Search

Hi, Community!See the Key Notes videos from Global Summit 2017 with new InterSystems IRIS Data Platform announcement.InterSystems Global Summit Keynote - Part 1InterSystems Global Summit Keynote - Part 2

AbstractA global provider of mobile telecommunications software tested the performance of InterSystems Caché and Oracle as the database in a simulated data mart application. They found Caché to be 41% faster than Oracle at building a data mart. When testing the response time to SQL queries of the data mart, Caché's performance ranged from 1.8 times to 513 times faster than Oracle.IntroductionTelecommunications companies, because they generate and must analyze enormous amounts of information, are among the most demanding database users in the world. In order to make practicable business intelligence solutions, telecommunications firms typically select key pieces of raw data to be loaded into a "data mart", where it is indexed and aggregated in various ways before being made available for analysis. Even so, the data marts in question may be hundreds of gigabytes in size. Database performance, both in the creation of the data mart, and in the query response time of the data mart, is critical to the timely analysis of information, and ultimately to the ability of the enterprise to identify and act upon changes in their business environment.This paper presents the results of comparative performance tests between InterSystems Caché and Oracle. They were performed by a global provider of mobile telecommunications software, as they evaluated database technology for incorporation into a new business intelligence applications relating to mobile phone usage.

IntroductionIn today's world, an ever-increasing number of purchases and payments are being made by credit card. Although merchants and service providers who accept credit cards have an obligation to protect customers' sensitive information, the software solutions they use may not support "best practices" for securing credit card information. To help combat this issue, a security standard for credit card information has been developed and is being widely adopted.The Payment Card Industry (PCI) Data Security Standard (DSS) is a set of guidelines for securely handling credit card information. Among its provisions are recommendations for storing customer information in a database. This paper will outline how software vendors can take advantage of the InterSystems Caché database - now and in the future - to comply with data storage guidelines within the PCI DSS.

InterSystems is pleased to announce that InterSystems IRIS Data Platform 2018.1.0 is now released.This press release was issued this morning.InterSystems IRIS is a complete, unified data platform that makes it faster and easier to build real-time, data-rich applications. It mirrors our design philosophy that software should be interoperable, reliable, intuitive, and scalable.For information about InterSystems IRIS, please visit our website here. You'll find out why InterSystems IRIS is the first complete data platform suitable for transactional, analytic, and hybrid transactional-analytic applications. See how our InterSystems Cloud Manager enables open and rapid deployment on public and private clouds, bare metal or virtual machines. Review our vertical and horizontal scaling, to ensure high performance regardless of your workloads, data sizes or concurrency needs. Discover how to use familiar tools and languages like Java and REST to interoperate with all your data.To interactively learn more about InterSystems IRIS, we've introduced the InterSystems IRIS Experience, a new environment that allows guided and open access to InterSystems IRIS Data Platform for exploring its powerful features. The Experience offers you the ability to solve challenges and build solutions using real-world data sets. With compelling scenarios, such as big data analytics and predicting patterns of financial fraud, developers and organizations can experience the power of InterSystems IRIS firsthand.You can also review the complete online documentation. What happens with ZEN in IRIS? %ZEN classes hidden in Class Reference and nothing in the documentation. Is it deprecated, anything else also disappear since Caché/Ensemble? - no DOCBOOK- no SAMPLES- no JDBC Driver in installbut there is C:\InterSystems\IRIS\dev\java\lib\JDK18\isc-jdbc-3.0.0.jar .....- and a very small icon for csystray.exe in Win Hi Robert,DocBook has now moved fully online, which is what the mgmt portal will link to: http://docs.intersystems.com/irisSAMPLES included quite a few outdated examples and was also not appropriate for many non-dev deployments, so we've also moved to a different model there, posting the most relevant ones on GitHub, giving us more flexibility to provide updates and new ones: https://github.com/intersystems?q=samplesJDBC driver: to what extent is this different from the past? It's always just been available as a jarfile, as is customary for JDBC drivers. We do hope to be able to post it through Maven repositories in the near future though.Small icons: yeah, to make our installer and (more importantly) the container images more lightweight, we had to economize on space. Next to the removal of DocBook and Samples, using smaller icons also reduces the size in bytes ;) ;)InterSystems IRIS is giving us the opportunity to adopt a contemporary deployment model, where we were somewhat restricted by long-term backwards compatibility commitments with Caché & Ensemble. Some of these will indeed catch your eye and might even feel a little strange at first, but we really believe the new model makes developing and deploying applications easier and faster. Of course, we're open to feedback on all of these evolution and this is a good channel to hear from you.Thanks!benjamin Hi Dmitry,Zen is indeed no longer a central piece of our application development strategy. We'll support it for some time to come (your Zen app still works on IRIS), but our focus is on providing a fast and scalable data management platform rather than GUI libraries. In that sense, you may already have noticed that recent courses we published on the topic of application development focus on leveraging the right technologies to connect to the backend (i.e. REST) and suggest using best-of-breed third-party technologies (i.e. Angular) for web development.InterSystems IRIS is a new product where we're taking advantage of our Caché & Ensemble heritage. It's meant to address today's challenges when building critical applications and we've indeed leveraged a number of capabilities from those products, but also added a few significant new ones like containers, cloud & horizontal scalability. We'll be providing an overview of elements to check for Caché & Ensemble customers that would like to migrate to InterSystems IRIS shortly (i.e. difference in supported platforms), but please don't consider this as merely an upgrade. You may already have noticed the installer doesn't support upgrading anyhow.Thanks,benjamin Hi Ben,- I like the idea of external samples. That offers definitely more flexibility .- DOCUMATIC is unchanged and works local! That's important. OK - JDBC: it isn't visible in Custom Install. You only see xDBC -> ODBC . Not an issue, rather a surprise. The .jar are where they used to be before.I'm really happy that we finally can get out of old chains imposed by 40yrs (DSM-11 and others) backward compatibility .Robert Hooray :) What date is the Zen end-of-life planned for (ie. when is it supported until and when will it be removed)? Hi Ben,I just installed a few samples as in description . CONGRATULATIONS !- not only that I get what I want and leave the rest aside- I also can split samples by subjects in several DBs and namespaces without being trapped by (hidden) dependencies.I think this makes live for training and teaching significantly easier!And allows central bug fixing independent from core release dates.A great idea! Thanks! And even anybody can offer own examples. Right !!A major step forward! Regarding Samples see also from the InterSystems IRIS documentation:Downloading Samples for Use with InterSystems IRIS What about Health Connect clients, does IRIS include components that are part of HealthShare Elite?ThanksYury Hi i need IR how to download it. Do you have login credentials for WRC Online at http://wrc.intersystems.com/ ?Once you have logged in, use the "Online Distributions" link. You'll need to contact your InterSystems account manager to request a license key.Alternatively, get your hands on InterSystems IRIS in the cloud here.

InterSystems is pleased to announce the release of InterSystems IRIS Data Platform 2018.1.1 This is our first follow up to InterSystems IRIS 2018.1.0, released earlier this year. InterSystems IRIS is a unified data platform for building scalable multi-workload, multi-model data management applications with native interoperability and an open analytics platform. As a reminder, you can learn more about InterSystems IRIS Data Platform from these Learning Services resources. Hundreds of fixes have been made in this release: Many changes that will greatly reduce future compatibility issues. Improvements to the documentation, user interface, and packaging to standardize naming of InterSystems IRIS components. Significant reliability improvements to our sharded query processing architecture, as well as unifying all application code into a single database. This unification eliminates redundancy and the potential for inconsistency, and provides a foundation for sharding and distributed processing across all our data models. Please read this document for complete details about the supported platforms, including cloud platforms and docker containers. For those of you who want to get hands-on quickly, we urge you to try out our InterSystems IRIS Experience. You’ll notice there is a brand new Java experience. The build corresponding to this release is 2018.1.1.643.0 To all the InterSystems Engineers. Great job and I am happy to use this product soon in my solutions. Great, but where is Caché 2018 (or at least the fieldtest) ? Are there any release notes? I can't seem to find any.RegardsGeorgewww.georgejames.com Caché and Ensemble 2018.1 Field Test is coming soon this summer. Unlike most maintenance releases we've not created separate release notes for this release. Relevant information has been added directly to the main documents. GREAT ! We're waiting for Caché 2018 to upgrade all of our instances, because 2017 has issues with ODBC. ...2017 has issues with ODBCHi Kurt,could you drop a few words on the subject: what kind of issues it has?(Just started moving the clients to 2017.2.1...) Hello,We're having issues with our Delphi-client where it receives empty datasets.Response fr om WRC : The problem was that Delphi switches from read uncommited to read commited mode and we had a problem there with the joins on null values. The fix will be included in the next maintenance kits for 2017.2.x. The change already went into 2018.1 so it will be in all versions 2018+. Has InterSystems yet published any guidance to help existing Caché or Ensemble users move their code onto IRIS? I haven't yet found that kind of document at https://docs.intersystems.com/ Yes, we maintain an adoption guide that covers exactly that purpose. In order to be able to properly follow up on questions you'd have, we're making it available through your technical account team (sales engineer or TAM) rather than ship it with the product. Maybe worth stating that in the product docs? Hi, what issues are you having with ODBC? We have a lot of clients connecting to our Cache DB through ODBC.

Design a production in a development environment using best practices. After you have built your first production in a test environment, you can start applying what you have learned and begin building in a development environment. Take the Designing Productions Non-Healthcare course to learn additional information to enable you to successfully design your production. Much of the information in this course is considered best practices.Learn More.

Hi, Community!Hope you know and use the Developer Community Analytics page which build with InterSystems DeepSee and DeepSee Web.We are playing with InterSystems iKnow analytics against Developer Community posts and introduced the new dashboard, which shows Top 60 concepts for all the posts:Or play with filters and see the top concepts for Atelier tag:Or the top concepts of a particular member:Click on the concept on the left and see the related articles on Developer Community. Here is small gif how it works: Next we plan to introduce concepts on Answers too and to fix tags and introduce new tags according to the concept stats.Your ideas and feedback would be much appreciated. Hi Evgeny,nice work!Maybe you can enhance the interface by also including an iKnow-based KPI to the dashboard exposing the similar or related entities for the concept clicked in the heat map. You can subclass this generic KPI and specify the query you want it to invoke, and then use it as the data source for a table widget. Let me know if I can help. thanks,benjamin Thank you, Benjamin!Yes, similar and related are concepts are in the roadmap too, thanks for the useful link!

Take this course to learn how data flows from HealthShare Information Exchange to Health Insight, along with the details of that data flow.Learn how to : - Relate a clinical scenario supported by Health Insight to its internal data structures and processes.- Identify the main data management components of HealthShare Information Exchange and Health Insight.- Describe the details of the data flow between HealthShare Information Exchange and Health Insight.- Differentiate between HL7 and CCD data handling in HealthShare Information Exchange.- Recognize configuration points in the system and how they affect system performance.- Define the HealthShare Information Exchange internal data structures and how they are used.Audience: HealthShare Customers. This course is for anyone who customizes or supports Health Insight, as well as power users of Health Insight who need an understanding of its technical details.Learn More.

Hi, Community!Hope you have already put in your schedule the visit to InterSystems Global Summit 2017 which will take place on 10-13 of September in remarkable JW Marriott Desert Springs Resort and Spa.This year we have Experience Lab, The Unconference, and 50 more other sessions, regarding performance, cloud, scalability, FHIR, high availability and other solutions and best practices.Global Summit it the most effective way to know what's new and what are the most powerful practices to make successful solutions with InterSystems technology.Today is the last day for early bird $999 tickets.But! You can get a free of charge ticket on InterSystems Global Masters Advocate hub!There are numerous ways to earn the points: write articles or answer the questions, publish testimonials or provide referrals, or simply watch and read articles and share it on social networks.To join Global Masters leave your comment in this post and we'll send you a personal invite link.Note!Allow us to recognize your contribution to Developer Community in Global Masters and register with the same email you have in Developer Community.Also, Community moderators are getting free tickets to Global Summit.This year they are -- [@Eduard.Lebedyuk], [@John.Murray], and [@Dmitry.Maslennikov].See you on InterSystems Global Summit 2017! Hi, Community!Just want to share the good news.Early bird registration for $999 prolonged until 14th of July, we also have $200 and $300 discounts for you on Global Masters. See the agenda of daily sessions on Solution Developers Conference.

Hi Community! New video "Deploying Shards Using InterSystems Cloud Manager" is available now on InterSystems Developers YouTube: In this demonstration, you will learn how to deploy a sharded cluster using InterSystems Cloud Manager (ICM), including defining the deployment, provisioning the architecture, and deploying and managing services. What is more? You are very welcome to watch all the videos about ICM in a dedicated InterSystems Cloud Manager playlist. Don`t forget to subscribe our InterSystems Developers YouTube Channel. Enjoy and stay tuned!

Hi all. Today we are going to install Jupyter Notebook and connect it to Apache Spark and InterSystems IRIS. Note: I have done the following on Ubuntu 18.04, Python 3.6.5. Introduction If you are looking for well-known, widely-spread and mainly popular among Python users notebook instead of Apache Zeppelin, you should choose Jupyter notebook. Jupyter notebook is a very powerful and great data science tool. it has a big community and a lot of additional software and integrations. Jupyter notebook allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more. And most importantly, it is a big community that will help you solve the problems you face. Check requirements If something doesn't work, look at the "Possible problems and solutions" paragraph in the bottom. First of all, ensure that you have Java 8 (java -version returns "1.8.x"). Next, download apache spark and unzip it. After, run the following in the terminal: pip3 install jupyter pip3 install toree jupyter toree install --spark_home=/path_to_spark/spark-2.3.1-bin-hadoop2.7 --interpreters=PySpark --user Now, open the terminal and run vim ~/.bashrc . Paste in the bottom the following code (this is environment variables): export JAVA_HOME=/usr/lib/jvm/installed java 8 export PATH="$PATH:$JAVA_HOME/bin" export SPARK_HOME=/path to spark/spark-2.3.1-bin-hadoop2.7 export PATH="$PATH:$SPARK_HOME/bin" export PYSPARK_DRIVER_PYTHON=jupyter export PYSPARK_DRIVER_PYTHON_OPTS="notebook" And run source ~/.bashrc. Check that it works Now, let us launch Jupyter notebook. Run pyspark in the terminal. Open in your browser the returned URL. Should be something like the image below: Click on new, choose Python 3, paste the following code into a paragraph: import sysprint(sys.version)sc Your output should look like this: Stop jupyter using ctrl-c in the terminal. Note: To add custom jars just move desired jars into $SPARK_HOME/jars. So, we want to work with intersystems-jdbc and intersystems-spark (we will also need a jpmml library). Let us copy required jars into spark. Run the following in the terminal: sudo cp /path to intersystems iris/dev/java/lib/JDK18/intersystems-jdbc-3.0.0.jar /path to spark/spark-2.3.1-bin-hadoop2.7/jars sudo cp /path to intersystems iris/dev/java/lib/JDK18/intersystems-spark-1.0.0.jar /path to spark/spark-2.3.1-bin-hadoop2.7/jars sudo cp /path to jpmml/jpmml-sparkml-executable-version.jar /path to spark/spark-2.3.1-bin-hadoop2.7/jars Ensure that is works. Run pyspark in the terminal again and run the following code (from the previous article): from pyspark.ml.linalg import Vectorsfrom pyspark.ml.feature import VectorAssemblerfrom pyspark.ml.clustering import KMeansfrom pyspark.ml import Pipelinefrom pyspark.ml.feature import RFormulafrom pyspark2pmml import PMMLBuilder dataFrame=spark.read.format("com.intersystems.spark").\option("url", "IRIS://localhost:51773/NAMESPACE").option("user", "dev").\option("password", "123").\option("dbtable", "DataMining.IrisDataset").load() # load iris dataset (trainingData, testData) = dataFrame.randomSplit([0.7, 0.3]) # split the data into two setsassembler = VectorAssembler(inputCols = ["PetalLength", "PetalWidth", "SepalLength", "SepalWidth"], outputCol="features") # add a new column with features kmeans = KMeans().setK(3).setSeed(2000) # clustering algorithm that we use pipeline = Pipeline(stages=[assembler, kmeans]) # First, passed data will run against assembler and after will run against kmeans.modelKMeans = pipeline.fit(trainingData) # pass training data pmmlBuilder = PMMLBuilder(sc, dataFrame, modelKMeans)pmmlBuilder.buildFile("KMeans.pmml") # create pmml model My output: The output file is a jpmml kmeans model. Everything works! Possible problems and solutions command not found: 'jupyter': vim ~/bashrc; add in the bottom export PATH="$PATH:~/.local/bin"; in terminal source ~/.bashrc. If it doesn't help, reinstall pip3 and jupyter. env: 'jupyter': No such file or directory: In ~/.bashrc export PYSPARK_DRIVER_PYTHON=/home/.../.local/bin/jupyter. TypeError: 'JavaPackage' object is not callable: Check that the required .jar file in /.../spark-2.3.1-bin-hadoop2.7/jars; Restart notebook. Java gateway process exited before sending the driver its port number: Your Java version should be 8 (probably works with Java 6/7 too, but I didn't check it); echo $JAVA_HOME should return to you Java 8 version. If not, change the path in ~/.bashrc; Paste sudo update-alternatives --config java in the terminal and choose a proper java version; Paste sudo update-alternatives --config javac in the terminal and choose a proper java version. PermissionError: [Errno 13] Permission denied: '/usr/local/share/jupyter' Add --user at the end of your command in the terminal Error executing Jupyter command 'toree': [Errno 2] No such file or directory Run the command without sudo. A specific error may appear if you use system variables like PYSPARK_SUBMIT_ARGS and other spark/pyspark variables or because of /.../spark-2.3.1-bin-hadoop2.7/conf/spark-env.sh changes. Delete these variables and check spark-env.sh. Links Jupyter Apache Toree Apache Spark Load a ML model into InterSystems IRIS K-Means clustering of the Iris Dataset The way to launch Apache Spark + Apache Zeppelin + InterSystems IRIS 💡 This article is considered as InterSystems Data Platform Best Practice.

Hi Community!This year will have a special section on Flash Talks which gives you an opportunity to introduce your tool or solution on InterSystems Global Summit 2018!What is Flash Talks? It's 15 min session you have on Technology Exchange scene: 10 min for your pitch, 5 min for Q&A. The session WILL BE live streamed on Developer Community YouTube Channel.Developer Community Flash Talks!Today, 10/02, Flash Talks Scene @ InterSystems Global Summit 2018!2 pm Open source approaches to work with Documents @Eduard.Lebedyuk, InterSystems2-15 InterSystems IRIS on Kubernetes by @Dmitry.Maslennikov2-30 Visual Studio Code IDE for InterSystems Data Platforms by @John.Murray, GeorgeJames Software2-45 Static Analysis for ObjectScript with CacheQuality by @Daniel.Tamajon, Lite Solutions3-00 InterSystems Open Exchange by @Evgeny.Shvarov, InterSystems3-15 Q&A Session on Developer Community, Global Masters, and Open Exchange Well!We already have two slots taken!One is:Static Analysis for ObjectScript with CacheQuality It's a CI compatible static analysis to for ObjectScript codebase which finds bugs and obeys code guidlines using a set of managed rules from LiteSolutions (which is now better will sound as IRISQuality, @Daniel.Tamajon? ;)Another slot is a secret topic, will be announced just before the GS.So we have two more, book your session on InterSystems IRIS Flash Talks from Developer Community! Yet another topic for InterSystems IRIS Flash Talks:InterSystems IRIS on Kubernetes by @Dmitry.Maslennikov Few days before Global Summit 2018!And I can announce yet another presenter: @Eduard.Lebedyuk, InterSystems.Title: Open source approaches to work with Documents.And we have the day and time!Find Developer Community Flash Talks "Share Your InterSystems IRIS Solution!" on Tuesday the 2nd of October on Flash Talks Stage from 2pm to 3-30 pm. And, we have another flash talk topic - "Visual Studio Code IDE for InterSystems Data Platforms" from @John.Murray!So many exciting topics! Looking forward! And one more topic from me: InterSystems Open Exchange - marketplace of Solutions, Tools and Adapters for InterSystems Data Platforms!The updated the agenda is here and in the topic:Tuesday 10/02, at Tech Exchange Global Summit 2018!2 pm Open source approaches to work with Documents @Eduard Lebedyuk, InterSystems2-15 InterSystems IRIS on Kubernetes by @Dmitry Maslennikov, 2-30 Visual Studio Code IDE for InterSystems Data Platforms by @John Murray, GeorgeJames Software2-45 Static Analysis for ObjectScript with CacheQuality by @Daniel Tamajon, Lite Solutions3-00 InterSystems Open Exchange by @Evgeny Shvarov, InterSystems3-15 Q&A Session on Developer Community, Global Masters and Open Exchange This is a live broadcast recording from Developer Community Flash Talks Great stuff. Thanks. Where can I get the sample files and/or some instructions on the use of Kubernetes as demonstrated? I bet @Dmitry.Maslennikov can share the info. Dmitry? I have done kubernetes deployments on a couple of projects, but both of them is private. And I can't share it. If you need some help, you can contact me directly, and I can share some ideas about how it can be done. Thanks Dmitry.

Hi Developers!New video from Global Summit 2018 is available now on InterSystems Developers YouTube Channel:Unit Test Coverage in InterSystems ObjectScript InterSystems is sharing a tool they use for measuring test coverage. Watch this session recording and learn how you can measure the effectiveness of your existing unit tests, identify weak areas, enable improvement, and track results over time.Takeaway: I know how to use the tool InterSystems provides to improve my unit tests.Presenter: @Timothy.Leavitt, Developer on HealthShare Development Team.Note: Test Coverage Tool is also available on InterSystems Open Exchange. And...Content related to this session, including slides, video and additional learning content can be found here.Don`t forget to subscribe our InterSystems Developers YouTube Channel.Enjoy and stay tuned!

Hi Developers! Here is the bonus results for the applications in InterSystems Grand Prix Programming Contest 2023: Project LLM AI or LangChain FHIR SQL Builder FHIR IntegratedML Native API Embedded Python Interoperability PEX Adaptive Analytics Tableau, PowerBI, Logi IRIS BI Columnar Index Docker ZPM Online Demo Unit Testing Community Idea Implementation First Article on DC Second Article on DC Code Quality First Time Contribution Video on YouTube Total Bonus Nominal 6 5 3 4 3 4 3 2 3 3 3 1 2 2 2 2 4 2 1 1 3 3 62 oex-mapping 4 3 2 2 2 2 2 1 1 3 22 appmsw-warm-home 2 2 2 2 1 9 RDUH Interface Analyst HL7v2 Browser Extension 3 3 6 irisapitester 4 2 2 2 1 1 3 15 oex-vscode-snippets-template 2 2 4 1 9 IRIS FHIR Transcribe Summarize Export 6 3 4 2 2 2 2 1 1 3 3 29 IntegratedMLandDashboardSample 4 3 2 2 1 12 iris-user-manager 2 2 1 5 irisChatGPT 6 5 4 2 2 2 2 1 1 3 28 fhir-chatGPT 6 3 4 2 1 16 iris-fhir-generative-ai 6 3 4 3 2 2 2 2 1 1 3 29 IRIS Data Migration Manager - - - 0 password-app-iris-db 3 2 2 2 3 3 15 interoperability_GPT 6 4 3 2 1 16 FHIR Editor 3 2 5 Recycler 3 - - 3 ZProfile 2 2 2 2 3 11 DevBox 6 2 3 11 FHIR - AI and OpenAPI Chain 6 3 2 2 2 2 1 1 3 3 25 IntegratedML-IRIS-PlatformEntryPrediction 4 3 3 10 Please apply with your comments for new implementations and corrections to be made here in the comments or in Discord. Hi @Evgeny.Shvarov ! I used Java to connect to IRIS in the application and associated an article with it, but I did not see it in bonus points. Can they be added to bonus points? Hi Zhang! We don't have points for using Java. What bonus are you talking about? If you mean Native API you haven't use it. You used only jdbc in your project without Native SDK . Hi @Evgeny.Shvarov Thanks for publishing the bonuses.Please note that I have added FHIR SQL Builder functionality in my new release of irisChatGPT application. So pls consider it.Thanks Hi Muhammad! Your points were added to the table! Thank you! Hi @Semion.Makarov I added a BI dashboard to do analytics on the app logs of iris-fhir-generative-ai to the release 1.0.9, and a second article explaining about such analytics. So, I'd like to ask for IRIS BI and Second article bonuses. PS: Sorry for publish this so late, but I had this idea just Sunday late. 😄 Thanks! Hi Jose! I've applied these bonuses to your app.

Demonstration example for the current Grand Prix contest for use of a more complex Parameter template to test the AI. Interview Questions There is documentation. A recruitment consultant wants to quickly challenge candidates with some relevant technical questions to a role. Can they automate making a list of questions and answers from the available documentation? Interview Answers and Learning One of the most effective ways to cement new facts into accessible long term memory is with phased recall. In essence you take a block of text information, reorganize it into a series of self-contained Questions and Facts. Now imagine two questions: What day of the week is the trash-bin placed outside for collection? When is the marriage anniversary? Quickly recalling correct answers can mean a happier life!! Recalling the answer to each question IS the mechanism to enforce a fact into memory. Phased Recall re-asks each question with longed and longer time gaps when the correct answer is recalled.For example: You consistently get the right answer: The question is asked again tomorrow, in 4 days, in 1 week, in 2 weeks, in 1 month. You consistently get the answer wrong: The question will be asked every day until it starts to be recalled. If you can easily see challenging answers, it is productive to re-work difficult answers, to make them more memorable. There is a free software package called Anki that provides this full phased recall process for you. If you can automate the creation of questions and answers into a text file, the Anki will create new flashcards for you. Hypothesis We can use LangChain to transform InterSystems PDF documentation into a series of Questions and answers to: Make interview questions and answers Make Learner Anki flash cards Create new virtual environment mkdir chainpdf cd chainpdf python -m venv . scripts\activate pip install openai pip install langchain pip install wget pip install lancedb pip install tiktoken pip install pypdf set OPENAI_API_KEY=[ Your OpenAI Key ] python Prepare the docs import glob import wget; url='https://docs.intersystems.com/irisforhealth20231/csp/docbook/pdfs.zip'; wget.download(url) # extract docs import zipfile with zipfile.ZipFile('pdfs.zip','r') as zip_ref: zip_ref.extractall('.') Extract PDF text from langchain.document_loaders import PyPDFLoader from langchain.embeddings.openai import OpenAIEmbeddings from langchain.text_splitter import CharacterTextSplitter from langchain.prompts.prompt import PromptTemplate from langchain import OpenAI from langchain.chains import LLMChain # To limit for the example # From the documentation site I could see that documentation sets # GCOS = Using ObjectScript # RCOS = ObjectScript Reference pdfFiles=['./pdfs/pdfs/GCOS.pdf','./pdfs/pdfs/RCOS.pdf'] # The prompt will be really big and need to leave space for the answer to be constructed # Therefore reduce the input string text_splitter = CharacterTextSplitter( separator = "\n\n", chunk_size = 200, chunk_overlap = 50, length_function = len, ) # split document text into chuncks documentsAll=[] for file_name in pdfFiles: loader = PyPDFLoader(file_name) pages = loader.load_and_split() # Strip unwanted padding for page in pages: del page.lc_kwargs page.page_content=("".join((page.page_content.split('\xa0')))) documents = text_splitter.split_documents(pages) # Ignore the cover pages for document in documents[2:]: # skip table of contents if document.page_content.__contains__('........'): continue documentsAll.append(document) Prep search template _GetDocWords_TEMPLATE = """From the following documents create a list of distinct facts. For each fact create a concise question that is answered by the fact. Do NOT restate the fact in the question. Output format: Each question and fact should be output on a seperate line delimited by a comma character Escape every double quote character in a question with two double quotes Add a double quote to the beginning and end of each question Escape every double quote character in a fact with two double quotes Add a double quote to the beginning and end of each fact Each line should end with {labels} The documents to reference to create facts and questions are as follows: {docs} """ PROMPT = PromptTemplate( input_variables=["docs","labels"], template=_GetDocWords_TEMPLATE ) llm = OpenAI(temperature=0, verbose=True) chain = LLMChain(llm=llm, prompt=PROMPT) Process each document and place output in file # open an output file with open('QandA.txt','w') as file: # iterate over each text chunck for document in documentsAll: # set the label for Anki flashcard source=document.metadata['source'] if source.__contains__('GCOS.pdf'): label='Using ObjectScript' else: label='ObjectScript Reference' output=chain.run(docs=document,labels=label) file.write(output+'\n') file.flush() There were some retry and force-close messages during loop. Anticipate this is limiting the openAI API to a fair use. Alternatively a local LLM could be applied instead. Examine the output file "What are the contexts in which ObjectScript can be used?", "You can use ObjectScript in any of the following contexts: Interactively from the command line of the Terminal, As the implementation language for methods of InterSystems IRIS object classes, To create ObjectScript routines, and As the implementation language for Stored Procedures and Triggers within InterSystems SQL.", Using ObjectScript, "What is a global?", "A global is a sparse, multidimensional database array.", Using ObjectScript, "What is the effect of the ##; comment on INT code line numbering?", "It does not change INT code line numbering.", Using ObjectScript, "What characters can be used in an explicit namespace name after the first character?", "letters, numbers, hyphens, or underscores", Using ObjectScript "Are string equality comparisons case-sensitive?", "Yes" Using ObjectScript, "What happens when the number of references to an object reaches 0?", "The system automatically destroys the object.",Using ObjectScript Question: "What operations can take an undefined or defined variable?", Fact: "The READ command, the $INCREMENT function, the $BIT function, and the two-argument form of the $GET function.", Using ObjectScript, a While a good attempt at formatting answers has occurred there is some deviation. Manually reviewing I can pick some questions and answers to continue the experiment. Importing FlashCards into Anki Reviewed text file: "What are the contexts in which ObjectScript can be used?", "You can use ObjectScript in any of the following contexts: Interactively from the command line of the Terminal, As the implementation language for methods of InterSystems IRIS object classes, To create ObjectScript routines, and As the implementation language for Stored Procedures and Triggers within InterSystems SQL.", "Using ObjectScript","What is a global?", "A global is a sparse, multidimensional database array.", "Using ObjectScript","What is the effect of the ##; comment on INT code line numbering?", "It does not change INT code line numbering.", "Using ObjectScript","What characters can be used in an explicit namespace name after the first character?", "letters, numbers, hyphens, or underscores", "Using ObjectScript""Are string equality comparisons case-sensitive?", "Yes", "Using ObjectScript","What happens when the number of references to an object reaches 0?", "The system automatically destroys the object.","Using ObjectScript""What operations can take an undefined or defined variable?", "The READ command, the $INCREMENT function, the $BIT function, and the two-argument form of the $GET function.", "Using ObjectScript" Creating new Anki card deck Open Anki and select File -> Import Select the reviewed text file Optionally create a new Card Deck for "Object Script" A basic card type is fine for this format There was mention of a "Field 4" so should check the records. Anki import success Lets Study Now choose the reinforcement schedule Happy Learning !! References Anki software is available from https://apps.ankiweb.net/