#iKnow

InterSystems' iKnow technology allows you to identify the concepts in natural language texts and the relations that link them together. As that's still a fairly abstract definition, we produced this video to explain what that means in more detail. But when meeting with customers, what really counts is a compelling demonstration, on data that makes sense to them, so they understand the value in identifying these concepts over classic top-down approaches. That's why it's probably worth spending a few articles on some of the demo apps and tools we've built to work with iKnow.

In the first article in this series, we'll start with the Knowledge Portal, a simple query interface to explore the contents of your domain.

Hello!

My group and I are currently doing a research project on natural language processing and iKnow plays a big role in this project. I am aware that the algorithms iKnow use aren't public, and I respect that.

My question is, are there any public documents/research that explains, at least part of, the algorthims iKnow uses and the motivations for using them?

This article contains the tutorial document for a Global Summit academy session on Text Categorization and provides a helpful starting point to learn about Text Categorization and how iKnow can help you to implement Text Categorization models. This document was originally prepared by Kerry Kirkham and Max Vershinin and should work based on the sample data provided in the SAMPLES namespace.

Hi,

I am using iknow text categorization to classify texts. I have 11 medical articles as my training set. Here is part of the source code:

In a conference call earlier this week, a customer described how they built an iKnow domain with clinical notes and now wanted to filter the contents of that domain based on the patient's diagnosis codes.

This is the third article in a series on iKnow demo applications, showcasing how the concepts and context provided through iKnow's unique bottom-up approach can be used to implement relevant use cases and help users be more productive in their daily tasks. Previous articles discussed the Knowledge Portal, a straightforward tool to browse iKnow indexing results, and the Set Analysis Demo, in which you can use the output of iKnow indexing to organize your texts according to their content, such as in patient cohort selection.

This week, we'll look into another demo application, the Dictionary Builder demo, in which we'll marry iKnow's bottom-up insights with top-down expertise, organizing our domain knowledge into dictionaries that are composed of the actual terms used in the data itself. Sticking to a top-down approach only, you'd risk missing out on some terminology used in the field that a domain expert sitting in his office wouldn't be aware of.

Earlier in this series, we've presented four different demo applications for iKnow, illustrating how its unique bottom-up approach allows users to explore the concepts and context of their unstructured data and then leverage these insights to implement real-world use cases. We started small and simple with core exploration through the Knowledge Portal, then organized our records according to content with the Set Analysis Demo, organized our domain knowledge using the Dictionary Builder Demo and finally build complex rules to extract nontrivial patterns from text with the Rules Builder Demo.

This time, we'll dive into a different area of the iKnow feature set: iFind. Where iKnow's core APIs are all about exploration and leveraging those results programmatically in applications and analytics, iFind is focused specifically on search scenarios in a pure SQL context. We'll be presenting a simple search portal implemented in Zen that showcases iFind's main features.

I have a class which, in the previous instance, was able to extract metadata field names and data from a text file, and load this information into a domain. I am trying to run this in the field test instance, but it is not loading the metadata - only the field names. I am not getting an error, but the data is not loaded.

The few changes I made to the original class:

Previously, this class also ran iTables. I commented all that code out.

To create the domain, I replaced the line:

Presenter: Benjamin De Boe
Task: Extract specialized information from your unstructured data
Approach: Combine InterSystems iKnow technology with third-party and custom text-processing tools

This session explains how you can easily combine ISC, third-party and custom text processing tools to get the broadest insights in your unstructured data.

Content related to this session, including slides, video and additional learning content can be found here.

Presenter: Misha Bouzinier
Task: Gain an understanding of natural language processing and the current state of the art
Approach: Discuss how InterSystems iKnow technology fits into the NLP ecosystem and complements the output of other components such as Lucene and Stanford NLP tools

A 101 session on Natural Language Processing that positions Intersystems tools in the broader ecosystem Problem: we’ve been touting “unstructured data” for five years, but many people both internally and externally still don’t know what it means to “process natural language” in general and how iKnow and our upcoming UIMA capabilities fit in this NLP ecosystem. This session will describe what a number of common technologies offer and how bare-bone NLP output typically needs to be complemented with more classic analytics or inference tooling to get the value out.

Content related to this session, including slides, video and additional learning content can be found here.

Hello!

We use iKnow's GetSimilar for decision making. Right now we have a domain with both good and bad documents and using GetSimilar we want to see if a document is more similar to the good ones or the bad ones. To do this we simply compare the weighted average of the score from the good ones and the bad ones that GetSimilar returns.

The problem is that GetSimilar doesn't always return the score to all other documents. Assuming we have 50 documents I would expect the following result:

Hello!

I'm trying to make a simple hello world program for iKnow but the documentation only shows examples using the scripting language or XML XData.

The domain is already created and configured, what I hope for is to be able to work with the data in some high-level language.

Is it possible to do something like this:

int domainId = 1;
IKnow iknow = new IKnow( domainId );
List concepts = iknow.getTopConcepts();

Any feedback will be greatly appreciated!