New post

#InterSystems Natural Language Processing (NLP, iKnow)

7

InterSystems IRIS Natural Language Processing (NLP), formerly known as iKnow, allows you to perform text analysis on unstructured data sources in a variety of natural languages without any prior knowledge of their content. It does this by applying language-specific rules that identify semantic entities. Because these rules are specific to the language, not the content, NLP can provide insight into the contents of texts without the use of a dictionary or ontology. Learn more.

All time

If you've worked with iKnow domain definitions, you know they allow you to easily define multiple data locations iKnow needs to fetch its data from when building a domain. If you've worked with DeepSee cube definitions, you'll know how they tie your cube to a source table and allow you to not just build your cube, but also synchronize it, only updating the facts that actually changed since the last time you built or synced the cube. As iKnow also supports loading from non-table data sources like files, globals and RSS feeds, the same tight synchronization link doesn't come out of the box. In this article, we'll explore two approaches for modelling DeepSee-like synchronization from table data locations using callbacks and other features of the iKnow domain definition infrastructure.

3 2
0 433

Hi Community!

Enjoy the video of the week about InterSystems iKnow Technology:

A Cure for Clinician Frustration

https://www.youtube.com/embed/johncrm-O0E
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

2 0
0 447

Hello!

My group and I are currently doing a research project on natural language processing and iKnow plays a big role in this project. I am aware that the algorithms iKnow use aren't public, and I respect that.

My question is, are there any public documents/research that explains, at least part of, the algorthims iKnow uses and the motivations for using them?

1 2
0 420

I am trying to create an iKnow domain programmatically like:

    Set dom = ##class(%iKnow.Domain).%New("TestDom")
    Do  dom.SetParameter("DefaultConfig", "MyConfiguration")
    Set sc = dom.%Save()

   ...

Although "MyConfiguration" sets the language to "ja", i.e. Japanese, it doesn't seem to be respected, and what I see on the top right pane in Knowledge Portal is related concepts, instead of proximity profiles, which I expect to see in Japanese language mode.

Also resulting segmentation of sentences looks to be it is in English mode.

0 2
0 418

Hi,

I created an iKnow domain, where I supplied dictionaries, blacklist, metadata and stemming. The datasource is a table.

I would like to use iFind semantic search feature. It is said in the documentation that iFind use iKnow semantic analysis. But I want iFind to use the iKnow domain configuration I created earlier earlier. How can I do that ?

Regards,

Jack Abdo.

0 7
0 408

I have a class which, in the previous instance, was able to extract metadata field names and data from a text file, and load this information into a domain. I am trying to run this in the field test instance, but it is not loading the metadata - only the field names. I am not getting an error, but the data is not loaded.

The few changes I made to the original class:

Previously, this class also ran iTables. I commented all that code out.

To create the domain, I replaced the line:

0 1
0 403

Hi Community!

New video "Treating Patients with REST and iKnow" is available now on DC YouTube Channel:

https://www.youtube.com/embed/HDTij5GS_qQ
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

1 0
1 399

Presenter: Danny Wijnschenk
Task: Help people make better decisions by letting application deal with all the data.
Approach: As an example, we’ll extend a demo asset management application for portfolio and trade compliance, using iKnow technology to translate agreements into rules that ensure portfolio compliance prior to trade execution.

In this session, we’ll discuss how easy it is to extend a classic application that deals with straightforward transactions, to also offer insights and actions based on more complex, unstructured data. We’ll present a use case on portfolio compliance from the financial services industry.

Content related to this session, including slides, video and additional learning content can be found here.

0 1
0 367

Hello!

We use iKnow's GetSimilar for decision making. Right now we have a domain with both good and bad documents and using GetSimilar we want to see if a document is more similar to the good ones or the bad ones. To do this we simply compare the weighted average of the score from the good ones and the bad ones that GetSimilar returns.

The problem is that GetSimilar doesn't always return the score to all other documents. Assuming we have 50 documents I would expect the following result:

0 1
0 384

Presenter: Benjamin De Boe
Task: Extract specialized information from your unstructured data
Approach: Combine InterSystems iKnow technology with third-party and custom text-processing tools

This session explains how you can easily combine ISC, third-party and custom text processing tools to get the broadest insights in your unstructured data.

Content related to this session, including slides, video and additional learning content can be found here.

0 0
0 381

I have iKnow domain of forum posts, their full text is an iKnow data, and each post also has a number of views as a metadata field.

I want to get a sum of views by concept. Let's say I have a concept called "TESTEST" and there are 10 sources that have this concept. Each source has some views. I want to get views total - impact of this concept so to say.

What's the best iKnow architecture for this use case?

So far I got this:

0 1
0 376

Hi-

I have a SQL Query using %iFind.Highlight which returns text highlighting certain words and phrases. %iFind.Highlight seems to remove cr/lf from the returned text.

Here's my query

ClassMethod Search(pSessionId As %String, pSearchString As %String) As %String
{
set tTags="<span style='background-color:yellow;'>"
&sql(
SELECT %iFind.Highlight(Text , :pSearchString , , :tTags) into :results
FROM SSA_OCR.TempSearchable where sessionId = :pSessionId)
quit results
}

0 1
0 374

Hi, Community!

Check a new session recording from Global Summit 2017:

iKnow What You'll Do Next Summer

https://www.youtube.com/embed/7Y5mdY3M3vo
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

1 0
0 353

In Cache'/Ensemble, by specifying the objectsPackage parameter, dictionaries (and other objects) get projected to tables that can be accessed by SQL queries.

But in IRIS (IRIS for UNIX (Ubuntu Server LTS for x86-64 Containers) 2019.1 (Build 507U) Mon Feb 25 2019 13:47:16 EST), when I created a dictionary with ##class(%iKnow.Matching.DictionaryAPI).CreateDictionary(), it does not get projected to a table.

The class APIs correctly retrieve information about this dictionary.

Am I missing something with IRIS, or is there any issues about this?

0 2
0 344

Hi Community,

Enjoy watching the new video on InterSystems Developers YouTube:

⏯ iKnow: Open Source NLP in InterSystems IRIS

https://www.youtube.com/embed/n-TL_Wh7Tv4
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

2 0
1 325

Presenter: Misha Bouzinier
Task: Gain an understanding of natural language processing and the current state of the art
Approach: Discuss how InterSystems iKnow technology fits into the NLP ecosystem and complements the output of other components such as Lucene and Stanford NLP tools

A 101 session on Natural Language Processing that positions Intersystems tools in the broader ecosystem Problem: we’ve been touting “unstructured data” for five years, but many people both internally and externally still don’t know what it means to “process natural language” in general and how iKnow and our upcoming UIMA capabilities fit in this NLP ecosystem. This session will describe what a number of common technologies offer and how bare-bone NLP output typically needs to be complemented with more classic analytics or inference tooling to get the value out.

Content related to this session, including slides, video and additional learning content can be found here.

0 0
0 316