New post
Question
· May 10, 2016

Adding sources to existing domain

#InterSystems Natural Language Processing (NLP, iKnow)

The iKnow documentation shows an example for adding sources to a domain after an initial loading of sources.

The example uses text files. However, our data is now in Cache SQL tables.

Is it possible to add sources from a Cache SQL table, and is there an example of how this is done?

Thank you.

Discussion (3)0
Log in or sign up to continue
Daniel Wijnschenk · May 17, 2016

Yes, it is possible to add sources to an existing iKnow domain, regardless of the type of sources.

Which method are you using to create iKnow domains : iKnow architect, subclassing %iKnow.DomainDefinitions or Loader & Lister ?

For the first two methods, you have to make sure you do not drop the domain before running the Build again.

iKnow will not process existing ID's in a domain when you launch the build a second time without the Drop option. It will issue warnings (Source with external ID already exists)  but continue to index the rest.

For the third method (with loader and lister), you can simply take the existing examples for creating a domain, but ommit the .DropData() part.

I always use a SQL lister with a Where clause to filter only the 'new' records to add. 

0 0
Fabio Goncalves · May 17, 2016

Hi Terri!

Yes. It is possible. Yoy can take a look at Aviation.ReportDomain class on SAMPLES namespace where there is an example of SQL load. 

Also It is possible via $System.iKnow.IndexTable(). If you take a look at %SYSTEM.iKnow:IndexTable you will find an example of %iKnow.Source.SQL.Lister use.

From the Administration Portal you can go to iKnow >Architect where there is some option to load SQL data and other Listers. (this resource is available from 2016.x)

Another option is: http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=...

Regards

0 0
Terri Tattan · Nov 4, 2016

Because I wish to add to the sources in an existing domain (without dropping data), I experimented with $System.iKnow.IndexTable(). 

The first time, the new sources were added to the existing domain successfully. But, when I tried to add additional sources a second time, I got errors. Does anyone know why this only worked the first time? No sources were added to the domain on the second attempt. Thank you!

 

do $SYSTEM.iKnow.IndexTable("V10132","XXXXX.NotesAndDemog","DOCNUM","sta3n","notetext",$lb("patientname","patientXXX","sta3n","tiustandardtitle","tiudocdeftype","servicesection","referencedate"),"patientXXX=40","XXXXXCFG")

zw %objlasterror
%objlasterror="0 "_[1m$lb([22m[1m$lb([22m5001,"WARNING: some sources were not processed successfully, call Loader.GetFailed() for details",,,,,,,,[1m$lb([22m,"VISTA",[1m$lb([22m"e^zCheckErrors+16^%iKnow.Source.Loader.1^1","d^zProcessBatch+138^%iKnow.Source.Loader.1^1","e^zIndexTable+53^%SYSTEM.iKnow.1^1","d^testindextbl+2^testindextbl^1","d^^^0"[1m)[22m[1m)[22m[1m)[22m[1m)[22m/* ERROR #5001: WARNING: some sources were not processed successfully, call Loader.GetFailed() for details */

 

do $system.iKnow.ListErrors("V10132")

Error report for domain 'V10132':
No errors

The following sources failed to load in this domain:
11/03/2016 12:31:31: ERROR #8089: Source with external ID already exists: (external ID: :SQL:500:1, location: %iKnow.Source.Temp.Processor:ProcessListElement)
11/03/2016 12:31:31: ERROR #8089: Source with external ID already exists: (external ID: :SQL:500:3, location: %iKnow.Source.Temp.Processor:ProcessListElement)
11/03/2016 12:31:31: ERROR #8089: Source with external ID already exists: (external ID: :SQL:500:4, location: %iKnow.Source.Temp.Processor:ProcessListElement)
11/03/2016 12:31:31: ERROR #8089: Source with external ID already exists: (external ID: :SQL:500:5, location: %iKnow.Source.Temp.Processor:ProcessListElement)
11/03/2016 12:31:31: ERROR #8089: Source with external ID already exists: (external ID: :SQL:500:6, location: %iKnow.Source.Temp.Processor:ProcessListElement)

No warnings

0 0