This is due to a logging issue that has been fixed in 2016.2 and should also be included in a future maintenance release of 2016.1

It's indeed tempting to just stuff interfaces like this in the kit, but it goes a bit beyond the objectives of pure system management interfaces that we'd typically pack with Caché. Also, in the specific case of this Dictionary Builder demo, it uses the programmatic APIs to create dictionaries (requiring allowCustomUpdates=true) and does not update the domain definition itself. We're actually working on making that a smoother process, so when that gets to a point where it can support the interactions implemented in this GUI (and when AngularJS becomes part of our kit), we can reconsider it.

Hi Terri,

[a little late perhaps (and we spoke briefly in Phoenix as well), but for the sake of completeness, here's your response]

The iKnow Architect page only shows domains based on domain definitions, subclasses of %iKnow.DomainDefinition. The ones created "the old way" (programmatically) through the %iKnow.Domain class do not carry the declarative information about where to load data from, hence the majority of the Architect GUI wouldn't be available on those.

As to the metadata loading question: I'm not sure which part of the script you're referring to was taking care of that metadata loading, but maybe this comes back to your question in a separate thread.

regards,

benjamin

Hi Orion,

I haven't heard about index data simply disappearing. There should be no reason for that other than calls to %PurgeIndices() or manually dropping the globals containing the data (including ^ISC.IF* ones containing entities and words shared across the namespace). Another potential issue may arise when importing (at a global level) only either the index or the shared data, resulting in them no longer being in sync. Any chance any of that could have happened?

For the missing class, that might be due to a class import as well, after which not all related classes were recompiled to reflect the changes.

Which version are you working on? Some of those recompiling issues may have been addressed in recent versions.

Perhaps the WRC is a better place to get the appropriate follow-up for specific issues like this one. 

regards,
benjamin

That functionality is not supported through the iKnow Architect. It is our intent to focus on the table and query data location options, as those are just a bit of COS development away from other sources of (meta)data.

Hi Benjamin,

the default algorithm indeed won't return scores for each record, but will only make the calculation for all records that contain at least a decent number of entities that are relevant in the source document. You can indeed simply approximate the other documents' score by taking 0.

For your specific use case, you may want to take a look at the text categorization infrastructure. I've posted a tutorial on the topic here.

regards,
benjamin

Hi Julie,

For XEP, the XEP guide in the product documentation is probably the best starting point. For iKnow, you can take a look at this video playlist introducing the technology. 

As you may know, InterSystems is also developing a new platform specifically aimed at big data use cases. Part of this new platform will be support for the UIMA standard, as a broader framework for dealing with unstructured data than iKnow's natural language processing alone, allowing you to combine it with third-party or custom utilities. Please send me an email if you'd like to discuss your big data project in more detail.

 

thanks,
benjamin

Hi Jack,

there's no need to normalize your search strings, as it's take care of automatically as part of executing your search when appropriate.

When you use DELETE FROM in SQL, or ##class(Your.Table).%DeleteExtent() in COS, the associated iFind indices' data will be erased as well. To drop just the indices data, use ##class(Your.Table).%PurgeIndices() (cf class ref for refinements). Note that, unless you are using index-local storage (new feature in 2016.1), the words and entities tables will not be wiped as they are shared between all iFind indices in your namespace (somewhat conserving space and indexing efficiency).

iFind can calculate a score representing how well a record satisfies a search string, largely based on TFIDF (although it'll leverage the more refined dominance scores for entities when it can). This is also new in 2016.1. See https://community.intersystems.com/code/ifind-search-portal for an example.

 

regards,
benjamin

Hi Evgeny, Jack,

 

Ranking is new in 2016.1, and will indeed allow you to retrieve a score expressing how well a record matches a search string. A packagename.tablename_indexnameRank function gets automatically generated when  you compile your class with an iFind index and can be invoked as follows:

SELECT %ID, 
Title,
FullText,
SomePackage.TheTable_MyIndexRank(%ID, 'cocktail* OR (hammock AND NOT bees)')
FROM SomePackage.TheTable
WHERE %ID %FIND search_index(MyIndex, 'cocktail* OR (hammock AND NOT bees)')
ORDER BY 4 DESC

There are no public demo servers exposing this functionality at this time.

 

regards,
benjamin