Using iKnow domain configuration from iFind

Question

Question

Jack Abdo · Jan 15, 2016

Hi,

I created an iKnow domain, where I supplied dictionaries, blacklist, metadata and stemming. The datasource is a table.

I would like to use iFind semantic search feature. It is said in the documentation that iFind use iKnow semantic analysis. But I want iFind to use the iKnow domain configuration I created earlier earlier. How can I do that ?

Regards,

Jack Abdo.

Discussion (7)0

Log in or sign up to continue

Benjamin De Boe · Jan 18, 2016

Hi Jack,

you can enable stemming by setting the INDEXOPTION index parameter to 1 (or by leveraging the more flexible TRANSFORMATIONSPEC index parameter if you are on 2016.1).

Class ThePackage.MyClass Extends %Persistent
{
	Property MyStringProperty As %String;
	
	Index MyBasicIndex On (MyStringProperty) As %iFind.Index.Basic(INDEXOPTION=1);
}

The class reference for %iFind.Index.Basic also explains how you can toggle between stemmed and normal search by using the search mode argument:

SELECT * FROM ThePackage.MyClass WHERE %ID %FIND search_index(MyBasicIndex, 'interesting')

for normal search vs using search option 1 for stemmed search:

SELECT * FROM ThePackage.MyClass WHERE %ID %FIND search_index(MyBasicIndex, 'interesting', 1)

We do not discard stop words in iFind, in order to ensure you can query for any literal word sequence afterwards. If you start looking at the projections for entities (cf %iKnow.Index.Analytic class ref), you'll see how iKnow offers you a more insightful view of what a sentence is about through the "entity" level, where classic search tech may only offer you the words of a sentence minus the stop words.

regards,

benjamin

0 0

score 0 · Answer 1 · 2016-01-18T07:45:42-05:00

Hi Jack,

thanks for sharing your question. iFind actually only uses the iKnow engine, the internal piece of machinery that analyzes natural language text to identify semantic entities and their context. It does not use the iKnow domain infrastructure, which most of the documentation is focused on, but files the output of the iKnow engine into index structures that can be queried using the %FIND syntax or through some of the additional projections in search scenarios.

In order to create an iFind index on your table, you simply add it to the class definition (more info here) and then call the regular %BuildIndices() method (if there was data in it already). In a sense, iFind is a more lightweight, search oriented SQL index type, whereas the iKnow domain infrastructure offers a broader environment for exploring entities and their context.

FYI, I've posted an example search portal built on top of iFind here.

score 0 · Answer 2 · 2016-01-18T09:11:48-05:00

Hi Benjamen,

Thank you for your reply.

Is there a way I can apply stemming and do stop words removal when indexing with iFind ?

Best regards,

Jack Abdo.

score 0 · Answer 3 · 2016-01-18T14:58:09-05:00

Hi Benjamin,

That is great, and simple.

Is there an API with iFind to do search using natural language ?

Best regards,

Jack Abdo.

score 0 · Answer 4 · 2016-01-19T04:17:16-05:00

Hi Jack,

what exactly do you mean with search using natural language? If you're referring to combined search strings like "snow AND (ski OR ice-skat*)", this is possible today. Or do you mean asking a real literal question? Can you give an example?

thanks,

benjamin

score 0 · Answer 5 · 2016-01-29T15:49:34-05:00

Hello Benjamin,

I meant asking real literal questions. In the example you gave a query "snow AND (ski OR ice-skat*)" based on concepts. What about extracting those concepts from a real literal question like: Where can I buy find the best ski or ice-skate models ?

So where => any location

ski and ice-skate are concepts

If iknow can analyse the semantic of my question this way, I can then build a query similar to the one above.

Regards,

Jack Abdo.

score 0 · Answer 6 · 2016-02-01T04:03:42-05:00

Hi Jack,

this is not an out-of-the-box feature of the iKnow technology. iKnow's semantic analysis is targeted at identifying the semantic entities of a sentence, but not at interpreting them, which is typically an application-specific activity. However, we do have some building blocks that will help you create such applications, combining the iKnow analysis of a sentence with domain knowledge you already have. If you look at the indexing results for such a sentence, you'll see that the entities iKnow identifies will usually already present a good structure for your sentence, and human questions are often not that complicated. However, if the database you'll be querying is just un-interpreted free text as well, you'll need much more magic. If you're looking at querying a well-known data structure, it's much more feasible. I once wrote a crude text-to-MDX query tool that translated natural language questions into MDX by matching the concepts in the question to the labels on the dimensions and measures of a DeepSee cube definition. In this case, iKnow played its part in decomposing the question into concepts and relationships, which were then easily "interpreted" by custom code as cube elements and MDX constructs.

So, in short, iKnow will help you in the semantic analysis of natural language text, but depending on the complexity of the domain, more dedicated (and expensive) tools are usually needed for the subsequent interpretation and inference of results.

benjamin