woops, I got that quite wrong indeed. I got confused by the mentions of TSQL earlier in the thread and mistook it for some other lock-related syntax that's only there for TSQL. you are correct: LOCK TABLE is part of IRIS SQL.

I'm not sure whether the error message you got would have further clues, but likely you're better off filing this with the WRC. Given that it works for embedded SQL, I'm suspicious it may have to do with the pre-parser for .NET, which does an initial parsing of the command on the client side.

Thanks @Kevin Chan for helping out (he's the author of the parser piece of LOAD DATA ;-) )

In addition to his comments on the encoding issues, you can also take a look at the %SQL_Diag.Result table which will likely have more detailed error info than what we're able to return through SQL semantics (though we're still working on ways to make it clearer). There's also a %SQL_Diag.Message table with row-level errors, which can help you figure out the reason for individual row failures.

The iFind search portal demo includes a simple class query to find similar documents within a single iFind index. It's only pretty basic and somewhat picky (assuming the demo setup), building on the dominance score for each entity, and may not guard against that difference in length issue you're seeing with BM25. There is a similar method in iKnow when your data would already be in an iKnow domain.

There would indeed be value in providing %SIMILARITY support for iFind indexed fields, leveraging the standard/enhanced algorithm on top of word tokens. I'll log that as an enhancement request and we can follow up internally. Obviously, I'm interested in experiences or advice of other DC members here 

You're spot on. The similarity between Python and ObjectScript plus its popularity (Python's, that is ;-) ) are exactly what drove us to build Embedded Python. We're not compiling it to .obj code though, but running it "as Python" in the kernel. @Bob Kuszewski and @David McCaldon are much better at explaining that nuance (and actually do in the intro webinar to our early access program for this upcoming feature.

Hi Nigel, 

glad you like the Python Gateway work. Perhaps you're also interested in participating in the embedded python EAP?

As for APL, we're typically looking for some critical mass of customer demand before embarking on such a development project and only recall it being mentioned once before. But thanks for bringing it up, as each critical mass starts somewhere :-). 

This said, we've had some really great work pioneered in the community here. The Python Gateway project is one example that originated here and its popularity (thanks @Eduard Lebedyuk and @Sergey Lukyanchikov 
!) inspired the one now released as part of the core InterSystems IRIS platform. Similar concepts, such as the Julia Gateway are still "incubating" as a community-driven project. Maybe APL could fit that same path?