go to post Benjamin De Boe · Apr 23, 2019 Depends a bit on what you want. If you want to use them in the WHERE clause, you can leave the "list of" structure as-is and use our %FOR SOME ELEMENT syntax. In your case, for retrieving them in the SELECT list, changing the projection or your data model is probably the pragmatic choice. Note that collections are a really powerful feature when working in the Object paradigm, but somewhat constrained by SQL standard operations when accessing through the relational paradigm.
go to post Benjamin De Boe · Apr 17, 2019 Still not sure I'm entirely on the same page wrt the goal, but $translate is to replace individual characters with other individual characters (so 1 by 1). If you're looking for a proper function in ObjectScript, take a look at $zconvert(), which supports HTML and XML-escaping.
go to post Benjamin De Boe · Apr 15, 2019 No, iKnow doesn't extract text from RTF prior to its NLP task. Besides the LibreOffice suggestion, I've also heard people who've worked with simple Java RTF extractors (part of regular JDK) and Tika in the past.
go to post Benjamin De Boe · Apr 11, 2019 Horita-san,not sure whether you mean the projection (table) itself is missing or the row you created through the API isn't showing up. This works fine for me, but in order to combine the use of APIs with a domain definition, you have to set the allowCustomUpdates flag to true (off by default). See also the notes in this article on the dictionary builder demo. When set to false, the API methods like CreateDictionary() will return an error (passed by reference, the returned ID will be below zero to indicate a failure).Hope this helps,benjamin
go to post Benjamin De Boe · Apr 1, 2019 Thanks for all your input thus far, which is proving very helpful inspiration for our planning process. Feel free to participate if you haven't done so or share with your colleagues, as we're still watching new inputs. Also, don't hesitate to share your thoughts directly on this thread. Positive feedback is great, but critical is often even more helpful for us :-)
go to post Benjamin De Boe · Mar 21, 2019 There is a simple regression calculator that is used internally for similar trend line work, iirc. The class reference is not spectacularly elaborate, but it's fairly straightforward to use. First you use the add() function to load up points and then the result() function will calculate a simple trend line and populate Slope and Intercept properties: USER>s stat = ##class(%DeepSee.extensions.utils.SimpleRegression).%New() USER>w stat.add(0,1) 1 USER>w stat.add(1,2) 1 USER>w stat.result(.b,.y0,.r) 1 USER>zw b,y0,r b=1 y0=1 r=1 USER>w stat.Slope 1 you can keep adding data and re-calculate: USER>w stat.add(1,1) 1 USER>w stat.result(.b,.y0,.r) 1 USER>zw b,y0,r b=.5 y0=1 r=.5
go to post Benjamin De Boe · Mar 19, 2019 Hi Joe,would you mind sharing some of your code (minus API key values :-) ) for signing AWS REST calls? I have almost scratched my head off trying to find out why things still aren't working when my StringToSign and SigningKey appear to be correct, but the hash I create from them isn't. I can even reproduce (aka "make the same mistake") using the sample Python code AWS provides.Relevant but not working (and therefore less relevant) code: Property AWSAccessKeyId As %String [ InitialExpression = "AKIDEXAMPLE" ]; Property AWSSecretAccessKey As %String [ InitialExpression = "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY" ]; Property Region As %String [ InitialExpression = "us-east-1" ]; Property Service As %String [ InitialExpression = "iam" ]; Method BuildAuthorizationHeader(pHttpRequest As %Net.HttpRequest, pOperation As %String = "", pURL As %String = "", Output pAuthorizationHeader As %String, pVerbose As %Boolean = 0) As %Status { set tSC = $$$OK try { if ..AWSAccessKeyId="" { set tSC = $$$ERROR($$$GeneralError, "No AWS Access Key ID provided") quit } if ..AWSSecretAccessKey="" { set tSC = $$$ERROR($$$GeneralError, "No AWS Secret Access Key provided") quit } set tAMZDateTime = $tr($zdatetime($h,8,7),":") // 20190319T151009Z //set tAMZDateTime = "20150830T123600Z" // for AWS samples set tAMZDate = $e(tAMZDateTime,1,8) // 20190319 set tLineBreak = $c(10) set pOperation = $$$UPPER(pOperation) // ensure the right date is set do pHttpRequest.SetHeader("X-Amz-Date", tAMZDateTime) // ************* TASK 1: CREATE A CANONICAL REQUEST ************* // http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html // Step 1 is to define the verb (GET, POST, etc.) -- inferred from pOperation // Step 2: Create canonical URI--the part of the URI from domain to query // string (use '/' if no path) set tCanonicalURL = $s($e(pURL,1)="/":pURL, $e(pURL,1)'="":"/"_pURL, 1:"/"_pHttpRequest.Location) // Step 3: Create the canonical query string. In this example (a GET request), // request parameters are in the query string. Query string values must // be URL-encoded (space=%20). The parameters must be sorted by name. // For this example, the query string is pre-formatted in the request_parameters variable. set tQueryString = $piece(tCanonicalURL,"?",2,*) set tCanonicalURL = $piece(tCanonicalURL,"?",1) // TODO: append pHttpRequest.Params content? // TODO: sort params! // Step 4: Create the canonical headers and signed headers. Header names // must be trimmed and lowercase, and sorted in code point order from // low to high. Note that there is a trailing \n. set tCanonicalHeaders = "content-type:" _ pHttpRequest.ContentType _ tLineBreak _ "host:" _ pHttpRequest.Server _ tLineBreak _ "x-amz-date:" _ tAMZDateTime _ tLineBreak // Step 5: Create the list of signed headers. This lists the headers // in the canonical_headers list, delimited with ";" and in alpha order. // Note: The request can include any headers; canonical_headers and // signed_headers lists those that you want to be included in the // hash of the request. "Host" and "x-amz-date" are always required. set tSignedHeaders = "content-type;host;x-amz-date" // Step 6: Create payload hash (hash of the request body content). For GET // requests, the payload is an empty string (""). if (pOperation = "GET") { set tPayload = "" } else { // TODO set tPayload = "" } set tPayloadHash = ..Hex($SYSTEM.Encryption.SHAHash(256,$zconvert("","O","UTF8"))) // Step 7: Combine elements to create canonical request set tCanonicalRequest = pOperation _ tLineBreak _ tCanonicalURL _ tLineBreak _ tQueryString _ tLineBreak _ tCanonicalHeaders _ tLineBreak _ tSignedHeaders _ tLineBreak _ tPayloadHash set tCanonicalRequestHash = ..Hex($SYSTEM.Encryption.SHAHash(256, tCanonicalRequest)) w:pVerbose !!,"Canonical request:",!,$replace(tCanonicalRequest,tLineBreak,"<"_$c(13,10)),!!,"Hash: ",tCanonicalRequestHash,! // ************* TASK 2: CREATE THE STRING TO SIGN************* // Match the algorithm to the hashing algorithm you use, either SHA-1 or // SHA-256 (recommended) set tAlgorithm = "AWS4-HMAC-SHA256" set tCredentialScope = tAMZDate _ "/" _ ..Region _ "/" _ ..Service _ "/" _ "aws4_request" set tStringToSign = tAlgorithm _ tLineBreak _ tAMZDateTime _ tLineBreak _ tCredentialScope _ tLineBreak _ tCanonicalRequestHash w:pVerbose !!,"String to sign:",!,$replace(tStringToSign,tLineBreak,$c(13,10)),! // ************* TASK 3: CALCULATE THE SIGNATURE ************* // Create the signing key using the function defined above. // def getSignatureKey(key, dateStamp, regionName, serviceName): set tSigningKey = ..GenerateSigningKey(tAMZDate) w:pVerbose !!,"Signing key:",!,..Hex(tSigningKey),! // Sign the string_to_sign using the signing_key set tSignature = ..Hex($SYSTEM.Encryption.HMACSHA(256, tStringToSign, tSigningKey)) // ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST ************* // The signing information can be either in a query string value or in // a header named Authorization. This code shows how to use a header. // Create authorization header and add to request headers set pAuthorizationHeader = tAlgorithm _ " Credential=" _ ..AWSAccessKeyId _ "/" _ tCredentialScope _ ", SignedHeaders=" _ tSignedHeaders _ ", Signature=" _ tSignature w:pVerbose !!,"Authorization header:",!,pAuthorizationHeader,!! b } catch (ex) { set tSC = ex.AsStatus() } quit tSC } Method GenerateSigningKey(pDate As %String) As %String { set kDate = $SYSTEM.Encryption.HMACSHA(256, pDate, $zconvert("AWS4" _ ..AWSSecretAccessKey,"O","UTF8")) //w !,"kDate: ",..Hex(kDate) set kRegion = $SYSTEM.Encryption.HMACSHA(256, ..Region, kDate) //w !,"kRegion: ",..Hex(kRegion) set kService = $SYSTEM.Encryption.HMACSHA(256, ..Service, kRegion) //w !,"kService: ",..Hex(kService) set tSigningKey = $SYSTEM.Encryption.HMACSHA(256, "aws4_request", kService) //w !,"kSigning: ",..Hex(tSigningKey),! quit tSigningKey } ClassMethod Hex(pRaw As %String) As %String [ Internal ] { set out="", l=$l(pRaw) for i = 1:1:l { set out=out_$zhex($ascii(pRaw,i)) } quit $$$LOWER(out) } ClassMethod SimpleTest() As %Status { set tSC = $$$OK try { set tAdapter = ..%New() set tAdapter.AWSAccessKeyId = "use yours" set tAdapter.AWSSecretAccessKey = "not mine" set tAdapter.Region = "us-east-1", tAdapter.Service = "iam" set tRequest = ##class(%Net.HttpRequest).%New() set tRequest.ContentType = "application/x-www-form-urlencoded" set tRequest.ContentCharset = "utf-8" set tRequest.Https = 1 set tRequest.SSLConfiguration = "SSL client" // simple empty SSL config set tRequest.Server = "iam.amazonaws.com" set tURL = "/?Action=ListUsers&Version=2010-05-08" set tSC = tAdapter.BuildAuthorizationHeader(tRequest, "GET", tURL, .tAuthorization, 1) quit:$$$ISERR(tSC) set tRequest.Authorization = tAuthorization set tSC = tRequest.Get(tURL) quit:$$$ISERR(tSC) Do tRequest.HttpResponse.OutputToDevice() } catch (ex) { set tSC = ex.AsStatus() } write:$$$ISERR(tSC) !!,$system.Status.GetErrorText(tSC),! quit tSC }
go to post Benjamin De Boe · Mar 12, 2019 plain variables don't really have a type, so it's worth knowing what you want to use your date for (storing as %Date, %PosixTime, %Timestamp,...). If you're starting from a string and want to convert to IRIS' internal date format, use the $zdateh() function and use 4 as the second argument for European date format. See the docs for that function for more info.
go to post Benjamin De Boe · Mar 6, 2019 You already got the expert answers, but maybe I'd just add this cautious recommendation: You should look at the IDKEY index keyword as a means to publish the internal rowid through a different name besides its default "%ID" alias. Unless you're mapping a class to an existing global structure, there's not many reasons nowadays to want to override it beyond that, as you may jeopardize some storage and runtime efficiencies like index options (eg bitmaps & bitslices).The primary key is what you as the schema designer decide to be the key for your table. If you don't choose one, we'll just default to that internal rowid for you (cf option 2 in Aviel's answer).
go to post Benjamin De Boe · Feb 22, 2019 IDENTITY fields have fairly specific characteristics wrt the physical storage of your table. Are you sure you want that particular field to be INSERTable by default for all tables (it's never UPDATEable)? Maybe a SERIAL field is more appropriate?
go to post Benjamin De Boe · Jan 22, 2019 If you just want to loop through your SQL resultset, you can do that either using cursors or using %SQL.Statements (aka Dynamic SQL)If you want to fetch all thoe %ID values into a single $listbuild(), check the %DLIST() aggregate function
go to post Benjamin De Boe · Jan 22, 2019 Hi Guillaume,I'm not sure what you're trying to get at. Our core JDBC driver supports batch processing through exactly the mechanism described in the tutorial you referenced, so that should work fine using default JDBC methods on the Java side. The JDBC SQL Adapter in EnsLib on the other hand was designed for a message-by-message processing and therefore the Adapter doesn't expose a batching mechanism. Maybe you can share a little more on the actual use case you're implementing? Buffering up messages for batch insertion or does the message carry a lot of data that deserves a batch insert by itself? Or am I totally on the wrong track here? :-)Thanks,benjamin
go to post Benjamin De Boe · Jan 14, 2019 my whole feeling of self-worth comes from the opinions of internet strangers!.. which means I cannot contribute without creating a profile under my dog's name :-)Still: great article! (and I don't have a dog)
go to post Benjamin De Boe · Jan 7, 2019 Thanks for posting Nikita. Your visualization has indeed been extremely helpful in showing what iKnow entities are all about to new audiences and is easily embeddable in applications where large numbers of entities need to be explored or navigated!
go to post Benjamin De Boe · Dec 10, 2018 Hi Evgeni,The default community edition should support 5 concurrent users. You might have to enable Atelier's REST API, which is not enabled by default for some security profiles. See this note on how to verify / enable it. If you chose BYOL (Bring Your Own License), it might be that you just still need to BYOL :-)Thanks,benjamin
go to post Benjamin De Boe · Dec 3, 2018 Hi Sean,IRIS uses different port numbers than Caché and Ensemble so port clashes are not an issue, but there are a few components that are typically shared across instances (e.g. ISCAgent) where consecutive installations of IRIS and Caché might cause trouble. We're documenting these and also other compatibility items of note (such as accessing one platform with the other's xDBC driver) in a guide that will be published shortly.The general recommendation remains to stick to instances of the same platform (so either all IRIS or all Caché) on a single server. Note that the use of VMs or Containers of course ensures a proper separation of libraries and enables you to run all your favourite cluster setups from the same physical server.
go to post Benjamin De Boe · Nov 19, 2018 Hi Yuri,Which endpoint did you use? And did you check whether there's any seed sentiment markers in your user dictionary as explained in this article? You can also check highlighting output in the general indexing results page to see if it's being picked up as expected.Feel free to reach out directly to me and/or Fabio if you want.Thanks,benjamin
go to post Benjamin De Boe · Nov 5, 2018 The SwaggerDoc for the iKnow REST APIs is also available online. Attribute information is relatively fine-grained and associated with entity occurrences aka sentence parts, which you can retrieve per sentence from the /domain/{domain}/sentences/{id}/details endpoint. The /domain/{domain}/entities series of endpoints are for unique entities, with which no occurrence-level information such as attributes is associated. Of course, you can also let attributes get rendered with highlighting rules, but I believe you want the attribute information in a more raw format, right? As an alternative to the REST API, you can get it from the SentenceAttribute and SourceAttribute views for your domain, as configured through the objectsPackage attribute in your domain definition (which you can of course wrap in a simple REST API of your own).You might already have seen this article which, despite dating back two years, provides a full overview of how to feed and use sentiment attributes.
go to post Benjamin De Boe · Oct 22, 2018 Perhaps also worth noting that %CacheObject is often mistakenly used for what should have been %ObjectHandle or %Base. Scenarios where user code would really need %Compiler.Type.Object are indeed very rare and thus intriguing :-)
go to post Benjamin De Boe · Sep 7, 2018 I just realized you're only on Caché 2012, which doesn't support table-valued functions, in which you can just SELECT from a function rather than having to use CALL, sorry.On the other hand, I'd expect a BI tool like Logi to be capable of providing exactly the sort of UI-side labelling of columns, if not drive the entire YoY calculation. Not that I want to fend off the question, but if there's a full-fledged BI tool sitting on top of these results anyhow, let's make sure to use its full set of fledges :-)