go to post Steven Hobbs · May 8 The %Stream.DynamicCharacter (%SDC) and %Stream.DynamicBinary (%SDB) classes are designed to hold a copy of the string data that is in an element of a %DynamicArray (%DA) or a %DynamicObject (%DO) class. The %SDC class, and its %SDB subclass, were designed to be small and efficient even when the string in the %DA/%DO class is very large, containing many billions of characters or bytes. The %SDC classes contain readonly data so you cannot modify the string data that they contain. However, you can create any other object in %Stream package of classes that supports the CopyFrom() method (such the %Stream.TmpCharacter, %Stream.TmpBinary, %Stream.FileCharacter or %Stream.FileBinary classes) and use the CopyFrom() method to create a writeable copy of the string data in the %SDC object. If your %SDC stream contains many billions of data elements then the DO tmpStream.CopyFrom(sdc)access will take much longer to load the tmpStream object than it took to create the original sdc object. A %DA/%DO object internally contains a data structure called a Packed Vector Array (PVA). The PVA supports rapid creation from JSON text and also rapid data lookup. It does not support rapid data modification, although the %Set() method can be used to modify and rearrange the data in a PVA. When a %DA/%DO contains a string element longer than a selected length (currently defaulting to about 4095 characters) then those characters are placed in a separate buffer object and a small object reference (oref) is placed in the PVA. The buffer classes are the %DynamicString, %DynamicBinary and %DynamicBase64 classes which are used to contain the characters and bytes of long strings. These classes are not very interesting to ordinary ObjectScript programers as they have no properties and no methods. When a %SDC object is created (using something like the SET sdc=dao.%Get(key,,"stream") ObjectScript statement) then the DAOData property in the %SDC object references the buffer class so that characters/bytes in the buffer do not need be copied. A buffer class can be shared by multiple %DA, %DO, %SDC and %SDB objects, which is why the buffer objects are read only.
go to post Steven Hobbs · May 2 The default 'mindate' value for the $ZDATE(hdate,dformat,monthlist,yearopt,startwin,endwin,mindate,maxdate,erropt,localeopt)system function call is 0, which represents 1840-12-31. However, you can supply your own 'mindate' argument with negative values down -672045 which does date conversions more distantly into the past. USER>zwrite $zdate(-672045,3,,,,,-672045) "0001-01-01" which is Gregorian date January 1st in the year 0001 CE. (That is the first day, in the first month in the first year of the first century.) I should note that the Gregorian calendar was not in use back in the year 0001 CE but I don't think IRIS has support for Julius Caesar's calendar. Passing a 'mindate' argument value more negative than -672045 will get you an <ILLEGAL VALUE> signal from $ZDATE() because there is some debate about whether the previous year is 0000 CE, -0001 CE or +0001 BCE.
go to post Steven Hobbs · May 1 $ZT is the abbreviation for the $ZTRAP system variable. You learn more about it at https://docs.intersystems.com/iris20251/csp/docbook/Doc.View.cls?KEY=RCO... Setting SET $ZTRAP="^%ETSDK" will call a routine name ^%ETSDK but I have no idea what the routine is. Maybe it is a special Error Trap routine used at your site. Executing SET $ZTRAP="^%ETN"in an ObjectScript routine will setup the ^%ETN utility routine as an error trap routine which will catch an error signal and then dump the process state into the namespace before terminating the process. Later, executing the DO ^%ERN utility will allow you to examine the various error trap dumps in a namespace.
go to post Steven Hobbs · Mar 17 If you are doing a moderate amount of computing between database access then there is no question that Python is faster then ObjectScript. If you are doing heavy computing then a call out to C++ or C would be faster than both ObjectScript and Python. But if you doing lots of database accesses with only simple string compares and string computations between each access then ObjectScript will usually be fastest because it is so close to the Globals engine that is doing all the accesses. Generally, manipulation of strings in ObjectScript is reasonably good. Parsing of JSON and XML is OK if you are using functions built into ObjectScript to do the parsing. But if you are doing character-by-character parsing of complex data structures that are not supported by ObjectScript built-ins then it might be best to call out of ObjectScript to do that parsing.
go to post Steven Hobbs · Mar 12 An aside: when you have two $LIST values, L1 and L2 ,that you want to concatenate then you can just compute L1_L2 and string concatenation will do the job. Some programmers loop over L2 one element at at time and add that element to the end of L1. This can take n-squared time based on some product of the $LISTLENGTHs of L1 and L2. Using string concatenation just depends on the sum on the lengths of L1 and L2.
go to post Steven Hobbs · Mar 12 Interestingly, for small examples, L<30, the first example is slower because it does both a $LISTBUILD plus a concatenation while the others just call a $LIST/$LISTUPDATE built-in. But for large examples, L>1000, the first example can win by at least one order of magnitude because the other examples include execution steps that take O(L**2) time.
go to post Steven Hobbs · Feb 25 You mention that you are using IRIS version 2024.1. The 3 argument form of %GetNext first appeared in IRIS version 2024.3 so it is possible that HealthShare is already using the 3 argument %GetNext method.
go to post Steven Hobbs · Feb 25 The %GetNext() method calls in the HS.FHIR.DTL.Util.JSON.Adapter class must have been written using 2 arguments and looking something like while iter.%GetNext(.Name, .Value) { When the two argument form of %GetNext encounters a %String value, it always return an ObjectScript string in the 2nd argument which limits the string length to 3641144 characters. However, there is a 3 argument form of %GetNext(.Name, .Value, .Type) which never signals <MAXSTRING>. If a string element in a %DynamicArray/%DynamicObject (%DA/%DO) element is too long then a %Stream oref is returned in the .Value argument while the .Type argument variable contains the value "string". If .Type is not "string" but is instead "oref" then that tells you the element in the %DA/%DO was originally a %Stream oref and not a %String value. See the Class Reference web page describing the %Iterator.Object class which will explain how to use the 3rd .Type argument to solve ambiguities that occur when %GetNext must convert the %DA/DO value returned in .Value to an ObjectScript value different from the original JSON value. You should suggest to HealthShare developers that their HS.FHIR.DTL.Util.JSON.Adapter class (and maybe some other classes) should be using the 3 argument form of %GetNext(.Name,.Value,.Type) instead of the 2 argument form %GetNext(.Name,.Value) in order to eliminate possible <MAXSTRING> and <MAXNUMBER> errors.
go to post Steven Hobbs · Jan 16 Is the file you are editing in IRISLIB or some other Read-Only database? If so, use the System Management Portal (SMP) to turn off the database Read-Only option.
go to post Steven Hobbs · Dec 16, 2024 I am not sure exactly what you want to do. Is check1 just an ordinary string, or might it sometimes be a string that starts with a number (like a %Status variable)? The unary plus operator (+) turns a string operand into a number. If the string starts with the syntax of a number then +string returns that number and ignores characters in the string following the numeric syntax. If the string does not start with a number (this includes the empty string) then +string returns 0. It is possible for +string to signal <MAXNUMBER> if the characters at the beginning of string are too large for conversion to the computational numbers supported by ObjectScript. You could try ret:+check1 check1which would return check1 only if it were a non-empty string that starts with a non-zero number. If you want to return a string value that could be tested as a boolean that you have to append "1" to the front of the string if +string=0 is true.
go to post Steven Hobbs · Nov 25, 2024 The Class Reference pages for IRIS, available in any IRIS installation, describe all the classes installed in the installation. The Class Reference for the %Regex.Matcher class documents that the %Regex.Matcher class comes from the International Components for Unicode (ICU). The ICU maintains web pages at https://icu.unicode.org . The class reference documentation also contains the following statement for ICU documentation specific to the IRIS %Regex: {quote}The definition and features of the ICU regular expression package can be found in https://unicode-org.github.io/icu/userguide/strings/regexp.html .{quote} Additional documentation on the Unicode Regex package specific to how it interacts with the Unicode character set can be found at https://www.unicode.org/reports/tr18/ .
go to post Steven Hobbs · Oct 15, 2024 A quick comment on performance. A TRY ... CATCH is very efficient at run time if no exception occurs. The only extra code that is executed is a jump at the at end of the TRY block to go around the CATCH block. However, TRY ... CATCH is slower than using the %ZTRAP when an execption does occur. At compile time TRY ... CATCH blocks build tables that include program counters that describe which object code locations are in each TRY block and where the corresponding CATCH block begins. When an error is signaled (or THROWn) at run time, it is necessary to scan the frame stack searching the compile-time generated TRY ... CATCH tables to find the CATCH location to go to after popping the intermediate stack frames. If there are no TRY ... CATCH blocks involved then it is usually faster to just jump to the most recent setting of the $ZTRAP variable. When exceptions are "exceptional" then TRY ... CATCH is most efficient since it avoids pushing $ZTRAP value in preparation for handling an exception and it also avoids popping the $ZTRAP value when the exception does not occur. If exceptions are part of a usual, successful, and often executed code path then it might be better to use $ZTRAP to catch those frequently executed signals.
go to post Steven Hobbs · Oct 8, 2024 Upgrading from an 8-bit instance to a Unicode instance is much simpler as you can just skip your export and import steps. Instead, just reinstall your original IRIS kit as an update kit. During the update, the installation will ask you: Do you want to convert 8-bit to Unicode <No>? Just answer Yes and the instance will be converted. Whenever a string value in a ^global variable contains only 8-bit characters then a Unicode IRIS instance stores that string in IRIS.DAT using 8-bit representation in order to save space. After the update, all your existing global data items are still there and the strings are all in 8-bit. IRIS Unicode instances use the UTF-16 Unicode encoding. If you have any 8-bit strings encoded in UTF-8 then you can use $ZCVT to convert UTF-8 strings to the IRIS default Unicode representation which uses UTF-16. Functions like $wlength, $wextract, etc. do not work on UTF-8 encoded 8-bit strings but they do work on the UTF-16 encoded strings. Note, if you do port IRIS.DAT files between different hardware instances and you also port between big-endian hardware and little-endian hardware (e.g., aix to windows) then there is a documented utility that describes how to convert the IRIS.DAT files between big-endian and little-endian representation. There is no support for automatic conversion starting from a Unicode IRIS.DAT file back to an 8-bit IRIS.DAT file. You can imagine this working if you are very lucky and the ported Unicode IRIS.DAT files just happen to have no Unicode strings, which will not happen with the "%SYS" namespace because the upgrade will add Unicode support to that namespace which will include some Unicode strings. With only a few, easily found Unicode strings then you can use %ZCVT to convert UTF-16 to 8-bit UTF-8. If are so lucky that you can do those conversions to completely remove all UTF-16 strings from a IRIS Unicode instance then you can try to install a new 8-bit instance and keep the IRISSYS and IRISLIB databases and replace the other database files with IRIS.DAT files that now just contain 8-bit user string data. If you fail to convert all the Unicode strings while trying to go back to an 8-bit instance then I believe you will get a <WIDE CHAR> signal if you attempt to access wide UTF-16 data.
go to post Steven Hobbs · Sep 26, 2024 You cannot store {{}} as it is an object reference (oref) and the translation of the oref to a %String is not useful. You can try: Parameter STATUSES = {{}}.%ToJSON(); and where you write 'w ..#STATUSES' you can instead write 'w [].%FromJSON(..#STATUSES)' I assume you can also write 'w [].%FromJSON(##class(Test...).#STATUSES)' although I am not certain that will do what you desire. [[ There is an ancient request for "DWIM" software (Do What I Mean software) to replace "DWIS" software (Do What I Said software). ]] And of course you Parameter modification can be written as: d $system.OBJ.UpdateConfigParam("Test","STATUSES",{"a":10}.%ToJSON())
go to post Steven Hobbs · Aug 13, 2024 The %DynamicAbstractObject (%DAO) subclasses, including the %DynamicArray and %DynamicObject classes, provide the best way to read/send JSON representation into/out-of IRIS databases. The %FromJSON method can create a tree of %DAO objects from external JSON representation. Then elements of %DAO data can be read or accessed using ObjectScript property syntax and using methods like %Get and %GetIterator. Data inside an IRIS database can be used to build a tree of %DAO objects by using the ObjectScript JSON constructor syntax, property assignment statements and using methods like %Set and %Push. If all you do is store values into previously undefined elements of a %DAO object then the performance is usually quite good. Then that %DAO tree can be turned into JSON representation using the %ToJSON method. It is not recommended that data in a tree of %DAO objects be extensively and repetitively modified/updated by the %Set method or by other ways that can modify data in a %DAO. This is especially true if you are repetitively changing the size of data elements. In such cases, the data in %DAO objects should be extracted and converted to some other optimized database structure. Often this can be Relational DataBase access if a mix of rectangular arrays provides the desirable access. It can be a %Stream if sequential access is needed. If a number of data elements are to be searched then columnar vectors can be used. IRIS can supply all these other access methods by layering them on top of ObjectScript multi-dimensional arrays. The IRIS multi-dimensional arrays (both global and local) provide the best tradeoffs across a large variety of differently ordered read/write/update/modification operations. If you have data in a %DAO tree and you need to make small or infrequent modifications of the data then it is acceptable to leave it in %DAO representation. You can also make repetitive reads of %DAO data without needing to convert to some other representation. But otherwise, %DAO objects are best limited to moving data into and out-of JSON representation.
go to post Steven Hobbs · Jul 25, 2024 Assuming that response.message_code is part of an ObjectScript expression then you want to evaluate response."message_code" because _ is an ObjectScript operator so you must quote "message_code" to change it into a method or property name. Otherwise, _ is the string concatenation operator so the value of response.message will be concatenated with the value of code.
go to post Steven Hobbs · Jul 16, 2024 I agree that LOG^%ETN, and ^%ETN as a process terminating trap routine, are the best way to log problems. But we should also mention that DO ^%ERN is the utility that will dump the data that LOG^%ETN places in the ^ERRORS multi-dim global. That dump includes state information, a complete call-stack dump, a list of all active local variables at each stack level, and in recent IRIS releases, it also includes a dump of all Class Objects with an oref active in memory. This is a lot of information for each ^ERRORS entry so you might not want to be over-enthusiastic with your calls on LOG^%ETN.
go to post Steven Hobbs · Jul 15, 2024 I don't know anything about the FhirBinary class but I do know why the %GetNext method in the %Iterator.Array or %Iterator.Object classes is giving the <MAXSTRING> signal. Let's assume the %GetNext in question is accessing a %DynamicObject as the %DynamicArray case is identical. A %DynamicObject (usually created by the %FromJSON method) limits it contained string elements only by the amount of memory that can be obtained from the Operating System running the IRIS instance. Fetching a long string element from a %DynamicObject as an ObjectScript string value will signal <MAXSTRING> when the %DynamicObject string element is too long. However, you can fetch a long string element as a %Stream.DynamicBinary or %Stream.DynamicCharacter class object. Such a %Stream object can contain any string that will into the system memory. Long %Stream contents longer than the ObjectScript maximum string length can be accessed by using the Read(.len,.status) method to access smaller substrings of the %Stream contents. Also, you can copy one class of long %Stream into a different class of long %Stream if you want to move a long string out of memory and into either a a file or an ObjectScript global array. The Class Reference documentation for the %Iterator.Object class does not directly appear in either Class Reference webpage on an IRIS instance nor in the Class Reference web page in the InterSystems IRIS network documentation. This strikes me as a documentation bug. Fortunately you can see the appropriate Class Reference documentation by indirectly going to the Class Reference page for the %Library.DynamicObject class (or %Library.DynamicArray class) and going to the documentation for %GetIterator method. That documentaton contains a link to the %Iterator.Object class. Click on that link and you will go the %Iterator.Object Class Reference page where you can see the documentation for the GetNext(.key,.value,.type) method. %GetNext method is used to iterate over the values in a %DynamicObject. The third argument, .type, is optional. The two argument form, %GetNext(.key,.value) will return the key index in .key and its associated ObjectScript value in .value. If the element value cannot be converted to an ObjectScript value then you will get a <MAXSTRING> or <MAXNUMBER> signal. However, the three argument form, %GetNext(.key,.value,.type) will not signal these errors. In place of a <MAXSTRING> signal, the .type argument will contain "string" and the the .value argument will contain the appropriate in-memory %Stream object value. In place of a <MAXNUMBER> signal, the .type argument will contain "number" and .value will contain a string with the decimal representation of the numeric value. More detail of other uses for the .type argument of the %GetNext(.key,.value,.type) method call can be found on the %Iterator.Object (or %Iterator.Array) Class Reference web page. When the Fhir.Binary class needs to iterate over the elements of a %DynamicArray/Object then it should not use the 2 argument form of %GetNext(.key,.value) but should instead use the the 3 argument form %GetNext(.key,.value,.type) and be prepared when the type of the the .value variable does not match the type specified by the .type argument.
go to post Steven Hobbs · Jun 25, 2024 We need to see more of your code to see what is truly happening. The routine method call Do obj.%Set("data",pStream,"stream>base64") should have no problem providing your system has enough memory to construct entire 'obj' Dynamic Object in your process memory. Your system will want to avoid having many large Dynamic Objects in memory at the same time. If the system runs out memory for all the active processes then the operating system will usually start aborting processes randomly until the operating system decides it now has enough free memory to continue execution. Calling %Set(,,) to read JSON text from a very large %Stream should only use moderate size buffers on the string stack. If a call on %Set(,,) writing from a %Stream generates a <STRINGSTACK> signal then that should be reported to InterSystems as a bug. However, your reply mentions the %ToJSON() function method call. Calling %ToJSON as a function will place the result on the String Stack so it is possible such a function call could signal a <STRINGSTACK> error although if the %ToJSON() function call appears in a very simple expression then I would expect a <MAXSTRING> signal instead of a <STRINGSTACK> signal. Calling %ToJSON() or %ToJSON(outstrm) as a routine method call in a DO statement should not get either a <STRINGSTACK> or a <MAXSTRING> signal since the result is not going to an ObjectScript string value.
go to post Steven Hobbs · Jun 24, 2024 The %FromJSONFile(file) method does not take a %File object as an argument but instead takes a string containing the name of file. You need to replace set newObject = {}.%FromJSONFile(file) with one of set newObject = {}.%FromJSONFile("/tmp/longObjectFile.txt")set newObject = {}.%FromJSON(file) Originally, %FromJSON(input) would accept the 'input' argument as a %File object, a %Stream object, a JSON text string or a filename text string. However, accepting an argument string as either JSON or a filename was considered to be security issue because a malicious user could be requested to enter a json-string but, being malicious, the user would instead enter a filename-string, or vice-versa. As a result %FromJSON no longer accepts filename strings while %FromJSONFile only accepts filename strings. No longer an issue, but it would be possible that legal VMS filename syntax could also be legal JSON text string syntax and such an argument string might be ambiguous on a VMS system. But IRIS no longer supports VMS and Caché for VMS did not support Dynamic Objects.