All
Accepted answers
Steven Hobbs · Jun 24, 2024 go to post

The %FromJSONFile(file) method does not take a %File object as an argument but instead takes a string containing the name of file.  You need to replace

set newObject = {}.%FromJSONFile(file)

with one of

set newObject = {}.%FromJSONFile("/tmp/longObjectFile.txt")
set newObject = {}.%FromJSON(file)

Originally, %FromJSON(input) would accept the 'input' argument as a %File object, a %Stream object, a JSON text string or a filename text string.  However, accepting an argument string as either JSON or a filename was considered to be security issue because a malicious user could be requested to enter a json-string but, being malicious, the user would instead enter a filename-string, or vice-versa.  As a result %FromJSON no longer accepts filename strings while %FromJSONFile only accepts filename strings.

No longer an issue, but it would be possible that legal VMS filename syntax could also be legal JSON text string syntax and such an argument string might be ambiguous on a VMS system.  But IRIS no longer supports VMS and Caché for VMS did not support Dynamic Objects.

Steven Hobbs · Jun 13, 2024 go to post

%DynamicAbstractObject-s provides the fastest for reading JSON from an external location and once in memory it provides excellent random access reading of data components.  Large JSON arrays/objects are supported.  I have done %FromJSON on a 20GB JSON file on a system with 16GB of physical memory—required paging of memory but still acceptable performance.  You don’t require random access performance and it is best to avoid using more memory than you need.  Maybe someday InterSystems can provide methods for reading JSON subobjects while remember how that object was nested in its parent objects.

Assuming your JSON is in a file or stream and the outer array contains only elements which are arrays or objects you can try the following.  1. First read the leading ‘[‘;  2. Then use %FromJSON to read the array element and process that JSON element (the read will stop on the closing ‘]’ or ‘}’);  3. Read single chars from your stream/file skipping white space and if that char is a ‘,’ then go back to step 2 but if that char is ‘]’ then you are done.

Steven Hobbs · May 17, 2024 go to post

The documentation is not quite clear on all this.

If you have written data to a temporary file before first calling the LinkToFile(filename) method then the previously written temporary file disappears and future Read and Write method calls go to the new filename.  If you later do a LinkToFile to a different filename then you will detach from the previous filename and attach to the new filename.  But the previous file data is still available if you later attach back to the old filename.

When you do a LinkToFile(filename) when there is no existing file with that filename then a new file is created.  If your filename describes an existing file then your %Stream.FileBinary is attached to the existing file.  If the first thing you do after LinkToFile(filename) are Read() method calls (or a Rewind() followed by Read() method calls) then you will Read the existing contents of the file.  If after the LinkToFile, you instead do a MoveToEnd() method call followed by Write() method calls, then you keep the current file contents and your data is appended to the end of the existing file.  If you do a Write() immediately after a LinkToFile() (or immediately after either a Rewind() or a Read()) then existing file data is deleted and you Write() to the beginning of the existing file.  It is possible to Read() data from an existing file by avoiding Write() method calls before you Read() the data.  it is possible to append new data to an existing file by using MoveToEnd() followed by Write() method calls.

Steven Hobbs · Apr 10, 2024 go to post

Most of the ObjectScript examples above (but not the Python example) take O(n**2) time (i.e., quadratic time) where 'n' is number of values in X.  This happens because the $PIECE(X,",",i) starts searching from the beginning of the string or because shortening the string X with

set x=$p(x,",",2,*)

will copy the string 'x' multiple times with the first element deleted before each copy.

The ObjectScript example from Timo Lindenschmid  using $LISTNEXT takes O(n) time (linear time).  This can be important if $L(X,",") is 1000 or more because such a list with 1000 elements will require about 500,000 scans of "," characters or about 500,000 copies of string characters.

Rich Taylor's Python example should also take linear time.

Another solution would be to build a %DynamicArray object and then iterate through its contents.

Finally, you could build a multi-dim ObjectScript array and iterate through the subscripts with $ORDER.  This is method closest to using standard MUMPS/ANSI M/ObjectScript features.

Steven Hobbs · Feb 15, 2024 go to post

Some additional picky details:

The unary + operator is a numeric operator so it converts its operand to a number.  If a string operand starts with a fractional number then unary + produces a fractional number (in its canonical numeric form) and it throws away any unneeded characters.  If you want your numeric result to be an integer then you need to throw away the fractional digits by doing an integer-division by 1.  Since the integer-division operator, \, is a numeric operator it always converts its operands to numbers so you no longer need the unary + to do the conversion of a string to numeric representation.  E.g.s:

USER>w "0012.543000abc"   
0012.543000abc
USER>w +"0012.543000abc"
12.543
USER>w +"0012.543000abc"\1
12
USER>w "0012.543000abc"\1 
12
 

Steven Hobbs · Jan 31, 2024 go to post

The third argument to %GetNext is not a string value.  It is a variable passed by reference:  iterator.%GetNext(.key,.value,.type).  When this returns then the variable type will contain a string value like “string”, “number”,”object”, etc.  If type contains “string” and $isobject(value) is true then value is a %Stream object containing your long string.

Steven Hobbs · Jan 30, 2024 go to post

If your method can handle %Stream data then you can copy the large string piece-by-piece into an appropriate %Stream.  If your method cannot handle %Stream data then you must modify it to test for a parameter that is a %Stream object instead of a string and then process that data by calling the stream.Read() method to scan through the %Stream.

A few built-in methods do have this capability.  For example, the %FromJSON(arg) method in the %DynamicAbstractObject class will accept 'arg' being either a string or a %Stream object.

Steven Hobbs · Jan 25, 2024 go to post

Well, my DO command suggestion won't work in ObjectScript because ObjectScript has extended the MUMPS DO command to accept syntax for the
   DO { code } WHILE expr,expr,...
statement.  The code block in the DO-WHILE extension does not assume the code block opens a new variable scope.

However, like most languages designed in the 1950s and 1960s, (including FORTRAN, BASIC and PL/I, but not C) MUMPS does not have reserved words.  Therefore, any newly added command can start with a new command word.  So we could have BLOCK { code } or SCOPE { code } as new commands that open a new variable scope.  We might be able to find a better command word than BLOCK or SCOPE.  We could even consider NEW { code }, although it might be considered confusing having an additional NEW command (one with a curly brace code block along with one with no arguments, one with list of argument names and one with a list of names surrounded by a pair of round braces).

Steven Hobbs · Jan 25, 2024 go to post

I always wanted to extend argumentless  DO so that you could get a new scope without needing the fairly ugly dot-indentation lines.  Then you write something like

Do {
   New $roles
   Set $roles=""
   If ($extract($roles,1,$length("%All"))="%All") Set tHaveAllRole = 1
}

You could even write the whole thing on one line like

Do {New $roles  Set $roles=""  If ($extract($roles,1,$length("%All"))="%All")  Set tHaveAllRole = 1}

Steven Hobbs · Jan 24, 2024 go to post

There are two places where a <MAXSTRING> signal could give you problems with JSON stored in a %DynamciAbstractObject.  It can occur if the input is too long and it can occur when evaluating a JSON element that is too large.

The %FromJSON(stream) method can read JSON syntax directly from a stream of any length providing the operating system will give you enough virtual memory to hold the %DynamicAbstractObject.  You seem to already understand this although you don't have to make a copy of an existing %Stream value (although it might need a stream.Rewind() call before you pass the %Stream to %FromJSON.)

Evaluating dynobj.%Get("pdfkey") can give you a <MAXSTRING> signal if the "pdfkey" element of the %DynamicObject referenced by the oref in the local variable 'dynobj' is very long.  But look at the class reference description of the %Get method in the %DynamicObject class and you will see that the %Get method can have optional 2nd and 3rd parameters.  The value of dynobj.%Get("pdfkey",,"stream") will return a %Stream containing the characters of the "pdfkey" element without generating a <MAXSTRING> signal.  Even better, the value of dynobj.%Get("pdfkey",,"stream<base64") will be a %Stream containing the "pdfkey" element decoded out of the base64 encoding.

If you want to iterate through all the elements in a %DynamicObject then you can use the dynobj%GetIterator() method to create a %Iterator.Object class object.  The %GetNext(key,.value,.type) method of the %Iterator.Object class can assign a %Stream to the .value parameter when the selected "key" element contains a long string value *providing* the optional .type parameter is also present.  The .type parameter receives the %GetTypeOf value of the original object element so you can tell the difference between a long "string" type element returning a %Stream versus an "object" type element that contains a %Stream oref value.

Steven Hobbs · Nov 13, 2023 go to post

You should remember that the types defined in the formal argument list of a Method declaration or defined when a local variable uses the #dim preprocessor directive are only advisory types.  Any ObjectScipt value or oref can passed as an actual arugment to a method.  A local variable specified in a #dim directive can contain any ObjectScript value or oref--not just the type/class specified in the #dim directive.

The type of a Property name is only checked when the %Save method (or other appropriate method) is applied to an object containing that Property.  While the object is active in memory you can assign any ObjectScript value or oref to that property.  You can choose to write you own specific PropertySet/PropertyGet methods containing code that applies your choice of run-time type checking and conversions during the modifications/evaluations of Property names.

The ObjectScript language has type-less variable contents and the types in the Class language extensions to ObjectScript are only enforced by run-time executed methods.

But of course, the Classes in the %Dictionary Package are useful for writing run-time methods that enforce the declared types of Properties.

Steven Hobbs · Nov 5, 2023 go to post

Internally, a %DynamicArray or %DynamicObject is usually limited only by the amount of memory that the OS is willing to allocate to the job,  Because the components of such objects are not limited by their size, some very large components need to be allocated by a non-standard allocator that does not participate with the $ZSTORAGE limit.  But some components of a %DynamicObject/Array are allocated by the usual memory allocator so %ZSTORAGE might limit the size.

Since %DynamicObject/Array classes are always stored in memory (I once built one that needed 20GB of memory) you want to limit how many large objects are active at the same time.  When you encounter a large %DynamicObject/Array you want to move its data to a file or global %Stream before you start reading in another large %DynamicObject/Array,

The %FromJSON(…) method can create a %DynamicObject/Array while reading JSON from any kind of %Stream.  The %ToJSON(…) method can write JSON text to any kind of %Stream.  A %Stream is not usually limited by the max supported length of an ObjectScript string and the %FromJSON and %ToJSON methods read/write %Stream data using buffers smaller than the max supported string length.

If you have a %DynamicObject/Array with a string component larger than the ObjectScript max string length then you can use the three argument %Get(key,default,type) method where the ‘type’ argument specifies a “stream” type.  The resulting %Stream is still stored in memory but that %Stream is readonly and will often share the memory allocation with string component in the %DynamicObject/Array. That in-memory %Stream can be copied into a file or global %Stream.  The three argument %Set(key,value,type) method can create a %DynamicObject/Array component of any length when the ‘type’ argument specifies that the ‘value’ argument is a %Stream.  If the %Set ‘value’ argument is a readonly, in-memory %Stream created by %Get(key,default,”stream”) then the component created by %Set will usually share in-memory data with its  ‘value’ argument stream.

See the class reference documentation for %Get and %Set.  The ‘type’ argument options include the capabilities to encode to, or to decode from, BASE64 representation.  This can be useful when working with a BASE64 encoded .pdf file in a JSON object.

Steven Hobbs · Oct 5, 2023 go to post

Yet another example:

USER>s ^abc(1,2)=3

USER>s temp="1,2"

USER>w @("^abc("_temp_")")
3

;; Three subscripts referencing an undefined ^abc entry
USER>set temp3 = "1,2,3"

USER>w @("^abc("_temp3_")")

W @("^abc("_temp3_")")
^
<UNDEFINED> ^abc(1,2,3)

Steven Hobbs · Aug 11, 2023 go to post

The numeric values inside a %DynamicObject can be a JSON number,  an ObjectScript Decimal Floating-Point number, an IEEE Binary Floating-Point number or an ObjectScript string containing the characters of an ObjectScript numeric literal.  The ObjectScript Decimal and the IEEE Binary numeric types do not keep track of trailing zeroes.  The JSON numbers do keep track of the trailing zero.  A JSON number or an ObjectScript string containing a numeric literal will loose their trailing zeroes as soon as an ObjectScript arithmetic expression uses those values as an operand.

Creating a %DynamicArray with the square-bracket syntax, [ ], or creating a %DynamicObject with the curly-bracket syntax, { }, will do parsing using JSON syntax which keeps the trailing zeroes.  However, if an ObjectScript expression is using the [ ] or { } syntax and an enclosed value is further enclosed inside round-bracket syntax then round brackets will enclose a single ObjectScript run-time expression which will be parsed using ObjectScript syntax rules.  (See json3 variable below for an example.)

If you evaluate a %DynamicObject element containing a JSON number with either ObjectScript property evaluation or with a simple ObjectScript %Get("propName") method evaluation then you get an ObjectScript Decimal value (or maybe an IEEE Binary value.)  E.g.:

USER>set json2 = { "decimal": 12.000, "decimal2":12.000E150}     

USER>zwrite json2                                                
json2={"decimal":12.000, "decimal2":12.000E150}  ; <DYNAMIC OBJECT>

USER>write json2.decimal,!,json2.%Get("decimal"),!,json2.decimal2
12
12
12000000000000000496000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Note that the JSON 12.000E150 was converted to the IEEE Binary value $DOUBLE(12.000E150) which is written containing the digits "496" as the 18th, 19th and 20th significant digits.  This occurs because of the rounding that occurs when doing the default conversions in both directions between binary floating-point values and decimal textual values.

[[ In case you want to see the fully accurate value of $DOUBLE(12.000E150), without any rounding modifications, the decimal value would be:
USER>write $FNUMBER(json2.decimal2, "G", 152)
12000000000000000496865878364855385636201248056368359870735338409167261627934342176354867336792577149743475436957677626049325994240678780913402110803968.0
which has 151significant decimal digits in its fully accurate representation in decimal. ]]

A more complex %Get method evaluation, %Get("decimal2", , "json") , will return a string containing the JSON syntax representation of the element.  E.g.:

USER>set json3 = { "decimal2":12.000E150, "OSnumber":(.12)}                                

USER>write json3.%Get("decimal2", , "json"),!,json3.OSnumber,!,json3.%Get("OSnumber", , "json")
12.000E150
.12
0.12

Note that .12 is an ObjectScript numeric value, while the strings 12.000E150 and 0.12 are syntactically legal JSON values.

It would be nice if the  %Set(key,value,type) method of the %DynamicObject class accepted "json" as a supported 'type' argument value, *BUT* it does not.  That would make it possible to add a JSON syntax numeric literal to an existing %DynamicObject using
    DO json3.%Set("decimal2", "12.000E150", "json")
Maybe someone should suggest such a feature extension to the %Set method.

Steven Hobbs · Jul 23, 2023 go to post

%FromJSON formerly accepted file names but this was reported as a security problem where input intended to be JSON was instead interpreted as a file name, or vice-versa.  As a result, recent %FromJSON method calls will not process file names but the %FromJSONFile method was added to support that case.

Steven Hobbs · Jul 21, 2023 go to post

The ##class(%DynamicAbstractObject).%FromJSON(s) class method, where 's' is a %String containing a JSON array/object or where 's' is a %Stream containing a JSON array/object, will parse the input JSON.  If the parse is successful then it returns an object reference of either %DynamicArray or %DynamicObject classes.  If the parse is unsuccessful then it will signal an error.

The parsing done by the %FromJSON class method is forgiving of at least one common syntax error.  The input string "[0.1,.1]" is not legal JSON because the second array number does not have a digit before the decimal point.  The %FromJSON class method will accept this input and silently supply the missing 0 digit.  If you apply the %ToJSON() method to the resulting %DynamicArray object then the result will be the string "[0.1,0.1]", which is legal JSON syntax.

There is also a %FromJSONFile (filename) class method which will accept a JSON array/object from a file with the specified 'filename'.

Steven Hobbs · Jun 13, 2023 go to post

On Caché and IRiS, both %ZU(0) and $ZU(1) will signal <FUNCTION> because they need more than 1 arugment.

On IRIS, $ZU(0,directoryName,....) with enough additional arguments will create an IRIS.DAT database file. and $ZU(1,...) will modify an existing IRIS.DAT database file.  On Caché these functions create/modify a CACHE.DAT database file.  These are legacy functions.

The Config.Database class provides ways to modify the CPF configuration file to create and configure databases.

Interactively you also use the System Management Portal (SMP) by clicking SystemAdministration > Configuration > SystemConfiguration > LocalDatabase to find the SMP webpage that can create or modify local databases.

Or you can use the DO ^|"%SYS"|DATABASE command line utility as a way to create and configure databases.

Steven Hobbs · May 26, 2023 go to post

InterSystems changed how <MAXNUMBER> was signaled during conversion from text representation to numeric representation when support for 64-bit IEEE binary floating point was added to Caché.  Textual numbers that overflowed the default ObjectScript decimal representation are converted to 64-bit IEEE binary floating point which supports a much larger range of magnitudes (but about 3 fewer digits of decimal precision.)  When a literal would exceed the magnitude of IEEE floating point, the choice of whether to signal <MAXNUMBER> depends on the run-time $SYSTEM.Process.IEEEError setting since IEEE floating-point overflow can either signal <MAXNUMBER> or it can return an IEEE floating-point infinity.  When the compiler sees a numeric literal that exceeds the finite IEEE number range then the decision to signal an error is delayed until run-time execution so the current $SYSTEM.Process.IEEEError setting can be obeyed.

Steven Hobbs · Feb 28, 2023 go to post

Can I assume this means that using the ObjectScript ZBREAK command to set a breakpoint on a method name will no longer need the leading "z" on the method name but the class name will still need the trailing ".1" ?

Steven Hobbs · Feb 8, 2023 go to post

The quoting of the slash character, "/", is optional in JSON.  When doing a $ZCVT output conversion ,"O", in JSON IRIS chooses to not quote the slash character.  It makes the output easier to read.  However, $ZCVT doing an input conversion, "I", will recognize a quoted slash character in JSON input.  E.g.:

USER>w $zcvt("abc\/def","I","JSON")
abc/def
USER>w $zcvt("abc/def","I","JSON") 
abc/def
 

Steven Hobbs · Jan 18, 2023 go to post

A future release of IRIS will restore the ability of %DynamicArray and %DynamicObject classes to dispatch method calls on the method names of the form "propertyGet" and "propertySet".  However,  $PROPERTY(dynobj,"Id") should be more efficient than $METHOD(dynobj,"IdGet") so we recommend using $PROPERTY over using $METHOD when working with %DynamicObject class objects.  When the property name is known before execution time then dynobj.Id is the best way to do a simple property access of a %DynamicObject element with key name "Id".

A change was made to how IRIS dispatched method calls because there existed a few unusual cases where there was no property named "property" but the programmer was trying to dynamically dispatch to a method with the name "propertyGet" and the result was an undefined property error rather than a method call.  The change was made so that a method named "propertyGet" could always be correctly called when there was no property named "property".

Every possible string is a legal %DynamicObject key name and if the key name element is not otherwise defined then that key name element contains the empty string value.  The implementation of the %DynamicObject class does not include a list of all legal property names since all key name strings are legal.  The new way of dynamically dispatching method calls with names like "propertyGet" would first check if the property named "property" existed before attempting a property dispatch instead of method dispatch.  Since a %DynamicObject class object did not contain the name "property" in the list of legal property names and did not contain a method called "propertyGet" in the list of legal method names, the result of evaluating $METHOD(dynobj,"propertyGet") was a <METHOD DOES NOT EXIST> signal.  The future release of IRIS will have $METHOD(dynobj,"propertyGet") dispatch a property access rather than attempting a dynamic dispatch of a method access.

I should point out that "%" is a legal key name string but $METHOD(dynobj,"%Get) did not fetch (and will not fetch) the element with key name "%" because a %DynamicObject does contain a method named "%Get".  A call on a defined method name always occurs before attempting a dynamic method dispatch.

See the documentation on the

methods which can be used with subclasses of the %RegisteredObject class.
Steven Hobbs · Dec 16, 2022 go to post

Another translation is

 Read *R:20
 ;; Test error case
 If '$Test { Use 0 Write !!!,"Expired time." Quit }
 ;; Test character "a" case
 If $c(R)="a" {
   Use 0 Write !!!,"A letter a has been read."
   Quit
 }
 ;; I added more code here to demonstrate "fall through" in original
 ;; when neither timeout nor "a" occurs
 Use 0
 Write !,"A character other than ""a"" was read"
 Quit

 ;; Since all 3 cases execute Use 0, this statement can be placed after Read and the other 3 copies deleted
 

Steven Hobbs · Dec 13, 2022 go to post

It might be easier just to always make your BREAK commands conditional like (Robert Cemper suggests above).  Do it like:

    BREAK:$GET(^|"USER"|MyNameBreakAppEnable)

Use your own name for "MyName".

Then you do SET ^|"USER"|MyNameBreakAppEnable=1 to enable break points and
KILL ^|"USER"|MyNameBreakAppEnable to turn break points off.

Change the name of "App" to conditionalize break points for use in different applications.  And "USER" is a scratch namespace available on all the instances you plan to do debugging on.

Steven Hobbs · Nov 30, 2022 go to post

The ObjectScript command
   ZBREAK %method^%CSP.Broker.1
or
   ZBREAK zmethod^%CSP.Broker.1
will allow you to set an ObjectScript debugging breakpoint at the entry of "%method" or "method" of that class.  The ObjectScript commands  BREAK "S" or BREAK "S+" would then allow you to single step through the statements in that method.  See documentation on ObjectScript debugging for more details.

Unfortunately, the ObjectScript routine %CSP.Broker.1 on my IRIS instance is "deployed" which means the %CSP.Broker.1.int routine source code is not available and I assume the %CSP.Broker.cls source code is also not available.  Without one of these source files you will be debugging and single stepping blind through the ObjectScript commands.  Some of InterSystems classes and routines are not "deployed" which means you can do Command-line Routine Debugging.  You can also modify those classes/routines after you change the IRISLIB database from readonly to read-write.

It is best that you take the advice above to contact support in the InterSystems WRC.

Steven Hobbs · Oct 19, 2022 go to post

I am assuming your problem is that request.HttpResponse.Data.Read() is complaining because you are reading the entire pdf file into an ObjectScript variable with its maximum supported string length of 3,641,144 characters.  You will have to read it out in smaller chunks that individually fit into an ObjectScript string.  The chunksize will be important as you pass the chunked data to $system.Encryption.Base64Encode(content) and your chunks cannot end between the boundaries between two different BASE64 encoding blocks.  The results of each Base64Encode must then be sent to some form of %Stream (probably %Stream.GlobalBinary or %Stream.FileBinary) since only a %Stream can hold a block of data larger than 3,641,144 characters.  Using a small, appropriate chuncksize will limit the in-memory resources used by this conversion.

If you don't mind having the entire PDF file in memory at one time you can use %DynamicObject to hold and decode that base64 data.  The %Library.DynamicObject and %Library.DynamicArray class objects are usually used to represent data that was originally JSON encoded.  These Dynamic Objects exist only in memory but you can serialize them into JSON textual representation using the %ToJSON(output) method.  But if the JSON text representation contains more than 3,641,144 characters then you better direct 'output' into some form of %Stream.

You can convert a binary pdf file into BASE64 encoding doing something like:

SET DynObj={}  ;; Creates an empty %DynamicObject
DO Dynobj.%Set("pdffile",request.HtttpResponse.Data,"stream")
SET Base64pdf=Dynobj.%Get("pdffile",,"stream>base64")

Then Base64pdf will a readonly, in-memory %Stream.DynamicBinary object which is encoded in BASE64.  You can use Base64pdf.Read(chunksize) to read the BASE64 out of Base64pdf in ObjectScript supported chunks.  You do not have to worry about making sure the chunksize is a multiple of 3 or a multiple of 4 or a multiple of 72.  You can also copy the data in Base64pdf into a writeable %Stream.FileBinary or a %Stream.GlobalBinary using the OtherStream.CopyFrom(Base64pdf) method call.

If your HttpResponse contains a BASE64 encoded pdf file instead of a binary pdf file then you can do the reverse decoding by:

SET DynObj={}
DO Dynobj.%Set("pdffile",request.HtttpResponse.Data,"stream<base64")
SET BinaryPDF=Dynobj.%Get("pdffile",,"stream")

Then BinaryPDF is a readonly %Stream.DynamicBinary containing the decoded pdf data.  You can copy it to a %Stream.FileBinary object which can then be examined using a pdf reader application.

Steven Hobbs · Sep 30, 2022 go to post

The %ToJSON method sends JSON text to a device or %Stream one element at a time.  Starting in IRIS 2019.1.0 it broke up long JSON strings into blocks of 5460 characters so that strings exceeding the maximum length could be sent to the device.  Make sure you are using an IRIS 2019.1.0 or later if you are getting a <MAXSTRING> signal.

In a future IRIS release (now out in a preview release) a change was made such that sequences of many small items would be blocked together and sent to the device in a larger buffer.  This is being done to improve the performance of %ToJSON method when sending many small elements to a %Stream.

Steven Hobbs · Sep 30, 2022 go to post

Although InterSystems allowed unlicensed use (with some limitations) of Caché by educational institutions very few colleges/universities took advantage of this.  The exception seemed to be universities in Russia.  When we had contests for the best Caché applications we would offer the university winner a free trip to Global Summit.  However, a recent Russian student was unable to attend because she could not get a Visa to visit the US.

Maybe IRIS with embedded Python could help increase university interest in IRIS databases.

Steven Hobbs · Sep 28, 2022 go to post

The %Library.DynamicArray and %Library.DynamicObject class objects are stored only in process memory and are not affected by the 3641144 character string length of ObjectScript strings.  But they are affected by the amount of virtual memory that the operating system will grant to a process so avoid having many such large things active.  These Dynamic Objects generally hold JSON format data but they can also hold ObjectScript values.  The %ToJSON(output) method can change the contents of Dynamic Object into JSON text and it will send that text to 'output', which can be a file, a device or a %Stream.  If you need to persist a Dynamic Object in a database then you can use %ToJSON(globalstream) to persist the JSON text in a %Stream.GlobalCharacter or %Stream.GlobalBinary class object.  There is a %FromJSON(input) which can build a new Dynamic Object from 'input' which is JSON text from a string, a file, a device or a %Stream.  In recent versions of IRIS, the %Get(key,default,type) and %Set(key,value,type) methods can let you choose 'type' as some form of %Stream so you can examine and modify elements of a Dynamic Object when that element needs to be larger than 3614411 characters.  The default %Stream-s generated by the %Get method use the %Stream.DynamicBinary and %String.DynamicCharacter classes, which also store data in process memory so you should avoid having many such large streams active at the same time.  They can be copied out to other streams which can be stored a database or into a file or a device.

Steven Hobbs · Aug 26, 2022 go to post

A canonical numeric string in ObjectScript can have a very large number of digits.  Such a string can be sorted with ObjectScript sorts-after operator, ]], and reasonably long canonical numeric strings can be used as subscripts and such numeric subscripts are arranged in numerical order before all the subscript strings that do not have the canonical numeric format.

However, when ObjectScript does other kinds arithmetic on a numeric string then that string is converted to an internal format, which has a restricted range and a restricted precision.  ObjectScript currently supports two internal formats.  The default format is a decimal floating-point representation with a precision of approximately 18.96 decimal digits and a maximum number about 9.2E145.  For customers doing scientific calculations or needing a larger range, ObjectScript also supports the IEEE double-precision binary floating-point representation with a precision around 16 decimal digits and a maximum number about 1.7E308.  You get the IEEE floating-point representation with its reduced precision but its greater range by using the $double(x) function or doing arithmetic on a string which would convert to a numeric value beyond the range of the ObjectScript decimal floating-point representation.  When doing arithmetic that combines ObjectScript decimal floating-point values with IEEE binary floating-point values then the decimal floating-point values will be converted to IEEE binary floating point before performing the arithmetic operation.

Here are more picky details.

The ObjectScript decimal floating-point representation has a 64-bit signed significand with a value between -9223372036854775808 and +9223372036854775807 combined with a decimal exponent multiplier between 10**-128 and 10**127.  I.e., a 64-bit twos-complement integer significand and a signed byte as the decimal exponent.  This decimal floating-point representation can exactly represent decimal fractions like 0.1 or 0.005.

The IEEE binary floating-point representation has a sign-bit, an 11-bit exponent exponent encoding, and a 52 bit significand encoding. The significand usually encodes a 53-bit range values between 1.0 and 2.0 - 2**-52 and the exponent usually encodes a power-of-two multiplier between 2**1023 and 2**-1022.  However, certain other encodings will handle +0, -0, +infinity, -infinity and a large number of NaNs (Not-a-Number symbols.)  There are also some encodings with less than 53 bits of precision for very small values in the underflow range of values.  IEEE 64-bit binary floating-point cannot exactly represent most decimal fractions.  The numbers $double(0.1) and $double(0.005) are approximated by values near 0.10000000000000000556 and 0.0050000000000000001041.

I have written some ObjectScript code that can do add, subtract and modulo on long canonical numeric strings for use in a banking application.  However, if you are doing some serious computations on large precision values then you should use the call-in/call-out capabilities of IRIS to access external code in a language other than ObjectScript. Python might be a good choice.  You could use canonical numeric strings as your call-in/call-out representation or you could invent a new encoding using binary string values that could be stored/fetched from an IRIS data base.

ObjectScript was designed to do efficient movements and rearrangements of data stored in data bases.  If you are doing some serious computations between your data base operations then using a programming language other than ObjectScript will probably provide better capabilities for solving your problem.

Steven Hobbs · Aug 1, 2022 go to post

The ObjectScript $ZDATETIME function (also-know-as $ZDT) contains lots of options, some of which are close to what your want.  [[ Note $HOROLOG is also-known-as $H; $ZTIMESTAMP is aka $ZTS. ]]

$ZDT($H,3,1) gives the current local time, where character positions 1-10 contain the date you want and positions 12-19 contain the time you want.  However, character position 11 contains a blank, " ", instead of a "T".

$ZDT($ZTS,3,1) gives the current UTC time with character position 11 containing a blank.

Assigning
    SET $EXTRACT(datetime,11)="T"
to your datetime result will fix the character position 11 issue.

Instead of using time format 1, you can use time formats 5  and 7 with $H.  $ZDT($H,3,5) gives local time in the format you want except character positions 20-27 contain the local offset from UTC.  $ZDT($H,3,7) converts the local $H date-time to UTC date-time and makes character position 20 contain "Z" to indicate the "Zulu" UTC time zone.  However, if your local time-zone includes a Daylight Saving Time (DST) offset change when DST "falls back" causing a local hour to be repeated then the time format 5 UTC offset or the time format 7 conversion from local to UTC will probably be incorrect during one of those repeated hours.