The king is dead, long live the king! is a traditional phrase that is proclaimed at the advent of a new monarch in France.

May I say: IRIS is dead, long live IRIS!

This is what I feel with the arrival of Embedded Python.

The old monarch spoke ObjectScript, he allowed us to accomplish great things but now let's stop looking back, let's move forward. Let's induct the new monarch who speaks Python. Let's stop worshipping his father and let's make room for the new generation.

That's why I think, we must stop coding in ObjectScript, we must strive to use Python, everywhere in IRIS and this without the wrapper ([language = python]).

Why be so radical?

  1. Coding in Python is to show by example and in a language that the uninitiated know and thus prove the extent of the possibilities of IRIS
  2. Coding in Python is to start answering and finding solutions to the new problems that we will encounter:
    a. How to integrate .py files in our CICD pipes
    b. How to integrate PyPI with ZPM
    c. How to elegantly expose future APIs coded on the Flash Framework with IRIS and CSP gateways
  3. Coding in Python encourages InterSystems to produce new packages in Python rather than ObjectScript.

Choosing your programming language in IRIS is a political act.

  • Coding in ObjectScript is to be conservative.
  • Coding in Python is to be liberal.

Choose your side.

I add the powershell version :

Invoke-WebRequest -Uri 'https://login.intersystems.com/login/SSO.UI.Login.cls?referrer=https%253A//wrc.intersystems.com/wrc/login.csp' -SessionVariable session -Method POST -Body 'UserName=<Your Login>&Password=<Your Password>' 

Invoke-WebRequest -WebSession $session -Uri https://wrc.intersystems.com/wrc/WRC.StreamServer.cls?FILE=/wrc/Live/ServerKits/IRIS-2020.1.1.408.0-win_x64.exe -outfile "iris.exe"

The difference between a stored procedure and SQL inserts is that the business logic remains on the application side and not on the database side to keep the principles (storage/logic/representation) separate.

Nevertheless, the problem doesn't seem to be at the SQL level but rather at the BPL level, so maybe it's due to the conversion of XML to object?
If it is the case, use different technique to parse the XML, like the SAX Parser which avoids to mount all the XML document in memory.
https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls...

Where do you struggle ? Is it in the BPL, in the Operation or during the insert statement to SQL Server ?

If the bottleneck is in the insert statement check out this community SQL adapter (work only on JDBC).
https://openexchange.intersystems.com/package/ETL-Interoperability-Adapter

This adapter provide you an access to insert batch statement, that can speed up the insert speed up to 10 times even 100 times.

In the github repository you have some example how to use it :
https://github.com/grongierisc/BatchSqlOutboundAdapter/blob/master/src/C...

Hi Rubén,

Another proposition on IRIS 2021.1+ can be this one with the use of the new window (OVER) function :

ClassMethod getPersonsPagWindow(iAge As %Integer, sortField As %String = 1, sortOrder As %String = 2, pageSize As %String = 20, pageIndex As %String = 1) As %DynamicObject
{
    set out = []
    set vFrom = ((pageIndex -1 ) * pageSize)+1
    set vTo = vFrom + (pageSize-1)

    set sql = "SELECT * "_
                "FROM ( SELECT persons.* "_
                "        , ROW_NUMBER() OVER (ORDER By "_sortField_" "_ $CASE(sortOrder,1:"ASC",2:"DESC",:"ASC")_
                "    ) rn "_
                "        FROM Sample.Person persons where Age > ? "_
                "    ) tmp "_
                "WHERE rn between "_vFrom_" and "_vTo_" "_
                "ORDER By "_sortField_" "_ $CASE(sortOrder,1:"ASC",2:"DESC",:"ASC")

    Set rs=##class(%ResultSet).%New("%DynamicQuery:SQL")
    set sc = rs.Prepare(sql)
    set sc = rs.Execute(iAge) If $$$ISERR(sc) Do DisplayError^%apiOBJ(sc) Quit

    while rs.%Next() {
        Do out.%Push({
                "pid": (rs.%Get("ID")),
                "ssn" : (rs.%Get("SSN")),
                "lastname" : (rs.%Get("LastName")) ,
                "givenname":    (rs.%Get("GivenName")),
                "secondaryname":       (rs.%Get("SecondaryName")) ,
                "gender": (rs.%Get("Gender")),
                "age": (rs.%Get("Age") )
                })
    }

    set outJson = []
    Do outJson.%Push({
                "pageSize":(pageSize),
                "pageIndex":(pageIndex),
                "fromIndex":(vFrom),
                "toIndex":(vTo),
                "resultSet":(out)
                })
    return outJson
}

I bench the two solutions on a dataset of 100 000 rows without index with a result of 20 elements on page 1 and here are the results :

"getPersonsPag timed : 1,647 secondes"
"getPersonsPagWindow timed : 0,247 secondes"

I guess that the window function is faster because you don't have to fetch all the data in a global before paging.

Hi Gregor,

First off all, try to connect to MySql directly by a shell command :

echo "select 1" | isql -v my-connector

Expected response :

+---------------------------------------+
| Connected!                            |
|                                       |
| sql-statement                         |
| help [tablename]                      |
| quit                                  |
|                                       |
+---------------------------------------+
SQL> select 1
+---------------------+
| 1                   |
+---------------------+
| 1                   |
+---------------------+
SQLRowCount returns 1
1 rows fetched

Where /etc/odbc.ini :

[my-connector]
Description           = MySQL connection to  database
Driver                = MySQL
Database              = example
Server                = localhost
User                  = example
Password              = example
Port                  = 3306
Socket                = /var/run/mysqld/mysqld.sock

and /etc/odbcinst.ini

[MySQL]
Description = ODBC for MySQL
Driver = /usr/local/lib/libmyodbc8a.so
Setup = /usr/local/lib/libmyodbc8w.so
FileUsage = 1

If you successfully connected to your mysql database, then you can use it in IRIS/Caché/Ensemble :

Here is an example with %SQLGatewayConnection

set gc=##class(%SQLGatewayConnection).%New()
set pDSN="my-connector"
set sc=gc.Connect(pDSN,"example","example")
set sc=gc.AllocateStatement(.hstmt) 
set pQuery= "select 1"
set sc=gc.Prepare(hstmt,pQuery)
set sc=gc.Execute(hstmt)
set sc=gc.Fetch(hstmt)
set sc=gc.GetData(hstmt, 1, 1, .val)
zw val
set sc=gc.CloseCursor(hstmt)
set sc=gc.Disconnect()

To go further check those links :
- https://docs.intersystems.com/irisforhealthlatest/csp/docbook/DocBook.UI...
- https://docs.intersystems.com/irisforhealthlatest/csp/docbook/DocBook.UI...

Or even better check this training, it's with an JDBC connector but most part is applicable has DSN will fit your odbc config.
- https://github.com/grongierisc/formation-template

Hi Eric,

First you are using &sql who is for internal SQL use : doc

If you want to do an external query to a remote database you can do it with Ensemble :

Include EnsSQLTypes
 
Class Batch.Example.SqlInsertOperation Extends Ens.BusinessOperation
{
 
Parameter ADAPTER = "EnsLib.SQL.OutboundAdapter";
 
Property Adapter As EnsLib.SQL.OutboundAdapter;
 
Parameter INVOCATION = "Queue";
 
Method SetResultSetView(pRequest As Ens.StringRequest, Output pResponse As Ens.StringResponse) As %Status
{
    set tStatus = $$$OK
    
    try{
                    
        set pResponse = ##class(Ens.StringResponse).%New()
    
        set SqlInsertView = "INSERT into ODS_Products (ID,ProductName,Date_Alimentation) values (?,?,TO_DATE(?,'yyyy-mm-dd hh24:mi:ss'))"
 
        set param(1) = 1
        set param(1,"SqlType")=$$$SqlInteger
 
        set param(2) = ##class(%PopulateUtils).Name()
        set param(2,"SqlType")=$$$SqlVarchar
            
        set param(3) = $ZDATETIME($NOW(),3)
        set param(3,"SqlType")=$$$SqlVarchar
 
        set param = 3
            
        $$$ThrowOnError(..Adapter.ExecuteUpdateBatchParamArray(.nrows,SqlInsertView,.param))
                                
    }
    catch exp
    {
        Set tStatus = exp.AsStatus()
    }
 
    Quit tStatus
}
 
XData MessageMap
{
<MapItems>
    <MapItem MessageType="Ens.StringRequest">
        <Method>SetResultSetView</Method>
    </MapItem>
</MapItems>
}
 
}

Or with the %SQLGatewayConnection :

    //Create new Gateway connection object
   set gc=##class(%SQLGatewayConnection).%New()
   If gc=$$$NULLOREF quit $$$ERROR($$$GeneralError,"Cannot create %SQLGatewayConnection.")
       
   //Make connection to target DSN
   s pDSN="Samples"
   s usr="_system"
   s pwd="SYS"
   set sc=gc.Connect(pDSN,usr,pwd,0)
   If $$$ISERR(sc) quit sc
   if gc.ConnectionHandle="" quit $$$ERROR($$$GeneralError,"Connection failed")
       
   set sc=gc.AllocateStatement(.hstmt)
   if $$$ISERR(sc) quit sc
       
   //Prepare statement for execution
   set pQuery= "select * from Sample.Person"
   set sc=gc.Prepare(hstmt,pQuery)
   if $$$ISERR(sc) quit sc
     //Execute statement
   set sc=gc.Execute(hstmt)
   if $$$ISERR(sc) quit sc

It depends on what you are looking for.

To create applications to expose data on IRIS with custom APIs and make custom applications, you can use IRIS Studio and/or VsCode with the IRIS extension.

For database administration, you can use the management portal.

For 100% SQL administration, I recommend DbBeaver.

You are right Benjamin, the R gateway go through the Java gateway with two helper classes :

  • com.intersystems.rgateway.Helper
  • org.rosuda.REngine.Rserve

An example can be found here :

If I may, I prefer the approach of Eduard for the R gateway : https://github.com/intersystems-community/RGateway who by pass the java gateway and directly use socket connection to the R Server.

@Eduard Lebedyuk : you are right no documentation a this time for the R Gateway.

You can't work directly in HSLIB, you have to create your own namespace to work with.

In a terminal :

zn "HSLIB"
// Install a Foundation namespace and change to it
Do ##class(HS.HC.Util.Installer).InstallFoundation("NEW_NAMESPACE")
zn "NEW_NAMESPACE"

If you are not familiar with Namespace, HSLIB, Ensemble first check for training here :

https://learning.intersystems.com/course/view.php?id=243

Hi Xiong,

To convert FHIR R4 to SDA you have to use this helper class :

HS.FHIR.DTL.Util.HC.FHIR.SDA3.Process

This class take in input a message of this kind : HS.FHIRServer.Interop.Request or HS.Message.FHIR.Request.

So if you want to convert an json FHIR file to SDA, you have to read the file from a business service cast you file to one of this message and send it to the helper class.

Here is and Business service example of reading fhir json files :

Class BS.FHIRFileService Extends Ens.BusinessService
{

Parameter ADAPTER = "EnsLib.File.InboundAdapter";

Property TargetConfigNames As %String(MAXLEN = 1000) [ InitialExpression = "BusinessProcess" ];

Parameter SETTINGS = "TargetConfigNames:Basic:selector?multiSelect=1&context={Ens.ContextSearch/ProductionItems?targets=1&productionName=@productionId}";

Method OnProcessInput(pDocIn As %RegisteredObject, Output pDocOut As %RegisteredObject) As %Status
{
    set status = $$$OK

    try {

        set pInput = ##class(HS.FHIRServer.Interop.Request).%New()
        set tQuickStream = ##class(HS.SDA3.QuickStream).%New()
        do tQuickStream.CopyFrom(pDocIn)
        set pInput.QuickStreamId= tQuickStream.Id


        for iTarget=1:1:$L(..TargetConfigNames, ",") {
            set tOneTarget=$ZStrip($P(..TargetConfigNames,",",iTarget),"<>W")  Continue:""=tOneTarget
            $$$ThrowOnError(..SendRequestSync(tOneTarget,pInput,.pDocOut))
        }
    } catch ex {
        set status = ex.AsStatus()
    }

    Quit status
}

}

You can find more information from this documentation :

https://docs.intersystems.com/irisforhealthlatest/csp/docbook/Doc.View.c...