What is the best approach to insert data to an external SQL database?
Hey All!
I wanted to reach out and get some advice on the best way to interact with a external SQL database, as I'm finding the documentation quite confusing to follow. I'm new to using SQL within InterSystem, and I want to make sure I'm following the best practices.
In essence, what I am trying to achieve, is extracting certain values from a HL7 message and then insert those message field values into an external database. The ODBC, DSN and Credentials details are all setup and working correctly.
The way that I've currently achieved this is by doing the following:
- I create a new class that contains the properties that I want to save and reference when calling the stored procedure
Class JWDEV.SQLTesting.PatientDetails Extends Ens.Request
{
Property name As %String;
Property mrn As %String;
Storage Default
{
<Data name="InsertDataDefaultData">
<Subscript>"InsertData"</Subscript>
<Value name="1">
<Value>name</Value>
</Value>
<Value name="2">
<Value>mrn</Value>
</Value>
</Data>
<DefaultData>InsertDataDefaultData</DefaultData>
<Type>%Storage.Persistent</Type>
}
} -
I create a new custom business process extending the "Ens.BusinessProcess" class.
- This class takes the HL7 message, and stores relevant message fields to the PatientDetails request object.
- Once the mrn and name values are saved to the req properties, it then passes over the req object to the "InsertPatientData" business operation
Class JWDEV.SQLTesting.SQLBusinessProcess Extends Ens.BusinessProcess { Method OnRequest(request As EnsLib.HL7.Message, response As JWDEV.SQLTesting.PatientDetails) As %Status { /*Create an new instance of the request class (this is the model of the data)*/ Set req = ##Class(JWDEV.SQLTesting.PatientDetails).%New() /*Set the properties within the model to values located in request HL7 argument*/ Set req.mrn = request.GetValueAt("MSH:9.1") Set req.name = request.GetValueAt("MSH:9.2") /*Send the request to the EnsLib.SQL.Operation.ProcOperation business operation*/ Set resultStatus = ..SendRequestSync("InsertPatientData", req, 30) return resultStatus } Method OnResponse(pInput As C21.CUH.IEP.IEPRequest) As %Status { Try { return $$$OK } Catch ex { Set tSC=ex.AsStatus() } } } - The business operation is configured with a request class of the "JWDEV.SQLTesting.PatientDetails", which allows me to directly access the "mrn" and "name" properties, which are passed as arguments to the stored procedure
This approach is functional and does insert the relevant fields into the database:
In other integration engines I've used, inserting into a external database wouldn't require the use of custom classes, I'd be able to directly access the HL7 message fields when needed or save global variables/properties that can be used as the stored procedure arguments, but with InterSystems I've struggled to find a similar approach. It is likely my inexperience using SQL within InterSystem, but the way I've accomplished this feels overengineered.
Is there a more appropriate approach to inserting data into an external table than the one I've shown above? If yes, please could I get some pointers on what that is, so I can improve on what I've learnt so far.
Also, I appreciate that the actual classes shown are quite barebones/messy, I just wanted to get basic functionality working to demonstrate a POC.
Thanks,
Josh
Comments
A simpler and appropriate pattern is to use a dedicated SQL Business Operation with EnsLib.SQL.OutboundAdapter, and send it a request object that contains only the values the SQL call needs. That is exactly the pattern shown in the external SQL production example: define a request class for the input values, define a response class if needed, then create a Business Operation that executes the SQL or stored procedure through the SQL adapter. [1]
In that example, the Business Operation extends Ens.BusinessOperation, sets Parameter ADAPTER = "EnsLib.SQL.OutboundAdapter", and executes the SQL against the external database via the adapter. The request object carries the needed fields, and the operation uses those values to build and run the SQL. [1]
Your current approach of:
- extracting HL7 fields from
EnsLib.HL7.Message - placing them into a custom request class
- passing that request to a SQL Business Operation
is therefore a valid InterSystems pattern rather than inherently overengineered. [1]
If you want to keep the logic lightweight, the documented interoperability patterns also support pulling values directly from the HL7 message in a process or BPL using request.GetValueAt(...), then sending only the values needed onward. The example BPL pattern shows extracting a field from an HL7 message and sending a simple request to an operation, which aligns with what you are doing conceptually. [2]
For stored procedures specifically, there is also an example of calling them from a SQL adapter operation using ..Adapter.ExecuteProcedureParmArray(...) with parameter arrays. That is useful when you want the operation itself to invoke the procedure directly and explicitly define input/output parameters. [3]
There is also an approach described for generating message classes from a stored procedure definition so the request class matches the procedure parameters automatically. In that pattern, you still use a message/request class and a Business Operation for the stored procedure call. [4]
So the main pointers are:
- Keep the external DB interaction inside a dedicated SQL Business Operation using
EnsLib.SQL.OutboundAdapter. [1] - Extract HL7 fields before the call and pass only the required values in a request object. [1][2]
- If you are calling a stored procedure, consider using
ExecuteProcedureParmArray(...)in the operation. [3] - If you have many procedures or many parameters, a generated request-class approach may reduce manual maintenance. [4]
One practical detail from the example: if you configure credentials on the SQL operation, remember they are namespace-specific. [1]
Sources:
I have many of examples of handling this using JDBC connection to Microsoft SQL Server. Either you can build an all-encompassing Custom Business Operation or use the EnsLib.SQL.Operation.ProcOperation. When inserting data into a table depending on how busy your Process is, I would suggest using Stored Procedures to insert the data.
Using a Dynamic SQL statement can cause some overhead because the Query is not Cached, so you are building it on your machine, then having to build it again when it is sent to the external connection. With a Patient Demographic (ADT) HL7 interface that extra overhead could cause everything to slow down.
The Begining of my Custom Business Operation looks like this...
Include (EnsSQLTypes, %occODBC)
Class osuwmc.Visit.VisitDBWriteOperation Extends Ens.BusinessOperation [ ClassType = "", ProcedureBlock ]
{
Parameter ADAPTER = "EnsLib.SQL.OutboundAdapter";
Parameter INVOCATION = "Queue";
Property InitDSN As %String;
Method OnInit() As %Status
{
Set ..InitDSN = ..Adapter.DSN
//Set ..Adapter.ConnectAttrs = "QueryTimeout:45" ; try this too just in case...
Kill $$$EnsRuntimeAppData(..%ConfigName)
Quit $$$OK
}
Then for each stored procedure I call I have a different Method that uses its own Data Message Class Structure, the Message Map takes the incoming Data Message Class, and it knows which Method to call.
Method InsertCaseScheduleSp(pRequest As osuwmc.Visit.DataStructures.InsertCaseSchedule, Output pResponse As Ens.Response) As %Status
{
set SPQuery = "{ ?= call dbo.usp_Interfaces_Insert_CaseSchedule_Ens(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}"
set par = 15
set par(1) = pRequest.CSN
set par(2) = pRequest.MRN
set par(3) = pRequest.CaseNum
set par(4) = pRequest.SchedDate
set par(5) = pRequest.OrderNum
set par(6) = pRequest.LocationDescription
set par(7) = pRequest.Duration
set par(8) = pRequest.DurationUnits
set par(9) = pRequest.CreationDate
set par(10) = pRequest.CreationUser
set par(11) = pRequest.ArrivalUser
set par(12) = pRequest.EditingUser
set par(13) = pRequest.NoShowUser
set par(14) = pRequest.CancelUser
set par(15) = pRequest.Status
set tSC = ..Adapter.ExecuteProcedureParmArray(.caseschedinsert,.outputs,SPQuery,"oiiiiiiiiiiiiiii",.par)
kill caseschedinsert
kill SPQuery
kill par
Quit tSC
}
Data MessageMap
{
<MapItems>
<MapItem MessageType="osuwmc.Visit.DataStructures.InsertCaseSchedule">
<Method>InsertCaseScheduleSp</Method>
</MapItem>
</MapItems>
}
Thanks Scott, this is really helpful, and much appreciated for taking the time to respond.
I've now got my classes more organised, and everything from a syntax perspective I believe is correct, however, I am still running into an issue when the BusinessOperation tries to bind the parameters, and I believe it's something to do with me using varchar (max) in my SQL table.
There actual error text is this:
ERROR #6022: Gateway failed: BindParameters.
ERROR <Ens>ErrGeneral: SQLState: (HY104) NativeError: [0] Message: [Microsoft][ODBC Driver 18 for SQL Server]Invalid precision valueThe same error occurs regardless of SQL ODBC driver I use, so I get the impression the issue is actually with the BusinessOperation, rather than with the ODBC or database.
When I run the stored procedure directly in SSMS with the exact same data contained within the "PatientRequest" (listed below) object it runs without any issues, so that rules out a stored procedure issue in my eyes.
Here is my basic BusinessOperation:
Class JWDEV.SQLTesting.JWDEVBusinessOperation Extends Ens.BusinessOperation
{
Parameter ADAPTER = "EnsLib.SQL.OutboundAdapter";
Property Adapter As EnsLib.SQL.OutboundAdapter;
Parameter INVOCATION = "Queue";
Method OnMessage(pRequest As JWDEV.SQLTesting.PatientRequest, Output pResponse As JWDEV.SQLTesting.PatientResponse) As %Status
{
Try{
Set query = "{call dbo.sp_insert_or_update_radiologyreports(?,?,?,?,?,?,?,?,?)}"
Set params = 9
Set params(1) = pRequest.patientid
Set params(2) = pRequest.accession
Set params(3) = pRequest.patientname
Set params(4) = pRequest.dob
Set params(5) = pRequest.sex
Set params(6) = pRequest.placerordernumber
Set params(7) = pRequest.universalserviceid
Set params(8) = pRequest.observationtext
Set params(9) = pRequest.observationdatetime
Set status = ..Adapter.ExecuteProcedureParmArray(,,query,"iiiiiiiii",.params)
If $$$ISERR(status){
$$$LOGERROR($SYSTEM.Status.GetErrorText(status))
Return status
}
Return status
} Catch(ex) {
Set status = ex.AsStatus()
$$$LOGERROR($SYSTEM.Status.GetErrorText(status))
return status
}
}
} Here is the Request Class:
Class JWDEV.SQLTesting.PatientRequest Extends Ens.Request
{
Property patientid As %String(MAXLEN = 50);
Property accession As %String(MAXLEN = 50);
Property patientname As %String(MAXLEN = 200);
Property dob As %String(MAXLEN = 25);
Property sex As %String(MAXLEN = 10);
Property placerordernumber As %String(MAXLEN = 25);
Property universalserviceid As %String(MAXLEN = 150);
Property observationtext As %String(MAXLEN = "");
Property observationdatetime As %String(MAXLEN = 25);
Storage Default
{
<Data name="InsertDataDefaultData">
<Subscript>"InsertData"</Subscript>
<Value name="1">
<Value>name</Value>
</Value>
<Value name="2">
<Value>mrn</Value>
</Value>
<Value name="3">
<Value>patientid</Value>
</Value>
<Value name="4">
<Value>accession</Value>
</Value>
<Value name="5">
<Value>patientname</Value>
</Value>
<Value name="6">
<Value>dob</Value>
</Value>
<Value name="7">
<Value>sex</Value>
</Value>
<Value name="8">
<Value>placerordernumber</Value>
</Value>
<Value name="9">
<Value>universalserviceid</Value>
</Value>
<Value name="10">
<Value>observationtext</Value>
</Value>
<Value name="11">
<Value>observationdatetime</Value>
</Value>
</Data>
<DefaultData>InsertDataDefaultData</DefaultData>
<Type>%Storage.Persistent</Type>
}
}
Here is my SQL table schema:
[PatientID] [nvarchar](50) NOT NULL,
[AccessionNumber] [nvarchar](50) NOT NULL,
[PatientName] [nvarchar](200) NOT NULL,
[DOB] [nvarchar](25) NOT NULL,
[Sex] [nvarchar](10) NOT NULL,
[PlacerOrderNumber] [nvarchar](25) NOT NULL,
[UniversalServiceID] [nvarchar](150) NOT NULL,
[ObservationText] [nvarchar](max) NOT NULL,
[ObservationDateTime] [nvarchar](25) NOT NULL,
[DateTimeInserted] [datetime] NOT NULL,
CONSTRAINT [PK_Radiology_Reports] PRIMARY KEY CLUSTERED
(
[PatientID] ASC,
[AccessionNumber] ASC
)I'm not sure whether varchar (max) needs to be handled different compared to other varchar types in ObjectScript. I did see it was possible to explicitly define parameter types within "Specifying Parameters in an InterSystems IRIS Multidimensional Array", and I had attempted this, but I've not had any success as of yet.
Have you had any experience with this error or how to correctly handle varchar (max) datatypes in ObjectScript?
In my experience, I have never used nvarchar within SQL, I have always just used varchar.
for your data class...
Property observationtext As %String(MAXLEN = "");
MAXLEN cannot be set to "", please see.. https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls?KEY=RCOS_appx_limits#RCOS_appx_limits_string_length
Thanks Scott, it was actually the DC AI and a few of the forums that appear to indicate that you could set "%String(MAXLEN = "")", but I have since updated the datatype to "%VarString" instead which I believe is the better choice.
I have now fixed my "Invalid precision value" error. Adding context below for anyone who runs into the same issue I did:
I'd started the debugging session by starting an ODBC trace, and then restarting the Business Operation. From the information I had seen online, the operation you need tracing must be restarted after the ODBC trace has been started, otherwise the trace won't be written to the ODBC .LOG file.
Once the trace had started and Business Operation restarted, I'd sent in a ORU^R01 result, processed it within my Business Operation and the same error "Invalid Precision Value" occurred.
The ODBC log file, contained the following output:
mgr" 610c-ed8 ENTER SQLBindParameter
HSTMT 0x0000020D973CB220
UWORD 8
SWORD 1 <SQL_PARAM_INPUT>
SWORD -8 <SQL_C_WCHAR>
SWORD 12 <SQL_VARCHAR>
SQLULEN 2147483647
SWORD 0
PTR 0x0000020D9AEFF5E0
SQLLEN 257
SQLLEN * 0x0000020D97055BD0
mgr" 610c-ed8 EXIT SQLBindParameter with return code -1 (SQL_ERROR)
HSTMT 0x0000020D973CB220
UWORD 8
SWORD 1 <SQL_PARAM_INPUT>
SWORD -8 <SQL_C_WCHAR>
SWORD 12 <SQL_VARCHAR>
SQLULEN 2147483647
SWORD 0
PTR 0x0000020D9AEFF5E0
SQLLEN 257
SQLLEN * 0x0000020D97055BD0With some assistance from ClaudeAI, I was able to interpret the "problematic" value was with the "SQLULEN" parameter and the ODBC driver not accepting an infinite fixed precision value for normal binding.
From reading similar posts on the forum, one suggestion was to treat the string as a LOB. To do that I'd updated my Business Operation logic so the observationtext was being written to a stream.
Set observationtextstream = ##class(%Stream.TmpCharacter).%New()
Do observationtextstream.Write(pRequest.observationtext)
Set params(8) = observationtextstream
....
Set status = ..Adapter.ExecuteProcedureParmArray(,,query,"iiiiiiiii",.params)
....
kill observationtextstream
Return statusBecause I have no need to interact with this property anywhere else, I've used the TmpCharacter stream class.
Once the logic was in place, I then restarted my BusinessOperation, resubmitted the ORU^R01 message, and the binding worked without any issue and successfully inserted the message into the table. I'm sure this approach is probably overkill, but it's the only way I've managed to get the ODBC/ObjectScript to play nicely together!

