Conversion from %Stream.GlobalBinary to %CharacterStream

Question

Question

Frederic Donnet... · Nov 23, 2023

Hi,

I'm receiving in one of our interface a XML file as a %Stream.GlobalBinary, and in the source Application that produces this XML file, the user sometimes add a special character that force the XML file to have the header encoding to UTF-8, instead of ISO-8859-1 :

<?xml version="1.0" encoding="UTF8"?>

When my current interface tries to convert the %Stream.GlobalBinary into %XML.XPATH.Document to this exceptional UTF8 header encoding of the XML file :

Set tSC=##class(%XML.XPATH.Document).CreateFromStream(stream,.tDoc) --> contains exceptionally UTF8 encoding

I have the following bug that blocks the whole interface :

invalid byte 'd' at position 2 of a 3-bytes sequence in at line 1 offset 39 (method zCreateFromStream)

So the idea I got is to create the following method I'm developping that forces to change the encoding to ISO-8859-1 of the XML file through the method ChangeXMLStreamEncoding :

Method CheckANSIXMLHeader(pStream As %Stream.GlobalBinary) As %Stream.GlobalBinary
{

#dim inputCharStream As %CharacterStream
#dim outputCharStream As %CharacterStream
#dim outputGlobalBinStream As %Stream.GlobalBinary Set tSC = $$$OK

//first convert %Stream.GlobalBinary to %CharacterStream back
    Set outputCharStream = ##class(Ens.Util.XML.Reader).ChangeXMLStreamEncoding(inputCharStream, "ISO-8859-1", , tSC)
//finally convert %CharacterStream back to %Stream.GlobalBinary

    Quit outputGlobalBinStream
}

This method ChangeXMLStreamEncoding uses %CharacterStream only.

So my problem is that I don't know how to convert %Stream.GlobalBinary to %CharacterStream, and vice-versa

Could you tell me how to perform such conversion ?

Thanks a lot

Frédéric Donnet-Monay, integrator, Centre Hospitalier Universitaire Vaud, Lausanne, Switzerland

Product version: IRIS 2022.1

$ZV: IRIS for UNIX (Red Hat Enterprise Linux 9 for x86-64) 2022.1.3 (Build 668U) Fri Apr 7 2023 12:41:06 EDT [HealthConnect:3.5.0-1.m1] [HealthConnect:3.5.0-1.m1]

Discussion (6)3

Log in or sign up to continue

score 0 · Answer 1 · 2023-11-23T05:14:06-05:00

You have to cast your %CharacterStream to a specific implementation, say, to %GlobalCharacterStream. The latter one implements %GlobalStreamAdaptor, which inherits from %AbstractStream and %Stream.Object. Finally, the %Stream.Object class exposes the CopyFrom method which allows you to copy data from your %Stream.GlobalBinary instance (and vice versa)

score 0 · Answer 2 · 2023-11-23T18:41:26-05:00

I'm not sure I understand your issue right, my understanding is that you receive an XML declared as UTF-8 that contains ISO-8859-1 characters.

If so, you can convert the encoding of the stream content using something like:

/// Convert the encoding of the content of a stream to UTF-8.
/// If OutStream is not passed (of any %Stream.* class), then a new %Stream.GlobalBinary is returned.
/// Note that OutStream IS NOT SAVED on exit
ClassMethod ConvertStreamToUTF8(InStream As %Stream.Object, ByRef OutStream As %Stream.Object = {##class(%Stream.GlobalBinary).%New()}) As %Status
{
	Set sc=$$$OK
	Set MaxRead=$$$MaxLocalLength\2 ; to be safe
	Try {
		While 'InStream.AtEnd {
			Set content=InStream.Read(MaxRead)
			Set sc=OutStream.Write($ZCONVERT(content,"O","UTF8"))
			If $$$ISERR(sc) Quit
		}
	} Catch CatchError {
		#dim CatchError as %Exception.SystemException
		Set sc=CatchError.AsStatus()
	}
	Quit sc
}

If the problem is different, please provide more info/details, maybe a sample XML (not necessarily the original XML).

Enrico

score 0 · Answer 3 · 2023-11-27T05:54:20-05:00

Thanks for yours answers, I tried your solutions, but it is hard and did not give good results.

To simplify my explanation : I'm reading an XML file from a file-share, and I want to change the XML header encoding from UTF8 to ISO-8859-1, if it happens sometimes.
So the result of my new method should transform the potential header
<?xml version="1.0" encoding="UTF8"?>

to a new header

<?xml version="1.0" encoding="ISO-8859-1"?>

score 0 · Answer 4 · 2023-11-27T07:31:12-05:00

Set inputBinaryStream=##class(%Stream.FileBinary).%New()
Set inputBinaryStream.Filename="\\server\your\share\file.xml"
Set outputBinaryStream=##class(%Stream.FileBinary).%New()
Set outputBinaryStream = ##class(Ens.Util.XML.Reader).ChangeXMLStreamEncoding(inputBinaryStream, "ISO-8859-1",outputBinaryStream, .tSC)

; Since the output stream is passed, you can just call (last line) with:
Do ##class(Ens.Util.XML.Reader).ChangeXMLStreamEncoding(inputBinaryStream, "ISO-8859-1",outputBinaryStream, .tSC)

Enrico

score 0 · Answer 5 · 2023-11-27T09:45:19-05:00

Thank you Enrico for your quick answer. I'm almost there.
To be more precise, we access the fileshare through a EnsLib.FTP.OutboundAdapter, through the method ..Adapter.GetStream(pRequest.StringValue, .tStream) .

pRequest.StringValue contains the file-share path of the file I want to return.

Here is the full method :

/// Return the CDA XML file as a %Stream.GlobalBinary
Method GetSoarianXMLCDA(pRequest As Ens.StringRequest, pResponse As %Stream.GlobalBinary) As %Status
{
// Create the variable tSC (temporary Status Code)
Set tSC = $$$OK

try{
//Create the stream
Set tStream=##class(%Stream.GlobalBinary).%New()

//set the files path to the FTP Adapter
Set tSC = ..Adapter.GetStream(pRequest.StringValue, .tStream)
If $$$ISERR(tSC){
$$$LOGERROR("Error while retrieving Stream from CDA file : tSC = " _ tSC)
}

$$$ASSERT($IsObject(tStream))

//call here Do ##class(Ens.Util.XML.Reader).ChangeXMLStreamEncoding...

If tStream.SizeGet() > 0{
Set pResponse = tStream
}
else{
  $$$LOGERROR("CDA file " _ pRequest.StringValue _" is empty or null")
}
}
catch ex {
$$$LOGERROR(##class(%SYSTEM.Status).GetErrorText(ex.AsStatus()))
Set tSC = ex.AsStatus()
}
    Quit tSC
}

The FTP adapter method ..Adapter.GetStream returns a %GlobalBinaryStream If I read the adapter's code correctly.

Could you please tell me how to use ChangeXMLStreamEncoding() in this methode above ?

Thanks again

Fred

score 0 · Answer 6 · 2023-11-27T09:55:11-05:00

Simply replace the line:
//call here Do ##class(Ens.Util.XML.Reader).ChangeXMLStreamEncoding...

With:
Set outputBinaryStream=##class(%Stream.FileBinary).%New()
Do ##class(Ens.Util.XML.Reader).ChangeXMLStreamEncoding(tStream, "ISO-8859-1",outputBinaryStream, .tSC)
//you may want to check tSC, just in case....
//now on use the header changed outputBinaryStream instead of tStream

Enrico