New post
Question
· Jan 30, 2017

WebMethod - Issue with encoding (special character)

#HealthShare #SOAP #ObjectScript #Frontend

Hello guys,

I've got this piece of code which runs the method "WebMethod", that belongs to %SOAP.WebBase.cls.

It grabs the outcome from an internal webservice we have and after that, it writes into a file.

The thing is, when I browse the file contents, I realize that in the place where a special character should be, I see a question mark.

By querying the same webservice from a special soap tool called "SoapSonar" (I've been using this for years), the outcome shows up this special character (shows it properly).

So I wonder... is there kind of encoding parameter I could use within the "WebMethod", so it'd return the correct output?

---

Method RunTree(Tree As %String, Inputs As %String, Debug As %Integer) As RMHCommon.Output [ Final, ProcedureBlock = 1, SoapBindingStyle = document, SoapBodyUse = literal, WebMethod ]
{
   Set p=##class(RMHCommon.Output).%New()
   Set p=..WebMethod("RunTree").Invoke(##this,"http://soap.uk/Interface.Soap.RunTree",.Tree,.Inputs,.Debug)

   set fs=##class(%Stream.FileBinary).%New()
   set fs.Filename="c:\TEMP\SoapTree.xml"
   set tSC=fs.CopyFrom(p.Value)
   set tSC=fs.%Save()
...
}

Many thanks!

Discussion (16)1
Log in or sign up to continue
John Murray · Jan 30, 2017

Please clarify what you mean by "browse the file contents".

Are you opening c:\TEMP\SoapTree.xml in a text editor?

Maybe that editor is assuming that the file is UTF8-encoded.

Can you view it in a tool that shows you the byte values it contains?

Did you have a particular reason for choosing to write the file using an instance of %Stream.FileBinary instead of %Stream.FileCharacter?

Also, be aware that the WebMethod classmethod of %SOAP.WebBase is tagged as "Internal" and commented thus:

/// This method is used internally by Caché. You should not make direct
/// use of it within your applications. There is no guarantee made about either
/// the behavior or future operation of this property.
 

1 0
Murillo Braga · Jan 30, 2017

Hello mate,

Thanks for you quick response. Actually there was no specific reason for using the FileBinary, so I managed to change it to FileCharacter and used also the property "TranslateTable = "UTF8", which hence allowed the file to be written with the special character I wanted.

By discovering that, I came back to the 1st place I originally thought the issue would be and now I guess that it's the right place to look at:

Method OnProcessInput(pInput As %Stream, Output pOutput As %RegisteredObject, ByRef pHint As %String) As %Status
{
    // Create an instance of %XML.Reader
    Set xread = ##class(%XML.Reader).%New()
    Set xread.IgnoreNull=1
        
    // Begin processing of the file
    d pInput.Rewind()
    
    s sc= xread.OpenStream(pInput)
   
     if $$$ISERR(sc)
{
set fs=##class(%Stream.FileCharacter).%New()
set fs.Filename="c:\TEMP\OnProcessInput00.xml"
set fs.TranslateTable = "UTF8"
do fs.Write($$$StatusDisplayString(sc))
set tSC=fs.%Save()
}

---

Here I had the file created and the error message written in it (ERROR #6301: SAX XML Parser Error: invalid character 0x18 while processing Anonymous Stream at line 1 offset 4627).

So I presume that
s sc= xread.OpenStream(pInput)

Is erroring, because the stream contains a special character.

What do you think?

Thanks

0 0
John Murray · Jan 31, 2017

What does the start of the pInput stream contain? One quick but ugly way of checking this would be to write it to a scratch global before the call to OpenStream which errors, e.g.

Set ^tMurillo=pInput.Read(255) Do pInput.Rewind()

Then afterwards run the following in Terminal:

w ^tMurillo

w $a(^tMurillo,1),!,$a(^tMurillo,2),!,$a(^tMurillo,3)

This should show us whether the creator of the stream started it with a BOM character or sequence, and also whether there is an XML header specifying an encoding.

That information is apparently important to the %XML.SAX.StreamAdapter used by %XML.SAX.Parser, which in turn is what the OpenStream method of %XML.Reader uses.

0 0
John Murray · Jan 31, 2017

There's no BOM, and the XML header claims that the content is UTF-8 encoded. But on your other post you reported that your content contains a left-single-quote character and that the SAX parser choked on a character, which I suspect was this character in Unicode form rather than UTF-8 encoded.

So, what originally wrote the pInput stream's content? Did it actually UTF-8 encode the data it wrote to the stream?

0 0
John Murray · Jan 31, 2017

Based on our comment thread I think the most likely cause is that the pInput %Library.GlobalCharacterStream (originating from the Value property of the object that was returned when you Invoke your webmethod) starts with an XML header claiming that its encoding="UTF-8" but I suspect that the characters within that global stream are actually Unicode rather than UTF8-encoded.

To test this theory I suggest you create a new %Library.FileCharacterStream, set its TranslateTable property to "UTF8", then use its CopyFromAndSave method to fill it with the contents of pInput. Now pass your FileCharacterStream to your %XML.Reader's OpenStream method and see if you still get the SAX error.

0 0
Murillo Braga · Feb 1, 2017

John,

Just to let you know, I am still a beginner with ObjectScript/Cache/Healthsare, so please excuseany primary mistakes you may encounter :)

Can you kindly help me out to write this piece of code you advised? (P.S: msg.Value is the stream)

set fs=##class(%Library.GlobalCharacterStream).%New()
set fs.TranslateTable = "UTF8"
set tSC=fs.CopyFromAndSave(msg.Value)

  s sc= xread.OpenStream(???)

if $$$ISERR(sc)
{
...
}

---

I came up with that a few moments ago: (at this point I have the stream available (msg.Value) and I will handle it in 2 different ways: by writing is a file and trying to open it (OpenFile) and by reading it (OpenStream))

  //
  // 1st approach - it succeeds, so the file errorOpenFile.xml is NOT generated
  //    
  d msg.Value.Rewind()
  set fs=##class(%Stream.FileCharacter).%New()
  set fs.Filename="D:\DATABASES\OVPATH\temp\test.xml"
  set fs.TranslateTable = "UTF8"
  set tSC=fs.CopyFrom(msg.Value)
  set tSC=fs.%Save()
    
  s reader=##class(%XML.Reader).%New()
  Set sc = reader.OpenFile("temp/test.xml")
    
  if $$$ISERR(sc)
  {
    set fs=##class(%Stream.FileCharacter).%New()
    set fs.Filename="D:\DATABASES\OVPATH\temp\errorOpenFile.xml"
    do fs.Write($$$StatusDisplayString(as))
    set tSC=fs.%Save()
  }    

  //
  // 2nd approach - it fails, so I have the file errorOpenStream.xml generated
  //    
  Set xread = ##class(%XML.Reader).%New()
  Set xread.IgnoreNull=1                  
  d msg.Value.Rewind()    
  s sc= xread.OpenStream(msg.Value)
    
  if $$$ISERR(sc)
  {
    set fs=##class(%Stream.FileCharacter).%New()
    set fs.Filename="D:\DATABASES\OVPATH\temp\errorOpenStream.xml"
    do fs.Write($$$StatusDisplayString(sc))
    set tSC=fs.%Save()
  }

0 0
John Murray · Feb 1, 2017

I think you could simplify your first approach a little by reverting to calling OpenStream on your reader object rather than using OpenFile:

  // 1st approach - it succeeds, so the file errorOpenFile.xml is NOT generated
  //    
  d msg.Value.Rewind()
  set fs=##class(%Stream.FileCharacter).%New()
  set fs.Filename="D:\DATABASES\OVPATH\temp\test.xml"
  set fs.TranslateTable = "UTF8"
  set tSC=fs.CopyFrom(msg.Value)
  set tSC=fs.%Save()
    
  s reader=##class(%XML.Reader).%New()

  d fs.Rewind() // might not be necessary, but won't hurt
  Set sc = reader.OpenStream(fs)

 

Anyhow, the fact that you don't get an error confirms my hypothesis that the original stream (msg.Value) contains Unicode data but the reader treats it as though it is UTF8-encoded.

In your code above I think you can also omit the line where you set fs.Filename and instead allow the stream to generate its own temporary file. Explicitly naming the file may be handy when debugging, but it will cause problems if more than one process runs this code concurrently.
 

0 0
Murillo Braga · Feb 2, 2017

Hello mate,

It generates the same error: ERROR #6301: SAX XML Parser Error: invalid character 0x18 while processing Anonymous Stream at line 1 offset 4183

Here is the code:

d msg.Value.Rewind()
set fs=##class(%Stream.FileCharacter).%New()
set fs.Filename="D:\DATABASES\OVPATH\temp\test0.xml"
set fs.TranslateTable = "UTF8"
set tSC=fs.CopyFrom(msg.Value)
set tSC=fs.%Save()

s reader=##class(%XML.Reader).%New()
d fs.Rewind() // might not be necessary, but won't hurt
Set sc = reader.OpenStream(fs)

if $$$ISERR(sc)
{
set fs=##class(%Stream.FileCharacter).%New()
set fs.Filename="D:\DATABASES\OVPATH\temp\errorOpen0.xml"
do fs.Write($$$StatusDisplayString(sc))
set tSC=fs.%Save()
}

0 0