New post
Question
· Apr 16, 2018

Reading a file and translating the content from UTF8 to 8-bit

#Caché #ObjectScript

Hi,

I need to read a UTF8 encoded text file and translate the content to 8-bit.

Using %File class and $ZCVT(TXT,"I","UTF8") works , but I see that if the content is larger than max string  (32000) and we cut the content

To max string chunks, we can get a <translate> error if we cut it in the "wrong" point..

Is there a better way to do this task?

My code looks like this:

    S file=##class(%File).%New(..LocalFileName)
    D file.Open("R")
    While 'file.AtEnd {    
        S Line=$ZCVT(Line,"I","UTF8")
    }
    D file.Close()

and an example of such an error:

USER>s str=$C(215)
USER>w $ZCVT(str,"I","UTF8")
W $ZCVT(str,"I","UTF8")
^
<TRANSLATE>

Regards,

Nael

Discussion (10)1
Log in or sign up to continue
Dmitry Maslennikov · Apr 16, 2018

I would recommend to use more suitable class for it. %Stream.FileCharacter when you can set TranslateTable property

Set stream=##class(%Stream.FileCharacter).%New()
Set sc=stream.LinkToFile("c:\myfile.txt")
Set stream.TranslateTable = "UTF8"
While 'stream.AtEnd {
	Set line=stream.Read()
	; Process the chunk here
}

And you don't need any conversions after that

3 0
Alexander Koblov · Apr 16, 2018

Nael,

I think you need to use 4th argument of $zconvert:

Set file=##class(%File).%New(..LocalFileName)
Do file.Open("R")
Set handle=""
While 'file.AtEnd { 
    Set Line=$ZCVT(file.Read() , "I", "UTF8", handle)
   // do something with Line
}
Do file.Close()

Handle "contains the remaining portion of string that could not be converted at the end of $ZCONVERT, and supplies this remaining portion to the next invocation of $ZCONVERT."

Please see reference for $zconvert

2 0