· Mar 13, 2017

Fastest way to strip invalid characters from stream

A method to convert certain non-readable ASCII characters in a  %Stream.FileCharacter object first copies it into a %Stream.FileBinary object and then loops through each character one at a time to find and convert these offensive characters to our interpretation of their readable ASCII equivalents  . The loop is sequential (while 'bStream.AtEnd) and is taking too long for large files. Any suggestions for an existing Cache method to do this or is there another way, besides a sequential loop, to examine every character of  a binary Stream?

Discussion (2)1
Log in or sign up to continue

Can you provide the code you are using currently so we have something definitive to base comments off, but have you tried using $translate and reading the data in big chunks e.g. 16k at a time?

While 'binarystream.AtEnd {

  Set sc=outputstream.Write($translate(binarystream.Read(16000),badchars,goodchars)


Where binarystream is your binary input stream, outputstream is your output stream with the converted characters and badchars is a list of the bad characters you need to convert and goodchars is the list of the values you want the badchars converted into.