· Apr 24

Line Terminators and File/Stream Classes - A Few Questions

Hello, I'm curious to see how other people deal with this: we have a text file that was created on someone's Windows machine and it was copied and pasted into a text file on someone's Mac machine.  After some examination we realized that the line feeds were originally CRLF (for Windows) and when copied and pasted they were changed to LF (Mac).  The diff program we used didn't pick up on this and the program we wrote to read the file was getting each line of the CRLF file and treating the whole file as one line for the LF file. 

I was able to use the text editor to change the LF file back to CRLF to fix the problem short term.  The long term solution was to simply set the line terminator to LF in the program itself.  My question is: is this reliable?  Why wouldn't the default line terminator just be LF in all cases?  That covers it for both Windows and Unix.  I'm trying to imagine the use case of why it's important to default to CRLF for Windows and LF for Mac.  Maybe someone can assist?

Maybe things are more complicated when you are dealing with things that aren't text files? I can understand the need to set the line terminator because it could be any character for any use case, but what I'm really curious about is the default for Windows and Unix when LF covers them both.  This could present problems if app users are trying to load files and through the magic of copying and pasting they change these control characters without even realizing.  

Speaking of line terminators, here's another quandary.  The file in question is loaded from a form submit from a CSP server page.  This means the file is presented to the server as a %CSP.BinaryStream.  Do folks agree that in general the methods to use %CSP.BinaryStream as similar but not totally consistent with %Stream.FileBinary and %Stream.FileCharacter?  Or are the similar enough that you shouldn't have a problem treating them as the same class really?

When it comes to line terminators however, %Stream.FileBinary does this:

property LineTerminator as %String (MAXLEN = 10) [ InitialExpression = $get(^%SYS("Stream","LineTerminator"),$select($$$isUNIX:$char(10),1:$char(13,10))) , Transient ];

and %CSP.BinaryStream does this:

property LineTerminator as %String (MAXLEN = 10) [ InitialExpression = $char(13,10) ];

Maybe I need a better primer on streams.  If you have a good resource, I'm happy to read!

Product version: IRIS 2023.1
Discussion (2)1
Log in or sign up to continue

Hi @Michael Davidovich !
you hit exactly what I addressed in in my recent article
  The bewitched line terminator

As long as you get (text-) files from your own OS the default is OK
BUT: if you work cross OS platforms - rather likely for CSP pages -
the approach of  
%CSP.BinaryStream is the best for import.

For output stream you have to decide based on the expected target OS.

My personal trigger to that subject was file transfer between
WIN as Docker host and a Container (Ubuntu) causing a lot of  confusion until detected.