ooRexx: Line Terminations in Various Implementations

I was trying to run a program that was originally written in Regina
Rexx, on the Windows platform. The input file is a spreadsheet saved
as tab separated values(TSV). When I use the linein bif, in oorexx,
to read the file, a single read, reads the entire file. I assumed
that the problem was that linein was expecting a different line
terminator than I had in the file. In checking the TSV input file, I
found that the line terminator appeared to be a single CR. This
input file was created on a Mac, and Mac OS uses CR for a line
terminator. My ooRexx program is also running on Mac OS. Since Mac
OS is UNIX, I assumed that it my be looking for a LF character as the
line terminator. To test this I changed my input file to have both
the CR and LF characters, i.e. DOS style line terminators ( a line
terminator for everyone ). Still when executed, the linein bif, read
the entire file in a single read.

My questions are:

1. What is the line terminator that ooRexx is expecting on Mac OS X?
2. Is there someway, other than recompiling the code to change the
default behavior?
3. Am I wrong in my assumption that this is a line termination
problem, and should look else where?
4. Is it possible that this is a bug.

Thanks-in-advance

Bruce

Greetings Bruce-

I have been under the assumption that there are three standards to line ends: DOS/Win, Lin, Mac.

Using the linein()/lineout() method of file I/O, I assume ooRexx uses the current OS EOL character as what it is looking for.

However, that is not the only way to do file I/O.

Certain methods were added a number of years ago to achieve high performance I/O for text files. The approach was to read the entire file in one call, then hand that big string to a method which
looked for EOL characters and busted each line up into a separate Array item. Likewise the reverse process is also supported: dumping an array out to a text file.

I have not tested if this method of I/O is safe even for multiple EOL chars existing in the same file... such as ooRexx and lineout() being used on different platforms to create entries to a single
file, and then read that all back in with the new high perf I/O method.

Anyway, I will paste in my suggested I/O method below. It would mean that you need ooRexx to run the app, but hopefully it would resolve your problem.

/************************************************** **********************************/
/* METHOD : READfile */
/* */
/* PARAMETERS : FINDfilename */
/* */
/* DESCRIPTION : Read a text file from disk and put it in memory */
/* */
/* RETURNS : Logical */
/************************************************** **********************************/
::method READfile
expose LDSSystemInfo LDSLogging LDSFileSystem FINDarray READmode
/* DEV */ if LDSSystemInfo~tracemode\='' then interpret 'trace 'LDSSystemInfo~tracemode
parse arg FINDfilename

/* DEV */ LDSLogging~Verbose('OBJECTCUR LDSTextFile~READfile, FINDfilename="' || FINDfilename || '"')

/* Check Params */
if FINDfilname='' then do
LDSLogging~Error('OBJECTCUR LDSTextFile~READfile -> Not Enough Paramaters RC=1')
return 0
end

/* Check to see that the file exists */
if \ LDSFileSystem~ISfile(FINDfilename) then do
LDSLogging~Logger('OBJECTCUR LDSTextFile~READfile -> FINDfilename="' || FINDfilename || '" is not on this machine')
return 0
end

/* Open a stream handle against the file to be read */
file=.stream~new(FINDfilename)
file~open(READmode)
if result\='READY:' then do
LDSLogging~Logger('OBJECTCUR LDSTextFile~READfile -> Error opening stream for READ, RC=' || result)
return 0
end

/* Check for how many lines the file has */
FILEsize=file~command('QUERY SIZE')

/* Read in the file */
/* This code requires at least ORexx 2.1.1 on Win32 - only tested with >=2.1.2 */
INstr=file~charin(1, FILEsize)
FINDarray=INstr~makearray()

/* Close the stream */
file~close()

/* DEV */ LDSLogging~Verbose('OBJECTCUR LDSTextFile~READfile -> Total Lines=' || self~Lines() || ' FINDarray[1]=' || self~GETline(1))

return 1

/************************************************** **********************************/
/* METHOD : WRITEfile */
/* */
/* PARAMETERS : FINDfilename */
/* */
/* DESCRIPTION : Write the data held in memory out to a file on disk */
/* */
/* RETURNS : Logical */
/************************************************** **********************************/
::method WRITEfile
expose LDSSystemInfo LDSLogging LDSFileSystem FINDarray WRITEmode
/* DEV */ if LDSSystemInfo~tracemode\='' then interpret 'trace 'LDSSystemInfo~tracemode
parse arg FINDfilename

/* DEV */ LDSLogging~Verbose('OBJECTCUR LDSTextFile~WRITEfile -> FINDfilename="' || FINDfilename || '" WRITEmode=' || WRITEmode || ' Total Lines=' || self~lines() || ' FINDarray.1=' || FINDarray.1)

/* Check Params */
if FINDfilname='' then do
LDSLogging~Error('OBJECTCUR LDSTextFile~WRITEfile -> Not Enough Paramaters RC=1')
return 0
end

/* Make sure we have some data to write out */
if self~lines()=0 then return 0

/* Decide how to write out the file */
select
when WRITEmode='QUICK' then do
/* Open a stream to write the file */
file=.stream~new(FINDfilename)
file~open('WRITE REPLACE')
if result\='READY:' then do
LDSLogging~logger('OBJECTCUR LDSTextFile~WRITEfile -> Error opening stream for WRITE, RC=' || result)
return 0
end
end /* WRITEmode=QUICK */

when WRITEmode='APPEND' then do
/* Open a stream to write the file */
file=.stream~new(FINDfilename)
file~open('WRITE APPEND')
if result\='READY:' then do
LDSLogging~logger('OBJECTCUR LDSTextFile~WRITEfile -> Error opening stream for WRITE APPEND, RC=' || result)
return 0
end
end /* WRITEmode=APPEND */

otherwise do
LDSLogging~Logger('OBJECTCUR LDSTextFile~WRITEfile -> Not valid WRITEmode found, WRITEmode=' || WRITEmode || ' RC=1')
return 0
end /* otherwise */
end /* select */

/* Write out the file */
/* This code requires at least ORexx 2.1.3 on Win32 */
if file~LineOut(FINDarray~MakeString()) then do
LDSLogging~Logger('OBJECTCUR LDSTextFile~WRITEfile -> Error calling LineOut, RC=' || rc)
file~close()
return 0
end

/* Close the stream */
file~close()

return 1


--
Michael Lueck
Lueck Data Systems
http://www.lueckdatasystems.com/

On Sep 6, 5:27*am, Michael Lueck <mlu...@lueckdatasystems.com> wrote:
> Greetings Bruce-
>
> I have been under the assumption that there are three standards to line ends: DOS/Win, Lin, Mac.
>
> Using the linein()/lineout() method of file I/O, I assume ooRexx uses thecurrent OS EOL character as what it is looking for.
>
> However, that is not the only way to do file I/O.
>
> Certain methods were added a number of years ago to achieve high performance I/O for text files. The approach was to read the entire file in one call, then hand that big string to a method which
> looked for EOL characters and busted each line up into a separate Array item. Likewise the reverse process is also supported: dumping an array out to a text file.
>
> I have not tested if this method of I/O is safe even for multiple EOL chars existing in the same file... such as ooRexx and lineout() being used on different platforms to create entries to a single
> file, and then read that all back in with the new high perf I/O method.
>
> Anyway, I will paste in my suggested I/O method below. It would mean thatyou need ooRexx to run the app, but hopefully it would resolve your problem.
>
> /************************************************** ************************ **********/
> /* METHOD * * * * : *READfile * * * * * * * * * * * * * * * * * * * * * * * * * * **/
> /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ** * * * **/
> /* PARAMETERS * * : *FINDfilename * * * * * * * * * * * * * * * * * * * * * * * * * */
> /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ** * * * **/
> /* DESCRIPTION * *: *Read a text file from disk and put it in memory * * * * * * * **/
> /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ** * * * **/
> /* RETURNS * * * *: *Logical * * * * * * * * * * * * * * * * * * * * * * * * * * ***/
> /************************************************** ************************ **********/
> ::method READfile
> * * expose LDSSystemInfo LDSLogging LDSFileSystem FINDarray READmode
> /* DEV */ if LDSSystemInfo~tracemode\='' then interpret 'trace 'LDSSystemInfo~tracemode
> * * parse arg FINDfilename
>
> /* DEV */ * LDSLogging~Verbose('OBJECTCUR LDSTextFile~READfile, FINDfilename="' || FINDfilename || '"')
>
> * * /* Check Params */
> * * if FINDfilname='' then do
> * * * *LDSLogging~Error('OBJECTCUR LDSTextFile~READfile -> Not Enough Paramaters RC=1')
> * * * *return 0
> * * end
>
> * * /* Check to see that the file exists */
> * * if \ LDSFileSystem~ISfile(FINDfilename) then do
> * * * *LDSLogging~Logger('OBJECTCUR LDSTextFile~READfile -> FINDfilename="' || FINDfilename || '" is not on this machine')
> * * * *return 0
> * * end
>
> * * /* Open a stream handle against the file to be read */
> * * file=.stream~new(FINDfilename)
> * * file~open(READmode)
> * * if result\='READY:' then do
> * * * *LDSLogging~Logger('OBJECTCUR LDSTextFile~READfile -> Erroropening stream for READ, RC=' || result)
> * * * *return 0
> * * end
>
> * * /* Check for how many lines the file has */
> * * FILEsize=file~command('QUERY SIZE')
>
> * * /* Read in the file */
> * * /* This code requires at least ORexx 2.1.1 on Win32 - only testedwith >=2.1.2 */
> * * INstr=file~charin(1, FILEsize)
> * * FINDarray=INstr~makearray()
>
> * * /* Close the stream */
> * * file~close()
>
> /* DEV */ * LDSLogging~Verbose('OBJECTCUR LDSTextFile~READfile -> TotalLines=' || self~Lines() || ' FINDarray[1]=' || self~GETline(1))
>
> return 1
>
> /************************************************** ************************ **********/
> /* METHOD * * * * : *WRITEfile * * * * * * * * * * * * * * * * * * * * * * * * * * **/
> /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ** * * * **/
> /* PARAMETERS * * : *FINDfilename * * * * * * * * * * * * * * * * * * * * * * * * * */
> /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ** * * * **/
> /* DESCRIPTION * *: *Write the data held in memory out to a file ondisk * * * * * **/
> /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ** * * * **/
> /* RETURNS * * * *: *Logical * * * * * * * * * * * * * * * * * * * * * * * * * * ***/
> /************************************************** ************************ **********/
> ::method WRITEfile
> * * expose LDSSystemInfo LDSLogging LDSFileSystem FINDarray WRITEmode
> /* DEV */ if LDSSystemInfo~tracemode\='' then interpret 'trace 'LDSSystemInfo~tracemode
> * * parse arg FINDfilename
>
> /* DEV */ * LDSLogging~Verbose('OBJECTCUR LDSTextFile~WRITEfile -> FINDfilename="' || FINDfilename || '" WRITEmode=' || WRITEmode || ' Total Lines=' || self~lines() || ' FINDarray.1=' || FINDarray.1)
>
> * * /* Check Params */
> * * if FINDfilname='' then do
> * * * *LDSLogging~Error('OBJECTCUR LDSTextFile~WRITEfile -> Not Enough Paramaters RC=1')
> * * * *return 0
> * * end
>
> * * /* Make sure we have some data to write out */
> * * if self~lines()=0 then return 0
>
> * * /* Decide how to write out the file */
> * * select
> * * * *when WRITEmode='QUICK' then do
> * * * * * /* Open a stream to write the file */
> * * * * * file=.stream~new(FINDfilename)
> * * * * * file~open('WRITE REPLACE')
> * * * * * if result\='READY:' then do
> * * * * * * *LDSLogging~logger('OBJECTCUR LDSTextFile~WRITEfile -> Error opening stream for WRITE, RC=' || result)
> * * * * * * *return 0
> * * * * * end
> * * * *end /* WRITEmode=QUICK */
>
> * * * *when WRITEmode='APPEND' then do
> * * * * * /* Open a stream to write the file */
> * * * * * file=.stream~new(FINDfilename)
> * * * * * file~open('WRITE APPEND')
> * * * * * if result\='READY:' then do
> * * * * * * *LDSLogging~logger('OBJECTCUR LDSTextFile~WRITEfile -> Error opening stream for WRITE APPEND, RC=' || result)
> * * * * * * *return 0
> * * * * * end
> * * * *end /* WRITEmode=APPEND */
>
> * * * *otherwise do
> * * * * * LDSLogging~Logger('OBJECTCUR LDSTextFile~WRITEfile ->Not valid WRITEmode found, WRITEmode=' || WRITEmode || ' RC=1')
> * * * * * return 0
> * * * *end /* otherwise */
> * * end /* select */
>
> * * /* Write out the file */
> * * /* This code requires at least ORexx 2.1.3 on Win32 */
> * * if file~LineOut(FINDarray~MakeString()) then do
> * * * *LDSLogging~Logger('OBJECTCUR LDSTextFile~WRITEfile -> Error calling LineOut, RC=' || rc)
> * * * *file~close()
> * * * *return 0
> * * end
>
> * * /* Close the stream */
> * * file~close()
>
> return 1
>
> --
> Michael Lueck
> Lueck Data Systemshttp://www.lueckdatasystems.com/

Wow Michael, That is some nice code. It is also longer than the
program that I was having problems with. It just occurred to me that
ooRexx should be self documenting, in that I would assume that linein
and lineout should behave the same way, so what ever line terminator I
get on a lineout should be the terminator linein is expecting.

Thanks -- Bruce

CVBruce wrote:
> It is also longer than the
> program that I was having problems with.

LOL! Well, when I want to use this class in a program, all it takes is a single ::requires line and all of this functionality is instantly imported.

_No going back to Classic Rexx!_

--
Michael Lueck
Lueck Data Systems
http://www.lueckdatasystems.com/