RfD - Enhanced local variable syntax, v4 (long)

Well, it's that time of year again, so I'm in proposal mode.
Don't forget to attend EuroForth 2008 and the Forth200x standards
meeting in gorgeous glorious Vienna from 25-28 September 2008.
http://www.euroforth.org

I have separated the enhanced local variable syntax from the
local buffer proposal. The main extension in this proposal
is the separation of initialised local arguments from
uninitialised local values.

Stephen

RfD - Enhanced local variable syntax, v4
====================================
Stephen Pelc - 11 August 2008

20080811 Removed references to local buffers as appropriate.
20070914 Moved local buffers to separate proposal.
20070607 Wordsmithing. Corrected reference implementation.
20060822 Added explanatory text.
Corrected reference implementation.
Updated ambiguous conditions.

Problem
=======
1) The current LOCALS| ... | notation explicitly forces all locals
to be initialised from the data stack.
2) 1) The current LOCALS| ... | notation defines locals in reverse
order to the normal stack notation.

This proposal is derived from implementations that have existed for
more than 15 years.


Solution
========
Base version
------------
The following syntax for local arguments and local values is
proposed. The sequence:
{ ni1 ni2 ... | lv1 lv2 ... -- o1 o2 ... }
defines local arguments, local values, and outputs. The local
arguments are automatically initialised from the data stack on
entry, the rightmost being taken from the top of the data stack.
Local arguments and local values can be referenced by name within
the word during compilation. The output names are dummies to allow
a complete stack comment to be generated.
The items between { and | are local arguments.
The items between | and -- are local values or local buffers.
The items between -- and } are outputs for formal comments only.

The outputs are provided in the notation so that complete stack
comments can be produced. However, all text between -- and } is
ignored. The facility is there to permit the notation to form a
complete stack comment. This eases documentation and current
users of the notation like this facility.

Local arguments and values return their values when referenced,
and must be preceded by TO to perform a store.

Any name ending in the '[' character is reserved for compatibility
with existing implementations.

In the example below, a and b are local arguments, a+b and a*b are
local values, and arr[ is a 10 byte local buffer.

: foo { a b | a+b a*b -- }
a b + to a+b
a b * to a*b
cr a+b . a*b .
;

Local types and extensions
--------------------------
Some current Forth systems use indicators to define local values
of sizes other than a cell. It is proposed that any name ending
in a ':' (colon) be reserved for this use.

: foo { a b | F: f1 F: f2 -- c }
...
;

At least one significant Forth implementation uses local value
names ending in the '[' character to indicate local buffers. This
character is reserved to prevent disenfranchising implementations
that have that behaviour. For similar reasons the use of '[' and
'\' as local argument or value names is also reserved.


Discussion
==========
The '|' (ASCII 0x7C) character is widely used as the separator
between local arguments and local values. Other characters
accepted in current Forth implementations are '\' (ASCII 0x5C) and
'¦' (ASCII 0xA6).. Since the ANS standard is defined in terms of
7 bit ASCII, and with regard to internationalistion, we propose only
to consider the '|' and '\' characters further. Only recognition of
the '|' separator is mandatory.

The use of local types is contentious as they only become useful
if TO is available for these. In practice, some current systems
permit TO to be used with floats (children of FVALUE) and other
data types. Such systems often provide additional operators such
as +TO (add from stack to item) for children of VALUE and FVALUE.
Standardisation of operators with (for example) floats needs to
be done before the local types extension can be incorporated into
Forth200x. Apart from forcing allocation of buffer space, no
additional functionality is provided by local types that cannot
be obtained using local buffers. More preparatory standardisation
needs to be done before local types can be standardised.

It has been noted that one widely used implementation uses brace
for multiline comments. However, inspection of the vendor's code
shows that this use only occurs during interpretation. The
interpretation semantics of brace in this proposal are undefined
in order for that implementation to coexist with this proposal.


Forth 200x text
===============
13.6.2.xxxx {
brace LOCAL EXT

Interpretation: Interpretation semantics for this word are undefined.

Compilation:
( "<spaces>arg1" ... "<spaces>argn" | "<spaces>lv1" ... "<spaces>lvn"
-- )

Create up to eight local arguments by repeatedly skipping leading
spaces, parsing arg, and executing implementation defined actions.
The list of local arguments to be defined is terminated by "|",
"--" or "}". Append the run-time semantics for local arguments
given below to the current definition. If a space delimited '|' is
encountered, create up to eight local values by repeatedly skipping
leading spaces, parsing the "lv" token, and creating the local
element. The list of local values to be defined is terminated by
"--" or "}". Append the run-time semantics for local values
given below to the current definition. If "--" has
been encountered, further text between "--" and } is ignored.

Local argument run-time: ( x1 ... xn -- )
Local value run-time: ( -- )

Initialize up to eight local arguments from the data stack
Local argument arg1 is initialized with x1, arg2 with x2 up
to argn from xn, which is on the top of the data stack. When
invoked, each local argument will return its value. The value
of a local argument may be changed using 13.6.1.2295 TO.

Initialize up to eight local values. The initial contents of local
values are undefined. When invoked, each local value
returns its value. The value of a local value may be changed
using 13.6.1.2295 TO. The size of a local value is a cell.
The user may make no assumption about the order and contiguity of
local values in memory.

Ambiguous conditions:
a) The { ... } text extends over more than one line.
b) { ... } is declared more than once in a word.
c) Parsing units '|', ']', '--' and '}' are not whitespace delimited.


Reference implementation
=========================
(currently untested)

0 [if]
BUILDLV c-addr u +n mode
When executed during compilation, BUILDLV passes a message to the
system identifying a new local argument whose definition name is
given by the string of characters identified by c-addr u. The size
of the data item is given by +n address units, and the mode
identifies the construction required as follows:
0 - finish construction of initialisation and data storage
allocation code. C-addr and u are ignored. +n is 0
(other values are reserved for future use).
1 - identify a local argument, +n = cell
2 - identify a local value, +n = cell
3+ - reserved for future use
-ve - implementation specific values

The result of executing BUILDLV during compilation of a definition
is to create a set of named local arguments, values and/or
buffers, each of which is a definition name, that only have
execution semantics within the scope of that definition's source.
[then]

: BUILDLV \ c-addr u +n mode --
\ Dummy for testing
CR 2SWAP TYPE SPACE SWAP . .
;

: TOKEN \ -- caddr u
\ Get the next space delimited token from the input stream.
\ Can be extended to permit multiple line declarations.
PARSE-NAME
;

: LTERM? \ caddr u -- flag
\ Return true if the string caddr/u is "--" or "}"
2DUP S" --" COMPARE 0= >R
S" }" COMPARE 0= R> OR
;

: LSEP? \ caddr u -- flag
\ Return true if the string caddr/u is the separator between
\ local arguments and local values or buffers.
2DUP S" |" COMPARE 0= >R
S" \" COMPARE 0= R> OR
;

: { ( -- )
0 >R \ indicate arguments
BEGIN
TOKEN 2DUP LTERM? 0=
WHILE \ -- caddr len
2DUP LSEP? IF \ if '|'
R> DROP 1 >R \ change to vars and buffers
ELSE
R@ 0= IF \ argument?
CELL 1
ELSE \ value or buffer
CELL 2
THEN
BUILDLV
THEN
REPEAT
BEGIN
S" }" COMPARE
WHILE
TOKEN
REPEAT
0 0 0 0 BUILDLV
R> DROP
; IMMEDIATE



--
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

Stephen Pelc wrote:
> In the example below, a and b are local arguments, a+b and a*b are
> local values, and arr[ is a 10 byte local buffer.
>
>: foo { a b | a+b a*b -- }
> a b + to a+b
> a b * to a*b
> cr a+b . a*b .
> ;

Looks like you missed this text when removing the local buffer stuff...

--Josh

Stephen Pelc <stephenXXX@mpeforth.com> wrote:
> The following syntax for local arguments and local values is
> proposed. The sequence:
> { ni1 ni2 ... | lv1 lv2 ... -- o1 o2 ... }
> defines local arguments, local values, and outputs. The local
> arguments are automatically initialised from the data stack on
> entry, the rightmost being taken from the top of the data stack.
> Local arguments and local values can be referenced by name within
> the word during compilation. The output names are dummies to allow
> a complete stack comment to be generated.
> The items between { and | are local arguments.
> The items between | and -- are local values or local buffers.
> The items between -- and } are outputs for formal comments only.

So far, reasonable enough.

> The outputs are provided in the notation so that complete stack
> comments can be produced. However, all text between -- and } is
> ignored. The facility is there to permit the notation to form a
> complete stack comment. This eases documentation and current
> users of the notation like this facility.

No, this is vile. Mixing semantically significant code with comments
in such an arbitrary way is a bad idea: it has no precedent in Forth,
and it is unnecessarily confusing. I thought that { a b -- c d }
meant that the contents of the variables after the "--" would be
automatically returned as the result, but this is not what happens.
While this convention may be in use in some systems it isn't suitable
for standardization.

In any case, it makes no sense to use the { } as the stack comment
for a word since not every to a word need be placed into a local
variable.

Andrew.

stephenXXX@mpeforth.com (Stephen Pelc) writes:

> Well, it's that time of year again, so I'm in proposal mode.
> Don't forget to attend EuroForth 2008 and the Forth200x standards
> meeting in gorgeous glorious Vienna from 25-28 September 2008.
> http://www.euroforth.org
>
> RfD - Enhanced local variable syntax, v4
> Stephen Pelc - 11 August 2008

....

> Any name ending in the '[' character is reserved for compatibility
> with existing implementations.

I don't like standards, which start from exceptions.
Why is this the only character reserved?
Why don't we reserve other characters from "!@#$%^&*()[]\~;':<>?,./" set?

Instead of conceptually clear syntax you're making precedent for all sort
of quirks. I strongly object to such attitude to the standard.
It is going to be yet another Perl at this pace.

If you want some characters to be reserved, put your locals library
into public domain and let it spread. At least we'll see, what is
actually used in community.


--
CE3OH...

On Mon, 11 Aug 2008 23:41:12 +0400, Aleksej Saushev <asau@inbox.ru>
wrote:

>stephenXXX@mpeforth.com (Stephen Pelc) writes:
>> Any name ending in the '[' character is reserved for compatibility
>> with existing implementations.
>
>I don't like standards, which start from exceptions.
>Why is this the only character reserved?

As you quoted
"for compatibility with existing implementations."
To prevent code breakage in VFX Forth systems both hosted and
embedded. MPE has been using the nortation
: foo { a b | c d e[ 5 cells ] f[ 7 cells ] -- }
for local arrays for 15 years or so. I'm not silly enough to
put forward a proposal that will break our own code. See the
local buffers proposal.

The point of this proposal is that it introduces uninitialised
local values as well as local arguments.

>If you want some characters to be reserved, put your locals library
>into public domain and let it spread. At least we'll see, what is
>actually used in community.

See the reference implementation(s). Yes, there's carnal knowledge
involved, but if you've already implemented (LOCAL), implementing
BUILDLV is trivial. AS I've already said we've used it for 15 years,
and variants with a similar notation exist in Win32Forth and gForth.

Stephen


--
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

On Mon, 11 Aug 2008 11:10:06 -0500, Andrew Haley
<andrew29@littlepinkcloud.invalid> wrote:

>> The outputs are provided in the notation so that complete stack
>> comments can be produced. However, all text between -- and } is
>> ignored. The facility is there to permit the notation to form a
>> complete stack comment. This eases documentation and current
>> users of the notation like this facility.
>
>No, this is vile. Mixing semantically significant code with comments
>in such an arbitrary way is a bad idea: it has no precedent in Forth,
>and it is unnecessarily confusing.

You don't have to do it that way
{ a b | c d }
is legal according to the proposal. There are, believe it or not,
Forth systems out there that use '--' as a comment to end of line.

To rephrase in a reductionist way:
"Mixing semantically significant code with comments
is a bad idea"
So all comments are a bad idea?

As you well know, the point of the '--' is to avoid having to
write both a stack comment and a very similar locals declaration.
As you well know, having those will be a source of error, so
in the best Forth practice, we eliminate the source of error.
After 15 years of use we and our clients know that this brace
notation works well. That it also exists in a similar form in
Win32Forth and gForth indicates utility. If you don't like
the '--', don't use it, and write a stack comment as well as
the locals declaration. Your choice.

Stephen

--
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

Andrew Haley wrote:
> Stephen Pelc <stephenXXX@mpeforth.com> wrote:
>> The following syntax for local arguments and local values is
>> proposed. The sequence:
>> { ni1 ni2 ... | lv1 lv2 ... -- o1 o2 ... }
>> defines local arguments, local values, and outputs. The local
>> arguments are automatically initialised from the data stack on
>> entry, the rightmost being taken from the top of the data stack.
>> Local arguments and local values can be referenced by name within
>> the word during compilation. The output names are dummies to allow
>> a complete stack comment to be generated.
>> The items between { and | are local arguments.
>> The items between | and -- are local values or local buffers.
>> The items between -- and } are outputs for formal comments only.
>
> So far, reasonable enough.
>
>> The outputs are provided in the notation so that complete stack
>> comments can be produced. However, all text between -- and } is
>> ignored. The facility is there to permit the notation to form a
>> complete stack comment. This eases documentation and current
>> users of the notation like this facility.
>
> No, this is vile. Mixing semantically significant code with comments
> in such an arbitrary way is a bad idea: it has no precedent in Forth,
> and it is unnecessarily confusing. I thought that { a b -- c d }
> meant that the contents of the variables after the "--" would be
> automatically returned as the result, but this is not what happens.
> While this convention may be in use in some systems it isn't suitable
> for standardization.
>
> In any case, it makes no sense to use the { } as the stack comment
> for a word since not every to a word need be placed into a local
> variable.

I strongly agree with Andrew. You shouldn't try to mix syntactic
elements with comments. Among other things, it creates the necessity
for reserving [ which Alexei deplores, and I do as well. This is
opening all sorts of Pandors's Boxes.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

On Mon, 11 Aug 2008 12:23:35 -1000, Elizabeth D Rather
<erather@forth.com> wrote:

>I strongly agree with Andrew. You shouldn't try to mix syntactic
>elements with comments. Among other things, it creates the necessity
>for reserving [ which Alexei deplores, and I do as well.

The [ reservation has *nothing* to do with ignoring the section
between '--' and '}'. The '--' marker is *optional*.

This proposal has existed in VFX Forth and its predecessors for 15+
years and is supported (without the '[' reservation) in Win32Forth
and gforth.

Stephen

--
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

On Mon, 11 Aug 2008 23:41:12 +0400, Aleksej Saushev <asau@inbox.ru>
wrote:

>Instead of conceptually clear syntax you're making precedent for all sort
>of quirks. I strongly object to such attitude to the standard.
>It is going to be yet another Perl at this pace.

It's a standards proposal. The conceptual clarity comes from
considering the brace notation as a formal comment that mirrors
the conventional stack comment notation:
( a b c -- d e }
This now extended to include local values by adding the '|'
separator and changing the brackets to braces:
{ a b c | x y z -- d e }
Since we permit the data stack and return stack to be manipulated
between the closing '}' and ';' we chose 15+ years ago to ignore
what is between the '--' and the closing brace.

>If you want some characters to be reserved, put your locals library
>into public domain and let it spread. At least we'll see, what is
>actually used in community.

The system is implemented in Win32Forth and gforth (without the '['
reservation. The reservation (should be an ambiguous condition?) is
simply to indicate that VFX Forth uses it to mean something, and that
there is body of code in the wild that uses that meaning. I suspect
that use in VFX Forth, Win32Forth and gforth means that the idea has
traction.

None of the proposed local buffer notations read particularly
well, yet the facility is available one one way or another in
most Forths used for large applications. However, because of the
flame wars that arose in the previous draft proposal, I split
the local buffer extension into a separate proposal. The new
local buffer proposal adopt's Ed's notation which arose during
the last round of discussion.

I propose that the discussion of '[' and friends now moves to
the local buffer thread.

Stephen

--
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

Andrew Haley wrote:

>> The outputs are provided in the notation so that complete stack
>> comments can be produced. However, all text between -- and } is
>> ignored. The facility is there to permit the notation to form a
>> complete stack comment. This eases documentation and current
>> users of the notation like this facility.
>
> No, this is vile. Mixing semantically significant code with comments
> in such an arbitrary way is a bad idea: it has no precedent in Forth,

It works just the same as what I've used on two different Forths.

> and it is unnecessarily confusing. I thought that { a b -- c d }
> meant that the contents of the variables after the "--" would be
> automatically returned as the result, but this is not what happens.

Thankfully. That is too much automation, IMHO.


> While this convention may be in use in some systems it isn't suitable
> for standardization.

Where is the harm? 9 times out of 10 the { ... } locals declaration
can
do double duty as a stack effect comment. Yes, there are times where
it
doesn't quite work and then we have to do something like:

: foo ( a b c -- d )
{ b c }

no big deal, really and it is obvious what is going on


> In any case, it makes no sense to use the { } as the stack comment
> for a word since not every to a word need be placed into a local
> variable.

It is easy to handle that case in a way similar to what I've shown
just
above.

Again, I strongly prefer the ability to have the locals declaration do
double duty as a stack comment. It does no harm and most of the time
is
a help (less typing if the stack comment matches the locals
declaration,
which is often the case).

-Doug