Advanced Forth app techniques

Most of what I find written about Forth programming involves the
basics; data stack, control structures, I/O, compilation. I'm
interested in learning ways that complete non-programming-language-
related applications (e.g. NOT Forth's written in Forth) have
leveraged Forth's unique capabilities to solve problems differently
from conventional practices.

Specifically I'd love learning about anything having to do with:
- Using Compile Time
- Usage of Wordlists (how was a program structured? how were wordlists
used? any app-specific abstractions that used wordlists?)
- Featuring the data stack
- Using the interpreter to process text files, or other text
processing applications
- Forth-style data structures
- How a complex application was structured (how did you avoid name-
collisions? any "scripts" compiled or run from within the
application?)
- Data storage implementation details (how memory and disk was used)
- Extensions to the compiler
- Modifications to the Forth system (extending the system into a new
domain-specific environment)

I'm eager to learn from the experience of others, to get a better idea
of what is possible in Forth and expand my bag of tricks, if anyone
has any interesting stories to tell or something useful to teach.

roger.levy@gmail.com wrote:
> Most of what I find written about Forth programming involves the
> basics; data stack, control structures, I/O, compilation. I'm
> interested in learning ways that complete non-programming-language-
> related applications (e.g. NOT Forth's written in Forth) have
> leveraged Forth's unique capabilities to solve problems differently
> from conventional practices.

It's a good question. It would be helpful to hear what you *have* read,
so we know where you're starting from. Have you read "Thinking Forth"
(Google for it)? Although it's somewhat dated, it does address some of
these issues.

Most of our application work nowadays is in embedded systems, which is
reflected in my comments below.

> Specifically I'd love learning about anything having to do with:
> - Using Compile Time

Aside from trivial things such as generating literals, this is a
technique that's used very rarely in my experience, but can be a
powerful weapon to keep in reserve for certain occasions.

One of my favorite good examples is the construction of a chip-select
table for a 68332. Single bits here and there in the cells of the table
control critical functions, and they make little sense examined in hex.
We made a set of words that set the appropriate bits at compile time
in ways that are absolutely obvious:

{ ---------------------------------------------------------------------
Chip select table

CHIPS is the chip select table for the NMIX-0332.
|CHIPS| is the table size in bytes.
--------------------------------------------------------------------- }

CREATE CHIPS
0 Select16 \ CS10 (A23)
Select16 \ CS9 (A22)
Select16 \ CS8 (A21)
Select16 \ CS7 (A20)
Select16 \ CS6 (A19)

0 Select16 \ CS5 (FC2)
Select16 \ CS4 (FC1)
Select16 \ CS3 (FC0)
Select16 \ CS2 (BGACK)
Select16 \ CS1 (BG)
Select16 \ CS0 (BR)
Select16 \ CSBOOT

( CSPAR0) W, ( CSPAR1) W,

Async Both Read AS 9 WS S/U 'PROM 128K \ CSBOOT
Async Both Read AS 9 WS S/U 'PROM 20000 + 128K \ CS0
Async Both R/W AS 1 WS S/U 0 128K \ CS1
Async Both R/W AS 1 WS S/U 20000 128K \ CS2
N/C \ CS3
N/C \ CS4
N/C \ CS5
N/C \ CS6
N/C \ CS7
N/C \ CS8
N/C \ CS9
N/C \ CS10

HERE CHIPS - EQU |CHIPS| IDATA

> - Usage of Wordlists (how was a program structured? how were wordlists
> used? any app-specific abstractions that used wordlists?)

We don't use wordlists very extensively in applications, but they are
absolutely vital in our systems, especially cross- and meta- compilers.
They're also critical to SwiftForth's SWOOP facility.

> - Featuring the data stack

Not sure just what you mean, here.

> - Using the interpreter to process text files, or other text
> processing applications

There's info on the in Forth Application Techniques (one of my books,
available through www.forth.com).

> - Forth-style data structures

See Thinking Forth and Forth Application Techniques. We use this
capability extensively in applications, most of which have app-specific
data structures of some kind. Examples range from circular buffers to
named bits in an I/O register. Automatically indexing arrays are also
very useful.

> - How a complex application was structured (how did you avoid name-
> collisions? any "scripts" compiled or run from within the
> application?)

We don't actually worry a lot about name collisions in applications. A
lot of application words are only used locally, and if there's a
different one somewhere the compiler will issue a warning. Sometimes we
just examine the other version, conclude there's no practical conflict,
and forge ahead.

> - Data storage implementation details (how memory and disk was used)

We used to have an extensive database package in polyFORTH. We have
made a modified version to use for flash databases, which we'll probably
include in SwiftX at some point.

In embedded systems, it's important to manage ROM/Flash and RAM
carefully. We have tools in SwiftX to do this, reflected in the
proposed Cross Compiler Standard, discussed in
http://www.forth.com/downloads/ANS/XCpaper.pdf

> - Extensions to the compiler

You mean like custom structure words? They're discussed here far more
than they're used in applications, in my experience.

> - Modifications to the Forth system (extending the system into a new
> domain-specific environment)

Well, that's what an application is, isn't it? The boundary between
"system" and "application" is pretty fuzzy in the Forth world. Every
time we write an application we end up with a system adapted to support
that application domain. Makes it really easy to do similar projects.
For example, GE Multilin in Vancouver has a whole line of multiplexers
programmed in Forth, with a lot of shared code (see
http://www.forth.com/resources/appNo...-Multilin.html).

> I'm eager to learn from the experience of others, to get a better idea
> of what is possible in Forth and expand my bag of tricks, if anyone
> has any interesting stories to tell or something useful to teach.

I also wish more of this lore could be captured and made available in
written form. Unfortunately, most of the programmers with good stories
to tell are too busy writing new apps to write about old ones, and most
companies consider their app code proprietary. I hope some other folks
here will chime in with more useful stuff.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

On Thu, 21 Aug 2008 10:55:23 -0700 (PDT), "roger.levy@gmail.com"
<roger.levy@gmail.com> wrote:

>Specifically I'd love learning about anything having to do with:
>- Using Compile Time
>- Usage of Wordlists (how was a program structured? how were wordlists
>used? any app-specific abstractions that used wordlists?)
>- Featuring the data stack
>- Using the interpreter to process text files, or other text
>processing applications
>- Forth-style data structures
>- How a complex application was structured (how did you avoid name-
>collisions? any "scripts" compiled or run from within the
>application?)
>- Data storage implementation details (how memory and disk was used)
>- Extensions to the compiler
>- Modifications to the Forth system (extending the system into a new
>domain-specific environment)

There's a chapter on using the Forth interpreter (including for
compilation) in my book
http://www.mpeforth.com/arena/ProgramForth.pdf

VFX Forth comes with a wide range of examples, including a state
machine system which illustrates much of what you want.

Stephen


--
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

"roger.levy@gmail.com" <roger.levy@gmail.com> writes:
>Most of what I find written about Forth programming involves the
>basics; data stack, control structures, I/O, compilation. I'm
>interested in learning ways that complete non-programming-language-
>related applications (e.g. NOT Forth's written in Forth) have
>leveraged Forth's unique capabilities to solve problems differently
>from conventional practices.

Even though it is programming-language-related (but definitely not
about implementing Forth), you might be interested in

@InProceedings{ertl99ef,
author = "M. Anton Ertl",
title = "Is {Forth} Code Compact? {A} Case Study",
booktitle = "EuroForth'99 Conference Proceedings",
year = "1999",
address = "St. Petersburg, Russia",
URL = "http://www.complang.tuwien.ac.at/papers/ertl99ef.ps.gz",
abstract = "Forth advocates often claim that Forth code is
smaller, faster, and requires less development time
than equivalent programs in other languages. This paper
investigates this claim by comparing a number of parser
generators written in various languages with respect to
source code size. The smallest parser generator (14
lines) in this comparison is written in Forth, and the
other Forth program is smaller than the others in its
class by a factor of 8 or more; however, the Forth
programs do not have all the features of their
counterparts. I took a closer look at Gray (in Forth)
and Coco/R (in Modula-2) and found that several Forth
features missing from Modula-2 give Gray more than a
factor of three advantage over Coco/R (even if the
other size differences were solely due to differences
in functionality): run-time code generation; access to
the parser and a simple, flexible syntax; and Forth's
dictionary.",
}

This paper mentions several of the techniques you ask about. I does
not give much detail, but if you want to learn more about that, you
can look at the source code of the program.

http://www.complang.tuwien.ac.at/forth/gray.zip

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2008: http://www.euroforth.org/ef08.html

> It's a good question. *It would be helpful to hear what you *have* read,
> so we know where you're starting from. *Have you read "Thinking Forth"
> (Google for it)? *Although it's somewhat dated, it does address some of
> these issues.

I've read and own both Starting Forth and Thinking Forth. I treat
them like bibles and re-read Thinking Forth for fun/encouragement
often. Though the knowledge in them is starting to get a little
stale ...

>
> Most of our application work nowadays is in embedded systems, which is
> reflected in my comments below.

It's frustrating; I know that embedded is where Forth thrives today.
But as I want pretty badly to become comfortable in Forth to build my
own apps on Windows, I puzzle without knowing for sure what works and
what doesn't and often I feel like I'm needlessly re-playing history,
and one that I don't quite understand. I'd like to see more source
code from serious applications, even if they're old, so long as they
competed with the other apps from their time. Ones written in "good"
Forth style, not some of the sources I've seen that look just like C
code with lots of nesting, long definitions, each line with a comment
basically repeating the statement...


>
> One of my favorite good examples is the construction of a chip-select
> table for a 68332. *Single bits here and there in the cells of the table
> control critical functions, and they make little sense examined in hex.
> * We made a set of words that set the appropriate bits at compile time
> in ways that are absolutely obvious:


>
> { ---------------------------------------------------------------------
> Chip select table
>
> CHIPS is the chip select table for the NMIX-0332.
> |CHIPS| is the table size in bytes.
> --------------------------------------------------------------------- }
>
> CREATE CHIPS
> 0 *Select16 * * \ CS10 (A23)
> * * Select16 * * \ CS9 (A22)
> * * Select16 * * \ CS8 (A21)
> * * Select16 * * \ CS7 (A20)
> * * Select16 * * \ CS6 (A19)
>
> 0 *Select16 * * \ CS5 (FC2)
> * * Select16 * * \ CS4 (FC1)
> * * Select16 * * \ CS3 (FC0)
> * * Select16 * * \ CS2 (BGACK)
> * * Select16 * * \ CS1 (BG)
> * * Select16 * * \ CS0 (BR)
> * * Select16 * * \ CSBOOT
>
> * * ( CSPAR0) W, *( CSPAR1) W,
>
> * * Async *Both * Read * AS * 9 WS *S/U *'PROM * * * * * 128K * *\ CSBOOT
> * * Async *Both * Read * AS * 9 WS *S/U *'PROM 20000 + * 128K * *\ CS0
> * * Async *Both * R/W * *AS * 1 WS *S/U * 0 * * ** * * *128K * *\ CS1
> * * Async *Both * R/W * *AS * 1 WS *S/U * 20000 * ** * *128K * *\ CS2
> * * N/C * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ CS3
> * * N/C * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ CS4
> * * N/C * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ CS5
> * * N/C * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ CS6
> * * N/C * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ CS7
> * * N/C * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ CS8
> * * N/C * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ CS9
> * * N/C * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ CS10
>
> HERE CHIPS - EQU |CHIPS| * IDATA

Interesting idea. I can't read the code (probably because I don't
have that much experience in embedded programming) but here's a
question I want to know: Were the words for defining the bit patterns
defined just before this code, or as part of the support code for the
entire app?

> We don't use wordlists very extensively in applications, but they are
> absolutely vital in our systems, especially cross- and meta- compilers.
> * They're also critical to SwiftForth's SWOOP facility.

I know you said you don't use them often in apps but how have you used
them?

> Not sure just what you mean, here.

It's from something I read by Dr. Ting. I guess basically it means,
taking advantage of the data stack to improve readability simply.

> See Thinking Forth and Forth Application Techniques. *We use this
> capability extensively in applications, most of which have app-specific
> data structures of some kind. *Examples range from circular buffers to
> named bits in an I/O register. *Automatically indexing arrays are also
> very useful.

What forms have these arrays taken? Are they usually a cell wide, or
have the been other data-widths? Have you made use of 2D arrays? Do
you use the [] suffix? What has/have been the most readable form(s),
if not that?

What have circular buffers been useful for?

> We don't actually worry a lot about name collisions in applications. *A
> lot of application words are only used locally, and if there's a
> different one somewhere the compiler will issue a warning. *Sometimes we
> just examine the other version, conclude there's no practical conflict,
> and forge ahead.

Ah, OK, so you don't try to make as many words as possible "global"?
I got the sense from Chuck Moore that that was a good approach. But
somehow it seemed a bit extreme too, since the context isn't always
clear in his writings or interviews, so I've been confused over it.

> We used to have an extensive database package in polyFORTH. *We have
> made a modified version to use for flash databases, which we'll probably
> include in SwiftX at some point.

Have you ever done apps using the OS's filesystem extensively?

Keeping on the database topic, what good are blocks, really? Are they
at all applicable to realtime applications? Is it ever useful to
access blocks only to copy the data to larger static buffers?


> You mean like custom structure words? *They're discussed here far more
> than they're used in applications, in my experience.

That's surprising. I implementing a fairly complex set of structures
for my library to implement a decentralized external object management
system. But now I'm thinking maybe that's not such a good idea
because it implies some possible complexities in the future when you
want to do things differently and the way you load resources (like
images, sounds) is locked into these words.

I also meant extensions to the compiler like IMMEDIATE words that look-
ahead to the next word to implement a special syntax or other
extensions to Forth as a language. For instance I taught SwiftForth
how to interpret fixed-point literals. (I plan on contributing it but
I'm not going to submit it yet, the wordset is incomplete and depends
on a C library for some words)

> Well, that's what an application is, isn't it? *The boundary between
> "system" and "application" is pretty fuzzy in the Forth world. Every
> time we write an application we end up with a system adapted to support
> that application domain. *Makes it really easy to do similar projects.
> For example, GE Multilin in Vancouver has a whole line of multiplexers
> programmed in Forth, with a lot of shared code (seehttp://www.forth.com/resources/appNotes/app-GE-Multilin.html).

That sounds wonderful. But how is it done? Is the application built
in two parts, a library and "the application"? Where is the
distinction between the re-used portion and the app-specific portion
made, if it all? Or do you not worry about that until the first app
is done and you come across a new but similar problem and then pick
the old app apart for reusable stuff? Or do you make a copy of the
old app and modify it until it suits the new problem?

"roger.levy@gmail.com" <roger.levy@gmail.com> writes:

> What have circular buffers been useful for?

They are very useful, since circular buffer is the simplest
data structure, that provides lock free consistent access.



--
HE CE3OH...

roger.levy@gmail.com wrote:
>> It's a good question. It would be helpful to hear what you *have* read,
>> so we know where you're starting from. Have you read "Thinking Forth"
>> (Google for it)? Although it's somewhat dated, it does address some of
>> these issues.
>
> I've read and own both Starting Forth and Thinking Forth. I treat
> them like bibles and re-read Thinking Forth for fun/encouragement
> often. Though the knowledge in them is starting to get a little
> stale ...

Starting Forth is very stale, particularly the paper copies (the online
versions have been partially updated). Since you say you have
SwiftForth, you have a copy of Forth Programmer's Handbook (pdf). You
might benefit from Forth Application Techniques, and it's pretty cheap.

>> Most of our application work nowadays is in embedded systems, which is
>> reflected in my comments below.
>
> It's frustrating; I know that embedded is where Forth thrives today.
> But as I want pretty badly to become comfortable in Forth to build my
> own apps on Windows, I puzzle without knowing for sure what works and
> what doesn't and often I feel like I'm needlessly re-playing history,
> and one that I don't quite understand. I'd like to see more source
> code from serious applications, even if they're old, so long as they
> competed with the other apps from their time. Ones written in "good"
> Forth style, not some of the sources I've seen that look just like C
> code with lots of nesting, long definitions, each line with a comment
> basically repeating the statement...

There's a fair amount of sample code with SwiftForth. Although they
aren't large real-world apps, they do illustrate useful approaches. And
I highly recommend the SwiftForth email group -- a number of the folks
there have very complex apps, and they can certainly share Windows
experiences and code. They don't chatter a lot, but are always willing
to respond to questions, like "how would you go about doing <this>?"

>> One of my favorite good examples is the construction of a chip-select
>> table for a 68332. Single bits here and there in the cells of the table
>> control critical functions, and they make little sense examined in hex.
>> We made a set of words that set the appropriate bits at compile time
>> in ways that are absolutely obvious:
>
>
>> { ---------------------------------------------------------------------
>> Chip select table
>>
>> CHIPS is the chip select table for the NMIX-0332.
>> |CHIPS| is the table size in bytes.
>> --------------------------------------------------------------------- }
>>
>> CREATE CHIPS
>> 0 Select16 \ CS10 (A23)
>> Select16 \ CS9 (A22)
>> Select16 \ CS8 (A21)
>> Select16 \ CS7 (A20)
>> Select16 \ CS6 (A19)
>>
>> 0 Select16 \ CS5 (FC2)
>> Select16 \ CS4 (FC1)
>> Select16 \ CS3 (FC0)
>> Select16 \ CS2 (BGACK)
>> Select16 \ CS1 (BG)
>> Select16 \ CS0 (BR)
>> Select16 \ CSBOOT
>>
>> ( CSPAR0) W, ( CSPAR1) W,
>>
>> Async Both Read AS 9 WS S/U 'PROM 128K \ CSBOOT
>> Async Both Read AS 9 WS S/U 'PROM 20000 + 128K \ CS0
>> Async Both R/W AS 1 WS S/U 0 128K \ CS1
>> Async Both R/W AS 1 WS S/U 20000 128K \ CS2
>> N/C \ CS3
>> N/C \ CS4
>> N/C \ CS5
>> N/C \ CS6
>> N/C \ CS7
>> N/C \ CS8
>> N/C \ CS9
>> N/C \ CS10
>>
>> HERE CHIPS - EQU |CHIPS| IDATA
>
> Interesting idea. I can't read the code (probably because I don't
> have that much experience in embedded programming) but here's a
> question I want to know: Were the words for defining the bit patterns
> defined just before this code, or as part of the support code for the
> entire app?

They are support code for configuring a 68332 for any application. Each
of these words is executed interpretively to set a bit in the
chip-select table to configure the chip. I've seen similar strategies
used to initialize other kinds of tables, but this example was easy to find.

>> We don't use wordlists very extensively in applications, but they are
>> absolutely vital in our systems, especially cross- and meta- compilers.
>> They're also critical to SwiftForth's SWOOP facility.
>
> I know you said you don't use them often in apps but how have you used
> them?

In applications where the user interface involves typing Forth words,
it's sometimes desirable to have a closed wordlist of user words, so
users can't/won't inadvertently type words that could cause trouble.
Those user-level words can have a lot of checks built in.

>> Not sure just what you mean, here.
>
> It's from something I read by Dr. Ting. I guess basically it means,
> taking advantage of the data stack to improve readability simply.

("Featuring the data stack") Well, if I had a word that recorded a data
sample, I'd call it SAMPLE. To take <n> samples, I'd define something like:

: SAMPLES ( n -- ) 0 DO SAMPLE LOOP ;

....because if elsewhere in the program you see

.... 100 SAMPLES ...

it will be pretty obvious what's going on. The relationship between a
single and plural noun plus the obvious use of the stack aids readability.

>> See Thinking Forth and Forth Application Techniques. We use this
>> capability extensively in applications, most of which have app-specific
>> data structures of some kind. Examples range from circular buffers to
>> named bits in an I/O register. Automatically indexing arrays are also
>> very useful.
>
> What forms have these arrays taken? Are they usually a cell wide, or
> have the been other data-widths? Have you made use of 2D arrays? Do
> you use the [] suffix? What has/have been the most readable form(s),
> if not that?

It totally depends on the data. The thing is, that you can define data
structures to accommodate the natural "shape" of the data, rather than
having to mush the data into a predefined structure. Arrays can have
byte elements, string elements, cell elements, double-cell elements,
floating point elements, whatever your app calls for. They can be 1D,
2D, 3D, whatever fits its inherent structure. I'm unfamiliar with a []
suffix.

> What have circular buffers been useful for?

Two very different examples:

a) polynomial coefficients: each time you invoke the name of the array
you get the next one, and it cycles automatically.

b) a data buffer, into which one task or an interrupt routine is putting
data, and another task is taking it out for further processing. Two
pointers, one where to put the next sample, one where to fetch the next.
Obviously the sample rate has to be slow enough (or the processing
task fast enough) that they don't wrap inauspiciously.

There are many more, but these two come to mind.

>> We don't actually worry a lot about name collisions in applications. A
>> lot of application words are only used locally, and if there's a
>> different one somewhere the compiler will issue a warning. Sometimes we
>> just examine the other version, conclude there's no practical conflict,
>> and forge ahead.
>
> Ah, OK, so you don't try to make as many words as possible "global"?
> I got the sense from Chuck Moore that that was a good approach. But
> somehow it seemed a bit extreme too, since the context isn't always
> clear in his writings or interviews, so I've been confused over it.

They may be technically global, but only interesting within a particular
section of the code. I see no reason to be paranoid about global names
(or variables).

>> We used to have an extensive database package in polyFORTH. We have
>> made a modified version to use for flash databases, which we'll probably
>> include in SwiftX at some point.
>
> Have you ever done apps using the OS's filesystem extensively?

Oh, yes. This is one of the areas where SF and TF are particularly
deficient, because they only considered completely self-contained
Forths. There are examples of using Windows files in Application
Techniques.

> Keeping on the database topic, what good are blocks, really? Are they
> at all applicable to realtime applications? Is it ever useful to
> access blocks only to copy the data to larger static buffers?

Blocks were invented for purely native Forths (no host OS), where they
were far faster and more reliable than OS-style files. They were
retained in ANS Forth for compatibility with native Forths. However,
nowadays, OS-hosted Forths are far more common, and blocks almost never
used (although we embedded systems types use a similar concept for
accessing large banks of flash).

>> You mean like custom structure words? They're discussed here far more
>> than they're used in applications, in my experience.
>
> That's surprising. I implementing a fairly complex set of structures
> for my library to implement a decentralized external object management
> system. But now I'm thinking maybe that's not such a good idea
> because it implies some possible complexities in the future when you
> want to do things differently and the way you load resources (like
> images, sounds) is locked into these words.

You're talking about data structures, here, aren't you? I was talking
about things like loops and conditionals. The standard Forth loops and
conditionals are perfectly adequate for 99% of applications, IMO. Data
structures, as I said above, should adapt to the natural shape and
structure of the data itself.

> I also meant extensions to the compiler like IMMEDIATE words that look-
> ahead to the next word to implement a special syntax or other
> extensions to Forth as a language. For instance I taught SwiftForth
> how to interpret fixed-point literals. (I plan on contributing it but
> I'm not going to submit it yet, the wordset is incomplete and depends
> on a C library for some words)

That should be very easy to do without involving C libraries.

>> Well, that's what an application is, isn't it? The boundary between
>> "system" and "application" is pretty fuzzy in the Forth world. Every
>> time we write an application we end up with a system adapted to support
>> that application domain. Makes it really easy to do similar projects.
>> For example, GE Multilin in Vancouver has a whole line of multiplexers
>> programmed in Forth, with a lot of shared code (see
>> http://www.forth.com/resources/appNo...-Multilin.html).
>
> That sounds wonderful. But how is it done? Is the application built
> in two parts, a library and "the application"? Where is the
> distinction between the re-used portion and the app-specific portion
> made, if it all? Or do you not worry about that until the first app
> is done and you come across a new but similar problem and then pick
> the old app apart for reusable stuff? Or do you make a copy of the
> old app and modify it until it suits the new problem?

Well, the system itself is organized into a number of files, each of
which is a well-defined set of related functions. There's a file that
loads the ones you need. Similarly, your app will consist of a number
of files of closely-related words, and a master file that loads the ones
you need in the right order. You try, first time around, to layer these
files with generic support functions separate from higher-level
processing, user interface, etc. Your next similar app can probably use
a lot of the same files, but will require some new ones, and some
modified versions of the old ones. It should have its own master load
file, which picks the ones appropriate for this app. Over time, you
build up your store of useful files, and structure them better (to
segregate domain-generic code from application-specific code). You may
find you need to modify some system functions; so you may have a
different system load-file to select your modified ones.

Some people set up complex sets of flags and conditional compilation
statements to manage this, but I don't like that approach. Much easier
to have a master file for each app (or variant) and use the right one.

Hope this helps.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

On Aug 29, 4:41 am, Elizabeth D Rather <erat...@forth.com> wrote:
> roger.l...@gmail.com wrote:
> >> It's a good question. It would be helpful to hear what you *have* read,
> >> so we know where you're starting from. Have you read "Thinking Forth"
> >> (Google for it)? Although it's somewhat dated, it does address some of
> >> these issues.
>
> > I've read and own both Starting Forth and Thinking Forth. I treat
> > them like bibles and re-read Thinking Forth for fun/encouragement
> > often. Though the knowledge in them is starting to get a little
> > stale ...
>
> Starting Forth is very stale, particularly the paper copies (the online
> versions have been partially updated). Since you say you have
> SwiftForth, you have a copy of Forth Programmer's Handbook (pdf). You
> might benefit from Forth Application Techniques, and it's pretty cheap.
>
> >> Most of our application work nowadays is in embedded systems, which is
> >> reflected in my comments below.
>
> > It's frustrating; I know that embedded is where Forth thrives today.
> > But as I want pretty badly to become comfortable in Forth to build my
> > own apps on Windows, I puzzle without knowing for sure what works and
> > what doesn't and often I feel like I'm needlessly re-playing history,
> > and one that I don't quite understand. I'd like to see more source
> > code from serious applications, even if they're old, so long as they
> > competed with the other apps from their time. Ones written in "good"
> > Forth style, not some of the sources I've seen that look just like C
> > code with lots of nesting, long definitions, each line with a comment
> > basically repeating the statement...
>
> There's a fair amount of sample code with SwiftForth. Although they
> aren't large real-world apps, they do illustrate useful approaches. And
> I highly recommend the SwiftForth email group -- a number of the folks
> there have very complex apps, and they can certainly share Windows
> experiences and code. They don't chatter a lot, but are always willing
> to respond to questions, like "how would you go about doing <this>?"
>
>
>
> >> One of my favorite good examples is the construction of a chip-select
> >> table for a 68332. Single bits here and there in the cells of the table
> >> control critical functions, and they make little sense examined in hex.
> >> We made a set of words that set the appropriate bits at compile time
> >> in ways that are absolutely obvious:
>
> >> { ---------------------------------------------------------------------
> >> Chip select table
>
> >> CHIPS is the chip select table for the NMIX-0332.
> >> |CHIPS| is the table size in bytes.
> >> --------------------------------------------------------------------- }
>
> >> CREATE CHIPS
> >> 0 Select16 \ CS10 (A23)
> >> Select16 \ CS9 (A22)
> >> Select16 \ CS8 (A21)
> >> Select16 \ CS7 (A20)
> >> Select16 \ CS6 (A19)
>
> >> 0 Select16 \ CS5 (FC2)
> >> Select16 \ CS4 (FC1)
> >> Select16 \ CS3 (FC0)
> >> Select16 \ CS2 (BGACK)
> >> Select16 \ CS1 (BG)
> >> Select16 \ CS0 (BR)
> >> Select16 \ CSBOOT
>
> >> ( CSPAR0) W, ( CSPAR1) W,
>
> >> Async Both Read AS 9 WS S/U 'PROM 128K \ CSBOOT
> >> Async Both Read AS 9 WS S/U 'PROM 20000 + 128K \ CS0
> >> Async Both R/W AS 1 WS S/U 0 128K \ CS1
> >> Async Both R/W AS 1 WS S/U 20000 128K \ CS2
> >> N/C \ CS3
> >> N/C \ CS4
> >> N/C \ CS5
> >> N/C \ CS6
> >> N/C \ CS7
> >> N/C \ CS8
> >> N/C \ CS9
> >> N/C \ CS10
>
> >> HERE CHIPS - EQU |CHIPS| IDATA
>
> > Interesting idea. I can't read the code (probably because I don't
> > have that much experience in embedded programming) but here's a
> > question I want to know: Were the words for defining the bit patterns
> > defined just before this code, or as part of the support code for the
> > entire app?
>
> They are support code for configuring a 68332 for any application. Each
> of these words is executed interpretively to set a bit in the
> chip-select table to configure the chip. I've seen similar strategies
> used to initialize other kinds of tables, but this example was easy to find.
>
> >> We don't use wordlists very extensively in applications, but they are
> >> absolutely vital in our systems, especially cross- and meta- compilers.
> >> They're also critical to SwiftForth's SWOOP facility.
>
> > I know you said you don't use them often in apps but how have you used
> > them?
>
> In applications where the user interface involves typing Forth words,
> it's sometimes desirable to have a closed wordlist of user words, so
> users can't/won't inadvertently type words that could cause trouble.
> Those user-level words can have a lot of checks built in.
>
> >> Not sure just what you mean, here.
>
> > It's from something I read by Dr. Ting. I guess basically it means,
> > taking advantage of the data stack to improve readability simply.
>
> ("Featuring the data stack") Well, if I had a word that recorded a data
> sample, I'd call it SAMPLE. To take <n> samples, I'd define something like:
>
> : SAMPLES ( n -- ) 0 DO SAMPLE LOOP ;
>
> ...because if elsewhere in the program you see
>
> ... 100 SAMPLES ...
>
> it will be pretty obvious what's going on. The relationship between a
> single and plural noun plus the obvious use of the stack aids readability.
>
> >> See Thinking Forth and Forth Application Techniques. We use this
> >> capability extensively in applications, most of which have app-specific
> >> data structures of some kind. Examples range from circular buffers to
> >> named bits in an I/O register. Automatically indexing arrays are also
> >> very useful.
>
> > What forms have these arrays taken? Are they usually a cell wide, or
> > have the been other data-widths? Have you made use of 2D arrays? Do
> > you use the [] suffix? What has/have been the most readable form(s),
> > if not that?
>
> It totally depends on the data. The thing is, that you can define data
> structures to accommodate the natural "shape" of the data, rather than
> having to mush the data into a predefined structure. Arrays can have
> byte elements, string elements, cell elements, double-cell elements,
> floating point elements, whatever your app calls for. They can be 1D,
> 2D, 3D, whatever fits its inherent structure. I'm unfamiliar with a []
> suffix.
>
> > What have circular buffers been useful for?
>
> Two very different examples:
>
> a) polynomial coefficients: each time you invoke the name of the array
> you get the next one, and it cycles automatically.
>
> b) a data buffer, into which one task or an interrupt routine is putting
> data, and another task is taking it out for further processing. Two
> pointers, one where to put the next sample, one where to fetch the next.
> Obviously the sample rate has to be slow enough (or the processing
> task fast enough) that they don't wrap inauspiciously.
>
> There are many more, but these two come to mind.
>
> >> We don't actually worry a lot about name collisions in applications. A
> >> lot of application words are only used locally, and if there's a
> >> different one somewhere the compiler will issue a warning. Sometimes we
> >> just examine the other version, conclude there's no practical conflict,
> >> and forge ahead.
>
> > Ah, OK, so you don't try to make as many words as possible "global"?
> They may be technically global, but only interesting within a particular
> section of the code. I see no reason to be paranoid about global names
> (or variables).

Not paranoid, simply wary.


> >> We used to have an extensive database package in polyFORTH. We have
> >> made a modified version to use for flash databases, which we'll probably
> >> include in SwiftX at some point.
What are these flash databases used for?

> > I implementing a fairly complex set of structures
> > for my library to implement a decentralized external object management
> > system. ...
> You're talking about data structures, here, aren't you?

Yes, but you're right, I guess what I was talking about wasn't a
compiler extension but data structures. It was a pretty open-ended
question; part of it is that I don't know what "extending the
compiler" really meant. Thanks for the hints.

> Well, the system itself is organized into a number of files, ....

Ok, just a few more questions about this. Is everything dumped into
the same folder (if I get what you said)? Or are you making copies of
old apps in new folders? What happens if a file from an old app has
just a slight bug that would require a change to the old app? And do
you use version numbering at the support-file level? I'm trying to
get a metric for the need to implement standards on the application
level as distinguished from the compiler level.

I may grab FAT. If it is really useful I'll recommend it.

roger.levy@gmail.com wrote:

...

> What have circular buffers been useful for?

...

A circular buffer is the software analog of a hardware FIFO. It is so
useful for some operations that DSPs have special hardware resources to
facilitate their operation.

Jerry
--
Engineering is the art of making what you want from things you can get.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

roger.levy@gmail.com wrote:
> On Aug 29, 4:41 am, Elizabeth D Rather <erat...@forth.com> wrote:
....
>>>> We don't actually worry a lot about name collisions in applications. A
>>>> lot of application words are only used locally, and if there's a
>>>> different one somewhere the compiler will issue a warning. Sometimes we
>>>> just examine the other version, conclude there's no practical conflict,
>>>> and forge ahead.
>>> Ah, OK, so you don't try to make as many words as possible "global"?
>> They may be technically global, but only interesting within a particular
>> section of the code. I see no reason to be paranoid about global names
>> (or variables).
>
> Not paranoid, simply wary.

Fortunately, any decent Forth will warn you of name collisions, so you
can use your judgement as to whether a particular one is a problem or not.

>>>> We used to have an extensive database package in polyFORTH. We have
>>>> made a modified version to use for flash databases, which we'll probably
>>>> include in SwiftX at some point.
> What are these flash databases used for?

Program and data storage, same as you'd use a disk for if you had one.
Most embedded systems nowadays do not have rotating media due to size,
power consumption, and reliability issues.

>>> I implementing a fairly complex set of structures
>>> for my library to implement a decentralized external object management
>>> system. ...
>> You're talking about data structures, here, aren't you?
>
> Yes, but you're right, I guess what I was talking about wasn't a
> compiler extension but data structures. It was a pretty open-ended
> question; part of it is that I don't know what "extending the
> compiler" really meant. Thanks for the hints.

Strictly speaking, a Forth "compiler" is what process the stuff between
a colon and a semicolon. Data structures are constructed by the text
interpreter. So, "extending the compiler" generally means adding
flow-of-control structures and similar things. Your custom literal
would also qualify.

>> Well, the system itself is organized into a number of files, ....
>
> Ok, just a few more questions about this. Is everything dumped into
> the same folder (if I get what you said)? Or are you making copies of
> old apps in new folders? What happens if a file from an old app has
> just a slight bug that would require a change to the old app? And do
> you use version numbering at the support-file level? I'm trying to
> get a metric for the need to implement standards on the application
> level as distinguished from the compiler level.

It really depends on the complexity of the application. My personal
preference is to have a directory for the "family" of related
applications. Files used by all variants without change are in it, as
are sub-directories for each variant. System files that are used
without change remain in the system directory where they are; any system
files that are modified for your application family or variants go in
the appropriate directory or subdirectory.

In your first application, all of your application files would go in the
same directory, unless it became so large and complex that you'd need
subdirectories for sets of related files. Then, as you add variants in
the future, the distinction between files shared between variants and
files customized or added in variant-specific ways would grow.

Look at the directory structure of SwiftForth for an example.

> I may grab FAT. If it is really useful I'll recommend it.

Good, let us know! The beginning chapters are very basic, you're
probably well beyond that. But it later moves into more detail on data
management, files, multitasking, and other more advanced topics.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================