Fundamental Problems of Lisp

Don Geddis <don@geddis.org> writes:

> "xahlee@gmail.com" <xahlee@gmail.com> wrote on Tue, 26 Aug 2008:
>> For example, semantics gave us: (cons 1 (cons 2 (cons 3 nil))), and
>> consequently all the problems of the cons business. If syntax has been
>> clearly thought of 30 years ago, it'd be {1,2,3} or (1,2,3) or [1,2,3], and
>> there wouldn't be the cons business.
>
> What about
> '(1 2 3)
> That seems to be a syntax just as simple as your three "clearly thought"
> examples.
>
> Perhaps Lisp syntax was "clearly thought of 30 years ago" after all?

Don, please stop.

1- it wasn't 30 years ago, it was 50 years ago.

2- the syntax for lisp code wasn't (1,2,3), it was list[1;2;3]
(and it was defined, not as CONSes,
but as combine[1;combine[2;combine[3,NIL]]]).

3- the syntax for lisp data was (1,2,3). So:

list[1;2;3] --> (1,2,3)

Which means that we don't write lisp CODE since ever LISP was
implemented 50 years ago, we feed it only lisp DATA, and it just
interprets it. Lisp code was only written in scientific papers.

--
__Pascal Bourguignon__ http://www.informatimago.com/

In a World without Walls and Fences,
who needs Windows and Gates?

2008-08-26

First of all, if you don't have a master degree in computer science,
and, if your specialization is not in computer languages design (i.e.
logics, mathematical formalism, some elements of computational
linguistics), then, you probably won't understand this post. However,
if you are bright, and have a open mind, then the views expressed post
can be meaningful or beneficial to you.

To begin, let me make it explicit that this post is about the concept
of syntax and semantics in computer languages. For example, are
computer languages defined by syntax and semantics? Does syntax and
semantics comprise the whole of a computer language? What is a
computer language? What is syntax, what is semantics? etc.

First, lets note few facts:

• there is no formal and universally accepted definition of what is
meant by “semantics�. There may be some, if so defined, they are
typically based on “models� over idealized, abstract hardware (i.e.
operations on values in storage; as in finite state machine).

• semantics is tied to syntax. There cannot be semantics without
syntax. We could conceptualize a semantic without any syntax, but this
is basically not done.

• a computer language, is typically mathematically defined as a setof
operations on a set of chars. This typical definition, covers
effectively just the syntax. (See http://en.wikipedia.org/wiki/Formal_language)

in general, the study of design of computer languages as languages for
practical use, does not have much formal background. Typically, some
computer scientists (the real ones are properly known as
Mathematicians), create a lang that is LOSELY based on their idea of
some mathematical theory (such as lambda calculus, or other sub
branches of logic.) Lisp, prolog, haskell, apl are such examples. For
vast majority of langs, such as pascal, logo, C, C++, Java, perl, awk,
sh, python, php, ruby, tcl, basic, fortran,... etc, basically their
design, creation, are far removed from any mathematics. It is pretty
much a so-called “art�. Any dummy, can conceive and create a computer
language from scratch based on basically just his personal experiences
in using lang and intuition. This does not mean the lang will be
“bad�. In fact, most successful or widely used langs, results from
such personal creativity.

One of the question we want to ask is, what is a computer language? As
we know, formal methods, such as those based on automata, has nothing
to do with actual langs and is basically just the study of syntax. A
formal languages, of course does also have semantics, but that
semantics basically means transformation of strings. This sense is far
different from what mean by “semantics� in pop books that deal with
real langs (such as the the text book “Programming Languages:
Application and Interpretation�", by Shriram Krishnamurth).

The other question we want to ask, related to the above, is “what
exactly is a computer language?�. There is a popular, unconcious
belief, that a computer language is made up of syntax and semantics.
However, as we know from above, there are no formal definition on what
exactly is semantics that can be applied to computer languages in
daily use.

So, what is then, a computer language? Is there a formal definition
that makes Java, C, perl, lisp, as computer languages? The answer is
just no in our current state of mathematics.

Why is syntax far more important than whatever we intuitively
understood as “semantics�? Because, it is syntax that we actually see
and use in a computer lang, and it is syntax that drives the
development of semantics. When you design a language, you start with
syntax, coupled with some vague idea of what u want to happen, then,
once you wrote a compiler, then is semantics born.

another reason syntax is far more important can be seen from the
history of math notations. Tech geekers might think, notation doesn't
mean much except some convenience aspects. But if you are acquainted
in history of math notation, you'll know that notation largely drove
many development and directions of mathematics. Examples include
arabic numeral with decimal positional notation, symbols for variables
→ abstract algebra, matrix notation → linear algebra, some notations →
calculus. (notations, is somewhat like terminology, in that sometimes
the existance of a notation/terminology have huge impact from
awareness of the concept to major influence on thought of a subject.)

Xah
∑ http://xahlee.org/

☄


On Aug 25, 1:58Â*pm, "xah...@gmail.com" <xah...@gmail.com> wrote:
> On Aug 25, 9:01 am, Ali <emailalicl...@gmail.com> wrote:
>
> > On Aug 25, 3:23 pm, "xah...@gmail.com" <xah...@gmail.com> wrote:
>
> > > On Aug 25, 6:09 am, Ali <emailalicl...@gmail.com> wrote:
> > > > c) Create a new lisp, but without these problems, and follow b)?
>
> > > Yes.
>
> > And one (hopefully last) question. Have you considered writing a lisp,
> > or other language, yourself without these irregularities?
>
> No. My knowledge and experience of parsers and compilers is basically
> zero. Say, below any studious person who just graduated from college
> with a computer science degree.
>
> I have some interest in parsers, so this year i started to read and
> learn tools about parsers. In particular, my interest in parsers is
> writing tools for text processing, in particular processing nestedsyntaxsuch as lisp and xml. I have no interest in compilers though.
>
> With the profileration of tools today... i probably can hackup a
> interpreter for some new language in say, perl. Even that will take me
> perhaps half a year or more, and when done, it'd be basically a
> useless toy.
>
> as for language'ssyntax, i do not find nestedsyntaxideal. I have
> been programing elisp as hobby on and off for 2 years. For all my love
> of uniformality, AI, functional programing, honestly i find the nestedsyntaxproblematic. (not counting lisp irregularity since the few lispsyntaxirregularity doesn't matter that much in practice).
>
> (i find nestedsyntaxproblematic for practical reasons. Namely,
> reading the source code and typing it. The other major reason that
> sting me as i start to write more complex programs, is that sequencing
> functions forces me into so much nesting (i.e. several nested mapcar).
> I have some detail in this article:
> “The Concepts and Confusions of Prefix, Infix, Postfix and Fully
> Nested Notations�http://xahlee.org/UnixResource_dir/writ/notations.html
> )
>
> Recently i've been thinking, a idealsyntaxthat completely ditchs any
> form of nesting insyntax. Â*That is, the source code is just a bunch
> of lines, each line is just a sequence of functions. For example,
> Mathematica's prefix notation “f@g@x� or postfix notation“x//g//f�,
> or unix pipe (e.g. “x | g | f�), and APL'ssyntaxis similar idea.
>
> The problem with such asyntaxis that it limits functions to one arg
> only. A solution might be that for any function that needs 2 or more
> args, the lang makes everything as list. But then, this means simple
> things like “1+2*3� becomes something like polish notation “[2 3] * 1
> +�, which is quite unreadable and unatural. APL solves it by allowing
> both binary and uninary functions ... but then i don't think that'd be
> a perfectsyntax.
>
> another train of thought i had recently, is that, i think all langs
> should have lisp's nestedsyntax(with no irregularities), then
> another layer on top of it, whatever it may be. This is how
> Mathematica'ssyntaxis, and i presume also liskell. Since all textual
> langs has a abstractsyntaxtree, and the nestedsyntaxis isomorphic
> to trees, so theoretically all langs could have a nestedsyntaxlike
> lisp's. I'm not sure how difficult this actually is to do, or how
> difficult for a existing lang to add such a layer. I guess its rather
> trivial in the context of compiler science, but i don't know much
> about parsers/compilers to be sure.
>
> if all lang support the pure nestedsyntax, then it will have a major
> impact on cross-lang understanding, usher practical major progress on
> programer's understanding of the role ofsyntaxin langs, and have
> major impact on language translation (aka compiling). (after all,
> theoretically, proper parser/compiler creates a AST in the parsing
> stage anyway. A pure nestedsyntaxcould be made as a explicit stage.
> That is, sugarsyntax→ nestedsyntax→ AST)
>
> btw, i have a question... are those AST created in the parsing process
> as some kinda explicit datatype? Wouldn't it be easy to print them out
> like lisp's nested paren?
>
> it is my wild guess, that the idea all or almost all languages will
> have a nestedsyntaxlike lisp (as a layer), might actually be a
> reality say in maybe 20 or 50 years.
>
> Â* Â* Â* Â* Â* Â* Â* * Â* Â* Â* Â* Â* Â* Â** Â* Â* Â* Â* Â* Â* Â**
>
> as for interests, i have much more interest in perfecting my expertise
> in elisp, and learning Haskell, and learning a theorem proving lang
> such as coq to enhance my understanding in mathematical logic.
>
> I also have far more interest in many math subjects and geometry
> programing subjects. I have much existing projects to be done in
> geometry of plane curves, tiling theory, algorithmic mathematical art,
> surface theory, etc. (you can find these on my website)
>
> my interest in computational mathematics (computational logic,
> programing geometry) outshine my interest in programing langs or
> typical computer science issues. (am expert in programing geometry; am
> newbie in computational logic)
>
> Â*Xah
> ∑http://xahlee.org/
>
> ☄

"xahlee@gmail.com" <xahlee@gmail.com> writes:
> To begin, let me make it explicit that this post is about the concept
> of syntax and semantics in computer languages. For example, are
> computer languages defined by syntax and semantics? Does syntax and
> semantics comprise the whole of a computer language? What is a
> computer language? What is syntax, what is semantics? etc.
>
> First, lets note few facts:
>
> • there is no formal and universally accepted definition of what is
> meant by “semantics�. There may be some, if so defined, they are
> typically based on “models� over idealized, abstract hardware (i.e.
> operations on values in storage; as in finite state machine).

I think that something along the lines of "the results of evaluating an
expression or statement written in the language" would be a suitable
working definition. No, I don't have a citation for that. But in my
experience, people don't tend to be too confused as to what "semantics"
means.

> • semantics is tied to syntax. There cannot be semantics without
> syntax. We could conceptualize a semantic without any syntax, but this
> is basically not done.

I think you have this backwards. Syntax is meaningless without
semantics. The same set of semantics can have multiple syntactic
expressions (this was the original intent behind Lisp's [lack of]
syntax, so far as I know). Further, similar syntaxes (sp?) can have
very different semantics. See the plethora of rather diverse languages
with C-based syntax for examples.

Semantics can exist without syntax. It's just difficult to talk about.

> • a computer language, is typically mathematically defined as a set of
> operations on a set of chars. This typical definition, covers
> effectively just the syntax. (See
> http://en.wikipedia.org/wiki/Formal_language)
>
> in general, the study of design of computer languages as languages for
> practical use, does not have much formal background. Typically, some
> computer scientists (the real ones are properly known as
> Mathematicians), create a lang that is LOSELY based on their idea of
> some mathematical theory (such as lambda calculus, or other sub
> branches of logic.) Lisp, prolog, haskell, apl are such examples. For
> vast majority of langs, such as pascal, logo, C, C++, Java, perl, awk,
> sh, python, php, ruby, tcl, basic, fortran,... etc, basically their
> design, creation, are far removed from any mathematics. It is pretty
> much a so-called “art�. Any dummy, can conceive and create a computer
> language from scratch based on basically just his personal experiences
> in using lang and intuition. This does not mean the lang will be
> “bad�. In fact, most successful or widely used langs, results from
> such personal creativity.

Many successful and widely used languages are built from such personal
creativity. Perl and Python come to mind.

There are exceptions, however: Standard ML has a complete mathematical
definition of its semantics for at least the core language (see "The
Definition of Standard ML"). Haskell also has a lot of formalism about
it, but I am not sure how thoroughly its semantics are defined.

> One of the question we want to ask is, what is a computer language? As
> we know, formal methods, such as those based on automata, has nothing
> to do with actual langs and is basically just the study of syntax. A
> formal languages, of course does also have semantics, but that
> semantics basically means transformation of strings.

Patently false. Formal languages, such as definitions of Turing
machines, are typically dealt with without any particular syntax. You
can express them in a variety of ways (an annotated state diagram, a
table, a list of functions), and they are semantically equivalent.

Lambda calculii probably have a more consistent syntax, but the syntax
remains just a representation of the abstract concepts.

> This sense is far different from what mean by “semantics� in pop books
> that deal with real langs (such as the the text book “Programming
> Languages: Application and Interpretation�", by Shriram Krishnamurth).

> The other question we want to ask, related to the above, is “what
> exactly is a computer language?�. There is a popular, unconcious
> belief, that a computer language is made up of syntax and semantics.
> However, as we know from above, there are no formal definition on what
> exactly is semantics that can be applied to computer languages in
> daily use.

The graduate level programming languages course I took in my
undergraduate years had a pretty good definition, I thought: "a language
capable of expressing every computable function."

> So, what is then, a computer language? Is there a formal definition
> that makes Java, C, perl, lisp, as computer languages? The answer is
> just no in our current state of mathematics.

Using the definition above, all that is necessary to show that they are
programming languages is to demonstrate that is possible to reduce a
Turing machine (the definition of computability) to them. That they are
computer languages is self-evident -- they are languages processable by
or controlling computers.

They may not be well-defined languages, but they certainly are
programming languages.

> Why is syntax far more important than whatever we intuitively
> understood as “semantics�? Because, it is syntax that we actually see
> and use in a computer lang, and it is syntax that drives the
> development of semantics. When you design a language, you start with
> syntax, coupled with some vague idea of what u want to happen, then,
> once you wrote a compiler, then is semantics born.

Again, not necessarily true. It may be true in some languages (perhaps
Perl), but not in others. As stated above, Standard ML has a complete
semantic definition. In the Python community, when adding new language
features, there have been times when they new most of what they wanted
semantically but it took some time to nail down what the syntax would
be.

I would argue that development goes in exactly the opposite description
you describe: one first thinks of what one wants to tell the computer to
do, and then figures out how to write it down.

> another reason syntax is far more important can be seen from the
> history of math notations. Tech geekers might think, notation doesn't
> mean much except some convenience aspects. But if you are acquainted
> in history of math notation, you'll know that notation largely drove
> many development and directions of mathematics. Examples include
> arabic numeral with decimal positional notation, symbols for variables
> → abstract algebra, matrix notation → linear algebra, some notations →
> calculus. (notations, is somewhat like terminology, in that sometimes
> the existance of a notation/terminology have huge impact from
> awareness of the concept to major influence on thought of a subject.)

Sure, syntax and notations are nice. But that does not make them more
important than semantics. It simply means that syntax makes it easier
to discuss semantics coherently.

- Michael

--
mouse, n: A device for pointing at the xterm in which you want to type.

"xahlee@gmail.com" <xahlee@gmail.com> wrote on Fri, 29 Aug 2008:
[...]
> To begin, let me make it explicit that this post is about the concept
> of syntax and semantics in computer languages.
[...]

Xah, I disagreed with much of what you wrote in that post. But rather than
comment negatively on the content, I wanted to comment positively on the
form. You didn't gratuitously insult your readers (much...), and you
attempted to be constructive. I appreciate the effort, at least (if perhaps
not so much the result).

-- Don
__________________________________________________ _____________________________
Don Geddis http://don.geddis.org/ don@geddis.org
Computer Science is no more about computers than astronomy is about telescopes.
-- E. W. Dijkstra

On 30 ago, 00:28, Don Geddis <d...@geddis.org> wrote:
> "xah...@gmail.com" <xah...@gmail.com> wrote on Fri, 29 Aug 2008:
>*You didn't gratuitously insult your readers (much...)

Yeah, here's him insulting all the verms below his gorgeous IQ:

"First of all, if you don't have a master degree in computer science,
and, if your specialization is not in computer languages design (i.e.
logics, mathematical formalism, some elements of computational
linguistics), then, you probably won't understand this post."

OTOH, his highness himself seems positively confused and unable to
make a distinction between syntax and semantics, despite being pointed
already more than once to simple examples that dispell all of his
verbose logic based on false premises...

xahlee@gmail.com wrote:


> First, lets note few facts:
>
> • there is no formal and universally accepted definition of what is
> meant by “semantics�.

This statement is false. Formal semantics is mathematics. Figuring
out what a statement is supposed to do is proving a theorem given the
defined axioms of the formal semantics. The semantics of a statement
are a transformation of one program state into another. Usually we
represent the program states as strings, hence the relevance of grammars
for transforming one string into another.

> There may be some, if so defined, they are
> typically based on “models� over idealized, abstract hardware (i.e.
> operations on values in storage; as in finite state machine).

Granted, actual implementations on machines with finite memory tend
to be unable to fully implement the formal semantics of a language,
in much the same way that automated theorem provers on machines with
finite memory are unable to prove some theorems.

> • semantics is tied to syntax. There cannot be semantics without
> syntax. We could conceptualize a semantic without any syntax, but this
> is basically not done.

In current languages, semantics is tied much more closely to the parse
tree of an expression than it is to the syntax, but whatever - there
is a connection, because the parse tree is computed from the grammar
and the syntax.

> One of the question we want to ask is, what is a computer language? As
> we know, formal methods, such as those based on automata, has nothing
> to do with actual langs and is basically just the study of syntax. A
> formal languages, of course does also have semantics, but that
> semantics basically means transformation of strings. This sense is far
> different from what mean by “semantics� in pop books that deal with
> real langs (such as the the text book “Programming Languages:
> Application and Interpretation�", by Shriram Krishnamurth).

Hm. Okay, granted, a grammar transforms strings into strings.
But if the input string is program text and the output string
is machine instructions, we call that grammar a compiler. This
is not how most actual compilers are written, but this is a
complete formal model of what a compiler is and does, and therefore
allows us to prove theorems about what is and isn't possible
for compilers.

Anyway, type-0 grammars are Turing equivalent, which means they
provide a general model of computability and can calculate any
computable function. What this means is that you can implement
the semantics of any computer language as a type-0 grammar that
transforms program source and input directly into output without
ever bothering to create instructions for a specific machine along
the way. You can also implement a grammar that transforms your
program text into another grammar, which will transform input
into output.

Even with the reduced power of a type-2 grammar (the context-free
languages, which includes nearly all modern computer languages)
you can directly solve most math problems using string manipulation
rules. I know, because I implemented exactly that as homework once.
You start by implementing incrementing, decrementing, doubling,
and halving as string transformation operations, introduce
nonterminal symbols that can prepend numbers indicating that one
of these operations is needed, and go from there.

I guess my point is that "transforming strings into strings"
*is* a pretty darn good model of what a compiler does, and a
pretty good model of semantics, and understanding the theory
*is* relevant far beyond dealing with the surface syntax of a
language.




> The other question we want to ask, related to the above, is “what
> exactly is a computer language?�. There is a popular, unconcious

true

> belief, that a computer language is made up of syntax and semantics.

....

> another reason syntax is far more important can be seen from the
> history of math notations. ... if you are acquainted
> in history of math notation, you'll know that notation largely drove
> many development and directions of mathematics. Examples include
> arabic numeral with decimal positional notation, symbols for variables
> → abstract algebra, matrix notation → linear algebra, some notations →
> calculus. (notations, is somewhat like terminology, in that sometimes
> the existance of a notation/terminology have huge impact from
> awareness of the concept to major influence on thought of a subject.)
>

This is definitely true. A good notation makes something easier to
reason about, usually by associating or "bundling" related concepts
in a more efficient or useful way.

Bear

Ray Dillinger <bear@sonic.net> wrote:
+---------------
| Even with the reduced power of a type-2 grammar (the context-free
| languages, which includes nearly all modern computer languages)
| you can directly solve most math problems using string manipulation
| rules. I know, because I implemented exactly that as homework once.
| You start by implementing incrementing, decrementing, doubling,
| and halving as string transformation operations, introduce
| nonterminal symbols that can prepend numbers indicating that one
| of these operations is needed, and go from there.
|
| I guess my point is that "transforming strings into strings"
| *is* a pretty darn good model of what a compiler does, and a
| pretty good model of semantics, and understanding the theory
| *is* relevant far beyond dealing with the surface syntax of a
| language.
+---------------

Indeed. In this regard Kent Dybvig's PhD thesis[1] may be of interest
to some. It's most frequently cited for its contributions to showing
how/when one could use stack-based (versus heap-based) lexical variables
in Scheme, but also included a string-based model. From the abstract:

This dissertation presents three implementation models for the
Scheme Programming Language. The first is a heap-based model used
in some form in most Scheme implementations to date; the second is
a new stack-based model that is considerably more effcient than
the heap-based model at executing most programs; and the third is
a new string-based model intended for use in a multiple-processor
implementation of Scheme. ... The string-based model allocates
versions of these structures right in the program text, which
is represented as a string of symbols. In the string-based model,
Scheme programs are translated into an FFP language designed
specifically to support Scheme. Programs in this language are
directly executed by the FFP machine, a multiple-processor
string-reduction computer.

And from "Chapter 5: The String-Based Model":

The string-based implementation model proposed in this chapter is
intended for the FFP machine... [which] employs string reduction,
evaluating expressions in FFP by reducing the text of the FFP
program piece by piece until each evaluable expression has been
fully reduced. It would be possible to employ the techniques
described herein to design a Scheme implementation for any
processor capable of string reduction.

"Rewrite-rule" or "reduction" computers were a quite active area
of research several decades ago, though not so much these days.


-Rob

[1] http://www.cs.indiana.edu/~dyb/papers/3imp.pdf
"Three Implementation Models for Scheme",
R. Kent Dybvig,
University of North Carolina Computer Science
Technical Report 87-011 [Ph.D. Dissertation], April 1987.

-----
Rob Warnock <rpw3@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607

rpw3@rpw3.org (Rob Warnock) writes:
> "Rewrite-rule" or "reduction" computers were a quite active area
> of research several decades ago, though not so much these days.

I think Haskell is just about always implemented with graph rewriting
(reduction), though specified in lambda calculus. Clean is specified
in terms of graph rewriting which the authors claim has advantages.