Another place to find free Common Lisp libraries

On Aug 25, 6:32*am, Ray Dillinger <b...@sonic.net> wrote:
> Pascal Costanza wrote:
> > ...but that new macro system _is_ unnecessarily complex.
>
> Your mouth to God's ear, friend. *
>
> Wouldn't it work just fine to base it on the CL defmacro form,
> with internal syntax like (with-captures ... ) enclosing code
> specifying when you _don't_ want to use renaming discipline?
> That way you get hygiene by default, avoid inadvertent capture,
> and get variable capture with a very simple additional syntax. *

Butbut... what are we going to write papers on, then?!

M/

Ray Dillinger wrote:
> Pascal Costanza wrote:
>
>> Ray Dillinger wrote:
>
>>> There are things I think that the most recent [scheme] report got wrong.
>>> They made it unnecessarily complex, but variable capture, at least,
>>> is now possible.
>
>> ...but that new macro system _is_ unnecessarily complex.
>
> Your mouth to God's ear, friend.
>
> Wouldn't it work just fine to base it on the CL defmacro form,
> with internal syntax like (with-captures ... ) enclosing code
> specifying when you _don't_ want to use renaming discipline?
> That way you get hygiene by default, avoid inadvertent capture,
> and get variable capture with a very simple additional syntax.

That's syntactic closures. Yes, that would be a fine solution.

I'd prefer to go even more minimalistic. I think Will Clinger's renaming
system is all you need.



Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Pascal Costanza <pc@p-cos.net> writes:

> Ray Dillinger wrote:
>> Pascal Costanza wrote:
>>
>>> Ray Dillinger wrote:
>>
>>>> There are things I think that the most recent [scheme] report got wrong.
>>>> They made it unnecessarily complex, but variable capture, at least,
>>>> is now possible.
>>
>>> ...but that new macro system _is_ unnecessarily complex.
>> Your mouth to God's ear, friend.
>>
>> Wouldn't it work just fine to base it on the CL defmacro form, with internal
>> syntax like (with-captures ... ) enclosing code specifying when you _don't_
>> want to use renaming discipline? That way you get hygiene by default, avoid
>> inadvertent capture, and get variable capture with a very simple additional
>> syntax.
>
> That's syntactic closures. Yes, that would be a fine solution.
>
> I'd prefer to go even more minimalistic. I think Will Clinger's renaming
> system is all you need.

Hmm, what exactly is "necessary" and what current macro systems provide it?

Apart from expressivness I think a good macro system would need the following
4 things (roughly in reverse order of importance):

1. ease of use (for writing macros))
2. robustness
3. plays well with interactive development
4. easy of use (for using macros viz. debugging support)

Is there any language/implementation that ticks all these boxes?

In my experience the core defect of CL's defmacro is not the often touted lack
of hygiene (see 2.) but extremely poor debuggability which IMO really
compromises CL's otherwise very good support for interactive development and
DSL creation.

I'm not sure to what extent poor debuggability is a quality of implementation
issue, but on the face of it the task seems much, much harder for procedural,
plain sexp-based macros than for e.g. define-syntax style macros (I can't
remember what the situation for syntax-case looked like, but I'm sure I
witnessed error reporting for exceptions originating in define-syntax macros
in scheme that is incomparably better than anything I've ever seen in CL
implementations).

'as

Alexander Schmolck <a.schmolck@gmail.com> writes:
> In my experience the core defect of CL's defmacro is not the often touted lack
> of hygiene (see 2.) but extremely poor debuggability which IMO really
> compromises CL's otherwise very good support for interactive development and
> DSL creation.

Bullshit.

(defmacro m (args...) body...)

is equivalent to:

(defun m* (args...) body...)
(defmacro m (args...) (m& args...))

and m* is debuggable as easily as any other function.



> I'm not sure to what extent poor debuggability is a quality of implementation
> issue, but on the face of it the task seems much, much harder for procedural,
> plain sexp-based macros than for e.g. define-syntax style macros (I can't
> remember what the situation for syntax-case looked like, but I'm sure I
> witnessed error reporting for exceptions originating in define-syntax macros
> in scheme that is incomparably better than anything I've ever seen in CL
> implementations).

Macros are compiler hooks. It is expected that macro writers make the
effort of testing and signaling meaningful compile-time errors.

In the case of define-syntax, the macro writer is the define-syntax
operator itself. (The poor programmer is just using some restricted
define-syntax DSL, and cannot do much by himself).



--
__Pascal Bourguignon__ http://www.informatimago.com/

IMPORTANT NOTICE TO PURCHASERS: The entire physical universe,
including this product, may one day collapse back into an
infinitesimally small space. Should another universe subsequently
re-emerge, the existence of this product in that universe cannot be
guaranteed.

pjb@informatimago.com (Pascal J. Bourguignon) writes:

> Alexander Schmolck <a.schmolck@gmail.com> writes:
>> In my experience the core defect of CL's defmacro is not the often touted lack
>> of hygiene (see 2.) but extremely poor debuggability which IMO really
>> compromises CL's otherwise very good support for interactive development and
>> DSL creation.
>
> Bullshit.
>
> (defmacro m (args...) body...)
>
> is equivalent to:
>
> (defun m* (args...) body...)
> (defmacro m (args...) (m& args...))
>
> and m* is debuggable as easily as any other function.

I should have made it clearer that the poor debuggability refers not primarily
to the macro itself, but to code that uses it. You normally don't get precise
error locations for unhandled conditions originating from code that was
(def)macro-generated.

>> I'm not sure to what extent poor debuggability is a quality of implementation
>> issue, but on the face of it the task seems much, much harder for procedural,
>> plain sexp-based macros than for e.g. define-syntax style macros (I can't
>> remember what the situation for syntax-case looked like, but I'm sure I
>> witnessed error reporting for exceptions originating in define-syntax macros
>> in scheme that is incomparably better than anything I've ever seen in CL
>> implementations).
>
> Macros are compiler hooks. It is expected that macro writers make the
> effort of testing and signaling meaningful compile-time errors.
>

Sure -- and this can inherently not be fully automated (but of course all else
being equal, a system that gives you better error messages out of the box for
the subset of things for which this is possible is preferable).

> In the case of define-syntax, the macro writer is the define-syntax
> operator itself. (The poor programmer is just using some restricted
> define-syntax DSL, and cannot do much by himself).

Well, but in return you get a rather convenient system for specifying the most
simple macros[*], plus, crucially, a pretty good inverse transform from a
location in the expansion to location in your actual source code. And of
course there's always syntax-case which allows you to do anything you can do
with defmacro. However since I haven't used it properly I don't know how it
compares in practice and I'd be curious to know how it does on my list of
criteria (the same for alternative scheme macro systems with less mind-share,
as well as solutions in other languages such as dylan).

'as
[*] And of course interesting intellectual challenges for those insisting on
doing anything of even moderate complexity in it

See e.g. http://eval.apply.googlepages.com/eccentric.txt

Alexander Schmolck <a.schmolck@gmail.com> writes:
> I should have made it clearer that the poor debuggability refers not primarily
> to the macro itself, but to code that uses it. You normally don't get precise
> error locations for unhandled conditions originating from code that was
> (def)macro-generated.

But almost the same can be said of even "normal" CL code. Remember
that in CL, there are only 24 Special Operators, vs. 91 Macros: most
of the Common Lisp language is made of macros, and even "normal" CL
code will be "hard" to debug given that it will be essentially
macroexpanded all over.


Nonetheless, this doesn't prevent a "source-level" debugger to be
developed for Common Lisp.


>> In the case of define-syntax, the macro writer is the define-syntax
>> operator itself. (The poor programmer is just using some restricted
>> define-syntax DSL, and cannot do much by himself).
>
> Well, but in return you get a rather convenient system for specifying the most
> simple macros[*], plus, crucially, a pretty good inverse transform from a
> location in the expansion to location in your actual source code. And of
> course there's always syntax-case which allows you to do anything you can do
> with defmacro. However since I haven't used it properly I don't know how it
> compares in practice and I'd be curious to know how it does on my list of
> criteria (the same for alternative scheme macro systems with less mind-share,
> as well as solutions in other languages such as dylan).

I would almost propose my standard answer: "do it yourself", but I'm
wondering if it's really worthwhile. As long as you have a simple
macro to write, it could look interesting to have a specialized macro
maker, but then when you need to make it more complex, you will have
to translate from the simple language back to Common Lisp defmacro. I
guess the best solution I've seen proposed is that to have utility
routines and macros to help writting simple defmacros. They would be
better integrated into CL.

--
__Pascal Bourguignon__ http://www.informatimago.com/
Cats meow out of angst
"Thumbs! If only we had thumbs!
We could break so much!"

Alexander Schmolck wrote:
> Pascal Costanza <pc@p-cos.net> writes:
>
>> Ray Dillinger wrote:
>>> Pascal Costanza wrote:
>>>
>>>> Ray Dillinger wrote:
>>>>> There are things I think that the most recent [scheme] report got wrong.
>>>>> They made it unnecessarily complex, but variable capture, at least,
>>>>> is now possible.
>>>
>>>> ...but that new macro system _is_ unnecessarily complex.
>>> Your mouth to God's ear, friend.
>>>
>>> Wouldn't it work just fine to base it on the CL defmacro form, with internal
>>> syntax like (with-captures ... ) enclosing code specifying when you _don't_
>>> want to use renaming discipline? That way you get hygiene by default, avoid
>>> inadvertent capture, and get variable capture with a very simple additional
>>> syntax.
>> That's syntactic closures. Yes, that would be a fine solution.
>>
>> I'd prefer to go even more minimalistic. I think Will Clinger's renaming
>> system is all you need.
>
> Hmm, what exactly is "necessary" and what current macro systems provide it?

None of this is necessary. There exists a lot of code written in Common
Lisp, and more code is being developed and will be developed in the
future. You don't hear a lot of complaints because the macro system is
"unreliable," you actually hear hardly any complaints about the macro
system at all, apart from beginners. The problems for real-world
development are elsewhere.

> Apart from expressivness I think a good macro system would need the following
> 4 things (roughly in reverse order of importance):
>
> 1. ease of use (for writing macros))
> 2. robustness
> 3. plays well with interactive development
> 4. easy of use (for using macros viz. debugging support)
>
> Is there any language/implementation that ticks all these boxes?
>
> In my experience the core defect of CL's defmacro is not the often touted lack
> of hygiene (see 2.) but extremely poor debuggability which IMO really
> compromises CL's otherwise very good support for interactive development and
> DSL creation.

Macros are functions that transform s-expressions into s-expressions. A
good way to ensure that you have "correct" macros is to do what you
should do with any function: Write test suites.

> I'm not sure to what extent poor debuggability is a quality of implementation
> issue, but on the face of it the task seems much, much harder for procedural,
> plain sexp-based macros than for e.g. define-syntax style macros (I can't
> remember what the situation for syntax-case looked like, but I'm sure I
> witnessed error reporting for exceptions originating in define-syntax macros
> in scheme that is incomparably better than anything I've ever seen in CL
> implementations).

The main reason why it is harder to implement good debuggers for CL's
macro system is because different parts of a macroexpansion may
partially share structure, or even may share structure with other
macroexpansions. This means that, in the general case, you cannot
unambiguously deduce from part of a macroexpansion which actual macro
invocation generated that code.

In the syntax-case macro system, that's different: Due to its reliance
on syntax objects, you can always infer which macro invocation generated
which target code.


Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Alexander Schmolck wrote:
> pjb@informatimago.com (Pascal J. Bourguignon) writes:
>> Macros are compiler hooks. It is expected that macro writers make the
>> effort of testing and signaling meaningful compile-time errors.
>
> Sure -- and this can inherently not be fully automated (but of course all else
> being equal, a system that gives you better error messages out of the box for
> the subset of things for which this is possible is preferable).

But all else is not equal...

>> In the case of define-syntax, the macro writer is the define-syntax
>> operator itself. (The poor programmer is just using some restricted
>> define-syntax DSL, and cannot do much by himself).
>
> Well, but in return you get a rather convenient system for specifying the most
> simple macros[*], plus, crucially, a pretty good inverse transform from a
> location in the expansion to location in your actual source code. And of
> course there's always syntax-case which allows you to do anything you can do
> with defmacro. However since I haven't used it properly I don't know how it
> compares in practice and I'd be curious to know how it does on my list of
> criteria (the same for alternative scheme macro systems with less mind-share,
> as well as solutions in other languages such as dylan).

A programming language is not a list of features that its designers
ticked off. A main question is how various parts of the language design
fit and work together. What I'm trying to get at: Yes, in terms of lists
of features given on paper, syntax-case gives you all you want. That
doesn't necessarily mean that it's a good macro system.

It's the old Turing tarpit again...


Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Alexander Schmolck <a.schmolck@gmail.com> writes:

> Well, but in return you get a rather convenient system for specifying
> the most simple macros[*], [...]

In my experience, the set of simple macros is mostly just macros which
are wrappers around CALL-WITH-FOO functions (cf. [1].) And those are
not only easily done with DEFMACRO, but also very readable.

Getting most other macros as wonderfully crafted as the macros in the
Common Lisp standard is quite a non-trivial task. The complexity does
not come from DEFMACRO, though, but from the (seemingly inherent)
complexity in language design.


The problems of source location tracking comes, as Pascal Costanza
rightfully mentioned, from the insufficiencies of conses to build up
source code. That's also a realization shared by people like
David A. Moon.

-T.

[1] http://random-state.net/log/3390120648.html