Alternatives : lisp

On Wed, 5 Nov 2008 19:04:29 -0800 (PST), Isaac Gouy <igouy2@yahoo.com>
wrote:

>On Nov 4, 8:59 am, budden <budde...@mtu-net.ru> wrote:
>...
>> What do you think about this:http://shootout.alioth.debian.org/u3...p?test=all?=sb...
>...
>> Memory consumption is not measured correctly (minimal SBCL image is
>> about 25Mb).
>
>How interesting!
>
>On x86 Ubuntu, System Monitor reports 1.9MiB for the SBCL fannkuch
>program memory use and the benchmarks game shows 5,572KB.
>
>http://shootout.alioth.debian.org/u3...lang=sbcl&id=2
>
>
>On x64 Ubuntu, System Monitor reports 33.1MiB for the SBCL fannkuch
>program memory use and the benchmarks game shows 38,024KB.
>
>http://shootout.alioth.debian.org/u6...lang=sbcl&id=2
>
>
>Any idea what's going on?
>

I haven't looked at the shootout pages - I don't really care - but
questions about memory use are complicated by virtual memory. At a
minimum you have to distinguish between the reserved set, the
committed set, the paged-in set, and the algorithm's working set (all
of which may change over time) in order to fairly compare memory use
between programs intended as "benchmarks".

All of these numbers are available, but rarely are they all reported.

George

ea> Dear Lispers, can you design a DSL (but true DSL!) in Lisp for
ea> generating XML (or HTML) that will give great (small) advantage over

ea> 1) Python/Ruby/Perl/Lisp functions with keywords:
ea> div(span('some text', style="span-style"),
ea> style="kuku", onClick="alert('kuku')")


DSL approach might have couple of advantages:
* better performance -- all constant parts can be compiled into
constant strings at compilation time. joining lots of small strings
might be sort of expensive performance-wise, and sometimes even function
calls might be rather expensive.
* with DSL you have more chances to verify correctness of structure of
your code at compile time (i think you can also do that in languages with
advanced type system and static type checking).
* in DSL you have clear separation between dynamically injected strings
and HTML pieces, so you can automatically escape those strings etc.

with a simplistic implementation via functions you will have everything
as string, so you can't do this automatically. you might choose to have
intermediate representation building sort of DOM, but then you'll have
even more overhead.
* but, huh, how DIV with two SPANs will look? probably you'll need
a construct to join adjacent elements:

div(span('some text', style="span-style")
+ span('other text', style="span-style"),
style="kuku", onClick="alert('kuku')")

in general, HTML generation does not map well onto function calls --
it is sort of ugly, and DSL can fix that.

but actually HTML generation is not the biggest reason to have DSLs.
i think there are more advantages for query languages. and of course,
this makes more sense in bigger code pieces, where you can see synergy
between several DSL -- for example, you directly output results of
query into HTML. and often, these advantages from DSLs are rather small,
but a bit here and a bit there -- and whole picture changes.

ea> 2) Given Smalltalk OO code ?

i dunno how Smalltalk OO code works, but i guess some points above
apply to it too.

On Nov 5, 8:50 pm, George Neuner <gneun...@comcast.net> wrote:
> On Wed, 5 Nov 2008 19:04:29 -0800 (PST), Isaac Gouy <igo...@yahoo.com>
> wrote:
>
>
>
> >On Nov 4, 8:59 am, budden <budde...@mtu-net.ru> wrote:
> >...
> >> What do you think about this:http://shootout.alioth.debian.org/u3...p?test=all?=sb...
> >...
> >> Memory consumption is not measured correctly (minimal SBCL image is
> >> about 25Mb).
>
> >How interesting!
>
> >On x86 Ubuntu, System Monitor reports 1.9MiB for the SBCL fannkuch
> >program memory use and the benchmarks game shows 5,572KB.
>
> >http://shootout.alioth.debian.org/u3...st=fannkuch&la....
>
> >On x64 Ubuntu, System Monitor reports 33.1MiB for the SBCL fannkuch
> >program memory use and the benchmarks game shows 38,024KB.
>
> >http://shootout.alioth.debian.org/u6...st=fannkuch&la....
>
> >Any idea what's going on?
>
> I haven't looked at the shootout pages - I don't really care - but
> questions about memory use are complicated by virtual memory.  At a
> minimum you have to distinguish between the reserved set, the
> committed set, the paged-in set, and the algorithm's working set (all
> of which may change over time) in order to fairly compare memory use
> between programs intended as "benchmarks".
>
> All of these numbers are available, but rarely are they all reported.
>
> George


Does anyone have a suggestion about what's going on with the numbers
that are reported?

For the same sbcl fannkuch program, for the same input values, top
shows

x86 Ubuntu
VIRT 523m
RES 5572
SHR 3628

x64 Ubuntu
VIRT 8243m
RES 37m
SHR 4160

Programs in other languages show a only modest increase in memory use
going from x86 to x64 - so what's going on with sbcl?

Isaac Gouy <igouy2@yahoo.com> writes:
> Does anyone have a suggestion about what's going on with the numbers
> that are reported?
>
> For the same sbcl fannkuch program, for the same input values, top
> shows
>
> x86 Ubuntu
> VIRT 523m
> RES 5572
> SHR 3628
>
> x64 Ubuntu
> VIRT 8243m
> RES 37m
> SHR 4160
>
> Programs in other languages show a only modest increase in memory use
> going from x86 to x64 - so what's going on with sbcl?

The VIRT is just due to sbcl reserving a larger default heap on x86-64
(0.5GB vs 8GB). We use the larger default larger address spaces are
the main reason to go 64-bit. The increase in RES is probably due to
sbcl's internal gc page tables, which will basically become larger
when the heap size increases, and which are touched at process startup
(hence being resident).

It should be pretty easy to frob things so that the whole page table
doesn't need to be touched at startup. Alternatively runnign with a
smaller heap size (e.g. --dynamic-space-size 500) should result in
roughly the same space usage as on x86.

And to answer the point made a couple of posts upthread, it's quite
correct for the resident set size to be smaller than the image size:
most of the lisp image is just mapped into memory, but not actually
accessed during the execution of a small benchmark.

--
Juho Snellman

On Nov 6, 9:16 am, Juho Snellman <jsn...@iki.fi> wrote:
> Isaac Gouy <igo...@yahoo.com> writes:
> > Does anyone have a suggestion about what's going on with the numbers
> > that are reported?
>
> > For the same sbcl fannkuch program, for the same input values, top
> > shows
>
> > x86 Ubuntu
> > VIRT 523m
> > RES 5572
> > SHR 3628
>
> > x64 Ubuntu
> > VIRT 8243m
> > RES 37m
> > SHR 4160
>
> > Programs in other languages show a only modest increase in memory use
> > going from x86 to x64 - so what's going on with sbcl?
>
> The VIRT is just due to sbcl reserving a larger default heap on x86-64
> (0.5GB vs 8GB). We use the larger default larger address spaces are
> the main reason to go 64-bit. The increase in RES is probably due to
> sbcl's internal gc page tables, which will basically become larger
> when the heap size increases, and which are touched at process startup
> (hence being resident).
>
> It should be pretty easy to frob things so that the whole page table
> doesn't need to be touched at startup. Alternatively runnign with a
> smaller heap size (e.g. --dynamic-space-size 500) should result in
> roughly the same space usage as on x86.
>
> And to answer the point made a couple of posts upthread, it's quite
> correct for the resident set size to be smaller than the image size:
> most of the lisp image is just mapped into memory, but not actually
> accessed during the execution of a small benchmark.
>
> --
> Juho Snellman


Thank you for taking the time to give a comprehensive answer.

On 5 ÎÏÑÂ, 23:57, "Alex Mizrahi" <udode...@users.sourceforge.net>
wrote:
> ea> Dear Lispers, can you design a DSL (but true DSL!) in Lisp for
> ea> generating XML (or HTML) that will give great (small) advantage over
>
> ea> 1) Python/Ruby/Perl/Lisp functions with keywords:
> ea> div(span('some text', style="span-style"),
> ea> style="kuku", onClick="alert('kuku')")
>
> DSL approach might have couple of advantages:
> * better performance -- all constant parts can be compiled into
> constant strings at compilation time. joining lots of small strings
> might be sort of expensive performance-wise, and sometimes even function
> calls might be rather expensive.
> * with DSL you have more chances to verify correctness of structure of
> your code at compile time (i think you can also do that in languages with
> advanced type system and static type checking).
> * in DSL you have clear separation between dynamically injected strings
> and HTML pieces, so you can automatically escape those strings etc.
>
> with a simplistic implementation via functions you will have everything
> as string, so you can't do this automatically. you might choose to have
> intermediate representation building sort of DOM, but then you'll have
> even more overhead.
> * but, huh, how DIV with two SPANs will look? probably you'll need
> a construct to join adjacent elements:
>
> div(span('some text', style="span-style")
> + span('other text', style="span-style"),
> style="kuku", onClick="alert('kuku')")
>
> in general, HTML generation does not map well onto function calls --
> it is sort of ugly, and DSL can fix that.
>
> but actually HTML generation is not the biggest reason to have DSLs.
> i think there are more advantages for query languages. and of course,
> this makes more sense in bigger code pieces, where you can see synergy
> between several DSL -- for example, you directly output results of
> query into HTML. and often, these advantages from DSLs are rather small,
> but a bit here and a bit there -- and whole picture changes.
>
> ea> 2) Given Smalltalk OO code ?
>
> i dunno how Smalltalk OO code works, but i guess some points above
> apply to it too.

Let's put aside performance/compile-time-checks for now and
concentrate on usability:
DIV can have any number of SPANs

# warning: toy example
def html_element(name):
'ha! mixing &key and &rest'
def inner(*args, **kwargs):
return '<%s %s>%s</%s>' % (name,
' '.join(['%s="%s"' % (k, v) for k,
v in kwargs.iteritems()]),
''.join(args),
name)
return inner

def define_html_element(name):
globals()[name] = html_element(name)

map(define_html_element, ['div', 'span', 'form', 'input'])

print div(span('first', style='span-style'),
span('second', ' span', style='second'),
style='default-div',
id="dsl")

It seems that HTML/XML generation maps perfectly well into function
calls. No need for macros here. And almost every book on Lisp has
chapter "Generating HTML with macros".

Query languages? Synergy between several DSLs? Great!

Dear Lispers, can you design a DSLs (imaginary) for
generating XML and for querying that together will give any advantage
over ordinary Python+SQL code? Please give an example of its usage.

On 2008-11-06, ershov.a_n@mail.ru <ershov.a_n@mail.ru> wrote:
> print div(span('first', style='span-style'),
> span('second', ' span', style='second'),
> style='default-div',
> id="dsl")
>
> It seems that HTML/XML generation maps perfectly well into function
> calls.

One problem is evaluation. All of the arguments to DIV are evaluated before it
is called. So if you're generating a long HTML, nothing happens until the end.

Those macro-based Lisp HTML generators are often directly coupled to HTTP
streams. Because they are macro-based, they have full control over the order of
translation of the S-expression to the HTML. HTML streaming can begin before
everything is evaluated.

With this function-based approach, you can't stream the HTML, because you're
stuck with generating the HTML bottom-up; the nested function calls generate
the inner pieces before the outer ones. The outermost div is called last, and
only then are the pieces integrated to form the outermost div block.

Moreover, the arguments to div, when it is finally called, are canned character
strings. The div function cannot do much more with the pieces other than
catenate them together. What if I want some function that I can use in place
of div that will walk the HTML tree and do something interesting to it? That
function will have to parse HTML strings.

See, everything /seems/ easy, if you flunked computer science and haven't made
up for it in later years by learning how to think.

Of course, we haven't proved that you need macros; it's possibly doable in a
functional language with lazy evaluation (normal argument passing), if you have
some tools to influence the actual evaluation order.

You might be able to hack it using Python generators. But Python generators
are not regular functions. Python generators use some special syntax which
cannot be developed in Python itself. Before the benevolent dictator declared
``let there be generators'', they were not possible to implement in any way
other than as a patch to the Python source code. Back to macros!

On 5 nov, 17:27, Kenny <kentil...@gmail.com> wrote:

> In all these bake-offs the anti-Lispers are making a fatal error: tiny
> little bits of code.

And where is the big code for Lisp? Ein?

Nobody sees it.

It is secret.

You know, for constructing nuclear weapons and things like that.

Or maybe indefinitely "under rapid development".

Franccesco wrote:
> On 5 nov, 17:27, Kenny <kentil...@gmail.com> wrote:
>
>
>>In all these bake-offs the anti-Lispers are making a fatal error: tiny
>>little bits of code.
>
>
> And where is the big code for Lisp? Ein?
>
> Nobody sees it.
>
> It is secret.
>
> You know, for constructing nuclear weapons and things like that.
>
> Or maybe indefinitely "under rapid development".

I was actually going to recommend Cells. It is a reasonable amount of
code but exercises Lisp (and Python, where Philip Eby has done something
inspired by Cells) pretty seriously. It is also a fine entry in the
functional reactive programming horse race, a way cool fun productive
paradigm, so at the same time whatshisface could gain admission to the
yearly FRP Conferences to be held in Monte Carlo.

Then we get a real comparison and a real contrib to Ruby hence the
world. Without macros Rubycells will not look as slick, but that's OK,
the bang is in the functionality.

peace,kxo

On 6 ÎÏÑÂ, 13:13, Kaz Kylheku <kkylh...@gmail.com> wrote:

> With this function-based approach, you can't stream the HTML, because you're
> stuck with generating the HTML bottom-up; the nested function calls generate
> the inner pieces before the outer ones. The outermost div is called last,and
> only then are the pieces integrated to form the outermost div block.

Again performance issue.
Here comes interesting moment: We can bypass this problem just with
lambdas, and it looks like this (in Ruby):

html(:style => 'default-div', :id => 'dsl') {
span(:style => 'span-style') {
print "first"
}
span(:style => 'second') {
print "second span"
}
}

,but Python has no good lambdas. So, bye, function approach:

from __future__ import with_statement

class Tag:
def __init__(self, name):
self.name = name

def __enter__(self):
pass

def __call__(self, **kwargs):
print("<%s %s>" % (self.name,
' '.join(['%s="%s"' % (k, v) for k, v in
kwargs.iteritems()])))
return self

def __exit__(self, *args):
print("</%s>" % self.name)

def html_element(name):
return Tag(name)

def define_html_element(name):
globals()[name] = html_element(name)

map(define_html_element, ['div', 'span', 'form', 'input'])

with div(style = 'default-div', id = 'dsl'):
with span(style='span-style'):
print("first")
with span(style='second'):
print("second span")

And we can begin HTML streaming.

Let's call this approach "hack with generators" (though it is not a
hack and doesn't use generators, but sounds great)


> Moreover, the arguments to div, when it is finally called, are canned character
> strings. The div function cannot do much more with the pieces other than
> catenate them together. What if I want some function that I can use in place
> of div that will walk the HTML tree and do something interesting to it? That
> function will have to parse HTML strings.

At least two options for Python here:
1) use XSL
2) use ElementTree instead of strings

each of this options is better than writing code walker in Lisp

> Of course, we haven't proved that you need macros; it's possibly doable in a
> functional language with lazy evaluation (normal argument passing), if you have
> some tools to influence the actual evaluation order.
>
> You might be able to hack it using Python generators. But Python generators
> are not regular functions. Python generators use some special syntax which
> cannot be developed in Python itself. Before the benevolent dictator declared
> ``let there be generators'', they were not possible to implement in any way
> other than as a patch to the Python source code. Back to macros!

That's right. Generators use special syntax. With_statements use
special syntax. And you cannot even imagine Lispy DSL for generating
XML that will be better than this special syntax.