R6RS test suite

On Jul 23, 7:08*am, leppie <xacc....@gmail.com> wrote:
> Line 44 of hashtables.sls: should be eqv?, not eq? EVER!

I think it should be `eq?' for an `eq?'-based hashtable, but I see
your point for other hashtables. I've adjusted the test to vary the
comparison based on the kind of hash table being testing.

> Also in line 161 of hashtables.sls: string-ci-hash and string-hash are
> not comparable, unless you define string-ci-hash as:
> (define (string-ci-hash str)
> * (string-hash (string-downcase str)))
>
> But that is inefficient.
>
> IMO the test is wrong, it should compared string-ci-hash to string-ci-
> hash with different cased strings.

Right.

These bugs are now fixed in SVN.

Thanks,
Matthew

On Jul 23, 2:35*pm, Matthew Flatt <mfl...@cs.utah.edu> wrote:
>
> > 2. Should record names interfere with variable names?
>
> Sorry - I'm not sure what you mean.
>
> Matthew

Say I do (on Ikarus):

> cons
#<procedure cons>
> (define-record-type cons)
> cons
Unhandled exception
Condition components:
1. &who: cons
2. &message: "invalid expression"
3. &syntax:
form: cons
subform: #f
4. &trace: #<syntax cons>

Why should a record name interfere? Now it seems 'cons' is some kind
of syntax, but yet it is only usable in 2 places (record-type-
descriptor & record-constructor-descriptor).

On Jul 23, 3:20*pm, Matthew Flatt <mfl...@cs.utah.edu> wrote:

> I think it should be `eq?' for an `eq?'-based hashtable, but I see
> your point for other hashtables. I've adjusted the test to vary the
> comparison based on the kind of hash table being testing.

Yeah, I missed that

I'd like to suggest two changes to the test suite. With
these two changes, the current development version of
Larceny is able to complete its execution of the entire
test suite in a reasonable amount of time, with some test
failures but without any fatal errors.

In tests/r6rs/io/ports.sls, I suggest changing the last
three calls to open-file-input/output-port to use a
transcoder with the latin-1-codec instead of utf-8-codec.
The R6RS does not specify a semantics for opening a file
for input/output with a variable-width codec. The R6RS
editors who insisted upon adding textual input/output
ports to the R6RS had the Posix semantics in mind, but
the Posix semantics for input/output ports assumes a
fixed-width encoding. Since the semantics of the R6RS
input/output ports with variable-width transcoding is
implementation-dependent, I believe the R6RS allows or
should allow implementations to raise an exception when
a program attempts to open a file for mixed input/output
with a variable-width encoding. Larceny does that, and
I believe this aspect of Larceny's behavior is a feature,
not a bug. Changing the test suite to use a fixed-wdith
encoding here would not alter the nature of the test.

The other change I suggest would reduce the code size of
tests/r6rs/arithmetic/fixnums.sls. Its large code size
is caused by this code:

;; If you put N numbers here, it expads to N^3 tests!
(carry-tests 0
[0 1 2 -1 -2 38734 -3843 2484598
-348732487 (greatest-fixnum) (least-fixnum)])

The following change would reduce those 1331 tests to
500 without losing much in the way of test coverage:

(carry-tests 0 [0 1 2 -1 -2])

(carry-tests 0 [2 -1 -2 38734 -3843])

(carry-tests 0 [2 -1 -2 (greatest-fixnum) (least-fixnum)])

(carry-tests 0 [-3843 2484598 -348732487
(greatest-fixnum) (least-fixnum)])

Thanks again for making these tests available to other
implementors of the R6RS.

Will

On Jul 23, 8:59*am, William D Clinger <cesur...@yahoo.com> wrote:
> In tests/r6rs/io/ports.sls, I suggest changing the last
> three calls to open-file-input/output-port to use a
> transcoder with the latin-1-codec instead of utf-8-codec.

Done in SVN.

> The other change I suggest would reduce the code size of
> tests/r6rs/arithmetic/fixnums.sls. *Its large code size
> is caused by this code:
>
> * * ;; If you put N numbers here, it expads to N^3 tests!
> * * (carry-tests 0
> * * * * * * * * *[0 1 2 -1 -2 38734 -3843 2484598
> * * * * * * * * * -348732487 (greatest-fixnum) (least-fixnum)])
>
> The following change would reduce those 1331 tests to
> 500 without losing much in the way of test coverage:
>
> * * (carry-tests 0 [0 1 2 -1 -2])
>
> * * (carry-tests 0 [2 -1 -2 38734 -3843])
>
> * * (carry-tests 0 [2 -1 -2 (greatest-fixnum) (least-fixnum)])
>
> * * (carry-tests 0 [-3843 2484598 -348732487
> * * * * * * * * * * (greatest-fixnum) (least-fixnum)])

I'm less sold on this one. Don't we want tests combining
`(greatest-fixnum)' with 0 and with 1, for example?

In fact, in SVN, I've made it worse. The `carry-tests' form was
meant to expand to tests of `fx+/carry' and `fx-/carry' in addition
to `fx*/carry'. So, now there are 3 times as many tests, instead of
1/3 as many tests. And even if I had re-organized as above, adding the
missing operators would leave us with more tests than before.

But it does seem unlikely that 3993 tests are really necessary.
Any ideas that will shrink the set of tests enough, even with the
added operators?


Thanks,
Matthew

On Jul 23, 9:53*am, Matthew Flatt <mfl...@cs.utah.edu> wrote:
> Any ideas that will shrink the set of tests enough, even with the
> added operators?

There's a really simple solution: instead of generating N test
expressions, just iterate over the inputs. (I was too stuck on
macros, which is a convenient way to track the original expression,
but obviously not the only way.)

Done in SVN.

Matthew

leppie wrote:
> Line 44 of hashtables.sls: should be eqv?, not eq? EVER! (this goes
> for the slatex and compiler benchmarks from Larceny too)

I didn't write either of those benchmarks. Spurred by
your message, I went through the R6RS version of the
latex benchmark and corrected all uses of eq? and memq
on characters. You can now obtain the corrected version
from Larceny's Trac site. I'll check in a corrected
version of the R5RS slatex benchmark after I test it.
The more important thing, of course, is to correct the
slatex program itself, which I haven't done.

With the compiler benchmark, it's harder to tell which
uses of eq?, memq, and assq need to be corrected. It
looks as though the easiest way to find out is to add
some instrumentation to the code. That's why I haven't
yet corrected the compiler benchmark, but I plan to get
it done eventually.

Will

On Jul 23, 9:08*pm, William D Clinger <cesur...@yahoo.com> wrote:
> leppie wrote:
> > Line 44 of hashtables.sls: should be eqv?, not eq? EVER! (this goes
> > for the slatex and compiler benchmarks from Larceny too)
>
> I didn't write either of those benchmarks. *Spurred by
> your message, I went through the R6RS version of the
> latex benchmark and corrected all uses of eq? and memq
> on characters. *You can now obtain the corrected version
> from Larceny's Trac site. *I'll check in a corrected
> version of the R5RS slatex benchmark after I test it.
> The more important thing, of course, is to correct the
> slatex program itself, which I haven't done.
>
> With the compiler benchmark, it's harder to tell which
> uses of eq?, memq, and assq need to be corrected. *It
> looks as though the easiest way to find out is to add
> some instrumentation to the code. *That's why I haven't
> yet corrected the compiler benchmark, but I plan to get
> it done eventually.
>
> Will

Thanks

I managed to get slatex working, totally gave up on the compiler
one

Cheers

leppie

Some more bugs (the conditions should all be wrapped with (record-type-
descriptor)):

- Line 80 of test.sls
- Line 264 & line 266 of flonums.sls

Cheers

leppie

On Jul 23, 2:20*pm, leppie <xacc....@gmail.com> wrote:
> Some more bugs (the conditions should all be wrapped with (record-type-
> descriptor)):
>
> - Line 80 of test.sls
> - Line 264 & line 266 of flonums.sls

Fixed in SVN.

Thanks,
Matthew