Re: Strange bugs in parser when reading in multiple threads?

Aaron Hsu wrote:
> (define (parse fname)
> (call-with-input-file fname
> (lambda (port)
> (ssax:xml->sxml port '()))))
>
> (define (run-test fname)
> (let ([m (make-mutex)]
> [i 0]
> [finished (make-condition)])
> (define (thunk)
> (parse fname)
> (set! i (1+ i))
> (condition-signal finished))
> (fork-thread thunk)
> (fork-thread thunk)
> (Fork-thread thunk)
> (mutex-acquire m)
> (let loop ()
> (if (= 3 i) #t
> (begin (condition-wait finished m)
> (loop))))))

I don't know why you got the error, but I have
three comments:

1. This is an implementation-specific question, but
you didn't identify the implementation.

2. There is an obvious race on i. If two threads
fetch the value of i in (1+ i) before either assigns
the result of that addition to i, then i will never
become 3.

3. Without knowing anything about the implementation,
my first guess would be that its i/o system isn't
completely thread-safe. Perhaps two of the three
concurrent calls to call-with-input-file ended up
sharing the same low-level file descriptor. I'd
suggest using sequential code to open the file three
times before passing the three distinct ports to the
three threads.

Will

In article
<3b3ff318-a4cb-4126-b9ea-8a323ef565f0@a1g2000hsb.googlegroups.com>,
William D Clinger <cesura17@yahoo.com> wrote:
> 1. This is an implementation-specific question, but
> you didn't identify the implementation.

It's likely this is an issue that has been discussed
on Oleg's SSAX mailing list before:
http://sourceforge.net/mailarchive/m...F38B4.30709%40
entricom.com

input-parse.scm is not thread-safe by default; the "offending"
section of code is given below:
; Procedure input-parse:init-buffer
; returns an initial buffer for next-token* procedures.
; The input-parse:init-buffer may allocate a new buffer per each
invocation:
; (define (input-parse:init-buffer) (make-string 32))
; Size 32 turns out to be fairly good, on average.
; That policy is good only when a Scheme system is multi-threaded with
; preemptive scheduling, or when a Scheme system supports shared
substrings.
; In all the other cases, it's better for input-parse:init-buffer to
; return the same static buffer. next-token* functions return a copy
; (a substring) of accumulated data, so the same buffer can be reused.
; We shouldn't worry about new token being too large: next-token will use
; a larger buffer automatically. Still, the best size for the static
buffer
; is to allow most of the tokens to fit in.
; Using a static buffer _dramatically_ reduces the amount of produced
garbage
; (e.g., during XML parsing).
(define input-parse:init-buffer
(let ((buffer (make-string 512)))
(lambda () buffer)))

As mentioned in the reference above, what others have done for
multi-threaded apps is change this code to something like:
> (define input-parse:init-buffer
> (lambda () (make-string 512)))

Jim