Re: Dangers of WITH-*-TO-* Procedures?

The with-input-from-X and with-output-to-X paradigm breaks
down in systems that provide concurrent threads. (It also
breaks down even in systems that use merely use call/cc to
simulate concurrent threads, which is one of the potential
problems that Pascal pointed out.)

Advocates of the with-input-from-X and with-output-to-X
paradigm will argue that these problems can be solved by
using thread-local parameters, but those alleged solutions
are slightly controversial and are certainly not mandated
by any of the Scheme standards. Portable code therefore
cannot rely upon them. Until a standard emerges for these
matters, the most portable solution is to avoid the
problematic paradigm.

Will

George Neuner wrote:
> Thread locals won't solve it alone because threads may share the file
> handle or be able to access the file through a different handle.

We seem to be concerned about different issues. As I see
it, sharing of ports (or file handles) between concurrent
threads is a separate issue. My opinion on that sharing
issue is that race conditions on ports are like races on
other mutable objects, and that they usually represent
bugs in the program(s) being executed. In my opinion,
implementors have a responsibility to prevent such bugs
from leading to catastrophic system failures, but are not
responsible for preventing such bugs from causing screwy
behavior in the buggy programs or in programs that rely
upon buggy programs.

> Question is: in the absence of deliberate call/cc abuse - that is
> permitting unwinding continuations only -

I don't share your assumption that using full continuations
constitutes abuse. On the other hand, the with-X paradigm
doesn't work without that assumption. That is why the
with-X paradigm is deprecated in Larceny.

> would it suffice for
> with-input to place a read-lock on the file and with-output to place a
> write-lock? (Where to lock - inside Scheme or in the OS is another
> topic.) I know that locking has implications for concurrent
> non-overlapping access, but since the with-io macros assume serial
> access, would there be any (non-performance) harm in locking?

Wouldn't that cause deadlock in programs that use the
with-X paradigm recursively?

Will

On Thu, 14 Aug 2008 15:19:04 -0700 (PDT), William D Clinger
<cesura17@yahoo.com> wrote:

>George Neuner wrote:
>
>> Question is: in the absence of deliberate call/cc abuse - that is
>> permitting unwinding continuations only -
>
>I don't share your assumption that using full continuations
>constitutes abuse.

I have no issues with continuations for computation, but where I/O is
concerned, I think full generality is a recipe for disaster ...
particularly where a continuation may be executed by a different
thread than the one that created it.


>On the other hand, the with-X paradigm
>doesn't work without that assumption. That is why the
>with-X paradigm is deprecated in Larceny.
>
>> would it suffice for
>> with-input to place a read-lock on the file and with-output to place a
>> write-lock? (Where to lock - inside Scheme or in the OS is another
>> topic.) I know that locking has implications for concurrent
>> non-overlapping access, but since the with-io macros assume serial
>> access, would there be any (non-performance) harm in locking?
>
>Wouldn't that cause deadlock in programs that use the
>with-X paradigm recursively?

That's an issue for the lock algorithm. I've used a number of
different lock implementations over the years - some where recursive
locks on the same region were superfluous. The only real problem I've
encountered has been multiple locks on the overlap of two regions. I
don't have a ready answer for that, but I haven't paid much attention
to lock algorithms for a while.

>Will

George

William D Clinger wrote:

> I don't share your assumption that using full continuations
> constitutes abuse. On the other hand, the with-X paradigm
> doesn't work without that assumption. That is why the
> with-X paradigm is deprecated in Larceny.

FWIW, I'm using "with" to establish dynamically-scoped
variables in the same way (and with the same syntax other
than keyword) that "let" establishes statically scoped
variables. That seemed to me a generalization of the way
scheme uses "with-foo" forms whenever it needs dynamically
scoped entities (although it's a bit dishonest to claim
that scheme has no dynamic variables, when implementors have
to do all the dynamic-binding machinery anyway to handle
entities such as output ports).

I haven't invested the amount of thought in threading and
concurrency issues that you have, Will. But I have run into
the issue of race conditions on I/O, and thought about that.
And, oddly, the techniques I've found and finally understood
may be a reinvention of the idea of monads, which had never
previously been described to me in a way that made me think
they were actually useful.

I used to think that monads were pretty useless; merely a
non-stateful and therefore somewhat inaccurate and ridiculously
complex way of talking and reasoning about a stateful
interaction. I'm less convinced of that now. I've found
myself implementing in a threaded application an "output"
method that accumulates the output from a thread in a
function-local variable and then passes the function back
to the parent thread, so that the parent can call the
function and by doing so output sequentially things that
were computed in parallel in different threads. And that's
suspiciously similar to the description of an output monad.
I still haven't found implementing something that reminds
me of the description of an input monad to actually be
useful, but I no longer dismiss the possibility.

There's something reducible about parallelism where a bunch
of threads all ask the same question and interpret answers
the same way. To the extent that these patterns of I/O are
similar, it ought to be possible to merge the outputs and
inputs across threads, such that all threads which ask the
same question should get the same answer while the user only
sees the question and enters the answer once (and other
threads are blocked because *their* questions haven't been
dealt with yet...). And that idea might be me arriving,
finally, at the "I/O monad" whose descriptions as "functions
that describe I/O" have never given me any impression of any
actual utility.

So... am I coming to an understanding of the _practical_
value of I/O monads, or am I off in the weeds somewhere with
a different idea?

The central and fundamental problem is that human attention
isn't really all that concurrent. To the extent that our
attention's modality mismatches the modality of the software
doing I/O with which we are interacting, we encounter
difficulties. If our brains were differently constituted
it might be otherwise.

Also, the problem arises less (but still arises) with lazy
evaluation than it does with eager evaluation. But this is
just a side effect of having fewer threads of evaluation
active at the same time.

Bear

On Aug 14, 4:19*am, William D Clinger <cesur...@yahoo.com> wrote:
> The with-input-from-X and with-output-to-X paradigm breaks
> down in systems that provide concurrent threads. *(It also
> breaks down even in systems that use merely use call/cc to
> simulate concurrent threads, which is one of the potential
> problems that Pascal pointed out.)
[...]
> Until a standard emerges for these
> matters, the most portable solution is to avoid the
> problematic paradigm.

By "problematic paradigm" you surely mean concurrent threads and call/
cc, right?

SCNR,
M/

Michael Weber wrote:

> On Aug 14, 4:19Â*am, William D Clinger <cesur...@yahoo.com> wrote:
>> The with-input-from-X and with-output-to-X paradigm breaks
>> down in systems that provide concurrent threads. Â*(It also
>> breaks down even in systems that use merely use call/cc to
>> simulate concurrent threads....)

> By "problematic paradigm" you surely mean concurrent threads and call/
> cc, right?

You may've put a smiley there, but I've had occasion to wonder
about that in all seriousness.

Call/cc does whatever you want, in terms of sequential control,
but it does seem like a possible wrong choice in terms of
systems where there is hardware multitasking. That said, I
don't see a candidate for the right choice, so.... not much
to say.

However, I can't help but point out that file I/O of whatever
type seems to break down in the same ways in multithreaded
programs of whatever language. Call/cc is like a very fancy
goto that preserves environments. But if someone is writing
a multithreaded application that does file I/O and uses goto,
I can't help but think that the crop of bugs is going to be
.... interesting ... no matter what language s/he's using.

Bear