GAWK: A fix for "missing file is a fatal error"

In article <g8hlu5$hvb$1@online.de>,
Janis Papanagnou <Janis_Papanagnou@hotmail.com> wrote:
>pk wrote:
>> On Wednesday 20 August 2008 16:16, Janis Papanagnou wrote:
>>
>>>>Maybe because that would be impractical if you're processing many file
>>>>arguments...?
>>>>
>>>> awk '{print FILENAME,$0}' prefix*.ext
>>>
>>>Oops, makes not much sense with wildcards.
>>
>> In that case the shell does all the work and the resulting file list
>> contains only files that actually exist.
>
>Yes, that's why I cancelled my original message and added this comment.
>I think it's still a Good Thing to let an "invisible" layer handle that
>instead of using explicit workarounds for each of the given files and
>avoiding "non-scalable" (as Kenny called it) shell constructs, which was
>the intention introducing my wildcard example in the first place to show
>the problem that arises with many file arguments.
>
>Janis

Good post. Thanks.

I like your concept of an "invisible layer".

On 8/20/2008 7:48 AM, Kenny McCormack wrote:
> In one of my scripts, I found that in GAWK, if a file is missing (can't
> be opened), GAWK bombs with a fatal error. I have a vague memory of there
> being some discussion of this issue in this group at some point in the
> distant past, and the consensus was that there wasn't anything you could
> do about it.
>
> Note to standards jockeys: No, this isn't a bug in your precious GAWK in
> the usual "standards" sense. So, don't even bother.
>
> The fact is that, under certain conditions, it *is* a mis-feature, and it
> would be nice to at least have the option of continuing.

I agree. A fatal error in this situation stinks. You could work around it, of
course, with an up-front getline test:

$ ls f1 f2 f3
ls: cannot access f2: No such file or directory
f1 f3

$ cat f1 f3
f1, line 1
f3, line 1

$ awk '1' f1 f2 f3
f1, line 1
awk: (FILENAME=f1 FNR=1) fatal: cannot open file `f2' for reading (No such file
or directory)

$ awk 'BEGIN{if ((getline<ARGV[2])<0) ARGV[2]="/dev/null"; else
close(ARGV[2])}1' f1 f2 f3
f1, line 1
f3, line 1

$ echo "f2, line 1" > f2

$ awk 'BEGIN{if ((getline<ARGV[2])<0) ARGV[2]="/dev/null"; else
close(ARGV[2])}1' f1 f2 f3
f1, line 1
f2, line 1
f3, line 1

You can't tell from the getline if the file's missing or just can't be opened
but you probably wouldn't care.

Ed.

On Aug 20, 6:08*pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
> $ awk 'BEGIN{if ((getline<ARGV[2])<0) ARGV[2]="/dev/null"; else
> close(ARGV[2])}1' f1 f2 f3
> f1, line 1
> f3, line 1

This approach seems sensible to me. And rather than use LD_PRELOAD
to solve the problem, why not use an xgawk include file? If
you stick the follwing file in /usr/share/xgawk/fixopen.awk

BEGIN {
for (i = 1; i < ARGC; i++) {
if ((getline < ARGV[i]) < 0)
delete ARGV[i]
else
close(ARGV[i])
}
}

then you can say:

bash-3.1$ xgawk '1; END {print "DONE"}' /tmp/does_not_exist
xgawk: cmd. line:1: fatal: cannot open file `/tmp/does_not_exist' for
reading (No such file or directory)

vs.

bash-3.1$ xgawk -i fixopen '1; END {print "DONE"}' /tmp/does_not_exist
DONE

That seems perhaps easier to maintain than using LD_PRELOAD. Plus
it has the advantage of deleting the file from the argument list,
so there's no need to open /dev/null in its place.

Regards,
Andy

P.S. I recognize that this has a problem in the case where the
user is passing other types of information (besides filenames)
on the command line. For some people that may be a problem;
I tend not to pass non-filename arguments very often...

In article <8535b7bd-e673-4184-aa28-405d55d803ce@e39g2000hsf.googlegroups.com>,
Andrew Schorr <aschorr@telemetry-investments.com> wrote:
>On Aug 20, 6:08*pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
>> $ awk 'BEGIN{if ((getline<ARGV[2])<0) ARGV[2]="/dev/null"; else
>> close(ARGV[2])}1' f1 f2 f3
>> f1, line 1
>> f3, line 1
>
>This approach seems sensible to me. And rather than use LD_PRELOAD
>to solve the problem, why not use an xgawk include file? If
>you stick the follwing file in /usr/share/xgawk/fixopen.awk

Read the rest of the thread and you will understand why this is a
non-starter.

In article <48AC95D0.6060606@lsupcaemnt.com>,
Ed Morton <morton@lsupcaemnt.com> wrote:
>On 8/20/2008 7:48 AM, Kenny McCormack wrote:
>> Note to standards jockeys: No, this isn't a bug in your precious GAWK in
>> the usual "standards" sense. So, don't even bother.
>>
>> The fact is that, under certain conditions, it *is* a mis-feature, and it
>> would be nice to at least have the option of continuing.
>
>I agree. A fatal error in this situation stinks.

It's historical practice. Unix awk has worked this way since forever. IF
you don't need the filenames, you could always use

cat /proc/*/cmdline 2>/dev/null | awk 'program text'

In any case, it would not be a good idea to change gawk's default
behavior in this case.

Kenny: You are, of course, welcome to fork the gawk code base and create
a language that works to your specifications. You have my blessings.
--
Aharon (Arnold) Robbins arnold AT skeeve DOT com
P.O. Box 354 Home Phone: +972 8 979-0381
Nof Ayalon Cell Phone: +972 50 729-7545
D.N. Shimshon 99785 ISRAEL

On Friday 22 August 2008 13:34, Aharon Robbins wrote:

> Kenny: You are, of course, welcome to fork the gawk code base and create
> a language that works to your specifications. You have my blessings.

In this particular case, I think a command line switch to enable the
behavior could be enough.

On 8/22/2008 7:08 AM, pk wrote:
> On Friday 22 August 2008 13:34, Aharon Robbins wrote:
>
>
>>Kenny: You are, of course, welcome to fork the gawk code base and create
>>a language that works to your specifications. You have my blessings.
>
>
> In this particular case, I think a command line switch to enable the
> behavior could be enough.
>

Yeah, but I think it'd make sense to see the default behavior changed and just
do the abort if a new switch or the existing "--compat/traditional" switch was
being used.

On the other hand, I've never actually encountered this problem in real use so
it's just an opinion...

Ed.

On Friday 22 August 2008 14:27, Ed Morton wrote:

>> In this particular case, I think a command line switch to enable the
>> behavior could be enough.
>>
>
> Yeah, but I think it'd make sense to see the default behavior changed and
> just do the abort if a new switch or the existing "--compat/traditional"
> switch was being used.

That would break the scripts that relay on the traditional behavior, but
again I never wrote one that does and I don't know how many of those are
there around (if any).

> On the other hand, I've never actually encountered this problem in real
> use so it's just an opinion...

Mine too.

On Friday 22 August 2008 14:39, pk wrote:

> That would break the scripts that relay

Should be "rely", of course.

On Aug 21, 5:26*pm, gaze...@shell.xmission.com (Kenny McCormack)
wrote:
> Read the rest of the thread and you will understand why this is a
> non-starter.

That's a rather cryptic response. I have read the thread. I have
always found LD_PRELOAD solutions to be hacks that are difficult to
maintain.

The point is that xgawk already exists as a testbed for new gawk
features. It currently contains some features that could help with
your problem. And it would also be a good place to add a patch to
address this particular issue, if you feel the existing xgawk
facilities
aren't rich enough.

Regards,
Andy