gfortran and the family of fortran array types

Hi all,

Last month, a colleague (more familiar with C/C++) was bitten with the
following
fortran source code, in which an "explicit-shape with constant bounds"
array is passed to a subroutine expecting an "assumed-shape dummy
array" :

program test
integer y(2)
y(1) = 123
y(2) = 12
call mysub (y)
end program test
subroutine mysub (x)
integer :: x(
print *, x(1)
print *, x(2)
end

Compiled with gfortran (version 4.4.0 20080603), that raised
a "segmentation violation" error. I suggested him to put a star "*"
instead of the colon ":" in the mysub subroutine (which seems strange
for a non-fortran initiated...).
It worked, which surprised my colleague and
sounds like a bug in gfortran, but is it really ?

Now that I have a little more time (and another related bug), I tried
more experiments. I especially compared the behaviour of gfortran
against Intel Fortran v10.1.
I experienced that gfortran may fail for a source code in which Intel
Fortran runs fine, and Intel Fortran generates a compile-time error
for a source code with which gfortran fails with a "segmentation
violation".

To make it short, here are the results of my small experiments.
In all the tests, I kept the same caller "program test", modifying
only the
callee "subroutine mysub", in the definition of its dummy arguments.
I tried the combinations detailed in the following table. In
parenthesis are the
fortran array type that I got from the book "Fortran 90/2003" by
Chapman, 3d edition, p437.
Sorry for these handcrafted tables, which look prettier
with a mono-spaced font.

Caller Fortran array
INTEGER y(2) Explicit-shape with constant bounds (1)

Callee :
Callee Fortran array
INTEGER :: x(2) Explicit-shape with constant bounds
(1)
INTEGER :: x(*) Assumed-size dummy array (2.c)
INTEGER :: x( Assumed-shape dummy array (2.b)
INTEGER :: n + INTEGER :: x(n) Explicit-shape dummy array (2.a)
INTEGER, POINTER :: x( Defered-shape array (4.)
INTEGER, ALLOCATABLE :: x( Defered-shape array (4.)

For each source code, I used the the "-fdump-tree-original" option
included
in gfortran, which generates an intermediate source code.

In the intermediate source code generated by gfortran, in the caller
part,
the array is declared as :

integer(kind=4) y[2];

The following table is the declaration in the intermediate source code
corresponding to the callee. Of course, the last lines in this
table are really absurd, but I added them to try all possibilities.

Callee Fortran
array gfortran intermediate source code
INTEGER :: x(2) Explicit-shape with constant bounds
(1.) integer(kind=4)[2] * x
INTEGER :: x(*) Assumed-size dummy array
(2.c) integer(kind=4)[0:] * x
INTEGER :: n + INTEGER :: x(n) Explicit-shape dummy array
(2.a) integer(kind=4)[0.626] * x
INTEGER :: x( Assumed-shape dummy array
(2.b) struct array01_integer(kind=4) & x
INTEGER, POINTER :: x( Defered-shape array
(4.) struct array01_integer(kind=4) & x
INTEGER, ALLOCATABLE :: x( Defered-shape array
(4.) struct array01_integer(kind=4) & x

The only missing array type is the automatic array, which is useless
in my example because what I want is to pass data to the callee,
and not have a temporary array.

The table let appear two types of arrays :
1.,2.c, 2.a : integer(kind=4), with optionnal upper bound ([2], [0:]
or [0.626])
2.b, 4. : struct array01_integer(kind=4),
which corresponds to the fact that the struct contains
additionnal informations on the array, such as the lower/upper
bounds, the stride and the offset.

I was surprised to see that Intel Fortran is able to run the
example where the callee declares the assumed-shape dummy array (2.b)
"integer :: x(", displays good values but generates the
following message at compile-time :

Warning: Required interface for passing assumed shape array is missing
from original source [Y]

More specifically, I am surprised that Intel compiles the code and
runs it perfectly, althought gfortran fails with a segmentation
violation.

Another surprising fact is that gfortran seems to be
unable to detect type mismatch in the arrays.
Indeed, when the subroutine expects a type 2.b or type 4. dummy array
and it is given a type 1.,2.c or 2.a array, there is an obvious type
mismatch. Intel Fortran generates the following error at compile-time,
with a type 4. defered-shape pointer dummy array :

Error: A pointer dummy argument may only be argument associated with a
pointer.

and with a type 4. defered-shape allocatable dummy array :

Error: An allocatable dummy argument may only be argument associated
with an allocatable actual argument.

With gfortran, I get a "segmentation violation" error (ouch !).

That leads to the following questions.

Don't all fortran compilers use the same principles to manage arrays,
that is,
what exactly is written in the fortran standard with respect to array
compatibilities
in the caller and the callee ?
Why does gfortran does not detect array type mismatch problems at
compile-time,
and, instead, fails at run-time, that is, is this a bug in gfortran or
not ?

Regards,

Michaël

relaxmike wrote:
....
> Last month, a colleague ... was bitten with the following
> fortran source code, in which an "explicit-shape with constant bounds"
> array is passed to a subroutine expecting an "assumed-shape dummy
> array" :
>
> program test
> integer y(2)
> y(1) = 123
> y(2) = 12
> call mysub (y)
> end program test
> subroutine mysub (x)
> integer :: x(
> print *, x(1)
> print *, x(2)
> end
>
> Compiled with gfortran (version 4.4.0 20080603), that raised
> a "segmentation violation" error. I suggested him to put a star "*"
> instead of the colon ":" in the mysub subroutine...
> It worked, which surprised my colleague and
> sounds like a bug in gfortran, but is it really ?

No, the change is from an assumed-shape to an assumed-size dummy array
which obtains it's dimensions from those of the actual argument.

The assumed-shape array _requires_ and explicit interface to pass the
associated information on shape and size. The above code does not have
the explicit interface either provided manually or by being included in
a module (the recommended technique).

That gfortran didn't find the problem at compilation is a result of
Fortran "compilation units" being a program and function or subroutine,
_NOT_ a file. Some compilers may look at global characteristics within
a file, but that's an implementation detail, not Standard.

The fact that IVF did it successfully is owing to a feature of the Intel
compiler in that it will generate missing explicit interfaces
"automagically" (I think by default unless the switch is explicitly
negated) so it did transparently supply the missing interface thereby
hiding the error (but the warning message told you what went on if you
were paying attention).

The short answer is your colleague made an error of not providing the
explicit interface and that it is required is specified by the Standard.

Again, the recommended way to avoid the problem and find many other
programming mistakes is to use modules religiously.

It seems futile to even consider all the permutations...

--

On 2008-08-25, relaxmike <michael.baudin@gmail.com> wrote:
> That leads to the following questions.
>
> Don't all fortran compilers use the same principles to manage arrays,
> that is,
> what exactly is written in the fortran standard with respect to array
> compatibilities
> in the caller and the callee ?

Well, yes and no. No in the sense that a compiler is not required to
detect arbitrary interface mismatches, since that would break the
separate compilation model used by most compilers.

> Why does gfortran does not detect array type mismatch problems at
> compile-time,
> and, instead, fails at run-time, that is, is this a bug in gfortran or
> not ?

It's a bug as in would-be-nice-to-have, not a standard violation. If you
want to help out, search the gfortran mailing list and bugzilla for the
"multiple decls per symbol" problem, which needs to be solved before
various interprocedural analysis stuff (such as checking interfaces for
procedures with implicit interfaces, which is the root cause for the
problems you describe) can be implemented.

From the end-user perspective, the "proper" solution is to always use
explicit interfaces, where this problem is already solved as the
standard requires.


--
JayBee

Thank you very much for your both messages.
Now I clearly understand what happens.

Back to the initial problem, I can implement
what is probably the solution used by Intel,
that is, the creation of a module which
contains the definition of the interface for the
subroutine (but Intel does that for me automatically).

module mymodule
interface
subroutine mysub (x)
integer :: x(
end subroutine mysub
end interface
end module mymodule
program test
use mymodule
integer y(2)
y(1) = 123
y(2) = 12
call mysub (y)
end program test
subroutine mysub (x)
integer :: x(
print *, x(1)
print *, x(2)
end subroutine mysub

When I look to the intermediate code generated
by gfortran, it appears that the data, which
is initially created in a basic data storage with
type integer(kind=4), is transferred to a struct, and
only the struct is passed to the subroutine so that it matches
the interface :

test ()
{
integer(kind=4) y[2];
static integer(kind=4) options.0[7] = {68, 255, 0, 0, 0, 1, 0};
_gfortran_set_options (7, (void *) &options.0);
y[0] = 123;
y[1] = 12;
{
struct array01_integer(kind=4) parm.1;
parm.1.dtype = 265;
parm.1.dim[0].lbound = 1;
parm.1.dim[0].ubound = 2;
parm.1.dim[0].stride = 1;
parm.1.data = (void *) &y[0];
parm.1.offset = -1;
mysub (&parm.1);
}
}

I have two more possibilities to solve my
problem :
- use a contains in the program, so that the
subroutine mysub is a internal subroutine of the
program,
- put the body of the subroutine directly into the
module.

It also clearly shows why there is a problem with
the interoperability with C, see the chasm project :
http://chasm-interop.sourceforge.net/

Regards,

Michaël

relaxmike wrote:

>
> I have two more possibilities to solve my
> problem :
> - use a contains in the program, so that the
> subroutine mysub is a internal subroutine of the
> program,
> - put the body of the subroutine directly into the
> module.

The latter is, generally, the preferred methodology. Two main reasons for me:
1) You don't have to worry about updating the interface in two places if/when it does
change, and
2) You can then use that module in other calling procedures/programs.

> It also clearly shows why there is a problem with
> the interoperability with C

The problem being... what?

cheers,

paulv

The following is mainly hypothetical, since I had no
opportunity to experiment C/Fortran interoperability
with respect to fortran 90 arrays.

My reference is the fortran 2003 pdf written by
Patrick Corde, Hervé Delouis from IDRIS :

http://www.idris.fr/data/cours/lang/...rtran_2003.pdf

There is also "The New Features of Fortran 2003" by John Reid,
especially p36 and p37 :

ftp://ftp.nag.co.uk/sc22wg5/N1551-N1600/N1579.pdf

I look at the chasm file GNU_dv.h which describes the gfortran
implementation of a fortran 90 array, based on a C struct :

http://chasm-interop.cvs.sourceforge....2&view=markup

typedef struct dope_vec_GNU_ {
void* base_addr; /* base address of the array */
void* base; /* base offset */
size_t dtype; /* elem_size, type (3 bits) and rank (3 bits)
*/
struct {
size_t stride_mult; /* distance between successive elements
(elements) */
size_t lower_bound; /* first array index for a given dimension */
size_t upper_bound; /* last array index for a given dimension */
} dim[7];
} dope_vec_GNU;

If I understand it well, only the field base_addr corresponds to
a C array, as a pointer to the first element of the array.
Of course, the implementation may depend on the compiler,
since the fortran 90 standard define only the features of the
array, not the real implementation. In fact, the fortran 90
standard introduced a problem for interoperability for
fortran 90 arrays with C (there was no problem for fortran 77
arrays). The fortran 2003 standard solves that problem, by
introducing
several operators, such as C_LOC.

Suppose that a fortran 90 program allocates an array and
want to pass that array to a C routine, as it is done on
page 33/174 of the Corde&Delouis pdf.

The fortran 90 array is declared as :

real(kind=C_FLOAT), dimension(:,, allocatable, target :: mat

To be manageable by the C routine, a conversion must be done so that
only the base_addr field is passed to the C routine. The fortran 2003
introduces the C_LOC operator for that purpose :

"C_LOC(X) is an inquiry function that returns the C address of an
object."

On the fortran side, the C function is passed C_LOC(mat). On the C
side,
one receives a float *vec, from which the matrix can be
accessed with some index algebra.

In the "Example of C calling Fortran" of the Reid pdf, p37, on the C
side, the array delta is declared with :

double delta[]

On the fortran side, the array is declared as :

REAL (C_DOUBLE),DIMENSION(*),INTENT(IN) :: DELTA

that is, with the fortran 77 syntax which corresponds to a
"assumed-size dummy array".

I did not understand these problems before I was able to
experiment with the "-fdump-tree-original" option
in gfortran. But what I just wrote may still contain several
confusions, which other experts in this forum may want to
fix.

Regards,

Michaël

relaxmike <michael.baudin@gmail.com> wrote:

> The fortran 90 array is declared as :
>
> real(kind=C_FLOAT), dimension(:,, allocatable, target :: mat
>
> To be manageable by the C routine, a conversion must be done so that
> only the base_addr field is passed to the C routine.

That is not so. Of course, prior to f2003, the standard didn't provide
any way at all to call C (or have the above C_FLOAT constant), but that
aside, there would be no problems at all with passing the above array to
C. It does *NOT* require the use of C_LOC.

You appear to be assuming that all arrays are inherently passed as dope
vectors, which is not so. I suspect that the confusion partly arises
from looking at argument passing conventions exactly backwards. To see
how an array is passed to a particular subroutine, you do *NOT* start
out by looking at the array in question. You start out by looking at the
interface of the subroutine. That's what defines the expected passing
mechanism.

If the subroutine has an assumed shape, pointer array, or allocatable
array dumy argument, then yes, something like a dope vector is likely to
get passed. But then a C function isn't going to have arguments like
that (unless someone went to a lot of work - and compiler-specific work
- just to make it that way).

If you just properly describe the interface of the C function, the array
will be passed properly, which is to say that just the starting address
will be passed. It does *NOT* matter that this array is allocatable.
That's a common error that people make, even without reference to C.
Just because an actual argument is allocatable, that doesn't mean that
the dummy argument has to be. (If a dummy argument is allocatable, then
th eactual argument must be, but that's a different matter). Once you
have allocated an allocatable array, it is now an array as "good" as any
other and it can be passed as an actual argument to procedures (or C
functions) that don't even need to know that it is allocatable. The same
applies to pointer arrays, I might add, though there is one slight
caveat to that case (involving the possibility that a pointer array
might point to a non-contiguous slice).

> The fortran 2003
> introduces the C_LOC operator for that purpose :

No. That is not the purpose of C_LOC. C_LOC is not needed at all in this
case. While you could use it, it is just redundant and needlessly
complicated for this case.

> In the "Example of C calling Fortran" of the Reid pdf, p37, on the C
> side, the array delta is declared with :
>
> double delta[]
>
> On the fortran side, the array is declared as :
>
> REAL (C_DOUBLE),DIMENSION(*),INTENT(IN) :: DELTA
>
> that is, with the fortran 77 syntax which corresponds to a
> "assumed-size dummy array".

That's a different scenario from Fortran calling C.

--
Richard Maine | Good judgement comes from experience;
email: last name at domain . net | experience comes from bad judgement.
domain: summertriangle | -- Mark Twain

Richard Maine wrote:

> relaxmike <michael.baudin@gmail.com> wrote:
(big snip)

>>The fortran 2003
>>introduces the C_LOC operator for that purpose :

> No. That is not the purpose of C_LOC. C_LOC is not needed at all in this
> case. While you could use it, it is just redundant and needlessly
> complicated for this case.

If you used C_LOC() it would pass the result of C_LOC in
the usual Fortran way, most likely the address of a temporary
variable containing the result. That might be useful if the
C routine wanted a pointer to the array (pointer to pointer to
the actual data, or (double**) in C notation).

You could also %VAL(C_LOC(array)), but that would be strange,
and require %VAL support.

-- glen

Richard Maine wrote:
| relaxmike <michael.baudin@gmail.com> wrote:
|
|| The fortran 90 array is declared as :
||
|| real(kind=C_FLOAT), dimension(:,, allocatable, target :: mat
||
|| To be manageable by the C routine, a conversion must be done so that
|| only the base_addr field is passed to the C routine.
|
| That is not so. Of course, prior to f2003, the standard didn't provide
| any way at all to call C (or have the above C_FLOAT constant), but that
| aside, there would be no problems at all with passing the above array to
| C. It does *NOT* require the use of C_LOC.
|
| You appear to be assuming that all arrays are inherently passed as dope
| vectors, which is not so. I suspect that the confusion partly arises
| from looking at argument passing conventions exactly backwards. To see
| how an array is passed to a particular subroutine, you do *NOT* start
| out by looking at the array in question. You start out by looking at the
| interface of the subroutine. That's what defines the expected passing
| mechanism.

But other than that, his post was pretty much spot on.

From a practical compiler implementation standpoint, interfacing F90 with C
(subtleties like CHARACTER aside) works pretty much as interfacing F90 with
F77. Neither C nor F77 know about assumed-shape, allocatable or pointer
arguments (and associated dope vector) -- all they "know" is an address of
the first element. In C it is expressed more explicitly with * operator,
while in F77 one uses an assumed-size (or explicit-shape) dummy.

Thus, (attempting to express Richard's point somewhat differently): a F90
compiler may (but need not) mantain a dope vector internally, as a bookkeeping
device; but whether it will pass it or not depends *only* on the called routine
interface/prototype. If it's a f90 assumed-shape/allocatable/pointer argument
(which, remind you, need an /explicit interface/), it will pass it (and
create one on the fly if necessary, such as in CALL FOO(A(1:8:2)) or
CALL Foo([12,8,2008,12,53]) ). Otherwise, it will "just" pass the address.

With F2003 C interop, C_LOC does that conversion explicitly and is
guaranteed to be portable. Before that, one would just pass the array
(assumed-size assumed by compiler), relying on both compilers to do
the right thing (they would most often do, but it wasn't guaranteed
by the Standard).

--
Jugoslav
___________
www.xeffort.com

Please reply to the newsgroup.
You can find my real e-mail on my home page above.

Jugoslav Dujic <jdujic@yahoo.com> wrote:

> With F2003 C interop, C_LOC does that conversion explicitly and is
> guaranteed to be portable. Before that, one would just pass the array
> (assumed-size assumed by compiler), relying on both compilers to do
> the right thing (they would most often do, but it wasn't guaranteed
> by the Standard).

Just to clarify (I hope)... withF2003 BIND(C) on an appropriate
interface the result is guaranteed to be portable. One could misread the
above as implying that C_LOC is needed for a guarantee of portability.
It isn't. In fact, it is the BIND(C) that gives you the portability
guarantee - not the C_LOC. Without the BIND(C), the C_LOC stuff wouldn't
be portable... and indeed, as Glen notes, would likely not work without
extra hacks; you'll end up passing the C_LOC result by reference.

--
Richard Maine | Good judgement comes from experience;
email: last name at domain . net | experience comes from bad judgement.
domain: summertriangle | -- Mark Twain