Fortran vs. Matlab on linear algebra

James Giles wrote:
> dpb wrote:
> ...
>> But, as has been noted, once one gets to the core routines in ML it's
>> likely they're about as good as it gets w/o really serious work unless
>> have multi-processor and can take advantage of them.
>
>
> I understand you intend ML to be an abbreviation for Matlab in
> this context. But there is a language called ML. Probably better
> not to confuse them.

I'll try to remember that for clf but wasn't aware of that, thanks --
"ML" is common vernacular on c.s-s.m so came naturally...

--

On 2008-08-06 19:52:14 -0300, nospam@see.signature (Richard Maine) said:

> James Giles <jamesgiles@worldnet.att.net> wrote:
>
>> dpb wrote:
>> ...
>>> But, as has been noted, once one gets to the core routines in ML it's
>>> likely they're about as good as it gets w/o really serious work unless
>>> have multi-processor and can take advantage of them.
>>
>> I understand you intend ML to be an abbreviation for Matlab in
>> this context. But there is a language called ML. Probably better
>> not to confuse them.
>
> Is that Mock Lisp or something else? Hmm. A quick google answers my own
> question. It appears to be something else. But that just reinforces your
> point about the confusion, as Mock Lisp was one of the things that
> occurred to me when I read your post.

Standard ML of New Jersey is one variant on the family. ML is MetaLanguage.
You are more likely to see CAML or OCaml which is Object Oriented cambridge
ML if I have it straight. The gentleman from Flying Frog Consulting (sic)
will undoubedly set us straight. The Microsoft version of OCaml is F#.

Last time I played with Matlab (about a year ago), it already had an
option to use multithreading (SMP). It was not enabled by default
though. Could it be that in your case Matlab had this option enabled?
That would explain such a dramatic difference.
Also, I would recommend Goto BLAS or ATLAS as a free alternative to
INTEL MKL.

Victor.

On Aug 5, 7:45*pm, Gib Bogle <g.bo...@auckland.no.spam.ac.nz> wrote:
> I have revisited a topic from 2001, using ifort 10.1.011 and Matlab 7.5.0, to compare the speed of
> matmul() with the speed of a matrix multiply in Matlab.
>
> Here is the code:
>
> ! To compare speed of Matlab and Fortran on matrix algebra
> program speed
>
> REAL(8) :: a(500,500), b(500,500), c(500,500), t1, t2
> integer, parameter :: N = 20
> integer :: i
>
> call RANDOM_NUMBER(a)
> call RANDOM_NUMBER(b)
>
> call cpu_time(t1)
>
> do i = 1,N
> * * *a(1,1) = a(1,1) + 0.1
> * * *c = MATMUL(a,b)
> * * *write(*,*) c(1,1)
> enddo
> call cpu_time(t2)
>
> write(*,*) 'Time: ',t2-t1
> end program
>
> ! Matlab code
> !function speed
> !A = rand(500);
> !B = rand(500);
> !C = zeros(500);
> !tic
> !for i = 1:20
> ! * *A(1,1) = A(1,1) + 0.1;
> ! * *C = A * B;
> ! * *C(1,1)
> !end
> !toc
>
> I was a bit surprised to find that Matlab takes about 0.92 sec while ifort takes 2.73 sec (this is
> the speed with O0, O1, or O2, while O3 takes 12.6 sec !?!)
> I am running on a recent Intel quad core Windows PC.
> ------------ And now a word from our sponsor ------------------
> Do your users want the best web-email gateway? Don't let your
> customers drift off to free webmail services install your own
> web gateway!
> -- *Seehttp://netwinsite.com/sponsor/sponsor_webmail.htm*----

Fly Away wrote:
> Last time I played with Matlab (about a year ago), it already had an
> option to use multithreading (SMP). It was not enabled by default
> though. Could it be that in your case Matlab had this option enabled?
> That would explain such a dramatic difference.
> Also, I would recommend Goto BLAS or ATLAS as a free alternative to
> INTEL MKL.

I checked, that timing is with no multithreading. It goes faster when
more than one CPU is used, although 4 is slightly slower than 3 - a
common result on the quad core, I've observed. Memory bus congestion
and/or cache contention is the cause, apparently.

On Aug 7, 3:43 am, Gib Bogle <bo...@ihug.too.much.spam.co.nz> wrote:
> Fly Away wrote:
> > Last time I played with Matlab (about a year ago), it already had an
> > option to use multithreading (SMP). It was not enabled by default
> > though. Could it be that in your case Matlab had this option enabled?
> > That would explain such a dramatic difference.
> > Also, I would recommend Goto BLAS or ATLAS as a free alternative to
> > INTEL MKL.
>
> I checked, that timing is with no multithreading. It goes faster when
> more than one CPU is used, although 4 is slightly slower than 3 - a
> common result on the quad core, I've observed. Memory bus congestion
> and/or cache contention is the cause, apparently.
Are you using an Intel machine? Seems like Matlab (7.01 and above)
also use the MKL underneath:

http://www.mathworks.com/support/sol...lution=1-JDIO3

"In MATLAB 7.0.1 (R14SP1) on computers that use Intel processors, the
default BLAS library is the Math Kernel Library (MKL) BLAS provided by
Intel."

That may explain why matmul was slower? With Fortran it is always a
good idea to use the vendor supplied/tuned blas/lapack (MKL for Intel,
ACML for AMD, Sunperf for Sun and so on). The implementations of
matmul may not always be highly optimized though there is little
excuse for not doing it.

Moreover using blas in MKL is also very simple, specially with the
Fortran 95 wrappers that come along with it. For example
"C=matmul(A,B)" is easily replaced by "call gemm(A,B,C)"

On Thu, 07 Aug 2008 07:11:30 -0700, rusi_pathan wrote:

> On Aug 7, 3:43 am, Gib Bogle <bo...@ihug.too.much.spam.co.nz> wrote:
>> Fly Away wrote:
>> > Last time I played with Matlab (about a year ago), it already had an
>> > option to use multithreading (SMP). It was not enabled by default
>> > though. Could it be that in your case Matlab had this option enabled?
>> > That would explain such a dramatic difference.
>> > Also, I would recommend Goto BLAS or ATLAS as a free alternative to
>> > INTEL MKL.
>>
>> I checked, that timing is with no multithreading. It goes faster when
>> more than one CPU is used, although 4 is slightly slower than 3 - a
>> common result on the quad core, I've observed. Memory bus congestion
>> and/or cache contention is the cause, apparently.
> Are you using an Intel machine? Seems like Matlab (7.01 and above)
> also use the MKL underneath:
>
> http://www.mathworks.com/support/sol...lution=1-JDIO3
>
> "In MATLAB 7.0.1 (R14SP1) on computers that use Intel processors, the
> default BLAS library is the Math Kernel Library (MKL) BLAS provided by
> Intel."
>
> That may explain why matmul was slower? With Fortran it is always a
> good idea to use the vendor supplied/tuned blas/lapack (MKL for Intel,
> ACML for AMD, Sunperf for Sun and so on). The implementations of
> matmul may not always be highly optimized though there is little
> excuse for not doing it.
>
> Moreover using blas in MKL is also very simple, specially with the
> Fortran 95 wrappers that come along with it. For example
> "C=matmul(A,B)" is easily replaced by "call gemm(A,B,C)"

Sadly the MKL is sold separately, rather than bundled into Intel's
Fortran. There was a time when it was a free download, but that was many
years ago. We just have to accept that Intel wish to hobble their compiler
to sell us additional products.

On Aug 7, 5:35 pm, none <n...@none.net> wrote:
> Sadly the MKL is sold separately, rather than bundled into Intel's
> Fortran. There was a time when it was a free download, but that was many
> years ago. We just have to accept that Intel wish to hobble their compiler
> to sell us additional products.

It is still a free download for non commercial use.

rusi_pathan wrote:
> On Aug 7, 3:43 am, Gib Bogle <bo...@ihug.too.much.spam.co.nz> wrote:
>> Fly Away wrote:
>>> Last time I played with Matlab (about a year ago), it already had an
>>> option to use multithreading (SMP). It was not enabled by default
>>> though. Could it be that in your case Matlab had this option enabled?
>>> That would explain such a dramatic difference.
>>> Also, I would recommend Goto BLAS or ATLAS as a free alternative to
>>> INTEL MKL.
>> I checked, that timing is with no multithreading. It goes faster when
>> more than one CPU is used, although 4 is slightly slower than 3 - a
>> common result on the quad core, I've observed. Memory bus congestion
>> and/or cache contention is the cause, apparently.
> Are you using an Intel machine? Seems like Matlab (7.01 and above)
> also use the MKL underneath:
>
> http://www.mathworks.com/support/sol...lution=1-JDIO3
>
> "In MATLAB 7.0.1 (R14SP1) on computers that use Intel processors, the
> default BLAS library is the Math Kernel Library (MKL) BLAS provided by
> Intel."
>
> That may explain why matmul was slower? With Fortran it is always a
> good idea to use the vendor supplied/tuned blas/lapack (MKL for Intel,
> ACML for AMD, Sunperf for Sun and so on). The implementations of
> matmul may not always be highly optimized though there is little
> excuse for not doing it.
>
> Moreover using blas in MKL is also very simple, specially with the
> Fortran 95 wrappers that come along with it. For example
> "C=matmul(A,B)" is easily replaced by "call gemm(A,B,C)"

Yes, Intel quad core.

On Aug 6, 3:45*am, Gib Bogle <g.bo...@auckland.no.spam.ac.nz> wrote:
> I have revisited a topic from 2001, using ifort 10.1.011 and Matlab 7.5.0, to compare the speed of
> matmul() with the speed of a matrix multiply in Matlab.
>
> Here is the code:
>
> ! To compare speed of Matlab and Fortran on matrix algebra
> program speed
>
> REAL(8) :: a(500,500), b(500,500), c(500,500), t1, t2
> integer, parameter :: N = 20
> integer :: i
>
> call RANDOM_NUMBER(a)
> call RANDOM_NUMBER(b)
>
> call cpu_time(t1)
>
> do i = 1,N
> * * *a(1,1) = a(1,1) + 0.1
> * * *c = MATMUL(a,b)
> * * *write(*,*) c(1,1)
> enddo
> call cpu_time(t2)
>
> write(*,*) 'Time: ',t2-t1
> end program
>
> ! Matlab code
> !function speed
> !A = rand(500);
> !B = rand(500);
> !C = zeros(500);
> !tic
> !for i = 1:20
> ! * *A(1,1) = A(1,1) + 0.1;
> ! * *C = A * B;
> ! * *C(1,1)
> !end
> !toc
>
> I was a bit surprised to find that Matlab takes about 0.92 sec while ifort takes 2.73 sec (this is
> the speed with O0, O1, or O2, while O3 takes 12.6 sec !?!)
> I am running on a recent Intel quad core Windows PC.

Hi,

I've tried it on my laptop with an intel core 2 duo processor,
2GHz running linux, playing Pink Floyd - the Wall in the background

ifort: 10.1.017
mkl: 10.0.3.020
matlab: 7.4.0.287 (R2007a)
octave: 3.0.0

My results are:

Fortran using matmul:
time [sec] optimization
18.9 -O0
5.0 -O1
2.4 -O2
1.0 -O3

matlab:
time [sec]
0.76->0.81

octave (w. default atlas?):
time [sec]
1.5

Fortran using blas95 (gemm)
time [sec] optimization
0.77->0.84 -O0,-O1,-O2,-O3

So, for this program on my laptop, Matlab and fortran+mkl are
practically the same, which is not surprising as they both use mkl.

I have however seen fortran beat matlab by a factor of 1000 on a
nontrivial problem (25 minutes vs 1.5 sec).


Wim

wim wrote:
> On Aug 6, 3:45 am, Gib Bogle <g.bo...@auckland.no.spam.ac.nz> wrote:
>> I have revisited a topic from 2001, using ifort 10.1.011 and Matlab 7.5.0, to compare the speed of
>> matmul() with the speed of a matrix multiply in Matlab.
>>
>> Here is the code:
>>
>> ! To compare speed of Matlab and Fortran on matrix algebra
>> program speed
>>
>> REAL(8) :: a(500,500), b(500,500), c(500,500), t1, t2
>> integer, parameter :: N = 20
>> integer :: i
>>
>> call RANDOM_NUMBER(a)
>> call RANDOM_NUMBER(b)
>>
>> call cpu_time(t1)
>>
>> do i = 1,N
>> a(1,1) = a(1,1) + 0.1
>> c = MATMUL(a,b)
>> write(*,*) c(1,1)
>> enddo
>> call cpu_time(t2)
>>
>> write(*,*) 'Time: ',t2-t1
>> end program
>>
>> ! Matlab code
>> !function speed
>> !A = rand(500);
>> !B = rand(500);
>> !C = zeros(500);
>> !tic
>> !for i = 1:20
>> ! A(1,1) = A(1,1) + 0.1;
>> ! C = A * B;
>> ! C(1,1)
>> !end
>> !toc
>>
>> I was a bit surprised to find that Matlab takes about 0.92 sec while ifort takes 2.73 sec (this is
>> the speed with O0, O1, or O2, while O3 takes 12.6 sec !?!)
>> I am running on a recent Intel quad core Windows PC.
>
> Hi,
>
> I've tried it on my laptop with an intel core 2 duo processor,
> 2GHz running linux, playing Pink Floyd - the Wall in the background
>
> ifort: 10.1.017
> mkl: 10.0.3.020
> matlab: 7.4.0.287 (R2007a)
> octave: 3.0.0
>
> My results are:
>
> Fortran using matmul:
> time [sec] optimization
> 18.9 -O0
> 5.0 -O1
> 2.4 -O2
> 1.0 -O3

For some reason, on my 4-core Windows XP box ifort with -O3 is much
slower than with -O2.