External pre-processing required? Plotting delta of data from 2 files

Hello.

Suppose there are the following two data files:

a.csv:
"svn","2008-08-21 06:52:38","14.143","14.079"
"svn","2008-08-21 07:06:46","14.125","12.999"
"svn","2008-08-21 08:49:24","38.832","27.451"

b.csv:
"fh","2008-08-21 13:47:36","26.234","17.234"
"fh","2008-08-21 14:09:40","19.922","20.125"
"fh","2008-08-21 14:38:25","16.766","13.966"

I now plot that using:

ab.gnuplot:
set datafile separator ","
set xdata time
set timefmt "%Y-%m-%d %H:%M:%S"
set xrange [ "2008-08-21 0:0" : * ]
set format x "%d.%m.%y\n%H:%M"

plot 'a.csv' using 2$3+$4) title 'S' with lines, \
'b.csv' using 2$3+$4) title 'F' with lines

I would now like to plot the delta between ($3+$4) from a.csv and
($3+$4) from b.csv. Ie. the values would be:

(14.143 + 14.079) - (26.234 + 17.234)
(14.125 + 12.999) - (19.922 + 20.125)
(38.832 + 27.451) - (16.766 + 13.966)

->

-15.246
-12.923
35.551

Can I do such a thing directly using Gnuplot, or would I need to do
those calculations outside of Gnuplot?

Thanks a lot,

Michael

On Aug 25, 4:58 am, Michael Schmarck <usenet-mich...@schmarck.cn>
wrote:
> I would now like to plot the delta between ($3+$4) from a.csv and
> ($3+$4) from b.csv. Ie. the values would be:
>
> (14.143 + 14.079) - (26.234 + 17.234)
> (14.125 + 12.999) - (19.922 + 20.125)
> (38.832 + 27.451) - (16.766 + 13.966)

> Can I do such a thing directly using Gnuplot, or would I need to do
> those calculations outside of Gnuplot?

i'm wondering about a similar question - in a related post titled
"Subtract two datasets "on the fly" ?" from 2003 :

http://tinyurl.com/65maw3 [comp.graphics.apps.gnuplot]

Hans-Bernhard suggested using command line tools in gnuplot as :

"< paste file1.dat file2.dat" using 1$4-$2)

.... works great when files are "one-to-one". i'm interested in
knowing how to go beyond that without, as you say, going "outside of
Gnuplot"... but actually, that should work for you, right?

HTH

-bryan

On Aug 25, 12:28*pm, bryan <bryanlep...@gmail.com> wrote:
> "< paste file1.dat file2.dat" using 1$4-$2)
>
> ... works great when files are "one-to-one". *i'm interested in
> knowing how to go beyond that without, as you say, going "outside of
> Gnuplot"

i was just experimenting a bit with this - indeed, you can put
multiple pipes/vertical bars "|" in the gnuplot command, so

"< paste file1.dat file2.dat | awk '{print $1/$2}'" using 1:2 [...]

the "<" issues a popen function that is in the gnuplot source code. i
am uncertain if more than one can be issued per `plot`. after one is
given, you need "|"

-bryan

bryan <bryanlepore@gmail.com> wrote:

> On Aug 25, 4:58 am, Michael Schmarck <usenet-mich...@schmarck.cn>
> wrote:
>> I would now like to plot the delta between ($3+$4) from a.csv and
>> ($3+$4) from b.csv. Ie. the values would be:
>>
>> (14.143 + 14.079) - (26.234 + 17.234)
>> (14.125 + 12.999) - (19.922 + 20.125)
>> (38.832 + 27.451) - (16.766 + 13.966)
>
>> Can I do such a thing directly using Gnuplot, or would I need to do
>> those calculations outside of Gnuplot?
>
> i'm wondering about a similar question - in a related post titled
> "Subtract two datasets "on the fly" ?" from 2003 :
>
> http://tinyurl.com/65maw3 [comp.graphics.apps.gnuplot]
>
> Hans-Bernhard suggested using command line tools in gnuplot as :
>
> "< paste file1.dat file2.dat" using 1$4-$2)
>
> ... works great when files are "one-to-one". i'm interested in
> knowing how to go beyond that without, as you say, going "outside of
> Gnuplot"... but actually, that should work for you, right?

I thought about that, but no, it won't work for me, as it occured to
me after I tought about it some more

My problem is, that I take samples on two systems (fh, svn) at not
precisely the same point in time. In reality, I take a sample every 30
minutes. But it can very well be, that one of the systems doesn't supply
data at a certain point in time.

So I guess I'd need a "graphical" solution, so to speak.

Have a look at the data:

a.csv:
"svn","2008-08-21 06:52:38","14.143","14.079"
"svn","2008-08-21 07:06:46","14.125","12.999"
"svn","2008-08-21 08:49:24","38.832","27.451"

b.csv:
"fh","2008-08-21 13:47:36","26.234","17.234"
"fh","2008-08-21 14:09:40","19.922","20.125"
"fh","2008-08-21 14:38:25","16.766","13.966"

The data in the 2nd column represents the point in time when the data
was taken. Suppose I had the following data instead:

a.csv:
"svn","2008-08-21 06:52:38","14.143","14.079"
"svn","2008-08-21 12:06:46","14.125","12.999"
"svn","2008-08-21 18:49:24","38.832","27.451"

b.csv:
"fh","2008-08-21 04:47:36","26.234","17.234"
"fh","2008-08-21 09:09:40","19.922","20.125"
"fh","2008-08-21 14:38:25","16.766","13.966"

I changed the point in times when the data was taken. If you plot that,
you get:

http://michael-schmarck.share.s3.ama...gag/ab.gnuplot
http://michael-schmarck.share.s3.ama...om/cgag/ab.png
http://michael-schmarck.share.s3.ama...com/cgag/a.csv
http://michael-schmarck.share.s3.ama...com/cgag/b.csv

Now, please have a look at the ab.png image (or plot the stuff yourself
*G*). As you can see from the data, I do not have a data point taken
at 08:00:00. But because I drawed lines, you can "guesss" that svn (red
line) had a value of about 28 at that time and fh (green) had about 42.

So I suppose I'd either need gnuplot to tell me what Y values the lines
had at a given X value, even for X values which haven't been supplied
(ie. the 08:00:00 example), or I'd need to preprocess the data outside
of gnuplot and do the "geometrical calculations".

Cheers,
Michael

On Aug 26, 8:54*am, Michael Schmarck <usenet-mich...@schmarck.cn>
wrote:
> bryan <bryanlep...@gmail.com> wrote:
> > On Aug 25, 4:58 am, Michael Schmarck <usenet-mich...@schmarck.cn>
> > wrote:
> >> I would now like to plot the delta between ($3+$4) from a.csv and
> >> ($3+$4) from b.csv. Ie. the values would be:
>
> >> (14.143 + 14.079) - (26.234 + 17.234)
> >> (14.125 + 12.999) - (19.922 + 20.125)
> >> (38.832 + 27.451) - (16.766 + 13.966)
>
> >> Can I do such a thing directly using Gnuplot, or would I need to do
> >> those calculations outside of Gnuplot?
>
> > i'm wondering about a similar question - in a related post titled
> > "Subtract two datasets "on the fly" ?" from 2003 *:
>
> >http://tinyurl.com/65maw3[comp.graphics.apps.gnuplot]
>
> > Hans-Bernhard suggested using command line tools in gnuplot as :
>
> > "< paste file1.dat file2.dat" using 1$4-$2)
>
> > ... works great when files are "one-to-one". *i'm interested in
> > knowing how to go beyond that without, as you say, going "outside of
> > Gnuplot"... but actually, that should work for you, right?
>
> I thought about that, but no, it won't work for me, as it occured to
> me after I tought about it some more
>
> My problem is, that I take samples on two systems (fh, svn) at not
> precisely the same point in time. In reality, I take a sample every 30
> minutes. But it can very well be, that one of the systems doesn't supply
> data at a certain point in time.
>
> So *I guess I'd need a "graphical" solution, so to speak.
>
> Have a look at the data:
>
> * * * * a.csv:
> * * * * "svn","2008-08-21 06:52:38","14.143","14.079"
> * * * * "svn","2008-08-21 07:06:46","14.125","12.999"
> * * * * "svn","2008-08-21 08:49:24","38.832","27.451"
>
> * * * * b.csv:
> * * * * "fh","2008-08-21 13:47:36","26.234","17.234"
> * * * * "fh","2008-08-21 14:09:40","19.922","20.125"
> * * * * "fh","2008-08-21 14:38:25","16.766","13.966"
>
> The data in the 2nd column represents the point in time when the data
> was taken. Suppose I had the following data instead:
>
> * * * * a.csv:
> * * * * "svn","2008-08-21 06:52:38","14.143","14.079"
> * * * * "svn","2008-08-21 12:06:46","14.125","12.999"
> * * * * "svn","2008-08-21 18:49:24","38.832","27.451"
>
> * * * * b.csv:
> * * * * "fh","2008-08-21 04:47:36","26.234","17.234"
> * * * * "fh","2008-08-21 09:09:40","19.922","20.125"
> * * * * "fh","2008-08-21 14:38:25","16.766","13.966"
>
> I changed the point in times when the data was taken. If you plot that,
> you get:
>
> * * * *http://michael-schmarck.share.s3.ama...gag/ab.gnuplot
> * * * *http://michael-schmarck.share.s3.ama...om/cgag/ab.png
> * * * *http://michael-schmarck.share.s3.ama...com/cgag/a.csv
> * * * *http://michael-schmarck.share.s3.ama...com/cgag/b.csv
>
> Now, please have a look at the ab.png image (or plot the stuff yourself
> *G*). As you can see from the data, I do not have a data point taken
> at 08:00:00. But because I drawed lines, you can "guesss" that svn (red
> line) had a value of about 28 at that time and fh (green) had about 42.
>
> So I suppose I'd either need gnuplot to tell me what Y values the lines
> had at a given X value, even for X values which haven't been supplied
> (ie. the 08:00:00 example), or I'd need to preprocess the data outside
> of gnuplot and do the "geometrical calculations".
>
> Cheers,
> Michael

So you want the diff between one real data point in one set and an
artificial interpolated value at the same x from the other.

To avoid creating garbage , unjustifiable output I think you really
need to do this sort of treatment outside of gnuplot , then plot the
results.

The only case where I can see gnuplot being helpful is if you can fit
a mathematical expression using fit command then plot it to a file as
a table (see help table).

You could just duplicate your example to a file , that way gnuplot
will output x in seconds since 1/1/2000 and save you the fag of
parsing the time data. It would then be trivial to do linear interp
between pts to get what you want.

/P