Error in "double = double - double" statement : basic.visual

I am trying to compute the differnece between two dwords in visual basic and
keep getting the wrong value being computed. I have the following structure
defind:

Public Type HighLowQuote
TradeDate As String
OpenPrice As Double
HighPrice As Double
LowPrice As Double
ClosePrice As Double
Volume As Long
AdjClosePrice As Double
AmtChange As Double
End Type

I create an instance of this type as 'q'. I then do the following:
q.highprice = variant
q.lowprice = variant

at this point the locals variable window shows the values as:
q.highprice = 4.1
q.lowprice = 4

I then fall through the following code:
q.AmtChange = q.highprice - q.lowprice

q.amtchange is then valued at 9.99999999999996E-02 in the locals window when
I believe it should be valued at .1.

Why is the value not being set to .1? Is there a change I can make to get
this calculation to work how I expect or is there some other way to perform
these calculations to get the results I am after?

Any help would be appreciated. Thanks in advance.

"Mensan" <don'tspamme@hotmail.com> wrote
> I am trying to compute the differnece between two dwords in visual basic and
> keep getting the wrong value being computed. I have the following structure
> defind:
>
> Public Type HighLowQuote
> TradeDate As String
> OpenPrice As Double
> HighPrice As Double
> LowPrice As Double
> ClosePrice As Double
> Volume As Long
> AdjClosePrice As Double
> AmtChange As Double
> End Type


Have you tried declaring your Price values as Currency instead of Double?

LFS





-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 100,000 Newsgroups - 19 Different Servers! =-----

"Mensan" <don'tspamme@hotmail.com> wrote in message
newsyJPb.8948$7D.8032@fe2.texas.rr.com...
> I am trying to compute the differnece between two dwords in visual
basic and
> keep getting the wrong value being computed. I have the following
structure
> defind:
>
> Public Type HighLowQuote
> TradeDate As String
> OpenPrice As Double
> HighPrice As Double
> LowPrice As Double
> ClosePrice As Double
> Volume As Long
> AdjClosePrice As Double
> AmtChange As Double
> End Type
>
> I create an instance of this type as 'q'. I then do the following:
> q.highprice = variant
> q.lowprice = variant
>
> at this point the locals variable window shows the values as:
> q.highprice = 4.1
> q.lowprice = 4
>
> I then fall through the following code:
> q.AmtChange = q.highprice - q.lowprice
>
> q.amtchange is then valued at 9.99999999999996E-02 in the locals
window when
> I believe it should be valued at .1.
>
> Why is the value not being set to .1? Is there a change I can make to
get
> this calculation to work how I expect or is there some other way to
perform
> these calculations to get the results I am after?
>
> Any help would be appreciated. Thanks in advance.
>
>

As Larry said, you may want to use the Currency type instead of doubles.

Welcome to the world of floating point numbers. In your debugger, try
examining the value of
?q.highprice
and then
?(q.highprice = 4.1)
and then
?q.highprice - 4

Depending on where the 4.1 came from, you may discover that it isn't
actually 4.1 in the first place. I can't really tell, because all you
said was q.highprice = variant.

Mensan <don'tspamme@hotmail.com> schreef in berichtnieuws
pyJPb.8948$7D.8032@fe2.texas.rr.com...

Hello Mensan,

[Snip]

> q.amtchange is then valued at 9.99999999999996E-02 in the locals window
when
> I believe it should be valued at .1.
>
> Why is the value not being set to .1? Is there a change I can make to get
> this calculation to work how I expect or is there some other way to
perform
> these calculations to get the results I am after?

But they are allmost the same ! They just differ a meager
0.000000000000004. That's not even a promille of a promille !

Even if you would get that ammount in dollars *every second*, you would,
after a year, not even have a cent. That would actually take something in
the vincinity of a mere 100,000 years :-)

9.99999999999996E-02. The E-02 means : shift the decimal-point left two
places to get the actual number. This means that the actual number is
0.0999999999999996 . And I think that's mighty close to 0.1 .


As others allready explained, that difference is a problem you can get when
a system that can only store numbers in powers of 2 tries to store a
fractional number that is based on powers of 10.

Just try to create the number 0.3 if you can only use 1/2 , 1/4 , 1/8 , 1/16
, etc (wich are 2^-1 , 2^-2, 2^-3, 2^-4, etc) which are the numbers the
computer has to use. I think you will see that you can't ...


To get rid of the *effect*, you could use a storage-type that automatically
rounds on the number of decimals you need (as suggested, use the
currency-type)

On computers that have not got that kind of variable-types you either ignore
the rounding-effect by displaying it with a { using "#####.##" } method
(which will do the rounding for you), or by not using decimals (calculate
the numbers in whole numbers (cents) only. Which, by the way, is what the
currency-type in VB might do).


By the way : This (your problem) is the reason why you should *never* try to
compare two doubles (or even two numbers having fractional parts) with each
other for equality (Like : if double1 = double2 then ....).

Regards,
Rudy Wieser