gawk integer conversions

Hi,

I am trying to pretty print some data presented as a bunch of hex
bytes:

0xf0 0xd8 - for a 16 bit example.

I use:
#little endian build a number
arg = 0
for (i = fld + len; i > fld ; i--) {
arg = (arg * 256) + strtonum($(i))
}

And this does collect the multi byte number 0xd8f0 and I can print it
as 55536 in decimal.

The problem is the data is signed. So -1 is represented as one byte of
0xff - which the above conversion will give me as 255.

Since gawk actually holds numbers as double floats, how can I sign
extend the integer sign?

Thanks, Steve

stevecalfee@gmail.com escreveu:
> Hi,
>
> I am trying to pretty print some data presented as a bunch of hex
> bytes:
>
> 0xf0 0xd8 - for a 16 bit example.
>
> I use:
> #little endian build a number
> arg = 0
> for (i = fld + len; i > fld ; i--) {
> arg = (arg * 256) + strtonum($(i))
> }
>
> And this does collect the multi byte number 0xd8f0 and I can print it
> as 55536 in decimal.
>
> The problem is the data is signed. So -1 is represented as one byte of
> 0xff - which the above conversion will give me as 255.
>
> Since gawk actually holds numbers as double floats, how can I sign
> extend the integer sign?
>
The best idea I can come is masking and testing the MSB multiplying the
result by -1 accordingly...

In article <g56as0$l2v$1@aioe.org>, Cesar Rabak <csrabak@yahoo.com.br> wrote:
>stevecalfee@gmail.com escreveu:
>> Hi,
>>
>> I am trying to pretty print some data presented as a bunch of hex
>> bytes:
>>
>> 0xf0 0xd8 - for a 16 bit example.
>>
>> I use:
>> #little endian build a number
>> arg = 0
>> for (i = fld + len; i > fld ; i--) {
>> arg = (arg * 256) + strtonum($(i))
>> }
>>
>> And this does collect the multi byte number 0xd8f0 and I can print it
>> as 55536 in decimal.
>>
>> The problem is the data is signed. So -1 is represented as one byte of
>> 0xff - which the above conversion will give me as 255.
>>
>> Since gawk actually holds numbers as double floats, how can I sign
>> extend the integer sign?
>>
>The best idea I can come is masking and testing the MSB multiplying the
>result by -1 accordingly...

Negating won't produce the correct number; the description indicates that the
data is stored in 2's complement.

For 16-bit numbers:

if (arg > 32767)
arg = arg - 65536

Similar for 8-bit, etc.

John
--
John DuBois spcecdt@armory.com KC6QKZ/AE http://www.armory.com/~spcecdt/

On Thu, 10 Jul 2008 09:23:32 -0700 (PDT), stevecalfee@gmail.com wrote:

>Hi,
>
>I am trying to pretty print some data presented as a bunch of hex
>bytes:
>
>0xf0 0xd8 - for a 16 bit example.
>
>I use:
>#little endian build a number
> arg = 0
> for (i = fld + len; i > fld ; i--) {
> arg = (arg * 256) + strtonum($(i))
> }
>
>And this does collect the multi byte number 0xd8f0 and I can print it
>as 55536 in decimal.
>
>The problem is the data is signed. So -1 is represented as one byte of
>0xff - which the above conversion will give me as 255.
>
>Since gawk actually holds numbers as double floats, how can I sign
>extend the integer sign?

grant@deltree:~$ awk -f xxx
0xf0 0xd8: -22768
0xf0 0x58: 22768
grant@deltree:~$ cat xxx
function convert_le(fld, len, arg, s)
{
# sign extend the first (ms) byte
if (and(strtonum($(fld -1 +len)), 0x80)) {
arg = 0xffffffff
s = 1
}
else
arg = s = 0
for (i = fld-1 + len; i >= fld ; i--) {
arg = (arg * 256) + strtonum($(i))
}
# assume 16 bit signed output
arg = and(arg, 0x7fff)
if (s)
return -arg
return arg
}
BEGIN {
$0 = "0xf0 0xd8"
printf "%s: %-d\n", $0, convert_le(1,2)
$0 = "0xf0 0x58"
printf "%s: %-d\n", $0, convert_le(1,2)
}

Hope this helps...

Grant.
--
http://bugsplatter.mine.nu/

John DuBois escreveu:
> In article <g56as0$l2v$1@aioe.org>, Cesar Rabak <csrabak@yahoo.com.br> wrote:
>> stevecalfee@gmail.com escreveu:
>>> Hi,
>>>
>>> I am trying to pretty print some data presented as a bunch of hex
>>> bytes:
>>>
>>> 0xf0 0xd8 - for a 16 bit example.
>>>
>>> I use:
>>> #little endian build a number
>>> arg = 0
>>> for (i = fld + len; i > fld ; i--) {
>>> arg = (arg * 256) + strtonum($(i))
>>> }
>>>
>>> And this does collect the multi byte number 0xd8f0 and I can print it
>>> as 55536 in decimal.
>>>
>>> The problem is the data is signed. So -1 is represented as one byte of
>>> 0xff - which the above conversion will give me as 255.
>>>
>>> Since gawk actually holds numbers as double floats, how can I sign
>>> extend the integer sign?
>>>
>> The best idea I can come is masking and testing the MSB multiplying the
>> result by -1 accordingly...
>
> Negating won't produce the correct number; the description indicates that the
> data is stored in 2's complement.
>
> For 16-bit numbers:
>
> if (arg > 32767)
> arg = arg - 65536
>
> Similar for 8-bit, etc.
>
Right. For 2's complement the math operation will have to be performed
afterward the conversion of more bit fiddling to arrive to the correct
result.

Grant <g_r_a_n_t_@dodo.com.au> wrote in
news:4hdd74lucnq1p8ausesqo8862e4cjgs2d2@4ax.com:

> On Thu, 10 Jul 2008 09:23:32 -0700 (PDT), stevecalfee@gmail.com wrote:
>
>>Hi,
>>
>>I am trying to pretty print some data presented as a bunch of hex
>>bytes:
>>
>>0xf0 0xd8 - for a 16 bit example.
>>
>>I use:
>>#little endian build a number
>> arg = 0
>> for (i = fld + len; i > fld ; i--) {
>> arg = (arg * 256) + strtonum($(i))Hi Grant,

You gave me the idea - Here is my final solution 2s complement, multi-
length byte strings, le conversion:

#little endian build a number
if (and(128,strtonum($(fld + len))) == 128 ) {
arg = -1;
} else {
arg = 0;
}
for (i = fld + len; i > fld ; i--) {
arg = (arg * 256) + strtonum($(i))
}

This way it doesn't even matter that it is an integer, everything happens
correctly as its internal double data type, and gawk knows it is negative,
because it built it that way. It is hard to quit thinking of everything as
integers and let double floats do their job.

Funny how this simple arithmetic stuff can confuse me!

I appreciate the help and ideas.

Regards, Steve
>> }
>>
>>And this does collect the multi byte number 0xd8f0 and I can print it
>>as 55536 in decimal.
>>
>>The problem is the data is signed. So -1 is represented as one byte of
>>0xff - which the above conversion will give me as 255.
>>
>>Since gawk actually holds numbers as double floats, how can I sign
>>extend the integer sign?
>
> grant@deltree:~$ awk -f xxx
> 0xf0 0xd8: -22768
> 0xf0 0x58: 22768
> grant@deltree:~$ cat xxx
> function convert_le(fld, len, arg, s)
> {
> # sign extend the first (ms) byte
> if (and(strtonum($(fld -1 +len)), 0x80)) {
> arg = 0xffffffff
> s = 1
> }
> else
> arg = s = 0
> for (i = fld-1 + len; i >= fld ; i--) {
> arg = (arg * 256) + strtonum($(i))
> }
> # assume 16 bit signed output
> arg = and(arg, 0x7fff)
> if (s)
> return -arg
> return arg
> }
> BEGIN {
> $0 = "0xf0 0xd8"
> printf "%s: %-d\n", $0, convert_le(1,2)
> $0 = "0xf0 0x58"
> printf "%s: %-d\n", $0, convert_le(1,2)
> }
>
Hi Grant,

You gave me the idea - Here is my final solution 2s complement, multi-
length byte strings, le conversion:

#little endian build a number
if (and(128,strtonum($(fld + len))) == 128 ) {
arg = -1;
} else {
arg = 0;
}
for (i = fld + len; i > fld ; i--) {
arg = (arg * 256) + strtonum($(i))
}

This way it doesn't even matter that it is an integer, everything happens
correctly as its internal double data type, and gawk knows it is negative,
because it built it that way. It is hard to quit thinking of everything as
integers and let double floats do their job.

Funny how this simple arithmetic stuff can confuse me!

I appreciate the help and ideas.

Regards, Steve