Read binary data file - Java

Windsor.Locks wrote:
>>> I do not have any say in the file format or how
>>> the file is written. My requirement is read this file and get the data

<http://java.sun.com/javase/6/docs/api/java/nio/ByteBuffer.html>
<http://java.sun.com/javase/6/docs/api/java/nio/ByteBuffer.html#order(java.nio.ByteOrder)>

--
Lew

Windsor.Locks wrote:
>> Here is how I read it in C++,
>>
>> Struct SOME_DATA
>> {
>> unsigned long data1;
>> unsigned short data2;
>> unsigned short data3;
>> unsigned long data4;
>>
>> }
>>
>> struct SOME_DATA someData;
>>
>> and read using
>>
>> fread(&someData, 12, 1, inputFile);
>>
>> Please give me some pointers, how do i read this using Java? Thanks.
....
> public class Convert {
>
> public static void main(String [] args) {
>
> int crap = 0, doublecrap = 0, counter = 0;
>
etc.
> }
> }

java.nio.ByteOrder will help you if you use the java.nio package as Roedy
suggested.

Please do not embed TABs in Usenet posts; it really fubars the alignment.

--
Lew

On Wed, 29 Aug 2007 16:47:00 -0700, Windsor.Locks wrote,
quoted or indirectly quoted someone who said :

>Thank you for all who tried to help. I got it working and in the
>interest of future programmers here is how I did it.

You are trying to read little-endian data. It is a lot easier with
LEDatastream.

float f = dis.readFloat();
double d = dis.readDouble();
int i = dis.readInt();
--
Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com

On Aug 29, 10:15 pm, Roedy Green <see_webs...@mindprod.com.invalid>
wrote:
> On Wed, 29 Aug 2007 16:47:00 -0700, Windsor.Lo... wrote,
> quoted or indirectly quoted someone who said :
>
> >Thank you for all who tried to help. I got it working and in the
> >interest of future programmers here is how I did it.
>
> You are trying to read little-endian data. It is a lot easier with
> LEDatastream.
>
> float f = dis.readFloat();
> double d = dis.readDouble();
> int i = dis.readInt();
> --
> Roedy Green Canadian Mind Products
> The Java Glossaryhttp://mindprod.com

Well, that actually does not work. See the reply above by "shakah"

On Aug 30, 10:26 am, Windsor.Lo... wrote:
> On Aug 29, 10:15 pm, Roedy Green <see_webs...@mindprod.com.invalid>
> wrote:
>
> > On Wed, 29 Aug 2007 16:47:00 -0700, Windsor.Lo... wrote,
> > quoted or indirectly quoted someone who said :
>
> > >Thank you for all who tried to help. I got it working and in the
> > >interest of future programmers here is how I did it.
>
> > You are trying to read little-endian data. It is a lot easier with
> > LEDatastream.
>
> > float f = dis.readFloat();
> > double d = dis.readDouble();
> > int i = dis.readInt();
> > --
> > Roedy Green Canadian Mind Products
> > The Java Glossaryhttp://mindprod.com
>
> Well, that actually does not work. See the reply above by "shakah"

He's suggesting you use his "little-endian DataInputStream" class,
where I'm guessing it would work:
http://mindprod.com/jgloss/ledatinputstream.html

On Aug 30, 12:54 am, ~kurt <actinouran...@earthlink.net> wrote:
> Hunter Gratzner <a24...@googlemail.com> wrote:
> > On Aug 29, 10:54 pm, Windsor.Lo... wrote:
> >> Thanks for your reply. I do not have any say in the file format or how
> >> the file is written. My requirement is read this file and get the data
> >> out of it. There is nothing more I can do.
>
> > Then the one "defining" this data format has no fucking clue. C/C++
> > structs have no well defined binary layout, except the order of
> > elements. C/C++ integer data types have no well defined binary
> > representation and no well defined size, except a minimum value range.
>
> And you are missing the point.

No, I don't. A C struct is not a suitable, unambiguous format
specification, binary or otherwise. That's the whole point. Giving
someone just a C struct and telling him to implement it in Java is a
pointless stupid act. It indicates that the one giving this file
format "definition" has no fucking clue what he is doing.

> Instead of telling us the one defining the data format has no clue (which
> you are wrong about), why don't you explain your solution to writing a binary
> file in C/C++, FORTRAN, or whatever, that will solve all the academic issues
> you have just brought up.

It did that previously in this same thread, but you are apparently
more interested in picking a fight.

> Reading binary files is almost always tricky, especially when you move from
> one platform, OS, or language to the next. There is no way to circumvent
> this.

Sure it is. By having an unambiguous format specification. A C struct
is not an unambiguous format specification.

> It is the price you pay to have the data in a binary format.

No, it is the price to pay when some fuckwit thinks that writing C
structs 1:1 to memory is a good idea.

There is no difference between a binary and a text format if you need
to move between platforms. Either the format is unambiguously defined,
then it's a straight forward job to implement it, or it isn't.

> What is really annoying is
> when you don't even know the endian or the size of the values (16 bit,
> 32 bit?) and need to experiment to get it right.

And why do you then think a C struct is a good definition of a binary
format?

Hunter Gratzner <a24...@googlemail.com> wrote:
> C/C++ integer data types have no well defined binary
> representation and no well defined size, except a minimum value range.

And the presence or absence of between-field padding isn't always
guaranteed. Still, if the files don't have to be cross-platform, reading
and writing structs will work just fine. Note: the *application* can be
portable across platforms, so long as the (for example) Solaris/Sparc
version won't have to read files written by the Windows/Intel version.

Hunter Gratzner <a24900@googlemail.com> wrote:
>
> No, I don't. A C struct is not a suitable, unambiguous format
> specification, binary or otherwise. That's the whole point. Giving
> someone just a C struct and telling him to implement it in Java is a
> pointless stupid act. It indicates that the one giving this file
> format "definition" has no fucking clue what he is doing.

It is hardly pointless. Most of the time, there is no format specification
because binary data is often not written with the intention of being used
outside of the application that writes it. Only later does an outside user
have a need for the data, and then one has to often reverse engineer a
solution. A C struct at least gives you an idea as to what type of data is in
the file. Knowing what platform it was written in helps out even more.

>> Instead of telling us the one defining the data format has no clue (which
>> you are wrong about), why don't you explain your solution to writing a binary
>> file in C/C++, FORTRAN, or whatever, that will solve all the academic issues
>> you have just brought up.
>
> It did that previously in this same thread, but you are apparently
> more interested in picking a fight.

I'm put off by your attitude that what the OP has to work with is due to
someone who has no clue. If you are saying a C structure makes a bad
ICD, then I agree with you. But, binary files are often not written with
portability in mind, and the implementation details exist only in the code
that reads/writes the data. There is nothing wrong with that when the
original intent of the data was for internal use only - and that is often
the case. Then, seeing how the data is read into a C structure is invaluable.

The soultion I saw you post was an example of how to read the data. I didn't
see anything but bitching regarding the data source.

> No, it is the price to pay when some fuckwit thinks that writing C
> structs 1:1 to memory is a good idea.

It is often the only reasonable idea, depending on the orignal intent of
the data. Like I said, I didn't see a better solution posted by you
on how to do this. Creating unecessary ICDs is a bad thing.

> And why do you then think a C struct is a good definition of a binary
> format?

It works as good as anything else for many uses. If you write a specification
describing how many bytes a number is supposed to take up, and the endian, and
the data is only to be used internally, then you are creating extra work for
youself when you port the code to other platforms (of course, you want to call
sizeoff() when reading in the structure instead of hard coding the size).

- Kurt

On Aug 30, 7:18 pm, ~kurt <actinouran...@earthlink.net> wrote:
> Hunter Gratzner <a24...@googlemail.com> wrote:
>
> > No, I don't. A C struct is not a suitable, unambiguous format
> > specification, binary or otherwise. That's the whole point. Giving
> > someone just a C struct and telling him to implement it in Java is a
> > pointless stupid act. It indicates that the one giving this file
> > format "definition" has no fucking clue what he is doing.
>
> It is hardly pointless. Most of the time, there is no format specification
> because binary data is often not written with the intention of being used
> outside of the application that writes it. Only later does an outside user
> have a need for the data, and then one has to often reverse engineer a
> solution. A C struct at least gives you an idea as to what type of data is in
> the file. Knowing what platform it was written in helps out even more.
>
> >> Instead of telling us the one defining the data format has no clue (which
> >> you are wrong about), why don't you explain your solution to writing a binary
> >> file in C/C++, FORTRAN, or whatever, that will solve all the academic issues
> >> you have just brought up.
>
> > It did that previously in this same thread, but you are apparently
> > more interested in picking a fight.
>
> I'm put off by your attitude that what the OP has to work with is due to
> someone who has no clue. If you are saying a C structure makes a bad
> ICD, then I agree with you. But, binary files are often not written with
> portability in mind, and the implementation details exist only in the code
> that reads/writes the data. There is nothing wrong with that when the
> original intent of the data was for internal use only - and that is often
> the case. Then, seeing how the data is read into a C structure is invaluable.
>
> The soultion I saw you post was an example of how to read the data. I didn't
> see anything but bitching regarding the data source.
>
> > No, it is the price to pay when some fuckwit thinks that writing C
> > structs 1:1 to memory is a good idea.
>
> It is often the only reasonable idea, depending on the orignal intent of
> the data. Like I said, I didn't see a better solution posted by you
> on how to do this. Creating unecessary ICDs is a bad thing.
>
> > And why do you then think a C struct is a good definition of a binary
> > format?
>
> It works as good as anything else for many uses. If you write a specification
> describing how many bytes a number is supposed to take up, and the endian, and
> the data is only to be used internally, then you are creating extra work for
> youself when you port the code to other platforms (of course, you want to call
> sizeoff() when reading in the structure instead of hard coding the size).
>
> - Kurt

Dear Friends (when did you guys become my friends?)

Let's review what the OP stated

A struct is given in C++

Data needs to read from a file in Java.

You have the following data types

unsigned long
unsigned short

As previously stated by other posters the Endianness of the operating
system should affect how the output file is encoded. I assume this to
be true but have not verified it to be true.

We assume all unsigned longs and unsigned short will ALWAYS have the
same bytesize.

The complete struct is given as

unsigned long data1;
unsigned short data2;
unsigned short data3;
unsigned long data4;

Can we also assume that the data will always be sequenced as described
in the STRUCT?
I don't see any argument why the data will be out of sequence as
defined in the STRUCT.

Does the input file get modified when it is transported from one
operating system to another?
I assume NO. This is not verified.

Are there equivalents of unsigned long and unsigned short in Java?
Are they the same byte size?
Do they encode the data the same?

Try to read in Java and verify with known data. If you don't know any
of the data values this becomes a harder task.