extract byte sequence from a binary fileThis is a discussion on extract byte sequence from a binary file within the awk forums in Programming Languages category; Hello! I have a large (up to 4 GB) binary file, an I want to extract the byte sequence of a certain length, beginning at a known position. For example, to take 50 000 bytes, beginning at position 200 000 (index count is starting from 0), I would normally execute head -c 250000 file.dat | tail -c 50000 > out.dat but head and tail are very slow and for a big file, I ran out of memory. So, is this task possible with awk? Any hints would be appreciated. Thanks in advance, Olli... | ||||||||
| | ||||||||
| ||||||||
| Register | FAQ | Calendar | Search | Today's Posts | Mark Forums Read |
|
#1
08-24-2008, 02:22 PM
| |||
| |||
 extract byte sequence from a binary file Hello! I have a large (up to 4 GB) binary file, an I want to extract the byte sequence of a certain length, beginning at a known position. For example, to take 50 000 bytes, beginning at position 200 000 (index count is starting from 0), I would normally execute head -c 250000 file.dat | tail -c 50000 > out.dat but head and tail are very slow and for a big file, I ran out of memory. So, is this task possible with awk? Any hints would be appreciated. Thanks in advance, Olli  |
|
#2
08-24-2008, 02:28 PM
| |||
| |||
| Re: extract byte sequence from a binary file In article <48b1a6f6$0$12956$9b4e6d93@newsspool2.arcor-online.net>, Oliver Gronau <ogronau@web.de> wrote: >Hello! > >I have a large (up to 4 GB) binary file, an I want to extract the byte >sequence of a certain length, beginning at a known position. > >For example, to take 50 000 bytes, beginning at position 200 000 (index >count is starting from 0), I would normally execute > > head -c 250000 file.dat | tail -c 50000 > out.dat > >but head and tail are very slow and for a big file, I ran out of memory. >So, is this task possible with awk? Any hints would be appreciated. > >Thanks in advance, > > Olli Platform? Version of AWK? |
|
#3
08-24-2008, 02:49 PM
| |||
| |||
| Re: extract byte sequence from a binary file Am 24.08.2008 20:28 schrieb Kenny McCormack: > In article <48b1a6f6$0$12956$9b4e6d93@newsspool2.arcor-online.net>, > Oliver Gronau <ogronau@web.de> wrote: >>Hello! >> >>I have a large (up to 4 GB) binary file, an I want to extract the byte >>sequence of a certain length, beginning at a known position. >> >>For example, to take 50 000 bytes, beginning at position 200 000 (index >>count is starting from 0), I would normally execute >> >> head -c 250000 file.dat | tail -c 50000 > out.dat >> >>but head and tail are very slow and for a big file, I ran out of memory. >>So, is this task possible with awk? Any hints would be appreciated. >> > Platform? Version of AWK? Kubuntu Linux, GNU Awk 3.1.6 Ciao Olli |
|
#4
08-24-2008, 03:02 PM
| |||
| |||
| Re: extract byte sequence from a binary file In article <48b1ad44$0$12943$9b4e6d93@newsspool2.arcor-online.net>, Oliver Gronau <ogronau@web.de> wrote: .... >> Platform? Version of AWK? > >Kubuntu Linux, GNU Awk 3.1.6 > > Ciao > Olli Ok, good. Just wanted to make sure we were talking about the same things. Lotsa of times we find out, at a late stage, that the OP is using something weird, such as Windows. For this sort of thing, I usually use "dd", setting the block sizes and skip paramters accordingly. So, for your example, I would so: dd if=inputfile of=outputfile bs=50000 skip=4 count=1 |
 |
« Previous Thread
|
Next Thread »
| Thread Tools | |
| Display Modes | |
Linear Mode
All times are GMT -5. The time now is 02:40 AM.
