byte[] to byte*... What is the fastest way?This is a discussion on byte[] to byte*... What is the fastest way? within the Framework and Interface Programming forums in category; hi, So, here is the code to get my checksum. If you see something that can be optimized, let me know. thanks ThunderMusic Code: private const UInt16 MOD_ADLER = 65521; private unsafe uint GetChecksum(byte[] databytes) { UInt32 a = 1, b = 0; fixed (byte* tmpdata = databytes) { byte* data = tmpdata; int len = databytes.Length; /* Length in bytes */ while (len > 0) { int tlen = len > 5550 ? 5550 : len; len -= tlen; do { a += *(data++); b += a; } while ((--tlen) > 0); a = (a & 0xffff) + (a ... | ||||||||
| | ||||||||
| ||||||||
![byte[] to byte*... What is the fastest way?](http://objectmix.com/iconimages/framework-interface-programming/byte-byte-what-fastest-way_ltr.gif)
| Register | FAQ | Calendar | Search | Today's Posts | Mark Forums Read |
|
#21
08-23-2007, 04:41 PM
| |||
| |||
 Re: byte[] to byte*... What is the fastest way? hi, So, here is the code to get my checksum. If you see something that can be optimized, let me know. thanks ThunderMusic Code:
private const UInt16 MOD_ADLER = 65521;
private unsafe uint GetChecksum(byte[] databytes)
{
UInt32 a = 1, b = 0;
fixed (byte* tmpdata = databytes)
{
byte* data = tmpdata;
int len = databytes.Length; /* Length in bytes */
while (len > 0)
{
int tlen = len > 5550 ? 5550 : len;
len -= tlen;
do
{
a += *(data++);
b += a;
} while ((--tlen) > 0);
a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER);
b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER);
}
/* It can be shown that a <= 0x1013a here, so a single subtract will
do. */
if (a >= MOD_ADLER)
a -= MOD_ADLER;
/* It can be shown that b can reach 0xffef1 here. */
b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER);
if (b >= MOD_ADLER)
b -= MOD_ADLER;
}
return ((b << 16) | a);
}
"Nicholas Paldino [.NET/C# MVP]" <mvp@spam.guard.caspershouse.com> wrote in message news:ewAkXGc5HHA.4712@TK2MSFTNGP04.phx.gbl... > ThunderMusic, > > If you are looking to tune this algoritm, I would say that the > iteration through the bytes is NOT the way to do it. There are probably a > bunch of other areas that can be improved upon. > > > -- > - Nicholas Paldino [.NET/C# MVP] > - mvp@spam.guard.caspershouse.com > > "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message > news:ukULcEc5HHA.2312@TK2MSFTNGP06.phx.gbl... >> hi, >> thanks for the really quick answer... Your solution works, but I can't >> do p++ because it's fixed. So I must use another pointer like byte* p2 = >> p; so now I can do p2++;... >> >> Actually, I'm looking for a faster way to compute a checksum on a byte >> array... For now, I'm using the Adler-32 algorithm, but I'm open to >> advises on a performant checksum algorithm. It will be for an error >> checking mecanism for tcp and udp communication on a closed network >> environment. So it doesn't need to be human-modification resistant, it's >> just to prevent modification due to the noise on the line (if it can >> happen)... >> >> Thanks >> >> ThunderMusic >> >> "Nicholas Paldino [.NET/C# MVP]" <mvp@spam.guard.caspershouse.com> wrote >> in message news:uezij7b5HHA.5360@TK2MSFTNGP03.phx.gbl... >>> ThunderMusic, >>> >>> Use unsafe code: >>> >>> byte[] bytes = ...; >>> >>> unsafe >>> { >>> fixed (byte* p = bytes) >>> { >>> // Work with pointer here. >>> } >>> } >>> >>> As a matter of fact, that's the only way to do it, as you need to pin >>> down the location of the array to prevent the reference from moving >>> around. >>> >>> Is there a reason you need the pointer, or are you just looking for a >>> faster way to iterate through the array? >>> >>> >>> -- >>> - Nicholas Paldino [.NET/C# MVP] >>> - mvp@spam.guard.caspershouse.com >>> >>> "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message >>> news:%23KkK5zb5HHA.5316@TK2MSFTNGP04.phx.gbl... >>>> Hi, >>>> The subject says it all... I want to use a byte[] and use it as byte* >>>> so I can increment the pointer to iterate through it. >>>> >>>> What is the fastest way of doing so in C#? >>>> >>>> Thanks >>>> >>>> ThunderMusic >>>> >>> >>> >> >> > >  |
|
#22
08-23-2007, 04:42 PM
| |||
| |||
| Re: byte[] to byte*... What is the fastest way? Hi, "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:eZjuTUc5HHA.4964@TK2MSFTNGP06.phx.gbl... > hi, > I know it may be overoptimizing, but this part of the code will be called > many many many times per seconds, so I'm must make sure it's as optimized > as it can be. It's for a checksum mecanism, so each time a message must be > sent, it is called to compute the checksum to append to the message and > will be computed again when the client receives it so it can verify the > validity of the message. > > Thanks for the comment tought... Why don't you do something, try using the "vanila" for statement versus the pointer arithmetic you want to use, I will be surprised if you find any noticeable difference. |
|
#23
08-23-2007, 04:42 PM
| |||
| |||
| Re: byte[] to byte*... What is the fastest way? Hi, "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:eZjuTUc5HHA.4964@TK2MSFTNGP06.phx.gbl... > hi, > I know it may be overoptimizing, but this part of the code will be called > many many many times per seconds, so I'm must make sure it's as optimized > as it can be. It's for a checksum mecanism, so each time a message must be > sent, it is called to compute the checksum to append to the message and > will be computed again when the client receives it so it can verify the > validity of the message. > > Thanks for the comment tought... Why don't you do something, try using the "vanila" for statement versus the pointer arithmetic you want to use, I will be surprised if you find any noticeable difference. |
|
#24
08-23-2007, 04:46 PM
| |||
| |||
| Re: byte[] to byte*... What is the fastest way? I understand well the "This code it called alot, therefore it needs to be optimized." frame of mind. I build big, high-performance, network socket servers for a living. I know this space really well, and have done lots in both TCP & UDP land at scales that are pretty signfigant. Put a profiler on this, and you'll likley find that this chunk of code won't even show up. Certainly the difference between the two (pointer vs iterator) is unlikley to show up at all. -- Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP http://www.coversant.com/blogs/cmullins "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:eZjuTUc5HHA.4964@TK2MSFTNGP06.phx.gbl... > hi, > I know it may be overoptimizing, but this part of the code will be called > many many many times per seconds, so I'm must make sure it's as optimized > as it can be. It's for a checksum mecanism, so each time a message must be > sent, it is called to compute the checksum to append to the message and > will be computed again when the client receives it so it can verify the > validity of the message. > > Thanks for the comment tought... > > ThunderMusic > > "Chris Mullins [MVP]" <cmullins@yahoo.com> wrote in message > news:uyDEaDc5HHA.484@TK2MSFTNGP06.phx.gbl... >> "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote: >> >>> The subject says it all... I want to use a byte[] and use it as byte* so >>> I can increment the pointer to iterate through it. >> >> I really hate to be pedantic, but I'm willing to bet that the difference >> in how you iterate through your array makes little to no difference in >> the overall performance of your code. >> >> People frequently are guilting of over-optimizing things that are already >> "Fast Enough". Unless you've verified this section is slow via a >> Profiler, you're better off not getting fancy with optimizations. >> >> -- >> Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP >> http://www.coversant.com/blogs/cmullins >> > > |
|
#25
08-23-2007, 04:46 PM
| |||
| |||
| Re: byte[] to byte*... What is the fastest way? I understand well the "This code it called alot, therefore it needs to be optimized." frame of mind. I build big, high-performance, network socket servers for a living. I know this space really well, and have done lots in both TCP & UDP land at scales that are pretty signfigant. Put a profiler on this, and you'll likley find that this chunk of code won't even show up. Certainly the difference between the two (pointer vs iterator) is unlikley to show up at all. -- Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP http://www.coversant.com/blogs/cmullins "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:eZjuTUc5HHA.4964@TK2MSFTNGP06.phx.gbl... > hi, > I know it may be overoptimizing, but this part of the code will be called > many many many times per seconds, so I'm must make sure it's as optimized > as it can be. It's for a checksum mecanism, so each time a message must be > sent, it is called to compute the checksum to append to the message and > will be computed again when the client receives it so it can verify the > validity of the message. > > Thanks for the comment tought... > > ThunderMusic > > "Chris Mullins [MVP]" <cmullins@yahoo.com> wrote in message > news:uyDEaDc5HHA.484@TK2MSFTNGP06.phx.gbl... >> "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote: >> >>> The subject says it all... I want to use a byte[] and use it as byte* so >>> I can increment the pointer to iterate through it. >> >> I really hate to be pedantic, but I'm willing to bet that the difference >> in how you iterate through your array makes little to no difference in >> the overall performance of your code. >> >> People frequently are guilting of over-optimizing things that are already >> "Fast Enough". Unless you've verified this section is slow via a >> Profiler, you're better off not getting fancy with optimizations. >> >> -- >> Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP >> http://www.coversant.com/blogs/cmullins >> > > |
|
#26
08-23-2007, 07:42 PM
| |||
| |||
| Re: byte[] to byte*... What is the fastest way? "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:OhgmWYc5HHA.2108@TK2MSFTNGP02.phx.gbl... > hi, > So, here is the code to get my checksum. If you see something that can be > optimized, let me know. > > thanks > > ThunderMusic > > Code: >
> private const UInt16 MOD_ADLER = 65521;
> private unsafe uint GetChecksum(byte[] databytes)
> {
> UInt32 a = 1, b = 0;
> fixed (byte* tmpdata = databytes)
> {
> byte* data = tmpdata;
> int len = databytes.Length; /* Length in bytes */
> while (len > 0)
> {
> int tlen = len > 5550 ? 5550 : len;
> len -= tlen;
> do
> {
> a += *(data++);
> b += a;
> } while ((--tlen) > 0);
> a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER);
> b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER);
> }
> /* It can be shown that a <= 0x1013a here, so a single subtract
> will do. */
> if (a >= MOD_ADLER)
> a -= MOD_ADLER;
> /* It can be shown that b can reach 0xffef1 here. */
> b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER);
> if (b >= MOD_ADLER)
> b -= MOD_ADLER;
> }
> return ((b << 16) | a);
> }
>
>
You can try to do some partial loop unrolling, this is what a C++ compiler does and what the C# compiler/JIT unfortunately fails to do. UInt32 a = 1, b = 0; fixed (byte* tmpdata = databytes) { byte* data = tmpdata; int len = databytes.Length; /* Length in bytes */ while(len > 0) { int tlen = len > 5550 ? 5550 : len; len -= tlen; for(int c = 0; c < tlen / 4; c++) { a += *(data++); b += a; a += *(data++); b += a; a += *(data++); b += a; a += *(data++); b += a; } a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); for(int c = 0; c < tlen % 4; c++) { a += *(data++); b += a; } a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); } if (a >= MOD_ADLER) a -= MOD_ADLER; /* It can be shown that b can reach 0xffef1 here. */ b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); if (b >= MOD_ADLER) b -= MOD_ADLER; } return ((b << 16) | a); Willy. |
|
#27
08-23-2007, 07:42 PM
| |||
| |||
| Re: byte[] to byte*... What is the fastest way? "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:OhgmWYc5HHA.2108@TK2MSFTNGP02.phx.gbl... > hi, > So, here is the code to get my checksum. If you see something that can be > optimized, let me know. > > thanks > > ThunderMusic > > Code: >
> private const UInt16 MOD_ADLER = 65521;
> private unsafe uint GetChecksum(byte[] databytes)
> {
> UInt32 a = 1, b = 0;
> fixed (byte* tmpdata = databytes)
> {
> byte* data = tmpdata;
> int len = databytes.Length; /* Length in bytes */
> while (len > 0)
> {
> int tlen = len > 5550 ? 5550 : len;
> len -= tlen;
> do
> {
> a += *(data++);
> b += a;
> } while ((--tlen) > 0);
> a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER);
> b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER);
> }
> /* It can be shown that a <= 0x1013a here, so a single subtract
> will do. */
> if (a >= MOD_ADLER)
> a -= MOD_ADLER;
> /* It can be shown that b can reach 0xffef1 here. */
> b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER);
> if (b >= MOD_ADLER)
> b -= MOD_ADLER;
> }
> return ((b << 16) | a);
> }
>
>
You can try to do some partial loop unrolling, this is what a C++ compiler does and what the C# compiler/JIT unfortunately fails to do. UInt32 a = 1, b = 0; fixed (byte* tmpdata = databytes) { byte* data = tmpdata; int len = databytes.Length; /* Length in bytes */ while(len > 0) { int tlen = len > 5550 ? 5550 : len; len -= tlen; for(int c = 0; c < tlen / 4; c++) { a += *(data++); b += a; a += *(data++); b += a; a += *(data++); b += a; a += *(data++); b += a; } a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); for(int c = 0; c < tlen % 4; c++) { a += *(data++); b += a; } a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); } if (a >= MOD_ADLER) a -= MOD_ADLER; /* It can be shown that b can reach 0xffef1 here. */ b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); if (b >= MOD_ADLER) b -= MOD_ADLER; } return ((b << 16) | a); Willy. |
|
#28
08-24-2007, 09:30 AM
| |||
| |||
| Re: byte[] to byte*... What is the fastest way? wow!! that's pretty impressive... this little modification made the loop 2 to 3 times faster!! thanks a lot ThunderMusic "Willy Denoyette [MVP]" <willy.denoyette@telenet.be> wrote in message news:uuXD%238d5HHA.3400@TK2MSFTNGP03.phx.gbl... > "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message > news:OhgmWYc5HHA.2108@TK2MSFTNGP02.phx.gbl... >> hi, >> So, here is the code to get my checksum. If you see something that can be >> optimized, let me know. >> >> thanks >> >> ThunderMusic >> >> Code: >>
>> private const UInt16 MOD_ADLER = 65521;
>> private unsafe uint GetChecksum(byte[] databytes)
>> {
>> UInt32 a = 1, b = 0;
>> fixed (byte* tmpdata = databytes)
>> {
>> byte* data = tmpdata;
>> int len = databytes.Length; /* Length in bytes */
>> while (len > 0)
>> {
>> int tlen = len > 5550 ? 5550 : len;
>> len -= tlen;
>> do
>> {
>> a += *(data++);
>> b += a;
>> } while ((--tlen) > 0);
>> a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER);
>> b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER);
>> }
>> /* It can be shown that a <= 0x1013a here, so a single subtract
>> will do. */
>> if (a >= MOD_ADLER)
>> a -= MOD_ADLER;
>> /* It can be shown that b can reach 0xffef1 here. */
>> b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER);
>> if (b >= MOD_ADLER)
>> b -= MOD_ADLER;
>> }
>> return ((b << 16) | a);
>> }
>>
>>
> > You can try to do some partial loop unrolling, this is what a C++ compiler > does and what the C# compiler/JIT unfortunately fails to do. > > UInt32 a = 1, b = 0; > fixed (byte* tmpdata = databytes) > { > byte* data = tmpdata; > int len = databytes.Length; /* Length in bytes */ > while(len > 0) > { > int tlen = len > 5550 ? 5550 : len; > len -= tlen; > for(int c = 0; c < tlen / 4; c++) > { > a += *(data++); > b += a; > a += *(data++); > b += a; > a += *(data++); > b += a; > a += *(data++); > b += a; > } > a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); > b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); > for(int c = 0; c < tlen % 4; c++) > { > a += *(data++); > b += a; > } > a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); > b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); > } > if (a >= MOD_ADLER) > a -= MOD_ADLER; > /* It can be shown that b can reach 0xffef1 here. */ > b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); > if (b >= MOD_ADLER) > b -= MOD_ADLER; > } > return ((b << 16) | a); > > > Willy. > |
|
#29
08-24-2007, 09:30 AM
| |||
| |||
| Re: byte[] to byte*... What is the fastest way? wow!! that's pretty impressive... this little modification made the loop 2 to 3 times faster!! thanks a lot ThunderMusic "Willy Denoyette [MVP]" <willy.denoyette@telenet.be> wrote in message news:uuXD%238d5HHA.3400@TK2MSFTNGP03.phx.gbl... > "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message > news:OhgmWYc5HHA.2108@TK2MSFTNGP02.phx.gbl... >> hi, >> So, here is the code to get my checksum. If you see something that can be >> optimized, let me know. >> >> thanks >> >> ThunderMusic >> >> Code: >>
>> private const UInt16 MOD_ADLER = 65521;
>> private unsafe uint GetChecksum(byte[] databytes)
>> {
>> UInt32 a = 1, b = 0;
>> fixed (byte* tmpdata = databytes)
>> {
>> byte* data = tmpdata;
>> int len = databytes.Length; /* Length in bytes */
>> while (len > 0)
>> {
>> int tlen = len > 5550 ? 5550 : len;
>> len -= tlen;
>> do
>> {
>> a += *(data++);
>> b += a;
>> } while ((--tlen) > 0);
>> a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER);
>> b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER);
>> }
>> /* It can be shown that a <= 0x1013a here, so a single subtract
>> will do. */
>> if (a >= MOD_ADLER)
>> a -= MOD_ADLER;
>> /* It can be shown that b can reach 0xffef1 here. */
>> b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER);
>> if (b >= MOD_ADLER)
>> b -= MOD_ADLER;
>> }
>> return ((b << 16) | a);
>> }
>>
>>
> > You can try to do some partial loop unrolling, this is what a C++ compiler > does and what the C# compiler/JIT unfortunately fails to do. > > UInt32 a = 1, b = 0; > fixed (byte* tmpdata = databytes) > { > byte* data = tmpdata; > int len = databytes.Length; /* Length in bytes */ > while(len > 0) > { > int tlen = len > 5550 ? 5550 : len; > len -= tlen; > for(int c = 0; c < tlen / 4; c++) > { > a += *(data++); > b += a; > a += *(data++); > b += a; > a += *(data++); > b += a; > a += *(data++); > b += a; > } > a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); > b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); > for(int c = 0; c < tlen % 4; c++) > { > a += *(data++); > b += a; > } > a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); > b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); > } > if (a >= MOD_ADLER) > a -= MOD_ADLER; > /* It can be shown that b can reach 0xffef1 here. */ > b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); > if (b >= MOD_ADLER) > b -= MOD_ADLER; > } > return ((b << 16) | a); > > > Willy. > |
|
#30
08-24-2007, 12:38 PM
| |||
| |||
| Re: byte[] to byte*... What is the fastest way? "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:u$0afLl5HHA.1208@TK2MSFTNGP03.phx.gbl... > wow!! that's pretty impressive... this little modification made the loop > 2 to 3 times faster!! > 2 to 3 times faster? It's hard to believe honestly, I would have expected something like a 20-30% improvement. Are you sure about your time measurement techniques? Willy. |
 |
« Previous Thread
|
Next Thread »
| Thread Tools | |
| Display Modes | |
Linear Mode
All times are GMT -5. The time now is 03:13 AM.
