Reliability of Java, sockets and TCP transmissions - Java

On Fri, 5 Oct 2007 23:52:19 +1000, Qu0ll wrote:
> So just how reliable are TCP and Java sockets over the actual
> internet? I mean do I need to implement some kind of "advanced"
> protocol whereby check sums are transmitted along with packets and
> the packet retransmitted if the check sum is invalid or is all this
> handled by either the Java sockets or the TCP protocol already?

"Java sockets" is a misnomer, since sockets are not specific to Java
or any other programming language. Java provides an interface to the
socket mechanism (sometimes called "Berkeley sockets") already
provided by your operating system.

TCP is a reliable protocol. It manages retransmission when data is out
of order, missing or corrupt, and it will break the connection when it
is unable to provide an error-free data stream, but not before making
a number of attempts to correct the situation first.

Your application can safely assume that as long as your connection is
up, your data stream contains exactly the data sent by the remote.

There are however a number of things that your application can do
wrong. The commonest ones to watch out for are thinking that TCP will
respect your message boundaries (it won't), or expecting that you'll
always read as much data as you requested (you won't).

/gordon

--

Qu0ll wrote:

> I am writing client and server components of an application that communicate
> using Socket, ServerSocket and TCP. I would like to know just how reliable
> this connection/protocol combination is in terms of transmission errors. So
> far I have only been able to run the application where the client and server
> are on the same local machine or separated by an intranet/LAN so I have no
> results of an internet deployment to report but I have not encountered any
> IO errors to this point.
>
> So just how reliable are TCP and Java sockets over the actual internet? I
> mean do I need to implement some kind of "advanced" protocol whereby check
> sums are transmitted along with packets and the packet retransmitted if the
> check sum is invalid or is all this handled by either the Java sockets or
> the TCP protocol already?
>

That is handled by TCP/IP. Each IP datagram contains multiple checksums. There
are at least checksums for the IP header and for the TCP segment (header+data).
Other protocols layered on top of TCP/IP may add their own.

Sockets are very reliable. The Internet is built on them.

--
Nigel Wade, System Administrator, Space Plasma Physics Group,
University of Leicester, Leicester, LE1 7RH, UK
E-mail : nmw@ion.le.ac.uk
Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555

"Gordon Beaton" <n.o.t@for.email> wrote in message
news:4706472b$0$3207$8404b019@news.wineasy.se...

> Your application can safely assume that as long as your connection is
> up, your data stream contains exactly the data sent by the remote.

This is good to know, except then you say:

> There are however a number of things that your application can do
> wrong. The commonest ones to watch out for are thinking that TCP will
> respect your message boundaries (it won't), or expecting that you'll
> always read as much data as you requested (you won't).

Can you please explain what you mean by not respecting message boundaries?
I am sending data in 512 byte "packets", are you saying that this won't be
respected by TCP?

And what do you mean by not reading as much data as requested? When would
this happen and why?

--
And loving it,

-Q
_________________________________________________
Qu0llSixFour@gmail.com
(Replace the "SixFour" with numbers to email me)

"Nigel Wade" <nmw@ion.le.ac.uk> wrote in message
news:fe5huj$9iv$1@south.jnrs.ja.net...

> That is handled by TCP/IP. Each IP datagram contains multiple checksums.
> There
> are at least checksums for the IP header and for the TCP segment
> (header+data).
> Other protocols layered on top of TCP/IP may add their own.
>
> Sockets are very reliable. The Internet is built on them.

Good to hear - thanks Nigel.

--
And loving it,

-Q
_________________________________________________
Qu0llSixFour@gmail.com
(Replace the "SixFour" with numbers to email me)

Qu0ll wrote:
> "Gordon Beaton" <n.o.t@for.email> wrote in message
> news:4706472b$0$3207$8404b019@news.wineasy.se...
>>
>> There are however a number of things that your application can do
>> wrong. The commonest ones to watch out for are thinking that TCP will
>> respect your message boundaries (it won't), or expecting that you'll
>> always read as much data as you requested (you won't).
>
> Can you please explain what you mean by not respecting message
> boundaries? I am sending data in 512 byte "packets", are you saying that
> this won't be respected by TCP?
>
> And what do you mean by not reading as much data as requested? When
> would this happen and why?

This is possibly not entirely correct but illustrates the issue nicely.


You write a bunch of 512 byte "packets".

TCP/IP sticks them all together and transmits when it deems fit - it may
send too-and-a-half of your "packets" in one message.

On the receiving side, when you read the first two times you get the 512
bytes you ask for. When you read the third time you might only get only
half of the 512 bytes you expect - you have to issue another read to
wait for the rest.

On Sat, 6 Oct 2007 16:38:57 +1000, Qu0ll wrote:
> Can you please explain what you mean by not respecting message
> boundaries? I am sending data in 512 byte "packets", are you saying
> that this won't be respected by TCP?

TCP gives you a data *stream*. So if you send 512 bytes and then
another 512 bytes, the only guarantee is that the 1024 bytes will
arrive in the correct order. There is nothing in the stream to
separate the two messages from each other, that's the responsibility
of your application, for example by terminating each message with a
unique character the application can recognize, or by prefixing each
message with its length.

> And what do you mean by not reading as much data as requested? When
> would this happen and why?

If you read the API documentation carefully, you'll see that all of
the methods for reading from a stream will only read "up to" the
requested amount.

Even though TCP provides a data stream, lower layers in the network
are packet based and your data will be sent in chunks that bear little
relation to the actual messages you sent. Sometimes two messages might
get sent in one IP packet, or a single message might span more than
one IP packet.

So an application that always requests 512 bytes might get 512 bytes
the first two times but only 436 bytes the third time, because the
remainder of the third message might still in transit, or needed to be
resent. The application needs to check how many bytes were actually
read, and read again until the desired number of bytes are received.

/gordon

--

"Gordon Beaton" <n.o.t@for.email> wrote in message
news:47073ae7$0$3209$8404b019@news.wineasy.se...

> TCP gives you a data *stream*. So if you send 512 bytes and then
> another 512 bytes, the only guarantee is that the 1024 bytes will
> arrive in the correct order. There is nothing in the stream to
> separate the two messages from each other, that's the responsibility
> of your application, for example by terminating each message with a
> unique character the application can recognize, or by prefixing each
> message with its length.
>
>> And what do you mean by not reading as much data as requested? When
>> would this happen and why?
>
> If you read the API documentation carefully, you'll see that all of
> the methods for reading from a stream will only read "up to" the
> requested amount.
>
> Even though TCP provides a data stream, lower layers in the network
> are packet based and your data will be sent in chunks that bear little
> relation to the actual messages you sent. Sometimes two messages might
> get sent in one IP packet, or a single message might span more than
> one IP packet.
>
> So an application that always requests 512 bytes might get 512 bytes
> the first two times but only 436 bytes the third time, because the
> remainder of the third message might still in transit, or needed to be
> resent. The application needs to check how many bytes were actually
> read, and read again until the desired number of bytes are received.

OK thanks to yourself and Thomas for the explanation. This explains why
sometimes not all data were being received by the applet. This had lead me
to conclude that TCP/Java/Sockets were unreliable but it appears to be fixed
now by re-reading until the full packet is received.

--
And loving it,

-Q
_________________________________________________
Qu0llSixFour@gmail.com
(Replace the "SixFour" with numbers to email me)

On Fri, 5 Oct 2007 23:52:19 +1000, "Qu0ll" <Qu0llSixFour@gmail.com>
wrote, quoted or indirectly quoted someone who said :

>So just how reliable are TCP and Java sockets over the actual internet?

If you look at the format of a TCP/IP packet you can get an idea.
see http://mindprod.com/jgloss/tcpip.html

TPC/IP packets have a 16 bit checksum. Any single bit error would give
you a different checksum. A multibit error has an 1 in 2^16 chance of
coming up with the valid checksum.

This is in addition to any packet-level checksums or hardware error
correction transparent to TCP/IP (e.g. error correcting modems).

The backbones now are fibre optic which very rarely get errors. The
problems comes from the rather wretched quality of the copper near
your end.

Perhaps someone knows of a tool to get the stats on the percentage of
packets getting through. The worse that number is the worse your
throughput and the greater your odds of an error sneaking through.

In my personal case they must be very rare. Nearly all my high volume
traffic is in ZIP files which have an additional checksum. I don't
see problems.

"But my tax return has to be correct. I submitted it with an
error-correcting modem."
--
Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com

On 06 Oct 2007 07:36:07 GMT, Gordon Beaton <n.o.t@for.email> wrote,
quoted or indirectly quoted someone who said :

>TCP gives you a data *stream*. So if you send 512 bytes and then
>another 512 bytes, the only guarantee is that the 1024 bytes will
>arrive in the correct order. There is nothing in the stream to
>separate the two messages from each other, that's the responsibility
>of your application, for example by terminating each message with a
>unique character the application can recognize, or by prefixing each
>message with its length.

If for some reason a message is garbled (by software or user error),
when you start reading, you won't necessarily pick up at the beginning
of the next message. You may pick up in the middle of the garbled
message, or part way through the next good one. If you want to be
able to recover, you need to reserve some magic start of message
pattern to scan for, that won't occur incidentally in data.

Socket communications generally require you to know the precise length
of things before you put them on the wire.
--
Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com