I started work on this project by the end of April when accepted student's list was published. I've created overall plan how I'm going to acomplish my task:
- Create some test code in C++ to test against libjingle's implementation and research how the protocol works (done already while applying)
- Establish packet level communication with C++ pseudotcp (ensure that header fields transfer correctly, byte order etc.)
- Finish refactoring code and transfer some data between Java and C++
- Write unit tests and make sure that all pass. Full set of test also exists in libjingle so they require porting.
- Test performance and compare it with C++.
- Create final Java interface for the protocol and integrate it with ice4j. A diagram with classes overview from my GSoC application can be found here. Now they have different naming, but the concept is almost the same.
As a starting point I've copy pasted PseudoTcp class directly from C++. In this class there is all protocol's logic and as I've mentioned in previous post it requires specific environment to run correctly. I had to create one thread for tracking time changes and another to handle network socket.
My goal was to exchange some sample packets and the first problem I encountered was that there aren't unsigned data types in Java. To handle header fields which are type of unsigned int 32 I use long primitive Java type. This allows to perform any calculations on this kind of fields. To transfer them through the network there are required special routines which will write and read them from byte buffers. For example function which stores long in a buffer may look something like this:
void long_to_bytes(long uInt, byte[] buf, int offset)
{
buf[offset] = (byte) (( uInt & 0xFF000000L) >>> 24);
buf[offset + 1] = (byte) (( uInt & 0x00FF0000L) >>> 16);
buf[offset + 2] = (byte) (( uInt & 0x0000FF00L) >>> 8);
buf[offset + 3] = (byte) (( uInt & 0x000000FFL));
}
There is also available special buffer class in java.nio called ByteBuffer, but it worked only in one direction and finally made no use for me. At first it looked very interesting as it has also included option to specify the byte order. I didn't knew by then that Java have network byte order by default.
PseudoTcp header
//
// 0 1 2 3
// 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 0 | Conversation Number |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 4 | Sequence Number |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 8 | Acknowledgment Number |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// | | |U|A|P|R|S|F| |
// 12 | Control | |R|C|S|S|Y|I| Window |
// | | |G|K|H|T|N|N| |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 16 | Timestamp sending |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 20 | Timestamp receiving |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 24 | data |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//
//////////////////////////////////////////////////////////////////////
- Conversation Number - identifies current conversation
- Sequence Number - info about the place of current packet in whole conversation's data stream
- Acknowledgment Number - idicates up to what sequence number receivied data has been acknowledged
- Control - one byte which identifies control action, currently only "connect" is used
- Window - bytes count that receivier is able to accept
- Timestamp sending - clock count in milliseconds idicates when packet was sent
- Timestamp receiving - clock count shows when we have receivied last packet from remote side
After packets were transferred sucessfully and header fields were properly parsed I moved on to data transfer. I spent most time on dealing with the fifo buffer class. Current protocol implementation uses one for storing receivied data and one to queue data which is waiting to be sent. It works like a queue of bytes, but has also few specific functions. For example writing or reading at specified offset from current buffer's position, but without affecting final buffer's position. It may be used to store packets which are delivered out of order.
My first implementation used java.nio.ByteBuffer internally. This class has ability only to write to the buffer or read from it at the same time. I kept track of read position not to rewind the buffer at every single read operation, but when it reached one half of total buffer's size. But it showed up to cause sending too small window to remote side because buffer's space was cleared too rarely. I discovered it only when window unit tests were failing. Increasing the frequency showed decent decrease in performance.
Second one which I called EfficentFifoBuffer :) was based on a single byte array. I stored read and write positions and used them to track available space and buffered data. They loop through the array with use of "modulo length" operation. This required much more work to have this working, but resolved performance issues.
Porting of the unit tests also required plenty of work as in libjingle there is some kind of custom threading model. Many operations are performed by posting some messages to the threads which they were processing. There is also an option to send delayed messages which were emulating delayed data packets in the unit tests.
At the moment almost all tests are completed except only one window test which I plan to finish later as I don't have any easy solution to make it working by now. I work on finishing class PseudoTcpSocket which is public interface of the protocol. It uses DatagramSockets to communicate through the network and controls required threads. Maybe it would be nice idea to use some interface instead of DatagramSocket directly. It would enable porting this code for example to Java ME which has different class to handling UDP datagrams. Also I have to discuss final interface on the dev list and determine how this will be integrated with ice4j library.
