Work summary

  As the GSoC is comming to an end, it's time to make a final summary of what's been done so far and what may be missing. It's been about a month since Java implementation of PseudoTCP had been commited to Ice4j's repository. From that time I've discovered miscellaneous bugs and I kept on fixing them. I am also very happy that some people started using things I've done. That is really motivating.

  Later I started working on file transfers between the Jitsi and the Google Talk. It required from me to learn about Jabber and XMPP, how all that stuff works. Also at the middle stage of transfer process there is ICE protocol involved which was causing me some troubles, but allowed me to learn even more. Finally I've managed to transfer succesfully files between both clients. Thanks to that I've found and fixed even more bugs in protocol's implementation. It looks that it works stable now, but for sure I can't say that it's a mature project.

  About the things that are not done yet. First I'll have to finish the transfer as now it's implemented only in one direction. I hope that it won't take too much as now I know how all that stuff works and there are less things that can surprise.

  Regards the protocol there is missing a closing procedure. The bad thing is that I can not just add it to my implementation as there are client apps that already use current version of the protocol. For now the only way to get around this issue is to have another method of notifying about transfer end. In file transfer which I've mentioned, we know the file size before the transfer because of the signaling protocol(Jabber/XMPP). Another method is to use some protocol in communication like HTTP where double CRLF means end of request/response and before content transfer we are informed about expected size. This is how multiple files are being transfered in as single sharing session with Google Talk client.

  There may be also few other things that need to be done or improve, but that is what will come with time. I'd like also write a few words about my general experience with the GSoC. It is a great opportunity to see how the open source world works and how fun it can be. During that time I had an occasion to meet many interesting people and a pleasure to work with them. Special thanks to my mentor Emil Ivov and whole Jitsi team for help and assitance. Also I'd like to mention that thanks to Emil I had an opportunity to go to Strasburg and spend some time with part of the team. Thanks to Yana and Vincent for a great time there :)

Integration with Ice4j

  Last time I worked on integrating pseudotcp code with ice4. It was quite simple as all I had to do was change package names and include my unit tests in the ice4j's test suite. The last one required to switch to older version of JUnit. It doesn't support annotations so I had to derive my base test class from the TestCase and rename all test methods to start with lower case "test" keyword.
  Next task was to create an example how pseudotcp may be used with Ice4j library. The basic scenario is that there are two ice clients establishing UDP connectivity. Then selected addresses are used for pseudotcp transfer. Optionally STUN and TURN harvesters may be used - by default it works only with local interfaces.
  Later I've finally switched to java.util.logging.Logger as a logging mechanism. The message templates helped me to resolve the issue with many string concatenations for messges that are not valid for current logging level. Profiler showed that it had visible impact on performance, as there are many messages generated while the protocol is working. Here's an example:

logger.log(Level.FINE,
           "--> <CONV={0}><FLG={1}><SEQ={2}:{3}><ACK={4}><WND={5}><TS={7}><TSR={8}><LEN={9}>",
           new Object[]{
                   seg.conv,seg.flags,seg.seq,(seg.seq+seg.len),seg.ack,seg.wnd,seg.tsval,seg.tsecr,seg.len
                   }); 


The main advantage of this approach is that the string will be generated only if message will be displayed(logging level will be set at least to FINE).

How's it going

   It's been a while since my last post as I had a small break for the exams. Recently I worked on fixing tcp window unit tests. I had some problems with porting them from libjingle because of different threading model. In libjingle these tests were based on thread's message queue, but here I didn't wanted to implement such thing, because it wouldn't fit to the whole ice4j library where practically only pseudo tcp would use such feature. I based my solution on standard wait() and notify() monitor methods of java Object's class.

   To test tcp window I used two instances of pseudo tcp called "local" and "remote". First local instance fills up send buffer. When it's filled, the byte count which were put into means the send window size. In this threading model in background runs another thread called the clock thread. Between two calls of the send method it may decide to send buffered data so we will fail to estimate the send window's size properly. So I used data 2x size of the send buffer to be sure that I'll fill it in a single call.

   Then it must wait until data will be available in remote instance's buffer. I used polling here for simplicity. It runs in a loop with sleep and checks if the data arrived to the remote instance or the remote's receive buffer is full. At this point I've encountered one issue with window scaling which I'll describe later.

  When all data put into the send buffer had arrived to remote or it's receive buffer is full I read all the data and store bytes count. This count means estimated receive window size. That sequence of send and read was used to estimate and check if both windows are set correctly.

   Another thing we wanted to check was if the receive window is fully opened after the data has been read(and if the info about this fact was correctly propageted to the local instance). To do that, the routine of send/receive was repeated twice and there were performed calculations on subsequent send positions compared with estimated receive window size. When this difference was close to receive window size it was considered a positive result.

   While working on this test I considered it hacky and I was thinking about dropping it. But it helped me to track few important bugs and also thanks to this I've decided to improve buffer class which is now much faster that the first implementation. First implementation was causing the receive window not to get fully opened as it was rewinded when it was filled up to 1/2. So for the half of time it was hiding available buffer's space.

    Another thing I learnt was about the window scaling. When this tests were run with buffer's size greater than "32 bit unsigned int" then the window scaling is involved(32 bits are reserved for window size in tcp header). In RFC it's defined as an extension of TCP for enhanced performance. In some networks it's good to send larger packets to decrease overhead.

   So in basic the window scale option is an additional byte which is used to bitwise shift window size value. For example when we want to use window of size 100 000 bytes we need to use scaling factor of 1. That means that we put into the header 100 000 >> 1 = 50 000 as window size and 1 as "window scale option". When the receivier gets that then he has to do the reverse operation: 50 000 << 1 = 100 000.

   As I described earlier I used polling to check if all data put into the buffer is receivied by remote side. When window scaling is involved I'm not able to transfer exactly all data in a single call because small part of window is hidden. It happens because window size isn't constant during conversation and when there will be only 1 byte left of the free space avaialble in a buffer we will lost it with the shift operation : 1 >> 1 = 0. When the receivier makes reverse operation he will still get 0. As we know when window gets closed(is equal to 0) that means that remote side have some problems with processing the data and we should wait for it to open before sending any more data.

   It was causing thread blocking and timeouts when none of these conditions were met: all data wasn't transferred and the receivier's buffer wasn't fully filled. I solved this by introducing a margin of error to that calcultion based on used window scale option. For scale option equal to 1 there will be 1 byte hidden, for 2 there will be 3 bytes hidden, for 3 -> 7 and so on. But I'm not sure if it's a good idea as it looks a bit hacky.

    That's not the issue in normal operation when something will read data on the remote side and receive window will open. This problem was specific only for this test altthough I was considering this as a bug at first.

   Now my task is to integrate the protocol with ice4j library and create some usage example of how we can use it together with an ice protocol. At this point I've managed to merge my project with ice4j and include unit tests inside. In about a week I should make first commit to the ice4j lib as there are still a few things I have to take care of. Then the temporary project "pseudo-tcp-java-impl" will probably disappear.

First progress report

    As I've mentioned in previous post my port of pseudo-tcp to java is at advanced stage. In this post I'll try to cover most of things which has been done until now. By the way - project's source code is published at http://code.google.com/p/pseudo-tcp-java-impl/

    I started work on this project by the end of April when accepted student's list was published. I've created overall plan how I'm going to acomplish my task:

1. Create some test code in C++ to test against libjingle's implementation and research how the protocol works (done already while applying)
2. Establish packet level communication with C++ pseudotcp (ensure that header fields transfer correctly, byte order etc.)
3. Finish refactoring code and transfer some data between Java and C++
4. Write unit tests and make sure that all pass. Full set of test also exists in libjingle so they require porting.
5. Test performance and compare it with C++.
6. Create final Java interface for the protocol and integrate it with ice4j. A diagram with classes overview from my GSoC application can be found here. Now they have different naming, but the concept is almost the same.

The beginning

    As a starting point I've copy pasted PseudoTcp class directly from C++. In this class there is all protocol's logic and as I've mentioned in previous post it requires specific environment to run correctly. I had to create one thread for tracking time changes and another to handle network socket.
    My goal was to exchange some sample packets and the first problem I encountered was that there aren't unsigned data types in Java. To handle header fields which are type of unsigned int 32 I use long primitive Java type. This allows to perform any calculations on this kind of fields. To transfer them through the network there are required special routines which will write and read them from byte buffers. For example function which stores long in a buffer may look something like this:

void long_to_bytes(long uInt, byte[] buf, int offset)
{
 buf[offset] = (byte) (( uInt  & 0xFF000000L) >>> 24);
 buf[offset + 1] = (byte) (( uInt  & 0x00FF0000L) >>> 16);
 buf[offset + 2] = (byte) (( uInt  & 0x0000FF00L) >>> 8);
 buf[offset + 3] = (byte) (( uInt  & 0x000000FFL));
}
    There is also available special buffer class in java.nio called ByteBuffer, but it worked only in one direction and finally made no use for me. At first it looked very interesting as it has also included option to specify the byte order. I didn't knew by then that Java have network byte order by default.

PseudoTcp header

//////////////////////////////////////////////////////////////////////
//
//    0                   1                   2                   3
//    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
//    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//  0 |                      Conversation Number                      |
//    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//  4 |                        Sequence Number                        |
//    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//  8 |                     Acknowledgment Number                     |
//    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//    |               |   |U|A|P|R|S|F|                               |
// 12 |    Control    |   |R|C|S|S|Y|I|            Window             |
//    |               |   |G|K|H|T|N|N|                               |
//    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 16 |                       Timestamp sending                       |
//    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 20 |                      Timestamp receiving                      |
//    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 24 |                             data                              |
//    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//
//////////////////////////////////////////////////////////////////////

• Conversation Number - identifies current conversation 
• Sequence Number - info about the place of current packet in whole  conversation's data stream
• Acknowledgment Number - idicates up to what sequence number receivied data has been acknowledged
• Control - one byte which identifies control action, currently only "connect" is used
• Window - bytes count that receivier is able to accept
• Timestamp sending - clock count in milliseconds idicates when packet was sent
• Timestamp receiving - clock count shows when we have receivied last packet from remote side

FifoBuffer

  After packets were transferred sucessfully and header fields were properly parsed I moved on to data transfer. I spent most time on dealing with the fifo buffer class. Current protocol implementation uses one for storing receivied data and one to queue data which is waiting to be sent. It works like a queue of bytes, but has also few specific functions. For example writing or reading at specified offset from current buffer's position, but without affecting final buffer's position. It may be used to store packets which are delivered out of order.

    My first implementation used java.nio.ByteBuffer internally. This class has ability only to write to the buffer or read from it at the same time. I kept track of read position not to rewind the buffer at every single read operation, but when it reached one half of total buffer's size. But it showed up to cause sending too small window to remote side because buffer's space was cleared too rarely. I discovered it only when window unit tests were failing. Increasing the frequency showed decent decrease in performance.

    Second one which I called EfficentFifoBuffer :) was based on a single byte array. I stored read and write positions and used them to track available space and buffered data. They loop through the array with use of "modulo length" operation. This required much more work to have this working, but resolved performance issues.

Unit tests

    Porting of the unit tests also required plenty of work as in libjingle there is some kind of custom threading model. Many operations are performed by posting some messages to the threads which they were processing. There is also an option to send delayed messages which were emulating delayed data packets in the unit tests. 

What next ?

    At the moment almost all tests are completed except only one window test which I plan to finish later as I don't have any easy solution to make it working by now. I work on finishing class PseudoTcpSocket which is public interface of the protocol. It uses DatagramSockets to communicate through the network and controls required threads. Maybe it would be nice idea to use some interface instead of DatagramSocket directly. It would enable porting this code for example to Java ME which has different class to handling UDP datagrams. Also I have to discuss final interface on the dev list and determine how this will be integrated with ice4j library.

Hello there !

After all efforts I've put into my application for GSoC 2012 I've managed to get involved in this event. My host organization is Jitsi.org under umbrella of XMPP Standards Foundation. The project that I work on is pseudo-tcp protocol implementation in Java. It's about reliable and controlled data transfer, but lightweight at the same time. This can be achived by using UDP as a transport layer and implementing only the most essantial parts of TCP specification. In fact the protocol is already implemented in Google's libjingle in C++. My implementation must be fully comapatible with this one.

My work has started when I was writing my application. I've spent a few days on digging into libjingle's code and documentation to build it and run. After that I realized that it wasn't so simple to run the protocol itself on some sockets. It was hidden below few other layers which were no use for me and it was impossible to get into it in such a small amount of time I had. There was available strictly protocol's logic class which requires specific environment to run. It exposes interface to get notified about progress in time and data receivied from another side. So in fact there must be one thread to track time and another to handle socket. It may be not much for someone who writes in C++ everyday, but I had some problems with that as I used it some time ago only for small projects during studies. Finally I've managed to create some code which transferred data between two UDP sockets.

The other task I worked on while applying for the project was to find out how the protocol may be integrated with ice4j library(ICE protocol for Java). So I had read introduction parts of ICE's RFC and tried to understand how it works. As I understood it's about choosing the best connection option of all that are available. When we are connecting using some application through the internet there are usually few routers on the way and our external ip address is different that the local one. Sometimes we can be accessed by local or external address or by using STUN server or maybe something else. ICE is there to choose the best option for us.

My first idea was to add new transport type to ICE library like there are UDP and TCP already. Later I thought that maybe there's no need for that and it would be enough when we will only discover UDP connectivity (there would be no checks if there's pseudo tcp on another side). Then just use UDP adresses to establish pseudo-tcp connection. In this case we should decide on some higher level that we are going to use pseudo-tcp on that sockets. However I'm still not sure which approach should I use and I'll decide about it later when the protocol's implementation will be finished. Probably jitsi dev team will help me with that.

It's been a while since I've started working on the project although official date was the 21st of May. Now the protocol is almost completed and I'll write about that next time.