System Overview

System Architecture

RakNet can be broadly defined as 3 libraries: network communication, plugin modules that use network communication, and supporting functionality.

Network communication is provided with two user classes. RakPeer.h, and TCPInterface.h. RakPeer is the primary class that games will use and is based on UDP. It provides functionality such as connections, connection management, congestion control, remote server detection, out of band messaging, connection statistics, latency and packetloss simulation, ban lists, and secure connectivity.

TCPInterface is a wrapper for TCP, and is used to communicate with external systems that are based on TCP. For example, the EmailSender class, used to report crashes from remote servers, interfaces with TCP. It is also supported by most plugin modules, and is particularly useful for file transfers such as with the autopatcher.

Plugin modules in RakNet are classes that you 'attach' to an instance of RakPeer or PacketizedTCP. Attached plugins update automatically when the base class updates, and filter or inject messages into the network stream. Plugins provide automatic functionality, such as host determination in a peer to peer enviroment, file transfers, NAT traversal, voice communication, remote procedure calls, and game object replication.

Supporting functionality includes crash reporting, email sending through Gmail pop servers, an SQL logging server, and PHP based server lists.

RakPeer architecture

RakPeer.h provides the base functionality for UDP communication, and it is expected that most applications will use RakPeer as opposed to TCP. On start, RakPeer starts two threads - one thread to wait for incoming datagrams, and another to perform periodic updates, such as determing lost connections, or pings. The user specifies the maximum number of connections and an array of RemoteSystem structures is allocated to this size. Each connection or connection attempt is assigned to a Remote System structure, which contains a class to manage congestion control between the two systems. Connections are identified by SystemAddress or RakNetGuid, the latter of which is randomly generated and unique to all instances of RakPeer.

Connections are established through UDP messages that contain a connection request payload, and an 'offline message' identifier. The 'offline message' identifier is used to differentiate between actual offline messages, and connected messages that happen to match the profile of an offline message. Connection requests are sent repeatedly over a short timeframe to account for packetloss, and if supported uses a decreasing MTU for path MTU detection.

When a connection requests arrives, RakPeer transmits internal state data, such as the RakNetGUID, and checks this connection against the ban list, repeat connection list, and other security measures. If the connection was flagged as secure, the secure connection protocol activates and additional data is sent. On success, the user is informed about this, either with ID_CONNECTION_REQUEST_ACCEPTED, or ID_NEW_INCOMING_CONNECTION. Failure conditions are also returned in a similar fashion.

Outgoing messages from the user to connected systems are copied and internally buffered. If the outgoing message plus the header is larger than the MTU, the message is fragmented internally. On a periodic interval, outgong messages are aggreggated into a single datagram and sent according to the transmission contraints of congestion control and the MTU size. Datagrams that do not get an ACK are resent. NAKs are sent for missing datagram sequence numbers. Messages sent unreliably and cannot be sent within a user-defined threshold are deleted. Messages are sent in order of priority. Acks are not congestion controlled. However, resends and new sends are, with resends taking priority over new sends.

As stated, incoming datagrams arrive on a blocking recvfrom thread. When a datagram arrives, the timestamp is immediately recorded, and the datagram is pushed into a thread-safe queue for the processing thread to manage. The processing thread is signaled, such that it will either immediately process the message (if sleeping) or process the message at the next available time.

Incoming datagrams are checked for the byte sequence that indicates if the sender thought it was connected or not. In addition, the source IP address is checked. If a message is flagged as unconnected, and the sender is not connected to us, the message is checked against a narrow range of accepted types, such as connection requests, or out of band messaging capabilities. If a message is flagged as connected, and the sender is connected to us, then it is processed through the ReliabilityLayer class, for congestion control and other communication related information (ACKS, NAKS, resends, encryption, reassembly of large messages).

Connected messages are first processed by RakPeer. This is currently only used for periodic pings and to detect lost connections should the user not send data within a given threshhold. All other messages are processed by plugins, or returned to the user. Calling RakPeer::Receive() ticks all plugins' update functions one time, and returns one message. Message returned to the user are returned from RakPeer::Receive(), one message per call. It is necessary to call Receive() in a loop to get all messages, until no further messages are returned.

Other systems

The NetworkIDObject class provides the ability for systems to refer to common objects and is used by object member remote function calls. It is a fairly simple system that has the server assign a number to objects as they are created. When a client creates an object, it is responsible for sending this event to the server, getting the number to use when the object is created on the server in turn, and then assigning this number to the object. This way systems can refer to objects on multiple systems which would otherwise be impossible since pointer addresses would probably not be the same. The easiest way to use this system is to derive from it. However, you can also include it as a member variable of your base class and call SetParent(parentClassPointer). The disadvantage of this architecture is that it relies on a single system being present to assign network IDs. This is addressed by having NetworkIDs be a RakNetGUID / NetworkID pair instead of a simple number by default.

The SystemAddress structure is what RakNet uses to represent remote systems. It is the binary encoding of the IP address along with the port of that system.

The BitStream class, located in BitStream.h, is natively supported by RakNet. This is both a user class and an internal class. It is primarily used to write single bits to a stream and for automatic endian swapping, which can be enabled by commenting out __BITSTREAM_NATIVE_END in RakNetDefines.h. It is disabled by default for speed, since most users do not need endian swapping.

For a visual representation of the architecture, refer to the UML Diagram
Next page: Detailed Implementation

Bandwidth overhead

Post 3.6201, using new congestion control

Per datagram:
1 byte for bitflags
4 bytes for timestamp, used to calculate RTT for congestion control
3 bytes for sequence number, used to lookup datagrams for ACKs

Per message:
1 byte for bitflags
2 bytes for message length
if (RELIABLE,RELIABLE_SEQUENCED,RELIABLE_ORDERED)
A. 3 bytes for sequence number, used to prevent returning to the user duplicated messages
if (UNRELIABLE_SEQUENCED,RELIABLE_SEQUENCED,RELIABLE_ORDERED)
A. 3 bytes for sequence number, used to reorganize messages on the same channel in order
B. 1 byte for ordering channel
if (message over MTU)
A. 4 bytes for number of splits, not compressed to improve performance
B. 2 bytes for identifier for which split this is
C. 4 bytes for index into number of splits, not compressed to improve performance

3.6201 and earlier in the 3.x series

Per datagram:
1 bit for bitflags
8 bytes for timestamp, used to calculate RTT for congestion control

Per message:
4 bytes for message length
4 bytes for sequence number, used to prevent returning to the user duplicated messages
4 bits for bitflags
if (UNRELIABLE_SEQUENCED,RELIABLE_SEQUENCED,RELIABLE_ORDERED)
A. 4 bytes for sequence number, used to reorganize messages in order
B. 5 bits for ordering channel
if (message over MTU)
A. 4 bytes for number of splits, but compressed, so used 1-2 bytes on average
B. 4 bytes for identifier for which split this is
C. 4 bytes for index into number of splits, but compressed, so used 1-2 bytes on average

A message is the data you send from the game. All messages you send to RakNet between RakNet update ticks are grouped together in one datagram. So if you send 1 message only, then the overhead is 1 datagram + 1 message. If you send 5 messages, then it's 1 datagram + 5 messages. If you send 1 message, but it's 10 times bigger than the MTU, then it sends 10 datagrams, each containing 1 message (the message gets split up)

See Also

Index Compiler Setup Detailed Implementation Introduction Tutorial UML Diagram