Designing a Distributed System for an Online Multiplayer Game — Game Server (Part 5–1)

Networking

There are two channels to communicate with the game server, the reliable channel to publish the game events (which need to be delivered to the game client with a guarantee and in order), and a fast channel to stream the game world states (which are called snapshots).

TCP Socket

For the main events of the games, we use the TCP protocol to guarantee that all events are delivered without corruption and in order. The WebSocket is useful here, it upgrades an HTTP request as a long-living connection and uses it to read and write bytes.

UDP Socket

All snapshots and users’ inputs must be delivered as fast as possible without any acknowledgment or latency overheads. The UDP is not aware of any connection and is stateless, but we need a virtual connection here to register and authenticate the user and send messages to it, to achieve this, I developed an open-sourced package to manage the UDP clients. It initiates a virtual connection between the server and the client. We’ll check this out in detail after other challenges.

Security

There is a common challenge for both TCP and UDP channels, security. All packets are transferred over the internet and they can be monitored by the man-in-the-middle. Thus, every data must be encrypted to be safe.

udpsocket

I developed an open-sourced UDP socket package to manage the clients & create a virtual connection:

Features:

  • Handle handshake
  • Authentication
  • Manage clients & connections
  • Encryption
  • Implementable with any other encryption methods, transporting data structure protocols, and authentication policies

Encryption

Each game server has a unique random RSA private key. The client downloads the game public key using the game manager HTTP server over TLS, so the public key is downloaded secure and the client (player) must have access to the game to download it, however, it’s the public key and doesn’t matter. The game client uses this public key to encrypt its AES encryption key for the game server.

UDP Handshake — Client Hello message encryption
Websocket Handshake key exchange

Authentication

I used Firebase for user authentication. After logging in to the game, it issues a token for each session, then the token is used to authorize user requests.

Encoding

To transport the data between the client and the server, we need to encode data before encryption. I used two different formats for UDP and TCP channels. as I mentioned before, the udpsocket package (which is used for the UDP channel) supports a custom implementation of the encoder, therefore I used the MessagePack as an encoder for the UDP and JSON for the TCP channel.

Event Broker

The game server publishes internal events to communicate with the game manager via the broker. Some events like GameServerReady with selected port numbers.

Pipeline

Let’s put these parts together and review the whole pipeline:
The game server tries to find a free port on the node to expose the TCP and UDP sockets. when the ports are bound, the game server publishes a readiness event with listening ports to the broker and waiting for the clients to connect.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sajjad Rad

Sajjad Rad

Currently a software engineer, always an adventurer