Protocol

Documentation Objective

This document describes the protocols used to communicate with Aether Engine.

Overview

Aether's engine simulation data is sent to the client via the muxer. In order for the data to be correctly processed by the muxer and the client, all the components need to agree in the data format.

Protocol

The protocol defines the format of the data that is sent over the wire from the simulation to the muxer and from the muxer to the client

The data format can be defined in different ways, here we will explore two used in the Aether SDK.

C structs

The simplest and quickest way to define a protocol is as set of trivially copyable structs (e.g it is valid to copy them using memcpy). This is the approach taken in the generic-netcode-protocol.

We can place the entity data we want to be sent inside a struct and send the in-memory representation directly. We use the HADEAN_PACK macro to eliminate padding to ensure that the data layout is consistent between different compilers.

HADEAN_PACK(struct Entity {
    vec3f position;
    quat rotation;
    unsigned int color;
    unsigned long id;
    float size;
});

Some of the drawbacks of this approach are:

does not account for endianness differences between platforms
does not perform any sort of optimization of the data sent
it may be cumbersome to read/write if the client is not written in C/C++

Cross platfrom protocol using FlatBuffers

To solve the issues related to endianness, platform and languages definitions a serialization library with implementations on the different platforms can be used.

The SDK provides an example of using FlatBuffers to define the protocol.

struct Entity {
  position: Vec3;
  rotation: Quat;
  color: uint;
  id: ulong;
  size: float;
}

The protocol definition is compiled using the FlatBuffers compiler to generate code for the different platforms where it will be used. See using the compiler for more info.

Integrating flatbuffers protocol with generic-netcode

The SDK provides an example of using FlatBuffers to define the protocol and using it with the provided generic netcode.

The generic netcode provides an marshalling interface to use for the protocol. Generic netcode also comes with a trivial marshaller class that can be used with a protocol defined as a set of trivially copyable structs as detailed in the generic-netcode-protocol. To use a different protocol with the generic protocol a new marshaller interface has to be defined. The marshaller will encode the information in and out of the protocol format.

In the SDK example the protocol is defined in PhysicsDemo/Simulation/Protocol/protocol.fbs. For example the information for a simulation entity is:

struct Entity {
  position: Vec3;
  rotation: Quat;
  color: uint;
  id: ulong;
  size: float;
}

In the SDK example the new marshalling interface for the simulation and the muxer is defined in PhysicsDemo/Simulation/Protocol/cpp/marshalling.h using the C++ structs generated by the FlatBuffers compiler from the definitions.

#include 'protocol_generated.h' // header generated by FlatBuffers compiler defines Entity and Worker class
struct physx_marshaller_traits {
    using entity_type = Entity;
    using static_data_type = aether::monostate;
    using per_worker_data_type = Worker;
    using config_type = physx_marshalling_config;
};
class physx_marshaller : public aether::netcode::marshaller<physx_marshaller_traits> {
    ...
    void add_entity(const Entity &entity) override;
    ...
};
class physx_demarshaller : public aether::netcode::demarshaller<physx_marshaller_traits> {
    ...
    std::vector<Entity> get_entities() const override {
    ...

Integrating a FlatBuffers-based protocol with the generic netcode on the client side

If the client is written in C++, as in the case of the OpenGL client in the SDK's physics demo, the same marshalling code can be reused on the client side. However, if the client is written in another language (as is the case for the Unity client) a new demarshaller is implemented using the C# definitions generated by the FlatBuffers compiler.

using FlatBuffers
public class Demarshaller
    {
        ...
        public static void DecodePacket(byte[] data, int size);
        public static List<global::Entity> getEntities();
        ...
    }

A high level overview of integrating Aether, Unity and FlatBuffers is described in the Unity integration overview. An example implementation of a C# demarshaller can be found in the UnityClient project in the Physics Demo installed with the SDK.

Integrating protocols with custom netcode

The generic netcode also imposes the restriction of using the same protocol from the simulation to the muxer as from the muxer to the client.

If the desired protocol cannot be written in a way that conforms to the marshalling interface then the generic netcode cannot be used for net relevancy. However, it is still possible to fork the generic netcode (as is done for Aether Hunt 2) or write a net relevancy implementation from scratch using the netcode interface provided by the muxer.

Protocol versioning

To prevent a mismatch in the protocol used to construct the messages between the simulation and the muxer or between the muxer and the client a version check is added in the marshalling.

Each protocol message contains a header with a protocol-specific magic number, a protocol version and a compression flag. This enables identification of the protocol and the protocol version used to encode the message and determine if the rest of the message is compressed or not. In the provided examples, the protocol versioning is done outside FlatBuffers so that the protocol versioning continues to work even when a different protocol specification mechanism is used.

int write_protocol_version(Writer &writer) const {
    using namespace aether::io::byte_order;
    const le_uint64_buf magic = PHYSX_MARSHALLER_MAGIC;
    const le_uint16_buf version = PHYSX_MARSHALLER_VERSION;
    auto ret = aether::write_all(writer, &magic, sizeof(magic));
    if (ret != 0) { return ret; }
    ret = aether::write_all(writer, &version, sizeof(version));
    return ret;
}
std::vector<char> encode() const override {
    std::vector<char> data;
    aether::io::in_memory_writer<char> writer(data);
    // Write protocol header
    int ret = write_protocol_version(writer);
    const uint8_t compression_on = config.use_zstd;
    // Write flag to indicate if compression is enabled
    ret = aether::write_all(writer, &compression_on, sizeof(compression_on));
    {
        if (compression_on) {
            aether::io::zstd_writer<decltype(writer)> compressed_writer(writer);
            ret = write_content(compressed_writer);
        } else {
            ret = write_content(writer);
        }
        assert(ret == 0);
    }
    return data;
}

In the demarshaller the header is read for each packet and the marshall magic number and protocol is checked:

int read_protocol_header(Reader &reader) {
    using namespace aether::io::byte_order;
    le_uint64_buf magic;
    auto ret = aether::read_exact(reader, &magic, sizeof(magic));
    if (ret != 0) { return ret; }
    le_uint16_buf version;
    ret = aether::read_exact(reader, &version, sizeof(version));
    if (ret != 0) { return ret; }
    if (magic != PHYSX_MARSHALLER_MAGIC) { return -1; }
    if (version != PHYSX_MARSHALLER_VERSION) { return -1; }
    return 0;
}

Logging

Introduction