Hadean Connect, referred to as Connect in these docs, provides a mechanism for producing a real-time filtered stream of state data to a large number of connected clients with each receiving their own unique view aggregating data from a single or multiple data sources, most typically a spatial simulation.
The main objectives of Connect's design are to:
- Aggregate data from multiple distributed simulation
- Offload interest management logic from a simulation to reduce load on servers and reduce bandwidth to clients
- Enable user customisable logic to determine what data is sent to clients and when to allow optimisation around the exact use case
- Be remotely deployable to put data closer to clients reducing "last mile" transmission costs and reducing latency
Connect will allow you to build larger simulations with lower latency and more connected clients than the standard client > server models
Connect is able to store state data locally in a queryable format to relay real time data streams out to multiple connected clients. Connect is also able to connect to multiple upstream servers to aggregate data, allowing large numbers of server instances to be combined into a single larger combined view than would otherwise be possible on a single server instance.
Hadean's Simulate product makes use of this capability to broadcast data generated from spatial simulations. Thousands of cloud distributed processes each work on a subset of the simulated space, then combine their state through Connect enabling connected clients to gain visibility into any region of space without worrying about which data centre / cloud provider the logic for that region may be running on.
Connect allows for "out of the box" or customised logic to be applied to data aggregated from your simulation before sending it out to connected clients, typical uses of this would include
- Implementing interest management (also known as network relevancy) allows data to be filtered both in terms of content but also in frequency of update ensuring that only important information is sent to a given client at a reasonable frequency. Connect comes supplied with high performance generic implementations of interest management, however it is also possible to build custom logic to best suit your implementation
- Example use cases
- Only information about entities that a client would be able to see can be filtered to dramatically reduce bandwidth usage
- Clients may only be interested in frequent updates about entities in close proximity while less frequent updates may be acceptable for distant ones.
- Clients may care about other clients on their team irrespective of distance, but only receive data about competing team clients if they are in visible range
- Adding security layers by limiting the data visible to any given player. E.g. ensuring a player doesn’t receive the private data of other players.
Connect clusters can be deployed to span multiple data centres across cloud providers around the world allowing clients to connect to Connect with the lowest latency available.
Connect in a network
While spatial simulation servers are often hosted in a single data centre, Connect nodes are deployed to edge data centres, so that each client has access to a Connect instance in their region. Only the connections between clients and Connect nodes go through the public internet, connections between clients and workers can use direct inter data centre connections which have greater bandwidth and lower latency.