Comment on page
Cell Initialisation
This section only applies to versions 4.2 and later.
Beginning with version 4.2 of Simulate, cell initialisation now occurs asynchronously to the main simulation; that is, the simulation will continue to execute whilst cell workers are spawned and initialised in the background. In previous versions, the simulation would block whilst cell workers were initialised and entity handover was completed which in some cases could cause noticeable delays - particularly if additional machines were required to be spawned, or if user code loaded external libraries or data. This section explains the asynchronous behaviour and how to configure or disable it if required.
Cell initialisation - or more completely, the spawning of additional worker processes across one or more extant or new machines and the setting up of their initial state - can occur in the following situations during a simulation:
- A cell divides as a result of the load factor increasing beyond the split threshold
- An entity moves into a region of unmanaged space
- Two cells merge as a result of the load factor dropping below the merge threshold
Simulate meets demand for additional cells by maintaining a worker pool.
The Simulate engine worker pool and how it behaves in each of these scenarios is outlined in the remainder of this section.
Simulate Engine maintains a pool of worker processes on standby ready to be brought into the simulation when required. If the number of standby processes falls below a certain threshold, new workers are spawned asynchronously.
As part of the spawning process, the
user_state object will be created. You can initialise custom code in user_state::initialise_cell providing that the code does not require access to any cell-specfic information as the worker is not currently simulating an active cell.Any resources created in
user_state::initialise_cell must be cleaned up in user_state::deinitialise_cell as the user_state object can be re-used with different workers.The worker pool can be configured via the octree_params struct passed during construction of the octree (see Passing Configuration Options to Other Aether Processes):
/* If the ratio of available workers in the worker pool drops below
this value, we will try to spawn more workers when refilling the pool */
double available_worker_target_ratio{0.2};
// Specify the maximum number of workers to spawn when refilling the worker pool
size_t worker_pool_max_refill{32};
Cell division occurs when a worker reports an estimated load above the configured threshold (see previous section Simulation Lifecyle). New workers will be initialised asynchronously to replace the current overloaded one.
During intialisation,
user_state::intialise_cell(...) will be called and you will have access to cell-specific area information, as the worker is now aware of the area it will be simulating. The get_tick() function will return the tick at which the split operation started.Note that
get_tick() may return a different value if called at the entry and exit of intialise_cell depending on how far the simulation has progressed whilst cell initialisation is in progress. If you need to refer to the value of get_tick() at different points in your custom initialisation code, it's recommended that you save it to a variable on entry.Once the new worker(s) have completed initialisation, the are added to the simulation, entities are handed over from the originating cell, and the worker process for the originating cell will be returned to the worker pool after
user_state::deinitialise_cell(...) has returned.As this process is executed asynchronously, the simulation does not pause whilst additional workers are initialised.
As with the Worker Pool, configuration is done within octree_params:
/* Threshold for splitting the worker cells as returned from
aether::user_cell_state_interface::estimate_load */
float split_at;
Cell merging occurs when the combined load of a group of workers falls below the threshold at which they are configured to merge. At this point, one worker process will be initialised to take over from the currently underutilised processes.
The newly initialised combined cell worker process follows the same initialisation and handover process as outlined above in Cell Division (Split).
If there are no worker processes available, the specified merge threshold will be overridden and incrementally increase until it reaches the split threshold. This will cause more cells to merge and free some worker processes to cover new simulation areas.
The merge threshold is specified within octree_params:
/* Threshold for merging the worker cells as returned from
aether::user_cell_state_interface::estimate_load */
float merge_at;
When an entity moves outside of the area currently managed by existing cells - i.e. outside of the entire simulated area - a new worker will be initialised to cover that area.
The newly initialised combined cell worker process follows the same initialisation and handover process as outlined above in Cell Division (Split).
As the new workers are being set up, the entity will continue to be simulated in its current cell even though it is outside the cell boundary. If your user code is unable to handle this behaviour - for example, due to an external library or data dependency - asynchronous worker creation for new areas should be disabled (see below).
Note that if two or more entities move outside of their respective cell boundaries, they may not be aware of each other despite being close together as they are managed by separate worker processes until the new area cell is fully initialised.
This new functionality introduces additional feature flags. See Runtime Configuration - Feature Flags for more information on using feature flags.
Flag | Purpose |
|---|---|
DISABLE_ASYNC_CELLS | Used to disable asynchronous cell initialisation. Set this flag if your simulation is required to be deterministic. Asynchronous mode means that cell creation will occur at different tick counts on subsequent runs of the same simulation. To prevent this, set this flag to revert to classic cell initialisation behaviour. |
DISABLE_ASYNC_NEW_AREA_CELLS | Set this flag to prevent asynchronous cell initialisation for new areas only. This will prevent the side-effect described in the above note but will retain async behaviour for cell splits and merges. |
DISABLE_CELL_RESIZE_OR_ALLOCATE | Setting this flag prevents the forced merge of cells when no additional worker processes are available, ensuring that the area a worker is responsible for remains the same. If this flag is set alongside DISABLE_ASYNC_CELLS the simulation will exit when the worker pool is exhausted. |