Metrics
Prerequisites
In order to gather metrics, make sure you have a simulation that is currently running. If that is not the case, please refer to the section on starting your simulation. You will also need your cluster key for accessing the metrics of remote simulations.
While your simulation is running on a cluster if you have set metrics_enabled to true in your configuration, our infrastructure will collect metrics from all the running servers populating a Prometheus database. The metrics_enable configuration can be found in the
runtime_config.toml file in the example configuration. Please see the platform documentation for more details.You can either connect directly to this database which will be running on your gateway server on port 9090, or alternatively the Hadean CLI includes a function to allow you to retrieve a text file including the Prometheus format metrics using the following command.
$ hadean cluter metric --name [cluster name]
# HELP metrics_server_dev_requests_served Number of requests served by this metrics server
# TYPE metrics_server_dev_requests_served counter
metrics_server_dev_requests_served 2
# HELP metrics_server_dev_uptime_ms Uptime of the metrics server
# TYPE metrics_server_dev_uptime_ms gauge
metrics_server_dev_uptime_ms 36360
# TYPE total_collisions counter
total_collisions{worker_id="5",hadean_pid="127.0.0.1.18021.0"} 11.000000
total_collisions{worker_id="6",hadean_pid="127.0.0.1.18027.0"} 20.000000
total_collisions{worker_id="7",hadean_pid="127.0.0.1.18015.0"} 11.000000
total_collisions{worker_id="4",hadean_pid="127.0.0.1.18019.0"} 15.000000
# TYPE dispatcher_manager_messages counter
dispatcher_manager_messages{hadean_pid="127.0.0.1.18011.0"} 70.000000
# HELP muxer_data_recv_from_workers_kib The ingress bandwidth received by a muxer from workers (in KiBs).
# TYPE muxer_data_recv_from_workers_kib counter
muxer_data_recv_from_workers_kib{hadean_pid="127.0.0.1.18007.0"} 49895.578125
muxer_data_recv_from_workers_kib{hadean_pid="127.0.0.1.18005.0"} 49558.5
# TYPE dispatcher_muxer_messages counter
dispatcher_muxer_messages{hadean_pid="127.0.0.1.18011.0"} 2.000000
# HELP aether_worker_received_messages_total The number of messages received by this worker.
# TYPE aether_worker_received_messages_total counter
aether_worker_received_messages_total{workerid="140720588402528",hadean_pid="127.0.0.1.18021.0"} 0.000000
aether_worker_received_messages_total{workerid="140736352035504",hadean_pid="127.0.0.1.18027.0"} 0.000000
aether_worker_received_messages_total{workerid="140736223445632",hadean_pid="127.0.0.1.18015.0"} 0.000000
aether_worker_received_messages_total{workerid="140730104848272",hadean_pid="127.0.0.1.18019.0"} 0.000000
# TYPE aether_time_elapsed_sec gauge
Last modified 9mo ago