Deploying an application to the cloud

Prerequisites
Make sure that you have logged into both the az CLI and hadean CLI by following the Prerequisites and Installing the SDK guides.
Creating and using a Hadean Platform cluster
There are four steps to getting an application running on a Hadean cluster:
1.​Create a cluster​
2.​Wait for your cluster to provision​
3.​Deploy an application​
4.​Run the application remotely​
1. Create a cluster
To create a cluster run
 hadean cluster -n <name> --cloud <provider> create 
passing in the desired name of your new cluster the as well as the cloud provider which can be set to either azure or aws
You can set defaults for your cloud provider and location for new clusters.
See Configuring your application.
Azure
Azure clusters must simply specify a location to deploy to.
1
hadean cluster --name demo create --cloud azure --location <location>
Copied!
For example, you can create a cluster in London using hadean cluster --name demo create -l uksouth. For other locations, read about configuring Azure location.
AWS
AWS clusters must specify a region, availability zone, and a domain name to use for the deployed cluster.
The region, availability zone, and domain name parameters all have aliases.
Checkout hadean cluster create --help to see these options.
1
hadean cluster --name demo create --cloud aws --region <region> --zone <zone> --domain <domain>
Copied!
For example, you can create a cluster in London zone a using hadean cluster --name demo create -l eu-west-2 --zone eu-west-2a --domain <domain>.
For other regions, read about configuring AWS region.
Most regions have three availability zones; a, b, and c. But you can also set a default availability zone just like you can for region.
On AWS, using your own domain name is currently a requirement. You'll need to have this setup in Route53. Learn more about this requirement.
Arguments
Flag
Effect
--name | -n
Specifies the name of the cluster to create
--cloud-provider | --cloud | --provider
Picks the cloud provider (azure / aws) to deploy the cluster to
--location | --region | -l
Sets the location to provision the cluster in
--availability-zone | --zone
Sets the availability zone to use for the cluster (Only currently available in AWS)
--domain-name | --domain
Names the exact registered domain to use for the cluster (Only currently available in AWS)
2. Wait for your cluster to provision
Once created, licensed customers can see your cluster in the Hadean Portal, which will also show its status.
The hadean cluster create command will provision a new cluster in the location specified. You can check the status of your cluster(s) with the hadean cluster list command.
You'll need to wait until your cluster is in the status Ready and this typically takes less than 5 minutes. You can watch the list command to avoid typing it in repeatedly.
1
watch -n 5 hadean cluster list
Copied!
3. Deploy an application
Once logged in you can deploy an application to the cluster - in this case we'll use the hello demo - by using the hadean cluster deploy command; pass the name of your cluster and the path to the application as arguments.
A cluster can only have one deployed application at a time and any time you modify an application you will need to deploy it again to see those changes in the remote cluster.
1
hadean cluster -n demo deploy ~/hadean/examples/rust/hello/target/release/hello
Copied!
If the binary isn't found here, make sure you've built the demo in release mode already.
For applications that need other files to be deployed, such as data files or libraries, you can use the --directory argument to specify a directory containing other files to deploy along side your application.
For example, if you have an application robot that depends on the libraries ./lib/arms.so and ./lib/legs.so, then you could upload the files in your lib directory:
1
hadean cluster -n demo deploy ./target/robot --directory ./lib
Copied!
The files inside of ./lib will then end up in the working directory of your application.
4. Run the application remotely
With the application deployed you can now run it on the cluster. 
To start the application use the hadean cluster run command; pass the name of your cluster and the path to the config file, see Configuring your application for details.
You can use the --debug argument to receive debug logs back from your application!
1
hadean cluster -n demo run ~/hadean/examples/configs/config.toml
Copied!
This command will produce output both from the platform and your application, here's a walk through of what you'll see when you run the application:
1
Uploading config...
2
Starting application...
3
Streaming application logs...
Copied!
Don't want timestamps and colours? You can use --simple to remove these.
The configuration is uploaded to the cluster and the application is run. At this point, unless you passed the --detached flag, the run command will automatically start streaming application logs back from the cluster.
1
Metrics will be stored in: /tmp/hadeanos-1001/hadeanos-metrics, which isn't a tmpfs. Exporting metrics might cause performance problems.
2
dynamic backend - creating...                                                                                                           
Copied!
At this point the dynamic backend needs to provision a machine for the application. To do this, it must first create a resource group and storage account to store terraform state within. This is typically quite fast, but it can sometimes fail. If this does fail, see our Hints, Tips, and Troubleshooting guide for more information.
1
dynamic backend - creation successful   
2
dynamic backend - getting manager       
3
unsupported attribute in locus: platform
Copied!
Next, the dynamic backend must scale up machines to run your application. In Azure, machine scale up typically happens in less than 5 minutes. Note that if you run repeatedly and you have your standby_machines set to greater than zero, you won't see this start up cost on every run. See Configuring your application for more information on how your configuration impacts scaling.
1
PLAY [Provision machine.] ******************************************************
2
​
3
TASK [Wait for SSH connection] *************************************************
4
ok: [51.140.4.108]
5
​
6
(...)
7
​
8
PLAY RECAP *********************************************************************
9
51.140.4.108               : ok=10   changed=9    unreachable=0    failed=0    skipped=3    rescued=0    ignored=0
Copied!
Once a machine is available, ansible is used to provision the machine with the manager and user binary as described in Creating Hadean Platform applications.
1
dynamic backend - getting manager was successful
2
Reserving resources for 20.117.71.210.20000.0 (before: cores=2,memory=4028332 KiB; after: cores=31/16,memory=4020140 KiB; delta: cores=1/16,memory=8 MiB)
3
Switching logging to use the global logger on Manager Manager { pid: 20.117.71.210.20000.0, free_resources: Resources { cores: Ratio { numer: 31, denom: 16 }, memory: Bytes { bytes: 4116623360 } }, reserved_resources: Resources { cores: Ratio { numer: 1, denom: 16 }, memory: Bytes { bytes: 8388608 } }, total_resources: Resources { cores: Ratio { numer: 2, denom: 1 }, memory: Bytes { bytes: 4125011968 } }, sender: … }
Copied!
The dynamic backend then connects to the manager and allocates resources for the application.
1
entry
2
parent start
3
Reserving resources for 20.117.71.210.20000.0 (before: cores=31/16,memory=4020140 KiB; after: cores=15/8,memory=4011948 KiB; delta: cores=1/16,memory=8 MiB)
4
entry
5
child start
6
child received: parent
7
parent received: child
8
parent end
9
child end
10
Releasing resources for 20.117.71.210.20000.0 (before: cores=15/8,memory=4011948 KiB; after: cores=31/16,memory=4020140 KiB; delta: cores=1/16,memory=8 MiB)
Copied!
The Hadean global logger preserves module names in the original JSON format of the logs. If you want to filter out the hadean logging and just focus on your application's logs, you can use --simple and --no-parse-logs to get out the raw log format. Then, you can use a tool like jq to process the JSON output and search for your module name.
At this point you'll see the logs from your application. In this case, we've run the hello example. So we can see the output from that program coming from the cluster.
1
Termination request received from scheduler                                              
2
Killing all children                                                                     
3
Wait for children to finish                                                              
4
dynamic backend - started destruction of machine (ip: 20.117.71.210)                     
5
dynamic backend - machine destruction was successful (identifier: 0, ip: 20.117.71.210)  
6
all managers have been terminated                                                        
Copied!
After the application exits, if your application used more resources than is specified in your standby_machines in your application config, the scheduler will down-scale your resources that are not in use. After this down scaling, exactly as many VMs as is specified in your config will remain.
After doing a run the time configured in machines_timeout will be used to destroy all standby machines if no new application is run within that time. This allows you to iterate rapidly without leaving resources around.
Clean up
Automatic clean up
The scheduler will automatically scale down resources when applications are not running (to the minimum specified in standby-machines in your config), so you only need to clean up resources manually when you are finished with the scheduler itself. You can configure how long the machine timeout is before they are cleaned up just as you can configure how many machines are held in standby.
Removing dynamic machines
If you have changed your configuration, you may find standby machines of a different configuration continue to be used. In this situation, you can use the clean-up command to fully remove all dynamic resources and start fresh.
1
hadean cluster --name <name> clean-up
Copied!
You'll have to stop any running application with the stop command first, if something is running. You can always check using the status command if you aren't sure. This command does not remove the scheduler itself, just all the resources that the scheduler has created.
Destroying a cluster
If you want to remove the whole cluster; dynamic resources and scheduler, use the destroy command:
1
hadean cluster --name <name> destroy
Copied!
This will:
1.Ask you to confirm that you do in fact want to destroy the named cluster.
2.Stop any running applications (this can take up to 5 minutes if the application is currently scaling up or down).
3.Clean up all dynamic resources.
4.Remove the scheduler resources.
5.Remove the cluster from the cluster list.
This will remove all virtual machines, elastic IPs, storage, and everything else associated with a cluster. There is no way to restore a cluster.
Currently, the following resources will remain in your cloud provider after destroy has been run:
1.The dynamic resources resource group. This will be named hadean-dynamic-{name}-{key}-resource-group in all providers. Where name is the first 6 characters of the cluster name, and key is the first 6 characters of the cluster key. You can see this key by using hadean cluster list before destroying.
2.The Azure Storage Account or AWS S3 bucket that contains the dynamic resources state.
You can find this using the resource group above.
3.The Azure Storage Account or AWS S3 bucket that contains the cluster resources state.
1.In AWS, the S3 bucket will be called hadean-tf-{region}-{name}-{key} with the same naming rules as the resource group in (1).
2.In Azure, the resource group containing the storage account will be called hadean-tf. The storage account name will be a hash. The storage container will have the name hadean-tf-{region}-{name}-{key}.
These remaining resources have minimal associated costs but should be cleaned up manually or via a scripted clean up process. This will help to avoid hitting quotas on your resources such as maximum allocated S3 buckets.
Getting started - Previous
Getting your cloud ready for the Hadean Platform
Next - Getting started
Configuring your application
Last modified 3mo ago
Copy link
Contents
Prerequisites
Creating and using a Hadean Platform cluster
1. Create a cluster
2. Wait for your cluster to provision
3. Deploy an application
4. Run the application remotely
Clean up