Comment on page
Getting your cloud ready for the Hadean Platform
First time administrative setup
The Hadean Platform requires that an administrator for your cloud provider go through an automated one time setup, so that clusters created by the platform have the necessary permissions for dynamic scaling.
The advantage of this approach is that the permissions of your clusters are managed, and separate from your user permissions. For example, if a user with admin permissions deploys a cluster, that cluster does not then inherit admin permissions as well. It only gets the specific managed set of permissions we create here.
If you have not done this setup, either provisioning may fail (resulting in clusters appearing in a "Failed" status in the cluster list) or if provisioning succeeds, attempting to run any application on the cluster will fail with an "Out of resources" error message.
If you plan to self-manage your cloud resources or need to run on physical machines, cloud providers other than Azure or AWS, or on-premises VMs, then you do not need to follow the steps in this section. See the Configuring pre-provisioned infrastructure section.
You can find a readme that will explain how to run the scripts we provide, and the scripts for your cloud providers first time setup in the
The readme's exact path is
~/.hadean/deployment/init/readme.mdand can be viewed either as raw text, or with a markdown renderer.
The location argument must be a valid Azure location name. You can find a list of these in our docs. This location specifies where the resource group for the resources we create will be deployed. You do not need to do this per location, and the location you choose will not impact the behaviour or the performance of the platform. Simply pick the location that you are likely to use the most, or whatever location is appropriate for your use of Azure.
The script automates the entire setup process. Once run, the following resources will exist:
The managed identity can then be assigned by users to the cluster scheduler. The cluster scheduler will login using
az login --identity, and use managed service identity to create and destroy dynamic resources.
If you need more restricted permissions, those must be created by your administrator as a role and assigned to the managed identity instead of the contributor role.
There are three steps to setting up AWS for use with the platform:
- 1.Deploy Policies
- 2.Assign User Policies
- 3.Hosted Zone Registration
aws cloudformation deploy --template-file ~/.hadean/deployment/init/aws/permissions.cloudformation.yaml --stack-name hadeanpermissions
The following resources are created:
- 1.A role that contains a policy and assume policy document called
HadeanPlatformClusterRolewhich clusters will use for their permissions. The policy in this role is tightly constrained to the specific permissions that the platform uses.
Once you have deployed the CloudFormation template, you must assign the
HadeanPlatformClusterUserPolicypolicy to the groups or users that are going to use the platform. This will give those users the ability to use the instance profile to setup the credentials for the scheduler, so that it can dynamically scale.
The last step for AWS setup is to ensure you have a hosted zone in Route53 that your users can list and create Route53 records for. More information on this requirement is available here. This step is not optional.
You can restrict these permissions to the specific hosted zone through the
Resourcepart of the policy, by setting it to the ARN of the hosted zone you created.
Once you are done, users can specify the hosted zone they want to create a cluster within by using the
--domain-nameargument to the
hadean cluster createcommand, specifying the name used by the hosted zone. For example, if you've registered
hadean.com., then your users must use
--domain-name hadean.comwhen creating clusters.