BYOC public preview docs for Azure/AWS/GCP #23182
+552
−49
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| - Review the [shared responsibility model for BYOC]({% link cockroachcloud/byoc-overview.md %}#shared-responsibility-model-for-byoc). Make sure you understand and acknowledge the responsibilities you hold for management of your cloud infrastucture and the necessary permissions you must grant to Cockroach Labs. | ||
|
|
||
| - [Create a CockroachDB {{ site.data.products.cloud }} organization]({% link cockroachcloud/create-an-account.md %}) if you do not already have one. | ||
|
|
||
| - The BYOC deployment option is not available by default and must be requested. Reach out to your account team to express interest in BYOC. | ||
|
|
||
| - Once your cloud account is prepared for a CockroachDB BYOC deployment, cluster configuration and management is identical to a CockroachDB {{ site.data.products.cloud }} {{ site.data.products.advanced }} cluster. Review the [Plan a CockroachDB {{ site.data.products.advanced }} Cluster]({% link cockroachcloud/plan-your-cluster-advanced.md %}) documentation to plan your cluster sizing and resource allocation. | ||
|
|
||
| - Review cloud service regions supported by [CockroachDB {{ site.data.products.cloud }} {{ site.data.products.advanced }}]({% link cockroachcloud/regions.md %}?filters=advanced). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,274 @@ | ||||||
| --- | ||||||
| title: Prepare a CockroachDB Cloud BYOC Deployment in Amazon Web Services | ||||||
| summary: Prepare an Amazon Web Services account to host a BYOC deployment of CockroachDB | ||||||
| toc: true | ||||||
| keywords: deployment, byoc | ||||||
| --- | ||||||
|
|
||||||
| This page describes how to prepare a cloud service account to host a [BYOC deployment]({% link cockroachcloud/byoc-overview.md %}) of CockroachDB {{ site.data.products.cloud }} {{ site.data.products.advanced }} in Amazon Web Services (AWS). | ||||||
|
|
||||||
| {{site.data.alerts.callout_info}} | ||||||
| {% include feature-phases/preview.md %} | ||||||
| {{site.data.alerts.end}} | ||||||
|
|
||||||
| ## Prerequisites | ||||||
|
|
||||||
| {% include cockroachcloud/byoc/byoc-common-prerequisites.md %} | ||||||
|
|
||||||
| - Create an [API service account]({% link cockroachcloud/managing-access.md %}#create-api-keys) to use the [{{ site.data.products.cloud }} API]({% link cockroachcloud/cloud-api.md %}) with your {{ site.data.products.cloud }} organization. | ||||||
|
|
||||||
| ## Step 1. Create a new AWS account | ||||||
|
|
||||||
| Provision a new AWS account with no existing infrastructure, dedicated to your CockroachDB {{ site.data.products.cloud }} deployment. The account configuration for BYOC requires you to grant Cockroach Labs permissions to access and modify resources in this account, so this step is necessary to isolate these permissions from non-CockroachDB Cloud resources. This account can be reused for multiple CockroachDB clusters. | ||||||
|
|
||||||
| ## Step 2. Collect the Cockroach Labs IAM role ARN | ||||||
|
|
||||||
| Cockroach Labs uses cross-account resource management to provision and manage resources in your AWS account. This requires two **IAM roles**: | ||||||
|
|
||||||
| - An IAM role owned by Cockroach Labs which must be granted permissions to access an IAM role in your AWS account. | ||||||
| - An intermediary IAM role in your AWS account which must be granted permissions to create and manage infrastructure. This IAM role is the target used by Cockroach Labs for cross-account management. | ||||||
|
|
||||||
| In this step, use the [{{ site.data.products.cloud }} API]({% link cockroachcloud/cloud-api.md %}) to get the **Amazon Resource Name (ARN)** of the IAM role provisioned by Cockroach Labs for your account. | ||||||
|
|
||||||
| Send a `GET` request to the `/v1/organization` endpoint of the [CockroachDB {{ site.data.products.cloud }} API](https://www.cockroachlabs.com/docs/api/cloud/v1.html#get-/api/v1/organization) similar to the following example: | ||||||
|
|
||||||
| {% include_cached copy-clipboard.html %} | ||||||
| ~~~ shell | ||||||
| curl --request GET \ | ||||||
| --url https://cockroachlabs.cloud/api/v1/organization \ | ||||||
| --header 'Authorization: Bearer {secret_key}' | ||||||
| ~~~ | ||||||
|
|
||||||
| Record the value of `cockroach_cloud_service_principals.aws.user_arn` in the response: | ||||||
|
|
||||||
| ~~~ json | ||||||
| { | ||||||
| "cockroach_cloud_service_principals": { | ||||||
| "aws": { | ||||||
| "user_arn": "arn:aws:iam::{AWS Account ID}:example/arn" | ||||||
| } | ||||||
| } | ||||||
| } | ||||||
| ~~~ | ||||||
|
|
||||||
| ## Step 3. Create intermediary IAM role and apply permissions | ||||||
|
|
||||||
| In this step, create the intermediary IAM role in your AWS account, then apply a trust relationship policy and permissions that allow Cockroach Labs to assume the intermediary role as needed. | ||||||
|
|
||||||
| Follow these steps to create the intermediate IAM role: | ||||||
|
|
||||||
| 1. Open the AWS IAM console. | ||||||
| 1. Create a new role. You can choose any name for this role. In these instructions the example role is named `CRLBYOCAdmin`. | ||||||
| 1. Use the following trust relationship policy for the new role, using the ARN collected in the previous step: | ||||||
| {% include_cached copy-clipboard.html %} | ||||||
| ~~~ json | ||||||
| { | ||||||
| "Version": "2012-10-17", | ||||||
| "Statement": [ | ||||||
| { | ||||||
| "Effect": "Allow", | ||||||
| "Principal": { | ||||||
| "AWS": <intermediate IAM user's ARN> | ||||||
| }, | ||||||
| "Action": "sts:AssumeRole" | ||||||
| } | ||||||
| ] | ||||||
| } | ||||||
| ~~~ | ||||||
| 1. Apply an IAM policy to the intermediate role granting the following list of permissions: | ||||||
| {% include_cached copy-clipboard.html %} | ||||||
| ~~~ text | ||||||
| // Auto Scaling permissions | ||||||
| "autoscaling:CreateAutoScalingGroup", | ||||||
| "autoscaling:DeleteAutoScalingGroup", | ||||||
| "autoscaling:Describe*", | ||||||
| "autoscaling:Get*", | ||||||
| "autoscaling:SetInstanceProtection", | ||||||
| "autoscaling:TerminateInstanceInAutoScalingGroup", | ||||||
| "autoscaling:UpdateAutoScalingGroup", | ||||||
|
|
||||||
| // EC2 permissions | ||||||
| "ec2:AcceptVpcEndpointConnections", | ||||||
| "ec2:AcceptVpcPeeringConnection", | ||||||
| "ec2:AssociateRouteTable", | ||||||
| "ec2:AssociateVpcCidrBlock", | ||||||
| "ec2:AttachInternetGateway", | ||||||
| "ec2:AuthorizeSecurityGroupEgress", | ||||||
| "ec2:AuthorizeSecurityGroupIngress", | ||||||
| "ec2:CreateFlowLogs", | ||||||
| "ec2:CreateInternetGateway", | ||||||
| "ec2:CreateLaunchTemplate", | ||||||
| "ec2:CreateLaunchTemplateVersion", | ||||||
| "ec2:CreateNatGateway", | ||||||
| "ec2:CreateRoute", | ||||||
| "ec2:CreateRouteTable", | ||||||
| "ec2:CreateSecurityGroup", | ||||||
| "ec2:CreateSubnet", | ||||||
| "ec2:CreateTags", | ||||||
| "ec2:CreateVpc", | ||||||
| "ec2:CreateVpcEndpoint", | ||||||
| "ec2:CreateVpcPeeringConnection", | ||||||
| "ec2:DeleteFlowLogs", | ||||||
| "ec2:DeleteInternetGateway", | ||||||
| "ec2:DeleteLaunchTemplate", | ||||||
| "ec2:DeleteLaunchTemplateVersions", | ||||||
| "ec2:DeleteNatGateway", | ||||||
| "ec2:DeleteRoute", | ||||||
| "ec2:DeleteRouteTable", | ||||||
| "ec2:DeleteSecurityGroup", | ||||||
| "ec2:DeleteSubnet", | ||||||
| "ec2:DeleteVpc", | ||||||
| "ec2:DeleteVpcEndpoints", | ||||||
| "ec2:DeleteVpcEndpointServiceConfigurations", | ||||||
| "ec2:DeleteVpcPeeringConnection", | ||||||
| "ec2:Describe*", | ||||||
| "ec2:DetachInternetGateway", | ||||||
| "ec2:DisableEbsEncryptionByDefault", | ||||||
| "ec2:DisassociateRouteTable", | ||||||
| "ec2:DisassociateVpcCidrBlock", | ||||||
| "ec2:EnableEbsEncryptionByDefault", | ||||||
| "ec2:Get*", | ||||||
| "ec2:List*", | ||||||
| "ec2:ModifySubnetAttribute", | ||||||
| "ec2:ModifyVolume", | ||||||
| "ec2:ModifyVpcAttribute", | ||||||
| "ec2:ModifyVpcEndpointServiceConfiguration", | ||||||
| "ec2:ModifyVpcEndpointServicePermissions", | ||||||
| "ec2:RejectVpcEndpointConnections", | ||||||
| "ec2:RevokeSecurityGroupEgress", | ||||||
| "ec2:RevokeSecurityGroupIngress", | ||||||
| "ec2:RunInstances", | ||||||
| "ec2:StartVpcEndpointServicePrivateDnsVerification", | ||||||
|
|
||||||
| // EKS permissions | ||||||
| "eks:AssociateAccessPolicy", | ||||||
| "eks:CreateAccessEntry", | ||||||
| "eks:CreateCluster", | ||||||
| "eks:DeleteAccessEntry", | ||||||
| "eks:DeleteCluster", | ||||||
| "eks:Describe*", | ||||||
| "eks:DisassociateAccessPolicy", | ||||||
| "eks:List*", | ||||||
| "eks:UpdateAccessEntry", | ||||||
| "eks:UpdateClusterConfig", | ||||||
| "eks:UpdateClusterVersion", | ||||||
|
|
||||||
| // Elastic Load Balancing permissions | ||||||
| "elasticloadbalancing:Describe*", | ||||||
|
|
||||||
| // IAM permissions | ||||||
| "iam:AddRoleToInstanceProfile", | ||||||
| "iam:AttachRolePolicy", | ||||||
| "iam:AttachUserPolicy", | ||||||
| "iam:CreateAccessKey", | ||||||
| "iam:CreateAccountAlias", | ||||||
| "iam:CreateInstanceProfile", | ||||||
| "iam:CreateOpenIDConnectProvider", | ||||||
| "iam:CreatePolicy", | ||||||
| "iam:CreateRole", | ||||||
| "iam:CreateServiceLinkedRole", | ||||||
| "iam:CreateUser", | ||||||
| "iam:DeleteAccessKey", | ||||||
| "iam:DeleteInstanceProfile", | ||||||
| "iam:DeleteLoginProfile", | ||||||
| "iam:DeleteOpenIDConnectProvider", | ||||||
| "iam:DeletePolicy", | ||||||
| "iam:DeletePolicyVersion", | ||||||
| "iam:DeleteRole", | ||||||
| "iam:DeleteRolePolicy", | ||||||
| "iam:DeleteUser", | ||||||
| "iam:DeleteUserPolicy", | ||||||
| "iam:DetachRolePolicy", | ||||||
| "iam:DetachUserPolicy", | ||||||
| "iam:Get*", | ||||||
| "iam:List*", | ||||||
| "iam:PassRole", | ||||||
| "iam:PutRolePolicy", | ||||||
| "iam:PutUserPolicy", | ||||||
| "iam:RemoveRoleFromInstanceProfile", | ||||||
| "iam:TagPolicy", | ||||||
|
|
||||||
| // Kafka permissions | ||||||
| "kafka:List*", | ||||||
|
|
||||||
| // CloudWatch Logs permissions | ||||||
| "logs:CreateLogGroup", | ||||||
| "logs:DeleteLogGroup", | ||||||
| "logs:Describe*", | ||||||
| "logs:Get*", | ||||||
| "logs:List*", | ||||||
| "logs:PutRetentionPolicy", | ||||||
| "logs:PutSubscriptionFilter", | ||||||
|
|
||||||
| // S3 permissions | ||||||
| "s3:CreateBucket", | ||||||
| "s3:DeleteBucketPolicy", | ||||||
| "s3:Describe*", | ||||||
| "s3:Get*", | ||||||
| "s3:List*", | ||||||
| "s3:PutBucketTagging", | ||||||
| "s3:PutEncryptionConfiguration", | ||||||
| "s3:PutLifecycleConfiguration", | ||||||
|
|
||||||
| // Service Quotas permissions | ||||||
| "servicequotas:GetServiceQuota", | ||||||
| ~~~ | ||||||
|
|
||||||
| ## Step 4. (Optional) Enable additional regions | ||||||
|
|
||||||
| If you plan to use non-default AWS regions, you must manually enable them in the AWS Management Console. You must also activate [global STS tokens](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_region-endpoints.html) for these regions to work with CockroachDB. | ||||||
|
|
||||||
| You may also need to adjust quotas for vCPU and EBS disk storage for the regions in which you plan to create your cluster. | ||||||
|
|
||||||
| ## Step 5. Create the CockroachDB {{ site.data.products.cloud }} cluster | ||||||
|
|
||||||
| In BYOC deployments, CockroachDB clusters can be deployed in the {{ site.data.products.cloud }} Console or with the [{{ site.data.products.cloud }} API]({% link cockroachcloud/cloud-api.md %}). | ||||||
|
|
||||||
| ### Create a cluster with the {{ site.data.products.cloud }} Console | ||||||
|
|
||||||
| Follow these steps to create a CockroachDB cluster in the {{ site.data.products.cloud }} console: | ||||||
|
|
||||||
| 1. Open the {{ site.data.products.cloud }} Console and select the organization that has been enabled for BYOC. | ||||||
| 1. Click **Create cluster**. | ||||||
| 1. Under **Select a plan**, click **{{ site.data.products.advanced }}**. | ||||||
| 1. Under **Cloud & Regions**, click **Bring Your Own Cloud** and select AWS. | ||||||
| 1. Under **Cloud account**, click **Select your cloud account > Add new cloud account**. Enter the ARN associated with your intermediate IAM role. | ||||||
|
Contributor
There was a problem hiding this comment.
Suggested change
Idk what the best way of clarifying this is but I found your phrasing a bit confusing, as I thought it was the ARN associated with the initial, Cockroach Labs IAM. I guess it's both? Either way, my suggested phrasing is less ambiguous.
Contributor
Author
There was a problem hiding this comment. It's actually referring to the ARN of the "intermediate IAM role" that you, the user, created in step 3. Agents are very bad at determining which "step 2" on the page you are referring to in a context like this. Will rewrite slightly to make it more clear |
||||||
| 1. Follow the rest of the **Create Cluster** steps to configure your cluster's regions, capacity, and features as desired. Read the [Plan a CockroachDB {{ site.data.products.advanced}} Cluster]({% link cockroachcloud/plan-your-cluster-advanced.md %}) documentation for more details. | ||||||
|
|
||||||
| ### Create a cluster with the {{ site.data.products.cloud }} API | ||||||
|
|
||||||
| Send a `POST` request to the the `/v1/clusters` endpoint to [create a CockroachDB {{ site.data.products.cloud }} {{ site.data.products.advanced }} cluster]({% link cockroachcloud/cloud-api.md %}#create-an-advanced-cluster). | ||||||
|
|
||||||
| The following example request creates a 3-node {{ site.data.products.advanced }} cluster in the `us-east-2` region, specifying the ARN associated with your intermediate IAM role: | ||||||
|
|
||||||
| {% include_cached copy-clipboard.html %} | ||||||
| ~~~ shell | ||||||
| curl --request POST \ | ||||||
| --url https://cockroachlabs.cloud/api/v1/clusters \ | ||||||
| --header 'Authorization: Bearer {secret_key}' \ | ||||||
| --json '{ | ||||||
| "name": "byoc-aws-cluster-1", | ||||||
| "provider": "AWS", | ||||||
| "spec": { | ||||||
| "dedicated": { | ||||||
| "hardware": { | ||||||
| "machine_spec": {"num_virtual_cpus": 4}, | ||||||
| "storage_gib": 16 | ||||||
| }, | ||||||
| "region_nodes": {"us-east-2": 3} | ||||||
| }, | ||||||
| "plan": "ADVANCED", | ||||||
| "customer_cloud_account": { | ||||||
| "aws": { | ||||||
| "arn": "arn:aws:iam::<AWS Account ID>:user/byoc/CockroachDB-Cloud-managed-BYOC_<org-label>" | ||||||
|
Contributor
There was a problem hiding this comment. This is such a nit but curly brackets for the placeholder would match the pre-existing example w/ placeholders in the Azure file.
Contributor
There was a problem hiding this comment. (this is relevant in a few places) |
||||||
| } | ||||||
| } | ||||||
| } | ||||||
| }' | ||||||
| ~~~ | ||||||
|
|
||||||
| ## Next steps | ||||||
|
|
||||||
| - [Connect to your cluster]({% link cockroachcloud/connect-to-an-advanced-cluster.md %}) | ||||||
| - [Manage your cluster using the {{ site.data.products.cloud }} API]({% link cockroachcloud/cloud-api.md %}) | ||||||
| - [Prepare your deployment for production]({% link cockroachcloud/production-checklist.md %}) | ||||||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.