Deploy Amazon OpenSearch Serverless with Terraform

Amazon OpenSearch Serverless supplies the search and analytical performance of OpenSearch with out the guide overhead of configuring, managing, and scaling OpenSearch clusters. It routinely scales the sources based mostly in your workload, and also you solely pay for the sources consumed. Managing OpenSearch Serverless is straightforward, however with infrastructure as code (IaC) software program like Terraform, you’ll be able to simplify your useful resource administration much more.

This put up demonstrates the right way to use Terraform to create, deploy, and clear up OpenSearch Serverless infrastructure.

Resolution overview

To create and deploy an OpenSearch Serverless assortment with safety and entry insurance policies utilizing Terraform, it’s essential comply with these steps:

1. Initialize the Terraform configuration.
2. Create an encryption coverage.
3. Create an OpenSearch Serverless assortment.
4. Create a community coverage.
5. Create a digital non-public cloud (VPC) endpoint.
6. Create a knowledge entry coverage.
7. Deploy utilizing Terraform.

Conditions

This put up assumes that you simply’re aware of GitHub and Git instructions.

For this walkthrough, you want the next:

Initialize the Terraform configuration

The pattern code is out there within the Terraform GitHub repo within the terraform-provider-aws/examples/opensearchserverless listing. This configuration will get you began with OpenSearch Serverless. First, clone the repository to your workstation and navigate to the listing:

$ git clone https://github.com/hashicorp/terraform-provider-aws.git &&  
cd ./terraform-provider-aws/examples/opensearchserverless

Initialize the configuration to put in the aws supplier by working the next command:

$ terraform init

The Terraform configuration first defines the model of Terraform required and configures the AWS supplier to launch sources within the Area outlined by the aws_region variable:

# Require Terraform 0.12 or higher
terraform {
  required_version = ">= 0.12"
}

# Set AWS supplier area
supplier "aws" {
  area = var.aws_region
}

The variables used on this Terraform configuration are outlined within the variables.tf file. This put up assumes the default values are used:

variable "aws_region" {
  description = "The AWS area to create issues in."
  default     = "us-east-1"
}

variable "collection_name" {
  description = "Identify of the OpenSearch Serverless assortment."
  default     = "example-collection"
}

Create an encryption coverage

Now that the supplier is put in and configured, the Terraform configuration strikes on to defining OpenSearch Serverless insurance policies for safety. OpenSearch Serverless makes use of AWS Key Administration Service (AWS KMS) to encrypt your information. The encryption is managed by an encryption coverage. To create an encryption coverage, use the aws_opensearchserverless_security_policy useful resource, which has a identify parameter, a kind of encryption, a JSON string that defines the coverage, and an non-compulsory description:

# Creates an encryption safety coverage
useful resource "aws_opensearchserverless_security_policy" "encryption_policy" {
  identify        = "example-encryption-policy"
  kind        = "encryption"
  description = "encryption coverage for ${var.collection_name}"
  coverage = jsonencode({
    Guidelines = [
      {
        Resource = [
          "collection/${var.collection_name}"
        ],
        ResourceType = "assortment"
      }
    ],
    AWSOwnedKey = true
  })
}

This encryption coverage is called example-encryption-policy, applies to a set named example-collection, and makes use of an AWS owned key to encrypt the info.

Create an OpenSearch Serverless assortment

You’ll be able to manage your OpenSearch indexes right into a logical grouping referred to as a assortment. Create a set utilizing the aws_opensearchserverless_collection useful resource, which has a identify parameter, and optionally, description, tags, and kind:

# Creates a set
useful resource "aws_opensearchserverless_collection" "assortment" {
  identify = var.collection_name

  depends_on = [aws_opensearchserverless_security_policy.encryption_policy]
}

This assortment is called example-collection. If kind just isn’t specified, a time collection assortment is created. Supported assortment sorts could be discovered within the Terraform documentation for the aws_opensearchserverless_collection useful resource. OpenSearch Serverless requires encryption at relaxation, so an relevant encryption coverage is required earlier than a set could be created. The Terraform configuration explicitly defines this dependency utilizing the depends_on meta-argument. Errors can come up if this dependency just isn’t outlined.

Now {that a} assortment has been created with an AWS owned KMS key, the Terraform configuration goes on to outline the community and information entry coverage to configure entry to the gathering.

Create a community coverage

A community coverage permits entry to your assortment both over the general public web or by means of OpenSearch Serverless-managed VPC endpoints. Just like the encryption coverage, to create a community coverage, use the aws_opensearchserverless_security_policy useful resource, which has a identify parameter, a kind of community, a JSON string that defines the coverage, and an non-compulsory description:

# Creates a community safety coverage
useful resource "aws_opensearchserverless_security_policy" "network_policy" {
  identify        = "example-network-policy"
  kind        = "community"
  description = "public entry for dashboard, VPC entry for assortment endpoint"
  coverage = jsonencode([
    {
      Description = "VPC access for collection endpoint",
      Rules = [
        {
          ResourceType = "collection",
          Resource = [
            "collection/${var.collection_name}"
          ]
        }
      ],
      AllowFromPublic = false,
      SourceVPCEs = [
        aws_opensearchserverless_vpc_endpoint.vpc_endpoint.id
      ]
    },
    {
      Description = "Public entry for dashboards",
      Guidelines = [
        {
          ResourceType = "dashboard"
          Resource = [
            "collection/${var.collection_name}"
          ]
        }
      ],
      AllowFromPublic = true
    }
  ])
}

This community coverage is called example-network-policy and applies to the gathering named example-collection. This coverage solely permits entry to the gathering’s OpenSearch endpoint by means of a VPC endpoint, however permits public entry to the OpenSearch Dashboards endpoint.

You’ll discover the VPC endpoint has not been outlined but, however it’s referenced within the community coverage. Terraform determines this dependency routinely and won’t create the community coverage till the VPC endpoint has been created.

Create a VPC endpoint

A VPC endpoint lets you privately entry your OpenSearch Serverless assortment utilizing AWS PrivateLink (for extra data, discuss with Entry AWS companies by means of AWS PrivateLink). Create a VPC endpoint utilizing the aws_opensearchserverless_vpc_endpoint useful resource, the place you outline identify, vpc_id, subnet_ids , and optionally, security_group_ids:

# Creates a VPC endpoint
useful resource "aws_opensearchserverless_vpc_endpoint" "vpc_endpoint" {
  identify               = "example-vpc-endpoint"
  vpc_id             = aws_vpc.vpc.id
  subnet_ids         = [aws_subnet.subnet.id]
  security_group_ids = [aws_security_group.security_group.id]
}

Making a VPC and all of the required networking sources is out of scope for this put up, however the minimal required VPC sources are created right here in a separate file to reveal the VPC endpoint performance. Seek advice from Getting Began with Amazon VPC to study extra.

Create a knowledge entry coverage

The configuration defines a information supply that appears up details about the context Terraform is presently working in. This information supply is used when defining the info entry coverage. Extra data could be discovered within the Terraform documentation for aws_caller_identity.

# Will get entry to the efficient Account ID by which Terraform is allowed
information "aws_caller_identity" "present" {}

A information entry coverage permits you to outline who has entry to collections and indexes. The information entry coverage is outlined utilizing the aws_opensearchserverless_access_policy useful resource, which has a identify parameter, a kind parameter set to information, a JSON string that defines the coverage, and an non-compulsory description:

# Creates a knowledge entry coverage
useful resource "aws_opensearchserverless_access_policy" "data_access_policy" {
  identify        = "example-data-access-policy"
  kind        = "information"
  description = "permit index and assortment entry"
  coverage = jsonencode([
    {
      Rules = [
        {
          ResourceType = "index",
          Resource = [
            "index/${var.collection_name}/*"
          ],
          Permission = [
            "aoss:*"
          ]
        },
        {
          ResourceType = "assortment",
          Useful resource = [
            "collection/${var.collection_name}"
          ],
          Permission = [
            "aoss:*"
          ]
        }
      ],
      Principal = [
        data.aws_caller_identity.current.arn
      ]
    }
  ])
}

This information entry coverage permits the present AWS function or person to carry out collection-related actions on the gathering named example-collection and index-related actions on the indexes within the assortment.

Deploy utilizing Terraform

Now that you’ve got configured the required sources, apply the configuration utilizing terraform apply. Earlier than creating the sources, Terraform will describe all of the sources that will likely be created so you’ll be able to confirm your configuration:

$ terraform apply

...

Terraform used the chosen suppliers to generate the next execution plan. Useful resource actions are indicated with the next symbols:
  + create

Terraform will carry out the next actions:

...

Plan: 13 so as to add, 0 to alter, 0 to destroy.

Modifications to Outputs:
  + collection_enpdoint = (recognized after apply)
  + dashboard_endpoint  = (recognized after apply)

Do you wish to carry out these actions?
  Terraform will carry out the actions described above.
  Solely 'sure' will likely be accepted to approve.

  Enter a price:

If that is the primary OpenSearch Serverless assortment in your account, making use of the configuration might take over 10 minutes as a result of Terraform waits for the gathering to develop into energetic.

Apply full! Assets: 13 added, 0 modified, 0 destroyed.

Outputs:

collection_enpdoint = "..."
dashboard_endpoint = "..."

You will have now deployed an OpenSearch Serverless time collection assortment with insurance policies to configure encryption and entry to the gathering!

Clear up

The sources will incur prices so long as they’re working, so clear up the sources if you end up carried out utilizing them. Use the terraform destroy command to do that:

$ terraform destroy

...

Terraform used the chosen suppliers to generate the next execution plan. Useful resource actions are indicated with the next symbols:
  - destroy

Terraform will carry out the next actions:

  ...

Plan: 0 so as to add, 0 to alter, 13 to destroy.

Modifications to Outputs:

...

Do you actually wish to destroy all sources?
  Terraform will destroy all of your managed infrastructure, as proven above.
  There is no such thing as a undo. Solely 'sure' will likely be accepted to verify.

  Enter a price:

Reply sure to run this plan and destroy the infrastructure.

Destroy full! Assets: 13 destroyed.

All sources created throughout this walkthrough have now been deleted.

Conclusion

On this put up, you created an OpenSearch Serverless assortment. Utilizing IaC software program like Terraform could make it easy to handle your sources like OpenSearch Serverless collections, encryption, community, and information entry insurance policies, and VPC endpoints.

Thanks to all of the open-source contributors who assist keep OpenSearch and Terraform.

Strive utilizing OpenSearch Serverless with Terraform to simplify your useful resource administration. Try the Getting began with Amazon OpenSearch Serverless workshop and the Amazon OpenSearch Serverless Developer Information to study extra about OpenSearch Serverless.

Concerning the authors

Joshua Luo is a Software program Growth Engineer for Amazon OpenSearch Serverless. He works on the programs that allow clients to handle and monitor their OpenSearch Serverless sources. He enjoys bouldering, pictures, and videography in his free time.

Satish Nandi is a Senior Technical Product Supervisor for Amazon OpenSearch Service.