CircleCI

The Sym API and CircleCI Orb combine for an easy, auditable approval flow for deployments.

Suggest Edits

📘

The Sym orb is available in the CircleCI registry as sym/sym.

There are several ways to configure your CircleCI pipeline with Sym, but in this tutorial we will configure a workflow with the following steps:

  1. sym/request API call to ask for deploy approval
  2. A CircleCI hold job that pauses the workflow
  3. An on_approve hook in the Flow's impl.py to trigger the CircleCI workflow to continue
  4. A sym/validate and a deploy job that wait on the pause workflow job

🚧

This tutorial assumes prior knowledge of:

Please visit the linked pages if you are not familiar with these concepts yet!

Create a bot token

In order to use the Sym API Orb on CircleCI you will need to create a SYM_JWT and set it as an environment variable available for use in your pipeline.

  1. Create a Bot user with symflow bots create [bot-name]
  2. Issue a SYM_JWT with symflow tokens issue -u [bot-name] -e [expiry-length]
  3. Save the JWT outputted by this previous step into an environment variable available to your workflow. You can do this by adding the SYM_JWT environment variable to a CircleCI Context OR the Project Settings.

For more detailed documentation about Sym Bots and Tokens, visit Bot Users and Tokens.

Configure Terraform

Declare a custom integration for CircleCI

This tutorial's impl.py will be making API calls to CircleCI, which requires a personal CircleCI API Token. For instructions on how to create a CircleCI API Token, visit CircleCI's documentation: Managing API Tokens.

📘

Consider using a dedicated bot user for the personal API token that you integrate with Sym. The approving user in CircleCi will be the user whose personal API token you set up for the integration.

After generating a CircleCI API Token, add a sym_secret and sym_integration resource to your Terraform. This will make your CircleCI Token available in impl.py. See Manage Secrets with AWS Secrets Manager and Set Up an Integration that Utilizes Secrets for more detailed steps about how to configure AWS Secrets Manager and set the secret value.

# Follow the tutorial in "Manage Secrets with AWS Secrets Manager"
# for instructions on how to configure AWS Secrets Manager for Sym secrets
# and how to set the secret value.
resource "sym_secret" "circleci_api_key" {
    path      = aws_secretsmanager_secret.circleci_api_key.name 
    source_id = sym_secrets.this.id
}

# This custom integration exposes the CircleCI token as `secrets[0]`
resource "sym_integration" "circleci" {
    type = "custom"
    name = "circleci"
  
    # A unique identifier for this integration
    external_id = "symopsio"

    settings = {
        secret_ids_json = jsonencode([sym_secret.circleci_api_key.id])
    }
}

Because circleci is not a built-in Sym integration type, we will utilize the custom type. sym_integrations with type = custom have a setting secret_ids_json which accepts a JSON-encoded list of sym_secret IDs. These will be available later in impl.py as event.flow.environment.integrations["circleci"].secrets. In this example, there is only one secret we need to provide for this integration.

Add the CircleCI Integration to your sym_environment

Add the sym_integration.circleci to your sym_environment integrations list so that it's available to the Sym Runtime.

resource "sym_environment" "this" {
    # ... truncated for brevity
    integrations = {
        # This integration will now be available as `event.flow.environment.integrations["circleci"]`
        circleci_id = sym_integration.circleci.id

        # ... other integrations
  }
}

Terraform a Sym Flow to be called by CircleCI

Now, we need to add a new Flow that can only be called by the Sym API. Set the allowed_sources to api to ensure this flow does not show up in the Sym Flow Selector.

This flow will require three inputs, workflow_id, workflow_url and merging_user. We will also specify that it can be called only via API so it doesn't appear in the user-facing Flow Selection modal in Slack, outside the context of a specific CircleCI run.

resource "sym_flow" "this" {
  # This flow is an approval-only flow, so there is no strategy!

  name  = "ci-approval"
  label = "CI Approval"

  implementation = "${path.module}/impl.py"
  environment_id = sym_environment.this.id
    
  params {
    # allowed_sources defines the sources from which this flow can be invoked.
    # For this tutorial, we are setting it to API only.
    allowed_sources = ["api"]

    # Each prompt_field defines an input the Flow needs to run.
    # For this tutorial, we want the Circle CI Workflow and User.
    prompt_field {
      name     = "workflow_url"
      label    = "CI Workflow URL"
      type     = "string"
      required = true
    }
  
    prompt_field {
      name     = "workflow_id"
      label    = "CI Workflow ID"
      type     = "string"
      required = true
    }
    
    prompt_field {
      name     = "merging_user"
      label    = "User who merged PR"
      type     = "string"
      required = true
    }
  }
}

Configure impl.py

Create the necessary CircleCI helper methods

This impl.py contains several helper methods that we'll use to manage the CircleCI piece of our workflow.

  • circleci_authentication_header: Grabs the CircleCI Token configured in the circleci sym_integration and generates a HTTP Header Circle-Token to authenticate
  • fetch_circle_ci_jobs: Calls CircleCI's API to get the workflow's jobs
  • approve_circle_ci_hold: Calls CircleCi's API to approve the pause workflow job and continue the workflow

Add the after_approve hook

Finally, we'll need an after_approve hook that uses the workflow_id prompt field to trigger the wait_for_sym_approval job in CircleCI. Please note that the job name needs to match what is configured in your CircleCI config, which we'll set in the next section.

import requests
from sym.sdk.annotations import hook


def circleci_authentication_header(event):
    """Grabs the Circle CI API Token from the environment integrations
        block (defined as circleci_id in the sym_environment Terraform resource)
     """
    integration = event.flow.environment.integrations["circleci"]
    token = integration.settings["secrets"][0].retrieve_value()

    return {"Circle-Token": token}


def fetch_circle_ci_jobs(session, workflow_id):
    """Get all jobs in the workflow"""
    response = session.get(f"https://circleci.com/api/v2/workflow/{workflow_id}/job")
    return response.json()


def approve_circle_ci_hold(session, workflow_id, approval_request_id):
    """Post request to approve CircleCI hold job"""
    response = session.post(
        f"https://circleci.com/api/v2/workflow/{workflow_id}/approve/{approval_request_id}"
    )
    return response.json()


@hook
def after_approve(event):
    # The workflow_id is defined in our sym_flow's prompt_fields_json
    workflow_id = event.payload.fields.get("workflow_id")

    with requests.Session() as session:
        # Set the Circle-Token header to authenticate our API requests
        session.headers.update(circleci_authentication_header(event))

        # The `wait_for_sym_approval` job will be defined in our CircleCI config
        job_list = fetch_circle_ci_jobs(session, workflow_id)
        circle_approval_step = [
            job["id"] for job in job_list["items"] if job["name"] == "wait_for_sym_approval"
        ]
        circle_approval_step_id = circle_approval_step[0]

        # Approve the pause workflow job, to trigger the workflow to continue
        approve_circle_ci_hold(session, workflow_id, circle_approval_step_id)

Add Sym to your CircleCI Pipeline

We'll use two convenience jobs from the Sym Orb to set up our workflow: sym/request and sym/validate.

Create a CircleCI Workflow with the following steps:

  1. A sym/request job
    • This will programmatically make a Sym Request
  2. A job with type: approval
    • This will pause your workflow until your Sym Request has been approved
    • This job should depend on the sym/request job
  3. A sym/validate job
    • This optional job makes sure that the approval really was triggered by Sym
    • This job should depend on the approval job
  4. Your deploy job
    • This job should depend on the sym/validate job
orbs:
  sym: sym/[email protected]
workflows:
  main:
    jobs:
      # This will start the Sym flow
      - sym/request:
          flow_srn: sym:flow:ci-approval-prod:latest
          # These environment variables are built into Circle CI!
          flow_inputs: '{
                      "workflow_url": "${CIRCLE_BUILD_URL}",
                      "merging_user": "${CIRCLE_USERNAME}",
                      "workflow_id": "${CIRCLE_WORKFLOW_ID}"
                  }'
          request_slug: "prod_deploy"
          
      # Once approved, Sym will resume the CircleCI flow from this step.
      - wait_for_sym_approval:
          type: approval
          requires:
            - sym/request

      # This is the step gated by Sym approval
      - sym/validate:
          request_slug: "prod_deploy"
          requires:
            - wait_for_sym_approval

      # This is the step gated by Sym approval
      - prod_deploy:
          requires:
            - sym/validate

Using flow_context

You can send additional context data to your Sym Flow by providing flow_context to the sym/request job.

Saving flow_context data to the sym/context path

If you want to supply context that is generated by a previous job step, you can do this by adding context files to your CircleCI Workspace. Any files you add to the sym/context path in your workspace will be automatically aggregated into a context.json file that is sent to your Sym Flow implementation.

Here we generate a list of changed files and save this to sym/context/diff.txt:

jobs:
  # Generate a list of changed files and save to the sym/context path
  git-diff:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run: mkdir -p sym/context
      - run: git diff --name-only HEAD^..${CIRCLE_SHA1} > sym/context/diff.txt
      - persist_to_workspace:
          root: .
          paths:
            - sym/context/diff.txt

Accessing context in your Flow implementation

Use the SDK's get_context to access your context data:

@hook
def on_request(event):
    """
    If the request include a diff, then check if this changes includes any Terraform files.
    if there are no Terraform files, then auto-approve the request.
    """
    context = event.get_context("request")
    diff = context.get("diff.txt", "")
    if no_terraform_files(diff):
        return ApprovalTemplate.approve(reason="No terraform changes, auto approved!")

Full Example

With these pieces, your CircleCI pipeline can now make a Sym Request, pause the workflow, and automatically continue after the request has been approved!

You can find the complete code for this example in our Approving a CircleCI Job with Sym Example.

Updated 11 days ago