CircleCI
The Sym API and CircleCI Orb combine for an easy, auditable approval flow for deployments.
The Sym orb is available in the CircleCI registry as
sym/sym
.
There are several ways to configure your CircleCI pipeline with Sym, but in this tutorial we will configure a workflow with the following steps:
sym/request
API call to ask for deploy approval- A CircleCI hold job that pauses the workflow
- An
on_approve
hook in the Flow'simpl.py
to trigger the CircleCI workflow to continue - A
sym/validate
and a deploy job that wait on the pause workflow job
This tutorial assumes prior knowledge of:
Please visit the linked pages if you are not familiar with these concepts yet!
Create a bot token
In order to use the Sym API Orb on CircleCI you will need to create a SYM_JWT
and set it as an environment variable available for use in your pipeline.
- Create a Bot user with
symflow bots create [bot-name]
- Issue a
SYM_JWT
withsymflow tokens issue -u [bot-name] -e [expiry-length]
- Save the JWT outputted by this previous step into an environment variable available to your workflow. You can do this by adding the
SYM_JWT
environment variable to a CircleCI Context OR the Project Settings.
For more detailed documentation about Sym Bots and Tokens, visit Bot Users and Tokens.
Configure Terraform
Declare a custom integration for CircleCI
This tutorial's impl.py
will be making API calls to CircleCI, which requires a personal CircleCI API Token. For instructions on how to create a CircleCI API Token, visit CircleCI's documentation: Managing API Tokens.
Consider using a dedicated bot user for the personal API token that you integrate with Sym. The approving user in CircleCi will be the user whose personal API token you set up for the integration.
After generating a CircleCI API Token, add a sym_secret
and sym_integration
resource to your Terraform. This will make your CircleCI Token available in impl.py
. See Manage Secrets with AWS Secrets Manager and Set Up an Integration that Utilizes Secrets for more detailed steps about how to configure AWS Secrets Manager and set the secret value.
# Follow the tutorial in "Manage Secrets with AWS Secrets Manager"
# for instructions on how to configure AWS Secrets Manager for Sym secrets
# and how to set the secret value.
resource "sym_secret" "circleci_api_key" {
path = aws_secretsmanager_secret.circleci_api_key.name
source_id = sym_secrets.this.id
}
# This custom integration exposes the CircleCI token as `secrets[0]`
resource "sym_integration" "circleci" {
type = "custom"
name = "circleci"
# A unique identifier for this integration
external_id = "symopsio"
settings = {
secret_ids_json = jsonencode([sym_secret.circleci_api_key.id])
}
}
Because circleci
is not a built-in Sym integration type, we will utilize the custom
type. sym_integrations
with type = custom
have a setting secret_ids_json
which accepts a JSON-encoded list of sym_secret
IDs. These will be available later in impl.py
as event.flow.environment.integrations["circleci"].secrets
. In this example, there is only one secret we need to provide for this integration.
Add the CircleCI Integration to your sym_environment
sym_environment
Add the sym_integration.circleci
to your sym_environment
integrations list so that it's available to the Sym Runtime.
resource "sym_environment" "this" {
# ... truncated for brevity
integrations = {
# This integration will now be available as `event.flow.environment.integrations["circleci"]`
circleci_id = sym_integration.circleci.id
# ... other integrations
}
}
Terraform a Sym Flow to be called by CircleCI
Now, we need to add a new Flow that can only be called by the Sym API. Set the allowed_sources to api
to ensure this flow does not show up in the Sym Flow Selector.
This flow will require three inputs, workflow_id
, workflow_url
and merging_user
. We will also specify that it can be called only via API so it doesn't appear in the user-facing Flow Selection modal in Slack, outside the context of a specific CircleCI run.
resource "sym_flow" "this" {
# This flow is an approval-only flow, so there is no strategy!
name = "ci-approval"
label = "CI Approval"
implementation = file("${path.module}/impl.py")
environment_id = sym_environment.this.id
params {
# allowed_sources defines the sources from which this flow can be invoked.
# For this tutorial, we are setting it to API only.
allowed_sources = ["api"]
# Each prompt_field defines an input the Flow needs to run.
# For this tutorial, we want the Circle CI Workflow and User.
prompt_field {
name = "workflow_url"
label = "CI Workflow URL"
type = "string"
required = true
}
prompt_field {
name = "workflow_id"
label = "CI Workflow ID"
type = "string"
required = true
}
prompt_field {
name = "merging_user"
label = "User who merged PR"
type = "string"
required = true
}
}
}
Configure impl.py
Create the necessary CircleCI helper methods
This impl.py
contains several helper methods that we'll use to manage the CircleCI piece of our workflow.
circleci_authentication_header
: Grabs the CircleCI Token configured in thecircleci
sym_integration
and generates a HTTP HeaderCircle-Token
to authenticatefetch_circle_ci_jobs
: Calls CircleCI's API to get the workflow's jobsapprove_circle_ci_hold
: Calls CircleCi's API to approve the pause workflow job and continue the workflow
Add the after_approve
hook
after_approve
hookFinally, we'll need an after_approve
hook that uses the workflow_id
prompt field to trigger the wait_for_sym_approval
job in CircleCI. Please note that the job name needs to match what is configured in your CircleCI config, which we'll set in the next section.
import requests
from sym.sdk.annotations import hook
def circleci_authentication_header(event):
"""Grabs the Circle CI API Token from the environment integrations
block (defined as circleci_id in the sym_environment Terraform resource)
"""
integration = event.flow.environment.integrations["circleci"]
token = integration.settings["secrets"][0].retrieve_value()
return {"Circle-Token": token}
def fetch_circle_ci_jobs(session, workflow_id):
"""Get all jobs in the workflow"""
response = session.get(f"https://circleci.com/api/v2/workflow/{workflow_id}/job")
return response.json()
def approve_circle_ci_hold(session, workflow_id, approval_request_id):
"""Post request to approve CircleCI hold job"""
response = session.post(
f"https://circleci.com/api/v2/workflow/{workflow_id}/approve/{approval_request_id}"
)
return response.json()
@hook
def after_approve(event):
# The workflow_id is defined in our sym_flow's prompt_fields_json
workflow_id = event.payload.fields.get("workflow_id")
with requests.Session() as session:
# Set the Circle-Token header to authenticate our API requests
session.headers.update(circleci_authentication_header(event))
# The `wait_for_sym_approval` job will be defined in our CircleCI config
job_list = fetch_circle_ci_jobs(session, workflow_id)
circle_approval_step = [
job["id"] for job in job_list["items"] if job["name"] == "wait_for_sym_approval"
]
circle_approval_step_id = circle_approval_step[0]
# Approve the pause workflow job, to trigger the workflow to continue
approve_circle_ci_hold(session, workflow_id, circle_approval_step_id)
Add Sym to your CircleCI Pipeline
We'll use two convenience jobs from the Sym Orb to set up our workflow: sym/request
and sym/validate
.
Create a CircleCI Workflow with the following steps:
- A
sym/request
job- This will programmatically make a Sym Request
- A job with
type: approval
- This will pause your workflow until your Sym Request has been approved
- This job should depend on the
sym/request
job
- A
sym/validate
job- This optional job makes sure that the approval really was triggered by Sym
- This job should depend on the approval job
- Your deploy job
- This job should depend on the
sym/validate
job
- This job should depend on the
orbs:
sym: sym/[email protected]
workflows:
main:
jobs:
# This will start the Sym flow
- sym/request:
flow_srn: sym:flow:ci-approval-prod:latest
# These environment variables are built into Circle CI!
flow_inputs: '{
"workflow_url": "${CIRCLE_BUILD_URL}",
"merging_user": "${CIRCLE_USERNAME}",
"workflow_id": "${CIRCLE_WORKFLOW_ID}"
}'
request_slug: "prod_deploy"
# Once approved, Sym will resume the CircleCI flow from this step.
- wait_for_sym_approval:
type: approval
requires:
- sym/request
# This is the step gated by Sym approval
- sym/validate:
request_slug: "prod_deploy"
requires:
- wait_for_sym_approval
# This is the step gated by Sym approval
- prod_deploy:
requires:
- sym/validate
Using flow_context
flow_context
You can send additional context data to your Sym Flow by providing flow_context
to the sym/request
job.
Saving flow_context
data to the sym/context path
flow_context
data to the sym/context pathIf you want to supply context that is generated by a previous job step, you can do this by adding context files to your CircleCI Workspace. Any files you add to the sym/context
path in your workspace will be automatically aggregated into a context.json
file that is sent to your Sym Flow implementation.
Here we generate a list of changed files and save this to sym/context/diff.txt
:
jobs:
# Generate a list of changed files and save to the sym/context path
git-diff:
docker:
- image: cimg/base:stable
steps:
- checkout
- run: mkdir -p sym/context
- run: git diff --name-only HEAD^..${CIRCLE_SHA1} > sym/context/diff.txt
- persist_to_workspace:
root: .
paths:
- sym/context/diff.txt
Accessing context in your Flow implementation
Use the SDK's get_context to access your context data:
@hook
def on_request(event):
"""
If the request include a diff, then check if this changes includes any Terraform files.
if there are no Terraform files, then auto-approve the request.
"""
context = event.get_context("request")
diff = context.get("diff.txt", "")
if no_terraform_files(diff):
return ApprovalTemplate.approve(reason="No terraform changes, auto approved!")
Full Example
With these pieces, your CircleCI pipeline can now make a Sym Request, pause the workflow, and automatically continue after the request has been approved!
You can find the complete code for this example in our Approving a CircleCI Job with Sym Example.
Updated 5 months ago