DCP User Guide

Document Purpose: This document describes how to use the Deploy Control Plane (DCP) for MuleSoft Service Teams.

  • Glossary
  • How Automated Deployments work
    • Failure handling
    • Cancelling a deployment
    • GOV processing for deployments
    • Deployment state lifecycles
    • TODO: Deployment Failure Troubleshooting Tips
  • DCP Events
  • Supporting Change Freezes
  • API Swagger locations
  • Deployment Events
  • DCP App
    • Find your team and service
    • Find deployments
    • Create a manual deployment
    • Cloning a Deployment
    • Configuring Slack notifications
    • Correcting Team or Service information via Muleteer and Github

Glossary

|Term |Description | |--- |--- | |Deployment |describes a series of phases by environment. | |Phase |a series of releases to cluster targets for a single environment | |Phase Prerequisite |items that must be completed before a phase can begin like a GUS change case approval or an approval for continuous-delivery prior to deploying to Production | |Release |a Helm deployment by CPC | |CPC |Core-Paas Client - used to deploy to our clusters | |Deploy Strategy |describes a progression of environments that an artifact manifest should be deployed to. | | | | | | |

How Automated Deployments Work

DCP will automatically deploy an artifact manifest to the cluster targets in each environment. DCP deployments are triggered by a BuildCompleted event published from the build tooling. DCP uses a database to manage deployments and their state.

  1. DCP receives the BuildCompleted event, which includes artifact manifest, MCN component name, channel, deployment strategy attributes and more.

  2. DCP creates a Deployment using details from the event payload.

    1. Note: DCP maintains a table of deployment targets (clusters by environments, etc) imported from channels.yaml.
    2. DCP finds the environments to include in this deployment and creates ordered Phases for each environment. DCP queries the targets table by deployment strategy and channel.
    3. Each Phase finds a list of targets for this phase’s environment.
    4. if this phase will go to GOV, we add GOV phases to the deployment. See 'GOV Processing for Deployments' below for more.

  3. Once a deployment is created, we then wait for the artifact's security scanning to complete. To do this, DCP initiates an Argo workflow that polls the artifact API every minute for scan status. When we detect the artifct is scanned, the worflow calls our DCP API to set the status to Scanned. This workflow does not time out, but will be cancelled ifß you cancel a deployment. See the Scanning/Scanned workflow states at Deployment States

  4. Once a deployment is scanned, we initiate the deployment’s first Phase.

  5. Depending on the environment’s configuration, a phase may require one or more Checks. If a check fails, the Phase is immediately cancelled. See Phase Checks

  6. Depending on the environment’s configuration, a phase may require a Change Case. If a Change Case is required, we will create a GUS change case in your team’s queue for you. See “Phase pre-requisites” below. See Phase Approval

  7. As we work though the Deployment, we send each Phase to an Argo workflow which concurrently calls CPC on each Release’s targets. Workflow notifies DCP as it progresses through each Phase and Release. Users can monitor the deployment progress on the DCP UI.

  8. Phase Pre-requisites: an environment can be configured with a variety of pre-requisites that require approval by humans, GUS cases, and more. See Phase Approval

  9. Once all Phases have been deployed, the Deployment is marked as Completed. If any of the Phases fail, the Deployment will be marked as Failed. If any of the Phases are cancelled, the Deployment will be marked as Cancelled. See the affected Phase's statusDetail field for more info. See Troubleshooting for more info.

GOV Processing for Deployments

DCP also deploys to GOV environments if the service's isGov attribute is true. We read the isGov attribute from the kilonova.yaml file in the service's repo. In the step described above where DCP queries the deployment targets table, DCP will appended GOV phases to the Deployment. Gov phases will create a separate change case from the Commercial Production change case and service teams will need to review the change case that is auto-created. Update the GUS change type, change risk, set the deploy window, and submit the GUS case for approvals. You will be notified via standard GUS notifications. Once the change case is in “scheduled-approved” status, the Phase will evaluate the case for approvals, pause for subsequent approvals and then begin deploying. If the GUS approval is rejected, or if the timeout of 14 hours is exceeded, the Phase will be automatically cancelled.

Failure Handling

When any Release fails, the workflow engine will notify DCP of the failure, including details about the error. A Release failure will quickly bubble up to a Phase failure and then a Deployment failure. A DeploymentPhaseFailed event is raised to the event bus and a slack notification is sent to the team (if their slack channel is configured). Users are linked to the DCP UI to view details for the error and enable troubleshooting. See Troubleshooting for more info.

Cancelling a Deployment

A user may wish to cancel a deployment for a variety of reasons. Perhaps the deployment has become “stuck“ and does not seem to be progressing for a long period of time or the user discovers that the deployment was mistakenly triggered. When a deployment is cancelled, the current Phase notifies the workflow to stop its processing and DCP does not progress to the next phase.

To cancel a deployment, open the DCP app and find your deployment in the list. Alternatively you may click the deployment link inside the Slack notifications your team was sent. Once found, click the Cancel button next to the deployment. (insert picture)

Supporting Change Freezes

When a change freeze is in place, GUS cases cannot be actioned to “scheduled-approved”. As DCP relies on this state, deployments to Prod are blocked until the freeze is lifted.

Authentication and Authorization

TBD

Configuring Slack notifications

Slack channel and workspace are imported from Github.

More topics

Version: 0.3.110
Last Updated: 2024-07-01T19:32:00+0000