Skip to main content

CI/CD

CodePipeline​

AWS CodePipeline is a fully managed continuous integration and continuous delivery (CI/CD) service by Amazon Web Services. It automates the build, test, and deployment phases of your release process, allowing you to quickly and reliably deliver applications and updates, thus enhancing software delivery pipelines.

Input and output artifacts​

CodePipeline integrates with development tools to check for code changes and then build and deploy through all of the stages of the continuous delivery process.

Stages use input and output artifacts that are stored in the Amazon S3 artifact bucket you chose when you created the pipeline. CodePipeline zips and transfers the files for input or output artifacts as appropriate for the action type in the stage.

For example:

  1. CodePipeline triggers your pipeline to run when there is a commit to the source repository, providing the output artifact (any files to be built) from the Source stage.
  2. The output artifact (any files to be built) from the previous step is ingested as an input artifact to the Build stage. An output artifact (the built application) from the Build stage can be an updated application or an updated Docker image built to a container.
  3. The output artifact from the previous step (the built application) is ingested as an input artifact to the Deploy stage, such as staging or production environments in the AWS Cloud. You can deploy applications to a deployment fleet, or you can deploy container-based applications to tasks running in ECS clusters.

Every action has a type. Depending on the type, the action might have one or both of the following:

  • An input artifact, which is the artifact it consumes or works on over the course of the action run.
  • An output artifact, which is the output of the action.

Every output artifact in the pipeline must have a unique name. Every input artifact for an action must match the output artifact of an action earlier in the pipeline, whether that action is immediately before the action in a stage or runs in a stage several stages earlier.

An artifact can be worked on by more than one action.

CodeDeploy vs CodePipeline​

Whereas CodeDeploy is a deployment service, CodePipeline is a continuous delivery service.

AWS CodeDeploy is a fully managed "deployment" service that automates software deployments to a variety of compute services such as Amazon EC2, AWS Fargate, AWS Lambda, and your on-premises servers. AWS CodeDeploy makes it easier for you to rapidly release new features, helps you avoid downtime during application deployment, and handles the complexity of updating your applications.

AWS CodePipeline is a fully managed "continuous delivery" service that helps you automate your release pipelines for fast and reliable application and infrastructure updates. CodePipeline automates the build, test, and deploy phases of your release process every time there is a code change, based on the release model you define. This enables you to rapidly and reliably deliver features and updates.

CodeCommit​

AWS CodeCommit is a fully managed source control service by Amazon Web Services. It provides secure and scalable Git repositories for hosting source code, artifacts, and other development assets. CodeCommit simplifies collaborative software development and integrates seamlessly with other AWS services and popular development tools.

Repositories are automatically encrypted in transit and at rest​

Data in AWS CodeCommit repositories is encrypted in transit and at rest. When data is pushed into an AWS CodeCommit repository (for example, by calling git push), AWS CodeCommit encrypts the received data as it is stored in the repository.

How to set up connections for using Git in CodeCommit​

  • The simplest way to set up connections to AWS CodeCommit repositories is to configure Git credentials for CodeCommit in the IAM console, and then use those credentials for HTTPS connections. 
  • You can also use these same credentials with any third-party tool or individual development environment (IDE) that supports HTTPS authentication using a static user name and password.

EC2 Rollback behavoir​

Q: You are a developer handling a deployment service that automates application deployments to Amazon EC2 instances. Most of the deployments consist of code, but sometimes web and configuration files. One of your deployments failed and was rolled back by AWS CodeDeploy to the last known good application revision.

During rollback which of the following instances did AWS CodeDeploy deploy first to?

A: To the failed instances: AWS CodeDeploy rolls back deployments by 

  1. Remove all files that were last successfully installed. demo
  2. Redeploy a previously deployed revision of an application as a new deployment on the failed instances.

AWS CodeBuild​

Speed up the build time​

You can use enable cache for dependencies on S3 you want to speed up the process when CodeBuild runs a build step takes ages due to its dependency size (more than 100 dependencies),

Downloading dependencies is a critical phase in the build process. These dependent files can range in size from a few KBs to multiple MBs. Because most of the dependent files do not change frequently between builds, you can noticeably reduce your build time by caching dependencies in S3.

Check the section of best practices for cache of this AWS blocg post - How to Enable Caching for AWS CodeBuild for details.

Integrate S3 and CloudWatch logs​

AWS CodeBuild monitors functions on your behalf and reports metrics through Amazon CloudWatch. These metrics include the number of total builds, failed builds, successful builds, and the duration of builds.

You can monitor your builds at two levels: Project level, AWS account level. You can export log data from your log groups to an Amazon S3 bucket and use this data in custom processing and analysis, or to load onto other systems.

Integrated with KMS​

In order to let AWS CodeBuild to encrypt its build output artifacts, it needs access to an AWS KMS customer master key (CMK).

By default, AWS CodeBuild uses the AWS-managed CMK for Amazon S3 in your AWS account. The following environment variable provides these details: CODEBUILD_KMS_KEY_ID: The identifier of the AWS KMS key that CodeBuild is using to encrypt the build output artifact (for example, arn:aws:kms:region-ID:account-ID:key/key-ID or alias/key-alias).

CodeDeploy​

Compute Platforms​

CDAF15-Overview-of-CodeDeploy-Compute-Platforms

Lifecycle​

You can specify one or more scripts to run in a hook. Each hook for a lifecycle event is specified with a string on a separate line. Please review the correct order of lifecycle events:

lifecycle-event-hook

Further reading: AppSpec 'hooks' section

Working with Code Deploy​

The below snapshot from the AWS documentation shows the structure of the Code Deploy service

how-codedeploy-work

Source: Working with Code Deploy

The different parts of CodeDeploy are as follows

  1. You first ensure that your code is available for deployment. The code package must also contain a file called appspec.yml. This AppSpec file is specific to AWS Code Deploy. It tells Code Deploy how to deploy the package to the underlying Instances.
  2. You then submit this as a release Code Deploy from either Github or from an S3 bucket.
  3. The code gets deployed to EC2 Instances which are put as part of a deployment Group. These EC@ instances need to have Code Deploy agents running on them. The agents in the EC2 Instances deployment group then polls Code Deploy and gets the details of the code to pull in.

You can have deployment groups for various stacks in your organization as shown below from the AWS documentation.

deployment-group

EC2 deployment​

We need to create 2 IAM Roles when deploying stuff to EC2

  • The first role is required for the EC2 Instances to work and communicate with the CodeDeploy service.
  • The second role is required by CodeDeploy to ensure it can work with the resources in the AWS account.

buildspec.yml & appspec.yml​

Alternative Products

  • CodeBuild - Jenkins
  • CodeDeploy - Ansible, chef, Puppet

Both yml are sitting in the root directory

  • CodeBuild - buildspec.yml
    • A build spec is a collection of build commands and related settings, in YAML format, that AWS CodeBuild uses to run a build.
    • You can include a build spec as part of the source code or you can define a build spec when you create a build project.
  • CodeDeploy - appspec.yml
    • Deployment TargetService can be Lambda, ECS, EC2
    • Define env, phases (install, pre_build ....), artifact, cache
    • Map the source files in your application revision to their destinations on the instance.
    • Specify custom permissions for deployed files.
    • Specify scripts to be run on each instance at various stages of the deployment process.
What is Jenkins & Ansible

Jenkins is a popular self hosted, open source CI (Continuous Integration) tool with numerous plugins, often leveraged to streamline DevOps pipeline.

The usages of Jenkins are automation of build and release for CI/CD; orchestration of CI/CD pipelines

Ansible is not strictly a CI/CD tool. However, it is a popular choice for automating application and system configurations and overseeing general IT automation needs.

The usages of Ansible are configuration management, intra-service orchestration and provisioning for complex cloud environments

Example​

Example from AWS - Create the buildspec file

version: 0.2

phases:
  install:
    runtime-versions:
      java: corretto11
  pre_build:
    commands:
      - echo Nothing to do in the pre_build phase...
  build:
    commands:
      - echo Build started on `date`
      - mvn install
  post_build:
    commands:
      - echo Build completed on `date`
artifacts:
  files:
    - target/messageUtil-1.0.jar

Important config​

  • CodeBuild
    • install, pre_build, build, post_build
    • artifacts
  • CodeDeploy - Deployment configurations
    • Target services (Lambda, ECS, EC2)
    • Lambda
      • LambdaAllAtOnce
      • LambdaLinear10PercentEvery10Minutes ... [Shifts 10 percent of traffic every 10 minutes until all traffic is shifted.]
      • LambdaCanary10Percent5Minutes... [Shifts 10 percent of traffic in the first increment. The remaining 90 percent is deployed five minutes later.]
    • ECS
      • ECSAllAtOnce
      • ECSCanary10Percent15Minutes..
      • ECSLinear10PercentEvery1Minutes...
    • EC2
      • AllAtOnce/ HalfAtATime / OneAtATime
        • In-place deployments
        • Blue/green deployments (Update DNS Routing)
Last updated on