CI/CD Pipelines
Production CI/CD with GitHub Actions, testing automation, and deployment strategies.
Quick Start
# .github/workflows/ci.yml
name: CI Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip'
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Run linting
run: ruff check .
- name: Run type checking
run: mypy src/
- name: Run tests
run: pytest tests/ --cov=src --cov-report=xml
- name: Upload coverage
uses: codecov/codecov-action@v3
with:
file: coverage.xml
Core Concepts
1. Complete CI/CD Pipeline
# .github/workflows/deploy.yml
name: Deploy Pipeline
on:
push:
branches: [main]
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- run: pip install -r requirements.txt && pytest
build:
needs: test
runs-on: ubuntu-latest
outputs:
image_tag: ${{ steps.meta.outputs.tags }}
steps:
- uses: actions/checkout@v4
- name: Log in to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=sha,prefix=
type=ref,event=branch
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
deploy-staging:
needs: build
runs-on: ubuntu-latest
environment: staging
steps:
- name: Deploy to staging
run: |
kubectl set image deployment/app \
app=${{ needs.build.outputs.image_tag }}
deploy-production:
needs: [build, deploy-staging]
runs-on: ubuntu-latest
environment: production
steps:
- name: Deploy to production
run: |
kubectl set image deployment/app \
app=${{ needs.build.outputs.image_tag }}
2. Matrix Testing
jobs:
test:
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12']
os: [ubuntu-latest, macos-latest]
database: [postgres, mysql]
exclude:
- os: macos-latest
database: mysql
runs-on: ${{ matrix.os }}
services:
postgres:
image: postgres:16
env:
POSTGRES_PASSWORD: test
options: >-
--health-cmd pg_isready
--health-interval 10s
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- run: pytest --db=${{ matrix.database }}
3. Reusable Workflows
# .github/workflows/python-ci.yml (reusable)
name: Python CI
on:
workflow_call:
inputs:
python-version:
required: false
type: string
default: '3.12'
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
- run: pip install -r requirements.txt && pytest
# Usage in another workflow
jobs:
ci:
uses: ./.github/workflows/python-ci.yml
with:
python-version: '3.11'
4. Deployment Strategies
# Blue-Green Deployment
deploy:
steps:
- name: Deploy to green
run: |
kubectl apply -f k8s/deployment-green.yaml
kubectl rollout status deployment/app-green
- name: Run smoke tests
run: ./scripts/smoke-test.sh $GREEN_URL
- name: Switch traffic
run: |
kubectl patch service app \
-p '{"spec":{"selector":{"version":"green"}}}'
- name: Cleanup blue
run: kubectl delete deployment app-blue
# Canary Deployment
deploy-canary:
steps:
- name: Deploy canary (10%)
run: |
kubectl apply -f k8s/deployment-canary.yaml
kubectl scale deployment/app-canary --replicas=1
kubectl scale deployment/app-stable --replicas=9
- name: Monitor canary
run: ./scripts/monitor-canary.sh --duration=30m
- name: Promote or rollback
run: |
if [ "$CANARY_SUCCESS" == "true" ]; then
kubectl scale deployment/app-canary --replicas=10
kubectl scale deployment/app-stable --replicas=0
else
kubectl delete deployment/app-canary
fi
Tools & Technologies
| Tool | Purpose | Version (2025) | |------|---------|----------------| | GitHub Actions | CI/CD platform | Latest | | GitLab CI | CI/CD platform | 16+ | | ArgoCD | GitOps for K8s | 2.10+ | | Terraform | Infrastructure | 1.6+ | | act | Local testing | 0.2+ |
Troubleshooting Guide
| Issue | Symptoms | Root Cause | Fix |
|-------|----------|------------|-----|
| Workflow Not Running | No job triggered | Wrong trigger config | Check on: section |
| Secret Not Available | Empty variable | Missing secret | Add in repo settings |
| Slow Builds | Long duration | No caching | Add cache steps |
| Flaky Tests | Random failures | Race conditions | Fix tests, add retries |
Best Practices
# ✅ DO: Cache dependencies
- uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
# ✅ DO: Use environments for deployments
environment: production
# ✅ DO: Pin action versions
- uses: actions/checkout@v4 # Not @main
# ✅ DO: Add timeouts
jobs:
test:
timeout-minutes: 10
# ❌ DON'T: Store secrets in code
# ❌ DON'T: Skip tests for faster deployments
Resources
Skill Certification Checklist:
- [ ] Can create CI pipelines with testing
- [ ] Can build and push Docker images
- [ ] Can deploy to Kubernetes
- [ ] Can implement deployment strategies
- [ ] Can create reusable workflows