Deployment Guide

Table of Contents

1. Prerequisites
2. Environment Setup
3. Docker Deployment
4. Kubernetes Deployment
5. Database Setup
6. Configuration Management
7. CI/CD Pipeline
8. Monitoring & Logging
9. Backup & Recovery
10. Troubleshooting

Prerequisites

System Requirements

• CPU: 2+ cores (4+ recommended for production)
• RAM: 4GB minimum (8GB+ recommended)
• Storage: 20GB minimum (100GB+ for production)
• OS: Linux (Ubuntu 20.04+, CentOS 8+), macOS, Windows Server 2019+

Software Requirements

• Docker 24.0+
• Docker Compose 2.20+
• Kubernetes 1.28+ (for K8s deployment)
• PostgreSQL 15+
• Redis 7+
• Go 1.24+ (for building from source)

Network Requirements

• Ports: 8080 (API), 5432 (PostgreSQL), 6379 (Redis)
• SSL certificates for HTTPS
• Domain name for production

Environment Setup

1. Clone Repository

git clone https://github.com/your-org/ms-project.git
cd ms-project

2. Environment Variables

Create .env file from template:

cp .env.example .env

Edit .env with your configuration:

# Server Configuration
NODE_ENV=production
PORT=8080

# Database Configuration
DATABASE_URL=postgresql://postgres:password@db:5432/ms_project?sslmode=require
DB_MAX_IDLE_CONNS=10
DB_MAX_OPEN_CONNS=100
DB_CONN_MAX_LIFETIME=1

# Redis Configuration
REDIS_HOST=redis
REDIS_PORT=6379
REDIS_PASSWORD=redis_password

# JWT Configuration
JWT_SECRET=your-secure-jwt-secret-min-32-chars

# S3 Storage
S3_ENDPOINT=https://s3.amazonaws.com
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=your_access_key
S3_SECRET_ACCESS_KEY=your_secret_key
S3_BUCKET_NAME=ms-project-files

# Service URLs
AUTH_SERVICE_URL=http://ms-auth:8080
NOTIFICATIONS_SERVICE_URL=http://ms-notifications:8080

# External URLs
DOMAIN=https://api.yourdomain.com
FRONTEND_URL=https://app.yourdomain.com

Docker Deployment

1. Build Docker Image

# Production build
docker build -t ms-project:latest \
  --build-arg GO_VERSION=1.24 \
  --target production .

# Development build with hot reload
docker build -t ms-project:dev \
  --target development .

2. Docker Compose Setup

Create docker-compose.yml:

version: '3.8'

services:
  app:
    image: ms-project:latest
    container_name: ms-project-app
    ports:
      - "8080:8080"
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgresql://postgres:password@db:5432/ms_project
      - REDIS_HOST=redis
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
    networks:
      - ms-project-network
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  db:
    image: postgres:15-alpine
    container_name: ms-project-db
    environment:
      POSTGRES_DB: ms_project
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./migrations:/docker-entrypoint-initdb.d
    ports:
      - "5432:5432"
    networks:
      - ms-project-network
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    container_name: ms-project-redis
    command: redis-server --appendonly yes --requirepass redis_password
    volumes:
      - redis_data:/data
    ports:
      - "6379:6379"
    networks:
      - ms-project-network
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5

  nginx:
    image: nginx:alpine
    container_name: ms-project-nginx
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./ssl:/etc/nginx/ssl
    depends_on:
      - app
    networks:
      - ms-project-network

volumes:
  postgres_data:
  redis_data:

networks:
  ms-project-network:
    driver: bridge

3. Nginx Configuration

Create nginx.conf:

events {
    worker_connections 1024;
}

http {
    upstream ms-project {
        server app:8080;
    }

    server {
        listen 80;
        server_name api.yourdomain.com;
        return 301 https://$server_name$request_uri;
    }

    server {
        listen 443 ssl http2;
        server_name api.yourdomain.com;

        ssl_certificate /etc/nginx/ssl/cert.pem;
        ssl_certificate_key /etc/nginx/ssl/key.pem;

        location / {
            proxy_pass http://ms-project;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            # WebSocket support
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }

        location /health {
            access_log off;
            proxy_pass http://ms-project/health;
        }
    }
}

4. Start Services

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f

# Stop services
docker-compose down

Kubernetes Deployment

1. Create Namespace

apiVersion: v1
kind: Namespace
metadata:
  name: ms-project

2. ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: ms-project-config
  namespace: ms-project
data:
  NODE_ENV: "production"
  PORT: "8080"
  DB_HOST: "postgres-service"
  DB_PORT: "5432"
  DB_NAME: "ms_project"
  REDIS_HOST: "redis-service"
  REDIS_PORT: "6379"

3. Secret

apiVersion: v1
kind: Secret
metadata:
  name: ms-project-secret
  namespace: ms-project
type: Opaque
data:
  db-password: <base64-encoded-password>
  jwt-secret: <base64-encoded-jwt-secret>
  redis-password: <base64-encoded-redis-password>

4. Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ms-project
  namespace: ms-project
spec:
  replicas: 3
  selector:
    matchLabels:
      app: ms-project
  template:
    metadata:
      labels:
        app: ms-project
    spec:
      containers:
      - name: ms-project
        image: ms-project:latest
        ports:
        - containerPort: 8080
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: ms-project-secret
              key: database-url
        - name: JWT_SECRET
          valueFrom:
            secretKeyRef:
              name: ms-project-secret
              key: jwt-secret
        envFrom:
        - configMapRef:
            name: ms-project-config
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5

5. Service

apiVersion: v1
kind: Service
metadata:
  name: ms-project-service
  namespace: ms-project
spec:
  selector:
    app: ms-project
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  type: LoadBalancer

6. Horizontal Pod Autoscaler

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: ms-project-hpa
  namespace: ms-project
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: ms-project
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

7. Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ms-project-ingress
  namespace: ms-project
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
  - hosts:
    - api.yourdomain.com
    secretName: ms-project-tls
  rules:
  - host: api.yourdomain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: ms-project-service
            port:
              number: 80

8. Deploy to Kubernetes

# Apply configurations
kubectl apply -f k8s/namespace.yaml
kubectl apply -f k8s/configmap.yaml
kubectl apply -f k8s/secret.yaml
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/service.yaml
kubectl apply -f k8s/hpa.yaml
kubectl apply -f k8s/ingress.yaml

# Check status
kubectl get all -n ms-project

# View logs
kubectl logs -f deployment/ms-project -n ms-project

# Scale manually
kubectl scale deployment/ms-project --replicas=5 -n ms-project

Database Setup

1. Initialize Database

# Connect to PostgreSQL
psql -h localhost -U postgres

# Create database
CREATE DATABASE ms_project;

# Create user
CREATE USER ms_project_user WITH PASSWORD 'secure_password';

# Grant privileges
GRANT ALL PRIVILEGES ON DATABASE ms_project TO ms_project_user;

2. Run Migrations

# Using migrate CLI
migrate -path ./migrations \
        -database "postgresql://user:password@localhost:5432/ms_project?sslmode=disable" \
        up

# Or using make
make migrate-up

3. Database Optimization

-- Performance tuning
ALTER SYSTEM SET shared_buffers = '256MB';
ALTER SYSTEM SET effective_cache_size = '1GB';
ALTER SYSTEM SET maintenance_work_mem = '64MB';
ALTER SYSTEM SET checkpoint_completion_target = 0.9;
ALTER SYSTEM SET wal_buffers = '7864kB';
ALTER SYSTEM SET default_statistics_target = 100;
ALTER SYSTEM SET random_page_cost = 1.1;

-- Reload configuration
SELECT pg_reload_conf();

Configuration Management

1. Using Consul

# consul-config.yaml
services:
  - name: ms-project
    port: 8080
    tags:
      - api
      - v1
    checks:
      - http: http://localhost:8080/health
        interval: 10s

2. Using Vault for Secrets

# Store secrets in Vault
vault kv put secret/ms-project \
  db_password="secure_password" \
  jwt_secret="secure_jwt_secret" \
  api_keys="key1,key2,key3"

# Retrieve secrets
vault kv get secret/ms-project

CI/CD Pipeline

GitHub Actions

name: Deploy MS-Project

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3

    - name: Setup Go
      uses: actions/setup-go@v4
      with:
        go-version: '1.24'

    - name: Run tests
      run: |
        go test -v -race -coverprofile=coverage.out ./...
        go tool cover -html=coverage.out -o coverage.html

    - name: Run linting
      run: |
        golangci-lint run --timeout=5m

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3

    - name: Build Docker image
      run: |
        docker build -t ms-project:${{ github.sha }} .

    - name: Push to registry
      run: |
        echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
        docker tag ms-project:${{ github.sha }} ${{ secrets.DOCKER_REGISTRY }}/ms-project:${{ github.sha }}
        docker push ${{ secrets.DOCKER_REGISTRY }}/ms-project:${{ github.sha }}

  deploy:
    needs: build
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
    - name: Deploy to Kubernetes
      run: |
        kubectl set image deployment/ms-project \
          ms-project=${{ secrets.DOCKER_REGISTRY }}/ms-project:${{ github.sha }} \
          -n ms-project

GitLab CI/CD

stages:
  - test
  - build
  - deploy

test:
  stage: test
  image: golang:1.24
  script:
    - go test -v -race ./...
    - golangci-lint run

build:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

deploy:
  stage: deploy
  image: bitnami/kubectl:latest
  script:
    - kubectl set image deployment/ms-project ms-project=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA -n ms-project
  only:
    - main

Monitoring & Logging

1. Prometheus Setup

# prometheus-config.yaml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'ms-project'
    static_configs:
      - targets: ['ms-project:8080']
    metrics_path: '/metrics'

2. Grafana Dashboards

Import dashboard JSON:

{
  "dashboard": {
    "title": "MS-Project Metrics",
    "panels": [
      {
        "title": "Request Rate",
        "targets": [
          {
            "expr": "rate(http_requests_total[5m])"
          }
        ]
      },
      {
        "title": "Response Time",
        "targets": [
          {
            "expr": "histogram_quantile(0.95, http_request_duration_seconds_bucket)"
          }
        ]
      }
    ]
  }
}

3. ELK Stack Setup

# filebeat.yml
filebeat.inputs:
- type: container
  paths:
    - '/var/lib/docker/containers/*/*.log'
  processors:
    - add_kubernetes_metadata:
        host: ${NODE_NAME}
        matchers:
        - logs_path:
            logs_path: "/var/lib/docker/containers/"

output.elasticsearch:
  hosts: ['elasticsearch:9200']

4. Application Metrics

// Instrument code with metrics
import "github.com/prometheus/client_golang/prometheus"

var (
    requestDuration = prometheus.NewHistogramVec(
        prometheus.HistogramOpts{
            Name: "http_request_duration_seconds",
            Help: "HTTP request duration in seconds",
        },
        []string{"method", "endpoint", "status"},
    )
)

func init() {
    prometheus.MustRegister(requestDuration)
}

Backup & Recovery

1. Database Backup

#!/bin/bash
# backup.sh

BACKUP_DIR="/backups"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
DB_NAME="ms_project"

# Create backup
pg_dump -h localhost -U postgres -d $DB_NAME \
  --format=custom --compress=9 \
  --file="$BACKUP_DIR/backup_$TIMESTAMP.dump"

# Upload to S3
aws s3 cp "$BACKUP_DIR/backup_$TIMESTAMP.dump" \
  s3://ms-project-backups/db/backup_$TIMESTAMP.dump

# Cleanup old backups (keep last 30 days)
find $BACKUP_DIR -name "*.dump" -mtime +30 -delete

2. Automated Backup CronJob

apiVersion: batch/v1
kind: CronJob
metadata:
  name: database-backup
  namespace: ms-project
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: postgres:15
            command: ["/scripts/backup.sh"]
            volumeMounts:
            - name: backup-script
              mountPath: /scripts
          volumes:
          - name: backup-script
            configMap:
              name: backup-scripts
          restartPolicy: OnFailure

3. Disaster Recovery

# Restore from backup
pg_restore -h localhost -U postgres -d ms_project_restore \
  --clean --if-exists --verbose backup_20240101.dump

# Point-in-time recovery
recovery_target_time = '2024-01-01 12:00:00'
restore_command = 'cp /archive/%f %p'

Troubleshooting

Common Issues

1. Database Connection Errors

# Check PostgreSQL status
docker-compose ps db
docker-compose logs db

# Test connection
psql -h localhost -U postgres -d ms_project -c "SELECT 1"

# Check connection pool
SELECT count(*) FROM pg_stat_activity;

2. High Memory Usage

# Check memory usage
docker stats ms-project-app

# Analyze Go memory
go tool pprof http://localhost:8080/debug/pprof/heap

# Increase memory limit
docker update --memory="1g" ms-project-app

3. Slow API Response

# Enable query logging
ALTER SYSTEM SET log_statement = 'all';
SELECT pg_reload_conf();

# Check slow queries
SELECT query, mean_exec_time
FROM pg_stat_statements
ORDER BY mean_exec_time DESC
LIMIT 10;

# Analyze query plan
EXPLAIN ANALYZE SELECT ...;

4. Container Restart Loop

# Check logs
kubectl logs -p deployment/ms-project -n ms-project

# Describe pod
kubectl describe pod <pod-name> -n ms-project

# Check events
kubectl get events -n ms-project --sort-by='.lastTimestamp'

Debug Mode

# Enable debug logging
environment:
  - LOG_LEVEL=debug
  - GIN_MODE=debug
  - DB_LOG=true

Health Check Endpoints

# Overall health
curl http://localhost:8080/health

# Database health
curl http://localhost:8080/health/db

# Redis health
curl http://localhost:8080/health/redis

# Detailed metrics
curl http://localhost:8080/metrics

Security Hardening

1. Network Security

# Network Policy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: ms-project-netpol
  namespace: ms-project
spec:
  podSelector:
    matchLabels:
      app: ms-project
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: nginx
    ports:
    - protocol: TCP
      port: 8080

2. Pod Security

securityContext:
  runAsNonRoot: true
  runAsUser: 1000
  fsGroup: 1000
  capabilities:
    drop:
    - ALL
  readOnlyRootFilesystem: true

3. Secrets Management

# Rotate secrets
kubectl create secret generic ms-project-secret \
  --from-literal=jwt-secret=$(openssl rand -base64 32) \
  --dry-run=client -o yaml | kubectl apply -f -

# Encrypt secrets at rest
kubectl patch secret ms-project-secret -p \
  '{"metadata":{"annotations":{"kubectl.kubernetes.io/encryption":"aescbc"}}}'

Performance Tuning

1. Application Optimization

# Resource limits
resources:
  requests:
    memory: "512Mi"
    cpu: "500m"
  limits:
    memory: "1Gi"
    cpu: "1000m"

# JVM-like tuning for Go
environment:
  - GOGC=100  # Garbage collection target
  - GOMEMLIMIT=900MiB  # Memory limit

2. Database Optimization

-- Connection pooling
ALTER SYSTEM SET max_connections = 200;

-- Query optimization
CREATE INDEX CONCURRENTLY idx_projects_status_created
ON projects(status, created_at DESC)
WHERE deleted_at IS NULL;

-- Vacuum schedule
ALTER TABLE projects SET (autovacuum_vacuum_scale_factor = 0.1);

3. Caching Strategy

# Redis configuration
redis:
  maxmemory: 512mb
  maxmemory-policy: allkeys-lru
  save: ""  # Disable persistence for cache-only

Conclusion

This deployment guide covers:

• Multiple deployment strategies (Docker, Kubernetes)
• Complete configuration management
• CI/CD pipeline setup
• Monitoring and logging
• Backup and recovery procedures
• Troubleshooting common issues
• Security hardening
• Performance optimization

Follow these guidelines to ensure a robust, scalable, and secure deployment of MS-Project.