market-data-service/docs/DEPLOYMENT.md

Production Deployment Guide

Note: This guide covers production deployment with fully containerized SSL certificate management. For local development, see LOCAL_DEV_SETUP.md. For Docker usage details, see DOCKER.md.

Complete guide for deploying Market Data Service to production with fully containerized SSL certificate management.

🎯 Overview

This deployment is fully containerized, including:

• ✅ Application (Node.js API)
• ✅ Database (PostgreSQL)
• ✅ Reverse Proxy (Nginx)
• ✅ SSL Certificate Obtain & Renewal (Certbot)
• ✅ Automatic certificate renewal every 12 hours
• ✅ Automatic nginx reload after certificate renewal

📋 Prerequisites

1. Docker and Docker Compose installed
2. Domain name pointing to your server's IP address
3. Ports 80 and 443 open in firewall
4. Email address for Let's Encrypt notifications

🚀 Deployment Steps

Step 1: Clone and Prepare

# Clone repository
git clone <your-repo-url>
cd market-data-service

# Create .env file for production
cat > .env << EOF
# Database Configuration
DB_USER=postgres
DB_PASSWORD=your_secure_password_here
DB_NAME=financial_data

# Application Configuration
NODE_ENV=production
PORT=3000
CORS_ORIGIN=https://yourdomain.com
JWT_SECRET=your_secure_jwt_secret_here
LOG_LEVEL=info

# SSL Configuration
DOMAIN_NAME=yourdomain.com
SSL_EMAIL=your-email@example.com
EOF

Important: Replace all placeholder values with your actual values.

Step 2: Start Services (Without SSL First)

# Start all services except certbot (we'll get certificates first)
docker-compose -f docker-compose.prod.yml up -d db api nginx

# Verify services are running
docker-compose -f docker-compose.prod.yml ps

# Check logs
docker-compose -f docker-compose.prod.yml logs -f

Step 3: Obtain SSL Certificate

Option A: Production Certificate (Recommended)

# Obtain production SSL certificate
DOMAIN_NAME=yourdomain.com \
SSL_EMAIL=your-email@example.com \
docker-compose -f docker-compose.prod.yml run --rm certbot-init

Option B: Staging Certificate (For Testing)

# Test with Let's Encrypt staging server (doesn't count against rate limits)
DOMAIN_NAME=yourdomain.com \
SSL_EMAIL=your-email@example.com \
SSL_STAGING=1 \
docker-compose -f docker-compose.prod.yml run --rm certbot-init

What this does:

• Requests SSL certificate from Let's Encrypt
• Uses webroot method (nginx serves the challenge)
• Stores certificates in Docker volume certbot_etc
• Certificates are automatically shared with nginx container

Step 4: Restart Nginx with SSL

# Restart nginx to load SSL certificates
docker-compose -f docker-compose.prod.yml restart nginx

# Start certbot for automatic renewal
docker-compose -f docker-compose.prod.yml up -d certbot

Step 5: Verify Deployment

# Check all services are running
docker-compose -f docker-compose.prod.yml ps

# Test HTTP (should redirect to HTTPS)
curl -I http://yourdomain.com

# Test HTTPS
curl https://yourdomain.com/health

# Test API endpoint
curl https://yourdomain.com/api/health

🔄 Automatic Certificate Renewal

Certificates are automatically renewed every 12 hours by the certbot container. The renewal process:

1. Certbot checks if certificates need renewal (30 days before expiry)
2. If renewal is needed, certbot obtains new certificates
3. Certbot signals nginx to reload
4. Nginx reloads within 60 seconds to use new certificates

No manual intervention required!

Manual Renewal (If Needed)

# Force certificate renewal
docker-compose -f docker-compose.prod.yml exec certbot certbot renew --force-renewal

# Restart nginx to load new certificates
docker-compose -f docker-compose.prod.yml restart nginx

Check Certificate Status

# View certificate information
docker-compose -f docker-compose.prod.yml exec certbot certbot certificates

# Check certificate expiry
docker-compose -f docker-compose.prod.yml exec certbot openssl x509 -in /etc/letsencrypt/live/yourdomain.com/cert.pem -noout -dates

📊 Monitoring

View Logs

# All services
docker-compose -f docker-compose.prod.yml logs -f

# Specific service
docker-compose -f docker-compose.prod.yml logs -f api
docker-compose -f docker-compose.prod.yml logs -f nginx
docker-compose -f docker-compose.prod.yml logs -f certbot
docker-compose -f docker-compose.prod.yml logs -f db

Health Checks

# API health
curl https://yourdomain.com/health

# Container health status
docker-compose -f docker-compose.prod.yml ps

🔧 Configuration

Environment Variables

Key environment variables in .env:

Variable	Description	Required
DB_PASSWORD	PostgreSQL password	Yes
JWT_SECRET	JWT secret key	Yes
DOMAIN_NAME	Your domain name	Yes
SSL_EMAIL	Email for Let's Encrypt	Yes
CORS_ORIGIN	Allowed CORS origins	Recommended

Nginx Configuration

Nginx configuration is in nginx/nginx.prod.conf. The domain name is automatically substituted at runtime from the DOMAIN_NAME environment variable.

SSL Certificate Paths

Certificates are stored in Docker volumes:

• Volume: certbot_etc → /etc/letsencrypt in containers
• Certificate path: /etc/letsencrypt/live/DOMAIN_NAME/fullchain.pem
• Key path: /etc/letsencrypt/live/DOMAIN_NAME/privkey.pem

🛠️ Troubleshooting

Certificate Obtain Fails

Problem: certbot-init fails with "Connection refused" or "Challenge failed"

Solutions:

1. Verify domain DNS points to your server:

   dig yourdomain.com
   nslookup yourdomain.com
   

2. Verify ports 80 and 443 are open:

   sudo ufw status
   # If needed: sudo ufw allow 80 && sudo ufw allow 443
   

3. Check nginx is running and accessible:

   curl http://yourdomain.com/.well-known/acme-challenge/test
   

4. Use staging server first to test:

   SSL_STAGING=1 docker-compose -f docker-compose.prod.yml run --rm certbot-init
   

Nginx Can't Find Certificates

Problem: Nginx fails to start with "SSL certificate not found"

Solutions:

1. Verify certificates exist:

   docker-compose -f docker-compose.prod.yml exec nginx ls -la /etc/letsencrypt/live/
   

2. Check DOMAIN_NAME matches certificate domain:

   # In .env file
   DOMAIN_NAME=yourdomain.com  # Must match exactly
   

3. Restart nginx:

   docker-compose -f docker-compose.prod.yml restart nginx
   

Certificate Renewal Not Working

Problem: Certificates expire without renewal

Solutions:

1. Check certbot container is running:

   docker-compose -f docker-compose.prod.yml ps certbot
   

2. Check certbot logs:

   docker-compose -f docker-compose.prod.yml logs certbot
   

3. Manually test renewal:

   docker-compose -f docker-compose.prod.yml exec certbot certbot renew --dry-run
   

Nginx Not Reloading After Renewal

Problem: New certificates not picked up by nginx

Solutions:

1. Check reload signal file:

   docker-compose -f docker-compose.prod.yml exec nginx ls -la /var/run/certbot-reload/
   

2. Manually reload nginx:

   docker-compose -f docker-compose.prod.yml exec nginx nginx -s reload
   

3. Restart nginx:

   docker-compose -f docker-compose.prod.yml restart nginx
   

🔐 Security Best Practices

1. Strong Passwords: Use strong, unique passwords for DB_PASSWORD and JWT_SECRET
2. Firewall: Only expose ports 80 and 443 to the internet
3. Regular Updates: Keep Docker images updated: bash docker-compose -f docker-compose.prod.yml pull docker-compose -f docker-compose.prod.yml up -d

4. Backup: Regularly backup database and SSL certificates: ```bash

   Backup database

   docker-compose -f docker-compose.prod.yml exec db pg_dump -U postgres financial_data > backup.sql

# Backup certificates (optional, they auto-renew) docker run --rm -v market-data-service_certbot_etc:/data -v $(pwd):/backup alpine tar czf /backup/certs_backup.tar.gz -C /data .

5. **Monitor Logs**: Set up log monitoring and alerting
6. **Rate Limiting**: Adjust rate limits in `nginx/nginx.prod.conf` as needed

## 📝 Maintenance

### Update Application

```bash
# Pull latest code
git pull

# Rebuild and restart
docker-compose -f docker-compose.prod.yml up -d --build

# Verify
docker-compose -f docker-compose.prod.yml ps
curl https://yourdomain.com/health

Database Backup

# Create backup
docker-compose -f docker-compose.prod.yml exec db pg_dump -U postgres financial_data > backup_$(date +%Y%m%d).sql

# Restore from backup
docker-compose -f docker-compose.prod.yml exec -T db psql -U postgres financial_data < backup_20250101.sql

View Certificate Expiry

docker-compose -f docker-compose.prod.yml exec certbot certbot certificates

🎉 Success Checklist

• All services running (docker-compose ps)
• HTTP redirects to HTTPS
• HTTPS endpoint accessible
• SSL certificate valid (check browser or curl -v)
• API endpoints responding
• Health check passing
• Certbot container running
• Automatic renewal working (check logs after 12 hours)

📚 Additional Resources

• Docker Documentation
• Let's Encrypt Documentation
• Nginx Documentation
• Certbot Documentation

🆘 Support

If you encounter issues:

1. Check logs: docker-compose -f docker-compose.prod.yml logs
2. Verify configuration: docker-compose -f docker-compose.prod.yml config
3. Check service status: docker-compose -f docker-compose.prod.yml ps
4. Review this guide's troubleshooting section