Quellcode durchsuchen

feat: Add complete Docker containerization for development and production

Implement comprehensive Docker setup for Market Data Service with support
for both local development and production deployment.

Features:
- Multi-stage Dockerfile with optimized production builds
- Development setup with live code reload (nodemon)
- Docker Compose configurations for dev and prod environments
- PostgreSQL database container with automatic initialization
- Nginx reverse proxy with WebSocket support for Socket.io
- Automatic database migrations on container startup
- Health checks for all services
- Data persistence via Docker volumes
- SSL/TLS support in production configuration
- Complete documentation in DOCKER.md

Files added:
- Dockerfile: Multi-stage build (dependencies + production)
- docker-compose.yml: Development environment
- docker-compose.prod.yml: Production environment
- nginx/nginx.conf: Development Nginx config (HTTP + WebSocket)
- nginx/nginx.prod.conf: Production Nginx config (HTTPS + SSL)
- docker/entrypoint.sh: API container entrypoint with migrations
- docker/wait-for-db.sh: Database readiness check
- docker/init-db.sh: Database initialization script
- .dockerignore: Docker build exclusions
- DOCKER.md: Complete containerization documentation

Files modified:
- .gitignore: Added Docker-related entries

This enables zero-dependency local development (only Docker Desktop required)
and production-ready containerized deployment with proper security, scaling,
and data persistence.
Hussain Afzal vor 3 Monaten
Ursprung
Commit
7c642d6730
10 geänderte Dateien mit 1202 neuen und 0 gelöschten Zeilen
  1. 22 0
      .gitignore
  2. 537 0
      DOCKER.md
  3. 77 0
      Dockerfile
  4. 97 0
      docker-compose.prod.yml
  5. 90 0
      docker-compose.yml
  6. 45 0
      docker/entrypoint.sh
  7. 27 0
      docker/init-db.sh
  8. 29 0
      docker/wait-for-db.sh
  9. 114 0
      nginx/nginx.conf
  10. 164 0
      nginx/nginx.prod.conf

+ 22 - 0
.gitignore

@@ -97,3 +97,25 @@ temp/
97
 
97
 
98
 # OS generated files
98
 # OS generated files
99
 Thumbs.db
99
 Thumbs.db
100
+
101
+# Docker
102
+.docker/
103
+docker-compose.override.yml
104
+*.dockerignore
105
+
106
+# Docker volumes (if using local volumes)
107
+docker-volumes/
108
+volumes/
109
+
110
+# SSL certificates (should not be committed)
111
+ssl/
112
+*.pem
113
+*.key
114
+*.crt
115
+*.cert
116
+
117
+# Backup files
118
+backups/
119
+*.sql.gz
120
+*.sql
121
+backup_*.sql

+ 537 - 0
DOCKER.md

@@ -0,0 +1,537 @@
1
+# Docker Containerization Guide
2
+
3
+Complete guide for running Market Data Service in Docker containers for both development and production environments.
4
+
5
+## Table of Contents
6
+
7
+- [Prerequisites](#prerequisites)
8
+- [Quick Start](#quick-start)
9
+- [Development Setup](#development-setup)
10
+- [Production Setup](#production-setup)
11
+- [Configuration](#configuration)
12
+- [Common Operations](#common-operations)
13
+- [Troubleshooting](#troubleshooting)
14
+- [Backup and Restore](#backup-and-restore)
15
+
16
+## Prerequisites
17
+
18
+- **Docker Desktop** (Mac/Windows) or **Docker Engine** (Linux)
19
+- **Docker Compose** v2.0+ (included with Docker Desktop)
20
+- **Git** (for cloning repository)
21
+
22
+No other dependencies required! Everything runs in containers.
23
+
24
+## Quick Start
25
+
26
+### Development Environment
27
+
28
+```bash
29
+# 1. Clone the repository
30
+git clone <repository-url>
31
+cd market-data-service
32
+
33
+# 2. Copy environment template
34
+cp .env.example .env
35
+
36
+# 3. Edit .env file with your settings (optional for development)
37
+# Default values work for local development
38
+
39
+# 4. Start all services
40
+docker-compose up -d
41
+
42
+# 5. Check service status
43
+docker-compose ps
44
+
45
+# 6. View logs
46
+docker-compose logs -f api
47
+```
48
+
49
+The API will be available at:
50
+- **HTTP**: http://localhost
51
+- **API Endpoints**: http://localhost/api/*
52
+- **Health Check**: http://localhost/health
53
+- **WebSocket**: ws://localhost/socket.io/
54
+
55
+### Production Environment
56
+
57
+```bash
58
+# 1. Set up environment variables
59
+cp .env.example .env
60
+# Edit .env with production values
61
+
62
+# 2. Set up SSL certificates (if using HTTPS)
63
+# Place certificates in ./ssl/certs/ and ./ssl/private/
64
+# Or update SSL_CERT_PATH and SSL_KEY_PATH in docker-compose.prod.yml
65
+
66
+# 3. Start production services
67
+docker-compose -f docker-compose.prod.yml up -d
68
+
69
+# 4. Check status
70
+docker-compose -f docker-compose.prod.yml ps
71
+```
72
+
73
+## Development Setup
74
+
75
+### Architecture
76
+
77
+The development environment consists of three services:
78
+
79
+1. **API Service** (`api`)
80
+   - Node.js/Express application
81
+   - Live code reload with volume mounts
82
+   - Runs on port 3000 (internal)
83
+   - Accessible via Nginx on port 80
84
+
85
+2. **Database Service** (`db`)
86
+   - PostgreSQL 15
87
+   - Exposed on port 5432 (for local tools)
88
+   - Data persisted in Docker volume
89
+
90
+3. **Nginx Service** (`nginx`)
91
+   - Reverse proxy
92
+   - WebSocket support for Socket.io
93
+   - Exposed on port 80
94
+
95
+### Starting Development Environment
96
+
97
+```bash
98
+# Start all services in detached mode
99
+docker-compose up -d
100
+
101
+# Start with logs visible
102
+docker-compose up
103
+
104
+# Rebuild containers after code changes
105
+docker-compose up -d --build
106
+```
107
+
108
+### Accessing Services
109
+
110
+- **API**: http://localhost
111
+- **Database**: localhost:5432
112
+- **API Direct**: http://localhost:3000 (if port mapping enabled)
113
+
114
+### Development Workflow
115
+
116
+1. **Make code changes** in `src/` directory
117
+2. **Changes are automatically reloaded** (nodemon watches for changes)
118
+3. **View logs**: `docker-compose logs -f api`
119
+4. **Restart service**: `docker-compose restart api`
120
+
121
+### Database Migrations
122
+
123
+Migrations run automatically on container startup. To run manually:
124
+
125
+```bash
126
+# Run migrations
127
+docker-compose exec api npx sequelize-cli db:migrate
128
+
129
+# Rollback last migration
130
+docker-compose exec api npx sequelize-cli db:migrate:undo
131
+
132
+# Check migration status
133
+docker-compose exec api npx sequelize-cli db:migrate:status
134
+```
135
+
136
+### Accessing Database
137
+
138
+```bash
139
+# Connect to PostgreSQL
140
+docker-compose exec db psql -U postgres -d financial_data
141
+
142
+# Run SQL commands
143
+docker-compose exec db psql -U postgres -d financial_data -c "SELECT * FROM symbols LIMIT 10;"
144
+
145
+# List tables
146
+docker-compose exec db psql -U postgres -d financial_data -c "\dt"
147
+```
148
+
149
+## Production Setup
150
+
151
+### Architecture
152
+
153
+Production environment uses optimized settings:
154
+
155
+- **No volume mounts** (code baked into image)
156
+- **Multi-stage builds** for smaller images
157
+- **Health checks** for all services
158
+- **SSL/TLS** support via Nginx
159
+- **Restart policies** for high availability
160
+
161
+### Prerequisites for Production
162
+
163
+1. **Environment Variables**
164
+   ```bash
165
+   # Set strong passwords and secrets
166
+   DB_PASSWORD=<strong_password>
167
+   JWT_SECRET=<strong_random_secret>
168
+   CORS_ORIGIN=https://your-domain.com
169
+   ```
170
+
171
+2. **SSL Certificates** (for HTTPS)
172
+   ```bash
173
+   # Option 1: Use Let's Encrypt (recommended)
174
+   # Certificates will be in /etc/letsencrypt/live/your-domain.com/
175
+   
176
+   # Option 2: Place certificates manually
177
+   mkdir -p ssl/certs ssl/private
178
+   # Copy fullchain.pem to ssl/certs/
179
+   # Copy privkey.pem to ssl/private/
180
+   ```
181
+
182
+3. **Update Nginx Configuration**
183
+   - Edit `nginx/nginx.prod.conf`
184
+   - Update SSL certificate paths
185
+   - Update `server_name` if using domain names
186
+
187
+### Starting Production Environment
188
+
189
+```bash
190
+# Build production images
191
+docker-compose -f docker-compose.prod.yml build
192
+
193
+# Start services
194
+docker-compose -f docker-compose.prod.yml up -d
195
+
196
+# Check health status
197
+docker-compose -f docker-compose.prod.yml ps
198
+```
199
+
200
+### Production Features
201
+
202
+- **Automatic migrations** on startup
203
+- **Health checks** for service monitoring
204
+- **SSL/TLS** encryption
205
+- **Security headers** (HSTS, X-Frame-Options, etc.)
206
+- **Rate limiting** (configurable in nginx.prod.conf)
207
+- **Data persistence** via Docker volumes
208
+
209
+## Configuration
210
+
211
+### Environment Variables
212
+
213
+Key environment variables (see `.env.example` for complete list):
214
+
215
+| Variable | Description | Default |
216
+|----------|-------------|---------|
217
+| `DB_HOST` | Database host | `db` (Docker) or `localhost` |
218
+| `DB_PORT` | Database port | `5432` |
219
+| `DB_NAME` | Database name | `financial_data` |
220
+| `DB_USER` | Database user | `postgres` |
221
+| `DB_PASSWORD` | Database password | **Required** |
222
+| `PORT` | API port | `3000` |
223
+| `NODE_ENV` | Environment | `development` or `production` |
224
+| `CORS_ORIGIN` | Allowed origins | `*` |
225
+| `JWT_SECRET` | JWT secret key | **Required** |
226
+| `LOG_LEVEL` | Logging level | `debug` (dev) or `info` (prod) |
227
+
228
+### Nginx Configuration
229
+
230
+- **Development**: `nginx/nginx.conf` (HTTP only)
231
+- **Production**: `nginx/nginx.prod.conf` (HTTPS with SSL)
232
+
233
+Key features:
234
+- WebSocket proxy for Socket.io
235
+- API endpoint routing
236
+- Health check endpoint
237
+- Security headers (production)
238
+- Rate limiting (production)
239
+
240
+### Database Configuration
241
+
242
+Database is automatically initialized with:
243
+- Database: `financial_data`
244
+- User: `postgres` (or from `DB_USER`)
245
+- Migrations run on API container startup
246
+
247
+## Common Operations
248
+
249
+### Viewing Logs
250
+
251
+```bash
252
+# All services
253
+docker-compose logs -f
254
+
255
+# Specific service
256
+docker-compose logs -f api
257
+docker-compose logs -f db
258
+docker-compose logs -f nginx
259
+
260
+# Last 100 lines
261
+docker-compose logs --tail=100 api
262
+```
263
+
264
+### Restarting Services
265
+
266
+```bash
267
+# Restart all services
268
+docker-compose restart
269
+
270
+# Restart specific service
271
+docker-compose restart api
272
+
273
+# Stop all services
274
+docker-compose down
275
+
276
+# Stop and remove volumes (WARNING: deletes data)
277
+docker-compose down -v
278
+```
279
+
280
+### Accessing Containers
281
+
282
+```bash
283
+# Execute command in API container
284
+docker-compose exec api sh
285
+
286
+# Execute command in database container
287
+docker-compose exec db sh
288
+
289
+# Run Node.js commands
290
+docker-compose exec api node -v
291
+docker-compose exec api npm list
292
+```
293
+
294
+### Updating Application
295
+
296
+```bash
297
+# 1. Pull latest code
298
+git pull
299
+
300
+# 2. Rebuild containers
301
+docker-compose build
302
+
303
+# 3. Restart services (migrations run automatically)
304
+docker-compose up -d
305
+
306
+# Or for production
307
+docker-compose -f docker-compose.prod.yml build
308
+docker-compose -f docker-compose.prod.yml up -d
309
+```
310
+
311
+### Health Checks
312
+
313
+```bash
314
+# Check container health
315
+docker-compose ps
316
+
317
+# Test API health endpoint
318
+curl http://localhost/health
319
+
320
+# Test database connection
321
+docker-compose exec db pg_isready -U postgres
322
+```
323
+
324
+## Troubleshooting
325
+
326
+### Database Connection Issues
327
+
328
+**Problem**: API can't connect to database
329
+
330
+**Solutions**:
331
+```bash
332
+# Check database is running
333
+docker-compose ps db
334
+
335
+# Check database logs
336
+docker-compose logs db
337
+
338
+# Test database connection manually
339
+docker-compose exec db psql -U postgres -d financial_data
340
+
341
+# Verify environment variables
342
+docker-compose exec api env | grep DB_
343
+```
344
+
345
+### Migration Failures
346
+
347
+**Problem**: Migrations fail on startup
348
+
349
+**Solutions**:
350
+```bash
351
+# Check migration logs
352
+docker-compose logs api | grep -i migration
353
+
354
+# Run migrations manually
355
+docker-compose exec api npx sequelize-cli db:migrate
356
+
357
+# Check migration status
358
+docker-compose exec api npx sequelize-cli db:migrate:status
359
+```
360
+
361
+### WebSocket Connection Issues
362
+
363
+**Problem**: WebSocket connections fail through Nginx
364
+
365
+**Solutions**:
366
+1. Verify Nginx configuration includes WebSocket upgrade headers
367
+2. Check Nginx logs: `docker-compose logs nginx`
368
+3. Test direct connection: `ws://localhost:3000/socket.io/`
369
+4. Verify Socket.io transport is set to 'websocket' only
370
+
371
+### Port Conflicts
372
+
373
+**Problem**: Port already in use
374
+
375
+**Solutions**:
376
+```bash
377
+# Check what's using the port
378
+lsof -i :80
379
+lsof -i :5432
380
+
381
+# Change ports in docker-compose.yml
382
+# Update PORT and DB_PORT environment variables
383
+```
384
+
385
+### Container Won't Start
386
+
387
+**Problem**: Container exits immediately
388
+
389
+**Solutions**:
390
+```bash
391
+# Check exit code
392
+docker-compose ps
393
+
394
+# View detailed logs
395
+docker-compose logs api
396
+
397
+# Check container status
398
+docker ps -a
399
+
400
+# Try starting without detached mode
401
+docker-compose up api
402
+```
403
+
404
+### Permission Issues
405
+
406
+**Problem**: Permission denied errors
407
+
408
+**Solutions**:
409
+```bash
410
+# Fix script permissions
411
+chmod +x docker/*.sh
412
+
413
+# Check file ownership
414
+ls -la docker/
415
+
416
+# Rebuild containers
417
+docker-compose build --no-cache
418
+```
419
+
420
+## Backup and Restore
421
+
422
+### Database Backup
423
+
424
+```bash
425
+# Create backup
426
+docker-compose exec db pg_dump -U postgres financial_data > backup_$(date +%Y%m%d_%H%M%S).sql
427
+
428
+# Or using docker directly
429
+docker exec market-data-db pg_dump -U postgres financial_data > backup.sql
430
+```
431
+
432
+### Database Restore
433
+
434
+```bash
435
+# Restore from backup
436
+docker-compose exec -T db psql -U postgres financial_data < backup.sql
437
+
438
+# Or using docker directly
439
+docker exec -i market-data-db psql -U postgres financial_data < backup.sql
440
+```
441
+
442
+### Volume Backup
443
+
444
+```bash
445
+# Backup database volume
446
+docker run --rm -v market_data_db_data:/data -v $(pwd):/backup alpine tar czf /backup/db_backup.tar.gz /data
447
+
448
+# Restore database volume
449
+docker run --rm -v market_data_db_data:/data -v $(pwd):/backup alpine tar xzf /backup/db_backup.tar.gz -C /
450
+```
451
+
452
+### Automated Backups
453
+
454
+Create a cron job or scheduled task:
455
+
456
+```bash
457
+# Daily backup script
458
+#!/bin/bash
459
+BACKUP_DIR="./backups"
460
+DATE=$(date +%Y%m%d_%H%M%S)
461
+mkdir -p $BACKUP_DIR
462
+docker-compose exec -T db pg_dump -U postgres financial_data | gzip > $BACKUP_DIR/backup_$DATE.sql.gz
463
+# Keep only last 30 days
464
+find $BACKUP_DIR -name "backup_*.sql.gz" -mtime +30 -delete
465
+```
466
+
467
+## MT5 EA Integration
468
+
469
+### Development
470
+
471
+Update MT5 EA settings:
472
+- `ApiBaseUrl`: `http://localhost` or `http://localhost:3000`
473
+
474
+### Production
475
+
476
+Update MT5 EA settings:
477
+- `ApiBaseUrl`: `https://your-domain.com`
478
+- Ensure SSL certificate is valid
479
+- WebSocket connections use `wss://` protocol
480
+
481
+## Performance Tuning
482
+
483
+### Database Optimization
484
+
485
+```bash
486
+# Increase shared buffers (edit postgresql.conf in container)
487
+# Or use environment variables in docker-compose.yml
488
+```
489
+
490
+### Nginx Optimization
491
+
492
+- Adjust `worker_processes` in nginx.conf
493
+- Tune `worker_connections` based on expected load
494
+- Enable caching for static assets (if any)
495
+
496
+### Container Resources
497
+
498
+Limit resources in `docker-compose.prod.yml`:
499
+
500
+```yaml
501
+services:
502
+  api:
503
+    deploy:
504
+      resources:
505
+        limits:
506
+          cpus: '2'
507
+          memory: 2G
508
+        reservations:
509
+          cpus: '1'
510
+          memory: 1G
511
+```
512
+
513
+## Security Best Practices
514
+
515
+1. **Change Default Passwords**: Always set strong `DB_PASSWORD` and `JWT_SECRET`
516
+2. **Use HTTPS in Production**: Configure SSL certificates
517
+3. **Limit CORS Origins**: Set specific `CORS_ORIGIN` in production
518
+4. **Regular Updates**: Keep Docker images updated
519
+5. **Backup Regularly**: Automate database backups
520
+6. **Monitor Logs**: Set up log monitoring and alerting
521
+7. **Use Secrets**: Consider Docker secrets for sensitive data
522
+
523
+## Additional Resources
524
+
525
+- [Docker Documentation](https://docs.docker.com/)
526
+- [Docker Compose Documentation](https://docs.docker.com/compose/)
527
+- [PostgreSQL Docker Image](https://hub.docker.com/_/postgres)
528
+- [Nginx Documentation](https://nginx.org/en/docs/)
529
+
530
+## Support
531
+
532
+For issues or questions:
533
+1. Check logs: `docker-compose logs`
534
+2. Review this documentation
535
+3. Check GitHub issues
536
+4. Contact development team
537
+

+ 77 - 0
Dockerfile

@@ -0,0 +1,77 @@
1
+# Multi-stage Dockerfile for Market Data Service API
2
+# Stage 1: Dependencies installation (for development)
3
+FROM node:18-alpine AS dependencies
4
+
5
+WORKDIR /app
6
+
7
+# Copy package files
8
+COPY package*.json ./
9
+
10
+# Install all dependencies (including dev dependencies for build tools)
11
+RUN npm ci
12
+
13
+# Copy application code and scripts (needed for development)
14
+COPY . .
15
+
16
+# Create logs directory
17
+RUN mkdir -p logs
18
+
19
+# Copy entrypoint and wait scripts
20
+COPY docker/entrypoint.sh /usr/local/bin/entrypoint.sh
21
+COPY docker/wait-for-db.sh /usr/local/bin/wait-for-db.sh
22
+RUN chmod +x /usr/local/bin/entrypoint.sh /usr/local/bin/wait-for-db.sh
23
+
24
+# Install PostgreSQL client for wait script (pg_isready)
25
+RUN apk add --no-cache postgresql-client
26
+
27
+# Expose port
28
+EXPOSE 3000
29
+
30
+# Default command (can be overridden in docker-compose)
31
+CMD ["npm", "start"]
32
+
33
+# Stage 2: Production build
34
+FROM node:18-alpine AS production
35
+
36
+WORKDIR /app
37
+
38
+# Copy package files
39
+COPY package*.json ./
40
+
41
+# Install only production dependencies
42
+RUN npm ci --only=production && npm cache clean --force
43
+
44
+# Copy application code
45
+COPY . .
46
+
47
+# Create logs directory
48
+RUN mkdir -p logs
49
+
50
+# Copy entrypoint script
51
+COPY docker/entrypoint.sh /usr/local/bin/entrypoint.sh
52
+COPY docker/wait-for-db.sh /usr/local/bin/wait-for-db.sh
53
+RUN chmod +x /usr/local/bin/entrypoint.sh /usr/local/bin/wait-for-db.sh
54
+
55
+# Install PostgreSQL client for wait script
56
+RUN apk add --no-cache postgresql-client
57
+
58
+# Set non-root user for security
59
+RUN addgroup -g 1001 -S nodejs && \
60
+    adduser -S nodejs -u 1001 && \
61
+    chown -R nodejs:nodejs /app
62
+
63
+USER nodejs
64
+
65
+# Expose port
66
+EXPOSE 3000
67
+
68
+# Health check
69
+HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
70
+  CMD node -e "require('http').get('http://localhost:3000/health', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"
71
+
72
+# Use entrypoint script
73
+ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
74
+
75
+# Default command
76
+CMD ["npm", "start"]
77
+

+ 97 - 0
docker-compose.prod.yml

@@ -0,0 +1,97 @@
1
+version: '3.8'
2
+
3
+services:
4
+  # PostgreSQL Database Service (Production)
5
+  db:
6
+    image: postgres:15-alpine
7
+    container_name: market-data-db-prod
8
+    environment:
9
+      POSTGRES_USER: ${DB_USER:-postgres}
10
+      POSTGRES_PASSWORD: ${DB_PASSWORD}
11
+      POSTGRES_DB: ${DB_NAME:-financial_data}
12
+      PGDATA: /var/lib/postgresql/data/pgdata
13
+    volumes:
14
+      - market_data_db_data_prod:/var/lib/postgresql/data
15
+      - ./docker/init-db.sh:/docker-entrypoint-initdb.d/init-db.sh
16
+    # No exposed ports in production (internal only)
17
+    healthcheck:
18
+      test: ["CMD-SHELL", "pg_isready -U ${DB_USER:-postgres} -d ${DB_NAME:-financial_data}"]
19
+      interval: 10s
20
+      timeout: 5s
21
+      retries: 5
22
+    networks:
23
+      - market-data-network
24
+    restart: unless-stopped
25
+    # Production security: run as non-root
26
+    user: "999:999"
27
+
28
+  # Node.js API Service (Production)
29
+  api:
30
+    build:
31
+      context: .
32
+      dockerfile: Dockerfile
33
+      target: production
34
+    container_name: market-data-api-prod
35
+    environment:
36
+      - NODE_ENV=production
37
+      - PORT=3000
38
+      - DB_HOST=db
39
+      - DB_PORT=5432
40
+      - DB_NAME=${DB_NAME:-financial_data}
41
+      - DB_USER=${DB_USER:-postgres}
42
+      - DB_PASSWORD=${DB_PASSWORD}
43
+      - CORS_ORIGIN=${CORS_ORIGIN:-*}
44
+      - JWT_SECRET=${JWT_SECRET}
45
+      - LOG_LEVEL=${LOG_LEVEL:-info}
46
+    # No volume mounts in production (code is baked into image)
47
+    # Optional: mount logs volume for persistence
48
+    volumes:
49
+      - market_data_logs_prod:/app/logs
50
+    # No exposed ports (accessed only through nginx)
51
+    depends_on:
52
+      db:
53
+        condition: service_healthy
54
+    networks:
55
+      - market-data-network
56
+    restart: unless-stopped
57
+    healthcheck:
58
+      test: ["CMD", "node", "-e", "require('http').get('http://localhost:3000/health', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"]
59
+      interval: 30s
60
+      timeout: 10s
61
+      start_period: 40s
62
+      retries: 3
63
+
64
+  # Nginx Reverse Proxy (Production)
65
+  nginx:
66
+    image: nginx:alpine
67
+    container_name: market-data-nginx-prod
68
+    ports:
69
+      - "80:80"
70
+      - "443:443"
71
+    volumes:
72
+      - ./nginx/nginx.prod.conf:/etc/nginx/nginx.conf:ro
73
+      # SSL certificates (mount your certbot certificates here)
74
+      - ${SSL_CERT_PATH:-./ssl/certs}:/etc/nginx/ssl/certs:ro
75
+      - ${SSL_KEY_PATH:-./ssl/private}:/etc/nginx/ssl/private:ro
76
+    depends_on:
77
+      api:
78
+        condition: service_healthy
79
+    networks:
80
+      - market-data-network
81
+    restart: unless-stopped
82
+    healthcheck:
83
+      test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost/health"]
84
+      interval: 30s
85
+      timeout: 10s
86
+      retries: 3
87
+
88
+volumes:
89
+  market_data_db_data_prod:
90
+    driver: local
91
+  market_data_logs_prod:
92
+    driver: local
93
+
94
+networks:
95
+  market-data-network:
96
+    driver: bridge
97
+

+ 90 - 0
docker-compose.yml

@@ -0,0 +1,90 @@
1
+version: '3.8'
2
+
3
+services:
4
+  # PostgreSQL Database Service
5
+  db:
6
+    image: postgres:15-alpine
7
+    container_name: market-data-db
8
+    environment:
9
+      POSTGRES_USER: ${DB_USER:-postgres}
10
+      POSTGRES_PASSWORD: ${DB_PASSWORD:-postgres}
11
+      POSTGRES_DB: ${DB_NAME:-financial_data}
12
+      PGDATA: /var/lib/postgresql/data/pgdata
13
+    volumes:
14
+      - market_data_db_data:/var/lib/postgresql/data
15
+      - ./docker/init-db.sh:/docker-entrypoint-initdb.d/init-db.sh
16
+    ports:
17
+      - "${DB_PORT:-5432}:5432"
18
+    healthcheck:
19
+      test: ["CMD-SHELL", "pg_isready -U ${DB_USER:-postgres} -d ${DB_NAME:-financial_data}"]
20
+      interval: 10s
21
+      timeout: 5s
22
+      retries: 5
23
+    networks:
24
+      - market-data-network
25
+    restart: unless-stopped
26
+
27
+  # Node.js API Service
28
+  api:
29
+    build:
30
+      context: .
31
+      dockerfile: Dockerfile
32
+      target: dependencies
33
+    container_name: market-data-api
34
+    environment:
35
+      - NODE_ENV=${NODE_ENV:-development}
36
+      - PORT=${PORT:-3000}
37
+      - DB_HOST=db
38
+      - DB_PORT=5432
39
+      - DB_NAME=${DB_NAME:-financial_data}
40
+      - DB_USER=${DB_USER:-postgres}
41
+      - DB_PASSWORD=${DB_PASSWORD:-postgres}
42
+      - CORS_ORIGIN=${CORS_ORIGIN:-*}
43
+      - JWT_SECRET=${JWT_SECRET:-change-me-in-production}
44
+      - LOG_LEVEL=${LOG_LEVEL:-debug}
45
+    volumes:
46
+      # Mount source code for live reload in development
47
+      - ./src:/app/src
48
+      - ./config:/app/config
49
+      - ./migrations:/app/migrations
50
+      - ./models:/app/models
51
+      - ./logs:/app/logs
52
+    ports:
53
+      - "${PORT:-3000}:3000"
54
+    depends_on:
55
+      db:
56
+        condition: service_healthy
57
+    networks:
58
+      - market-data-network
59
+    restart: unless-stopped
60
+    # Override entrypoint for development to use nodemon with live reload
61
+    entrypoint: /bin/sh
62
+    command: -c "/usr/local/bin/wait-for-db.sh && npx sequelize-cli db:migrate && npm install -g nodemon && nodemon src/server.js"
63
+
64
+  # Nginx Reverse Proxy
65
+  nginx:
66
+    image: nginx:alpine
67
+    container_name: market-data-nginx
68
+    ports:
69
+      - "80:80"
70
+    volumes:
71
+      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
72
+    depends_on:
73
+      - api
74
+    networks:
75
+      - market-data-network
76
+    restart: unless-stopped
77
+    healthcheck:
78
+      test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost/health"]
79
+      interval: 30s
80
+      timeout: 10s
81
+      retries: 3
82
+
83
+volumes:
84
+  market_data_db_data:
85
+    driver: local
86
+
87
+networks:
88
+  market-data-network:
89
+    driver: bridge
90
+

+ 45 - 0
docker/entrypoint.sh

@@ -0,0 +1,45 @@
1
+#!/bin/sh
2
+set -e
3
+
4
+echo "Starting Market Data Service API container..."
5
+
6
+# Wait for database to be ready
7
+if [ -f /usr/local/bin/wait-for-db.sh ]; then
8
+    /usr/local/bin/wait-for-db.sh
9
+else
10
+    echo "Waiting for database using pg_isready..."
11
+    host="${DB_HOST:-db}"
12
+    port="${DB_PORT:-5432}"
13
+    user="${DB_USER:-postgres}"
14
+    max_attempts=30
15
+    attempt=0
16
+    
17
+    while [ $attempt -lt $max_attempts ]; do
18
+        if PGPASSWORD="${DB_PASSWORD}" pg_isready -h "$host" -p "$port" -U "$user" >/dev/null 2>&1; then
19
+            echo "Database is ready!"
20
+            break
21
+        fi
22
+        attempt=$((attempt + 1))
23
+        echo "Waiting for database... ($attempt/$max_attempts)"
24
+        sleep 2
25
+    done
26
+    
27
+    if [ $attempt -eq $max_attempts ]; then
28
+        echo "ERROR: Database failed to become ready"
29
+        exit 1
30
+    fi
31
+fi
32
+
33
+# Run database migrations
34
+echo "Running database migrations..."
35
+npx sequelize-cli db:migrate || {
36
+    echo "ERROR: Database migration failed"
37
+    exit 1
38
+}
39
+
40
+echo "Migrations completed successfully"
41
+
42
+# Start the application
43
+echo "Starting Node.js application..."
44
+exec "$@"
45
+

+ 27 - 0
docker/init-db.sh

@@ -0,0 +1,27 @@
1
+#!/bin/bash
2
+set -e
3
+
4
+# Database initialization script for PostgreSQL
5
+# This script creates the database if it doesn't exist
6
+# This runs automatically when PostgreSQL container starts for the first time
7
+
8
+DB_NAME="${POSTGRES_DB:-financial_data}"
9
+DB_USER="${POSTGRES_USER:-postgres}"
10
+
11
+echo "Initializing database ${DB_NAME}..."
12
+
13
+# The database is already created by POSTGRES_DB environment variable
14
+# We just need to ensure it exists and set up permissions
15
+psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
16
+    -- Grant all privileges to the postgres user
17
+    GRANT ALL PRIVILEGES ON DATABASE ${DB_NAME} TO ${DB_USER};
18
+    
19
+    -- Connect to the database and set up schema
20
+    \c ${DB_NAME}
21
+    
22
+    -- Create extensions if needed
23
+    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
24
+EOSQL
25
+
26
+echo "Database ${DB_NAME} initialized successfully."
27
+

+ 29 - 0
docker/wait-for-db.sh

@@ -0,0 +1,29 @@
1
+#!/bin/sh
2
+set -e
3
+
4
+# Wait for PostgreSQL to be ready
5
+# Usage: wait-for-db.sh [host] [port] [user] [database]
6
+
7
+host="${DB_HOST:-db}"
8
+port="${DB_PORT:-5432}"
9
+user="${DB_USER:-postgres}"
10
+database="${DB_NAME:-financial_data}"
11
+max_attempts=30
12
+attempt=0
13
+
14
+echo "Waiting for PostgreSQL to be ready at ${host}:${port}..."
15
+
16
+while [ $attempt -lt $max_attempts ]; do
17
+    if PGPASSWORD="${DB_PASSWORD}" pg_isready -h "$host" -p "$port" -U "$user" -d "$database" >/dev/null 2>&1; then
18
+        echo "PostgreSQL is ready!"
19
+        exit 0
20
+    fi
21
+    
22
+    attempt=$((attempt + 1))
23
+    echo "Attempt $attempt/$max_attempts: PostgreSQL is not ready yet. Waiting..."
24
+    sleep 2
25
+done
26
+
27
+echo "PostgreSQL failed to become ready after $max_attempts attempts"
28
+exit 1
29
+

+ 114 - 0
nginx/nginx.conf

@@ -0,0 +1,114 @@
1
+# Nginx configuration for development environment
2
+# HTTP only, WebSocket support enabled
3
+
4
+user nginx;
5
+worker_processes auto;
6
+error_log /var/log/nginx/error.log warn;
7
+pid /var/run/nginx.pid;
8
+
9
+events {
10
+    worker_connections 1024;
11
+}
12
+
13
+http {
14
+    include /etc/nginx/mime.types;
15
+    default_type application/octet-stream;
16
+
17
+    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
18
+                    '$status $body_size_sent "$http_referer" '
19
+                    '"$http_user_agent" "$http_x_forwarded_for"';
20
+
21
+    access_log /var/log/nginx/access.log main;
22
+
23
+    sendfile on;
24
+    tcp_nopush on;
25
+    tcp_nodelay on;
26
+    keepalive_timeout 65;
27
+    types_hash_max_size 2048;
28
+    client_max_body_size 50M;
29
+
30
+    # Gzip compression
31
+    gzip on;
32
+    gzip_vary on;
33
+    gzip_proxied any;
34
+    gzip_comp_level 6;
35
+    gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss;
36
+
37
+    # Upstream API server
38
+    upstream api {
39
+        server api:3000;
40
+    }
41
+
42
+    server {
43
+        listen 80;
44
+        server_name localhost;
45
+
46
+        # Health check endpoint
47
+        location /health {
48
+            proxy_pass http://api/health;
49
+            proxy_http_version 1.1;
50
+            proxy_set_header Host $host;
51
+            proxy_set_header X-Real-IP $remote_addr;
52
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
53
+            proxy_set_header X-Forwarded-Proto $scheme;
54
+            
55
+            access_log off;
56
+        }
57
+
58
+        # WebSocket support for Socket.io
59
+        location /socket.io/ {
60
+            proxy_pass http://api;
61
+            proxy_http_version 1.1;
62
+            
63
+            # WebSocket upgrade headers
64
+            proxy_set_header Upgrade $http_upgrade;
65
+            proxy_set_header Connection "upgrade";
66
+            
67
+            # Standard proxy headers
68
+            proxy_set_header Host $host;
69
+            proxy_set_header X-Real-IP $remote_addr;
70
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
71
+            proxy_set_header X-Forwarded-Proto $scheme;
72
+            
73
+            # WebSocket timeouts (Socket.io uses long-lived connections)
74
+            proxy_read_timeout 86400s;
75
+            proxy_send_timeout 86400s;
76
+            proxy_connect_timeout 60s;
77
+            
78
+            # Disable buffering for real-time data
79
+            proxy_buffering off;
80
+            proxy_cache off;
81
+        }
82
+
83
+        # API endpoints
84
+        location /api/ {
85
+            proxy_pass http://api;
86
+            proxy_http_version 1.1;
87
+            proxy_set_header Host $host;
88
+            proxy_set_header X-Real-IP $remote_addr;
89
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
90
+            proxy_set_header X-Forwarded-Proto $scheme;
91
+            
92
+            # Timeout settings
93
+            proxy_connect_timeout 60s;
94
+            proxy_send_timeout 60s;
95
+            proxy_read_timeout 60s;
96
+            
97
+            # Buffer settings
98
+            proxy_buffering on;
99
+            proxy_buffer_size 4k;
100
+            proxy_buffers 8 4k;
101
+        }
102
+
103
+        # Root endpoint
104
+        location / {
105
+            proxy_pass http://api;
106
+            proxy_http_version 1.1;
107
+            proxy_set_header Host $host;
108
+            proxy_set_header X-Real-IP $remote_addr;
109
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
110
+            proxy_set_header X-Forwarded-Proto $scheme;
111
+        }
112
+    }
113
+}
114
+

+ 164 - 0
nginx/nginx.prod.conf

@@ -0,0 +1,164 @@
1
+# Nginx configuration for production environment
2
+# HTTP to HTTPS redirect, SSL/TLS, WebSocket support
3
+
4
+user nginx;
5
+worker_processes auto;
6
+error_log /var/log/nginx/error.log warn;
7
+pid /var/run/nginx.pid;
8
+
9
+events {
10
+    worker_connections 2048;
11
+    use epoll;
12
+}
13
+
14
+http {
15
+    include /etc/nginx/mime.types;
16
+    default_type application/octet-stream;
17
+
18
+    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
19
+                    '$status $body_size_sent "$http_referer" '
20
+                    '"$http_user_agent" "$http_x_forwarded_for"';
21
+
22
+    access_log /var/log/nginx/access.log main;
23
+
24
+    sendfile on;
25
+    tcp_nopush on;
26
+    tcp_nodelay on;
27
+    keepalive_timeout 65;
28
+    types_hash_max_size 2048;
29
+    client_max_body_size 50M;
30
+
31
+    # Hide nginx version
32
+    server_tokens off;
33
+
34
+    # Gzip compression
35
+    gzip on;
36
+    gzip_vary on;
37
+    gzip_proxied any;
38
+    gzip_comp_level 6;
39
+    gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss;
40
+
41
+    # Rate limiting (optional, adjust as needed)
42
+    limit_req_zone $binary_remote_addr zone=api_limit:10m rate=100r/m;
43
+    limit_req_zone $binary_remote_addr zone=ws_limit:10m rate=10r/s;
44
+
45
+    # Upstream API server
46
+    upstream api {
47
+        server api:3000;
48
+        keepalive 32;
49
+    }
50
+
51
+    # HTTP server - redirect to HTTPS
52
+    server {
53
+        listen 80;
54
+        server_name _;
55
+
56
+        # Allow Let's Encrypt challenges
57
+        location /.well-known/acme-challenge/ {
58
+            root /var/www/certbot;
59
+        }
60
+
61
+        # Redirect all other traffic to HTTPS
62
+        location / {
63
+            return 301 https://$host$request_uri;
64
+        }
65
+    }
66
+
67
+    # HTTPS server
68
+    server {
69
+        listen 443 ssl http2;
70
+        server_name _;
71
+
72
+        # SSL Configuration
73
+        # Update these paths to match your SSL certificate locations
74
+        ssl_certificate /etc/nginx/ssl/certs/fullchain.pem;
75
+        ssl_certificate_key /etc/nginx/ssl/private/privkey.pem;
76
+
77
+        # SSL Security Settings
78
+        ssl_protocols TLSv1.2 TLSv1.3;
79
+        ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384';
80
+        ssl_prefer_server_ciphers off;
81
+        ssl_session_cache shared:SSL:10m;
82
+        ssl_session_timeout 10m;
83
+        ssl_session_tickets off;
84
+
85
+        # Security Headers
86
+        add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
87
+        add_header X-Frame-Options "DENY" always;
88
+        add_header X-Content-Type-Options "nosniff" always;
89
+        add_header X-XSS-Protection "1; mode=block" always;
90
+        add_header Referrer-Policy "strict-origin-when-cross-origin" always;
91
+
92
+        # Health check endpoint (no rate limiting)
93
+        location /health {
94
+            proxy_pass http://api/health;
95
+            proxy_http_version 1.1;
96
+            proxy_set_header Host $host;
97
+            proxy_set_header X-Real-IP $remote_addr;
98
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
99
+            proxy_set_header X-Forwarded-Proto $scheme;
100
+            
101
+            access_log off;
102
+        }
103
+
104
+        # WebSocket support for Socket.io (with rate limiting)
105
+        location /socket.io/ {
106
+            limit_req zone=ws_limit burst=20 nodelay;
107
+            
108
+            proxy_pass http://api;
109
+            proxy_http_version 1.1;
110
+            
111
+            # WebSocket upgrade headers
112
+            proxy_set_header Upgrade $http_upgrade;
113
+            proxy_set_header Connection "upgrade";
114
+            
115
+            # Standard proxy headers
116
+            proxy_set_header Host $host;
117
+            proxy_set_header X-Real-IP $remote_addr;
118
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
119
+            proxy_set_header X-Forwarded-Proto $scheme;
120
+            
121
+            # WebSocket timeouts (Socket.io uses long-lived connections)
122
+            proxy_read_timeout 86400s;
123
+            proxy_send_timeout 86400s;
124
+            proxy_connect_timeout 60s;
125
+            
126
+            # Disable buffering for real-time data
127
+            proxy_buffering off;
128
+            proxy_cache off;
129
+        }
130
+
131
+        # API endpoints (with rate limiting)
132
+        location /api/ {
133
+            limit_req zone=api_limit burst=50 nodelay;
134
+            
135
+            proxy_pass http://api;
136
+            proxy_http_version 1.1;
137
+            proxy_set_header Host $host;
138
+            proxy_set_header X-Real-IP $remote_addr;
139
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
140
+            proxy_set_header X-Forwarded-Proto $scheme;
141
+            
142
+            # Timeout settings
143
+            proxy_connect_timeout 60s;
144
+            proxy_send_timeout 60s;
145
+            proxy_read_timeout 60s;
146
+            
147
+            # Buffer settings
148
+            proxy_buffering on;
149
+            proxy_buffer_size 4k;
150
+            proxy_buffers 8 4k;
151
+        }
152
+
153
+        # Root endpoint
154
+        location / {
155
+            proxy_pass http://api;
156
+            proxy_http_version 1.1;
157
+            proxy_set_header Host $host;
158
+            proxy_set_header X-Real-IP $remote_addr;
159
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
160
+            proxy_set_header X-Forwarded-Proto $scheme;
161
+        }
162
+    }
163
+}
164
+