market-data-service/docs/LOCAL_DEV_SETUP.md

Local Development Setup Guide

This guide will help you set up the Market Data Service for local development with MT5 EA connectivity.

Quick Start (5 minutes)

Prerequisites

• Docker and Docker Compose installed
• Git installed
• Administrator access (script will prompt for password automatically on macOS)

Step 1: Clone and Setup Hosts File

# Clone the repository
git clone https://git.mqldevelopment.com/muhammad.uzair/market-data-service.git
cd market-data-service

# Run the automated setup script (fully automated - will prompt for password on macOS)
# On Linux, you may need: sudo bash scripts/setup-local-dev.sh
bash scripts/setup-local-dev.sh

Note: On macOS, the script automatically prompts for your administrator password using a system dialog. On Linux, you may need to run with sudo.

Alternative (Manual Setup): If you prefer to do it manually:

# Add market-data.local to your hosts file
echo "127.0.0.1 market-data.local" | sudo tee -a /etc/hosts

Step 2: Start Docker Services

# Start all services (database, API, Nginx)
docker-compose up -d

# Verify services are running
docker-compose ps

Step 3: Verify Setup

# Test health endpoint
curl http://market-data.local/health

# Test API endpoint
curl http://market-data.local/api/symbols

You should see JSON responses. If you do, your setup is complete! ✅

---

Why market-data.local?

MT5 Expert Advisors require valid domain URLs in their "Allowed URLs" list. MT5 does not allow:

• ❌ localhost
• ❌ 127.0.0.1
• ❌ http://localhost:80

MT5 requires a domain-like URL:

• ✅ http://market-data.local

The market-data.local domain resolves to your local Docker setup via the /etc/hosts file, allowing MT5 EA to connect to your local development server.

---

Complete Setup Instructions

1. Hosts File Configuration

The setup script (bash scripts/setup-local-dev.sh) automatically adds the required entry. On macOS, it will prompt for your administrator password using a system dialog. On Linux, you may need to run it with sudo.

Verify the entry:

# View current entry
cat /etc/hosts | grep market-data.local

Manual setup (if needed):

# Add entry manually (macOS/Linux)
echo "127.0.0.1 market-data.local" | sudo tee -a /etc/hosts

Windows:

1. Open Notepad as Administrator
2. Open C:\Windows\System32\drivers\etc\hosts
3. Add line: 127.0.0.1 market-data.local
4. Save and close

2. Docker Services

Start Services:

docker-compose up -d

View Logs:

# All services
docker-compose logs -f

# Specific service
docker-compose logs -f api
docker-compose logs -f nginx
docker-compose logs -f db

Stop Services:

docker-compose down

Restart Services:

docker-compose restart

3. Verify Services

Check Service Status:

docker-compose ps

All services should show "Up" status.

Test Endpoints:

# Health check
curl http://market-data.local/health

# API endpoints
curl http://market-data.local/api/symbols
curl http://market-data.local/api/health

# Test with browser
open http://market-data.local/health  # macOS
# or visit http://market-data.local/health in your browser

---

MT5 EA Configuration

Step 1: Add URL to MT5 Allowed List

1. Open MT5
2. Go to: Tools → Options → Expert Advisors
3. Click the "Allowed URLs" tab
4. Click "Add"
5. Enter: http://market-data.local
6. Click "OK"

Step 2: Configure EA

1. Attach MarketDataSender.mq5 to any chart
2. In EA Inputs, set:
   • ApiBaseUrl: http://market-data.local
   • HistoricalCandleCount: 1000 (or your preference)
   • LivePriceIntervalSeconds: 5 (or your preference)
3. Click "OK"

Step 3: Verify Connection

1. Check the Experts tab in MT5 Terminal
2. Look for EA logs:
   • ✅ ✅ Symbols initialized: X
   • ✅ ✅ Successfully sent live prices to API
   • ❌ If you see errors, check troubleshooting section below

---

Testing the Setup

Test 1: Basic Connectivity

# Health check
curl http://market-data.local/health

# Expected response:
# {"success":true,"message":"Market Data Service is running",...}

Test 2: API Endpoints

# Get symbols
curl http://market-data.local/api/symbols

# Get health
curl http://market-data.local/api/health

Test 3: MT5 EA Connection

1. Start MT5 EA with ApiBaseUrl = "http://market-data.local"
2. Check MT5 Experts tab for logs
3. Verify symbols are being synced
4. Check that live prices are being sent

Test 4: Database Verification

# Connect to database
docker-compose exec db psql -U postgres -d financial_data

# Check tables
\dt

# Check symbols
SELECT * FROM symbols LIMIT 5;

# Check live prices
SELECT * FROM live_prices LIMIT 5;

# Exit
\q

---

Troubleshooting

Issue: "curl: Could not resolve host: market-data.local"

Solution:

# Verify hosts entry exists
cat /etc/hosts | grep market-data.local

# If missing, add it
echo "127.0.0.1 market-data.local" | sudo tee -a /etc/hosts

# Flush DNS cache (macOS)
sudo dscacheutil -flushcache

# Flush DNS cache (Linux)
sudo systemd-resolve --flush-caches

Issue: "502 Bad Gateway" or "Connection refused"

Solution:

# Check if services are running
docker-compose ps

# If not running, start them
docker-compose up -d

# Check service logs
docker-compose logs api
docker-compose logs nginx

# Restart services
docker-compose restart

Issue: MT5 EA shows "WebRequest connection error"

Solution:

1. Verify URL is in MT5 Allowed URLs list
2. Check URL format: http://market-data.local (no trailing slash)
3. Test URL manually: curl http://market-data.local/health
4. Verify Docker services are running: docker-compose ps
5. Check Nginx logs: docker-compose logs nginx

Issue: "Failed to fetch symbols from API: HTTP 502"

Solution:

# Check API service
docker-compose logs api

# Check database connection
docker-compose exec db pg_isready -U postgres

# Restart API service
docker-compose restart api

Issue: Database connection errors

Solution:

# Check database is running
docker-compose ps db

# Check database logs
docker-compose logs db

# Restart database
docker-compose restart db

# Wait for database to be ready
docker-compose exec db pg_isready -U postgres

Issue: Port 80 already in use

Solution:

# Check what's using port 80
sudo lsof -i :80

# Stop conflicting service or change Nginx port in docker-compose.yml
# Edit docker-compose.yml, change "80:80" to "8080:80"
# Then use: http://market-data.local:8080

---

Development Workflow

Making Code Changes

The API service uses volume mounts for live code reloading:

# Code changes in src/ are automatically reloaded
# No need to restart containers

# View API logs to see changes
docker-compose logs -f api

Database Migrations

Migrations run automatically on container start. To run manually:

# Run migrations
docker-compose exec api npx sequelize-cli db:migrate

# Check migration status
docker-compose exec api npx sequelize-cli db:migrate:status

Viewing Logs

# All services
docker-compose logs -f

# Specific service
docker-compose logs -f api
docker-compose logs -f nginx
docker-compose logs -f db

# Last 100 lines
docker-compose logs --tail=100 api

---

Switching Between Local and Production

Local Development

• URL: http://market-data.local
• MT5 EA: ApiBaseUrl = "http://market-data.local"

Production

• URL: http://market-price.insightbull.io
• MT5 EA: ApiBaseUrl = "http://market-price.insightbull.io"

You can easily switch by changing the ApiBaseUrl input in MT5 EA settings.

---

Cleanup

Remove Hosts Entry (if needed)

macOS/Linux:

# Remove the entry
sudo sed -i '' '/market-data.local/d' /etc/hosts  # macOS
sudo sed -i '/market-data.local/d' /etc/hosts     # Linux

Windows:

1. Open Notepad as Administrator
2. Open C:\Windows\System32\drivers\etc\hosts
3. Remove line: 127.0.0.1 market-data.local
4. Save and close

Stop Docker Services

# Stop and remove containers
docker-compose down

# Stop and remove containers + volumes (⚠️ deletes database data)
docker-compose down -v

---

Additional Resources

• Main README: See README.md for general project information
• Docker Guide: See DOCKER.md for Docker-specific details
• API Documentation: See docs/API_CONTRACT.md
• MT5 Operation: See docs/MT5_OPERATION.md

---

Quick Reference

Essential Commands

# Setup (one-time, fully automated on macOS)
bash scripts/setup-local-dev.sh
# On Linux, you may need: sudo bash scripts/setup-local-dev.sh

# Start services
docker-compose up -d

# Stop services
docker-compose down

# View logs
docker-compose logs -f api

# Test endpoint
curl http://market-data.local/health

# Check services
docker-compose ps

URLs

• Local API: http://market-data.local
• Health Check: http://market-data.local/health
• API Endpoints: http://market-data.local/api/*

MT5 Configuration

• Allowed URL: http://market-data.local
• EA ApiBaseUrl: http://market-data.local

---

Need Help?

If you encounter issues not covered in this guide:

1. Check the troubleshooting section above
2. Review service logs: docker-compose logs
3. Verify hosts entry: cat /etc/hosts | grep market-data.local
4. Test connectivity: curl http://market-data.local/health

For additional support, check the main README.md or project documentation.