feat: Add local development domain support for MT5 EA connectivity

- Update Nginx config to accept market-data.local domain alongside localhost
- Add automated setup script with macOS password prompt support
- Update MT5 EA with local dev URL documentation
- Add comprehensive local development setup guide
- Update README with Docker-based local dev instructions
- Remove redundant documentation files (LOCAL_DEV_URL_ANALYSIS.md, QUICK_START.md)

This enables MT5 Expert Advisors to connect to local development environment
using market-data.local domain, as MT5 doesn't allow localhost/127.0.0.1 URLs.

Setup is now fully automated on macOS (prompts for password) and works
on Linux with sudo fallback.

MT5/Experts/MarketDataSender.mq5

@@ -6,6 +6,9 @@
 
 #include <Trade\SymbolInfo.mqh>
 
+// API Base URL Configuration
+// Production: "http://market-price.insightbull.io"
+// Local Dev:  "http://market-data.local" (requires hosts file setup - see docs/LOCAL_DEV_SETUP.md)
 input string ApiBaseUrl = "http://market-price.insightbull.io";
 input int HistoricalCandleCount = 1000;
 input ENUM_TIMEFRAMES HistoricalTimeframe = PERIOD_H1;

README.md

@@ -360,31 +360,131 @@ curl -X POST https://your-domain.com/api/live-prices/bulk \
 
 ## 💻 LOCAL DEVELOPMENT SETUP
 
-This section explains how to run the project locally on your development machine.
+This section provides a quick overview. For detailed instructions, troubleshooting, and advanced configuration, see **[docs/LOCAL_DEV_SETUP.md](./docs/LOCAL_DEV_SETUP.md)**.
 
-### Prerequisites for Local Development
-- **Node.js**: v18.x LTS or higher
-- **PostgreSQL**: v13.x or higher
-- **Git**: Latest version
+### Prerequisites
+- **Docker** and **Docker Compose** installed
+- **Git** installed
+- **Administrator access** (script will prompt for password automatically on macOS)
 
-### Step-by-Step Local Setup
+### Quick Start
 
 #### 1. Clone the Repository
 ```bash
 # Clone from GitHub
 git clone https://git.mqldevelopment.com/muhammad.uzair/market-data-service.git
 cd market-data-service
+```
 
-# Install dependencies
+#### 2. Setup Local Domain (One-time, Required for MT5 EA)
+```bash
+# Run automated setup script (fully automated on macOS - will prompt for password)
+# On Linux, you may need to run with: sudo bash scripts/setup-local-dev.sh
+bash scripts/setup-local-dev.sh
+```
+
+**Note:** On macOS, the script automatically prompts for your administrator password. On Linux, you may need to run with `sudo`.
+
+**Alternative (Manual):**
+```bash
+# Add market-data.local to hosts file manually
+echo "127.0.0.1 market-data.local" | sudo tee -a /etc/hosts
+```
+
+#### 3. Start Docker Services
+```bash
+# Start all services (database, API, Nginx)
+docker-compose up -d
+
+# Verify services are running
+docker-compose ps
+```
+
+#### 4. Verify Setup
+```bash
+# Test health endpoint
+curl http://market-data.local/health
+
+# Test API endpoints
+curl http://market-data.local/api/symbols
+curl http://market-data.local/api/health
+```
+
+**✅ Setup Complete!** Your API is now accessible at `http://market-data.local`
+
+### Why market-data.local?
+
+MT5 Expert Advisors require valid domain URLs and **do not allow** `localhost` or `127.0.0.1`. The `market-data.local` domain resolves to your local Docker setup, enabling MT5 EA connectivity.
+
+### MT5 EA Configuration
+
+1. **Add URL to MT5 Allowed List:**
+   - Open MT5 → Tools → Options → Expert Advisors
+   - Click "Allowed URLs" tab → Add → Enter: `http://market-data.local`
+
+2. **Configure EA:**
+   - Attach `MarketDataSender.mq5` to any chart
+   - Set `ApiBaseUrl = "http://market-data.local"`
+   - Other settings as needed
+
+3. **Verify Connection:**
+   - Check MT5 Experts tab for EA logs
+   - Look for: `✅ Symbols initialized`
+
+### Development Workflow
+
+**View Logs:**
+```bash
+# All services
+docker-compose logs -f
+
+# Specific service
+docker-compose logs -f api
+docker-compose logs -f nginx
+```
+
+**Code Changes:**
+- Code in `src/` is automatically reloaded (no container restart needed)
+- View API logs to see changes: `docker-compose logs -f api`
+
+**Stop Services:**
+```bash
+docker-compose down
+```
+
+**Restart Services:**
+```bash
+docker-compose restart
+```
+
+### Need More Help?
+
+📖 **For comprehensive setup instructions, troubleshooting, and advanced configuration, see:**
+- **[docs/LOCAL_DEV_SETUP.md](./docs/LOCAL_DEV_SETUP.md)** - Complete local development guide with detailed steps, troubleshooting, and MT5 EA configuration
+
+### Alternative: Non-Docker Local Setup
+
+If you prefer to run without Docker (not recommended for MT5 EA):
+
+<details>
+<summary>Click to expand non-Docker setup instructions</summary>
+
+#### Prerequisites
+- **Node.js**: v18.x LTS or higher
+- **PostgreSQL**: v13.x or higher
+
+#### Setup Steps
+
+1. **Install Dependencies:**
+```bash
 npm install
 ```
 
-#### 2. Set Up PostgreSQL Database
+2. **Set Up Database:**
 ```bash
-# Create local database
 createdb financial_data
 
-# Set up environment variables
+# Create .env file
 cat > .env << EOF
 DB_TYPE=postgres
 DB_HOST=localhost
@@ -392,55 +492,32 @@ DB_PORT=5432
 DB_NAME=financial_data
 DB_USER=your_local_username
 DB_PASSWORD=your_local_password
-
 PORT=3001
 NODE_ENV=development
-
 JWT_SECRET=your_development_jwt_secret_key
 CORS_ORIGIN=http://localhost:3000
-
 LOG_LEVEL=debug
 EOF
 ```
 
-#### 3. Run Database Migrations
+3. **Run Migrations:**
 ```bash
-# Install Sequelize CLI (if not already installed)
-npm install -g sequelize-cli
-
-# Run migrations to create tables
 npx sequelize-cli db:migrate
-
-# Verify tables were created
-psql -d financial_data -c "\dt"
 ```
 
-#### 4. Start Development Server
+4. **Start Server:**
 ```bash
-# Option 1: Using npm script (recommended)
 npm run dev
-
-# Option 2: Using nodemon directly
-npx nodemon src/server.js
-
-# Option 3: Using node directly
-node src/server.js
 ```
 
-#### 5. Verify Local Setup
+5. **Test:**
 ```bash
-# Test health endpoint
 curl http://localhost:3001/health
+```
 
-# Test API endpoints
-curl http://localhost:3001/api/symbols
-curl http://localhost:3001/api/live-prices
+**Note:** This setup won't work with MT5 EA (requires domain URL). Use Docker setup for MT5 integration.
 
-# Test bulk endpoint
-curl -X POST http://localhost:3001/api/live-prices/bulk \
-  -H "Content-Type: application/json" \
-  -d '{"prices": [{"symbolId": 1, "price": 123.45}]}'
-```
+</details>
 
 
 ## 🔄 UPDATING EXISTING DEPLOYMENT

docker-compose.yml

@@ -62,6 +62,8 @@ services:
     command: -c "/usr/local/bin/wait-for-db.sh && node docker/sync-models.js && (npx sequelize-cli db:migrate || echo 'No migrations to run or migration skipped') && npm install -g nodemon && nodemon src/server.js"
 
   # Nginx Reverse Proxy
+  # For local development with MT5 EA, use: http://market-data.local
+  # Run: sudo bash scripts/setup-local-dev.sh (one-time setup)
   nginx:
     image: nginx:alpine
     container_name: market-data-nginx

docs/LOCAL_DEV_SETUP.md

@@ -0,0 +1,462 @@
+# Local Development Setup Guide
+
+This guide will help you set up the Market Data Service for local development with MT5 EA connectivity.
+
+## Quick Start (5 minutes)
+
+### Prerequisites
+- Docker and Docker Compose installed
+- Git installed
+- Administrator access (script will prompt for password automatically on macOS)
+
+### Step 1: Clone and Setup Hosts File
+
+```bash
+# Clone the repository
+git clone https://git.mqldevelopment.com/muhammad.uzair/market-data-service.git
+cd market-data-service
+
+# Run the automated setup script (fully automated - will prompt for password on macOS)
+# On Linux, you may need: sudo bash scripts/setup-local-dev.sh
+bash scripts/setup-local-dev.sh
+```
+
+**Note:** On macOS, the script automatically prompts for your administrator password using a system dialog. On Linux, you may need to run with `sudo`.
+
+**Alternative (Manual Setup):**
+If you prefer to do it manually:
+```bash
+# Add market-data.local to your hosts file
+echo "127.0.0.1 market-data.local" | sudo tee -a /etc/hosts
+```
+
+### Step 2: Start Docker Services
+
+```bash
+# Start all services (database, API, Nginx)
+docker-compose up -d
+
+# Verify services are running
+docker-compose ps
+```
+
+### Step 3: Verify Setup
+
+```bash
+# Test health endpoint
+curl http://market-data.local/health
+
+# Test API endpoint
+curl http://market-data.local/api/symbols
+```
+
+You should see JSON responses. If you do, your setup is complete! ✅
+
+---
+
+## Why market-data.local?
+
+MT5 Expert Advisors require valid domain URLs in their "Allowed URLs" list. MT5 **does not allow**:
+- ❌ `localhost`
+- ❌ `127.0.0.1`
+- ❌ `http://localhost:80`
+
+MT5 **requires** a domain-like URL:
+- ✅ `http://market-data.local`
+
+The `market-data.local` domain resolves to your local Docker setup via the `/etc/hosts` file, allowing MT5 EA to connect to your local development server.
+
+---
+
+## Complete Setup Instructions
+
+### 1. Hosts File Configuration
+
+The setup script (`bash scripts/setup-local-dev.sh`) automatically adds the required entry. On macOS, it will prompt for your administrator password using a system dialog. On Linux, you may need to run it with `sudo`.
+
+**Verify the entry:**
+```bash
+# View current entry
+cat /etc/hosts | grep market-data.local
+```
+
+**Manual setup (if needed):**
+```bash
+# Add entry manually (macOS/Linux)
+echo "127.0.0.1 market-data.local" | sudo tee -a /etc/hosts
+```
+
+**Windows:**
+1. Open Notepad as Administrator
+2. Open `C:\Windows\System32\drivers\etc\hosts`
+3. Add line: `127.0.0.1 market-data.local`
+4. Save and close
+
+### 2. Docker Services
+
+**Start Services:**
+```bash
+docker-compose up -d
+```
+
+**View Logs:**
+```bash
+# All services
+docker-compose logs -f
+
+# Specific service
+docker-compose logs -f api
+docker-compose logs -f nginx
+docker-compose logs -f db
+```
+
+**Stop Services:**
+```bash
+docker-compose down
+```
+
+**Restart Services:**
+```bash
+docker-compose restart
+```
+
+### 3. Verify Services
+
+**Check Service Status:**
+```bash
+docker-compose ps
+```
+
+All services should show "Up" status.
+
+**Test Endpoints:**
+```bash
+# Health check
+curl http://market-data.local/health
+
+# API endpoints
+curl http://market-data.local/api/symbols
+curl http://market-data.local/api/health
+
+# Test with browser
+open http://market-data.local/health  # macOS
+# or visit http://market-data.local/health in your browser
+```
+
+---
+
+## MT5 EA Configuration
+
+### Step 1: Add URL to MT5 Allowed List
+
+1. Open MT5
+2. Go to: **Tools → Options → Expert Advisors**
+3. Click the **"Allowed URLs"** tab
+4. Click **"Add"**
+5. Enter: `http://market-data.local`
+6. Click **"OK"**
+
+### Step 2: Configure EA
+
+1. Attach `MarketDataSender.mq5` to any chart
+2. In EA Inputs, set:
+   - **ApiBaseUrl**: `http://market-data.local`
+   - **HistoricalCandleCount**: `1000` (or your preference)
+   - **LivePriceIntervalSeconds**: `5` (or your preference)
+3. Click **"OK"**
+
+### Step 3: Verify Connection
+
+1. Check the **Experts** tab in MT5 Terminal
+2. Look for EA logs:
+   - ✅ `✅ Symbols initialized: X`
+   - ✅ `✅ Successfully sent live prices to API`
+   - ❌ If you see errors, check troubleshooting section below
+
+---
+
+## Testing the Setup
+
+### Test 1: Basic Connectivity
+
+```bash
+# Health check
+curl http://market-data.local/health
+
+# Expected response:
+# {"success":true,"message":"Market Data Service is running",...}
+```
+
+### Test 2: API Endpoints
+
+```bash
+# Get symbols
+curl http://market-data.local/api/symbols
+
+# Get health
+curl http://market-data.local/api/health
+```
+
+### Test 3: MT5 EA Connection
+
+1. Start MT5 EA with `ApiBaseUrl = "http://market-data.local"`
+2. Check MT5 Experts tab for logs
+3. Verify symbols are being synced
+4. Check that live prices are being sent
+
+### Test 4: Database Verification
+
+```bash
+# Connect to database
+docker-compose exec db psql -U postgres -d financial_data
+
+# Check tables
+\dt
+
+# Check symbols
+SELECT * FROM symbols LIMIT 5;
+
+# Check live prices
+SELECT * FROM live_prices LIMIT 5;
+
+# Exit
+\q
+```
+
+---
+
+## Troubleshooting
+
+### Issue: "curl: Could not resolve host: market-data.local"
+
+**Solution:**
+```bash
+# Verify hosts entry exists
+cat /etc/hosts | grep market-data.local
+
+# If missing, add it
+echo "127.0.0.1 market-data.local" | sudo tee -a /etc/hosts
+
+# Flush DNS cache (macOS)
+sudo dscacheutil -flushcache
+
+# Flush DNS cache (Linux)
+sudo systemd-resolve --flush-caches
+```
+
+### Issue: "502 Bad Gateway" or "Connection refused"
+
+**Solution:**
+```bash
+# Check if services are running
+docker-compose ps
+
+# If not running, start them
+docker-compose up -d
+
+# Check service logs
+docker-compose logs api
+docker-compose logs nginx
+
+# Restart services
+docker-compose restart
+```
+
+### Issue: MT5 EA shows "WebRequest connection error"
+
+**Solution:**
+1. Verify URL is in MT5 Allowed URLs list
+2. Check URL format: `http://market-data.local` (no trailing slash)
+3. Test URL manually: `curl http://market-data.local/health`
+4. Verify Docker services are running: `docker-compose ps`
+5. Check Nginx logs: `docker-compose logs nginx`
+
+### Issue: "Failed to fetch symbols from API: HTTP 502"
+
+**Solution:**
+```bash
+# Check API service
+docker-compose logs api
+
+# Check database connection
+docker-compose exec db pg_isready -U postgres
+
+# Restart API service
+docker-compose restart api
+```
+
+### Issue: Database connection errors
+
+**Solution:**
+```bash
+# Check database is running
+docker-compose ps db
+
+# Check database logs
+docker-compose logs db
+
+# Restart database
+docker-compose restart db
+
+# Wait for database to be ready
+docker-compose exec db pg_isready -U postgres
+```
+
+### Issue: Port 80 already in use
+
+**Solution:**
+```bash
+# Check what's using port 80
+sudo lsof -i :80
+
+# Stop conflicting service or change Nginx port in docker-compose.yml
+# Edit docker-compose.yml, change "80:80" to "8080:80"
+# Then use: http://market-data.local:8080
+```
+
+---
+
+## Development Workflow
+
+### Making Code Changes
+
+The API service uses volume mounts for live code reloading:
+
+```bash
+# Code changes in src/ are automatically reloaded
+# No need to restart containers
+
+# View API logs to see changes
+docker-compose logs -f api
+```
+
+### Database Migrations
+
+Migrations run automatically on container start. To run manually:
+
+```bash
+# Run migrations
+docker-compose exec api npx sequelize-cli db:migrate
+
+# Check migration status
+docker-compose exec api npx sequelize-cli db:migrate:status
+```
+
+### Viewing Logs
+
+```bash
+# All services
+docker-compose logs -f
+
+# Specific service
+docker-compose logs -f api
+docker-compose logs -f nginx
+docker-compose logs -f db
+
+# Last 100 lines
+docker-compose logs --tail=100 api
+```
+
+---
+
+## Switching Between Local and Production
+
+### Local Development
+- **URL**: `http://market-data.local`
+- **MT5 EA**: `ApiBaseUrl = "http://market-data.local"`
+
+### Production
+- **URL**: `http://market-price.insightbull.io`
+- **MT5 EA**: `ApiBaseUrl = "http://market-price.insightbull.io"`
+
+You can easily switch by changing the `ApiBaseUrl` input in MT5 EA settings.
+
+---
+
+## Cleanup
+
+### Remove Hosts Entry (if needed)
+
+**macOS/Linux:**
+```bash
+# Remove the entry
+sudo sed -i '' '/market-data.local/d' /etc/hosts  # macOS
+sudo sed -i '/market-data.local/d' /etc/hosts     # Linux
+```
+
+**Windows:**
+1. Open Notepad as Administrator
+2. Open `C:\Windows\System32\drivers\etc\hosts`
+3. Remove line: `127.0.0.1 market-data.local`
+4. Save and close
+
+### Stop Docker Services
+
+```bash
+# Stop and remove containers
+docker-compose down
+
+# Stop and remove containers + volumes (⚠️ deletes database data)
+docker-compose down -v
+```
+
+---
+
+## Additional Resources
+
+- **Main README**: See [README.md](../README.md) for general project information
+- **Docker Guide**: See [DOCKER.md](../DOCKER.md) for Docker-specific details
+- **API Documentation**: See [docs/API_CONTRACT.md](./API_CONTRACT.md)
+- **MT5 Operation**: See [docs/MT5_OPERATION.md](./MT5_OPERATION.md)
+
+---
+
+## Quick Reference
+
+### Essential Commands
+
+```bash
+# Setup (one-time, fully automated on macOS)
+bash scripts/setup-local-dev.sh
+# On Linux, you may need: sudo bash scripts/setup-local-dev.sh
+
+# Start services
+docker-compose up -d
+
+# Stop services
+docker-compose down
+
+# View logs
+docker-compose logs -f api
+
+# Test endpoint
+curl http://market-data.local/health
+
+# Check services
+docker-compose ps
+```
+
+### URLs
+
+- **Local API**: `http://market-data.local`
+- **Health Check**: `http://market-data.local/health`
+- **API Endpoints**: `http://market-data.local/api/*`
+
+### MT5 Configuration
+
+- **Allowed URL**: `http://market-data.local`
+- **EA ApiBaseUrl**: `http://market-data.local`
+
+---
+
+## Need Help?
+
+If you encounter issues not covered in this guide:
+
+1. Check the troubleshooting section above
+2. Review service logs: `docker-compose logs`
+3. Verify hosts entry: `cat /etc/hosts | grep market-data.local`
+4. Test connectivity: `curl http://market-data.local/health`
+
+For additional support, check the main [README.md](../README.md) or project documentation.
+

nginx/nginx.conf

@@ -41,7 +41,9 @@ http {
 
     server {
         listen 80;
-        server_name localhost;
+        # Accept both localhost and market-data.local for local development
+        # market-data.local is required for MT5 EA connectivity (MT5 doesn't allow localhost/127.0.0.1)
+        server_name localhost market-data.local;
 
         # Health check endpoint
         location /health {

scripts/setup-local-dev.sh

@@ -0,0 +1,90 @@
+#!/bin/bash
+# Setup script for local development with market-data.local domain
+# This script automates the hosts file entry required for MT5 EA connectivity
+
+set -e
+
+DOMAIN="market-data.local"
+HOSTS_ENTRY="127.0.0.1 ${DOMAIN}"
+HOSTS_FILE="/etc/hosts"
+
+echo "🚀 Setting up local development environment for Market Data Service"
+echo ""
+
+# Check if running on macOS
+if [[ "$OSTYPE" == "darwin"* ]]; then
+    OS="macOS"
+elif [[ "$OSTYPE" == "linux-gnu"* ]]; then
+    OS="Linux"
+else
+    echo "⚠️  Warning: Unsupported OS. Please manually add to hosts file:"
+    echo "   ${HOSTS_ENTRY}"
+    exit 1
+fi
+
+# Check if entry already exists
+if grep -q "${DOMAIN}" "${HOSTS_FILE}" 2>/dev/null; then
+    echo "✅ Hosts entry for ${DOMAIN} already exists"
+    echo ""
+    echo "Current entry:"
+    grep "${DOMAIN}" "${HOSTS_FILE}" | head -1
+    echo ""
+    echo "✅ Setup complete! You can now use http://${DOMAIN}"
+    exit 0
+fi
+
+# Check if running as root (required for hosts file modification)
+if [ "$EUID" -ne 0 ]; then
+    echo "📝 Adding ${DOMAIN} to hosts file requires administrator privileges"
+    echo ""
+    
+    # Try to use osascript on macOS for interactive password prompt
+    if [[ "$OSTYPE" == "darwin"* ]] && command -v osascript >/dev/null 2>&1; then
+        echo "   Prompting for administrator password..."
+        echo ""
+        
+        # Use osascript to run the command with admin privileges
+        osascript -e "do shell script \"echo '${HOSTS_ENTRY}' >> ${HOSTS_FILE}\" with administrator privileges" 2>&1
+        
+        if [ $? -ne 0 ]; then
+            echo ""
+            echo "❌ Failed to add entry. Please run manually:"
+            echo "   sudo bash scripts/setup-local-dev.sh"
+            echo ""
+            echo "Or manually add this line to ${HOSTS_FILE}:"
+            echo "   ${HOSTS_ENTRY}"
+            exit 1
+        fi
+    else
+        # Not macOS or osascript not available, require manual sudo
+        echo "Please run this script with sudo:"
+        echo "   sudo bash scripts/setup-local-dev.sh"
+        echo ""
+        echo "Or manually add this line to ${HOSTS_FILE}:"
+        echo "   ${HOSTS_ENTRY}"
+        exit 1
+    fi
+else
+    # Already running as root, just add the entry
+    echo "📝 Adding ${DOMAIN} to ${HOSTS_FILE}..."
+    echo "${HOSTS_ENTRY}" >> "${HOSTS_FILE}"
+fi
+
+# Verify the entry was added
+if grep -q "${DOMAIN}" "${HOSTS_FILE}"; then
+    echo "✅ Successfully added ${DOMAIN} to hosts file"
+    echo ""
+    echo "Added entry:"
+    grep "${DOMAIN}" "${HOSTS_FILE}" | tail -1
+    echo ""
+    echo "✅ Setup complete!"
+    echo ""
+    echo "Next steps:"
+    echo "  1. Start Docker services: docker-compose up -d"
+    echo "  2. Test the domain: curl http://${DOMAIN}/health"
+    echo "  3. Configure MT5 EA with: ApiBaseUrl = \"http://${DOMAIN}\""
+else
+    echo "❌ Failed to add entry to hosts file"
+    exit 1
+fi
+