# Local Development Setup Guide

This guide will help you set up the Market Data Service for local development with MT5 EA connectivity.

## Quick Start (5 minutes)

### Prerequisites
- Docker and Docker Compose installed
- Git installed
- Administrator access (script will prompt for password automatically on macOS)

### Step 1: Clone and Setup Hosts File

```bash
# Clone the repository
git clone https://git.mqldevelopment.com/muhammad.uzair/market-data-service.git
cd market-data-service

# Run the automated setup script (fully automated - will prompt for password on macOS)
# On Linux, you may need: sudo bash scripts/setup-local-dev.sh
bash scripts/setup-local-dev.sh
```

**Note:** On macOS, the script automatically prompts for your administrator password using a system dialog. On Linux, you may need to run with `sudo`.

**Alternative (Manual Setup):**
If you prefer to do it manually:
```bash
# Add market-data.local to your hosts file
echo "127.0.0.1 market-data.local" | sudo tee -a /etc/hosts
```

### Step 2: Start Docker Services

```bash
# Start all services (database, API, Nginx)
docker-compose up -d

# Verify services are running
docker-compose ps
```

### Step 3: Verify Setup

```bash
# Test health endpoint
curl http://market-data.local/health

# Test API endpoint
curl http://market-data.local/api/symbols
```

You should see JSON responses. If you do, your setup is complete! ✅

---

## Why market-data.local?

MT5 Expert Advisors require valid domain URLs in their "Allowed URLs" list. MT5 **does not allow**:
- ❌ `localhost`
- ❌ `127.0.0.1`
- ❌ `http://localhost:80`

MT5 **requires** a domain-like URL:
- ✅ `http://market-data.local`

The `market-data.local` domain resolves to your local Docker setup via the `/etc/hosts` file, allowing MT5 EA to connect to your local development server.

---

## Complete Setup Instructions

### 1. Hosts File Configuration

The setup script (`bash scripts/setup-local-dev.sh`) automatically adds the required entry. On macOS, it will prompt for your administrator password using a system dialog. On Linux, you may need to run it with `sudo`.

**Verify the entry:**
```bash
# View current entry
cat /etc/hosts | grep market-data.local
```

**Manual setup (if needed):**
```bash
# Add entry manually (macOS/Linux)
echo "127.0.0.1 market-data.local" | sudo tee -a /etc/hosts
```

**Windows:**
1. Open Notepad as Administrator
2. Open `C:\Windows\System32\drivers\etc\hosts`
3. Add line: `127.0.0.1 market-data.local`
4. Save and close

### 2. Docker Services

**Start Services:**
```bash
docker-compose up -d
```

**View Logs:**
```bash
# All services
docker-compose logs -f

# Specific service
docker-compose logs -f api
docker-compose logs -f nginx
docker-compose logs -f db
```

**Stop Services:**
```bash
docker-compose down
```

**Restart Services:**
```bash
docker-compose restart
```

### 3. Verify Services

**Check Service Status:**
```bash
docker-compose ps
```

All services should show "Up" status.

**Test Endpoints:**
```bash
# Health check
curl http://market-data.local/health

# API endpoints
curl http://market-data.local/api/symbols
curl http://market-data.local/api/health

# Test with browser
open http://market-data.local/health  # macOS
# or visit http://market-data.local/health in your browser
```

---

## MT5 EA Configuration

### Step 1: Add URL to MT5 Allowed List

1. Open MT5
2. Go to: **Tools → Options → Expert Advisors**
3. Click the **"Allowed URLs"** tab
4. Click **"Add"**
5. Enter: `http://market-data.local`
6. Click **"OK"**

### Step 2: Configure EA

1. Attach `MarketDataSender.mq5` to any chart
2. In EA Inputs, set:
   - **ApiBaseUrl**: `http://market-data.local`
   - **HistoricalCandleCount**: `1000` (or your preference)
   - **LivePriceIntervalSeconds**: `5` (or your preference)
3. Click **"OK"**

### Step 3: Verify Connection

1. Check the **Experts** tab in MT5 Terminal
2. Look for EA logs:
   - ✅ `✅ Symbols initialized: X`
   - ✅ `✅ Successfully sent live prices to API`
   - ❌ If you see errors, check troubleshooting section below

---

## Testing the Setup

### Test 1: Basic Connectivity

```bash
# Health check
curl http://market-data.local/health

# Expected response:
# {"success":true,"message":"Market Data Service is running",...}
```

### Test 2: API Endpoints

```bash
# Get symbols
curl http://market-data.local/api/symbols

# Get health
curl http://market-data.local/api/health
```

### Test 3: MT5 EA Connection

1. Start MT5 EA with `ApiBaseUrl = "http://market-data.local"`
2. Check MT5 Experts tab for logs
3. Verify symbols are being synced
4. Check that live prices are being sent

### Test 4: Database Verification

```bash
# Connect to database
docker-compose exec db psql -U postgres -d financial_data

# Check tables
\dt

# Check symbols
SELECT * FROM symbols LIMIT 5;

# Check live prices
SELECT * FROM live_prices LIMIT 5;

# Exit
\q
```

---

## Troubleshooting

### Issue: "curl: Could not resolve host: market-data.local"

**Solution:**
```bash
# Verify hosts entry exists
cat /etc/hosts | grep market-data.local

# If missing, add it
echo "127.0.0.1 market-data.local" | sudo tee -a /etc/hosts

# Flush DNS cache (macOS)
sudo dscacheutil -flushcache

# Flush DNS cache (Linux)
sudo systemd-resolve --flush-caches
```

### Issue: "502 Bad Gateway" or "Connection refused"

**Solution:**
```bash
# Check if services are running
docker-compose ps

# If not running, start them
docker-compose up -d

# Check service logs
docker-compose logs api
docker-compose logs nginx

# Restart services
docker-compose restart
```

### Issue: MT5 EA shows "WebRequest connection error"

**Solution:**
1. Verify URL is in MT5 Allowed URLs list
2. Check URL format: `http://market-data.local` (no trailing slash)
3. Test URL manually: `curl http://market-data.local/health`
4. Verify Docker services are running: `docker-compose ps`
5. Check Nginx logs: `docker-compose logs nginx`

### Issue: "Failed to fetch symbols from API: HTTP 502"

**Solution:**
```bash
# Check API service
docker-compose logs api

# Check database connection
docker-compose exec db pg_isready -U postgres

# Restart API service
docker-compose restart api
```

### Issue: Database connection errors

**Solution:**
```bash
# Check database is running
docker-compose ps db

# Check database logs
docker-compose logs db

# Restart database
docker-compose restart db

# Wait for database to be ready
docker-compose exec db pg_isready -U postgres
```

### Issue: Port 80 already in use

**Solution:**
```bash
# Check what's using port 80
sudo lsof -i :80

# Stop conflicting service or change Nginx port in docker-compose.yml
# Edit docker-compose.yml, change "80:80" to "8080:80"
# Then use: http://market-data.local:8080
```

---

## Development Workflow

### Making Code Changes

The API service uses volume mounts for live code reloading:

```bash
# Code changes in src/ are automatically reloaded
# No need to restart containers

# View API logs to see changes
docker-compose logs -f api
```

### Database Migrations

Migrations run automatically on container start. To run manually:

```bash
# Run migrations
docker-compose exec api npx sequelize-cli db:migrate

# Check migration status
docker-compose exec api npx sequelize-cli db:migrate:status
```

### Viewing Logs

```bash
# All services
docker-compose logs -f

# Specific service
docker-compose logs -f api
docker-compose logs -f nginx
docker-compose logs -f db

# Last 100 lines
docker-compose logs --tail=100 api
```

---

## Switching Between Local and Production

### Local Development
- **URL**: `http://market-data.local`
- **MT5 EA**: `ApiBaseUrl = "http://market-data.local"`

### Production
- **URL**: `http://market-price.insightbull.io`
- **MT5 EA**: `ApiBaseUrl = "http://market-price.insightbull.io"`

You can easily switch by changing the `ApiBaseUrl` input in MT5 EA settings.

---

## Cleanup

### Remove Hosts Entry (if needed)

**macOS/Linux:**
```bash
# Remove the entry
sudo sed -i '' '/market-data.local/d' /etc/hosts  # macOS
sudo sed -i '/market-data.local/d' /etc/hosts     # Linux
```

**Windows:**
1. Open Notepad as Administrator
2. Open `C:\Windows\System32\drivers\etc\hosts`
3. Remove line: `127.0.0.1 market-data.local`
4. Save and close

### Stop Docker Services

```bash
# Stop and remove containers
docker-compose down

# Stop and remove containers + volumes (⚠️ deletes database data)
docker-compose down -v
```

---

## Additional Resources

- **Main README**: See [README.md](../README.md) for general project information
- **Docker Guide**: See [DOCKER.md](../DOCKER.md) for Docker-specific details
- **API Documentation**: See [docs/API_CONTRACT.md](./API_CONTRACT.md)
- **MT5 Operation**: See [docs/MT5_OPERATION.md](./MT5_OPERATION.md)

---

## Quick Reference

### Essential Commands

```bash
# Setup (one-time, fully automated on macOS)
bash scripts/setup-local-dev.sh
# On Linux, you may need: sudo bash scripts/setup-local-dev.sh

# Start services
docker-compose up -d

# Stop services
docker-compose down

# View logs
docker-compose logs -f api

# Test endpoint
curl http://market-data.local/health

# Check services
docker-compose ps
```

### URLs

- **Local API**: `http://market-data.local`
- **Health Check**: `http://market-data.local/health`
- **API Endpoints**: `http://market-data.local/api/*`

### MT5 Configuration

- **Allowed URL**: `http://market-data.local`
- **EA ApiBaseUrl**: `http://market-data.local`

---

## Need Help?

If you encounter issues not covered in this guide:

1. Check the troubleshooting section above
2. Review service logs: `docker-compose logs`
3. Verify hosts entry: `cat /etc/hosts | grep market-data.local`
4. Test connectivity: `curl http://market-data.local/health`

For additional support, check the main [README.md](../README.md) or project documentation.