# Market Data Service

A high-performance financial data API that provides comprehensive market data for various financial instruments including cryptocurrencies, stocks, forex, and commodities through both RESTful endpoints and real-time WebSocket connections.

## Features

- **Multi-Asset Support**: Handles cryptocurrencies, stocks, forex, and commodities
- **Real-time Data**: Live price feeds with bid/ask spreads
- **Historical Data**: OHLCV candle data with flexible timeframes
- **RESTful API**: Well-structured endpoints for all operations
- **Data Validation**: Comprehensive input validation using Joi
- **Error Handling**: Robust error handling with detailed responses
- **Security**: Helmet.js for security headers, CORS support
- **Logging**: Winston-based logging with multiple transports
- **Database**: PostgreSQL with Sequelize ORM
- **Scalable Architecture**: Modular design with controllers, routes, and middleware

## Tech Stack

- **Backend**: Node.js, Express.js
- **Database**: PostgreSQL
- **ORM**: Sequelize
- **Validation**: Joi
- **Security**: Helmet, CORS
- **Logging**: Winston, Morgan
- **Testing**: Jest
- **Development**: Nodemon, ESLint

## Project Structure

```
market-data-service/
├── src/
│   ├── config/
│   │   └── database.js          # Database configuration
│   ├── controllers/
│   │   ├── symbolController.js      # Symbol CRUD operations
│   │   ├── candleController.js      # Candle data operations
│   │   └── livePriceController.js   # Live price operations
│   ├── middleware/
│   │   ├── errorHandler.js          # Global error handling
│   │   └── validation.js            # Request validation
│   ├── models/
│   │   ├── Symbol.js                # Symbol model
│   │   ├── Candle1h.js              # 1-hour candle model
│   │   ├── LivePrice.js             # Live price model
│   │   └── index.js                 # Model associations
│   ├── routes/
│   │   ├── symbols.js               # Symbol routes
│   │   ├── candles.js               # Candle routes
│   │   └── livePrices.js            # Live price routes
│   ├── utils/
│   │   └── logger.js                # Logging utility
│   ├── app.js                       # Express app configuration
│   └── server.js                    # Server startup
├── tests/                           # Test files
├── schema.sql                       # Database schema
├── .env                             # Environment variables
├── .gitignore                       # Git ignore rules
├── package.json                     # Dependencies and scripts
└── README.md                        # This file
```

## Installation

1. **Clone the repository**
   ```bash
   git clone https://git.mqldevelopment.com/muhammad.uzair/market-data-service.git
   cd market-data-service
   ```

2. **Install dependencies**
   ```bash
   npm install
   ```

3. **Set up environment variables**
   ```bash
   cp .env.example .env
   ```
   Edit `.env` with your database credentials and other configuration.

4. **Set up the database**
   ```bash
   # Create PostgreSQL database
   createdb market_data

   # Run the schema
   psql -d market_data -f schema.sql
   ```

5. **Start the development server**
   ```bash
   npm run dev
   ```

The server will start on `http://localhost:3000`

## Environment Variables

Create a `.env` file in the root directory with the following variables:

```env
# Database Configuration
DB_HOST=localhost
DB_PORT=5432
DB_NAME=market_data
DB_USER=your_username
DB_PASSWORD=your_password

# Server Configuration
PORT=3000
NODE_ENV=development

# JWT Configuration (if needed for authentication)
JWT_SECRET=your_jwt_secret_key

# API Keys (if needed for external services)
# BINANCE_API_KEY=your_api_key
# BINANCE_API_SECRET=your_api_secret

# CORS Configuration
CORS_ORIGIN=*
```

## API Endpoints

### Health Check
- `GET /health` - Check service health

### Symbols
- `GET /api/symbols` - Get all symbols (with filtering)
- `GET /api/symbols/search` - Search symbols by name
- `GET /api/symbols/:id` - Get symbol by ID
- `POST /api/symbols` - Create new symbol
- `PUT /api/symbols/:id` - Update symbol
- `DELETE /api/symbols/:id` - Delete symbol (soft delete)

### Candles
- `GET /api/candles` - Get candles with filtering
- `GET /api/candles/ohlc` - Get OHLC data
- `GET /api/candles/:symbolId/latest` - Get latest candle for symbol
- `POST /api/candles` - Create new candle
- `POST /api/candles/bulk` - Bulk create candles

### Live Prices
- `GET /api/live-prices` - Get all live prices
- `GET /api/live-prices/exchange/:exchange` - Get live prices by exchange
- `GET /api/live-prices/type/:type` - Get live prices by instrument type
- `GET /api/live-prices/:symbolId` - Get live price for symbol
- `POST /api/live-prices` - Create/update live price
- `POST /api/live-prices/bulk` - Bulk update live prices
- `DELETE /api/live-prices/:symbolId` - Delete live price

## Database Schema

The service uses three main tables:

1. **symbols** - Trading symbols metadata
2. **candles_1h** - Hourly OHLCV data
3. **live_prices** - Current market prices

See `schema.sql` for complete table definitions.

## Development

### Available Scripts

- `npm start` - Start production server
- `npm run dev` - Start development server with auto-reload
- `npm test` - Run tests
- `npm run test:watch` - Run tests in watch mode
- `npm run lint` - Run ESLint
- `npm run lint:fix` - Fix ESLint issues

### Code Style

This project uses ESLint for code linting. Run `npm run lint` to check for issues and `npm run lint:fix` to automatically fix them.

### Testing

Tests are written using Jest. Add your test files in the `tests/` directory.

## Deployment

1. Set `NODE_ENV=production` in your environment
2. Run `npm start` to start the production server
3. Consider using a process manager like PM2 for production deployments

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new features
5. Ensure all tests pass
6. Submit a pull request

## License

ISC License - see LICENSE file for details.

## Support

For support or questions, please contact the development team.