Hussain Afzal

hussain.afzal pushed to master at MQL-Development/market-data-service

2dfc5f7ba6 Fix CORS configuration and HTTPS port 443 issue - Add support for multiple CORS origins (comma-separated) * Updated src/app.js to parse comma-separated CORS_ORIGIN env variable * Updated src/server.js to support multiple origins for Socket.io WebSocket connections * Allows frontend apps from localhost, beta, and production to access the API - Fix nginx configuration typo preventing HTTPS from working * Fixed 'body_size_sent' -> 'body_bytes_sent' in log_format * This typo was preventing nginx from starting with SSL configuration * Port 443 is now properly listening and accepting HTTPS connections - Update docker-compose.prod.yml with default CORS origins * Set default CORS_ORIGIN to include all frontend environments: - http://localhost:3000 (local development) - https://beta-app.insightbull.io (beta/staging) - https://app.insightbull.io (production) These changes enable: - Frontend team to access API from all environments (localhost, beta, production) - HTTPS connections to work properly (port 443) - WebSocket connections to work from all allowed origins - Proper CORS headers for cross-origin requests Fixes connection issues reported by frontend team where: - CORS was blocking localhost:3000 requests - HTTPS port 443 was refusing connections

2 months ago

hussain.afzal pushed to master at MQL-Development/market-data-service

c69e943a3b feat: Add production deployment with automated SSL certificate management This commit introduces a complete production deployment setup with fully automated SSL certificate management using Let's Encrypt and Certbot. ## Key Features ### SSL Certificate Management - Automatic SSL certificate obtainment via Certbot - Automatic certificate renewal every 12 hours - Automatic nginx reload after certificate renewal - Support for Let's Encrypt staging server for testing - Fully containerized certificate management ### Infrastructure Improvements - Updated docker-compose.prod.yml with Certbot services - Dynamic nginx configuration that adapts to certificate availability - Shared Docker volumes for certificate persistence - Health checks for all services ### New Scripts - docker/nginx-entrypoint.sh: Dynamic nginx config processing with SSL support - docker/certbot-init.sh: SSL certificate initialization wrapper - docker/init-ssl.sh: Initial SSL certificate obtainment script - docker/renew-ssl.sh: SSL certificate renewal script ### Configuration Updates - nginx/nginx.prod.conf: Updated for Let's Encrypt certificate paths - .env.example: Added production environment variables (DOMAIN_NAME, SSL_EMAIL) - docker-compose.prod.yml: Integrated Certbot services and volumes ### Documentation - docs/DEPLOYMENT.md: Complete production deployment guide - docs/CONFIGURATION.md: Environment variables and configuration guide - docs/README.md: Documentation index - Updated DOCKER.md and README.md with deployment references ### Security - All sensitive files (.env) remain in .gitignore - No hardcoded secrets or passwords - Secure certificate storage in Docker volumes ## Deployment Process 1. Set environment variables in .env file 2. Start services: docker-compose -f docker-compose.prod.yml up -d db api nginx 3. Obtain SSL certificate: docker-compose run --rm certbot-init 4. Services automatically handle certificate renewal ## Testing - HTTP endpoint working: http://market-price.insightbull.io/health - SSL certificates obtained and configured - Automatic renewal configured - All services healthy and running This deployment has been tested and verified on production server.
cae42ecf1f feat: Add production deployment with automated SSL certificate management This commit introduces a complete production deployment setup with fully automated SSL certificate management using Let's Encrypt and Certbot. ## Key Features ### SSL Certificate Management - Automatic SSL certificate obtainment via Certbot - Automatic certificate renewal every 12 hours - Automatic nginx reload after certificate renewal - Support for Let's Encrypt staging server for testing - Fully containerized certificate management ### Infrastructure Improvements - Updated docker-compose.prod.yml with Certbot services - Dynamic nginx configuration that adapts to certificate availability - Shared Docker volumes for certificate persistence - Health checks for all services ### New Scripts - docker/nginx-entrypoint.sh: Dynamic nginx config processing with SSL support - docker/certbot-init.sh: SSL certificate initialization wrapper - docker/init-ssl.sh: Initial SSL certificate obtainment script - docker/renew-ssl.sh: SSL certificate renewal script ### Configuration Updates - nginx/nginx.prod.conf: Updated for Let's Encrypt certificate paths - .env.example: Added production environment variables (DOMAIN_NAME, SSL_EMAIL) - docker-compose.prod.yml: Integrated Certbot services and volumes ### Documentation - docs/DEPLOYMENT.md: Complete production deployment guide - docs/CONFIGURATION.md: Environment variables and configuration guide - docs/README.md: Documentation index - Updated DOCKER.md and README.md with deployment references ### Security - All sensitive files (.env) remain in .gitignore - No hardcoded secrets or passwords - Secure certificate storage in Docker volumes ## Deployment Process 1. Set environment variables in .env file 2. Start services: docker-compose -f docker-compose.prod.yml up -d db api nginx 3. Obtain SSL certificate: docker-compose run --rm certbot-init 4. Services automatically handle certificate renewal ## Testing - HTTP endpoint working: http://market-price.insightbull.io/health - SSL certificates obtained and configured - Automatic renewal configured - All services healthy and running This deployment has been tested and verified on production server.
View comparison for these 2 commits »

2 months ago

hussain.afzal pushed to master at MQL-Development/market-data-service

cae42ecf1f feat: Add production deployment with automated SSL certificate management This commit introduces a complete production deployment setup with fully automated SSL certificate management using Let's Encrypt and Certbot. ## Key Features ### SSL Certificate Management - Automatic SSL certificate obtainment via Certbot - Automatic certificate renewal every 12 hours - Automatic nginx reload after certificate renewal - Support for Let's Encrypt staging server for testing - Fully containerized certificate management ### Infrastructure Improvements - Updated docker-compose.prod.yml with Certbot services - Dynamic nginx configuration that adapts to certificate availability - Shared Docker volumes for certificate persistence - Health checks for all services ### New Scripts - docker/nginx-entrypoint.sh: Dynamic nginx config processing with SSL support - docker/certbot-init.sh: SSL certificate initialization wrapper - docker/init-ssl.sh: Initial SSL certificate obtainment script - docker/renew-ssl.sh: SSL certificate renewal script ### Configuration Updates - nginx/nginx.prod.conf: Updated for Let's Encrypt certificate paths - .env.example: Added production environment variables (DOMAIN_NAME, SSL_EMAIL) - docker-compose.prod.yml: Integrated Certbot services and volumes ### Documentation - docs/DEPLOYMENT.md: Complete production deployment guide - docs/CONFIGURATION.md: Environment variables and configuration guide - docs/README.md: Documentation index - Updated DOCKER.md and README.md with deployment references ### Security - All sensitive files (.env) remain in .gitignore - No hardcoded secrets or passwords - Secure certificate storage in Docker volumes ## Deployment Process 1. Set environment variables in .env file 2. Start services: docker-compose -f docker-compose.prod.yml up -d db api nginx 3. Obtain SSL certificate: docker-compose run --rm certbot-init 4. Services automatically handle certificate renewal ## Testing - HTTP endpoint working: http://market-price.insightbull.io/health - SSL certificates obtained and configured - Automatic renewal configured - All services healthy and running This deployment has been tested and verified on production server.

2 months ago

hussain.afzal pushed to master at MQL-Development/market-data-service

2b037e3119 Fix MT5 EA: precision, instrument type, and cleanup improvements - Increase candle price precision from %.5f to %.8f to match API requirements - Change instrument type from 'metal' to 'commodity' for XAU/XAG symbols (API validation) - Fix cleanup function to handle all timeframes instead of just default '1h' - Update timer comments to accurately reflect 10-minute sync interval - Improve cleanup to iterate through all timeframes (15m, 30m, 1h, 1D, 1W, 1M) These fixes ensure: - Candle data is sent with correct precision (8 decimals) - Symbol creation won't fail for metal instruments - Old candles are properly cleaned up across all timeframes

2 months ago

hussain.afzal pushed to master at MQL-Development/market-data-service

4c16f0f502 feat: Add local development domain support for MT5 EA connectivity - Update Nginx config to accept market-data.local domain alongside localhost - Add automated setup script with macOS password prompt support - Update MT5 EA with local dev URL documentation - Add comprehensive local development setup guide - Update README with Docker-based local dev instructions - Remove redundant documentation files (LOCAL_DEV_URL_ANALYSIS.md, QUICK_START.md) This enables MT5 Expert Advisors to connect to local development environment using market-data.local domain, as MT5 doesn't allow localhost/127.0.0.1 URLs. Setup is now fully automated on macOS (prompts for password) and works on Linux with sudo fallback.

3 months ago

hussain.afzal pushed to master at MQL-Development/market-data-service

ba0b4a0aa3 refactor: Remove unnecessary migrations for fresh deployment Remove old migration files entirely as they are not needed for first-time deployment. The migrations were designed to upgrade an old database schema (candles_1h -> candles, adding timeframe, etc.), but since this is a fresh deployment, the Sequelize models already define the correct schema. The models are the single source of truth: - Candle model: includes timeframe, proper constraints and indexes - Symbol model: includes all instrument types including 'index', proper indexes - LivePrice model: properly defined Database schema is created entirely by model sync on first deployment. Future schema changes should be handled through new migrations if needed.

3 months ago

hussain.afzal pushed to master at MQL-Development/market-data-service

fe44f0b9b7 fix: Fix Docker deployment issues and configure for local development Fix multiple issues encountered during local Docker deployment: Docker Configuration: - Fix Dockerfile to handle package-lock.json properly (use npm install if missing) - Update .dockerignore to include package-lock.json in builds - Add docker/sync-models.js script to sync Sequelize models before migrations - Update docker-compose.yml to sync models before running migrations - Configure API to use port 3001 (to avoid conflict with port 3000) - Update docker-compose.yml to read PORT from .env file Database Configuration: - Update config/config.json for Docker environment (host: db, password: postgres) - Make migrations resilient by checking table existence before operations - Update all three migrations to skip if tables don't exist (tables created by model sync) Nginx Configuration: - Fix nginx.conf log format (body_size_sent -> body_bytes_sent) - Update nginx upstream to proxy to api:3001 instead of api:3000 Entrypoint Script: - Add model sync step to entrypoint.sh before migrations These changes ensure: - Containers start successfully without migration errors - Database tables are created via model sync before migrations run - Migrations are idempotent and can run on fresh databases - Port configuration is consistent across all services - Local development works with port 3001 to avoid conflicts

3 months ago

hussain.afzal pushed to master at MQL-Development/market-data-service

38582f44c8 refactor: Remove PM2 deployment references in favor of Docker Remove all PM2 process manager references and replace with Docker-based deployment instructions throughout the project. Changes: - Delete ecosystem.config.js (PM2 configuration file) - Replace PM2 installation step with Docker deployment step in README - Update deployment update procedures to use Docker Compose - Replace PM2 troubleshooting section with Docker container troubleshooting - Replace PM2 management commands with Docker management commands - Update deployment section to reference Docker exclusively The project now uses Docker containers exclusively for both development and production deployment, providing a consistent and containerized deployment strategy.

3 months ago

hussain.afzal pushed to master at MQL-Development/market-data-service

7c642d6730 feat: Add complete Docker containerization for development and production Implement comprehensive Docker setup for Market Data Service with support for both local development and production deployment. Features: - Multi-stage Dockerfile with optimized production builds - Development setup with live code reload (nodemon) - Docker Compose configurations for dev and prod environments - PostgreSQL database container with automatic initialization - Nginx reverse proxy with WebSocket support for Socket.io - Automatic database migrations on container startup - Health checks for all services - Data persistence via Docker volumes - SSL/TLS support in production configuration - Complete documentation in DOCKER.md Files added: - Dockerfile: Multi-stage build (dependencies + production) - docker-compose.yml: Development environment - docker-compose.prod.yml: Production environment - nginx/nginx.conf: Development Nginx config (HTTP + WebSocket) - nginx/nginx.prod.conf: Production Nginx config (HTTPS + SSL) - docker/entrypoint.sh: API container entrypoint with migrations - docker/wait-for-db.sh: Database readiness check - docker/init-db.sh: Database initialization script - .dockerignore: Docker build exclusions - DOCKER.md: Complete containerization documentation Files modified: - .gitignore: Added Docker-related entries This enables zero-dependency local development (only Docker Desktop required) and production-ready containerized deployment with proper security, scaling, and data persistence.

3 months ago

hussain.afzal pushed to master at MQL-Development/market-data-service

5140bd7081 feat(tests): Add live price controller tests and documentation updates - Added comprehensive test suite for livePriceController - Updated README.md testing section with controller test coverage - Verified documentation consistency with current implementation
82c01cac5c feat: update project dependencies and live price controller - Fixed API endpoints and tested all GET APIs - Resolved issues in live price controller for better data handling - Updated package dependencies for improved stability
59bd38cbf9 feat(MT5): enhance integration and data integrity - Add unique constraint to candles table to prevent duplicate entries - Implement retry logic with exponential backoff for MT5 connections - Enforce 15-decimal precision in data validation for accuracy - Create API contract and MT5 operation documentation - Update README with technical specifications and setup instructions This commit improves the reliability of MT5 data integration by preventing duplicate data entries and ensuring connection stability. The added documentation helps developers understand the system architecture and operational procedures.
eda6f70d7e feat: overhaul project structure with Sequelize integration - README.md: • Added Supertest and Sequelize CLI to tooling • Updated database setup instructions to use migrations • Enhanced testing and migration documentation • Improved development workflow commands - config/config.json: • Added database configuration for development/test/production environments - models/index.js: • Implemented Sequelize model loader and association setup - package.json: • Added devDependencies: @jest/globals, jest, sequelize-cli, supertest - src/models/Candle1h.js: • Enhanced model definition with validations • Added association configurations - src/models/LivePrice.js: • Improved model structure with timestamp handling • Added foreign key constraints - tests/candleController.test.js: • Created initial test suite for candle endpoints • Implemented database cleanup routines • Added CRUD operation tests with Supertest - General: • Standardized Sequelize configuration across environments • Prepared project for migration-based database management
bf57b9271b feat: Add MT5 integration with automatic symbol registration and data streaming This commit introduces: - MT5 Expert Advisor (MarketDataSender.mq5) that: * Automatically registers symbols with the API * Sends last 1000 historical candles on initialization * Streams live prices via bulk updates every second * Handles error conditions and retries - Comprehensive README documentation for MT5 setup - Symbol mapping between MT5 and database IDs
View comparison for these 6 commits »

3 months ago

hussain.afzal transfered repository muhammad.uzair/market-data-service to MQL-Development/market-data-service

3 months ago

hussain.afzal pushed to feat/initial-project-setup at muhammad.uzair/market-data-service

5140bd7081 feat(tests): Add live price controller tests and documentation updates - Added comprehensive test suite for livePriceController - Updated README.md testing section with controller test coverage - Verified documentation consistency with current implementation

3 months ago

hussain.afzal pushed to feat/initial-project-setup at muhammad.uzair/market-data-service

59bd38cbf9 feat(MT5): enhance integration and data integrity - Add unique constraint to candles table to prevent duplicate entries - Implement retry logic with exponential backoff for MT5 connections - Enforce 15-decimal precision in data validation for accuracy - Create API contract and MT5 operation documentation - Update README with technical specifications and setup instructions This commit improves the reliability of MT5 data integration by preventing duplicate data entries and ensuring connection stability. The added documentation helps developers understand the system architecture and operational procedures.
fc955241ef Enhance MT5 integration and data integrity - Added unique constraint to prevent duplicate candle entries - Implemented MT5 retry logic with exponential backoff - Enforced 15-decimal precision in data validation - Created comprehensive documentation: • API contract specifications • MT5 operation guide - Updated README with technical specifications
View comparison for these 2 commits »

4 months ago

hussain.afzal pushed to feat/initial-project-setup at muhammad.uzair/market-data-service

fc955241ef Enhance MT5 integration and data integrity - Added unique constraint to prevent duplicate candle entries - Implemented MT5 retry logic with exponential backoff - Enforced 15-decimal precision in data validation - Created comprehensive documentation: • API contract specifications • MT5 operation guide - Updated README with technical specifications

4 months ago

hussain.afzal pushed to feat/initial-project-setup at muhammad.uzair/market-data-service

eda6f70d7e feat: overhaul project structure with Sequelize integration - README.md: • Added Supertest and Sequelize CLI to tooling • Updated database setup instructions to use migrations • Enhanced testing and migration documentation • Improved development workflow commands - config/config.json: • Added database configuration for development/test/production environments - models/index.js: • Implemented Sequelize model loader and association setup - package.json: • Added devDependencies: @jest/globals, jest, sequelize-cli, supertest - src/models/Candle1h.js: • Enhanced model definition with validations • Added association configurations - src/models/LivePrice.js: • Improved model structure with timestamp handling • Added foreign key constraints - tests/candleController.test.js: • Created initial test suite for candle endpoints • Implemented database cleanup routines • Added CRUD operation tests with Supertest - General: • Standardized Sequelize configuration across environments • Prepared project for migration-based database management
bf57b9271b feat: Add MT5 integration with automatic symbol registration and data streaming This commit introduces: - MT5 Expert Advisor (MarketDataSender.mq5) that: * Automatically registers symbols with the API * Sends last 1000 historical candles on initialization * Streams live prices via bulk updates every second * Handles error conditions and retries - Comprehensive README documentation for MT5 setup - Symbol mapping between MT5 and database IDs
View comparison for these 2 commits »

4 months ago

hussain.afzal pushed to feat/initial-project-setup at muhammad.uzair/market-data-service

7ece7a970d feat: initial project setup with complete market data service architecture - Add comprehensive Node.js/Express.js backend structure - Implement PostgreSQL database schema with Sequelize ORM - Create RESTful API endpoints for symbols, candles, and live prices - Add data validation using Joi and error handling middleware - Configure Winston logging and security with Helmet.js - Set up modular architecture with controllers, routes, and models - Update .gitignore with standard Node.js exclusions - Enhance README with detailed documentation, installation, and API guides This commit establishes the foundational architecture for a high-performance financial data API supporting multiple asset classes including cryptocurrencies, stocks, forex, and commodities with both RESTful endpoints and real-time WebSocket capabilities.
ae138a0f48 Initial commit

4 months ago

hussain.afzal transfered repository tayyab.akhlaq/htf_delta_hedge_strategy_mt5 to MQL-Development/htf_delta_hedge_strategy_mt5

1 year ago

hussain.afzal merged pull request MQL-Development/mm-ui-tradescopier#25

Update fetchUsers.js

1 year ago

hussain.afzal closed pull request MQL-Development/mm-ui-tradescopier#25

Update fetchUsers.js

1 year ago

hussain.afzal merged pull request MQL-Development/mm-ui-tradescopier#24

Free Master Product Fixed & Master Master Product Subscription is added

1 year ago