================================================================================ SMARTBULLS BACKEND - SYSTEM ARCHITECTURE & DOCUMENTATION ================================================================================ SmartBulls is a modern, high-performance virtual trading and contest platform. This readme describes the overall system topology, Docker orchestrations, individual container specifications, and folder directories. -------------------------------------------------------------------------------- 1. THE MAIN SYSTEM LAYER: DOCKER -------------------------------------------------------------------------------- Docker acts as the unified system layer, providing strict containerization, microservice isolation, secure localized bridge networks, and reproducible staging and production configurations. All microservices are orchestrated together via docker-compose.yml, allowing seamless startup, configuration management, and dynamic scale control. -------------------------------------------------------------------------------- 2. THE STARTED CONTAINERS FLOW GRAPH -------------------------------------------------------------------------------- The SmartBulls environment starts exactly 7 containers connected via a private bridge network named "app-network": [ Public Web Client / Mobile Apps ] │ (HTTP 80 / HTTPS 443) ▼ ┌─────────────────────────────────────────────────────────────┐ │ web_server (Nginx:alpine) │ └───────┬────────────────────────┬────────────────────┬───────┘ │ (FastCGI:9000) │ (Proxy Pass:4000) │ (Proxy:80) ▼ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ ┌────────────┐ │ php_app │ │ node_app │ │ pma │ │ (PHP-FPM) │ │ (Node.js API) │ │(PhpMyAdmin)│ └────────┬────────┘ └────────┬────────┘ └──────┬─────┘ │ │ │ │ ┌──────┴──────┐ │ │ │ kite_ticker │ │ │ └──────┬──────┘ │ │ (MySQL:3306) │ (Redis:6379) │ ▼ ▼ │ ┌──────────────────────────────────────────┐ │ │ mysql_db (MySQL 8.0) │◄────────────┘ └──────────────────────────────────────────┘ ▲ │ ┌────────────────────┴─────────────────────┐ │ cron_worker (Cron) │ └──────────────────────────────────────────┘ ▲ │ (Redis Cache / Lock) ▼ ┌──────────────────────────────────────────┐ │ redis_cache (Redis 7) │ └──────────────────────────────────────────┘ -------------------------------------------------------------------------------- 3. INDIVIDUAL CONTAINER ROLE & CONFIGURATION DETAILS -------------------------------------------------------------------------------- [1] Container Name: web_server (Image: nginx:alpine) - Port Bindings: 80:80 (HTTP), 443:443 (HTTPS) - Primary Role: Acts as the primary ingress controller and SSL termination proxy. Distributes traffic dynamically: * Static Assets: Served directly from the local public volume mount. * General / Legacy API requests: Handled via FastCGI to php_app. * WebSocket (Socket.io) / Live Tickers: Proxied to node_app (port 5000). * Express Auth & Trading Endpoints: Proxied to node_app (port 4000). * phpMyAdmin Proxy: Routed to phpmyadmin container (port 80). - Configurations: * Rate Limiting: General requests limited to 15r/s (mylimit). Sensitive APIs (Login/Register) limited to 5r/s (api_limit). Connection limit 30. [2] Container Name: mysql_db (Image: mysql:8.0) - Port Bindings: Exposed internally on port 3306 within app-network. - Primary Role: Persistent relational database storing users, contests, joins, closed orders, positions, watchlists, plans, and historic ledgers. - Configurations: * Init Script: mounts ./docker/mysql-init.sql to auto-bootstrap schema. * Storage Volume: mounts persistent external volume `dbdata` to `/var/lib/mysql`. * High Load Settings: max_connections=1000, innodb_buffer_pool_size=1G. * Strict Modes: STRICT_TRANS_TABLES, ERROR_FOR_DIVISION_BY_ZERO, etc. [3] Container Name: pma (Image: phpmyadmin/phpmyadmin) - Port Bindings: 8081:80 (External access at port 8081) - Primary Role: Web-based GUI administration client for managing mysql_db. - Configurations: * Host: Binded to container "db" on port 3306. * URL Routing: PMA_ABSOLUTE_URI is set to: https://app.edistry.com/phpmyadmin/ * Upload Limit: UPLOAD_LIMIT: 300M (for importing large databases). [4] Container Name: redis_cache (Image: redis:7-alpine) - Port Bindings: None publicly exposed (Isolated internally on port 6379 within app-network) - Primary Role: High-speed caching layer, locking provider, and dashboard feed serializer to maximize performance. - Configurations: * Security Password: --requirepass cspl2023 * Memory Policy: --maxmemory 512mb --maxmemory-policy allkeys-lru * Persistence: mounts `redisdata` persistent volume. [5] Container Name: cron_worker (Image: custom php/cron build) - Port Bindings: Running completely backgrounded within the network. - Primary Role: Triggers all schedule-based background scripts, automated contest management, stock expiries, and database settlement tasks. - Active Cronjobs (crontab-set.txt): * cronjob-notifier.php (Every 5 mins): Dispatches SMS and Firebase push notifications for trading alerts and administrative actions. * cronjob-r.php (Every 5 mins): Resolves contest rank-based reward distribution and handles user settlement ledgers. * cronjob.php (Every minute): Executes minute-by-minute order square-offs, calculates unrealized contest PnL, and checks position states. * cash-price-update.php (Daily at 09:10 AM): Syncs index cash prices and contract parameters with external stock data feeds. * check-stock-expiry.php (Daily at 11:00 PM): Sweeps expiring derivative contracts and marks them as settled or expired. * pending-group-user-join.php (Every minute M-F): Processes queued requests for private group contest entries during market hours. * cronjob-plan.php (Hourly): Evaluates subscriber membership durations and auto-renews/renews active plans. * auto_add_contest_schedule.php (Daily at 09:20 AM): Creates tomorrow's contests based on active tournament templates. * run-workers.sh (Every 25 minutes): Monitors and cleans leaking/stuck background worker threads to secure system memory. * balance-settlement.js (Every 10 minutes): Triggers rapid batch Node.js settlements for contest joins and balances. [6] Container Name: php_app (Image: custom php-fpm build) - Port Bindings: Running internally on port 9000 (connected to web_server) - Primary Role: Runs the Legacy PHP API and the administrative admin panel. - Folder Structure: * /public: Web accessible entry points (index.php, CSS, JS). * /public/admin: All views, controllers, and screens for the admin panel. * /public/admin/cron: Backend PHP cron files executed by cron_worker. * /helpers: Shared functions (DB connectors, notification dispatchers). * /vendor: Composer dependencies. [7] Container Name: node_app (Image: custom Node.js build) - Port Bindings: 3000:3000, 4000:4000, 5000:5000 - Primary Role: Executes the express API backend (running under PM2) for low-latency trading operations, socket servers, and Kite ticker listeners. - Security / DDoS Mitigation: * Connection limits enforced by Nginx layer (max 30 connections). * Dual-tiered rate-limiters at Nginx (mylimit 15r/s, api_limit 5r/s burst 10). * Local Node rate-limiting middleware blocks massive bulk API requests. * Signature headers prevent unauthorized third-party requests. - Folder Structure (/node_api/src): * /controllers: Core route handlers executing business flows. * /routes: Express API endpoints mapped per module. * /services: Heavy-lifting layers (such as trading_engine.service.js). * /models: SQL query models doing optimized database transactions. * /websocket: Websocket listeners (binance, kite feeds). - API Route Modules: * Auth (auth.routes.js): Performs secure login, OTP registration, and profile edits. * Content & Misc (content_and_misc.routes.js): Delivers holidays, news, and app settings. * Contest (contest.routes.js): Manages dynamic pools, join logs, and lists. * Social & Performance (social_and_performance.routes.js): Serves leaderboards and performance metrics. * Stock Management (stock_management.routes.js): Indexes stock lookups, details, and static watchlists. * Stock NFO (stock_nfo.routes.js): Yields contracts details and parent margins. * Trading Engine (trading_engine.routes.js): Handles super low-latency transaction orders. * Wallet & Plans (wallet_and_plans.routes.js): Tracks plan entries, cashouts, and dynamic ledgers. ================================================================================ 4. LOG MANAGEMENT & AUTO-CLEANING SYSTEM ================================================================================ A production-grade automatic log cleaning and rotation system is configured for both Node (PM2) and PHP cron logs to prevent disk space issues: [1] PM2 Log Auto-Rotation (100MB threshold) - Tool: pm2-logrotate (baked directly into the Node container image) - Threshold: Rotates log files automatically when they exceed 100MB. - Compression: Rotated files are gzipped (.gz) automatically (saves >90% disk space). - Retention: Keeps the 10 most recent rotated backups per app (older backups are auto-purged). - Commands: * View live configurations: docker exec node_app pm2 conf pm2-logrotate * Manually force-clear all PM2 logs: docker exec node_app pm2 flush [2] PHP Cron Logs Auto-Clean (100MB threshold) - Tool: Automated script `docker/cron/clean-large-logs.sh` running inside `cron_worker` - Threshold: Truncates log files to 0 bytes safely (without breaking file writes) if they exceed 100MB. - Directories Monitored: * /var/www/html/public/logdata/ (PHP cron logs) * /var/www/html/public/ (Legacy root logs) * /var/www/html/volumes/logdata/ (Staged/volume log files) - Execution Interval: Runs automatically every 30 minutes via crontab: */30 * * * * /bin/bash /var/www/html/docker/cron/clean-large-logs.sh >> /var/www/html/public/logdata/clean-large-logs.log 2>&1 - Commands: * Manually trigger the cron clean-up script right now: docker exec cron_worker /bin/bash /var/www/html/docker/cron/clean-large-logs.sh * View clean-up logs: cat /var/www/html/public/logdata/clean-large-logs.log Remove docker logs command :- truncate -s 0 $(docker inspect --format='{{.LogPath}}' php_app) sudo sh -c 'truncate -s 0 /var/lib/docker/containers/*/*-json.log' 2>/dev/null || true Docker clean command all image volume container stack network all docker compose -f docker-compose.yml down -v --remove-orphans ; docker compose -f docker-compose.live.yml down -v --remove-orphans ; docker system prune -a --volumes -f I checked and compared all configuration files from your Host/Live Server configuration folder (server configuration/) with your Dockerized environment (docker/ and docker-compose.yml). Here is the breakdown of the differences across all services in your stack: 1. Web Server: Apache (Live) vs Nginx (Docker) Your live server used Apache2 directly on the host, whereas the Dockerized setup uses Nginx for high performance and reverse-proxying. WebSocket Routing: Live (Apache): Routed /ws/ requests to ws://localhost:5000/ using mod_proxy_wstunnel. Docker (Nginx): Correctly mirrors this routing, sending /ws and /socket.io/ requests to http://node:5000. SSL Certificates: Live (Apache): Pointed directly to Host path /etc/letsencrypt/live/app.stoxbaba.com/. Docker (Nginx): Mounted certificate folder via docker-compose.yml to the container path /var/www/html/secure/. Rewrite Rules / Hiding .php Extension: Live (Apache): Handled via .htaccess files inside /var/www/html. Docker (Nginx): Handled natively in the Nginx template via try_files $uri $uri/ $uri.php?$query_string /index.php?$query_string;. 2. PHP-FPM Configurations Parameter Live Server (php.ini & www.conf) Docker Container Description pm.max_children 200 150 Maximum parallel PHP processes allowed. pm.start_servers 40 30 Idle worker processes spawned on startup. pm.max_requests 500 1000 Process recycling rate. Docker recycles processes less frequently to minimize CPU overhead. memory_limit 128M 1024M Docker is optimized. Docker allows up to 1GB RAM per script, preventing script crashes on heavy data generation. upload_max_filesize 2M 1024M Docker is optimized. Allows up to 1GB uploads (critical for posters/charts). post_max_size 8M 1024M Same as above. 3. MySQL Database Parameter Live Server Config (mysqld.cnf) Docker Config (docker-compose.yml) Performance Impact innodb_buffer_pool_size 6G (for 8GB RAM host) 1G Critical. Live keeps the database fully cached in RAM. Docker only uses 1GB, forcing disk reads on large tables. innodb_flush_log_at_trx_commit 2 (Good write speed) 1 (Safe but slower) Critical for order placement. 2 writes logs to disk in batches once per second, speeding up transaction writes. max_connections 2000 1000 Maximum concurrent connections allowed. sql_mode Relaxed Stricter (has NO_ZERO_DATE) Compatibility. Docker will reject zero dates (0000-00-00), whereas the live server accepts them. 4. Redis Cache Your live server runs Redis bound strictly to localhost: Live (redis.conf): bind 127.0.0.1 -::1 (Restricts access to host loopback). Docker (Redis container): Accessible to all container services inside the Docker bridge network (app-network), but the port 6379 is not exposed to the outer public internet, maintaining high security. Both environments use standard memory-limit settings (unlimited/noeviction by default). 🛠️ Next Step recommendation To make your Dockerized database behave exactly like your live database (fast transaction write speeds, compatibility with zero-dates, and higher connection limit) while keeping memory usage safe for a shared 8GB staging environment, we should update the db service command inside docker-compose.yml. Would you like me to update docker-compose.yml to apply these database optimizations? . Registration & OTP Verification Flow Mobile Number Validation: When a user registers or requests an OTP, the system validates the number using a regular expression: !/^[6-9]\d{9}$/.test(cleanMobile). It must be a valid 10-digit mobile number (starting with 6-9). If User Goes Back and Tries to Register Again: For an Already Registered User: If the user registration completes and they attempt to register again with that number, the system will block the request and return: "User already exists. Please login." Before verification (Unverified user): If the user registers, receives the OTP, and immediately exits, goes back, and registers again within 60 seconds: The system detects the rate limit lock and returns: "Please wait 60 seconds before requesting another OTP." After the 60-Second Lock Expires: The user is allowed to request another OTP. Once the lock is removed, if they write the correct details and submit, they will receive a new OTP, verify it, and proceed to the Home screen. 2. Login with PIN & Lockout System (Newly Implemented) How it works: Previously, there was no limit on wrong PIN entries. We have implemented a secure lockout using Redis to prevent brute-force attacks. Limitations and Rules: Max Wrong PIN Attempts: 5 times. Lockout Period: 15 minutes. Flow details: If the user enters an incorrect PIN, the system tracks the failure under login:attempts:${mobile} in Redis with a 15-minute expiry. The response returns: "Incorrect PIN entered. You have X attempts remaining." (e.g. 4, 3, 2, 1). If the user inputs the wrong PIN for the 5th consecutive time, a lock is activated in Redis for 15 minutes (login:lock:${mobile}). For the next 15 minutes, any login attempts will be immediately rejected with the message: "Too many incorrect PIN attempts. Your account is locked for 15 minutes." Upon entering the correct PIN, the attempts counter is immediately reset. 3. Forgot PIN flow with OTP Validation details: The system verifies if the mobile number is registered. If not, it returns: "Mobile number is not registered." If registered, it generates and sends a new OTP. Limitations: 60-Second Lock: You must wait at least 60 seconds before requesting another OTP. Any request within 60 seconds returns: "Please wait 60 seconds before requesting another OTP." Daily Limit: You can request a maximum of 10 OTPs per day. If you exceed 10 OTP requests, the system blocks further requests and returns: "Daily OTP limit reached. Please try again tomorrow." (This limit resets at midnight). 4. Logout System Flow: When a user logs out, the app sends a request to the server, clearing the user's active token. Fix Implemented: We fixed the duplicate key collision on logout. Multiple users can now log out simultaneously without getting blocked by the unique key constraint index users.idx_users_apptoken (the backend now updates apptoken and fcmtoken to NULL instead of ''). Run docker project docker compose up -d --build --scale app=2 --scale node-api=2 pre-load data docker exec -it html-node-api-1 node src/workers/terminal_cache_preloader.js