All 12 Next.js API routes that proxy requests to the ML service
(INFERENCE_API_URL / localhost:8001) now include the X-API-Key header
read from process.env.API_KEY. Affected routes:
- /api/predict
- /api/predict/batch
- /api/model/info
- /api/model/load
- /api/training/start
- /api/training/runs
- /api/training/runs/[run_id] (DELETE)
- /api/training/dataset-info
- /api/training/active
- /api/training/build-dataset
- /api/patterns/available
- /api/patterns/detect
Marks task 3.3 as complete in openspec/changes/code-review-fix/tasks.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Import Header, Depends, Security from fastapi
- Add verify_api_key dependency: reads API_KEY env var, checks X-API-Key
header, raises HTTP 401 if key mismatch; fail-open if env var not set
- Apply Depends(verify_api_key) to all 14 non-health endpoints
- /health endpoint remains unauthenticated for liveness probes
- Mark task 3.2 as complete in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Create src/middleware.ts with Next.js middleware
- Reads API_KEY env var and checks X-API-Key header on all /api/* routes
- Skips auth for /api/health endpoint
- Fails open (with warning) when API_KEY is not configured
- Returns 401 Unauthorized when key is missing or mismatched
- Mark task 3.1 as complete in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace hardcoded allow_origins=['*'] with dynamic configuration
- Read CORS_ORIGINS environment variable (comma-separated list)
- Default to 'http://localhost:3000' if CORS_ORIGINS is not set
- Support multiple origins by splitting and stripping whitespace from env var
- Validate filename ends with .csv (case-insensitive)
- Validate MIME type is text/* or application/csv or text/csv
- Return HTTP 400 with error message if validation fails
- Mark task 2.4 as complete
- Reject uploads larger than 10MB before reading file content
- Reject CSVs with more than 500,000 data rows after parsing
- Checks placed as early as possible in the handler flow
- Mark task 2.3 as done in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add `import re` to services/ml/app/main.py
- In POST /model/load: validate run_id matches ^[a-zA-Z0-9_-]+$ before DB lookup; use Path.resolve() + directory containment check before loading model artifact
- In DELETE /training/runs/{run_id}: validate run_id matches ^[a-zA-Z0-9_-]+$ before any processing; use Path.resolve() + directory containment check before deleting model artifact
- Both endpoints return HTTP 400 with {"detail": "Invalid run_id format"} on invalid input
- Mark task 2.2 as completed in openspec/changes/code-review-fix/tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Validate that run_id matches /^[a-zA-Z0-9_-]+$ regex before interpolating into the API URL.
Returns HTTP 400 with 'Invalid run_id format' error if validation fails.
This prevents potential URL injection attacks and invalid identifier usage.
Changes:
- Updated docker-compose.yml MLflow service port binding from 5000:5000 to 127.0.0.1:5000:5000
to restrict access to localhost only for security
- Marked task 1.7 as complete in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Changed PostgreSQL service port binding from 5432:5432 to 127.0.0.1:5432:5432 in docker-compose.yml
- This restricts PostgreSQL to listen only on localhost, improving security by preventing access from other interfaces
- Marked task 1.6 as completed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All DATABASE_URL values and postgres service env vars now use
\${POSTGRES_USER}, \${POSTGRES_PASSWORD}, \${POSTGRES_DB} interpolation
instead of hardcoded ml_user/ml_password/candle_annotator values.
Also updated pg_isready healthcheck to use the same env vars.
Closes task 1.5.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove hardcoded SQL comments containing 'ml_user' and 'ml_password'
- Remove fallback default credentials in DATABASE_URL construction
- Add fail-fast validation: raise RuntimeError if DATABASE_URL env var is missing or empty
- Mark task 1.4 as complete in code-review-fix/tasks.md
- Replace ml_password with change_me_to_a_strong_password placeholder
- Replace ml_user with your_db_user placeholder
- Mark task 1.3 as completed in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add build_dataset_from_db() that exports candles from DB, runs feature
engineering, and ingests span annotations into labeled CSV
- Call it automatically in _run_training_background before training starts
- Add POST /training/build-dataset endpoint for standalone use
- Add Next.js proxy route /api/training/build-dataset
- Update TrainingPanel: remove dataset-missing block on Start Training,
show informational message that dataset builds automatically
- Rewrite scripts/run-migrations.js for PostgreSQL (was better-sqlite3)
- Rewrite scripts/load-initial-data.js for PostgreSQL (was better-sqlite3)
- Make db connection lazy in src/lib/db/index.ts to avoid build-time errors
when DATABASE_URL is not available in Docker build stage
- Fix SpanAnnotationManager price range calculation: use direct candle
filtering by time range instead of dummy Candle objects that fell back
to 0/0 high/low, causing invisible rectangles
- Add handleDeleteAllAnnotations in page.tsx (deletes all span annotations
and regular annotations for current chart)
- Add 'Delete All Annotations' button in sidebar below TA-Lib panel
- TalibPatternPanel: pattern checkboxes, detect button, results summary, clear-all and per-pattern delete
- TrainingPanel: model type selector, dataset info, start training, polling, run history
- ModelSelector: dropdown of completed runs, wired into PredictionPanel for model switching
- page.tsx: integrate all three panels into sidebar, wire callbacks (model load, annotations refresh)
- tasks.md: mark all 39 tasks complete
- Archived change to openspec/changes/archive/2026-02-17-ml-db-consolidation/
- Created new postgres-data-layer spec with PostgreSQL connection, schema definitions, Drizzle migrations, npm deps, and SQLite migration requirements
- Updated docker-deployment spec: Docker Compose now PostgreSQL-based (postgres dependency, ml-data volume, DATABASE_URL); env vars updated (DATABASE_URL added, DATABASE_PATH removed); database persistence updated to PostgreSQL volumes; health check updated to PostgreSQL
- Updated ml-training spec: added database name scenario (candle_annotator) and new direct annotation data access requirement
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Archived change to openspec/changes/archive/2026-02-17-line-rectangle-annotations/
- Updated annotation-tools spec: added rectangle tool mode, TrendLine plugin rendering, line hit testing, line selection handles; updated line drawing and delete requirements; removed SVG overlay rendering
- Created new rectangle-annotation spec with full requirements for rectangle drawing, rendering, hit testing, selection, deletion, and database storage
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Convert numpy.int64 to Python int before passing to SQLAlchemy queries
- Prevents psycopg2.ProgrammingError: can't adapt type 'numpy.int64'
- Applied to get_candles(), get_span_annotations(), and get_point_annotations()
- All ML service database access tests now passing successfully
- Updated .env to use DATABASE_URL instead of DATABASE_PATH
- Tested all API endpoints: health, charts, candles, span annotations
- Confirmed JSONB fields work correctly (geometry, sub_spans, model_prediction)
- All 2,836 rows accessible via API
- Database connection pooling working correctly
- Created scripts/migrate-sqlite-to-postgres.py as alternative to TypeScript version
- Handles all type conversions: timestamps, booleans, and JSONB fields
- Successfully migrated all 2,836 rows from SQLite to PostgreSQL
- Verified data integrity: all 6 tables migrated correctly
- Charts: 1, Candles: 2,592, Annotations: 4, Span annotations: 223
