Commit graph

384 commits

Author SHA1 Message Date
Marko Djordjevic
15adf09b73 fix: add parseInt(value, 10) with isNaN() guards to all integer query param parsing
- Add radix 10 to all parseInt() calls parsing integer query/path parameters
- Add isNaN() guards returning HTTP 400 with descriptive error messages
- Updated routes: annotations, candles, export, export/spans, annotation-types/[id], span-annotations, span-annotations/[id], span-label-types/[id]
- Ensures strict integer parsing and prevents invalid parameter values from reaching database queries
2026-02-18 11:19:26 +01:00
Marko Djordjevic
1678da2d9d fix: wrap chart cascade delete in db.transaction() and add spanAnnotations deletion
- Import spanAnnotations from schema
- Wrap all delete operations in db.transaction() for atomicity
- Delete spanAnnotations first to satisfy FK constraints, then annotations, candles, chart
- Mark task 4.8 as done in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:17:19 +01:00
Marko Djordjevic
103bfa89cb fix: require chartId for bulk delete in annotations route (task 4.7)
Reject DELETE ?all=true without chartId with HTTP 400 to prevent
accidental deletion of annotations across all charts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:16:37 +01:00
Marko Djordjevic
aace19b7f4 fix: replace error.message with generic "Internal server error" in all API catch blocks
Prevents leaking internal error details to clients across 7 route files:
health, candles, annotations, annotations/[id], upload, export, span-annotations/export.
Server-side console.error logging preserved for debugging.

Closes task 4.6.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:16:02 +01:00
Marko Djordjevic
81e3554d82 feat: add Zod schema validation to patterns/detect route
- Import z from zod
- Add CandleSchema validating time, open, high, low, close (number), volume (optional number)
- Add PatternDetectRequestSchema validating candles array and patterns array of non-empty strings
- Use safeParse() and return HTTP 400 with error details on validation failure
- Forward only validated data to the inference service
- Mark task 4.5 as completed in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:14:36 +01:00
Marko Djordjevic
2e02d155af feat: add Zod schema validation to training/start route (task 4.4)
Validates model_type as a non-empty string using .safeParse(); returns
HTTP 400 with error details on invalid input. Marks task 4.4 as done.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:14:00 +01:00
Marko Djordjevic
4cffc223b3 feat: add Zod schema validation to model/load route
Validate run_id in POST /api/model/load using Zod:
- run_id must be a non-empty string matching /^[a-zA-Z0-9_-]+$/
- Returns HTTP 400 with error details if validation fails
- Validated data is forwarded to the inference service

Marks task 4.3 as complete in tasks.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:13:13 +01:00
Marko Djordjevic
5c399037c3 feat: add Zod validation to predict/batch route (task 4.2)
Add BatchPredictRequestSchema with Zod to validate pair, timeframe,
start_date, and end_date fields. Returns HTTP 400 with flattened error
details on invalid input. Forward only validated data to the inference
service.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:12:38 +01:00
Marko Djordjevic
3361236d3f feat: add Zod schema validation to predict API route
- Add CandleSchema validating time, open, high, low, close (number) and optional volume
- Add PredictRequestSchema validating pair (non-empty string), timeframe (non-empty string), candles array
- Use safeParse() and return HTTP 400 with error details on invalid input
- Forward only validated data to the inference service
- Mark task 4.1 as done in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:11:58 +01:00
Marko Djordjevic
c023702644 feat: add API_KEY to .env.example with placeholder and instructions
- Add API_KEY environment variable with placeholder value 'change_me_to_a_strong_random_key'
- Include helpful comment explaining its purpose: authentication between Next.js and ML service
- Provide command for generating strong random value: openssl rand -hex 32
- Mark task 3.4 as completed
2026-02-18 11:06:47 +01:00
Marko Djordjevic
4a3e4a48ba feat: forward X-API-Key header from Next.js proxy routes to ML service
All 12 Next.js API routes that proxy requests to the ML service
(INFERENCE_API_URL / localhost:8001) now include the X-API-Key header
read from process.env.API_KEY. Affected routes:
- /api/predict
- /api/predict/batch
- /api/model/info
- /api/model/load
- /api/training/start
- /api/training/runs
- /api/training/runs/[run_id] (DELETE)
- /api/training/dataset-info
- /api/training/active
- /api/training/build-dataset
- /api/patterns/available
- /api/patterns/detect

Marks task 3.3 as complete in openspec/changes/code-review-fix/tasks.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:06:18 +01:00
Marko Djordjevic
5f569d9134 feat(ml): add API key authentication via FastAPI Depends() on all endpoints except /health
- Import Header, Depends, Security from fastapi
- Add verify_api_key dependency: reads API_KEY env var, checks X-API-Key
  header, raises HTTP 401 if key mismatch; fail-open if env var not set
- Apply Depends(verify_api_key) to all 14 non-health endpoints
- /health endpoint remains unauthenticated for liveness probes
- Mark task 3.2 as complete in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:04:41 +01:00
Marko Djordjevic
577bb2e56e feat: add API key auth middleware for /api/* routes (task 3.1)
- Create src/middleware.ts with Next.js middleware
- Reads API_KEY env var and checks X-API-Key header on all /api/* routes
- Skips auth for /api/health endpoint
- Fails open (with warning) when API_KEY is not configured
- Returns 401 Unauthorized when key is missing or mismatched
- Mark task 3.1 as complete in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:02:51 +01:00
Marko Djordjevic
0cd21887e4 mark: task 2.5 complete (CORS configuration) 2026-02-18 11:02:10 +01:00
Marko Djordjevic
26ff80a682 fix: implement CORS origins configuration via environment variable
- Replace hardcoded allow_origins=['*'] with dynamic configuration
- Read CORS_ORIGINS environment variable (comma-separated list)
- Default to 'http://localhost:3000' if CORS_ORIGINS is not set
- Support multiple origins by splitting and stripping whitespace from env var
2026-02-18 11:02:00 +01:00
Marko Djordjevic
94bc5768d1 feat: add file type validation to upload endpoint
- Validate filename ends with .csv (case-insensitive)
- Validate MIME type is text/* or application/csv or text/csv
- Return HTTP 400 with error message if validation fails
- Mark task 2.4 as complete
2026-02-18 11:01:28 +01:00
Marko Djordjevic
0e239dc3da security: add file size (10MB) and row count (500k) limits to upload route
- Reject uploads larger than 10MB before reading file content
- Reject CSVs with more than 500,000 data rows after parsing
- Checks placed as early as possible in the handler flow
- Mark task 2.3 as done in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:01:02 +01:00
Marko Djordjevic
67dd7aa2f0 security: validate run_id format and add path containment check in ML service
- Add `import re` to services/ml/app/main.py
- In POST /model/load: validate run_id matches ^[a-zA-Z0-9_-]+$ before DB lookup; use Path.resolve() + directory containment check before loading model artifact
- In DELETE /training/runs/{run_id}: validate run_id matches ^[a-zA-Z0-9_-]+$ before any processing; use Path.resolve() + directory containment check before deleting model artifact
- Both endpoints return HTTP 400 with {"detail": "Invalid run_id format"} on invalid input
- Mark task 2.2 as completed in openspec/changes/code-review-fix/tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:00:19 +01:00
Marko Djordjevic
870f92d208 feat: add run_id format validation in DELETE training/runs endpoint
Validate that run_id matches /^[a-zA-Z0-9_-]+$ regex before interpolating into the API URL.
Returns HTTP 400 with 'Invalid run_id format' error if validation fails.
This prevents potential URL injection attacks and invalid identifier usage.
2026-02-18 10:58:54 +01:00
Marko Djordjevic
4e5ce321b9 chore: bind ML service port to 127.0.0.1:8001:8001 for localhost-only access
- Changed ML service port binding from 8001:8001 to 127.0.0.1:8001:8001 in docker-compose.yml
- Marks task 1.8 as complete in tasks.md
2026-02-18 10:58:31 +01:00
Marko Djordjevic
c327ba3370 bind: MLflow port to 127.0.0.1:5000:5000 in docker-compose.yml
Changes:
- Updated docker-compose.yml MLflow service port binding from 5000:5000 to 127.0.0.1:5000:5000
  to restrict access to localhost only for security
- Marked task 1.7 as complete in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 10:58:11 +01:00
Marko Djordjevic
9efa1dbbcc fix: Bind PostgreSQL port to 127.0.0.1:5432:5432 for localhost-only access
- Changed PostgreSQL service port binding from 5432:5432 to 127.0.0.1:5432:5432 in docker-compose.yml
- This restricts PostgreSQL to listen only on localhost, improving security by preventing access from other interfaces
- Marked task 1.6 as completed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 10:57:55 +01:00
Marko Djordjevic
e3469ec39f fix: replace hardcoded DB credentials with env var interpolation in docker-compose.yml
All DATABASE_URL values and postgres service env vars now use
\${POSTGRES_USER}, \${POSTGRES_PASSWORD}, \${POSTGRES_DB} interpolation
instead of hardcoded ml_user/ml_password/candle_annotator values.
Also updated pg_isready healthcheck to use the same env vars.

Closes task 1.5.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 10:57:31 +01:00
Marko Djordjevic
9bc82b822c security: remove credential SQL comments and add DATABASE_URL fail-fast check
- Remove hardcoded SQL comments containing 'ml_user' and 'ml_password'
- Remove fallback default credentials in DATABASE_URL construction
- Add fail-fast validation: raise RuntimeError if DATABASE_URL env var is missing or empty
- Mark task 1.4 as complete in code-review-fix/tasks.md
2026-02-18 10:56:49 +01:00
Marko Djordjevic
55ee9c936a fix: replace real credentials in .env.example with placeholders
- Replace ml_password with change_me_to_a_strong_password placeholder
- Replace ml_user with your_db_user placeholder
- Mark task 1.3 as completed in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 10:56:23 +01:00
Marko Djordjevic
099f334fe9 chore: mark task 1.2 as completed 2026-02-18 10:55:56 +01:00
Marko Djordjevic
ea89543c0f feat: add models/ and *.pkl to .gitignore, remove tracked model files from git history 2026-02-18 10:55:48 +01:00
Marko Djordjevic
4ba1327a53 task 1.1: add .env to .gitignore and untrack from git 2026-02-18 10:55:24 +01:00
Marko Djordjevic
c0f5654450 sync: ml-ui-connection delta specs to main specs 2026-02-18 10:21:05 +01:00
Marko Djordjevic
9a549b8c7a fix: make volume column optional in feature engineering, skip MFI when absent 2026-02-18 01:01:32 +01:00
Marko Djordjevic
d3dcfcea7d feat: auto-build training dataset from DB annotations before training
- Add build_dataset_from_db() that exports candles from DB, runs feature
  engineering, and ingests span annotations into labeled CSV
- Call it automatically in _run_training_background before training starts
- Add POST /training/build-dataset endpoint for standalone use
- Add Next.js proxy route /api/training/build-dataset
- Update TrainingPanel: remove dataset-missing block on Start Training,
  show informational message that dataset builds automatically
2026-02-18 00:24:39 +01:00
Marko Djordjevic
b4956f3fb9 fix: call init_db() on ML service startup to create training_runs table 2026-02-18 00:08:23 +01:00
Marko Djordjevic
d5fc4662e9 fix: replace all SQLite references with PostgreSQL in scripts and lazy-init db connection
- Rewrite scripts/run-migrations.js for PostgreSQL (was better-sqlite3)
- Rewrite scripts/load-initial-data.js for PostgreSQL (was better-sqlite3)
- Make db connection lazy in src/lib/db/index.ts to avoid build-time errors
  when DATABASE_URL is not available in Docker build stage
2026-02-17 23:48:47 +01:00
Marko Djordjevic
2481fda68c fix: remove all SQLite references (migrate.ts, migration script, package.json) 2026-02-17 23:34:12 +01:00
Marko Djordjevic
28725caaa8 fix: exclude scripts/ from TypeScript build to fix deployment 2026-02-17 23:29:52 +01:00
Marko Djordjevic
69634909d1 fix: correct timestamp/boolean types for PostgreSQL schema (Date not int, bool not 0/1) 2026-02-17 22:50:31 +01:00
Marko Djordjevic
e00bd4d804 fix: await params in DELETE route for Next.js 15 async params 2026-02-17 22:33:38 +01:00
Marko Djordjevic
6ef102cf21 fix: training panel stuck button, stale runs on startup, add delete model 2026-02-17 22:23:50 +01:00
Marko Djordjevic
d34dc9d729 fix: make sidebar scrollable so training/predictions panels are always visible 2026-02-17 22:10:54 +01:00
Marko Djordjevic
2faa8879f3 feat: add keyboard shortcuts R/S/L/D/T/? with shortcuts modal 2026-02-17 20:19:25 +01:00
Marko Djordjevic
028a3382d6 fix: pass confidence=null for talib span annotations (DB column is integer, not float) 2026-02-17 20:14:13 +01:00
Marko Djordjevic
d416aa409c fix: show TA-lib pattern spans on chart and add delete all annotations button
- Fix SpanAnnotationManager price range calculation: use direct candle
  filtering by time range instead of dummy Candle objects that fell back
  to 0/0 high/low, causing invisible rectangles
- Add handleDeleteAllAnnotations in page.tsx (deletes all span annotations
  and regular annotations for current chart)
- Add 'Delete All Annotations' button in sidebar below TA-Lib panel
2026-02-17 20:12:23 +01:00
Marko Djordjevic
7901a1a257 fix: use annotation's saved color for TrendLine instead of annotationType default color 2026-02-17 20:03:54 +01:00
Marko Djordjevic
c404e79678 fix: normalize span annotation timestamps to Unix epoch seconds in all API responses 2026-02-17 20:02:14 +01:00
Marko Djordjevic
900443d078 fix: convert Unix epoch seconds to Date for span annotation start_time/end_time on insert 2026-02-17 20:01:09 +01:00
Marko Djordjevic
4352732630 fix: change is_active type from number to boolean to match API response 2026-02-17 19:59:48 +01:00
Marko Djordjevic
a75c9307d4 feat: add Settings button in sidebar with links to annotation-types and span-label-types 2026-02-17 19:51:42 +01:00
Marko Djordjevic
3a114cccd0 fix: convert Unix epoch seconds to Date before inserting annotation timestamp 2026-02-17 19:46:24 +01:00
Marko Djordjevic
148fe22cd7 fix: guard crosshair move handler against re-entrant recursion from applyOptions 2026-02-17 19:44:37 +01:00
Marko Djordjevic
0ba3592679 fix: make Line tool button static next to Rect and Span 2026-02-17 19:42:21 +01:00