- Validate filename ends with .csv (case-insensitive)
- Validate MIME type is text/* or application/csv or text/csv
- Return HTTP 400 with error message if validation fails
- Mark task 2.4 as complete
- Reject uploads larger than 10MB before reading file content
- Reject CSVs with more than 500,000 data rows after parsing
- Checks placed as early as possible in the handler flow
- Mark task 2.3 as done in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add `import re` to services/ml/app/main.py
- In POST /model/load: validate run_id matches ^[a-zA-Z0-9_-]+$ before DB lookup; use Path.resolve() + directory containment check before loading model artifact
- In DELETE /training/runs/{run_id}: validate run_id matches ^[a-zA-Z0-9_-]+$ before any processing; use Path.resolve() + directory containment check before deleting model artifact
- Both endpoints return HTTP 400 with {"detail": "Invalid run_id format"} on invalid input
- Mark task 2.2 as completed in openspec/changes/code-review-fix/tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Validate that run_id matches /^[a-zA-Z0-9_-]+$ regex before interpolating into the API URL.
Returns HTTP 400 with 'Invalid run_id format' error if validation fails.
This prevents potential URL injection attacks and invalid identifier usage.
Changes:
- Updated docker-compose.yml MLflow service port binding from 5000:5000 to 127.0.0.1:5000:5000
to restrict access to localhost only for security
- Marked task 1.7 as complete in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Changed PostgreSQL service port binding from 5432:5432 to 127.0.0.1:5432:5432 in docker-compose.yml
- This restricts PostgreSQL to listen only on localhost, improving security by preventing access from other interfaces
- Marked task 1.6 as completed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All DATABASE_URL values and postgres service env vars now use
\${POSTGRES_USER}, \${POSTGRES_PASSWORD}, \${POSTGRES_DB} interpolation
instead of hardcoded ml_user/ml_password/candle_annotator values.
Also updated pg_isready healthcheck to use the same env vars.
Closes task 1.5.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove hardcoded SQL comments containing 'ml_user' and 'ml_password'
- Remove fallback default credentials in DATABASE_URL construction
- Add fail-fast validation: raise RuntimeError if DATABASE_URL env var is missing or empty
- Mark task 1.4 as complete in code-review-fix/tasks.md
- Replace ml_password with change_me_to_a_strong_password placeholder
- Replace ml_user with your_db_user placeholder
- Mark task 1.3 as completed in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add build_dataset_from_db() that exports candles from DB, runs feature
engineering, and ingests span annotations into labeled CSV
- Call it automatically in _run_training_background before training starts
- Add POST /training/build-dataset endpoint for standalone use
- Add Next.js proxy route /api/training/build-dataset
- Update TrainingPanel: remove dataset-missing block on Start Training,
show informational message that dataset builds automatically
- Rewrite scripts/run-migrations.js for PostgreSQL (was better-sqlite3)
- Rewrite scripts/load-initial-data.js for PostgreSQL (was better-sqlite3)
- Make db connection lazy in src/lib/db/index.ts to avoid build-time errors
when DATABASE_URL is not available in Docker build stage
- Fix SpanAnnotationManager price range calculation: use direct candle
filtering by time range instead of dummy Candle objects that fell back
to 0/0 high/low, causing invisible rectangles
- Add handleDeleteAllAnnotations in page.tsx (deletes all span annotations
and regular annotations for current chart)
- Add 'Delete All Annotations' button in sidebar below TA-Lib panel
- TalibPatternPanel: pattern checkboxes, detect button, results summary, clear-all and per-pattern delete
- TrainingPanel: model type selector, dataset info, start training, polling, run history
- ModelSelector: dropdown of completed runs, wired into PredictionPanel for model switching
- page.tsx: integrate all three panels into sidebar, wire callbacks (model load, annotations refresh)
- tasks.md: mark all 39 tasks complete
- Archived change to openspec/changes/archive/2026-02-17-ml-db-consolidation/
- Created new postgres-data-layer spec with PostgreSQL connection, schema definitions, Drizzle migrations, npm deps, and SQLite migration requirements
- Updated docker-deployment spec: Docker Compose now PostgreSQL-based (postgres dependency, ml-data volume, DATABASE_URL); env vars updated (DATABASE_URL added, DATABASE_PATH removed); database persistence updated to PostgreSQL volumes; health check updated to PostgreSQL
- Updated ml-training spec: added database name scenario (candle_annotator) and new direct annotation data access requirement
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Archived change to openspec/changes/archive/2026-02-17-line-rectangle-annotations/
- Updated annotation-tools spec: added rectangle tool mode, TrendLine plugin rendering, line hit testing, line selection handles; updated line drawing and delete requirements; removed SVG overlay rendering
- Created new rectangle-annotation spec with full requirements for rectangle drawing, rendering, hit testing, selection, deletion, and database storage
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Convert numpy.int64 to Python int before passing to SQLAlchemy queries
- Prevents psycopg2.ProgrammingError: can't adapt type 'numpy.int64'
- Applied to get_candles(), get_span_annotations(), and get_point_annotations()
- All ML service database access tests now passing successfully
- Updated .env to use DATABASE_URL instead of DATABASE_PATH
- Tested all API endpoints: health, charts, candles, span annotations
- Confirmed JSONB fields work correctly (geometry, sub_spans, model_prediction)
- All 2,836 rows accessible via API
- Database connection pooling working correctly
- Created scripts/migrate-sqlite-to-postgres.py as alternative to TypeScript version
- Handles all type conversions: timestamps, booleans, and JSONB fields
- Successfully migrated all 2,836 rows from SQLite to PostgreSQL
- Verified data integrity: all 6 tables migrated correctly
- Charts: 1, Candles: 2,592, Annotations: 4, Span annotations: 223
- Replace green hacker theme with professional blue-toned design
- Light theme default, manual toggle only (no system detection)
- Compact w-60 sidebar with collapsible sections
- New CSS tokens: sidebar, chart, candle, annotation colors
- Tools displayed as compact grid buttons
- Color swatches as inline bar
- Chart top bar with keyboard shortcut hints
- Inter + JetBrains Mono font pairing
- All components updated for compact styling
- Tailwind config extended with sidebar/chart tokens
- Add scripts/run-migrations.js to run migrations before data loading
- Fix startup.sh ordering: migrations -> data load -> app start
- Fix migration 0005 missing statement-breakpoint between ALTER TABLE statements
- Add migration 0005 to drizzle journal (was missing)
- Fix load-initial-data.js to check table existence before querying
- Fix load-initial-data.js to create chart record before inserting candles (chart_id NOT NULL constraint)
- Simplify db/index.ts migration error handling (remove overly broad 'already exists' catch)
- Add pre-migration check for inconsistent DB state (tables without migration tracking)
