Commit graph

11 commits

Author SHA1 Message Date
Marko Djordjevic
73f44bf447 Add data migration script for user-accounts (task 2.5)
Create scripts/migrate-users.ts that:
- Creates a default admin user from DEFAULT_ADMIN_EMAIL/DEFAULT_ADMIN_PASSWORD env vars
- Backfills user_id on all existing rows in charts, annotations, annotation_types,
  span_annotations, span_label_types
- Is idempotent (safe to run multiple times)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 09:51:23 +01:00
Marko Djordjevic
d5fc4662e9 fix: replace all SQLite references with PostgreSQL in scripts and lazy-init db connection
- Rewrite scripts/run-migrations.js for PostgreSQL (was better-sqlite3)
- Rewrite scripts/load-initial-data.js for PostgreSQL (was better-sqlite3)
- Make db connection lazy in src/lib/db/index.ts to avoid build-time errors
  when DATABASE_URL is not available in Docker build stage
2026-02-17 23:48:47 +01:00
Marko Djordjevic
2481fda68c fix: remove all SQLite references (migrate.ts, migration script, package.json) 2026-02-17 23:34:12 +01:00
Marko Djordjevic
69634909d1 fix: correct timestamp/boolean types for PostgreSQL schema (Date not int, bool not 0/1) 2026-02-17 22:50:31 +01:00
Marko Djordjevic
bfe437857b feat: add Python migration script and successfully test SQLite to PostgreSQL data migration
- Created scripts/migrate-sqlite-to-postgres.py as alternative to TypeScript version
- Handles all type conversions: timestamps, booleans, and JSONB fields
- Successfully migrated all 2,836 rows from SQLite to PostgreSQL
- Verified data integrity: all 6 tables migrated correctly
- Charts: 1, Candles: 2,592, Annotations: 4, Span annotations: 223
2026-02-17 14:01:21 +01:00
Marko Djordjevic
2bde38d0bf fix: resolve database migration failures and startup ordering
- Add scripts/run-migrations.js to run migrations before data loading
- Fix startup.sh ordering: migrations -> data load -> app start
- Fix migration 0005 missing statement-breakpoint between ALTER TABLE statements
- Add migration 0005 to drizzle journal (was missing)
- Fix load-initial-data.js to check table existence before querying
- Fix load-initial-data.js to create chart record before inserting candles (chart_id NOT NULL constraint)
- Simplify db/index.ts migration error handling (remove overly broad 'already exists' catch)
- Add pre-migration check for inconsistent DB state (tables without migration tracking)
2026-02-16 19:59:47 +01:00
Marko Djordjevic
63486bc7b5 fix(ml): add CCI to hlc_indicators list
CCI (Commodity Channel Index) requires high, low, and close prices
2026-02-15 21:08:20 +01:00
Marko Djordjevic
57240d4eea fix(scripts): add created_at timestamps to annotation import
Set created_at field for both span_label_types and span_annotations
to satisfy NOT NULL constraint
2026-02-15 19:36:55 +01:00
Marko Djordjevic
847ff67986 feat(ml): add TA-Lib annotation generation and import workflow
Add complete workflow for using TA-Lib to bootstrap training data:

- generate_talib_annotations.py: Python script to run TA-Lib CDL* functions
  and output span annotations in UI-compatible format
- import_talib_annotations.ts: TypeScript script to import generated
  annotations into the UI database with auto-label-type creation
- npm script 'import-annotations' for easy execution
- TALIB_WORKFLOW.md: Comprehensive guide covering the full cycle:
  * Generate patterns with TA-Lib
  * Import into UI
  * Review and edit in browser
  * Export and train model
  * Compare predictions with TA-Lib detections
  * Iterate for improvement

This enables the intended workflow: use TA-Lib for initial annotations,
manually refine them, then train a model that learns from corrections.
2026-02-15 19:18:28 +01:00
Marko Djordjevic
dadf515406 feat: add database schema, migrations, and API endpoints for span annotations
- Add span_label_types and span_annotations tables to schema
- Seed default span label types (bull_flag, bear_flag, etc.)
- Implement CRUD API endpoints for span label types
- Implement CRUD API endpoints for span annotations
- Add time swap validation in POST endpoint (start_time <= end_time)
2026-02-14 05:56:28 +01:00
Marko Djordjevic
6522f501b6 feat: add EURUSD.csv data loading on Docker startup 2026-02-12 18:48:54 +01:00