Commit graph

150 commits

Author SHA1 Message Date
Marko Djordjevic
28725caaa8 fix: exclude scripts/ from TypeScript build to fix deployment 2026-02-17 23:29:52 +01:00
Marko Djordjevic
69634909d1 fix: correct timestamp/boolean types for PostgreSQL schema (Date not int, bool not 0/1) 2026-02-17 22:50:31 +01:00
Marko Djordjevic
e00bd4d804 fix: await params in DELETE route for Next.js 15 async params 2026-02-17 22:33:38 +01:00
Marko Djordjevic
6ef102cf21 fix: training panel stuck button, stale runs on startup, add delete model 2026-02-17 22:23:50 +01:00
Marko Djordjevic
d34dc9d729 fix: make sidebar scrollable so training/predictions panels are always visible 2026-02-17 22:10:54 +01:00
Marko Djordjevic
2faa8879f3 feat: add keyboard shortcuts R/S/L/D/T/? with shortcuts modal 2026-02-17 20:19:25 +01:00
Marko Djordjevic
028a3382d6 fix: pass confidence=null for talib span annotations (DB column is integer, not float) 2026-02-17 20:14:13 +01:00
Marko Djordjevic
d416aa409c fix: show TA-lib pattern spans on chart and add delete all annotations button
- Fix SpanAnnotationManager price range calculation: use direct candle
  filtering by time range instead of dummy Candle objects that fell back
  to 0/0 high/low, causing invisible rectangles
- Add handleDeleteAllAnnotations in page.tsx (deletes all span annotations
  and regular annotations for current chart)
- Add 'Delete All Annotations' button in sidebar below TA-Lib panel
2026-02-17 20:12:23 +01:00
Marko Djordjevic
7901a1a257 fix: use annotation's saved color for TrendLine instead of annotationType default color 2026-02-17 20:03:54 +01:00
Marko Djordjevic
c404e79678 fix: normalize span annotation timestamps to Unix epoch seconds in all API responses 2026-02-17 20:02:14 +01:00
Marko Djordjevic
900443d078 fix: convert Unix epoch seconds to Date for span annotation start_time/end_time on insert 2026-02-17 20:01:09 +01:00
Marko Djordjevic
4352732630 fix: change is_active type from number to boolean to match API response 2026-02-17 19:59:48 +01:00
Marko Djordjevic
a75c9307d4 feat: add Settings button in sidebar with links to annotation-types and span-label-types 2026-02-17 19:51:42 +01:00
Marko Djordjevic
3a114cccd0 fix: convert Unix epoch seconds to Date before inserting annotation timestamp 2026-02-17 19:46:24 +01:00
Marko Djordjevic
148fe22cd7 fix: guard crosshair move handler against re-entrant recursion from applyOptions 2026-02-17 19:44:37 +01:00
Marko Djordjevic
0ba3592679 fix: make Line tool button static next to Rect and Span 2026-02-17 19:42:21 +01:00
Marko Djordjevic
04a8303f4f fix: convert timestamp columns to Unix epoch seconds in candles and annotations API 2026-02-17 19:40:48 +01:00
Marko Djordjevic
a9bfe36e47 fix: handle test_ prefixed metric keys from training runs API 2026-02-17 19:00:22 +01:00
Marko Djordjevic
12a9603fce feat: add TalibPatternPanel, TrainingPanel, ModelSelector UI components (tasks 5-8)
- TalibPatternPanel: pattern checkboxes, detect button, results summary, clear-all and per-pattern delete
- TrainingPanel: model type selector, dataset info, start training, polling, run history
- ModelSelector: dropdown of completed runs, wired into PredictionPanel for model switching
- page.tsx: integrate all three panels into sidebar, wire callbacks (model load, annotations refresh)
- tasks.md: mark all 39 tasks complete
2026-02-17 18:55:52 +01:00
Marko Djordjevic
2a02669222 feat: add FastAPI model/load endpoint and all Next.js proxy routes (tasks 2-4) 2026-02-17 18:47:04 +01:00
Marko Djordjevic
b8e649e333 feat: add FastAPI pattern detection endpoints (Section 1)
- Extract CDL pattern detection logic into services/ml/app/patterns.py
  with TALIB_PATTERNS dict, get_available_patterns(), validate_pattern_names(),
  and detect_patterns(candles, pattern_names) functions
- Add GET /patterns/available endpoint returning all 54 supported CDL pattern
  names with display names
- Add POST /patterns/detect endpoint accepting {candles, patterns}, running
  selected CDL functions, returning span annotations with source "talib"
- Add input validation: reject invalid pattern names with HTTP 400,
  treat empty patterns list as "run all"

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 18:34:14 +01:00
Marko Djordjevic
38df874255 chore: archive ml-db-consolidation change and sync specs
- Archived change to openspec/changes/archive/2026-02-17-ml-db-consolidation/
- Created new postgres-data-layer spec with PostgreSQL connection, schema definitions, Drizzle migrations, npm deps, and SQLite migration requirements
- Updated docker-deployment spec: Docker Compose now PostgreSQL-based (postgres dependency, ml-data volume, DATABASE_URL); env vars updated (DATABASE_URL added, DATABASE_PATH removed); database persistence updated to PostgreSQL volumes; health check updated to PostgreSQL
- Updated ml-training spec: added database name scenario (candle_annotator) and new direct annotation data access requirement

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 18:22:28 +01:00
Marko Djordjevic
0e8dcc6707 chore: archive line-rectangle-annotations change and sync specs
- Archived change to openspec/changes/archive/2026-02-17-line-rectangle-annotations/
- Updated annotation-tools spec: added rectangle tool mode, TrendLine plugin rendering, line hit testing, line selection handles; updated line drawing and delete requirements; removed SVG overlay rendering
- Created new rectangle-annotation spec with full requirements for rectangle drawing, rendering, hit testing, selection, deletion, and database storage

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 18:16:49 +01:00
Marko Djordjevic
d1557a3846 fix: resolve numpy type conversion issues in ML service data access
- Convert numpy.int64 to Python int before passing to SQLAlchemy queries
- Prevents psycopg2.ProgrammingError: can't adapt type 'numpy.int64'
- Applied to get_candles(), get_span_annotations(), and get_point_annotations()
- All ML service database access tests now passing successfully
2026-02-17 14:10:21 +01:00
Marko Djordjevic
5377431c9d test: verify Next.js application works with PostgreSQL
- Updated .env to use DATABASE_URL instead of DATABASE_PATH
- Tested all API endpoints: health, charts, candles, span annotations
- Confirmed JSONB fields work correctly (geometry, sub_spans, model_prediction)
- All 2,836 rows accessible via API
- Database connection pooling working correctly
2026-02-17 14:02:22 +01:00
Marko Djordjevic
bfe437857b feat: add Python migration script and successfully test SQLite to PostgreSQL data migration
- Created scripts/migrate-sqlite-to-postgres.py as alternative to TypeScript version
- Handles all type conversions: timestamps, booleans, and JSONB fields
- Successfully migrated all 2,836 rows from SQLite to PostgreSQL
- Verified data integrity: all 6 tables migrated correctly
- Charts: 1, Candles: 2,592, Annotations: 4, Span annotations: 223
2026-02-17 14:01:21 +01:00
Marko Djordjevic
5f70f13da3 feat: migrate from SQLite to PostgreSQL - complete schema and API updates
- Remove better-sqlite3, add pg driver
- Convert schema to PostgreSQL types (serial, timestamp, boolean, jsonb)
- Generate fresh PostgreSQL migrations
- Update database connection layer with pg.Pool
- Fix all API routes: remove JSON.parse/stringify, use native timestamps and booleans
- Update drizzle.config.ts and .env.example for PostgreSQL
2026-02-17 13:43:06 +01:00
Marko Djordjevic
4605283d2b feat: redesign UI to match lovable compact sidebar layout
- Replace green hacker theme with professional blue-toned design
- Light theme default, manual toggle only (no system detection)
- Compact w-60 sidebar with collapsible sections
- New CSS tokens: sidebar, chart, candle, annotation colors
- Tools displayed as compact grid buttons
- Color swatches as inline bar
- Chart top bar with keyboard shortcut hints
- Inter + JetBrains Mono font pairing
- All components updated for compact styling
- Tailwind config extended with sidebar/chart tokens
2026-02-16 20:50:30 +01:00
Marko Djordjevic
2bde38d0bf fix: resolve database migration failures and startup ordering
- Add scripts/run-migrations.js to run migrations before data loading
- Fix startup.sh ordering: migrations -> data load -> app start
- Fix migration 0005 missing statement-breakpoint between ALTER TABLE statements
- Add migration 0005 to drizzle journal (was missing)
- Fix load-initial-data.js to check table existence before querying
- Fix load-initial-data.js to create chart record before inserting candles (chart_id NOT NULL constraint)
- Simplify db/index.ts migration error handling (remove overly broad 'already exists' catch)
- Add pre-migration check for inconsistent DB state (tables without migration tracking)
2026-02-16 19:59:47 +01:00
Marko Djordjevic
573efea5b5 fix: improve migration error handling to gracefully handle existing tables 2026-02-16 19:43:32 +01:00
Marko Djordjevic
bbaf330020 fix: skip migrations during build phase to prevent 'table already exists' errors 2026-02-16 19:41:22 +01:00
Marko Djordjevic
ecb23855b5 fix: add curl to ml-service for healthcheck 2026-02-16 17:31:08 +01:00
Marko Djordjevic
e48bff46cf docs: document RTK npm run bug workaround 2026-02-16 16:37:15 +01:00
Marko Djordjevic
511a557c42 add for docker until i fix it 2026-02-16 16:21:46 +01:00
Marko Djordjevic
23131b2c33 fix: resolve TypeScript narrowing errors and renamed lightweight-charts exports 2026-02-16 16:05:54 +01:00
Marko Djordjevic
efcf5c185f docs: document RTK bug with npm run command 2026-02-16 16:01:20 +01:00
Marko Djordjevic
025cf6d973 fix: resolve TypeScript error by renaming conflicting lineHit variable 2026-02-16 15:42:13 +01:00
Marko Djordjevic
08bd9625ae fix: build TA-Lib from source in ML Dockerfile 2026-02-16 14:58:07 +01:00
Marko Djordjevic
2cde02b722 docs: mark all line-rectangle-annotations tasks as complete 2026-02-16 12:14:22 +01:00
Marko Djordjevic
73e07c9050 feat: complete SVG overlay removal and line endpoint dragging (tasks 6.1-7.3) 2026-02-16 12:13:29 +01:00
Marko Djordjevic
aea1791122 feat: complete rectangle annotation tool (tasks 4.1-5.2)
- Add rectangle primitive management in CandleChart
- Handle chart switching with proper primitive cleanup
- Implement rectangle selection via hitTest
- Add rectangle deletion in delete tool
- Add rectangle tool button to Toolbox
- Wire rectangle tool with toggle behavior
2026-02-16 11:58:49 +01:00
Marko Djordjevic
82fd5ce819 feat: wire up drawing interaction for line and rectangle tools (tasks 3.1-3.5)
- Add drawing state and preview primitive refs
- Implement two-click drawing flow via subscribeClick
- Add crosshair move handler to update preview in real-time
- Add Escape key handler to cancel drawing
- Manage TrendLine primitives for saved line annotations
2026-02-16 11:55:06 +01:00
Marko Djordjevic
bec0aeb6ca feat: enhance TrendLine plugin and create RectangleDrawingPrimitive
- Add hitTest, setSelected, attached/detached lifecycle methods to TrendLine
- Add preview mode support with dashed lines and reduced opacity
- Draw selection handles on endpoints when selected
- Create RectangleDrawingPrimitive plugin with full ISeriesPrimitive implementation
- Support preview mode, selection, hit testing, and autoscaling for rectangles
- Set z-order to bottom for rectangles to render behind candlesticks

Tasks completed: 1.1-1.4, 2.1-2.7
2026-02-16 11:51:07 +01:00
Marko Djordjevic
28e3f83cf7 archive: candle-backend change complete 2026-02-16 11:44:53 +01:00
Marko Djordjevic
7e0579f65d sync: migrate delta specs to main openspec/specs
- Added 5 new capabilities: feature-engineering, annotation-ingestion, ml-training, ml-inference, prediction-ui
- Updated 2 existing capabilities: backend-api, span-annotation
- All specs synced from openspec/changes/candle-backend/specs/
2026-02-16 11:44:34 +01:00
Marko Djordjevic
65f00e6ce7 feat: complete prediction UI feedback tasks (11.2, 11.4, 11.5)
- Implement disagreement visual highlighting with distinct colors
  - Yellow highlight for 'missed_by_human' predictions
  - Orange for 'label_mismatch' disagreements
  - Warning icon on disagreement markers
- Add click-to-convert prediction feedback
  - Click disagreement predictions to create span annotations
  - Auto-fill with predicted label and times
  - Set source as 'model_confirmed' or 'model_corrected'
- Add dismiss action for false positive predictions
  - Alt+Click or Ctrl+Click to dismiss predictions
  - Saves negative annotation with label 'O'
  - Records original prediction in model_prediction field
- Filter predictions when 'Show only disagreements' is enabled
2026-02-16 11:40:55 +01:00
Marko Djordjevic
a18c6d110a Remove __pycache__ from git tracking 2026-02-15 22:29:35 +01:00
Marko Djordjevic
e3184ea86c save mlruns dir to git 2026-02-15 22:26:01 +01:00
Marko Djordjevic
f7b154f866 fix(ml): extract class labels from model when metadata is missing
The model info returned empty labels array because the pkl file has
no metadata dict. Now extracts labels from model.classes_ or
model.model.classes_ as fallback.
2026-02-15 22:18:16 +01:00
Marko Djordjevic
be09cef098 fix(ml): add missing numpy import in main.py 2026-02-15 22:08:41 +01:00