- Replace green hacker theme with professional blue-toned design
- Light theme default, manual toggle only (no system detection)
- Compact w-60 sidebar with collapsible sections
- New CSS tokens: sidebar, chart, candle, annotation colors
- Tools displayed as compact grid buttons
- Color swatches as inline bar
- Chart top bar with keyboard shortcut hints
- Inter + JetBrains Mono font pairing
- All components updated for compact styling
- Tailwind config extended with sidebar/chart tokens
- Add scripts/run-migrations.js to run migrations before data loading
- Fix startup.sh ordering: migrations -> data load -> app start
- Fix migration 0005 missing statement-breakpoint between ALTER TABLE statements
- Add migration 0005 to drizzle journal (was missing)
- Fix load-initial-data.js to check table existence before querying
- Fix load-initial-data.js to create chart record before inserting candles (chart_id NOT NULL constraint)
- Simplify db/index.ts migration error handling (remove overly broad 'already exists' catch)
- Add pre-migration check for inconsistent DB state (tables without migration tracking)
- Add hitTest, setSelected, attached/detached lifecycle methods to TrendLine
- Add preview mode support with dashed lines and reduced opacity
- Draw selection handles on endpoints when selected
- Create RectangleDrawingPrimitive plugin with full ISeriesPrimitive implementation
- Support preview mode, selection, hit testing, and autoscaling for rectangles
- Set z-order to bottom for rectangles to render behind candlesticks
Tasks completed: 1.1-1.4, 2.1-2.7
- Implement disagreement visual highlighting with distinct colors
- Yellow highlight for 'missed_by_human' predictions
- Orange for 'label_mismatch' disagreements
- Warning icon on disagreement markers
- Add click-to-convert prediction feedback
- Click disagreement predictions to create span annotations
- Auto-fill with predicted label and times
- Set source as 'model_confirmed' or 'model_corrected'
- Add dismiss action for false positive predictions
- Alt+Click or Ctrl+Click to dismiss predictions
- Saves negative annotation with label 'O'
- Records original prediction in model_prediction field
- Filter predictions when 'Show only disagreements' is enabled
The model info returned empty labels array because the pkl file has
no metadata dict. Now extracts labels from model.classes_ or
model.model.classes_ as fallback.
The model was trained on 94-candle sliding windows flattened to 2820
features (94 candles x 30 features). Inference was sending raw per-candle
features (27 columns).
Changes:
- Rewrite preprocessing to return (X, window_times) tuple
- Add sliding window creation with correct feature ordering
- Fill missing columns (average, barCount) with 0 for feature parity
- Fill NaN from indicator warmup with 0 instead of dropping rows
- Always compute all indicators (including MFI) for feature parity
- Update predict and batch predict endpoints for new signature
The Pydantic model sets volume=None when absent, creating an all-NaN
column rather than a missing column. Check isna().all() in addition
to column existence.
- Fill volume with 0 when column is absent from candle data
- Skip MFI/OBV/AD/ADOSC indicators when no real volume data available
- Fix pandas FutureWarning for inplace fillna in candle_features
- Remove temporary debug NaN logging
- Add GET handler to /api/charts/[id] route to fetch chart metadata
- Fix batch prediction to use regular /predict endpoint with database candles
- Remove /predict/batch usage (was designed for file-based predictions)
- Make volume field optional in CandleData model (database candles don't have volume)
- Convert timestamps to ISO dates for batch requests
Known issue: TA-Lib indicators failing with 'input array type is not double'
- May need to ensure candle data is float64/double type before processing
- Change pair and timeframe fields from required to optional
- Frontend only sends candles array, not pair/timeframe metadata
- These fields are only used for logging, not prediction logic
- Update logging to handle None values with 'unknown' fallback
- Fixes 422 validation error on /predict endpoint
- Update TypeScript types to match flat backend response structure
- Remove nested model_info and metrics objects
- Remove label_config, use labels array and per_class_metrics array
- Update all component references to use new structure
- Generate default colors for prediction labels in CandleChart
- Fix TypeScript type errors for nullable model_version
- Remove accuracy/F1 metrics display (not in new response)
- Fix CCI indicator to use HLC prices instead of close only
- Parse datetime column when loading enriched CSV
- Strip timezone from annotation timestamps
- Fix TA-Lib pattern names (CDL3WHITESOLDIERS, CDL3BLACKCROWS)
- Exclude programmatic label columns from training features
- Fix classification report to handle missing classes
- Update MLflow tracking to use localhost:5000
- Grant PostgreSQL permissions to ml_user
Pipeline now runs successfully end-to-end:
- Feature engineering: 2543 rows, 31 columns
- Annotation ingestion: 286 samples
- Training: 89.47% test accuracy with Random Forest
- Convert date strings to Unix timestamps in load_ohlcv()
- Fix duplicate pattern names (CDL3WHITESOLDIERS/CDL3BLACKCROWS)
- Ensure time column is always integer type
Add complete workflow for using TA-Lib to bootstrap training data:
- generate_talib_annotations.py: Python script to run TA-Lib CDL* functions
and output span annotations in UI-compatible format
- import_talib_annotations.ts: TypeScript script to import generated
annotations into the UI database with auto-label-type creation
- npm script 'import-annotations' for easy execution
- TALIB_WORKFLOW.md: Comprehensive guide covering the full cycle:
* Generate patterns with TA-Lib
* Import into UI
* Review and edit in browser
* Export and train model
* Compare predictions with TA-Lib detections
* Iterate for improvement
This enables the intended workflow: use TA-Lib for initial annotations,
manually refine them, then train a model that learns from corrections.
Complete step-by-step guide covering:
- Creating training data through annotations
- Exporting annotations and candle data
- Running the full ML pipeline
- Verifying model creation and loading
- Getting predictions in the UI
- Troubleshooting common issues
- Iteration through active learning loop
- Add disagreement detection logic comparing human annotations vs predictions
- Display prediction summary in PredictionPanel (agreements/disagreements)
- Wire up 'Show only disagreements' filter toggle
- Add loading overlay during prediction fetching
- Update docker-compose.yml with healthchecks for all services
- Update DEPLOYMENT.md with comprehensive ML service setup instructions
- Update README.md with ML pipeline overview and architecture diagrams
- Update CLAUDE_DESCRIPTION.md with v3.0.0 ML integration details
Remaining tasks (11.2, 11.4, 11.5) deferred - core functionality complete
- Add histogram series to CandleChart for per-bar prediction colors (15% opacity)
- Add series markers showing label name and confidence % at prediction span starts
- Implement confidence threshold filtering for both histogram and markers
- Implement label type filtering from PredictionPanel checkboxes
- Implement prediction layer visibility toggle (show/hide)
- Add getVisibleCandles method to CandleChartHandle for on-demand prediction fetching
- Pass prediction state props from page.tsx to CandleChart
Tasks 10.1-10.5 complete.
- Create prediction type definitions in src/types/predictions.ts
- Add prediction state management to page.tsx with caching
- Implement PredictionPanel component with:
- Master visibility toggle
- Model info display (name, version, type, metrics)
- Action buttons (Run on Visible, Predict All)
- Confidence threshold slider
- Label filter checkboxes with per-class metrics
- Disagreement filter toggle
- Prediction summary display
- Model server offline banner
- Add on-demand and batch prediction fetching
- Implement prediction caching by chart and model version
- Add health polling for inference API (30s interval when offline)
- Ensure annotation tools work independently of prediction API
Tasks completed: 9.1-9.5, 12.1-12.3 (59/78 total)
- Add GET /api/span-annotations/export endpoint for ML pipeline JSON/CSV export
- Add source and model_prediction fields to span_annotations schema
- Update POST endpoint to accept source (human/model/human_correction) and model_prediction metadata
- Support negative annotations (label 'O' for user corrections to model predictions)
- Create migration 0005 for new schema fields
Completes tasks 8.1-8.4 of candle-backend change
- Create RandomForestModel and XGBoostModel wrappers with class weight support
- Implement temporal and random train/val/test splitting
- Add MLflow experiment tracking with full parameter and metric logging
- Create evaluation module for confusion matrix, feature importance, and classification reports
- Implement model training with sklearn/xgboost flavor logging and optional registry registration
- Store training run metadata in PostgreSQL
- Wire training stage into pipeline.py orchestrator
- Support both RandomForest and XGBoost models with configurable hyperparameters
