candle-annotator/openspec/specs/prediction-ui/spec.md

ADDED Requirements

Requirement: Prediction state management

The system SHALL maintain a separate prediction state alongside the existing annotation state. The prediction state SHALL include: spans (array of prediction spans), isLoading, error, modelInfo, visible (toggle), confidenceThreshold (filter), selectedLabels (filter), and autoPredict (toggle). Prediction state SHALL be independent from annotation state.

Scenario: Initial prediction state

• WHEN the app loads
• THEN predictions are empty, visible is true, confidenceThreshold defaults to 0.70, autoPredict is false, and selectedLabels includes all labels

Requirement: On-demand prediction fetching

The system SHALL fetch predictions on demand when the user clicks "Run on Visible". The system SHALL send the currently visible candles to /api/predict and update the prediction state with results. Predictions are ephemeral — not persisted, re-fetched on demand.

Scenario: Run on visible candles

• WHEN user clicks "Run on Visible" button
• THEN the system sends the visible candle range to /api/predict, shows a loading state, and renders returned predictions on the chart

Scenario: Batch predict all

• WHEN user clicks "Predict All" button
• THEN the system sends a batch request to /api/predict/batch for the full dataset and renders all returned predictions

Requirement: Prediction caching

The system SHALL cache predictions in memory keyed by ${pair}_${timeframe}_${startTime}_${endTime}_${modelVersion}. When the user scrolls to a range with cached predictions, the system SHALL use the cache instead of re-fetching. Cache SHALL be invalidated when the model version changes.

Scenario: Cache hit

• WHEN user scrolls back to a previously predicted range with the same model version
• THEN the system renders cached predictions without making an API call

Scenario: Cache invalidation on model change

• WHEN the model version changes (detected via /api/model/info)
• THEN all cached predictions are cleared

Requirement: Prediction rendering on chart

The system SHALL render model predictions as a visual layer on the lightweight-charts instance, visually distinct from human annotations. Predictions SHALL use a histogram series with per-bar colors mapped to predicted pattern labels at reduced opacity (10-20%). Series markers SHALL be added at the start of each prediction span showing {label} ({confidence}%) positioned below bars.

Scenario: Render prediction spans

• WHEN predictions are loaded and visible is true
• THEN colored histogram bars appear behind candles for predicted patterns, with markers showing labels and confidence

Scenario: Predictions hidden

• WHEN the user toggles predictions off (visible = false)
• THEN the prediction histogram series and markers are removed from the chart

Scenario: Visual distinction from annotations

• WHEN both human annotations and model predictions exist for the same range
• THEN human annotations render as solid colored rectangles (above bars) and predictions render as low-opacity histogram bars (below bars) — they are visually distinguishable

Requirement: Confidence threshold filter

The system SHALL filter displayed predictions by confidence. Only predictions with confidence >= confidenceThreshold SHALL be rendered. The threshold is adjustable via a slider in the controls panel (range 0.0 to 1.0).

Scenario: Filter low confidence

• WHEN confidenceThreshold is 0.70 and a prediction has confidence 0.55
• THEN that prediction is not rendered on the chart

Scenario: Adjust threshold

• WHEN user moves the confidence slider from 0.70 to 0.50
• THEN previously hidden predictions with confidence between 0.50 and 0.70 become visible

Requirement: Label type filter

The system SHALL allow users to toggle visibility of individual pattern labels via checkboxes in the controls panel. Only predictions for checked labels are rendered.

Scenario: Hide specific label

• WHEN user unchecks "double_bottom" in the label filter
• THEN all "double_bottom" predictions are hidden from the chart

Requirement: Prediction controls panel

The system SHALL display a prediction controls panel in the sidebar with: master on/off toggle, model selector dropdown (listing available trained models), model info (name, version, type, training date), action buttons ("Run on Visible", "Predict All"), auto-predict toggle, confidence threshold slider, label checkboxes with per-class precision/recall metrics, prediction count, agreement count, and a "Show only disagreements" filter.

Scenario: Display model info

• WHEN the prediction panel loads and the inference API is available
• THEN the panel fetches /api/model/info and displays model name, version, type, and training date

Scenario: Inference API unavailable

• WHEN the prediction panel loads and /api/model/info returns an error
• THEN the panel shows "Model server offline — predictions unavailable" and all controls are disabled

Scenario: Per-class metrics display

• WHEN model info includes per-class metrics
• THEN each label checkbox shows precision and recall values (e.g., "bull_flag (P:0.89 R:0.76)")

Scenario: Model selector integrated

• WHEN the prediction panel renders with trained models available
• THEN a model selector dropdown appears above the action buttons, allowing the user to switch the active model

Requirement: Disagreement detection

The system SHALL compare human annotation spans with model prediction spans to identify disagreements. For each human annotation, check if any prediction span overlaps (>50% time overlap). Disagreement types: "missed_by_model" (human annotated, model predicted "O"), "missed_by_human" (model predicted pattern, no human annotation), "label_mismatch" (both see a pattern but different labels).

Scenario: Missed by model

• WHEN a human annotation exists at T10-T20 but no prediction span overlaps it
• THEN the system identifies this as "missed_by_model"

Scenario: Missed by human

• WHEN a prediction span exists at T30-T40 with no overlapping human annotation
• THEN the system identifies this as "missed_by_human"

Scenario: Label mismatch

• WHEN a human annotation labels T10-T20 as "bull_flag" and the prediction labels the same range as "wedge_up"
• THEN the system identifies this as "label_mismatch"

Requirement: Disagreement rendering

The system SHALL render disagreements with distinct visual styles: "missed_by_model" shows a red dashed border around the human annotation, "missed_by_human" shows a yellow highlight around the prediction, "label_mismatch" shows an orange border with both labels displayed.

Scenario: Render missed_by_human highlight

• WHEN a "missed_by_human" disagreement is detected and disagreement rendering is enabled
• THEN the prediction span is highlighted with a yellow border/glow to draw attention

Scenario: Show only disagreements

• WHEN user clicks "Show only disagreements" filter
• THEN only prediction spans involved in disagreements are rendered, hiding agreement spans

Requirement: Prediction-to-annotation feedback

When a user clicks on a "missed_by_human" prediction, the system SHALL open the span annotation dialog pre-filled with the prediction's start_time, end_time, and label. The user can confirm (save as new annotation), correct (change label, then save), or dismiss.

Scenario: Confirm prediction as annotation

• WHEN user clicks a "missed_by_human" prediction and clicks Save in the pre-filled dialog
• THEN the system creates a new span annotation with the model's suggested label and timestamps

Scenario: Correct and save

• WHEN user clicks a "missed_by_human" prediction, changes the label in the dialog, and clicks Save
• THEN the system creates a new span annotation with the corrected label

Scenario: Dismiss as not-a-pattern

• WHEN user clicks a "missed_by_human" prediction and clicks "Not a pattern"
• THEN the system saves a negative annotation with label "O", source "human_correction", and records the model's original prediction and confidence

Requirement: Prediction cache invalidation on model change

The system SHALL cache predictions in memory keyed by ${pair}_${timeframe}_${startTime}_${endTime}_${modelVersion}. When the user scrolls to a range with cached predictions, the system SHALL use the cache instead of re-fetching. Cache SHALL be invalidated when the model version changes OR when the user switches models via the model selector.

Scenario: Cache hit

• WHEN user scrolls back to a previously predicted range with the same model version
• THEN the system renders cached predictions without making an API call

Scenario: Cache invalidation on model switch

• WHEN the user switches to a different model via the model selector
• THEN all cached predictions are cleared and the chart removes displayed predictions

Requirement: Inference API connection monitoring

The system SHALL poll /api/model/info every 30 seconds when the inference API is unavailable. When the API becomes available, the system SHALL auto-reconnect and enable prediction controls. Human annotation SHALL never be blocked by inference API availability.

Scenario: Auto-reconnect

• WHEN the inference API was unavailable and becomes reachable
• THEN the prediction panel re-enables controls and shows "Model server online"

Scenario: Annotation independence

• WHEN the inference API is unavailable
• THEN all human annotation tools continue to work normally

Requirement: AbortController for prediction requests

The prediction fetching functions (fetchPredictions, handleFetchBatchPredictions) SHALL use AbortController to cancel previous in-flight requests when a new request is initiated. A ref SHALL hold the current controller. Stale responses SHALL be discarded.

Scenario: Rapid clicks cancel previous request

• WHEN the user clicks "Run on Visible" twice quickly
• THEN the first request is aborted and only the second response is rendered

Scenario: Batch prediction cancellation

• WHEN the user clicks "Predict All" while a previous batch prediction is in progress
• THEN the previous batch request is aborted

Requirement: Bounded prediction cache

The predictionCacheRef Map SHALL have a maximum size of 100 entries. When the cache exceeds the limit, the oldest entry (first inserted) SHALL be evicted. The cache SHALL function as a simple FIFO bounded map.

Scenario: Cache eviction at limit

• WHEN the cache has 100 entries and a new prediction result is cached
• THEN the oldest entry is deleted before the new one is inserted

Scenario: Cache size never exceeds limit

• WHEN 200 unique prediction ranges are fetched
• THEN the cache contains at most 100 entries at any point

Requirement: Stable modelInfo reference for cache key

The generateCacheKey function SHALL read modelInfo from a ref (not from the predictionState closure) to prevent stale cache keys.

Scenario: Model change produces correct cache key

• WHEN the user loads a new model and then runs predictions
• THEN the cache key includes the new model version (not the old one from a stale closure)

Requirement: Confirmation dialog for delete-all annotations

The "Delete All" annotations action SHALL show a confirmation dialog before executing. The dialog SHALL display "Are you sure you want to delete all annotations?" with "Cancel" and "Delete" buttons.

Scenario: User confirms deletion

• WHEN the user clicks "Delete All" and then clicks "Delete" in the confirmation dialog
• THEN all annotations for the active chart are deleted

Scenario: User cancels deletion

• WHEN the user clicks "Delete All" and then clicks "Cancel" in the confirmation dialog
• THEN no annotations are deleted