427 lines
16 KiB
Markdown
427 lines
16 KiB
Markdown
# Inference Integration: ML Predictions → Next.js Annotation Tool
|
|
|
|
## Context
|
|
|
|
We have an existing Next.js frontend app that uses TradingView Lightweight Charts for candlestick rendering and span annotation of forex patterns. We also have a Python ML pipeline that trains pattern recognition models and serves predictions via a REST API (FastAPI on port 8001).
|
|
|
|
This prompt describes how to connect the inference API to the frontend so users can see model predictions overlaid on their charts alongside their own human annotations.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────┐
|
|
│ Next.js Frontend │
|
|
│ │
|
|
│ ┌───────────────────────────────────────────────┐ │
|
|
│ │ Lightweight Charts │ │
|
|
│ │ │ │
|
|
│ │ [Candles] ← raw OHLCV │ │
|
|
│ │ [Human Annotations] ← solid colored overlays │ │
|
|
│ │ [Model Predictions] ← dashed/hatched overlays│ │
|
|
│ │ [Disagreements] ← highlighted borders │ │
|
|
│ └───────────────────────────────────────────────┘ │
|
|
│ │
|
|
│ ┌──────────────┐ ┌──────────────┐ │
|
|
│ │ Annotation │ │ Prediction │ │
|
|
│ │ Panel │ │ Panel │ │
|
|
│ └──────────────┘ └──────────────┘ │
|
|
└──────────────────┬──────────────────────────────────┘
|
|
│
|
|
Next.js API routes
|
|
/api/predict
|
|
/api/predict/batch
|
|
│
|
|
┌───────────▼───────────┐
|
|
│ Python Inference API │
|
|
│ FastAPI :8001 │
|
|
│ │
|
|
│ /predict │
|
|
│ /predict/batch │
|
|
│ /model/info │
|
|
│ /model/labels │
|
|
└───────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## 1. Next.js API Routes (Proxy to Python Backend)
|
|
|
|
Create Next.js API routes that proxy requests to the Python inference server. This avoids CORS issues and lets you add auth/rate-limiting on the Next.js side.
|
|
|
|
### `/api/predict` — Predict patterns for visible candles
|
|
|
|
```typescript
|
|
// app/api/predict/route.ts
|
|
|
|
// Request body:
|
|
interface PredictRequest {
|
|
pair: string; // e.g. "EURUSD"
|
|
timeframe: string; // e.g. "1H"
|
|
candles: {
|
|
time: number; // unix timestamp
|
|
open: number;
|
|
high: number;
|
|
low: number;
|
|
close: number;
|
|
volume: number;
|
|
}[];
|
|
}
|
|
|
|
// Response body from Python API:
|
|
interface PredictResponse {
|
|
predictions: {
|
|
time: number;
|
|
label: string; // "bull_flag", "head_and_shoulders", "O" (no pattern)
|
|
confidence: number; // 0.0 - 1.0
|
|
}[];
|
|
spans: { // predictions grouped into continuous spans
|
|
start_time: number;
|
|
end_time: number;
|
|
label: string;
|
|
avg_confidence: number;
|
|
}[];
|
|
model_info: {
|
|
model_name: string;
|
|
model_version: string;
|
|
trained_at: string;
|
|
dataset_version: string;
|
|
};
|
|
}
|
|
```
|
|
|
|
### `/api/predict/batch` — Predict on full historical dataset
|
|
|
|
Same interface but accepts a date range instead of raw candles. The Python backend loads the data from its own OHLCV store and returns predictions. Useful for backfill — "show me all patterns the model finds in the last 6 months."
|
|
|
|
```typescript
|
|
interface BatchPredictRequest {
|
|
pair: string;
|
|
timeframe: string;
|
|
start_date: string; // ISO 8601
|
|
end_date: string;
|
|
}
|
|
```
|
|
|
|
### `/api/model/info` — Get current model metadata
|
|
|
|
```typescript
|
|
interface ModelInfoResponse {
|
|
model_name: string;
|
|
model_version: string;
|
|
model_type: string; // "xgboost", "cnn_1d", etc.
|
|
trained_at: string;
|
|
dataset_version: string;
|
|
feature_engineering: boolean;
|
|
labels: string[]; // all pattern labels the model knows
|
|
per_class_metrics: {
|
|
[label: string]: {
|
|
precision: number;
|
|
recall: number;
|
|
f1: number;
|
|
training_samples: number;
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 2. Prediction State Management
|
|
|
|
Add a prediction layer to the existing app state. Keep it separate from annotation state — they are independent data sources that render on the same chart.
|
|
|
|
```typescript
|
|
// types/predictions.ts
|
|
|
|
interface PredictionSpan {
|
|
id: string; // generated from start_time + label
|
|
startTime: number;
|
|
endTime: number;
|
|
label: string;
|
|
avgConfidence: number;
|
|
source: "model"; // always "model", vs "human" for annotations
|
|
}
|
|
|
|
interface PredictionState {
|
|
spans: PredictionSpan[];
|
|
isLoading: boolean;
|
|
error: string | null;
|
|
modelInfo: ModelInfoResponse | null;
|
|
visible: boolean; // toggle prediction layer on/off
|
|
confidenceThreshold: number; // filter: only show predictions above this
|
|
selectedLabels: string[]; // filter: which pattern types to show
|
|
autoPredict: boolean; // auto-run predictions when chart scrolls
|
|
}
|
|
```
|
|
|
|
### When to fetch predictions
|
|
|
|
Predictions should be fetched:
|
|
- **On demand:** User clicks a "Run Model" button
|
|
- **On chart scroll/zoom (if `autoPredict` is on):** When the visible candle range changes, debounce 500ms, then send the visible candles to `/api/predict`. Only send candles that haven't been predicted yet (cache previous results by time range).
|
|
- **On batch backfill:** User selects a date range and clicks "Predict All"
|
|
|
|
### Caching
|
|
|
|
Cache predictions in a Map keyed by `${pair}_${timeframe}_${startTime}_${endTime}_${modelVersion}`. Invalidate cache when the model version changes. This prevents re-predicting the same candles on every scroll.
|
|
|
|
---
|
|
|
|
## 3. Rendering Predictions on Lightweight Charts
|
|
|
|
Predictions render as a separate visual layer on the same chart instance as human annotations. They must be visually distinct from human annotations.
|
|
|
|
### Visual Distinction
|
|
|
|
| Property | Human Annotations | Model Predictions |
|
|
|---|---|---|
|
|
| Background fill | Solid color, 20% opacity | Diagonal hatched pattern or 10% opacity |
|
|
| Border | Solid 2px | Dashed 2px |
|
|
| Label tag | Solid background | Outlined/hollow background |
|
|
| Label text | "bull_flag" | "bull_flag (87%)" with confidence |
|
|
| Position | Above candles | Below candles (avoid overlap) |
|
|
|
|
### Implementation with Lightweight Charts
|
|
|
|
Lightweight Charts doesn't have a native "span highlight" feature, so use one of these approaches:
|
|
|
|
**Option A: Custom Series Markers + Box Plugin**
|
|
|
|
Use the `createBox` or a custom plugin to draw rectangles behind candle ranges. Lightweight Charts v4+ supports plugins that can draw on the canvas.
|
|
|
|
```typescript
|
|
// Pseudo-code for rendering a prediction span
|
|
|
|
function renderPredictionSpan(chart, span: PredictionSpan, labelConfig: LabelConfig) {
|
|
// Use a box/rectangle primitive
|
|
// Position: from span.startTime to span.endTime on X axis
|
|
// Full price range of candles in that span on Y axis
|
|
// Style: dashed border, hatched or low-opacity fill
|
|
// Color: from labelConfig based on span.label
|
|
|
|
// Add a marker at the first candle of the span with label text
|
|
series.setMarkers([
|
|
...existingMarkers,
|
|
{
|
|
time: span.startTime,
|
|
position: 'belowBar', // below for predictions, above for human
|
|
color: labelConfig.color,
|
|
shape: 'square',
|
|
text: `${span.label} (${Math.round(span.avgConfidence * 100)}%)`,
|
|
}
|
|
]);
|
|
}
|
|
```
|
|
|
|
**Option B: Background color per candle using a secondary series**
|
|
|
|
Create a histogram series behind the candles that uses color to indicate predictions. Each bar's color maps to a pattern label. Simpler but less visually rich.
|
|
|
|
```typescript
|
|
const predictionSeries = chart.addHistogramSeries({
|
|
priceScaleId: '', // overlay on main scale
|
|
color: 'transparent',
|
|
lastValueVisible: false,
|
|
priceLineVisible: false,
|
|
});
|
|
|
|
// Set data with per-bar colors
|
|
predictionSeries.setData(
|
|
predictedCandles.map(c => ({
|
|
time: c.time,
|
|
value: c.high, // height of the bar
|
|
color: c.label !== 'O'
|
|
? `${labelColorMap[c.label]}33` // 20% opacity hex
|
|
: 'transparent',
|
|
}))
|
|
);
|
|
```
|
|
|
|
**Option C: Custom drawing on the chart canvas (most control)**
|
|
|
|
Use the Lightweight Charts plugin API to draw directly on the canvas. This gives full control over hatching, dashed borders, etc.
|
|
|
|
```typescript
|
|
import { createChart, IChartApi } from 'lightweight-charts';
|
|
|
|
// After chart is created, access the canvas and draw overlays
|
|
// Use requestAnimationFrame to sync with chart rendering
|
|
// Listen to chart.timeScale().subscribeVisibleLogicalRangeChange()
|
|
// to redraw when the user scrolls
|
|
```
|
|
|
|
**Recommended approach:** Start with Option B (histogram series) for a quick implementation, then upgrade to Option C (canvas plugin) for the polished version. Option A works if you find or build a good box plugin.
|
|
|
|
---
|
|
|
|
## 4. Prediction Controls Panel
|
|
|
|
Add a panel (sidebar or floating) next to the annotation panel with these controls:
|
|
|
|
### Controls
|
|
|
|
```
|
|
┌─────────────────────────────────────────┐
|
|
│ Model Predictions [ON] │ ← master toggle
|
|
├─────────────────────────────────────────┤
|
|
│ Model: candlestick_v1 (v3) │
|
|
│ Type: XGBoost │
|
|
│ Trained: 2024-03-20 │
|
|
│ Dataset: v12 (847 annotations) │
|
|
├─────────────────────────────────────────┤
|
|
│ [Run on Visible] [Predict All] │ ← action buttons
|
|
│ Auto-predict on scroll: [OFF] │
|
|
├─────────────────────────────────────────┤
|
|
│ Confidence threshold: ━━━━━━●━━ 0.70 │ ← slider, filters
|
|
│ │
|
|
│ Show patterns: │
|
|
│ ☑ bull_flag (P:0.89 R:0.76) │ ← per-class metrics
|
|
│ ☑ bear_flag (P:0.82 R:0.71) │
|
|
│ ☑ head_shoulders (P:0.74 R:0.65) │
|
|
│ ☐ double_bottom (P:0.68 R:0.52) │ ← low recall, user may hide
|
|
│ ☑ wedge_up (P:0.85 R:0.79) │
|
|
├─────────────────────────────────────────┤
|
|
│ Predictions visible: 12 │
|
|
│ Agreements with human: 8/12 │
|
|
│ Disagreements: 4 │
|
|
│ → [Show only disagreements] │
|
|
└─────────────────────────────────────────┘
|
|
```
|
|
|
|
### Per-class metrics display
|
|
|
|
Fetch from `/api/model/info` on load. Show precision (P) and recall (R) next to each label checkbox so the user knows which patterns the model is reliable for. Low-metric patterns can be hidden by default.
|
|
|
|
---
|
|
|
|
## 5. Disagreement Detection
|
|
|
|
The most valuable feature: comparing human annotations with model predictions.
|
|
|
|
### Logic
|
|
|
|
For each time range, compare human annotation spans with prediction spans:
|
|
|
|
```typescript
|
|
interface Disagreement {
|
|
time_range: { start: number; end: number };
|
|
human_label: string | null; // null if model predicted but human didn't annotate
|
|
model_label: string | null; // null if human annotated but model missed
|
|
model_confidence: number;
|
|
type: "missed_by_model" // human annotated, model said "O"
|
|
| "missed_by_human" // model predicted pattern, human didn't annotate
|
|
| "label_mismatch"; // both see a pattern but disagree on which
|
|
}
|
|
```
|
|
|
|
### How to compute disagreements
|
|
|
|
1. Get all human annotation spans for the visible range.
|
|
2. Get all model prediction spans for the visible range.
|
|
3. For each human span, check if any prediction span overlaps (>50% time overlap):
|
|
- No overlap → `missed_by_model`
|
|
- Overlap but different label → `label_mismatch`
|
|
- Overlap and same label → agreement
|
|
4. For each prediction span not matched to a human span → `missed_by_human`
|
|
|
|
### Rendering disagreements
|
|
|
|
- `missed_by_model`: Red dashed border around the human annotation (model couldn't see this)
|
|
- `missed_by_human`: Yellow pulsing/blinking highlight (model found something you didn't label — review it!)
|
|
- `label_mismatch`: Orange border with both labels shown
|
|
|
|
`missed_by_human` is especially valuable — the model may be finding patterns you haven't annotated yet. Add a quick action: clicking a `missed_by_human` prediction opens the annotation dialog pre-filled with the model's suggested label so you can confirm or correct it with one click.
|
|
|
|
---
|
|
|
|
## 6. Feedback Loop: Predictions → New Annotations
|
|
|
|
This is what makes the system improve over time.
|
|
|
|
### Flow
|
|
|
|
```
|
|
1. Model predicts a pattern the user didn't annotate (missed_by_human)
|
|
2. User sees it highlighted on the chart
|
|
3. User clicks it → annotation dialog opens pre-filled:
|
|
- Start/end time from prediction span
|
|
- Label from prediction
|
|
- Confidence shown
|
|
4. User either:
|
|
a. Confirms → saves as a new human annotation
|
|
b. Corrects label → saves with corrected label
|
|
c. Dismisses → optionally mark as "not a pattern" (negative example)
|
|
5. New annotation is exported → fed into next training cycle
|
|
```
|
|
|
|
### "Not a pattern" negative examples
|
|
|
|
When the model predicts something and the user explicitly says "this is not a pattern," save this as a negative annotation:
|
|
|
|
```json
|
|
{
|
|
"start_time": "...",
|
|
"end_time": "...",
|
|
"label": "O",
|
|
"source": "human_correction",
|
|
"model_predicted": "bull_flag",
|
|
"model_confidence": 0.72
|
|
}
|
|
```
|
|
|
|
These negative examples are extremely valuable for training — they teach the model to stop making specific false positive mistakes.
|
|
|
|
---
|
|
|
|
## 7. API Error Handling & Loading States
|
|
|
|
### When inference API is unavailable
|
|
|
|
- Show a subtle banner: "Model server offline — predictions unavailable"
|
|
- All prediction UI controls become disabled/greyed out
|
|
- Human annotation continues to work normally (never block annotation on inference)
|
|
- Poll `/api/model/info` every 30 seconds, auto-reconnect when available
|
|
|
|
### Loading states
|
|
|
|
- While predictions are loading, show a skeleton/shimmer overlay on the chart area
|
|
- For batch predictions on large date ranges, show a progress indicator
|
|
- Allow the user to cancel a long-running batch prediction
|
|
|
|
### Stale predictions
|
|
|
|
- When the user scrolls to a range that has cached predictions from an older model version, show a subtle indicator: "Predictions from model v2 — click to refresh with v3"
|
|
- When a new model is deployed, invalidate all cached predictions and show "New model available — rerun predictions?"
|
|
|
|
---
|
|
|
|
## 8. Environment Configuration
|
|
|
|
```env
|
|
# .env.local
|
|
|
|
# Inference API
|
|
INFERENCE_API_URL=http://localhost:8001
|
|
INFERENCE_API_TIMEOUT=10000 # ms, for single prediction requests
|
|
INFERENCE_BATCH_TIMEOUT=120000 # ms, for batch predictions
|
|
|
|
# Feature flags
|
|
NEXT_PUBLIC_PREDICTIONS_ENABLED=true
|
|
NEXT_PUBLIC_AUTO_PREDICT_DEFAULT=false
|
|
NEXT_PUBLIC_DEFAULT_CONFIDENCE_THRESHOLD=0.70
|
|
```
|
|
|
|
---
|
|
|
|
## Summary of Integration Points
|
|
|
|
1. **Next.js API routes** proxy to Python FastAPI inference server
|
|
2. **Prediction state** lives alongside annotation state, fetched on demand or auto-scroll
|
|
3. **Rendering** uses Lightweight Charts histogram series (quick) or canvas plugin (polished) with visual distinction from human annotations
|
|
4. **Controls panel** lets user toggle predictions, filter by confidence/label, view model metrics
|
|
5. **Disagreement detection** compares human vs model and highlights mismatches
|
|
6. **Feedback loop** lets user confirm/correct/dismiss model predictions as new annotations
|
|
7. **Negative examples** from dismissed predictions feed back into training
|
|
8. **Error handling** ensures annotation tool works independently of inference availability
|