openspec: span annotation
This commit is contained in:
Marko Djordjevic
parent
88e7347918
commit
8a7eb1fb08
5 changed files with 497 additions and 0 deletions
185
openspec/changes/span-annotation/design.md
Normal file
185
openspec/changes/span-annotation/design.md
Normal file
|
|
@ -0,0 +1,185 @@
|
|||
## Context
|
||||
|
||||
The candle annotator currently supports two annotation types: point markers (Break Up/Down displayed as arrow markers via `series.setMarkers()`) and trend lines (drawn on an SVG overlay via `SvgOverlay.tsx`). All annotations are stored in a single `annotations` table with `timestamp`, `label_type`, `geometry` (JSON), and `color` fields.
|
||||
|
||||
The application uses Next.js 16 + React 19, lightweight-charts v4.2.3, SQLite + Drizzle ORM, and Tailwind CSS + shadcn/ui. State management is in `page.tsx` with props passed down to `CandleChart`, `SvgOverlay`, and `Toolbox`.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Add span/range annotation: select start and end candles, assign a pattern label, render as colored rectangle
|
||||
- Rectangles must snap precisely to candle time coordinates (not free-floating like SVG lines)
|
||||
- Support optional metadata: confidence (1-5), outcome, notes
|
||||
- Sub-span support within a parent span
|
||||
- Sidebar list for span annotations with search/filter/select/delete
|
||||
- Export in 3 ML formats: Windowed CSV, BIO-tagged CSV, Raw JSON
|
||||
- User-configurable pattern label categories with colors and hotkeys
|
||||
|
||||
**Non-Goals:**
|
||||
- Overlapping span visual stacking (v1 will render overlapping spans at the same level; vertical offset is a future enhancement)
|
||||
- Real-time collaboration or multi-user support
|
||||
- Undo/redo system
|
||||
- Image crop export format (mentioned as optional in spec, deferred)
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Rendering: lightweight-charts `ISeriesPrimitive` API vs SVG overlay
|
||||
|
||||
**Decision:** Use the `ISeriesPrimitive` plugin API (available since lw-charts v4.1) to render span rectangles as canvas-native primitives.
|
||||
|
||||
**Why not SVG overlay (current line approach):**
|
||||
- SVG overlay positions are in pixel space and must be manually recalculated on every zoom/pan
|
||||
- No automatic coordinate snapping to candle positions
|
||||
- Separate DOM layer causes z-index issues and performance overhead
|
||||
- No built-in hit-testing
|
||||
|
||||
**Why `ISeriesPrimitive`:**
|
||||
- Renders directly on the chart canvas — pixel-perfect alignment with candles
|
||||
- Coordinates are defined in data space `{time, price}` — automatic repositioning on zoom/pan
|
||||
- Built-in `hitTest()` for click detection
|
||||
- `zOrder: 'bottom'` renders behind candles (the desired look for span backgrounds)
|
||||
- `autoscaleInfo()` ensures spans are visible in the price range
|
||||
- Follows the official lightweight-charts plugin pattern (rectangle-drawing-tool example exists)
|
||||
|
||||
**Implementation:** Create a `SpanRectanglePrimitive` class implementing `ISeriesPrimitive`. Each span annotation gets one primitive instance attached via `series.attachPrimitive(rect)`. The primitive converts `{time, price}` corners to pixels in `updateAllViews()` and draws a filled/stroked rectangle in the renderer's `draw()` method using `useBitmapCoordinateSpace` for HiDPI support.
|
||||
|
||||
**Rectangle corners:** For a span from candle A to candle B, the rectangle corners are:
|
||||
- Top-left: `{time: A.time, price: max(high) of candles in range}`
|
||||
- Bottom-right: `{time: B.time, price: min(low) of candles in range}`
|
||||
|
||||
This makes the rectangle hug the actual price range of the pattern, not some arbitrary height.
|
||||
|
||||
### 2. Data model: New table vs extending `annotations`
|
||||
|
||||
**Decision:** Create a separate `span_annotations` table rather than extending the existing `annotations` table.
|
||||
|
||||
**Rationale:**
|
||||
- Span annotations have fundamentally different fields (start_time, end_time, label, confidence, outcome, notes, sub_spans) vs point annotations (single timestamp, label_type)
|
||||
- Mixing them in one table would require many nullable columns and type-checking everywhere
|
||||
- Separate table keeps queries clean and allows independent evolution
|
||||
- The existing `annotations` table and API routes remain untouched — zero risk of breaking existing functionality
|
||||
|
||||
**Schema:**
|
||||
```sql
|
||||
span_annotations (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
chart_id INTEGER NOT NULL REFERENCES charts(id),
|
||||
start_time INTEGER NOT NULL, -- Unix timestamp of first candle
|
||||
end_time INTEGER NOT NULL, -- Unix timestamp of last candle
|
||||
label TEXT NOT NULL, -- pattern name (e.g., 'bull_flag')
|
||||
confidence INTEGER, -- 1-5 scale, nullable
|
||||
outcome TEXT, -- 'win'|'loss'|'breakeven'|null
|
||||
notes TEXT, -- free-text, nullable
|
||||
sub_spans TEXT, -- JSON array of sub-span objects, nullable
|
||||
color TEXT NOT NULL DEFAULT '#2196F3',
|
||||
created_at INTEGER NOT NULL
|
||||
)
|
||||
```
|
||||
|
||||
### 3. Label configuration: Reuse `annotationTypes` vs new table
|
||||
|
||||
**Decision:** Create a new `span_label_types` table separate from `annotationTypes`.
|
||||
|
||||
**Rationale:**
|
||||
- Existing `annotationTypes` has fields specific to point annotations (`icon`, `category` with 'marker'/'line')
|
||||
- Span labels need different fields: `color`, `hotkey`, no icon, no category
|
||||
- Keeping them separate avoids overloading the existing type system
|
||||
- The annotation-types management page (`/annotation-types`) stays focused on its current scope
|
||||
|
||||
**Schema:**
|
||||
```sql
|
||||
span_label_types (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT NOT NULL UNIQUE, -- internal name (e.g., 'bull_flag')
|
||||
display_name TEXT NOT NULL, -- UI label (e.g., 'Bull Flag')
|
||||
color TEXT NOT NULL, -- hex color for rectangle fill
|
||||
hotkey TEXT, -- keyboard shortcut (e.g., '1')
|
||||
is_active INTEGER NOT NULL DEFAULT 1,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
created_at INTEGER NOT NULL
|
||||
)
|
||||
```
|
||||
|
||||
Seed with default labels: bull_flag, bear_flag, head_and_shoulders, double_bottom, wedge_up, wedge_down, custom.
|
||||
|
||||
### 4. Span tool interaction flow
|
||||
|
||||
**Decision:** Two-click interaction with live preview rectangle, followed by an inline popover for label/metadata.
|
||||
|
||||
**Flow:**
|
||||
1. User activates "Span" tool (button in Toolbox or hotkey)
|
||||
2. User clicks candle A → start candle is highlighted with a vertical marker
|
||||
3. As mouse moves, a preview rectangle stretches from candle A to the candle under cursor (snapped to nearest candle time). The rectangle height covers the high/low range of the candles in the preview span.
|
||||
4. User clicks candle B → span is defined
|
||||
5. A popover/dialog appears near the chart with:
|
||||
- Label dropdown (from `span_label_types`)
|
||||
- Confidence slider (1-5, optional)
|
||||
- Outcome select (win/loss/breakeven/none, optional)
|
||||
- Notes textarea (optional)
|
||||
- Save / Cancel buttons
|
||||
6. On Save → POST to API, attach primitive to chart, update sidebar list
|
||||
7. On Cancel or Escape → discard, clear preview
|
||||
|
||||
**Candle snapping:** On click, convert pixel x to time via `coordinateToTime()`, then find the nearest candle from the loaded candle data array. This ensures the span always starts/ends exactly on a candle boundary.
|
||||
|
||||
### 5. Component architecture
|
||||
|
||||
**Decision:** Create focused new components rather than extending existing ones.
|
||||
|
||||
- `SpanRectanglePrimitive.ts` — lightweight-charts `ISeriesPrimitive` implementation (pure TypeScript, no React)
|
||||
- `SpanAnnotationManager.tsx` — React component that manages span state, handles click interactions, creates/removes primitives. Rendered inside `CandleChart.tsx` alongside `SvgOverlay`.
|
||||
- `SpanPopover.tsx` — Label/metadata assignment popover (shadcn Popover or Dialog)
|
||||
- `SpanAnnotationList.tsx` — Sidebar section for span annotations (similar to label list in `Toolbox.tsx`)
|
||||
|
||||
**State flow:** `page.tsx` holds `spanAnnotations[]`, `selectedSpanId`, and passes them to child components. The span tool mode is added to the existing `activeTool` state (type union extended).
|
||||
|
||||
### 6. API design
|
||||
|
||||
**Decision:** Separate API routes for span annotations, mirroring the existing annotation API pattern.
|
||||
|
||||
- `GET /api/span-annotations?chartId=X` — list spans for chart
|
||||
- `POST /api/span-annotations` — create span
|
||||
- `PATCH /api/span-annotations/[id]` — update span (label, metadata, sub-spans)
|
||||
- `DELETE /api/span-annotations/[id]` — delete span
|
||||
- `GET /api/span-label-types` — list configured labels
|
||||
- `POST /api/span-label-types` — create label type
|
||||
- `PATCH /api/span-label-types/[id]` — update label type
|
||||
- `DELETE /api/span-label-types/[id]` — delete label type
|
||||
- `GET /api/export/spans?chartId=X&format=windowed|bio|json` — export spans
|
||||
|
||||
### 7. Export formats
|
||||
|
||||
**Decision:** Single export endpoint with `format` query parameter. The endpoint generates the requested format server-side.
|
||||
|
||||
- **Windowed CSV (`format=windowed`)**: One row per span with flattened OHLCV columns. Configurable `context_padding` parameter (default 10 candles before/after).
|
||||
- **BIO-tagged CSV (`format=bio`)**: One row per candle across the entire dataset. BIO tagging: B-{label} for first candle, I-{label} for continuation, O for outside. Multi-label columns for overlapping spans.
|
||||
- **Raw JSON (`format=json`)**: Full annotation objects with metadata summary.
|
||||
|
||||
### 8. Sub-spans
|
||||
|
||||
**Decision:** Store sub-spans as a JSON array in the `sub_spans` column of `span_annotations`. Sub-spans reference time ranges within the parent span.
|
||||
|
||||
**Interaction:** After creating a span, user can edit it and add sub-spans via the edit popover. Each sub-span has: `label` (text), `start_time`, `end_time`. Sub-spans render as slightly different-shaded rectangles within the parent or with thin divider lines.
|
||||
|
||||
**Deferred to later phase:** Sub-span visual rendering and editing UI are complex. Phase 1 stores the data model; Phase 2 adds the UI.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
**[Risk] `ISeriesPrimitive` API complexity** — The primitive API requires implementing several interfaces (pane views, renderers, HiDPI bitmap scaling). This is more code than an SVG rect.
|
||||
- Mitigation: Follow the official rectangle-drawing-tool plugin example closely. The pattern is well-documented and proven.
|
||||
|
||||
**[Risk] Performance with many span annotations** — Each span creates a separate primitive instance with its own renderer.
|
||||
- Mitigation: For v1, this is acceptable (users won't have hundreds of overlapping spans). If needed later, batch multiple rectangles into a single primitive.
|
||||
|
||||
**[Risk] Popover positioning** — The label assignment popover needs to appear near the chart click point without being clipped by viewport edges.
|
||||
- Mitigation: Use shadcn Popover with `side="top"` and `avoidCollisions={true}`. Anchor to the end-click pixel position.
|
||||
|
||||
**[Risk] Click interaction conflict** — When span tool is active, clicks must go to the canvas for span creation but the SVG overlay currently captures pointer events for line mode.
|
||||
- Mitigation: The SVG overlay already conditionally enables pointer-events only for `line` and `delete` tools. Span mode will bypass the SVG overlay and use the chart's native `subscribeClick`.
|
||||
|
||||
**[Trade-off] Separate tables vs unified annotations** — Separate tables mean two different annotation systems to query/manage. But this avoids schema migration complexity and keeps the existing system stable.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should span labels be manageable from the existing `/annotation-types` page or a new dedicated page? (Recommendation: new section on the same page, since users manage all annotation config in one place)
|
||||
- Should hotkeys for span labels work globally or only when span tool is active? (Recommendation: only when span tool is active, to avoid conflicts with existing hotkeys)
|
||||
Loading…
Add table
Add a link
Reference in a new issue