feat: add charts table schema and migration with data backfill
- Add charts table with id, name (unique), created_at - Add chart_id FK to candles table with composite unique on (chart_id, time) - Add chart_id FK to annotations table - Custom migration handles existing data: creates 'Imported Data' chart and backfills chart_id - Recreates tables for NOT NULL constraint (SQLite limitation)
This commit is contained in:
Marko Djordjevic
parent
178834f3b2
commit
92d3339a48
14 changed files with 913 additions and 3 deletions
115
openspec/changes/multi-chart-management/design.md
Normal file
115
openspec/changes/multi-chart-management/design.md
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
## Context
|
||||
|
||||
The Candle Annotator is a single-user Next.js app with a SQLite backend (Drizzle ORM). It currently stores candle data in a flat `candles` table and annotations in a flat `annotations` table — both globally scoped. Uploading a CSV wipes all existing candles (`db.delete(candles)`) and replaces them. Annotations have no association to any dataset and persist across uploads, becoming meaningless when the underlying candle data changes.
|
||||
|
||||
The frontend (`page.tsx`) manages state with React hooks — `annotations`, `activeTool`, `selectedLabelId`, etc. Data fetching happens via `GET /api/candles` (all candles) and `GET /api/annotations` (all annotations). There is no concept of a "chart" or "dataset" entity.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Users can upload multiple CSV files, each becoming a distinct named chart
|
||||
- Users can switch between charts without losing data
|
||||
- Annotations are scoped per-chart — switching charts loads only that chart's annotations
|
||||
- Existing data is preserved via a migration that assigns current candles/annotations to a default chart
|
||||
- The "Manage Annotation Types" link matches the app's theme system
|
||||
|
||||
**Non-Goals:**
|
||||
- Chart renaming or metadata editing (keep it simple for now)
|
||||
- Comparing multiple charts side-by-side
|
||||
- Sharing charts between users (single-user app)
|
||||
- Chart data editing after upload
|
||||
- Pagination or lazy-loading of charts list (unlikely to have hundreds)
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. New `charts` table as the parent entity
|
||||
|
||||
Add a `charts` table with `id`, `name`, `created_at`. Both `candles` and `annotations` gain a `chart_id` foreign key column.
|
||||
|
||||
**Rationale**: This is the natural relational model. A chart owns its candles and annotations. The alternative — separate SQLite databases per chart — would complicate queries and make migrations harder.
|
||||
|
||||
**Schema addition:**
|
||||
```
|
||||
charts: id (PK), name (text, unique), created_at (integer)
|
||||
candles: + chart_id (integer, FK → charts.id, NOT NULL)
|
||||
annotations: + chart_id (integer, FK → charts.id, NOT NULL)
|
||||
```
|
||||
|
||||
The `candles.time` unique constraint changes to a composite unique on `(chart_id, time)` since different charts can have candles at the same timestamp.
|
||||
|
||||
### 2. Upload creates a new chart, named from filename
|
||||
|
||||
When a CSV is uploaded, the server:
|
||||
1. Strips the `.csv` extension from the filename to derive the chart name
|
||||
2. If a chart with that name already exists, appends a numeric suffix (e.g., `btc-daily-2`)
|
||||
3. Creates a row in `charts`
|
||||
4. Inserts all candle rows with the new `chart_id`
|
||||
5. Returns the new chart's `id` and `name` in the response
|
||||
|
||||
**Rationale**: Using the filename is the most intuitive naming. Users typically name their CSV files meaningfully (e.g., `BTC-1H.csv`, `ETH-daily.csv`). No extra UI input required.
|
||||
|
||||
**Alternative considered**: Prompting the user for a name before upload. Rejected because it adds friction and the filename is usually sufficient.
|
||||
|
||||
### 3. Chart selector in sidebar, above file upload
|
||||
|
||||
Add a dropdown/select component in the sidebar header area showing the active chart name. Clicking it reveals the list of available charts. Selecting a chart sets `activeChartId` in state, triggering data refetch.
|
||||
|
||||
**Placement**: Between the app title block and the file upload section. This keeps it prominent and logically grouped — you pick your chart, then work with its tools.
|
||||
|
||||
**Rationale**: A sidebar dropdown is consistent with the existing layout. It's always visible and doesn't require navigation. The alternative — a separate page for chart management — adds unnecessary complexity for what is essentially a picker.
|
||||
|
||||
### 4. API scoping via query parameter `?chartId=`
|
||||
|
||||
All existing endpoints gain chart awareness:
|
||||
- `GET /api/candles?chartId=N` — returns candles for chart N
|
||||
- `GET /api/annotations?chartId=N` — returns annotations for chart N
|
||||
- `POST /api/annotations` — body includes `chart_id`
|
||||
- `GET /api/export?chartId=N` — exports annotations for chart N
|
||||
- New `GET /api/charts` — lists all charts
|
||||
- New `DELETE /api/charts/[id]` — deletes chart and cascading candles/annotations
|
||||
|
||||
**Rationale**: Query parameters are the simplest approach. The alternative — nested routes like `/api/charts/[id]/candles` — is more RESTful but would require creating new route files and updating all frontend fetch calls to different URLs. Query params let us modify existing routes in place.
|
||||
|
||||
### 5. Frontend state: `activeChartId` in page.tsx
|
||||
|
||||
Add `activeChartId` and `charts` state to `page.tsx`. When `activeChartId` changes:
|
||||
1. Fetch candles for that chart via `GET /api/candles?chartId=N`
|
||||
2. Fetch annotations for that chart via `GET /api/annotations?chartId=N`
|
||||
3. Re-render the chart and sidebar
|
||||
|
||||
On initial load, select the most recently created chart (or show empty state if none exist).
|
||||
|
||||
**Rationale**: Keeping state in `page.tsx` is consistent with the existing pattern. No need for React Context or a state library — the app is a single page with a flat component hierarchy.
|
||||
|
||||
### 6. Migration strategy: default chart for existing data
|
||||
|
||||
The Drizzle migration will:
|
||||
1. Create the `charts` table
|
||||
2. Add `chart_id` column to `candles` and `annotations` as nullable initially
|
||||
3. Insert a default chart row named "Imported Data" if any candles exist
|
||||
4. Update all existing candles and annotations to reference the default chart's ID
|
||||
5. Make `chart_id` NOT NULL (via table rebuild in SQLite, which Drizzle handles)
|
||||
6. Drop the old unique constraint on `candles.time`, add composite unique on `(chart_id, time)`
|
||||
|
||||
**Rationale**: SQLite doesn't support `ALTER TABLE ADD CONSTRAINT` or `ALTER COLUMN SET NOT NULL` directly. Drizzle's migration tooling handles this by recreating the table. The two-step nullable → backfill → not-null approach ensures no data loss.
|
||||
|
||||
### 7. Theme-aware "Manage Annotation Types" link
|
||||
|
||||
Replace the hardcoded `text-blue-600 hover:text-blue-800 underline` with a styled button or link using the app's theme variables. Use `text-muted-foreground hover:text-foreground` classes and remove the underline to match the sidebar's existing aesthetic.
|
||||
|
||||
**Rationale**: The current blue link is the only element not respecting the theme system. Using theme-aware Tailwind classes (`text-muted-foreground`, `hover:text-foreground`) ensures it works in both light and dark modes.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
**[Migration complexity with SQLite]** → SQLite doesn't support adding NOT NULL columns with foreign keys easily. Drizzle's migration tool handles table rebuilds, but the migration should be tested against a copy of production data before deploying. Keep a backup of `data/candles.db` before running.
|
||||
|
||||
**[Breaking API change]** → Existing API calls without `chartId` will break. Mitigation: If `chartId` is omitted, fall back to the most recently created chart. This provides backward compatibility during the transition and for the export endpoint.
|
||||
|
||||
**[Chart deletion cascading]** → Deleting a chart must also delete its candles and annotations. SQLite foreign key support requires `PRAGMA foreign_keys = ON`. Alternatively, handle cascading deletes in application code (delete candles + annotations + chart in a transaction). Application-level cascading is more reliable since SQLite foreign key enforcement requires per-connection pragma.
|
||||
|
||||
**[No chart limit]** → Users could upload many CSVs and accumulate large amounts of data. For now this is acceptable (single-user app, SQLite handles reasonable sizes). If needed later, add a chart count limit or storage warning.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should deleting a chart require confirmation? (Likely yes — implement a confirmation dialog in the UI.)
|
||||
- Should the chart selector show metadata like date range or candle count? (Nice to have, not required for v1.)
|
||||
Loading…
Add table
Add a link
Reference in a new issue