- Add charts table with id, name (unique), created_at - Add chart_id FK to candles table with composite unique on (chart_id, time) - Add chart_id FK to annotations table - Custom migration handles existing data: creates 'Imported Data' chart and backfills chart_id - Recreates tables for NOT NULL constraint (SQLite limitation)
115 lines
7.6 KiB
Markdown
115 lines
7.6 KiB
Markdown
## Context
|
|
|
|
The Candle Annotator is a single-user Next.js app with a SQLite backend (Drizzle ORM). It currently stores candle data in a flat `candles` table and annotations in a flat `annotations` table — both globally scoped. Uploading a CSV wipes all existing candles (`db.delete(candles)`) and replaces them. Annotations have no association to any dataset and persist across uploads, becoming meaningless when the underlying candle data changes.
|
|
|
|
The frontend (`page.tsx`) manages state with React hooks — `annotations`, `activeTool`, `selectedLabelId`, etc. Data fetching happens via `GET /api/candles` (all candles) and `GET /api/annotations` (all annotations). There is no concept of a "chart" or "dataset" entity.
|
|
|
|
## Goals / Non-Goals
|
|
|
|
**Goals:**
|
|
- Users can upload multiple CSV files, each becoming a distinct named chart
|
|
- Users can switch between charts without losing data
|
|
- Annotations are scoped per-chart — switching charts loads only that chart's annotations
|
|
- Existing data is preserved via a migration that assigns current candles/annotations to a default chart
|
|
- The "Manage Annotation Types" link matches the app's theme system
|
|
|
|
**Non-Goals:**
|
|
- Chart renaming or metadata editing (keep it simple for now)
|
|
- Comparing multiple charts side-by-side
|
|
- Sharing charts between users (single-user app)
|
|
- Chart data editing after upload
|
|
- Pagination or lazy-loading of charts list (unlikely to have hundreds)
|
|
|
|
## Decisions
|
|
|
|
### 1. New `charts` table as the parent entity
|
|
|
|
Add a `charts` table with `id`, `name`, `created_at`. Both `candles` and `annotations` gain a `chart_id` foreign key column.
|
|
|
|
**Rationale**: This is the natural relational model. A chart owns its candles and annotations. The alternative — separate SQLite databases per chart — would complicate queries and make migrations harder.
|
|
|
|
**Schema addition:**
|
|
```
|
|
charts: id (PK), name (text, unique), created_at (integer)
|
|
candles: + chart_id (integer, FK → charts.id, NOT NULL)
|
|
annotations: + chart_id (integer, FK → charts.id, NOT NULL)
|
|
```
|
|
|
|
The `candles.time` unique constraint changes to a composite unique on `(chart_id, time)` since different charts can have candles at the same timestamp.
|
|
|
|
### 2. Upload creates a new chart, named from filename
|
|
|
|
When a CSV is uploaded, the server:
|
|
1. Strips the `.csv` extension from the filename to derive the chart name
|
|
2. If a chart with that name already exists, appends a numeric suffix (e.g., `btc-daily-2`)
|
|
3. Creates a row in `charts`
|
|
4. Inserts all candle rows with the new `chart_id`
|
|
5. Returns the new chart's `id` and `name` in the response
|
|
|
|
**Rationale**: Using the filename is the most intuitive naming. Users typically name their CSV files meaningfully (e.g., `BTC-1H.csv`, `ETH-daily.csv`). No extra UI input required.
|
|
|
|
**Alternative considered**: Prompting the user for a name before upload. Rejected because it adds friction and the filename is usually sufficient.
|
|
|
|
### 3. Chart selector in sidebar, above file upload
|
|
|
|
Add a dropdown/select component in the sidebar header area showing the active chart name. Clicking it reveals the list of available charts. Selecting a chart sets `activeChartId` in state, triggering data refetch.
|
|
|
|
**Placement**: Between the app title block and the file upload section. This keeps it prominent and logically grouped — you pick your chart, then work with its tools.
|
|
|
|
**Rationale**: A sidebar dropdown is consistent with the existing layout. It's always visible and doesn't require navigation. The alternative — a separate page for chart management — adds unnecessary complexity for what is essentially a picker.
|
|
|
|
### 4. API scoping via query parameter `?chartId=`
|
|
|
|
All existing endpoints gain chart awareness:
|
|
- `GET /api/candles?chartId=N` — returns candles for chart N
|
|
- `GET /api/annotations?chartId=N` — returns annotations for chart N
|
|
- `POST /api/annotations` — body includes `chart_id`
|
|
- `GET /api/export?chartId=N` — exports annotations for chart N
|
|
- New `GET /api/charts` — lists all charts
|
|
- New `DELETE /api/charts/[id]` — deletes chart and cascading candles/annotations
|
|
|
|
**Rationale**: Query parameters are the simplest approach. The alternative — nested routes like `/api/charts/[id]/candles` — is more RESTful but would require creating new route files and updating all frontend fetch calls to different URLs. Query params let us modify existing routes in place.
|
|
|
|
### 5. Frontend state: `activeChartId` in page.tsx
|
|
|
|
Add `activeChartId` and `charts` state to `page.tsx`. When `activeChartId` changes:
|
|
1. Fetch candles for that chart via `GET /api/candles?chartId=N`
|
|
2. Fetch annotations for that chart via `GET /api/annotations?chartId=N`
|
|
3. Re-render the chart and sidebar
|
|
|
|
On initial load, select the most recently created chart (or show empty state if none exist).
|
|
|
|
**Rationale**: Keeping state in `page.tsx` is consistent with the existing pattern. No need for React Context or a state library — the app is a single page with a flat component hierarchy.
|
|
|
|
### 6. Migration strategy: default chart for existing data
|
|
|
|
The Drizzle migration will:
|
|
1. Create the `charts` table
|
|
2. Add `chart_id` column to `candles` and `annotations` as nullable initially
|
|
3. Insert a default chart row named "Imported Data" if any candles exist
|
|
4. Update all existing candles and annotations to reference the default chart's ID
|
|
5. Make `chart_id` NOT NULL (via table rebuild in SQLite, which Drizzle handles)
|
|
6. Drop the old unique constraint on `candles.time`, add composite unique on `(chart_id, time)`
|
|
|
|
**Rationale**: SQLite doesn't support `ALTER TABLE ADD CONSTRAINT` or `ALTER COLUMN SET NOT NULL` directly. Drizzle's migration tooling handles this by recreating the table. The two-step nullable → backfill → not-null approach ensures no data loss.
|
|
|
|
### 7. Theme-aware "Manage Annotation Types" link
|
|
|
|
Replace the hardcoded `text-blue-600 hover:text-blue-800 underline` with a styled button or link using the app's theme variables. Use `text-muted-foreground hover:text-foreground` classes and remove the underline to match the sidebar's existing aesthetic.
|
|
|
|
**Rationale**: The current blue link is the only element not respecting the theme system. Using theme-aware Tailwind classes (`text-muted-foreground`, `hover:text-foreground`) ensures it works in both light and dark modes.
|
|
|
|
## Risks / Trade-offs
|
|
|
|
**[Migration complexity with SQLite]** → SQLite doesn't support adding NOT NULL columns with foreign keys easily. Drizzle's migration tool handles table rebuilds, but the migration should be tested against a copy of production data before deploying. Keep a backup of `data/candles.db` before running.
|
|
|
|
**[Breaking API change]** → Existing API calls without `chartId` will break. Mitigation: If `chartId` is omitted, fall back to the most recently created chart. This provides backward compatibility during the transition and for the export endpoint.
|
|
|
|
**[Chart deletion cascading]** → Deleting a chart must also delete its candles and annotations. SQLite foreign key support requires `PRAGMA foreign_keys = ON`. Alternatively, handle cascading deletes in application code (delete candles + annotations + chart in a transaction). Application-level cascading is more reliable since SQLite foreign key enforcement requires per-connection pragma.
|
|
|
|
**[No chart limit]** → Users could upload many CSVs and accumulate large amounts of data. For now this is acceptable (single-user app, SQLite handles reasonable sizes). If needed later, add a chart count limit or storage warning.
|
|
|
|
## Open Questions
|
|
|
|
- Should deleting a chart require confirmation? (Likely yes — implement a confirmation dialog in the UI.)
|
|
- Should the chart selector show metadata like date range or candle count? (Nice to have, not required for v1.)
|