exe.dev SSH proxy blocks direct SSH from GitHub Actions. Use webhook listener on port 9000 instead. Co-authored-by: Shelley <shelley@exe.dev> |
||
|---|---|---|
| .github/workflows | ||
| drizzle | ||
| openspec | ||
| public | ||
| src | ||
| .dockerignore | ||
| .env.example | ||
| .eslintrc.json | ||
| .gitignore | ||
| CLAUDE.md | ||
| CLAUDE_DESCRIPTION.md | ||
| components.json | ||
| DEPLOYMENT.md | ||
| docker-compose.yml | ||
| Dockerfile | ||
| drizzle.config.ts | ||
| EURUSD.csv | ||
| EURUSD.csv~ | ||
| EURUSD_1000.csv | ||
| LINE_DRAWING_IMPROVEMENTS.md | ||
| next-env.d.ts | ||
| next.config.js | ||
| package-lock.json | ||
| package.json | ||
| PLAN.md | ||
| postcss.config.js | ||
| README.md | ||
| tailwind.config.ts | ||
| tsconfig.json | ||
| tsconfig.tsbuildinfo | ||
Candle Annotator
A web-based tool for manually annotating candlestick charts with pattern labels and trend lines. Built for creating labeled training data for machine learning models in trading analysis.
Overview
Candle Annotator provides a TradingView-like charting interface that allows traders and researchers to:
- Upload historical OHLC (Open, High, Low, Close) candle data from CSV files
- Visualize candlestick charts with interactive zoom and pan
- Mark breakout patterns (Break Up, Break Down) directly on candles
- Draw custom trend lines with two-click interaction
- Delete annotations with a dedicated tool
- Export all annotations as CSV for ML training pipelines
Features
Data Management
- CSV Upload: Import OHLC data with support for both Unix timestamps and date strings
- SQLite Storage: All candle data and annotations stored locally in SQLite database
- Data Persistence: Annotations and candles persist between sessions
Chart Visualization
- Interactive Candlestick Chart: Powered by lightweight-charts library
- Dark Theme: Eye-friendly slate color scheme
- Zoom & Pan: Mouse wheel zoom and drag-to-pan functionality
- Crosshair: Precise price and time tracking
Annotation Tools
- Break Up Markers: Green arrow markers below candles indicating upward breakouts
- Break Down Markers: Red arrow markers above candles indicating downward breakouts
- Trend Lines: Two-click line drawing with real-time preview
- Delete Tool: Remove any annotation (markers or lines) by clicking on them
- Tool Toggle: Click tool button again to deactivate
Label Management
- Label Sidebar: View all annotations in collapsible sidebar with:
- Click Selection: Click markers on chart or in sidebar to select/highlight
- Keyboard Delete: Press Delete or Backspace to remove selected label
- Individual Delete: Delete button on each list item
- Search: Search annotations by timestamp
- Filter: Filter by Break Up, Break Down, or All types
- Count Display: See how many Break Up vs Break Down markers exist
- Visual Highlight: Selected markers highlighted with glow effect
UI Theme
- Hacker Theme: Terminal-inspired dark aesthetic with:
- Matrix green (#00ff41) on dark background (#0a0e0a)
- Monospace font (JetBrains Mono) throughout
- Glow effects on button hover and active states
- Custom scrollbars styled to match theme
- High contrast for accessibility
Export & Deployment
- CSV Export: Download all annotations with timestamp, label type, and price data
- ML-Ready Format: Structured data suitable for training ML models
- Docker Deployment: One-command deployment with persistent data volume
- Health Check: Built-in /api/health endpoint for monitoring
Tech Stack
- Frontend: Next.js 16 (App Router), React 19, TypeScript
- Styling: Tailwind CSS 3, shadcn/ui components
- Charting: lightweight-charts 4.x (TradingView)
- Icons: lucide-react
- Backend: Next.js API Routes
- Database: SQLite with better-sqlite3
- ORM: Drizzle ORM
- CSV Parsing: papaparse
Getting Started
Docker Quickstart (Recommended)
The fastest way to get running with Docker:
docker-compose up --build
Then open http://localhost:3000
See DEPLOYMENT.md for detailed Docker instructions.
Prerequisites
- Node.js 18.x or higher (for local development)
- npm 9.x or higher (for local development)
- Docker & docker-compose (for containerized deployment)
- Build tools for native modules (see DEPLOYMENT.md)
Local Development Installation
-
Clone the repository:
git clone <repository-url> cd candle_annotator -
Install dependencies:
npm install -
Start the development server:
npm run dev -
Open http://localhost:3000 in your browser
Usage
- Upload Data: Click "Choose CSV File" and select a CSV with columns:
time,open,high,low,close - View Chart: The candlestick chart renders automatically after upload
- Add Annotations:
- Click "Label: Break Up" or "Label: Break Down" then click on a candle
- Click "Draw Line" then click two points to draw a trend line
- Press Escape to cancel line drawing
- Delete Annotations: Click "Delete" tool, then click on markers or lines to remove them
- Export: Click "Export CSV" to download all annotations
CSV File Format
Input Format
Your CSV file should have these columns:
time,open,high,low,close
1700000000,1.0500,1.0520,1.0490,1.0510
1700000060,1.0510,1.0530,1.0505,1.0525
Time column accepts:
- Unix timestamps (seconds):
1700000000 - Date strings:
2024-01-15,2024-01-15 10:30:00
Export Format
The exported CSV includes:
timestamp,label_type,price
1700000000,break_up,1.0510
1700000120,break_down,1.0505
1700000000,line,1.0500
- timestamp: Unix timestamp of the annotation
- label_type:
break_up,break_down, orline - price: Close price for markers, start price for lines
Database Schema
Candles Table
{
id: integer (PK, auto-increment),
time: integer (Unix timestamp, unique),
open: real,
high: real,
low: real,
close: real
}
Annotations Table
{
id: integer (PK, auto-increment),
timestamp: integer (Unix timestamp),
label_type: text ('break_up' | 'break_down' | 'line'),
geometry: text (JSON string for line coordinates, null for markers),
created_at: integer (Unix timestamp)
}
API Endpoints
POST /api/upload
Upload CSV file and store candle data
Request: multipart/form-data with file field
Response: { success: true, count: number } or { error: string }
GET /api/candles
Retrieve all candle records
Response: Array of candle objects ordered by time
GET /api/annotations
Retrieve all annotations
Response: Array of annotation objects with parsed geometry
POST /api/annotations
Create a new annotation
Request: { timestamp: number, label_type: string, geometry?: object }
Response: Created annotation object with ID
DELETE /api/annotations/[id]
Delete an annotation by ID
Response: { success: true } or { error: string }
GET /api/export
Export annotations as downloadable CSV
Response: CSV file download with Content-Disposition header
Architecture
Component Structure
- page.tsx: Main page composition, manages active tool state
- Toolbox.tsx: Sidebar with tool buttons and export functionality
- FileUpload.tsx: CSV upload component with status messages
- CandleChart.tsx: Core chart wrapper with lightweight-charts integration
- Initializes chart with dark theme
- Handles marker annotations (Break Up/Down)
- Manages click events for annotation creation
- Exposes
refreshData()method for parent updates
- SvgOverlay.tsx: Transparent SVG layer for line drawing
- Coordinate transformation between data and pixels
- Two-click line drawing with preview
- Line hit detection for deletion
Data Flow
- User uploads CSV → POST /api/upload → SQLite storage
- Chart mounts → GET /api/candles + GET /api/annotations → Render
- User clicks with active tool → POST /api/annotations → Refresh chart
- User deletes → DELETE /api/annotations/[id] → Refresh chart
- User exports → GET /api/export → CSV download
Development
Project Structure
candle_annotator/
├── src/
│ ├── app/
│ │ ├── api/ # API route handlers
│ │ │ ├── upload/
│ │ │ ├── candles/
│ │ │ ├── annotations/
│ │ │ └── export/
│ │ ├── globals.css # Tailwind styles
│ │ ├── layout.tsx # Root layout with dark theme
│ │ └── page.tsx # Main page
│ ├── components/
│ │ ├── ui/ # shadcn/ui components
│ │ ├── CandleChart.tsx
│ │ ├── SvgOverlay.tsx
│ │ ├── Toolbox.tsx
│ │ └── FileUpload.tsx
│ └── lib/
│ ├── db/
│ │ ├── index.ts # Drizzle client
│ │ ├── schema.ts # Table definitions
│ │ └── migrate.ts # Migration runner
│ └── utils.ts # Utility functions
├── data/ # SQLite database directory
├── drizzle/ # Migration files
├── DEPLOYMENT.md # Deployment instructions
└── README.md # This file
Key Technical Decisions
- lightweight-charts v4: Stable API with good candlestick and marker support
- SQLite with better-sqlite3: Synchronous access, perfect for single-user local apps
- SVG Overlay for Lines: Maintains separate rendering layer from chart, easier coordinate management
- Drizzle ORM: Type-safe queries with minimal overhead
- Next.js App Router: Server-side API routes co-located with frontend code
Known Limitations
- Single User: No authentication or concurrent access support
- No Undo: Can only delete annotations, not undo placement
- Memory: Large CSV files (100k+ rows) may cause slow uploads
- Line Snapping: Lines don't snap to candles, free-form placement only
Troubleshooting
See DEPLOYMENT.md for detailed troubleshooting steps.
Common issues:
- better-sqlite3 binding errors: Run
npm rebuild better-sqlite3 - Port 3000 in use: Use
PORT=3001 npm run dev - Database corruption: Delete
data/candles.dband restart
License
ISC
Contributing
This is a focused tool for a specific use case. For questions or issues, please open a GitHub issue.