## Why The project currently runs two separate database servers: SQLite (via Drizzle ORM) for the Next.js frontend and PostgreSQL for the ML service. This creates unnecessary operational complexity — two different ORMs, two migration systems, two backup strategies, and no ability for the ML service to directly query annotation/candle data. Consolidating to PostgreSQL as the single database simplifies deployment, enables direct cross-service data access, and reduces the infrastructure footprint. ## What Changes - **BREAKING**: Replace SQLite/better-sqlite3/Drizzle with PostgreSQL/Drizzle (pg driver) for the Next.js frontend - Remove the `candle-data` Docker volume (SQLite file storage) and `DATABASE_PATH` env var - Migrate all frontend tables (charts, candles, annotations, annotation_types, span_annotations, span_label_types) into the existing PostgreSQL instance - Update Drizzle schema and config to target PostgreSQL instead of SQLite - Regenerate Drizzle migrations for PostgreSQL dialect (column types change: `integer` → `serial`, `real` → `double precision`, timestamps as proper `timestamp` types, etc.) - Update the ML service to share the same PostgreSQL database (or a separate schema within it) so it can directly query candle/annotation data instead of relying on CSV/JSON exports - Update docker-compose.yml to remove SQLite volume dependency and point the frontend at PostgreSQL - Update environment variables: frontend gets `DATABASE_URL` pointing to PostgreSQL ## Capabilities ### New Capabilities - `postgres-data-layer`: Unified PostgreSQL data access layer for the Next.js frontend, replacing the SQLite/better-sqlite3 setup with Drizzle's PostgreSQL driver ### Modified Capabilities - `docker-deployment`: Container configuration changes — remove SQLite volume, add PostgreSQL dependency for the frontend service, update environment variables - `ml-training`: ML service can now query annotations and candle data directly from PostgreSQL instead of requiring CSV/JSON file exports ## Impact - **Database schema**: All 6 frontend tables move to PostgreSQL with type adaptations (SQLite integers → PostgreSQL serial/integer/timestamp) - **ORM layer**: `src/lib/db/index.ts` switches from `better-sqlite3` to `postgres` driver; schema types in `src/lib/db/schema.ts` change to PostgreSQL equivalents - **Dependencies**: Remove `better-sqlite3`, add `postgres` (or `pg`) npm package for Drizzle's PostgreSQL adapter - **Migrations**: Existing SQLite migrations become obsolete; new PostgreSQL migrations needed - **Docker**: `candle-annotator` service gains `depends_on: postgres`, loses `candle-data` volume mount - **Environment**: `.env` and `.env.example` updated with PostgreSQL connection string for frontend - **ML service**: `services/ml/app/db.py` gains access to frontend tables (candles, annotations) for direct querying - **Data migration**: Existing SQLite data needs a one-time migration script to PostgreSQL - **API routes**: All Next.js API routes using `db` from `src/lib/db` continue working (Drizzle abstracts the driver change), but queries using SQLite-specific syntax may need adjustment