Commit graph

177 commits

Author SHA1 Message Date
Marko Djordjevic
ed05bd6972 code-review-fix task 10.2: mark task 10.2 as done in tasks.md 2026-02-18 20:27:36 +01:00
Marko Djordjevic
4bd9d6f9da code-review-fix task 10.1: mark task 10.1 as done in tasks.md 2026-02-18 20:27:05 +01:00
Marko Djordjevic
43b6b5da9e code-review-fix task 9.8: mark task 9.8 as done in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 20:26:05 +01:00
Marko Djordjevic
1b637dc45e task 9.7: mark task 9.7 as done in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:49:43 +01:00
Marko Djordjevic
28a0d0790e task 9.6: mark task 9.6 as completed in tasks.md 2026-02-18 15:46:06 +01:00
Marko Djordjevic
2be91ceea8 task 9.5: mark task 9.5 as complete in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:45:36 +01:00
Marko Djordjevic
76c440d879 task 9.4: mark task 9.4 as done in tasks.md
src/types/predictions.ts already existed with PredictionSpan, PredictionState, and ModelInfoResponse interfaces matching actual usage across page.tsx, CandleChart.tsx, and PredictionPanel.tsx.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:44:24 +01:00
Marko Djordjevic
a7087ef945 task 9.3: mark task 9.3 as done in tasks.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:43:36 +01:00
Marko Djordjevic
f05a0081f7 feat: add shared Annotation, AnnotationType, Geometry interfaces in src/types/annotations.ts
Creates src/types/annotations.ts with typed interfaces matching actual
usage in CandleChart.tsx, page.tsx, and DB schema. Marks task 9.2 done.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:27:13 +01:00
Marko Djordjevic
8b40a2cb9a feat: create src/types/candles.ts with Candle interface (task 9.1)
Add shared Candle interface with time, open, high, low, close, and
optional volume fields. Volume is optional because the DB schema does
not store volume but the predict/patterns API routes accept it as an
optional field.

Mark task 9.1 as complete in tasks.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:26:14 +01:00
Marko Djordjevic
90d62cf187 chore: mark task 8.8 as completed 2026-02-18 15:25:23 +01:00
Marko Djordjevic
2f1e4944de Mark task 8.7 as completed - Extract magic numbers to named constants
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:24:41 +01:00
Marko Djordjevic
d6d844a003 fix: detect candle interval dynamically instead of hardcoded 60s
Replace the hardcoded 60-second assumption in CandleChart.tsx with
dynamic interval detection: compute interval as candles[1].time -
candles[0].time when at least 2 candles are available, fall back to
60 otherwise. Used for span-to-prediction-time mapping iteration.

Marks task 8.6 as done in tasks.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:23:11 +01:00
Marko Djordjevic
8395f23744 feat: add bounded prediction cache (max 100 entries, FIFO eviction)
- Before inserting into predictionCacheRef, check if size >= 100
- If so, evict oldest entry via cache.keys().next().value (FIFO)
- Applied at both cache write sites (lines ~549 and ~671 in page.tsx)
- Marks task 8.5 as done in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:22:38 +01:00
Marko Djordjevic
acf31d3edf perf: split SpanAnnotationManager reconciliation into two effects
Separate the single reconciliation effect into:
1. Full reconciliation effect — runs only when spanAnnotations list,
   series, chart, or candles change. Rebuilds all primitives.
2. Selection-only effect — runs only when selectedSpanId changes.
   Calls setSelected() on existing primitives instead of detaching
   and reattaching all of them.

This avoids tearing down and rebuilding every primitive on each
selection change, which was the main unnecessary overhead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:21:46 +01:00
Marko Djordjevic
aa27fe93f6 perf: remove fitContent() from reconciliation effect, call only on initial load
Move chart.timeScale().fitContent() out of the span annotation reconciliation
effect (which ran on every annotation/selection change) into a dedicated effect
guarded by a hasInitializedRef, so it fires only once when chart and series
first become available.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:20:37 +01:00
Marko Djordjevic
131d2912d5 fix: use chart.applyOptions() for theme changes instead of re-creating chart
Remove resolvedTheme from the chart initialization useEffect dependency
array so theme changes no longer destroy and re-create the entire chart.
Add a separate useEffect that calls chartRef.current.applyOptions() with
the new layout, grid, timeScale and rightPriceScale colors whenever
resolvedTheme changes, avoiding the memory leak and flicker caused by
full chart reconstruction on every theme toggle.

Closes task 8.2.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:19:44 +01:00
Marko Djordjevic
0cd7a34c99 fix: replace useState with useRef for preview primitive to prevent memory leak
The preview primitive in SpanAnnotationManager was stored in useState,
causing unnecessary re-renders and potential memory leaks since the
primitive was not cleaned up on unmount. Changed to useRef, updated all
read/write sites, removed from dependency arrays, and added unmount
cleanup effect that detaches the primitive from the series.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:18:48 +01:00
Marko Djordjevic
e5b0cc2540 fix: resolve stale closure in SpanAnnotationManager keyboard handler
Use selectedSpanIdRef to avoid capturing stale selectedSpanId in the
keyboard event handler closure. Wrap handleDeleteSpan in useCallback
with stable dependencies (reads selectedSpanIdRef.current instead of
the prop directly). Remove selectedSpanId from the useEffect dependency
array since the ref is used instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:16:55 +01:00
Marko Djordjevic
73c88f8211 fix: resolve stale closures in CandleChart click handler
Convert drawingState, selectedLineId, dragState, and annotations to
refs that stay in sync with state via useEffect. The chart click handler
(subscribeClick) captured stale closure values for these variables.
Now reads from refs (e.g. drawingStateRef.current) so the handler
always sees the latest state, and removed the stale state variables
from the useEffect dependency array.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:15:24 +01:00
Marko Djordjevic
45a23047dd fix: add AbortController to fetchPredictions and handleFetchBatchPredictions
Prevent race conditions by aborting in-flight requests when a new
request is triggered. Each function now:
- Aborts the previous request via a stored AbortController ref
- Passes signal to all fetch() calls
- Silently discards AbortError in catch blocks

Completes task 7.2.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:12:40 +01:00
Marko Djordjevic
838c063b5b mark task 7.1 done: stale closure fix for modelInfo already applied
The modelInfoRef with useEffect sync was already in place in page.tsx
(lines 201-205), and fetchPredictions/generateCacheKey already use
modelInfoRef.current instead of the stale closure value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 15:11:20 +01:00
Marko Djordjevic
300752da3a fix: use curl consistently for healthcheck in Dockerfile and docker-compose.yml
- Replace wget with curl in root Dockerfile healthcheck command (line 50)
- Add curl to apk dependencies in root Dockerfile (line 22)
- Align healthcheck parameters between Dockerfile and docker-compose.yml
- Mark task 6.8 as complete

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:37:52 +01:00
Marko Djordjevic
5896e56faa feat: add sha256 pinning TODO comments to both Dockerfiles
Add TODO comments above each FROM instruction in Dockerfile and
services/ml/Dockerfile instructing how to pin base images to sha256
digests for reproducible builds. Marks task 6.7 as complete.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:37:17 +01:00
Marko Djordjevic
38b270140e fix: remove unnecessary node_modules copy from Dockerfile
Standalone Next.js output includes all required dependencies in the
/app/.next/standalone directory, making the separate node_modules
copy unnecessary. This reduces image size and build complexity.

Closes task 6.6.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:36:35 +01:00
Marko Djordjevic
e146de2e05 security: add SHA256 checksum verification for TA-Lib download in ML Dockerfile
Splits the monolithic TA-Lib build RUN command to insert an ARG for the
expected SHA256 hash and a sha256sum -c verification step immediately after
the wget download, before extraction and build. Marks task 6.5 complete.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:36:06 +01:00
Marko Djordjevic
a6e0697ab5 fix: Change TA-Lib download URL to HTTPS in Dockerfile
Updated the wget command in services/ml/Dockerfile line 10 to use HTTPS instead of HTTP for downloading TA-Lib source. This improves security by ensuring encrypted transport.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:35:15 +01:00
Marko Djordjevic
b0e4a618f0 fix: update .dockerignore with all required entries and mark task 6.3 complete 2026-02-18 11:34:52 +01:00
Marko Djordjevic
1438e474e8 security: add non-root appuser to services/ml/Dockerfile
Create system user appuser with useradd, set ownership of /app,
and switch to non-root user before CMD to reduce container attack surface.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:34:26 +01:00
Marko Djordjevic
34e543ea96 feat: add security headers to next.config.js (task 6.1)
Add async headers() function with X-Frame-Options, X-Content-Type-Options,
Referrer-Policy, Permissions-Policy, and Content-Security-Policy headers
applied to all routes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:33:52 +01:00
Marko Djordjevic
3d8672121e feat: add run_id format validation to GET /training/runs/{run_id} endpoint
- Add new GET endpoint for retrieving a specific training run by run_id
- Validate run_id format with regex pattern ^[a-zA-Z0-9_-]+$ before DB access
- Return HTTP 400 for invalid run_id format, HTTP 404 for non-existent runs
- Ensure DELETE endpoint validation is correctly placed before any DB access
- Both endpoints now provide consistent security validation
- Mark task 5.8 as completed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:33:12 +01:00
Marko Djordjevic
3dc0014328 feat: add training resource limits (500MB size check + 30-min timeout)
- Import concurrent.futures for timeout support
- In _run_training_background: check df.memory_usage(deep=True).sum()
  after loading the labeled dataset; raise ValueError if > 500MB
- Wrap model.fit() in a ThreadPoolExecutor with a 1800s timeout;
  on TimeoutError update DB status to "failed" with message
  "Training timed out after 30 minutes" and return early
- Mark task 5.7 as done in openspec/changes/code-review-fix/tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:31:33 +01:00
Marko Djordjevic
f94d16c6ab fix: implement real health checks in ML service /health endpoint
- Execute SELECT 1 against PostgreSQL via SQLAlchemy session to verify DB connectivity
- HTTP GET to ${MLFLOW_TRACKING_URI}/health (default: http://localhost:5000/health) to verify MLflow
- Return HTTP 503 if any component is unhealthy, HTTP 200 only when both are healthy
- Values: "healthy" / "unhealthy" for database and mlflow fields
- Add `import requests as http_requests` and `sa_text` imports

Closes task 5.6 in openspec/changes/code-review-fix/tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:30:12 +01:00
Marko Djordjevic
d75b05b585 feat: add candle time-sort and duplicate timestamp validation to POST /predict
Sort incoming candles by their time field in ascending chronological order
before processing to ensure correct feature engineering regardless of input
order. Also validate that no duplicate timestamps are present, returning
HTTP 400 with details if duplicates are found.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:28:02 +01:00
Marko Djordjevic
ff952aa083 feat(ml): add date range validation to POST /predict/batch
- Parse start_date and end_date as datetime objects
- Return HTTP 400 if end_date is before start_date
- Return HTTP 400 if date range exceeds 365 days (1 year)

Closes task 5.4 in code-review-fix tasks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:27:12 +01:00
Marko Djordjevic
b9beea1574 fix(ml): add _model_swap_lock to prediction reads for thread-safe model access
In /predict and /predict/batch endpoints, grab the model reference under
_model_swap_lock before running inference. Inference itself runs outside
the lock (using a local variable) to avoid blocking model swaps during
potentially slow computation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:26:33 +01:00
Marko Djordjevic
ff15adc847 feat: add SHA256 model integrity check before joblib.load()
Add verify_model_checksum() that validates model files against a
models/checksums.sha256 manifest before loading. Fails open when
manifest is missing or file not listed (backward compat), raises
HTTP 500 on hash mismatch. Created empty manifest placeholder.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:25:14 +01:00
Marko Djordjevic
b7f9b2e04d security: replace exception details with generic error in ML service HTTP responses
Replace all instances of `detail=str(e)`, `detail=f"...{exc}"`, and similar
patterns that exposed internal exception messages to HTTP clients in
services/ml/app/main.py. All exception details are now logged server-side
only via logger.error(), while clients receive a generic "Internal server error"
message. Fixes 9 handlers across predict, batch predict, pattern detection,
training start, training runs fetch, training run delete, dataset info,
build dataset, and model load endpoints.

Mark task 5.1 as completed in tasks.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:23:41 +01:00
Marko Djordjevic
4c53ef7cae fix: add response.ok checks before .json() in CandleChart.tsx
Added HTTP error checks before calling .json() in the three fetch
functions (fetchCandles, fetchAnnotations, fetchAnnotationTypes)
to prevent silent failures on non-2xx responses.

Marks task 4.12 as complete in tasks.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:21:54 +01:00
Marko Djordjevic
4436cd655f fix: add response.ok checks before .json() in page.tsx fetch calls
Guard all four fetch() calls in src/app/page.tsx against non-2xx HTTP
responses by throwing before attempting to parse the body as JSON.
Affected functions: fetchCharts, fetchAnnotations, fetchSpanAnnotations,
fetchSpanLabelTypes.

Marks task 4.11 as completed in code-review-fix/tasks.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:21:21 +01:00
Marko Djordjevic
b2129ad626 security: add CSV injection protection to all export routes
Add sanitizeCsvCell() helper to both export routes that prefixes cell
values starting with =, +, @, or - with a single quote to prevent CSV
formula injection attacks.

Applied to:
- src/app/api/export/route.ts: timestamp and label_type columns
- src/app/api/span-annotations/export/route.ts: start_time, end_time,
  label, and outcome columns

Closes task 4.10.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:20:36 +01:00
Marko Djordjevic
160f146ab4 mark: task 4.9 complete - integer parameter parsing with isNaN guards 2026-02-18 11:19:34 +01:00
Marko Djordjevic
1678da2d9d fix: wrap chart cascade delete in db.transaction() and add spanAnnotations deletion
- Import spanAnnotations from schema
- Wrap all delete operations in db.transaction() for atomicity
- Delete spanAnnotations first to satisfy FK constraints, then annotations, candles, chart
- Mark task 4.8 as done in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:17:19 +01:00
Marko Djordjevic
103bfa89cb fix: require chartId for bulk delete in annotations route (task 4.7)
Reject DELETE ?all=true without chartId with HTTP 400 to prevent
accidental deletion of annotations across all charts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:16:37 +01:00
Marko Djordjevic
aace19b7f4 fix: replace error.message with generic "Internal server error" in all API catch blocks
Prevents leaking internal error details to clients across 7 route files:
health, candles, annotations, annotations/[id], upload, export, span-annotations/export.
Server-side console.error logging preserved for debugging.

Closes task 4.6.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:16:02 +01:00
Marko Djordjevic
81e3554d82 feat: add Zod schema validation to patterns/detect route
- Import z from zod
- Add CandleSchema validating time, open, high, low, close (number), volume (optional number)
- Add PatternDetectRequestSchema validating candles array and patterns array of non-empty strings
- Use safeParse() and return HTTP 400 with error details on validation failure
- Forward only validated data to the inference service
- Mark task 4.5 as completed in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:14:36 +01:00
Marko Djordjevic
2e02d155af feat: add Zod schema validation to training/start route (task 4.4)
Validates model_type as a non-empty string using .safeParse(); returns
HTTP 400 with error details on invalid input. Marks task 4.4 as done.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:14:00 +01:00
Marko Djordjevic
4cffc223b3 feat: add Zod schema validation to model/load route
Validate run_id in POST /api/model/load using Zod:
- run_id must be a non-empty string matching /^[a-zA-Z0-9_-]+$/
- Returns HTTP 400 with error details if validation fails
- Validated data is forwarded to the inference service

Marks task 4.3 as complete in tasks.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:13:13 +01:00
Marko Djordjevic
5c399037c3 feat: add Zod validation to predict/batch route (task 4.2)
Add BatchPredictRequestSchema with Zod to validate pair, timeframe,
start_date, and end_date fields. Returns HTTP 400 with flattened error
details on invalid input. Forward only validated data to the inference
service.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:12:38 +01:00
Marko Djordjevic
3361236d3f feat: add Zod schema validation to predict API route
- Add CandleSchema validating time, open, high, low, close (number) and optional volume
- Add PredictRequestSchema validating pair (non-empty string), timeframe (non-empty string), candles array
- Use safeParse() and return HTTP 400 with error details on invalid input
- Forward only validated data to the inference service
- Mark task 4.1 as done in tasks.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:11:58 +01:00