ai-teacher/specs/002-image-aware-embedding/plan.md

Implementation Plan: Enhanced Embedding with Image Parsing and Metadata

Branch: 002-image-aware-embedding | Date: 2026-04-04 | Spec: spec.md
Input: Feature specification from /specs/002-image-aware-embedding/spec.md

Summary

Enhance the PDF embedding pipeline to extract figures and generate AI descriptions for them, making image content semantically searchable alongside text. PDF parsing and figure extraction are delegated to a local Marker server (http://localhost:8000/marker/upload), which returns reading-order text and pre-cropped figure images (base64) in a single JSON response, eliminating the need for PDFBox column heuristics and figure bbox rendering.

Technical Context

Language/Version: Java 25 (backend), TypeScript / Node 20 (frontend)
Primary Dependencies: Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API (embeddings + GPT-4o vision), PDFBox 3.0.3 (via spring-ai-pdf-document-reader — retained transitively, no longer used directly), Marker local HTTP API (http://localhost:8000/marker/upload)
Storage: PostgreSQL (JPA + Flyway), pgvector (Spring AI VectorStore), S3-compatible object store (figure images via FigureStorageService)
Testing: Maven / JUnit 5 (spring-boot-starter-test)
Target Platform: Linux server
Project Type: Web application (backend API + frontend client)
Performance Goals: SC-003 — book processing time ≤ 3× text-only for ≤ 500 pages
Constraints: REST API only (Constitution III); Marker server must be running locally; S3-compatible storage configured via env vars
Scale/Scope: POC — handful of books, <10 users

Constitution Check

GATE: Must pass before Phase 0 research. Re-checked after Phase 1 design.

Principle	Status	Notes
I. KISS	✅ Justified	Marker replaces a bespoke PDFBox column heuristic + Google Cloud SDK with one HTTP call. Net complexity reduction vs. the Document AI approach.
II. Easy to Change	✅	MarkerPageParser is the only class that knows about Marker; swap the implementation to replace Marker with any other parser. PageResult DTO remains unchanged.
III. Web-First	✅	Internal pipeline change; no public API contract change.
IV. Documentation	✅	README must be updated to show Marker as a local external service.

Project Structure

Documentation (this feature)

specs/002-image-aware-embedding/
├── plan.md              # This file
├── research.md          # Phase 0 output
├── data-model.md        # Phase 1 output
├── quickstart.md        # Phase 1 output
├── contracts/
│   ├── api.md           # HTTP API contracts (unchanged from initial plan)
│   └── marker-page-result.md  # Internal DTO contract (MarkerPageParser → downstream)
└── tasks.md             # Phase 2 output (/speckit.tasks — not created here)

Source Code

backend/
├── src/main/java/com/aiteacher/
│   ├── config/
│   │   └── MarkerConfig.java          # NEW: RestClient bean + base-url property
│   ├── document/
│   │   ├── MarkerPageParser.java      # NEW: replaces DocumentAiPageParser + PdfStructureParser
│   │   ├── PageResult.java            # UPDATED: FigureBbox → FigureData (bytes not bbox)
│   │   ├── FigureExtractionService.java  # UPDATED: no PDFBox render; decode bytes directly
│   │   ├── TextChunkingService.java   # UNCHANGED
│   │   ├── VisionDescriptionService.java # UNCHANGED
│   │   └── [removed] DocumentAiPageParser.java
│   ├── book/
│   │   └── BookEmbeddingService.java  # MINOR UPDATE: inject MarkerPageParser, drop DocumentAiPageParser
│   └── [removed] config/DocumentAiConfig.java
├── src/main/resources/
│   └── application.yaml               # UPDATED: remove document-ai.*, add marker.base-url
└── pom.xml                            # UPDATED: remove google-cloud-document-ai

Structure Decision: Option 2 (backend + frontend) per constitution Technology Constraints. Frontend changes are display-only (render figure citations inline).

Complexity Tracking

No constitution violations — Marker reduces complexity compared to the previous Google Document AI approach (fewer dependencies, no GCP credentials, no 15-page batching).