ai-teacher/README.md

# AI Teacher — Neurosurgeon RAG Learning Platform

A web application for neurosurgeons to upload medical textbooks (PDF), have them
embedded into a pgvector store, then select from a predefined topic list to receive
AI-generated cross-book summaries, and engage in grounded RAG chat.

## Architecture

```mermaid
graph TD
    User["Neurosurgeon (Browser)"]
    FE["Frontend\nVue.js 3 / Vite\n:5173"]
    BE["Backend\nSpring Boot 4 / Spring AI\n:8080"]
    DB["PostgreSQL + pgvector\n(source of truth)"]
    FS["File Store\nuploads/ (local disk)\nExtracted figure PNGs"]
    LLM["LLM Provider\n(OpenAI)\nEmbeddings + Chat + Vision"]

    User -->|HTTP| FE
    FE -->|REST /api/v1/...| BE
    BE -->|"JDBC — books, chapters,\nsections, figures, refs"| DB
    BE -->|"pgvector — text chunks\n+ figure caption vectors"| DB
    BE -->|"PNG read/write\n(figure extraction)"| FS
    FE -->|"GET /api/v1/figures/**\n(static file serving)"| BE
    BE -->|"Embedding + Chat\n+ Vision (image description)"| LLM

    subgraph "Embedding Pipeline (per PDF upload)"
        EP1["Parse pages → SectionEntity"]
        EP2["Extract images → FigureEntity"]
        EP3["Vision describe → embed caption"]
        EP4["Chunk text → embed chunks"]
        EP5["Link chunks ↔ figures"]
        EP1 --> EP2
        EP1 --> EP4
        EP2 --> EP3
        EP4 --> EP5
        EP3 --> EP5
    end

    subgraph "Retrieval Pipeline (per chat query)"
        RP1["Text chunk search (topK=5)"]
        RP2["Figure caption search (topK=3)"]
        RP3["Expand chunks → full section text"]
        RP4["Fetch linked figures (chunk_figure_ref)"]
        RP5["Merge + deduplicate figures"]
        RP6["Build LLM prompt + call"]
        RP1 --> RP3
        RP1 --> RP4
        RP2 --> RP5
        RP4 --> RP5
        RP3 --> RP6
        RP5 --> RP6
    end
```

## Marker API Response Structure

The PDF parsing pipeline calls a local [Marker](https://github.com/VikParuchuri/marker) server (`POST /marker/upload`).

### Top-level envelope

```json
{
  "format": "json",
  "output": "<JSON-encoded string>"
}
```

`output` is a **JSON-encoded string** (not a nested object) and must be parsed a second time to get the document tree.

### Parsed `output` shape

```
{
  "children": [ <Page block>, ... ]
}
```

### Block types

Every block shares these fields:

| Field            | Type              | Notes                                      |
|------------------|-------------------|--------------------------------------------|
| `id`             | string            | e.g. `/page/0/Picture/2`                   |
| `block_type`     | string            | see table below                            |
| `html`           | string            | rendered HTML; may contain `<content-ref>` |
| `bbox`           | `[x0,y0,x1,y1]`  | bounding box in page coordinates           |
| `children`       | array or null     | nested blocks                              |
| `images`         | object or null    | base64 PNG map (leaf image blocks only)    |
| `section_hierarchy` | object         | heading ancestry                           |

#### Known `block_type` values

| block_type       | Category | Notes                                                 |
|------------------|----------|-------------------------------------------------------|
| `Page`           | structure | Top-level; direct children are the page content       |
| `SectionHeader`  | text      | Section / chapter heading                             |
| `Text`           | text      |                                                       |
| `TextInlineMath` | text      |                                                       |
| `ListItem`       | text      |                                                       |
| `Table`          | text      |                                                       |
| `Code`           | text      |                                                       |
| `Equation`       | text      |                                                       |
| `Footnote`       | text      |                                                       |
| `Caption`        | text      | Usually a child of a `*Group` block                   |
| `PageHeader`     | text      |                                                       |
| `PageFooter`     | text      |                                                       |
| `Handwriting`    | text      |                                                       |
| `Picture`        | image     | Leaf block; `images` map holds base64 PNG keyed by ID |
| `Figure`         | image     | Leaf block; same as `Picture`                         |
| `PictureGroup`   | container | Wraps one `Picture` + one `Caption` child             |
| `FigureGroup`    | container | Wraps one `Figure` + one `Caption` child              |

### Image extraction

Images are only present on **leaf** image blocks (`Picture`, `Figure`).
Group blocks (`PictureGroup`, `FigureGroup`) have `images: null` — the base64 PNG lives on the child leaf block.

```
PictureGroup
├── Picture   ← images: { "/page/0/Picture/2": "<base64 PNG>" }
└── Caption   ← html: "<p>Figure 1 — ...</p>"
```

## Stack

- **Backend**: Spring Boot 4.0.5 + Spring AI 2.0.0-M4, Java 21, Maven
- **Frontend**: Vue.js 3 + Vite + TypeScript + Pinia + Axios
- **Database**: PostgreSQL 16 + pgvector extension
- **Auth**: HTTP Basic (single shared in-memory user)

## Quick Start

See [specs/001-neuro-rag-learning/quickstart.md](specs/001-neuro-rag-learning/quickstart.md) for full instructions.

### Local Dev

```bash
# Start the database
docker compose up -d

# Backend
cd backend
mvn spring-boot:run

# Frontend
cd frontend
npm install
npm run dev
```

### Environment Variables

| Variable | Required | Description |
|----------|----------|-------------|
| `OPENAI_API_KEY` | Yes | OpenAI API key for embeddings and chat |
| `APP_PASSWORD` | Yes | Shared password for HTTP Basic auth |
| `DB_URL` | Yes | JDBC URL, e.g. `jdbc:postgresql://localhost:5432/aiteacher` |
| `DB_USERNAME` | Yes | Database username |
| `DB_PASSWORD` | Yes | Database password |
| `FIGURE_STORAGE_PATH` | No | Base path for uploaded PDFs and extracted figures (default: `./uploads`) |