first implementation - image/drawing integration

This commit is contained in:
Adrien
2026-04-04 12:56:56 +02:00
parent fc5b22fba1
commit 5acfdd33c1
42 changed files with 2854 additions and 151 deletions
+37 -4
View File
@@ -11,13 +11,45 @@ graph TD
User["Neurosurgeon (Browser)"]
FE["Frontend\nVue.js 3 / Vite\n:5173"]
BE["Backend\nSpring Boot 4 / Spring AI\n:8080"]
DB["PostgreSQL + pgvector\n(provided)"]
LLM["LLM Provider\n(OpenAI / configurable)"]
DB["PostgreSQL + pgvector\n(source of truth)"]
FS["File Store\nuploads/ (local disk)\nExtracted figure PNGs"]
LLM["LLM Provider\n(OpenAI)\nEmbeddings + Chat + Vision"]
User -->|HTTP| FE
FE -->|REST /api/v1/...| BE
BE -->|JDBC / pgvector| DB
BE -->|Embedding + Chat API| LLM
BE -->|"JDBC — books, chapters,\nsections, figures, refs"| DB
BE -->|"pgvector — text chunks\n+ figure caption vectors"| DB
BE -->|"PNG read/write\n(figure extraction)"| FS
FE -->|"GET /api/v1/figures/**\n(static file serving)"| BE
BE -->|"Embedding + Chat\n+ Vision (image description)"| LLM
subgraph "Embedding Pipeline (per PDF upload)"
EP1["Parse pages → SectionEntity"]
EP2["Extract images → FigureEntity"]
EP3["Vision describe → embed caption"]
EP4["Chunk text → embed chunks"]
EP5["Link chunks ↔ figures"]
EP1 --> EP2
EP1 --> EP4
EP2 --> EP3
EP4 --> EP5
EP3 --> EP5
end
subgraph "Retrieval Pipeline (per chat query)"
RP1["Text chunk search (topK=5)"]
RP2["Figure caption search (topK=3)"]
RP3["Expand chunks → full section text"]
RP4["Fetch linked figures (chunk_figure_ref)"]
RP5["Merge + deduplicate figures"]
RP6["Build LLM prompt + call"]
RP1 --> RP3
RP1 --> RP4
RP2 --> RP5
RP4 --> RP5
RP3 --> RP6
RP5 --> RP6
end
```
## Stack
@@ -56,3 +88,4 @@ npm run dev
| `DB_URL` | Yes | JDBC URL, e.g. `jdbc:postgresql://localhost:5432/aiteacher` |
| `DB_USERNAME` | Yes | Database username |
| `DB_PASSWORD` | Yes | Database password |
| `FIGURE_STORAGE_PATH` | No | Base path for uploaded PDFs and extracted figures (default: `./uploads`) |