first implementation - image/drawing integration
This commit is contained in:
@@ -11,13 +11,45 @@ graph TD
|
||||
User["Neurosurgeon (Browser)"]
|
||||
FE["Frontend\nVue.js 3 / Vite\n:5173"]
|
||||
BE["Backend\nSpring Boot 4 / Spring AI\n:8080"]
|
||||
DB["PostgreSQL + pgvector\n(provided)"]
|
||||
LLM["LLM Provider\n(OpenAI / configurable)"]
|
||||
DB["PostgreSQL + pgvector\n(source of truth)"]
|
||||
FS["File Store\nuploads/ (local disk)\nExtracted figure PNGs"]
|
||||
LLM["LLM Provider\n(OpenAI)\nEmbeddings + Chat + Vision"]
|
||||
|
||||
User -->|HTTP| FE
|
||||
FE -->|REST /api/v1/...| BE
|
||||
BE -->|JDBC / pgvector| DB
|
||||
BE -->|Embedding + Chat API| LLM
|
||||
BE -->|"JDBC — books, chapters,\nsections, figures, refs"| DB
|
||||
BE -->|"pgvector — text chunks\n+ figure caption vectors"| DB
|
||||
BE -->|"PNG read/write\n(figure extraction)"| FS
|
||||
FE -->|"GET /api/v1/figures/**\n(static file serving)"| BE
|
||||
BE -->|"Embedding + Chat\n+ Vision (image description)"| LLM
|
||||
|
||||
subgraph "Embedding Pipeline (per PDF upload)"
|
||||
EP1["Parse pages → SectionEntity"]
|
||||
EP2["Extract images → FigureEntity"]
|
||||
EP3["Vision describe → embed caption"]
|
||||
EP4["Chunk text → embed chunks"]
|
||||
EP5["Link chunks ↔ figures"]
|
||||
EP1 --> EP2
|
||||
EP1 --> EP4
|
||||
EP2 --> EP3
|
||||
EP4 --> EP5
|
||||
EP3 --> EP5
|
||||
end
|
||||
|
||||
subgraph "Retrieval Pipeline (per chat query)"
|
||||
RP1["Text chunk search (topK=5)"]
|
||||
RP2["Figure caption search (topK=3)"]
|
||||
RP3["Expand chunks → full section text"]
|
||||
RP4["Fetch linked figures (chunk_figure_ref)"]
|
||||
RP5["Merge + deduplicate figures"]
|
||||
RP6["Build LLM prompt + call"]
|
||||
RP1 --> RP3
|
||||
RP1 --> RP4
|
||||
RP2 --> RP5
|
||||
RP4 --> RP5
|
||||
RP3 --> RP6
|
||||
RP5 --> RP6
|
||||
end
|
||||
```
|
||||
|
||||
## Stack
|
||||
@@ -56,3 +88,4 @@ npm run dev
|
||||
| `DB_URL` | Yes | JDBC URL, e.g. `jdbc:postgresql://localhost:5432/aiteacher` |
|
||||
| `DB_USERNAME` | Yes | Database username |
|
||||
| `DB_PASSWORD` | Yes | Database password |
|
||||
| `FIGURE_STORAGE_PATH` | No | Base path for uploaded PDFs and extracted figures (default: `./uploads`) |
|
||||
|
||||
Reference in New Issue
Block a user