Compare commits
14 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 820734c251 | |||
| 0711e40c66 | |||
| 0db31e91ab | |||
| d480d04145 | |||
| c2d034d1fe | |||
| 0908355704 | |||
| 8e227a9429 | |||
| d8bcdce879 | |||
| aee6a9dfba | |||
| 0cf318f0a7 | |||
| e5d53b4e80 | |||
| 5c641f4bcc | |||
| ea1276dc2e | |||
| b154e29f2d |
@@ -0,0 +1,27 @@
|
|||||||
|
.git/
|
||||||
|
.gitignore
|
||||||
|
*.md
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
# Java build artifacts
|
||||||
|
target/
|
||||||
|
*.class
|
||||||
|
*.jar
|
||||||
|
|
||||||
|
# Node
|
||||||
|
node_modules/
|
||||||
|
dist/
|
||||||
|
*.log
|
||||||
|
|
||||||
|
# Env files (never bake secrets into images)
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
!.env.example
|
||||||
|
|
||||||
|
# Spec / docs
|
||||||
|
specs/
|
||||||
|
|
||||||
|
# Editor
|
||||||
|
.vscode/
|
||||||
|
.idea/
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
# Copy this file to .env and fill in your values before running docker-compose.native.yml
|
||||||
|
# .env is gitignored — never commit real credentials
|
||||||
|
|
||||||
|
# OpenAI
|
||||||
|
OPENAI_API_KEY=sk-...
|
||||||
|
|
||||||
|
# AWS S3 (figure storage — leave blank if using local filesystem)
|
||||||
|
AWS_ACCESS_KEY_ID=
|
||||||
|
AWS_SECRET_ACCESS_KEY=
|
||||||
|
AWS_REGION=eu-west-1
|
||||||
|
|
||||||
|
# S3 bucket name (if S3 storage enabled)
|
||||||
|
APP_STORAGE_S3_BUCKET=ai-teacher-figures
|
||||||
@@ -1,10 +1,23 @@
|
|||||||
# ai-teacher Development Guidelines
|
# ai-teacher Development Guidelines
|
||||||
|
|
||||||
Auto-generated from all feature plans. Last updated: 2026-04-03
|
Auto-generated from all feature plans. Last updated: 2026-04-10
|
||||||
|
|
||||||
## Active Technologies
|
## Active Technologies
|
||||||
- Java 25 (backend), TypeScript / Node 20 (frontend) + Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API (embeddings + chat), PDFBox (via Spring AI PDF reader dependency) (002-image-aware-embedding)
|
- Java 25 (backend), TypeScript / Node 20 (frontend) + Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API (embeddings + chat), PDFBox (via Spring AI PDF reader dependency) (002-image-aware-embedding)
|
||||||
- PostgreSQL (JPA + Flyway), pgvector (Spring AI `VectorStore`), local file system (extracted images — `/uploads/figures/`) (002-image-aware-embedding)
|
- PostgreSQL (JPA + Flyway), pgvector (Spring AI `VectorStore`), local file system (extracted images — `/uploads/figures/`) (002-image-aware-embedding)
|
||||||
|
- Java 25 (backend), TypeScript / Node 20 (frontend) + Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API, PDFBox (rendering only), `com.google.cloud:google-cloud-documentai` (~2.40.x) (002-image-aware-embedding)
|
||||||
|
- PostgreSQL (JPA + Flyway), pgvector (Spring AI VectorStore), S3 / local filesystem (figure images) (002-image-aware-embedding)
|
||||||
|
- PostgreSQL (JPA + Flyway), pgvector (Spring AI `VectorStore`), S3-compatible (002-image-aware-embedding)
|
||||||
|
- Java 21 (backend) / TypeScript + Node 20 (frontend) + Spring Boot 4.0.5, Spring Security (already included), Vue 3.4, Vue Router 4.3, Pinia 2.1, Axios 1.7 (003-basic-login)
|
||||||
|
- No new storage — credentials held in browser `sessionStorage` (frontend only) (003-basic-login)
|
||||||
|
- Java 21 (backend), TypeScript / Node 20 (frontend) + Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API (chat + embeddings), pgvector, Vue 3.4, Pinia 2.1 (004-rag-retrieval-quality)
|
||||||
|
- PostgreSQL (sections, figures, messages — unchanged). No new tables needed. (004-rag-retrieval-quality)
|
||||||
|
- Java 21 (backend), TypeScript / Node 20 (frontend) + Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API (chat + embeddings), Vue 3.4, Pinia 2.1, Axios 1.7 (004-rag-retrieval-quality)
|
||||||
|
- PostgreSQL (JPA + Flyway), pgvector (`VectorStore`) (004-rag-retrieval-quality)
|
||||||
|
- Java 25 (backend), TypeScript / Node 20 (frontend) + Spring Boot 4.0.5, Spring AI 2.0.0-M4, `native-maven-plugin` 0.10.6, (005-native-image-deployment)
|
||||||
|
- PostgreSQL 16 + pgvector (unchanged) (005-native-image-deployment)
|
||||||
|
- TypeScript / Node 20 (frontend only) + Vue 3.4, Vue Router 4.3, Pinia 2.1 — no changes (006-mobile-responsive-ui)
|
||||||
|
- N/A (frontend-only change) (006-mobile-responsive-ui)
|
||||||
|
|
||||||
- Java 21 (backend), TypeScript / Node 20 (frontend) (001-neuro-rag-learning)
|
- Java 21 (backend), TypeScript / Node 20 (frontend) (001-neuro-rag-learning)
|
||||||
|
|
||||||
@@ -24,9 +37,10 @@ npm test && npm run lint
|
|||||||
Java 21 (backend), TypeScript / Node 20 (frontend): Follow standard conventions
|
Java 21 (backend), TypeScript / Node 20 (frontend): Follow standard conventions
|
||||||
|
|
||||||
## Recent Changes
|
## Recent Changes
|
||||||
- 002-image-aware-embedding: Added Java 25 (backend), TypeScript / Node 20 (frontend) + Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API (embeddings + chat), PDFBox (via Spring AI PDF reader dependency)
|
- 006-mobile-responsive-ui: Added TypeScript / Node 20 (frontend only) + Vue 3.4, Vue Router 4.3, Pinia 2.1 — no changes
|
||||||
|
- 005-native-image-deployment: Added Java 25 (backend), TypeScript / Node 20 (frontend) + Spring Boot 4.0.5, Spring AI 2.0.0-M4, `native-maven-plugin` 0.10.6,
|
||||||
|
- 004-rag-retrieval-quality: Added Java 21 (backend), TypeScript / Node 20 (frontend) + Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API (chat + embeddings), Vue 3.4, Pinia 2.1, Axios 1.7
|
||||||
|
|
||||||
- 001-neuro-rag-learning: Added Java 21 (backend), TypeScript / Node 20 (frontend)
|
|
||||||
|
|
||||||
<!-- MANUAL ADDITIONS START -->
|
<!-- MANUAL ADDITIONS START -->
|
||||||
<!-- MANUAL ADDITIONS END -->
|
<!-- MANUAL ADDITIONS END -->
|
||||||
|
|||||||
@@ -0,0 +1,566 @@
|
|||||||
|
# Marker
|
||||||
|
|
||||||
|
Marker converts documents to markdown, JSON, chunks, and HTML quickly and accurately.
|
||||||
|
|
||||||
|
- Converts PDF, image, PPTX, DOCX, XLSX, HTML, EPUB files in all languages
|
||||||
|
- Formats tables, forms, equations, inline math, links, references, and code blocks
|
||||||
|
- Extracts and saves images
|
||||||
|
- Removes headers/footers/other artifacts
|
||||||
|
- Extensible with your own formatting and logic
|
||||||
|
- Does structured extraction, given a JSON schema (beta)
|
||||||
|
- Optionally boost accuracy with LLMs (and your own prompt)
|
||||||
|
- Works on GPU, CPU, or MPS
|
||||||
|
|
||||||
|
For our managed API or on-prem document intelligence solution, check out [our platform here](https://datalab.to?utm_source=gh-marker).
|
||||||
|
|
||||||
|
## Performance
|
||||||
|
|
||||||
|
<img src="data/images/overall.png" width="800px"/>
|
||||||
|
|
||||||
|
Marker benchmarks favorably compared to cloud services like Llamaparse and Mathpix, as well as other open source tools.
|
||||||
|
|
||||||
|
The above results are running single PDF pages serially. Marker is significantly faster when running in batch mode, with a projected throughput of 25 pages/second on an H100.
|
||||||
|
|
||||||
|
See [below](#benchmarks) for detailed speed and accuracy benchmarks, and instructions on how to run your own benchmarks.
|
||||||
|
|
||||||
|
## Hybrid Mode
|
||||||
|
|
||||||
|
For the highest accuracy, pass the `--use_llm` flag to use an LLM alongside marker. This will do things like merge tables across pages, handle inline math, format tables properly, and extract values from forms. It can use any gemini or ollama model. By default, it uses `gemini-2.0-flash`. See [below](#llm-services) for details.
|
||||||
|
|
||||||
|
Here is a table benchmark comparing marker, gemini flash alone, and marker with use_llm:
|
||||||
|
|
||||||
|
<img src="data/images/table.png" width="400px"/>
|
||||||
|
|
||||||
|
As you can see, the use_llm mode offers higher accuracy than marker or gemini alone.
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
| PDF | File type | Markdown | JSON |
|
||||||
|
|-----|-----------|------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------|
|
||||||
|
| [Think Python](https://greenteapress.com/thinkpython/thinkpython.pdf) | Textbook | [View](https://github.com/VikParuchuri/marker/blob/master/data/examples/markdown/thinkpython/thinkpython.md) | [View](https://github.com/VikParuchuri/marker/blob/master/data/examples/json/thinkpython.json) |
|
||||||
|
| [Switch Transformers](https://arxiv.org/pdf/2101.03961.pdf) | arXiv paper | [View](https://github.com/VikParuchuri/marker/blob/master/data/examples/markdown/switch_transformers/switch_trans.md) | [View](https://github.com/VikParuchuri/marker/blob/master/data/examples/json/switch_trans.json) |
|
||||||
|
| [Multi-column CNN](https://arxiv.org/pdf/1804.07821.pdf) | arXiv paper | [View](https://github.com/VikParuchuri/marker/blob/master/data/examples/markdown/multicolcnn/multicolcnn.md) | [View](https://github.com/VikParuchuri/marker/blob/master/data/examples/json/multicolcnn.json) |
|
||||||
|
|
||||||
|
# Commercial usage
|
||||||
|
|
||||||
|
Our model weights use a modified AI Pubs Open Rail-M license (free for research, personal use, and startups under $2M funding/revenue) and our code is GPL. For broader commercial licensing or to remove GPL requirements, visit our pricing page [here](https://www.datalab.to/pricing?utm_source=gh-marker).
|
||||||
|
|
||||||
|
# Hosted API & On-prem
|
||||||
|
|
||||||
|
There's a [hosted API](https://www.datalab.to?utm_source=gh-marker) and [painless on-prem solution](https://www.datalab.to/blog/self-serve-on-prem-licensing) for marker - it's free to sign up, and we'll throw in credits for you to test it out.
|
||||||
|
|
||||||
|
The API:
|
||||||
|
- Supports PDF, image, PPT, PPTX, DOC, DOCX, XLS, XLSX, HTML, EPUB files
|
||||||
|
- Is 1/4th the price of leading cloud-based competitors
|
||||||
|
- Fast - ~15s for a 250 page PDF
|
||||||
|
- Supports LLM mode
|
||||||
|
- High uptime (99.99%)
|
||||||
|
|
||||||
|
# Community
|
||||||
|
|
||||||
|
[Discord](https://discord.gg//KuZwXNGnfH) is where we discuss future development.
|
||||||
|
|
||||||
|
# Installation
|
||||||
|
|
||||||
|
You'll need python 3.10+ and [PyTorch](https://pytorch.org/get-started/locally/).
|
||||||
|
|
||||||
|
Install with:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
pip install marker-pdf
|
||||||
|
```
|
||||||
|
|
||||||
|
If you want to use marker on documents other than PDFs, you will need to install additional dependencies with:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
pip install marker-pdf[full]
|
||||||
|
```
|
||||||
|
|
||||||
|
# Usage
|
||||||
|
|
||||||
|
First, some configuration:
|
||||||
|
|
||||||
|
- Your torch device will be automatically detected, but you can override this. For example, `TORCH_DEVICE=cuda`.
|
||||||
|
- Some PDFs, even digital ones, have bad text in them. Set `--force_ocr` to force OCR on all lines, or the `strip_existing_ocr` to keep all digital text, and strip out any existing OCR text.
|
||||||
|
- If you care about inline math, set `force_ocr` to convert inline math to LaTeX.
|
||||||
|
|
||||||
|
## Interactive App
|
||||||
|
|
||||||
|
I've included a streamlit app that lets you interactively try marker with some basic options. Run it with:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
pip install streamlit streamlit-ace
|
||||||
|
marker_gui
|
||||||
|
```
|
||||||
|
|
||||||
|
## Convert a single file
|
||||||
|
|
||||||
|
```shell
|
||||||
|
marker_single /path/to/file.pdf
|
||||||
|
```
|
||||||
|
|
||||||
|
You can pass in PDFs or images.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--page_range TEXT`: Specify which pages to process. Accepts comma-separated page numbers and ranges. Example: `--page_range "0,5-10,20"` will process pages 0, 5 through 10, and page 20.
|
||||||
|
- `--output_format [markdown|json|html|chunks]`: Specify the format for the output results.
|
||||||
|
- `--output_dir PATH`: Directory where output files will be saved. Defaults to the value specified in settings.OUTPUT_DIR.
|
||||||
|
- `--paginate_output`: Paginates the output, using `\n\n{PAGE_NUMBER}` followed by `-` * 48, then `\n\n`
|
||||||
|
- `--use_llm`: Uses an LLM to improve accuracy. You will need to configure the LLM backend - see [below](#llm-services).
|
||||||
|
- `--force_ocr`: Force OCR processing on the entire document, even for pages that might contain extractable text. This will also format inline math properly.
|
||||||
|
- `--block_correction_prompt`: if LLM mode is active, an optional prompt that will be used to correct the output of marker. This is useful for custom formatting or logic that you want to apply to the output.
|
||||||
|
- `--strip_existing_ocr`: Remove all existing OCR text in the document and re-OCR with surya.
|
||||||
|
- `--redo_inline_math`: If you want the absolute highest quality inline math conversion, use this along with `--use_llm`.
|
||||||
|
- `--disable_image_extraction`: Don't extract images from the PDF. If you also specify `--use_llm`, then images will be replaced with a description.
|
||||||
|
- `--debug`: Enable debug mode for additional logging and diagnostic information.
|
||||||
|
- `--processors TEXT`: Override the default processors by providing their full module paths, separated by commas. Example: `--processors "module1.processor1,module2.processor2"`
|
||||||
|
- `--config_json PATH`: Path to a JSON configuration file containing additional settings.
|
||||||
|
- `config --help`: List all available builders, processors, and converters, and their associated configuration. These values can be used to build a JSON configuration file for additional tweaking of marker defaults.
|
||||||
|
- `--converter_cls`: One of `marker.converters.pdf.PdfConverter` (default) or `marker.converters.table.TableConverter`. The `PdfConverter` will convert the whole PDF, the `TableConverter` will only extract and convert tables.
|
||||||
|
- `--llm_service`: Which llm service to use if `--use_llm` is passed. This defaults to `marker.services.gemini.GoogleGeminiService`.
|
||||||
|
- `--help`: see all of the flags that can be passed into marker. (it supports many more options then are listed above)
|
||||||
|
|
||||||
|
The list of supported languages for surya OCR is [here](https://github.com/VikParuchuri/surya/blob/master/surya/recognition/languages.py). If you don't need OCR, marker can work with any language.
|
||||||
|
|
||||||
|
## Convert multiple files
|
||||||
|
|
||||||
|
```shell
|
||||||
|
marker /path/to/input/folder
|
||||||
|
```
|
||||||
|
|
||||||
|
- `marker` supports all the same options from `marker_single` above.
|
||||||
|
- `--workers` is the number of conversion workers to run simultaneously. This is automatically set by default, but you can increase it to increase throughput, at the cost of more CPU/GPU usage. Marker will use 5GB of VRAM per worker at the peak, and 3.5GB average.
|
||||||
|
|
||||||
|
## Convert multiple files on multiple GPUs
|
||||||
|
|
||||||
|
```shell
|
||||||
|
NUM_DEVICES=4 NUM_WORKERS=15 marker_chunk_convert ../pdf_in ../md_out
|
||||||
|
```
|
||||||
|
|
||||||
|
- `NUM_DEVICES` is the number of GPUs to use. Should be `2` or greater.
|
||||||
|
- `NUM_WORKERS` is the number of parallel processes to run on each GPU.
|
||||||
|
|
||||||
|
## Use from python
|
||||||
|
|
||||||
|
See the `PdfConverter` class at `marker/converters/pdf.py` function for additional arguments that can be passed.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from marker.converters.pdf import PdfConverter
|
||||||
|
from marker.models import create_model_dict
|
||||||
|
from marker.output import text_from_rendered
|
||||||
|
|
||||||
|
converter = PdfConverter(
|
||||||
|
artifact_dict=create_model_dict(),
|
||||||
|
)
|
||||||
|
rendered = converter("FILEPATH")
|
||||||
|
text, _, images = text_from_rendered(rendered)
|
||||||
|
```
|
||||||
|
|
||||||
|
`rendered` will be a pydantic basemodel with different properties depending on the output type requested. With markdown output (default), you'll have the properties `markdown`, `metadata`, and `images`. For json output, you'll have `children`, `block_type`, and `metadata`.
|
||||||
|
|
||||||
|
### Custom configuration
|
||||||
|
|
||||||
|
You can pass configuration using the `ConfigParser`. To see all available options, do `marker_single --help`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from marker.converters.pdf import PdfConverter
|
||||||
|
from marker.models import create_model_dict
|
||||||
|
from marker.config.parser import ConfigParser
|
||||||
|
|
||||||
|
config = {
|
||||||
|
"output_format": "json",
|
||||||
|
"ADDITIONAL_KEY": "VALUE"
|
||||||
|
}
|
||||||
|
config_parser = ConfigParser(config)
|
||||||
|
|
||||||
|
converter = PdfConverter(
|
||||||
|
config=config_parser.generate_config_dict(),
|
||||||
|
artifact_dict=create_model_dict(),
|
||||||
|
processor_list=config_parser.get_processors(),
|
||||||
|
renderer=config_parser.get_renderer(),
|
||||||
|
llm_service=config_parser.get_llm_service()
|
||||||
|
)
|
||||||
|
rendered = converter("FILEPATH")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Extract blocks
|
||||||
|
|
||||||
|
Each document consists of one or more pages. Pages contain blocks, which can themselves contain other blocks. It's possible to programmatically manipulate these blocks.
|
||||||
|
|
||||||
|
Here's an example of extracting all forms from a document:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from marker.converters.pdf import PdfConverter
|
||||||
|
from marker.models import create_model_dict
|
||||||
|
from marker.schema import BlockTypes
|
||||||
|
|
||||||
|
converter = PdfConverter(
|
||||||
|
artifact_dict=create_model_dict(),
|
||||||
|
)
|
||||||
|
document = converter.build_document("FILEPATH")
|
||||||
|
forms = document.contained_blocks((BlockTypes.Form,))
|
||||||
|
```
|
||||||
|
|
||||||
|
Look at the processors for more examples of extracting and manipulating blocks.
|
||||||
|
|
||||||
|
## Other converters
|
||||||
|
|
||||||
|
You can also use other converters that define different conversion pipelines:
|
||||||
|
|
||||||
|
### Extract tables
|
||||||
|
|
||||||
|
The `TableConverter` will only convert and extract tables:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from marker.converters.table import TableConverter
|
||||||
|
from marker.models import create_model_dict
|
||||||
|
from marker.output import text_from_rendered
|
||||||
|
|
||||||
|
converter = TableConverter(
|
||||||
|
artifact_dict=create_model_dict(),
|
||||||
|
)
|
||||||
|
rendered = converter("FILEPATH")
|
||||||
|
text, _, images = text_from_rendered(rendered)
|
||||||
|
```
|
||||||
|
|
||||||
|
This takes all the same configuration as the PdfConverter. You can specify the configuration `force_layout_block=Table` to avoid layout detection and instead assume every page is a table. Set `output_format=json` to also get cell bounding boxes.
|
||||||
|
|
||||||
|
You can also run this via the CLI with
|
||||||
|
```shell
|
||||||
|
marker_single FILENAME --use_llm --force_layout_block Table --converter_cls marker.converters.table.TableConverter --output_format json
|
||||||
|
```
|
||||||
|
|
||||||
|
### OCR Only
|
||||||
|
|
||||||
|
If you only want to run OCR, you can also do that through the `OCRConverter`. Set `--keep_chars` to keep individual characters and bounding boxes.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from marker.converters.ocr import OCRConverter
|
||||||
|
from marker.models import create_model_dict
|
||||||
|
|
||||||
|
converter = OCRConverter(
|
||||||
|
artifact_dict=create_model_dict(),
|
||||||
|
)
|
||||||
|
rendered = converter("FILEPATH")
|
||||||
|
```
|
||||||
|
|
||||||
|
This takes all the same configuration as the PdfConverter.
|
||||||
|
|
||||||
|
You can also run this via the CLI with
|
||||||
|
```shell
|
||||||
|
marker_single FILENAME --converter_cls marker.converters.ocr.OCRConverter
|
||||||
|
```
|
||||||
|
|
||||||
|
### Structured Extraction (beta)
|
||||||
|
|
||||||
|
You can run structured extraction via the `ExtractionConverter`. This requires an llm service to be setup first (see [here](#llm-services) for details). You'll get a JSON output with the extracted values.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from marker.converters.extraction import ExtractionConverter
|
||||||
|
from marker.models import create_model_dict
|
||||||
|
from marker.config.parser import ConfigParser
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
class Links(BaseModel):
|
||||||
|
links: list[str]
|
||||||
|
|
||||||
|
schema = Links.model_json_schema()
|
||||||
|
config_parser = ConfigParser({
|
||||||
|
"page_schema": schema
|
||||||
|
})
|
||||||
|
|
||||||
|
converter = ExtractionConverter(
|
||||||
|
artifact_dict=create_model_dict(),
|
||||||
|
config=config_parser.generate_config_dict(),
|
||||||
|
llm_service=config_parser.get_llm_service(),
|
||||||
|
)
|
||||||
|
rendered = converter("FILEPATH")
|
||||||
|
```
|
||||||
|
|
||||||
|
Rendered will have an `original_markdown` field. If you pass this back in next time you run the converter, as the `existing_markdown` config key, you can skip re-parsing the document.
|
||||||
|
|
||||||
|
# Output Formats
|
||||||
|
|
||||||
|
## Markdown
|
||||||
|
|
||||||
|
Markdown output will include:
|
||||||
|
|
||||||
|
- image links (images will be saved in the same folder)
|
||||||
|
- formatted tables
|
||||||
|
- embedded LaTeX equations (fenced with `$$`)
|
||||||
|
- Code is fenced with triple backticks
|
||||||
|
- Superscripts for footnotes
|
||||||
|
|
||||||
|
## HTML
|
||||||
|
|
||||||
|
HTML output is similar to markdown output:
|
||||||
|
|
||||||
|
- Images are included via `img` tags
|
||||||
|
- equations are fenced with `<math>` tags
|
||||||
|
- code is in `pre` tags
|
||||||
|
|
||||||
|
## JSON
|
||||||
|
|
||||||
|
JSON output will be organized in a tree-like structure, with the leaf nodes being blocks. Examples of leaf nodes are a single list item, a paragraph of text, or an image.
|
||||||
|
|
||||||
|
The output will be a list, with each list item representing a page. Each page is considered a block in the internal marker schema. There are different types of blocks to represent different elements.
|
||||||
|
|
||||||
|
Pages have the keys:
|
||||||
|
|
||||||
|
- `id` - unique id for the block.
|
||||||
|
- `block_type` - the type of block. The possible block types can be seen in `marker/schema/__init__.py`. As of this writing, they are ["Line", "Span", "FigureGroup", "TableGroup", "ListGroup", "PictureGroup", "Page", "Caption", "Code", "Figure", "Footnote", "Form", "Equation", "Handwriting", "TextInlineMath", "ListItem", "PageFooter", "PageHeader", "Picture", "SectionHeader", "Table", "Text", "TableOfContents", "Document"]
|
||||||
|
- `html` - the HTML for the page. Note that this will have recursive references to children. The `content-ref` tags must be replaced with the child content if you want the full html. You can see an example of this at `marker/output.py:json_to_html`. That function will take in a single block from the json output, and turn it into HTML.
|
||||||
|
- `polygon` - the 4-corner polygon of the page, in (x1,y1), (x2,y2), (x3, y3), (x4, y4) format. (x1,y1) is the top left, and coordinates go clockwise.
|
||||||
|
- `children` - the child blocks.
|
||||||
|
|
||||||
|
The child blocks have two additional keys:
|
||||||
|
|
||||||
|
- `section_hierarchy` - indicates the sections that the block is part of. `1` indicates an h1 tag, `2` an h2, and so on.
|
||||||
|
- `images` - base64 encoded images. The key will be the block id, and the data will be the encoded image.
|
||||||
|
|
||||||
|
Note that child blocks of pages can have their own children as well (a tree structure).
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "/page/10/Page/366",
|
||||||
|
"block_type": "Page",
|
||||||
|
"html": "<content-ref src='/page/10/SectionHeader/0'></content-ref><content-ref src='/page/10/SectionHeader/1'></content-ref><content-ref src='/page/10/Text/2'></content-ref><content-ref src='/page/10/Text/3'></content-ref><content-ref src='/page/10/Figure/4'></content-ref><content-ref src='/page/10/SectionHeader/5'></content-ref><content-ref src='/page/10/SectionHeader/6'></content-ref><content-ref src='/page/10/TextInlineMath/7'></content-ref><content-ref src='/page/10/TextInlineMath/8'></content-ref><content-ref src='/page/10/Table/9'></content-ref><content-ref src='/page/10/SectionHeader/10'></content-ref><content-ref src='/page/10/Text/11'></content-ref>",
|
||||||
|
"polygon": [[0.0, 0.0], [612.0, 0.0], [612.0, 792.0], [0.0, 792.0]],
|
||||||
|
"children": [
|
||||||
|
{
|
||||||
|
"id": "/page/10/SectionHeader/0",
|
||||||
|
"block_type": "SectionHeader",
|
||||||
|
"html": "<h1>Supplementary Material for <i>Subspace Adversarial Training</i> </h1>",
|
||||||
|
"polygon": [
|
||||||
|
[217.845703125, 80.630859375], [374.73046875, 80.630859375],
|
||||||
|
[374.73046875, 107.0],
|
||||||
|
[217.845703125, 107.0]
|
||||||
|
],
|
||||||
|
"children": null,
|
||||||
|
"section_hierarchy": {
|
||||||
|
"1": "/page/10/SectionHeader/1"
|
||||||
|
},
|
||||||
|
"images": {}
|
||||||
|
},
|
||||||
|
...
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
## Chunks
|
||||||
|
|
||||||
|
Chunks format is similar to JSON, but flattens everything into a single list instead of a tree. Only the top level blocks from each page show up. It also has the full HTML of each block inside, so you don't need to crawl the tree to reconstruct it. This enable flexible and easy chunking for RAG.
|
||||||
|
|
||||||
|
## Metadata
|
||||||
|
|
||||||
|
All output formats will return a metadata dictionary, with the following fields:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"table_of_contents": [
|
||||||
|
{
|
||||||
|
"title": "Introduction",
|
||||||
|
"heading_level": 1,
|
||||||
|
"page_id": 0,
|
||||||
|
"polygon": [...]
|
||||||
|
}
|
||||||
|
], // computed PDF table of contents
|
||||||
|
"page_stats": [
|
||||||
|
{
|
||||||
|
"page_id": 0,
|
||||||
|
"text_extraction_method": "pdftext",
|
||||||
|
"block_counts": [("Span", 200), ...]
|
||||||
|
},
|
||||||
|
...
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
# LLM Services
|
||||||
|
|
||||||
|
When running with the `--use_llm` flag, you have a choice of services you can use:
|
||||||
|
|
||||||
|
- `Gemini` - this will use the Gemini developer API by default. You'll need to pass `--gemini_api_key` to configuration.
|
||||||
|
- `Google Vertex` - this will use vertex, which can be more reliable. You'll need to pass `--vertex_project_id`. To use it, set `--llm_service=marker.services.vertex.GoogleVertexService`.
|
||||||
|
- `Ollama` - this will use local models. You can configure `--ollama_base_url` and `--ollama_model`. To use it, set `--llm_service=marker.services.ollama.OllamaService`.
|
||||||
|
- `Claude` - this will use the anthropic API. You can configure `--claude_api_key`, and `--claude_model_name`. To use it, set `--llm_service=marker.services.claude.ClaudeService`.
|
||||||
|
- `OpenAI` - this supports any openai-like endpoint. You can configure `--openai_api_key`, `--openai_model`, and `--openai_base_url`. To use it, set `--llm_service=marker.services.openai.OpenAIService`.
|
||||||
|
- `Azure OpenAI` - this uses the Azure OpenAI service. You can configure `--azure_endpoint`, `--azure_api_key`, and `--deployment_name`. To use it, set `--llm_service=marker.services.azure_openai.AzureOpenAIService`.
|
||||||
|
|
||||||
|
These services may have additional optional configuration as well - you can see it by viewing the classes.
|
||||||
|
|
||||||
|
# Internals
|
||||||
|
|
||||||
|
Marker is easy to extend. The core units of marker are:
|
||||||
|
|
||||||
|
- `Providers`, at `marker/providers`. These provide information from a source file, like a PDF.
|
||||||
|
- `Builders`, at `marker/builders`. These generate the initial document blocks and fill in text, using info from the providers.
|
||||||
|
- `Processors`, at `marker/processors`. These process specific blocks, for example the table formatter is a processor.
|
||||||
|
- `Renderers`, at `marker/renderers`. These use the blocks to render output.
|
||||||
|
- `Schema`, at `marker/schema`. The classes for all the block types.
|
||||||
|
- `Converters`, at `marker/converters`. They run the whole end to end pipeline.
|
||||||
|
|
||||||
|
To customize processing behavior, override the `processors`. To add new output formats, write a new `renderer`. For additional input formats, write a new `provider.`
|
||||||
|
|
||||||
|
Processors and renderers can be directly passed into the base `PDFConverter`, so you can specify your own custom processing easily.
|
||||||
|
|
||||||
|
## API server
|
||||||
|
|
||||||
|
There is a very simple API server you can run like this:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
pip install -U uvicorn fastapi python-multipart
|
||||||
|
marker_server --port 8001
|
||||||
|
```
|
||||||
|
|
||||||
|
This will start a fastapi server that you can access at `localhost:8001`. You can go to `localhost:8001/docs` to see the endpoint options.
|
||||||
|
|
||||||
|
You can send requests like this:
|
||||||
|
|
||||||
|
```
|
||||||
|
import requests
|
||||||
|
import json
|
||||||
|
|
||||||
|
post_data = {
|
||||||
|
'filepath': 'FILEPATH',
|
||||||
|
# Add other params here
|
||||||
|
}
|
||||||
|
|
||||||
|
requests.post("http://localhost:8001/marker", data=json.dumps(post_data)).json()
|
||||||
|
```
|
||||||
|
|
||||||
|
Note that this is not a very robust API, and is only intended for small-scale use. If you want to use this server, but want a more robust conversion option, you can use the hosted [Datalab API](https://www.datalab.to/plans).
|
||||||
|
|
||||||
|
# Troubleshooting
|
||||||
|
|
||||||
|
There are some settings that you may find useful if things aren't working the way you expect:
|
||||||
|
|
||||||
|
- If you have issues with accuracy, try setting `--use_llm` to use an LLM to improve quality. You must set `GOOGLE_API_KEY` to a Gemini API key for this to work.
|
||||||
|
- Make sure to set `force_ocr` if you see garbled text - this will re-OCR the document.
|
||||||
|
- `TORCH_DEVICE` - set this to force marker to use a given torch device for inference.
|
||||||
|
- If you're getting out of memory errors, decrease worker count. You can also try splitting up long PDFs into multiple files.
|
||||||
|
|
||||||
|
## Debugging
|
||||||
|
|
||||||
|
Pass the `debug` option to activate debug mode. This will save images of each page with detected layout and text, as well as output a json file with additional bounding box information.
|
||||||
|
|
||||||
|
# Benchmarks
|
||||||
|
|
||||||
|
## Overall PDF Conversion
|
||||||
|
|
||||||
|
We created a [benchmark set](https://huggingface.co/datasets/datalab-to/marker_benchmark) by extracting single PDF pages from common crawl. We scored based on a heuristic that aligns text with ground truth text segments, and an LLM as a judge scoring method.
|
||||||
|
|
||||||
|
| Method | Avg Time | Heuristic Score | LLM Score |
|
||||||
|
|------------|----------|-----------------|-----------|
|
||||||
|
| marker | 2.83837 | 95.6709 | 4.23916 |
|
||||||
|
| llamaparse | 23.348 | 84.2442 | 3.97619 |
|
||||||
|
| mathpix | 6.36223 | 86.4281 | 4.15626 |
|
||||||
|
| docling | 3.69949 | 86.7073 | 3.70429 |
|
||||||
|
|
||||||
|
Benchmarks were run on an H100 for markjer and docling - llamaparse and mathpix used their cloud services. We can also look at it by document type:
|
||||||
|
|
||||||
|
<img src="data/images/per_doc.png" width="1000px"/>
|
||||||
|
|
||||||
|
| Document Type | Marker heuristic | Marker LLM | Llamaparse Heuristic | Llamaparse LLM | Mathpix Heuristic | Mathpix LLM | Docling Heuristic | Docling LLM |
|
||||||
|
|----------------------|------------------|------------|----------------------|----------------|-------------------|-------------|-------------------|-------------|
|
||||||
|
| Scientific paper | 96.6737 | 4.34899 | 87.1651 | 3.96421 | 91.2267 | 4.46861 | 92.135 | 3.72422 |
|
||||||
|
| Book page | 97.1846 | 4.16168 | 90.9532 | 4.07186 | 93.8886 | 4.35329 | 90.0556 | 3.64671 |
|
||||||
|
| Other | 95.1632 | 4.25076 | 81.1385 | 4.01835 | 79.6231 | 4.00306 | 83.8223 | 3.76147 |
|
||||||
|
| Form | 88.0147 | 3.84663 | 66.3081 | 3.68712 | 64.7512 | 3.33129 | 68.3857 | 3.40491 |
|
||||||
|
| Presentation | 95.1562 | 4.13669 | 81.2261 | 4 | 83.6737 | 3.95683 | 84.8405 | 3.86331 |
|
||||||
|
| Financial document | 95.3697 | 4.39106 | 82.5812 | 4.16111 | 81.3115 | 4.05556 | 86.3882 | 3.8 |
|
||||||
|
| Letter | 98.4021 | 4.5 | 93.4477 | 4.28125 | 96.0383 | 4.45312 | 92.0952 | 4.09375 |
|
||||||
|
| Engineering document | 93.9244 | 4.04412 | 77.4854 | 3.72059 | 80.3319 | 3.88235 | 79.6807 | 3.42647 |
|
||||||
|
| Legal document | 96.689 | 4.27759 | 86.9769 | 3.87584 | 91.601 | 4.20805 | 87.8383 | 3.65552 |
|
||||||
|
| Newspaper page | 98.8733 | 4.25806 | 84.7492 | 3.90323 | 96.9963 | 4.45161 | 92.6496 | 3.51613 |
|
||||||
|
| Magazine page | 98.2145 | 4.38776 | 87.2902 | 3.97959 | 93.5934 | 4.16327 | 93.0892 | 4.02041 |
|
||||||
|
|
||||||
|
## Throughput
|
||||||
|
|
||||||
|
We benchmarked throughput using a [single long PDF](https://www.greenteapress.com/thinkpython/thinkpython.pdf).
|
||||||
|
|
||||||
|
| Method | Time per page | Time per document | VRAM used |
|
||||||
|
|---------|---------------|-------------------|---------- |
|
||||||
|
| marker | 0.18 | 43.42 | 3.17GB |
|
||||||
|
|
||||||
|
The projected throughput is 122 pages per second on an H100 - we can run 22 individual processes given the VRAM used.
|
||||||
|
|
||||||
|
## Table Conversion
|
||||||
|
|
||||||
|
Marker can extract tables from PDFs using `marker.converters.table.TableConverter`. The table extraction performance is measured by comparing the extracted HTML representation of tables against the original HTML representations using the test split of [FinTabNet](https://developer.ibm.com/exchanges/data/all/fintabnet/). The HTML representations are compared using a tree edit distance based metric to judge both structure and content. Marker detects and identifies the structure of all tables in a PDF page and achieves these scores:
|
||||||
|
|
||||||
|
| Method | Avg score | Total tables |
|
||||||
|
|------------------|-----------|--------------|
|
||||||
|
| marker | 0.816 | 99 |
|
||||||
|
| marker w/use_llm | 0.907 | 99 |
|
||||||
|
| gemini | 0.829 | 99 |
|
||||||
|
|
||||||
|
The `--use_llm` flag can significantly improve table recognition performance, as you can see.
|
||||||
|
|
||||||
|
We filter out tables that we cannot align with the ground truth, since fintabnet and our layout model have slightly different detection methods (this results in some tables being split/merged).
|
||||||
|
|
||||||
|
## Running your own benchmarks
|
||||||
|
|
||||||
|
You can benchmark the performance of marker on your machine. Install marker manually with:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
git clone https://github.com/VikParuchuri/marker.git
|
||||||
|
poetry install
|
||||||
|
```
|
||||||
|
|
||||||
|
### Overall PDF Conversion
|
||||||
|
|
||||||
|
Download the benchmark data [here](https://drive.google.com/file/d/1ZSeWDo2g1y0BRLT7KnbmytV2bjWARWba/view?usp=sharing) and unzip. Then run the overall benchmark like this:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python benchmarks/overall.py --methods marker --scores heuristic,llm
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
|
||||||
|
- `--use_llm` use an llm to improve the marker results.
|
||||||
|
- `--max_rows` how many rows to process for the benchmark.
|
||||||
|
- `--methods` can be `llamaparse`, `mathpix`, `docling`, `marker`. Comma separated.
|
||||||
|
- `--scores` which scoring functions to use, can be `llm`, `heuristic`. Comma separated.
|
||||||
|
|
||||||
|
### Table Conversion
|
||||||
|
The processed FinTabNet dataset is hosted [here](https://huggingface.co/datasets/datalab-to/fintabnet-test) and is automatically downloaded. Run the benchmark with:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python benchmarks/table/table.py --max_rows 100
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
|
||||||
|
- `--use_llm` uses an llm with marker to improve accuracy.
|
||||||
|
- `--use_gemini` also benchmarks gemini 2.0 flash.
|
||||||
|
|
||||||
|
# How it works
|
||||||
|
|
||||||
|
Marker is a pipeline of deep learning models:
|
||||||
|
|
||||||
|
- Extract text, OCR if necessary (heuristics, [surya](https://github.com/VikParuchuri/surya))
|
||||||
|
- Detect page layout and find reading order ([surya](https://github.com/VikParuchuri/surya))
|
||||||
|
- Clean and format each block (heuristics, [texify](https://github.com/VikParuchuri/texify), [surya](https://github.com/VikParuchuri/surya))
|
||||||
|
- Optionally use an LLM to improve quality
|
||||||
|
- Combine blocks and postprocess complete text
|
||||||
|
|
||||||
|
It only uses models where necessary, which improves speed and accuracy.
|
||||||
|
|
||||||
|
# Limitations
|
||||||
|
|
||||||
|
PDF is a tricky format, so marker will not always work perfectly. Here are some known limitations that are on the roadmap to address:
|
||||||
|
|
||||||
|
- Very complex layouts, with nested tables and forms, may not work
|
||||||
|
- Forms may not be rendered well
|
||||||
|
|
||||||
|
Note: Passing the `--use_llm` and `--force_ocr` flags will mostly solve these issues.
|
||||||
|
|
||||||
|
# Usage and Deployment Examples
|
||||||
|
|
||||||
|
You can always run `marker` locally, but if you wanted to expose it as an API, we have a few options:
|
||||||
|
- Our platform API which is powered by `marker` and `surya` and is easy to test out - it's free to sign up, and we'll include credits, [try it out here](https://datalab.to)
|
||||||
|
- Our painless on-prem solution for commercial use, which you can [read about here](https://www.datalab.to/blog/self-serve-on-prem-licensing) and gives you privacy guarantees with high throughput inference optimizations.
|
||||||
|
- [Deployment example with Modal](./examples/README_MODAL.md) that shows you how to deploy and access `marker` through a web endpoint using [`Modal`](https://modal.com). Modal is an AI compute platform that enables developers to deploy and scale models on GPUs in minutes.
|
||||||
@@ -9,14 +9,20 @@ AI-generated cross-book summaries, and engage in grounded RAG chat.
|
|||||||
```mermaid
|
```mermaid
|
||||||
graph TD
|
graph TD
|
||||||
User["Neurosurgeon (Browser)"]
|
User["Neurosurgeon (Browser)"]
|
||||||
|
Login["Login Page\n(username + password form)"]
|
||||||
FE["Frontend\nVue.js 3 / Vite\n:5173"]
|
FE["Frontend\nVue.js 3 / Vite\n:5173"]
|
||||||
BE["Backend\nSpring Boot 4 / Spring AI\n:8080"]
|
BE["Backend\nSpring Boot 4 / Spring AI\n:8080"]
|
||||||
|
Auth["Spring Security\nHTTP Basic Auth"]
|
||||||
DB["PostgreSQL + pgvector\n(source of truth)"]
|
DB["PostgreSQL + pgvector\n(source of truth)"]
|
||||||
FS["File Store\nuploads/ (local disk)\nExtracted figure PNGs"]
|
FS["File Store\nuploads/ (local disk)\nExtracted figure PNGs"]
|
||||||
LLM["LLM Provider\n(OpenAI)\nEmbeddings + Chat + Vision"]
|
LLM["LLM Provider\n(OpenAI)\nEmbeddings + Chat + Vision"]
|
||||||
|
|
||||||
User -->|HTTP| FE
|
User -->|"First visit / unauthenticated"| Login
|
||||||
FE -->|REST /api/v1/...| BE
|
Login -->|"POST credentials\n(GET /api/v1/auth/check)"| Auth
|
||||||
|
Auth -->|"401 → back to login\n200 → app access"| Login
|
||||||
|
Login -->|"Authenticated"| FE
|
||||||
|
FE -->|"REST /api/v1/...\n(HTTP Basic on every request)"| Auth
|
||||||
|
Auth --> BE
|
||||||
BE -->|"JDBC — books, chapters,\nsections, figures, refs"| DB
|
BE -->|"JDBC — books, chapters,\nsections, figures, refs"| DB
|
||||||
BE -->|"pgvector — text chunks\n+ figure caption vectors"| DB
|
BE -->|"pgvector — text chunks\n+ figure caption vectors"| DB
|
||||||
BE -->|"PNG read/write\n(figure extraction)"| FS
|
BE -->|"PNG read/write\n(figure extraction)"| FS
|
||||||
@@ -37,24 +43,101 @@ graph TD
|
|||||||
end
|
end
|
||||||
|
|
||||||
subgraph "Retrieval Pipeline (per chat query)"
|
subgraph "Retrieval Pipeline (per chat query)"
|
||||||
|
RP0["Query expansion\n(QueryExpansionService)\nlay → clinical terms"]
|
||||||
RP1["Text chunk search (topK=5)"]
|
RP1["Text chunk search (topK=5)"]
|
||||||
RP2["Figure caption search (topK=3)"]
|
RP2["Figure caption search (topK=3)"]
|
||||||
RP3["Expand chunks → full section text"]
|
RP3["Expand chunks → ±1-page section text"]
|
||||||
RP4["Fetch linked figures (chunk_figure_ref)"]
|
RP4["Fetch linked figures (chunk_figure_ref)"]
|
||||||
RP5["Merge + deduplicate figures"]
|
RP5["Merge + deduplicate figures"]
|
||||||
RP6["Build LLM prompt + call"]
|
RP6["Build labelled prompt\n[S1],[F1]… tags"]
|
||||||
|
RP7["LLM chat call"]
|
||||||
|
RP8["Citation validation\n(CitationValidatorService)\nstrip hallucinated refs"]
|
||||||
|
RP0 --> RP1
|
||||||
|
RP0 --> RP2
|
||||||
RP1 --> RP3
|
RP1 --> RP3
|
||||||
RP1 --> RP4
|
RP1 --> RP4
|
||||||
RP2 --> RP5
|
RP2 --> RP5
|
||||||
RP4 --> RP5
|
RP4 --> RP5
|
||||||
RP3 --> RP6
|
RP3 --> RP6
|
||||||
RP5 --> RP6
|
RP5 --> RP6
|
||||||
|
RP6 --> RP7
|
||||||
|
RP7 --> RP8
|
||||||
end
|
end
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Marker API Response Structure
|
||||||
|
|
||||||
|
The PDF parsing pipeline calls a local [Marker](https://github.com/VikParuchuri/marker) server (`POST /marker/upload`).
|
||||||
|
|
||||||
|
### Top-level envelope
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"format": "json",
|
||||||
|
"output": "<JSON-encoded string>"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`output` is a **JSON-encoded string** (not a nested object) and must be parsed a second time to get the document tree.
|
||||||
|
|
||||||
|
### Parsed `output` shape
|
||||||
|
|
||||||
|
```
|
||||||
|
{
|
||||||
|
"children": [ <Page block>, ... ]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Block types
|
||||||
|
|
||||||
|
Every block shares these fields:
|
||||||
|
|
||||||
|
| Field | Type | Notes |
|
||||||
|
|------------------|-------------------|--------------------------------------------|
|
||||||
|
| `id` | string | e.g. `/page/0/Picture/2` |
|
||||||
|
| `block_type` | string | see table below |
|
||||||
|
| `html` | string | rendered HTML; may contain `<content-ref>` |
|
||||||
|
| `bbox` | `[x0,y0,x1,y1]` | bounding box in page coordinates |
|
||||||
|
| `children` | array or null | nested blocks |
|
||||||
|
| `images` | object or null | base64 PNG map (leaf image blocks only) |
|
||||||
|
| `section_hierarchy` | object | heading ancestry |
|
||||||
|
|
||||||
|
#### Known `block_type` values
|
||||||
|
|
||||||
|
| block_type | Category | Notes |
|
||||||
|
|------------------|----------|-------------------------------------------------------|
|
||||||
|
| `Page` | structure | Top-level; direct children are the page content |
|
||||||
|
| `SectionHeader` | text | Section / chapter heading |
|
||||||
|
| `Text` | text | |
|
||||||
|
| `TextInlineMath` | text | |
|
||||||
|
| `ListItem` | text | |
|
||||||
|
| `Table` | text | |
|
||||||
|
| `Code` | text | |
|
||||||
|
| `Equation` | text | |
|
||||||
|
| `Footnote` | text | |
|
||||||
|
| `Caption` | text | Usually a child of a `*Group` block |
|
||||||
|
| `PageHeader` | text | |
|
||||||
|
| `PageFooter` | text | |
|
||||||
|
| `Handwriting` | text | |
|
||||||
|
| `Picture` | image | Leaf block; `images` map holds base64 PNG keyed by ID |
|
||||||
|
| `Figure` | image | Leaf block; same as `Picture` |
|
||||||
|
| `PictureGroup` | container | Wraps one `Picture` + one `Caption` child |
|
||||||
|
| `FigureGroup` | container | Wraps one `Figure` + one `Caption` child |
|
||||||
|
|
||||||
|
### Image extraction
|
||||||
|
|
||||||
|
Images are only present on **leaf** image blocks (`Picture`, `Figure`).
|
||||||
|
Group blocks (`PictureGroup`, `FigureGroup`) have `images: null` — the base64 PNG lives on the child leaf block.
|
||||||
|
|
||||||
|
```
|
||||||
|
PictureGroup
|
||||||
|
├── Picture ← images: { "/page/0/Picture/2": "<base64 PNG>" }
|
||||||
|
└── Caption ← html: "<p>Figure 1 — ...</p>"
|
||||||
|
```
|
||||||
|
|
||||||
## Stack
|
## Stack
|
||||||
|
|
||||||
- **Backend**: Spring Boot 4.0.5 + Spring AI 2.0.0-M4, Java 21, Maven
|
- **Backend**: Spring Boot 4.0.5 + Spring AI 2.0.0-M4, Java 25, Maven
|
||||||
- **Frontend**: Vue.js 3 + Vite + TypeScript + Pinia + Axios
|
- **Frontend**: Vue.js 3 + Vite + TypeScript + Pinia + Axios
|
||||||
- **Database**: PostgreSQL 16 + pgvector extension
|
- **Database**: PostgreSQL 16 + pgvector extension
|
||||||
- **Auth**: HTTP Basic (single shared in-memory user)
|
- **Auth**: HTTP Basic (single shared in-memory user)
|
||||||
@@ -63,7 +146,7 @@ graph TD
|
|||||||
|
|
||||||
See [specs/001-neuro-rag-learning/quickstart.md](specs/001-neuro-rag-learning/quickstart.md) for full instructions.
|
See [specs/001-neuro-rag-learning/quickstart.md](specs/001-neuro-rag-learning/quickstart.md) for full instructions.
|
||||||
|
|
||||||
### Local Dev
|
### Local Dev (JVM)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Start the database
|
# Start the database
|
||||||
@@ -79,8 +162,73 @@ npm install
|
|||||||
npm run dev
|
npm run dev
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Native Image Build
|
||||||
|
|
||||||
|
Produces a GraalVM native binary packaged into a minimal Docker image via Jib.
|
||||||
|
|
||||||
|
**Prerequisite**: GraalVM 25 must be installed and set as `JAVA_HOME`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install GraalVM 25 CE via sdkman (one-time)
|
||||||
|
sdk install java 25-graalce
|
||||||
|
sdk use java 25-graalce
|
||||||
|
|
||||||
|
# Build native executable + Docker image (requires Docker daemon)
|
||||||
|
cd backend
|
||||||
|
mvn -Pnative package jib:build -DskipTests
|
||||||
|
mvn -Pnative jib:build -Djib.to.auth.username=admin -Djib.to.auth.password=""
|
||||||
|
```
|
||||||
|
|
||||||
|
### Frontend build
|
||||||
|
```
|
||||||
|
buildah build \
|
||||||
|
--platform linux/arm64 \
|
||||||
|
--tag zot.immich-ad.ovh/ai-teacher-frontend:latest \
|
||||||
|
frontend/
|
||||||
|
buildah login zot.immich-ad.ovh
|
||||||
|
```
|
||||||
|
Push to the private repository:
|
||||||
|
|
||||||
|
```
|
||||||
|
buildah push --tls-verify=false zot.immich-ad.ovh/ai-teacher-frontend:latest
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
### Run Native Stack (Docker Compose)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Copy and fill in secrets
|
||||||
|
cp .env.example .env
|
||||||
|
# edit .env — add OPENAI_API_KEY at minimum
|
||||||
|
|
||||||
|
# Start PostgreSQL + native backend
|
||||||
|
docker compose -f docker-compose.native.yml up
|
||||||
|
```
|
||||||
|
|
||||||
|
App available at `http://localhost:8080`.
|
||||||
|
|
||||||
|
### Build Pipeline (Native)
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph LR
|
||||||
|
SRC["Source Code\n(Java 25)"]
|
||||||
|
AOT["Spring Boot AOT\n(process-aot)"]
|
||||||
|
NI["GraalVM native-image\n(native-maven-plugin)"]
|
||||||
|
EXE["Native Executable\ntarget/ai-teacher-backend"]
|
||||||
|
JIB["Jib\n(jib-native-image-extension)"]
|
||||||
|
IMG["Docker Image\nai-teacher-backend:latest\n(distroless base)"]
|
||||||
|
|
||||||
|
SRC --> AOT
|
||||||
|
AOT --> NI
|
||||||
|
NI --> EXE
|
||||||
|
EXE --> JIB
|
||||||
|
JIB --> IMG
|
||||||
|
```
|
||||||
|
|
||||||
### Environment Variables
|
### Environment Variables
|
||||||
|
|
||||||
|
#### Backend
|
||||||
|
|
||||||
| Variable | Required | Description |
|
| Variable | Required | Description |
|
||||||
|----------|----------|-------------|
|
|----------|----------|-------------|
|
||||||
| `OPENAI_API_KEY` | Yes | OpenAI API key for embeddings and chat |
|
| `OPENAI_API_KEY` | Yes | OpenAI API key for embeddings and chat |
|
||||||
@@ -89,3 +237,14 @@ npm run dev
|
|||||||
| `DB_USERNAME` | Yes | Database username |
|
| `DB_USERNAME` | Yes | Database username |
|
||||||
| `DB_PASSWORD` | Yes | Database password |
|
| `DB_PASSWORD` | Yes | Database password |
|
||||||
| `FIGURE_STORAGE_PATH` | No | Base path for uploaded PDFs and extracted figures (default: `./uploads`) |
|
| `FIGURE_STORAGE_PATH` | No | Base path for uploaded PDFs and extracted figures (default: `./uploads`) |
|
||||||
|
| `UPLOAD_ENABLED` | No | Set to `false` to disable the book upload endpoint (default: `true`) |
|
||||||
|
| `DELETE_ENABLED` | No | Set to `false` to disable the book delete endpoint (default: `true`) |
|
||||||
|
|
||||||
|
#### Frontend
|
||||||
|
|
||||||
|
| Variable | Required | Description |
|
||||||
|
|----------|----------|-------------|
|
||||||
|
| `VITE_API_URL` | No | Backend API base URL (default: `/api/v1`) |
|
||||||
|
| `VITE_APP_PASSWORD` | Yes | Shared password for HTTP Basic auth (must match `APP_PASSWORD`) |
|
||||||
|
| `VITE_UPLOAD_ENABLED` | No | Set to `false` to hide the upload UI (default: `true`) |
|
||||||
|
| `VITE_DELETE_ENABLED` | No | Set to `false` to hide the delete button (default: `true`) |
|
||||||
|
|||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# Java build artifacts
|
||||||
|
target/
|
||||||
|
*.class
|
||||||
|
*.jar
|
||||||
|
|
||||||
|
# Git
|
||||||
|
.git/
|
||||||
|
.gitignore
|
||||||
|
|
||||||
|
# Editor
|
||||||
|
.vscode/
|
||||||
|
.idea/
|
||||||
|
*.iml
|
||||||
|
|
||||||
|
# OS
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
# Logs
|
||||||
|
*.log
|
||||||
|
|
||||||
|
# Environment
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
# ---- Pull Maven from its official image (avoids microdnf under QEMU) ----
|
||||||
|
FROM docker.io/library/maven:3.9.9-eclipse-temurin-21 AS maven-dist
|
||||||
|
|
||||||
|
# ---- Build stage: GraalVM 25 + Maven ----
|
||||||
|
FROM ghcr.io/graalvm/native-image-community:25 AS build
|
||||||
|
|
||||||
|
# Copy Maven from the official Maven image — no package installation needed
|
||||||
|
COPY --from=maven-dist /usr/share/maven /opt/maven
|
||||||
|
ENV PATH="/opt/maven/bin:$PATH"
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Cache dependency resolution separately from source compilation
|
||||||
|
COPY pom.xml .
|
||||||
|
RUN mvn -Pnative dependency:resolve dependency:resolve-plugins -q
|
||||||
|
|
||||||
|
# Build native executable
|
||||||
|
COPY src ./src
|
||||||
|
RUN mvn -Pnative package -DskipTests
|
||||||
|
|
||||||
|
# ---- Runtime stage: slim Debian with glibc + libz (required by GraalVM native binary) ----
|
||||||
|
FROM docker.io/library/debian:12-slim
|
||||||
|
COPY --from=build /app/target/ai-teacher-backend /app/ai-teacher-backend
|
||||||
|
EXPOSE 8080
|
||||||
|
ENTRYPOINT ["/app/ai-teacher-backend"]
|
||||||
+119
-2
@@ -32,6 +32,13 @@
|
|||||||
<type>pom</type>
|
<type>pom</type>
|
||||||
<scope>import</scope>
|
<scope>import</scope>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>software.amazon.awssdk</groupId>
|
||||||
|
<artifactId>bom</artifactId>
|
||||||
|
<version>2.30.14</version>
|
||||||
|
<type>pom</type>
|
||||||
|
<scope>import</scope>
|
||||||
|
</dependency>
|
||||||
</dependencies>
|
</dependencies>
|
||||||
</dependencyManagement>
|
</dependencyManagement>
|
||||||
|
|
||||||
@@ -101,13 +108,19 @@
|
|||||||
<artifactId>spring-ai-pdf-document-reader</artifactId>
|
<artifactId>spring-ai-pdf-document-reader</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<!-- PDFBox — explicit for image extraction per page -->
|
<!-- PDFBox — page rendering and cropping for figure extraction -->
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.apache.pdfbox</groupId>
|
<groupId>org.apache.pdfbox</groupId>
|
||||||
<artifactId>pdfbox</artifactId>
|
<artifactId>pdfbox</artifactId>
|
||||||
<version>3.0.3</version>
|
<version>3.0.3</version>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
|
<!-- AWS SDK v2 — S3 figure storage -->
|
||||||
|
<dependency>
|
||||||
|
<groupId>software.amazon.awssdk</groupId>
|
||||||
|
<artifactId>s3</artifactId>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
<!-- Jackson (JSON) -->
|
<!-- Jackson (JSON) -->
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>com.fasterxml.jackson.core</groupId>
|
<groupId>com.fasterxml.jackson.core</groupId>
|
||||||
@@ -127,15 +140,119 @@
|
|||||||
</dependency>
|
</dependency>
|
||||||
</dependencies>
|
</dependencies>
|
||||||
|
|
||||||
|
|
||||||
<build>
|
<build>
|
||||||
<plugins>
|
<plugins>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.graalvm.buildtools</groupId>
|
||||||
|
<artifactId>native-maven-plugin</artifactId>
|
||||||
|
</plugin>
|
||||||
|
|
||||||
<plugin>
|
<plugin>
|
||||||
<groupId>org.springframework.boot</groupId>
|
<groupId>org.springframework.boot</groupId>
|
||||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||||
</plugin>
|
</plugin>
|
||||||
|
|
||||||
|
<!-- Jib — package native executable (or fat-jar) into Docker image -->
|
||||||
|
<plugin>
|
||||||
|
<groupId>com.google.cloud.tools</groupId>
|
||||||
|
<artifactId>jib-maven-plugin</artifactId>
|
||||||
|
<version>3.5.1</version>
|
||||||
|
<configuration>
|
||||||
|
<from>
|
||||||
|
<!-- distroless glibc base — includes libz + libssl needed by GraalVM native binary -->
|
||||||
|
<image>gcr.io/distroless/base-debian12</image>
|
||||||
|
</from>
|
||||||
|
<to>
|
||||||
|
<image>zot.immich-ad.ovh/ai-teacher-backend</image>
|
||||||
|
<tags>
|
||||||
|
<tag>latest</tag>
|
||||||
|
</tags>
|
||||||
|
</to>
|
||||||
|
<container>
|
||||||
|
<format>OCI</format>
|
||||||
|
<ports>
|
||||||
|
<port>8080</port>
|
||||||
|
</ports>
|
||||||
|
<!-- invoke the native binary directly — no JVM -->
|
||||||
|
<entrypoint>
|
||||||
|
<arg>/app/ai-teacher-backend</arg>
|
||||||
|
</entrypoint>
|
||||||
|
</container>
|
||||||
|
<!-- copy the GraalVM-compiled binary from target/ into /app/ -->
|
||||||
|
<extraDirectories>
|
||||||
|
<paths>
|
||||||
|
<path>
|
||||||
|
<from>${project.build.directory}</from>
|
||||||
|
<into>/app</into>
|
||||||
|
<includes>ai-teacher-backend</includes>
|
||||||
|
</path>
|
||||||
|
</paths>
|
||||||
|
<permissions>
|
||||||
|
<permission>
|
||||||
|
<file>/app/ai-teacher-backend</file>
|
||||||
|
<mode>755</mode>
|
||||||
|
</permission>
|
||||||
|
</permissions>
|
||||||
|
</extraDirectories>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
</plugins>
|
</plugins>
|
||||||
</build>
|
</build>
|
||||||
|
|
||||||
|
<profiles>
|
||||||
|
<profile>
|
||||||
|
<id>native</id>
|
||||||
|
<build>
|
||||||
|
<plugins>
|
||||||
|
|
||||||
|
<!-- skip jib in native builds — use Dockerfile.native + buildah instead -->
|
||||||
|
<plugin>
|
||||||
|
<groupId>com.google.cloud.tools</groupId>
|
||||||
|
<artifactId>jib-maven-plugin</artifactId>
|
||||||
|
<configuration>
|
||||||
|
<skip>true</skip>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
|
|
||||||
|
<!-- GraalVM native-image compilation -->
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.graalvm.buildtools</groupId>
|
||||||
|
<artifactId>native-maven-plugin</artifactId>
|
||||||
|
<version>1.0.0</version>
|
||||||
|
<executions>
|
||||||
|
<execution>
|
||||||
|
<id>add-reachability-metadata</id>
|
||||||
|
<goals>
|
||||||
|
<goal>add-reachability-metadata</goal>
|
||||||
|
</goals>
|
||||||
|
</execution>
|
||||||
|
<execution>
|
||||||
|
<id>compile</id>
|
||||||
|
<goals>
|
||||||
|
<goal>compile-no-fork</goal>
|
||||||
|
</goals>
|
||||||
|
<phase>package</phase>
|
||||||
|
</execution>
|
||||||
|
</executions>
|
||||||
|
<configuration>
|
||||||
|
<imageName>ai-teacher-backend</imageName>
|
||||||
|
<buildArgs>
|
||||||
|
<buildArg>--initialize-at-build-time=org.slf4j,ch.qos.logback</buildArg>
|
||||||
|
<buildArg>-H:+ReportExceptionStackTraces</buildArg>
|
||||||
|
<buildArg>--gc=serial</buildArg>
|
||||||
|
<buildArg>-Os</buildArg>
|
||||||
|
<buildArg>-H:+RemoveUnusedSymbols</buildArg>
|
||||||
|
<buildArg>-H:-EnableLoggingFeature</buildArg>
|
||||||
|
<buildArg>-R:MaxHeapSize=128m</buildArg>
|
||||||
|
<buildArg>-R:MinHeapSize=32m</buildArg>
|
||||||
|
<!-- Limit native-image compiler RAM (build time, not runtime) -->
|
||||||
|
<buildArg>-J-Xmx8g</buildArg>
|
||||||
|
</buildArgs>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
|
</plugins>
|
||||||
|
</build>
|
||||||
|
</profile>
|
||||||
|
</profiles>
|
||||||
|
|
||||||
</project>
|
</project>
|
||||||
|
|||||||
@@ -1,11 +1,15 @@
|
|||||||
package com.aiteacher;
|
package com.aiteacher;
|
||||||
|
|
||||||
|
import org.springframework.context.annotation.ImportRuntimeHints;
|
||||||
import org.springframework.boot.SpringApplication;
|
import org.springframework.boot.SpringApplication;
|
||||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||||
import org.springframework.scheduling.annotation.EnableAsync;
|
import org.springframework.scheduling.annotation.EnableAsync;
|
||||||
|
|
||||||
|
import com.aiteacher.config.NativeHintsConfig;
|
||||||
|
|
||||||
@SpringBootApplication
|
@SpringBootApplication
|
||||||
@EnableAsync
|
@EnableAsync
|
||||||
|
@ImportRuntimeHints(NativeHintsConfig.class)
|
||||||
public class AiTeacherApplication {
|
public class AiTeacherApplication {
|
||||||
|
|
||||||
public static void main(String[] args) {
|
public static void main(String[] args) {
|
||||||
|
|||||||
@@ -0,0 +1,19 @@
|
|||||||
|
package com.aiteacher.auth;
|
||||||
|
|
||||||
|
import org.springframework.http.ResponseEntity;
|
||||||
|
import org.springframework.web.bind.annotation.GetMapping;
|
||||||
|
import org.springframework.web.bind.annotation.RequestMapping;
|
||||||
|
import org.springframework.web.bind.annotation.RestController;
|
||||||
|
|
||||||
|
import java.security.Principal;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
@RestController
|
||||||
|
@RequestMapping("/api/v1/auth")
|
||||||
|
public class AuthController {
|
||||||
|
|
||||||
|
@GetMapping("/check")
|
||||||
|
public ResponseEntity<Map<String, String>> check(Principal principal) {
|
||||||
|
return ResponseEntity.ok(Map.of("username", principal.getName()));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -2,7 +2,10 @@ package com.aiteacher.book;
|
|||||||
|
|
||||||
import com.aiteacher.document.FigureEntity;
|
import com.aiteacher.document.FigureEntity;
|
||||||
import com.aiteacher.document.FigureRepository;
|
import com.aiteacher.document.FigureRepository;
|
||||||
|
import com.aiteacher.document.MarkdownStorageService;
|
||||||
|
import org.springframework.beans.factory.annotation.Value;
|
||||||
import org.springframework.http.HttpStatus;
|
import org.springframework.http.HttpStatus;
|
||||||
|
import org.springframework.http.MediaType;
|
||||||
import org.springframework.http.ResponseEntity;
|
import org.springframework.http.ResponseEntity;
|
||||||
import org.springframework.web.bind.annotation.*;
|
import org.springframework.web.bind.annotation.*;
|
||||||
import org.springframework.web.multipart.MultipartFile;
|
import org.springframework.web.multipart.MultipartFile;
|
||||||
@@ -18,14 +21,24 @@ public class BookController {
|
|||||||
|
|
||||||
private final BookService bookService;
|
private final BookService bookService;
|
||||||
private final FigureRepository figureRepository;
|
private final FigureRepository figureRepository;
|
||||||
|
private final MarkdownStorageService markdownStorageService;
|
||||||
|
|
||||||
public BookController(BookService bookService, FigureRepository figureRepository) {
|
@Value("${app.features.upload-enabled:true}")
|
||||||
|
private boolean uploadEnabled;
|
||||||
|
|
||||||
|
@Value("${app.features.delete-enabled:true}")
|
||||||
|
private boolean deleteEnabled;
|
||||||
|
|
||||||
|
public BookController(BookService bookService, FigureRepository figureRepository,
|
||||||
|
MarkdownStorageService markdownStorageService) {
|
||||||
this.bookService = bookService;
|
this.bookService = bookService;
|
||||||
this.figureRepository = figureRepository;
|
this.figureRepository = figureRepository;
|
||||||
|
this.markdownStorageService = markdownStorageService;
|
||||||
}
|
}
|
||||||
|
|
||||||
@PostMapping(consumes = "multipart/form-data")
|
@PostMapping(consumes = "multipart/form-data")
|
||||||
public ResponseEntity<?> upload(@RequestParam("file") MultipartFile file) throws IOException {
|
public ResponseEntity<?> upload(@RequestParam("file") MultipartFile file) throws IOException {
|
||||||
|
if (!uploadEnabled) return ResponseEntity.status(HttpStatus.METHOD_NOT_ALLOWED).build();
|
||||||
Book book = bookService.upload(file);
|
Book book = bookService.upload(file);
|
||||||
return ResponseEntity.status(HttpStatus.ACCEPTED).body(toSummaryResponse(book));
|
return ResponseEntity.status(HttpStatus.ACCEPTED).body(toSummaryResponse(book));
|
||||||
}
|
}
|
||||||
@@ -46,6 +59,7 @@ public class BookController {
|
|||||||
|
|
||||||
@DeleteMapping("/{id}")
|
@DeleteMapping("/{id}")
|
||||||
public ResponseEntity<Void> delete(@PathVariable UUID id) {
|
public ResponseEntity<Void> delete(@PathVariable UUID id) {
|
||||||
|
if (!deleteEnabled) return ResponseEntity.status(HttpStatus.METHOD_NOT_ALLOWED).build();
|
||||||
bookService.delete(id);
|
bookService.delete(id);
|
||||||
return ResponseEntity.noContent().build();
|
return ResponseEntity.noContent().build();
|
||||||
}
|
}
|
||||||
@@ -59,6 +73,17 @@ public class BookController {
|
|||||||
));
|
));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@GetMapping(value = "/{id}/pages/{pageNumber}/html", produces = MediaType.TEXT_HTML_VALUE)
|
||||||
|
public ResponseEntity<String> getPageHtml(@PathVariable UUID id,
|
||||||
|
@PathVariable int pageNumber) {
|
||||||
|
bookService.getById(id); // 404 if not found
|
||||||
|
try {
|
||||||
|
return ResponseEntity.ok(markdownStorageService.getText(id, pageNumber));
|
||||||
|
} catch (Exception e) {
|
||||||
|
return ResponseEntity.notFound().build();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
@GetMapping("/{id}/figures")
|
@GetMapping("/{id}/figures")
|
||||||
public ResponseEntity<List<FigureResponse>> figures(@PathVariable UUID id) {
|
public ResponseEntity<List<FigureResponse>> figures(@PathVariable UUID id) {
|
||||||
bookService.getById(id); // 404 if not found
|
bookService.getById(id); // 404 if not found
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package com.aiteacher.book;
|
|||||||
|
|
||||||
import com.aiteacher.document.*;
|
import com.aiteacher.document.*;
|
||||||
import com.aiteacher.figure.FigureStorageService;
|
import com.aiteacher.figure.FigureStorageService;
|
||||||
|
|
||||||
import org.slf4j.Logger;
|
import org.slf4j.Logger;
|
||||||
import org.slf4j.LoggerFactory;
|
import org.slf4j.LoggerFactory;
|
||||||
import org.springframework.ai.document.Document;
|
import org.springframework.ai.document.Document;
|
||||||
@@ -23,13 +24,7 @@ public class BookEmbeddingService {
|
|||||||
|
|
||||||
private final VectorStore vectorStore;
|
private final VectorStore vectorStore;
|
||||||
private final BookRepository bookRepository;
|
private final BookRepository bookRepository;
|
||||||
|
private final MarkerPageParser markerPageParser;
|
||||||
@Value("${app.embedding.batch-size:50}")
|
|
||||||
private int embeddingBatchSize;
|
|
||||||
|
|
||||||
@Value("${app.embedding.batch-delay-ms:1000}")
|
|
||||||
private long embeddingBatchDelayMs;
|
|
||||||
private final PdfStructureParser pdfStructureParser;
|
|
||||||
private final FigureExtractionService figureExtractionService;
|
private final FigureExtractionService figureExtractionService;
|
||||||
private final VisionDescriptionService visionDescriptionService;
|
private final VisionDescriptionService visionDescriptionService;
|
||||||
private final TextChunkingService textChunkingService;
|
private final TextChunkingService textChunkingService;
|
||||||
@@ -39,11 +34,21 @@ public class BookEmbeddingService {
|
|||||||
private final FigureRepository figureRepository;
|
private final FigureRepository figureRepository;
|
||||||
private final ChunkFigureRefRepository chunkFigureRefRepository;
|
private final ChunkFigureRefRepository chunkFigureRefRepository;
|
||||||
private final FigureStorageService figureStorageService;
|
private final FigureStorageService figureStorageService;
|
||||||
|
private final MarkdownStorageService markdownStorageService;
|
||||||
|
|
||||||
|
@Value("${app.embedding.batch-size:50}")
|
||||||
|
private int embeddingBatchSize;
|
||||||
|
|
||||||
|
@Value("${app.embedding.batch-delay-ms:1000}")
|
||||||
|
private long embeddingBatchDelayMs;
|
||||||
|
|
||||||
|
@Value("${app.embedding.skip-embedding:false}")
|
||||||
|
private boolean skipEmbedding;
|
||||||
|
|
||||||
public BookEmbeddingService(
|
public BookEmbeddingService(
|
||||||
VectorStore vectorStore,
|
VectorStore vectorStore,
|
||||||
BookRepository bookRepository,
|
BookRepository bookRepository,
|
||||||
PdfStructureParser pdfStructureParser,
|
MarkerPageParser markerPageParser,
|
||||||
FigureExtractionService figureExtractionService,
|
FigureExtractionService figureExtractionService,
|
||||||
VisionDescriptionService visionDescriptionService,
|
VisionDescriptionService visionDescriptionService,
|
||||||
TextChunkingService textChunkingService,
|
TextChunkingService textChunkingService,
|
||||||
@@ -52,10 +57,11 @@ public class BookEmbeddingService {
|
|||||||
ChapterRepository chapterRepository,
|
ChapterRepository chapterRepository,
|
||||||
FigureRepository figureRepository,
|
FigureRepository figureRepository,
|
||||||
ChunkFigureRefRepository chunkFigureRefRepository,
|
ChunkFigureRefRepository chunkFigureRefRepository,
|
||||||
FigureStorageService figureStorageService) {
|
FigureStorageService figureStorageService,
|
||||||
|
MarkdownStorageService markdownStorageService) {
|
||||||
this.vectorStore = vectorStore;
|
this.vectorStore = vectorStore;
|
||||||
this.bookRepository = bookRepository;
|
this.bookRepository = bookRepository;
|
||||||
this.pdfStructureParser = pdfStructureParser;
|
this.markerPageParser = markerPageParser;
|
||||||
this.figureExtractionService = figureExtractionService;
|
this.figureExtractionService = figureExtractionService;
|
||||||
this.visionDescriptionService = visionDescriptionService;
|
this.visionDescriptionService = visionDescriptionService;
|
||||||
this.textChunkingService = textChunkingService;
|
this.textChunkingService = textChunkingService;
|
||||||
@@ -65,11 +71,12 @@ public class BookEmbeddingService {
|
|||||||
this.figureRepository = figureRepository;
|
this.figureRepository = figureRepository;
|
||||||
this.chunkFigureRefRepository = chunkFigureRefRepository;
|
this.chunkFigureRefRepository = chunkFigureRefRepository;
|
||||||
this.figureStorageService = figureStorageService;
|
this.figureStorageService = figureStorageService;
|
||||||
|
this.markdownStorageService = markdownStorageService;
|
||||||
}
|
}
|
||||||
|
|
||||||
@Async
|
@Async
|
||||||
public void embedBook(UUID bookId, String bookTitle, Path pdfPath) {
|
public void embedBook(UUID bookId, String bookTitle, Path pdfPath) {
|
||||||
log.info("Starting image-aware embedding for book {} ({})", bookId, bookTitle);
|
log.info("Starting Marker-powered embedding for book {} ({})", bookId, bookTitle);
|
||||||
|
|
||||||
Book book = bookRepository.findById(bookId).orElse(null);
|
Book book = bookRepository.findById(bookId).orElse(null);
|
||||||
if (book == null) {
|
if (book == null) {
|
||||||
@@ -81,68 +88,94 @@ public class BookEmbeddingService {
|
|||||||
book.setStatus(BookStatus.PROCESSING);
|
book.setStatus(BookStatus.PROCESSING);
|
||||||
bookRepository.save(book);
|
bookRepository.save(book);
|
||||||
|
|
||||||
// Step 1: Parse PDF into page-level sections persisted in Postgres
|
|
||||||
List<SectionEntity> sections = pdfStructureParser.parse(bookId, bookTitle, pdfPath);
|
|
||||||
String chapterId = bookId + "-ch1";
|
String chapterId = bookId + "-ch1";
|
||||||
|
ChapterEntity chapter = new ChapterEntity(chapterId, bookId, 1, bookTitle, 1);
|
||||||
|
chapterRepository.save(chapter);
|
||||||
|
|
||||||
// Step 2: Build and embed text chunks for all sections in batches
|
// Step 1: Parse with Marker — split into 100-page chunks, then merge results
|
||||||
|
ParsedBook parsed = markerPageParser.parse(pdfPath);
|
||||||
|
|
||||||
|
List<PageResult> pageResults = parsed.pages();
|
||||||
|
|
||||||
|
// Step 2: Build SectionEntity per page and persist
|
||||||
|
List<SectionEntity> sections = buildAndSaveSections(bookId, bookTitle, chapterId, pageResults);
|
||||||
|
|
||||||
|
// Step 3: Chunk and embed text
|
||||||
List<Document> allChunks = new ArrayList<>();
|
List<Document> allChunks = new ArrayList<>();
|
||||||
for (SectionEntity section : sections) {
|
for (SectionEntity section : sections) {
|
||||||
List<Document> chunks = textChunkingService.chunk(section, bookTitle);
|
allChunks.addAll(textChunkingService.chunk(section, bookTitle));
|
||||||
allChunks.addAll(chunks);
|
}
|
||||||
|
if (skipEmbedding) {
|
||||||
|
log.info("skip-embedding=true — skipping text embedding for book {}", bookId);
|
||||||
|
} else {
|
||||||
|
embedInBatches(allChunks, bookId);
|
||||||
|
log.info("Embedded {} text chunks for book {}", allChunks.size(), bookId);
|
||||||
}
|
}
|
||||||
embedInBatches(allChunks, bookId);
|
|
||||||
log.info("Embedded {} text chunks for book {}", allChunks.size(), bookId);
|
|
||||||
|
|
||||||
// Step 3: Extract images from the PDF, save to file store, persist FigureEntity
|
// Step 4: Decode pre-cropped figures from Marker output
|
||||||
List<FigureEntity> figures = figureExtractionService.extract(
|
FigureExtractionService.ExtractionResult extraction =
|
||||||
bookId, chapterId, sections, pdfPath);
|
figureExtractionService.extract(bookId, chapterId, pageResults);
|
||||||
|
List<FigureEntity> figures = extraction.figures();
|
||||||
|
|
||||||
|
// Step 4b: Save per-page HTML to S3, replacing Marker image src with API URLs
|
||||||
|
parsed.htmlByPage().forEach((pageNumber, html) -> {
|
||||||
|
String resolved = resolveImageSrcs(html, bookId, extraction.blockIdToFigureId());
|
||||||
|
markdownStorageService.save(bookId, pageNumber, resolved);
|
||||||
|
});
|
||||||
|
log.info("Saved {} HTML pages to S3 for book {}", parsed.htmlByPage().size(), bookId);
|
||||||
|
|
||||||
|
// Step 5: Vision analysis (description + visible text) → embed figure chunks
|
||||||
|
Map<String, SectionEntity> sectionById = new HashMap<>();
|
||||||
|
for (SectionEntity s : sections) sectionById.put(s.getId(), s);
|
||||||
|
|
||||||
// Step 4: For each figure, generate vision description and embed caption
|
|
||||||
for (FigureEntity figure : figures) {
|
for (FigureEntity figure : figures) {
|
||||||
Path imagePath = figureStorageService.resolve(figure.getImagePath());
|
// Prefer caption extracted from the linked section's full text
|
||||||
String description = visionDescriptionService.describe(
|
|
||||||
imagePath, figure.getCaption());
|
|
||||||
|
|
||||||
// Use description as caption fallback if no caption was detected
|
|
||||||
if (figure.getCaption() == null || figure.getCaption().isBlank()) {
|
if (figure.getCaption() == null || figure.getCaption().isBlank()) {
|
||||||
figure.setCaption(description);
|
String sectionCaption = extractCaptionFromSection(sectionById.get(figure.getSectionId()));
|
||||||
figureRepository.save(figure);
|
if (sectionCaption != null) {
|
||||||
|
figure.setCaption(sectionCaption);
|
||||||
|
figureRepository.save(figure);
|
||||||
|
} else {
|
||||||
|
byte[] imageBytes = figureStorageService.getBytes(figure.getImagePath());
|
||||||
|
VisionDescriptionService.ImageAnalysis analysis =
|
||||||
|
visionDescriptionService.analyze(imageBytes, figure.getCaption());
|
||||||
|
figure.setCaption(analysis.description());
|
||||||
|
figureRepository.save(figure);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Content for embedding = vision description + caption for maximum signal
|
// Embedding content: description
|
||||||
String embeddingContent = description
|
String embeddingContent = (figure.getCaption() != null ? "\n" + figure.getCaption() : "");
|
||||||
+ (figure.getCaption() != null ? "\n" + figure.getCaption() : "");
|
|
||||||
|
|
||||||
String embeddingId = UUID.randomUUID().toString();
|
String embeddingId = UUID.randomUUID().toString();
|
||||||
Map<String, Object> metadata = buildFigureMetadata(figure, bookTitle, embeddingId);
|
if (!skipEmbedding) {
|
||||||
Document figureDoc = new Document(embeddingId, embeddingContent, metadata);
|
Document figureDoc = new Document(embeddingId, embeddingContent,
|
||||||
vectorStore.add(List.of(figureDoc));
|
buildFigureMetadata(figure, bookTitle, embeddingId, ""));
|
||||||
|
vectorStore.add(List.of(figureDoc));
|
||||||
figure.setCaptionEmbeddingId(UUID.fromString(embeddingId));
|
figure.setCaptionEmbeddingId(UUID.fromString(embeddingId));
|
||||||
|
}
|
||||||
figureRepository.save(figure);
|
figureRepository.save(figure);
|
||||||
}
|
}
|
||||||
log.info("Embedded {} figure captions for book {}", figures.size(), bookId);
|
log.info("Embedded {} figure chunks for book {}", figures.size(), bookId);
|
||||||
|
|
||||||
// Step 5: Link text chunks to figures via text references
|
// Step 6: Link text chunks to figures via in-text references
|
||||||
for (SectionEntity section : sections) {
|
for (SectionEntity section : sections) {
|
||||||
List<Document> sectionChunks = allChunks.stream()
|
List<Document> sectionChunks = allChunks.stream()
|
||||||
.filter(d -> section.getId().equals(d.getMetadata().get("section_id")))
|
.filter(d -> section.getId().equals(d.getMetadata().get("section_id")))
|
||||||
.toList();
|
.toList();
|
||||||
List<FigureEntity> sectionFigures = figures.stream()
|
List<FigureEntity> sectionFigures = figures.stream()
|
||||||
.filter(f -> section.getId().equals(f.getSectionId()))
|
.filter(f -> section.getId().equals(f.getSectionId()))
|
||||||
.toList();
|
.toList();
|
||||||
chunkFigureRefService.linkChunksToFigures(
|
chunkFigureRefService.linkChunksToFigures(sectionChunks, sectionFigures, section.getPageStart());
|
||||||
sectionChunks, sectionFigures, section.getPageStart());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
book.setStatus(BookStatus.READY);
|
book.setStatus(BookStatus.READY);
|
||||||
book.setPageCount(sections.size());
|
book.setPageCount(parsed.htmlByPage().size());
|
||||||
book.setProcessedAt(Instant.now());
|
book.setProcessedAt(Instant.now());
|
||||||
bookRepository.save(book);
|
bookRepository.save(book);
|
||||||
|
|
||||||
log.info("Finished embedding book {} — {} pages, {} figures",
|
log.info("Finished embedding book {} — {} pages, {} figures",
|
||||||
bookId, sections.size(), figures.size());
|
bookId, sections.size(), figures.size());
|
||||||
|
|
||||||
} catch (Exception ex) {
|
} catch (Exception ex) {
|
||||||
log.error("Failed to embed book {}", bookId, ex);
|
log.error("Failed to embed book {}", bookId, ex);
|
||||||
@@ -156,53 +189,63 @@ public class BookEmbeddingService {
|
|||||||
public void deleteBookChunks(UUID bookId) {
|
public void deleteBookChunks(UUID bookId) {
|
||||||
log.info("Deleting all data for book {}", bookId);
|
log.info("Deleting all data for book {}", bookId);
|
||||||
try {
|
try {
|
||||||
// Delete chunk-figure refs (by figureId for this book)
|
|
||||||
List<String> figureIds = figureRepository.findAllByBookId(bookId)
|
List<String> figureIds = figureRepository.findAllByBookId(bookId)
|
||||||
.stream().map(FigureEntity::getId).toList();
|
.stream().map(FigureEntity::getId).toList();
|
||||||
if (!figureIds.isEmpty()) {
|
if (!figureIds.isEmpty()) {
|
||||||
chunkFigureRefRepository.deleteByFigureIdIn(figureIds);
|
chunkFigureRefRepository.deleteByFigureIdIn(figureIds);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Delete figures from Postgres
|
|
||||||
figureRepository.deleteAllByBookId(bookId);
|
figureRepository.deleteAllByBookId(bookId);
|
||||||
|
|
||||||
// Delete figure files from disk
|
|
||||||
figureStorageService.deleteAll(bookId);
|
figureStorageService.deleteAll(bookId);
|
||||||
|
markdownStorageService.deleteAll(bookId);
|
||||||
// Delete sections and chapters from Postgres
|
|
||||||
sectionRepository.deleteAllByBookId(bookId);
|
sectionRepository.deleteAllByBookId(bookId);
|
||||||
chapterRepository.deleteAllByBookId(bookId);
|
chapterRepository.deleteAllByBookId(bookId);
|
||||||
|
|
||||||
// Delete vector store entries (text chunks + figure embeddings)
|
|
||||||
FilterExpressionBuilder b = new FilterExpressionBuilder();
|
FilterExpressionBuilder b = new FilterExpressionBuilder();
|
||||||
vectorStore.delete(b.eq("book_id", bookId.toString()).build());
|
vectorStore.delete(b.eq("book_id", bookId.toString()).build());
|
||||||
|
|
||||||
} catch (Exception ex) {
|
} catch (Exception ex) {
|
||||||
log.warn("Error during cleanup for book {}: {}", bookId, ex.getMessage());
|
log.warn("Error during cleanup for book {}: {}", bookId, ex.getMessage());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- Private helpers ---
|
||||||
|
|
||||||
|
private List<SectionEntity> buildAndSaveSections(UUID bookId, String bookTitle,
|
||||||
|
String chapterId,
|
||||||
|
List<PageResult> pageResults) {
|
||||||
|
List<SectionEntity> sections = new ArrayList<>();
|
||||||
|
for (PageResult page : pageResults) {
|
||||||
|
if (page.orderedText().isBlank()) continue;
|
||||||
|
|
||||||
|
String sectionId = bookId + "-p" + page.pageNumber();
|
||||||
|
String title = truncate(page.headingTitle() != null ? page.headingTitle() : "Page " + page.pageNumber(), 500);
|
||||||
|
|
||||||
|
SectionEntity section = new SectionEntity(
|
||||||
|
sectionId, chapterId, bookId,
|
||||||
|
String.valueOf(page.pageNumber()),
|
||||||
|
title,
|
||||||
|
page.pageNumber(), page.pageNumber(),
|
||||||
|
page.orderedText());
|
||||||
|
sections.add(sectionRepository.save(section));
|
||||||
|
}
|
||||||
|
return sections;
|
||||||
|
}
|
||||||
|
|
||||||
private void embedInBatches(List<Document> docs, UUID bookId) {
|
private void embedInBatches(List<Document> docs, UUID bookId) {
|
||||||
int total = docs.size();
|
int total = docs.size();
|
||||||
for (int i = 0; i < total; i += embeddingBatchSize) {
|
for (int i = 0; i < total; i += embeddingBatchSize) {
|
||||||
List<Document> batch = docs.subList(i, Math.min(i + embeddingBatchSize, total));
|
List<Document> batch = docs.subList(i, Math.min(i + embeddingBatchSize, total));
|
||||||
vectorStore.add(batch);
|
vectorStore.add(batch);
|
||||||
int batchNum = i / embeddingBatchSize + 1;
|
log.debug("Embedded batch {}/{} for book {}",
|
||||||
int totalBatches = (total - 1) / embeddingBatchSize + 1;
|
i / embeddingBatchSize + 1, (total - 1) / embeddingBatchSize + 1, bookId);
|
||||||
log.debug("Embedded batch {}/{} for book {}", batchNum, totalBatches, bookId);
|
|
||||||
if (i + embeddingBatchSize < total) {
|
if (i + embeddingBatchSize < total) {
|
||||||
try {
|
try { Thread.sleep(embeddingBatchDelayMs); }
|
||||||
Thread.sleep(embeddingBatchDelayMs);
|
catch (InterruptedException e) { Thread.currentThread().interrupt(); }
|
||||||
} catch (InterruptedException e) {
|
|
||||||
Thread.currentThread().interrupt();
|
|
||||||
log.warn("Embedding batch sleep interrupted for book {}", bookId);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private Map<String, Object> buildFigureMetadata(FigureEntity figure, String bookTitle,
|
private Map<String, Object> buildFigureMetadata(FigureEntity figure, String bookTitle,
|
||||||
String embeddingId) {
|
String embeddingId, String imageText) {
|
||||||
Map<String, Object> m = new HashMap<>();
|
Map<String, Object> m = new HashMap<>();
|
||||||
m.put("type", "FIGURE");
|
m.put("type", "FIGURE");
|
||||||
m.put("book_id", figure.getBookId().toString());
|
m.put("book_id", figure.getBookId().toString());
|
||||||
@@ -215,9 +258,37 @@ public class BookEmbeddingService {
|
|||||||
m.put("label", figure.getLabel() != null ? figure.getLabel() : "");
|
m.put("label", figure.getLabel() != null ? figure.getLabel() : "");
|
||||||
m.put("page", figure.getPage());
|
m.put("page", figure.getPage());
|
||||||
m.put("embedding_id", embeddingId);
|
m.put("embedding_id", embeddingId);
|
||||||
|
m.put("image_text", imageText); // verbatim text visible inside the image
|
||||||
return m;
|
return m;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Replaces Marker's {@code src='{blockId}'} image attributes with resolved API URLs.
|
||||||
|
* Block IDs look like {@code /page/0/Figure/2}.
|
||||||
|
*/
|
||||||
|
private String resolveImageSrcs(String html, UUID bookId, Map<String, String> blockIdToFigureId) {
|
||||||
|
for (Map.Entry<String, String> entry : blockIdToFigureId.entrySet()) {
|
||||||
|
String blockId = entry.getKey();
|
||||||
|
String figureId = entry.getValue();
|
||||||
|
String apiUrl = "/api/v1/figures/" + bookId + "/" + figureId + ".png";
|
||||||
|
// Marker emits both single and double-quoted src attributes
|
||||||
|
html = html.replace("src='" + blockId + "'", "src='" + apiUrl + "'");
|
||||||
|
html = html.replace("src=\"" + blockId + "\"", "src=\"" + apiUrl + "\"");
|
||||||
|
}
|
||||||
|
return html;
|
||||||
|
}
|
||||||
|
|
||||||
|
private String extractCaptionFromSection(SectionEntity section) {
|
||||||
|
if (section == null) return null;
|
||||||
|
for (String line : section.getFullText().split("\n")) {
|
||||||
|
String trimmed = line.strip();
|
||||||
|
if (trimmed.startsWith("Fig.") || trimmed.startsWith("Figure") || trimmed.startsWith("Algorithm")) {
|
||||||
|
return trimmed;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
private String truncate(String msg, int max) {
|
private String truncate(String msg, int max) {
|
||||||
if (msg == null) return null;
|
if (msg == null) return null;
|
||||||
return msg.length() <= max ? msg : msg.substring(0, max);
|
return msg.length() <= max ? msg : msg.substring(0, max);
|
||||||
|
|||||||
@@ -5,10 +5,11 @@ import com.aiteacher.book.BookStatus;
|
|||||||
import com.aiteacher.book.NoKnowledgeSourceException;
|
import com.aiteacher.book.NoKnowledgeSourceException;
|
||||||
import com.aiteacher.document.FigureEntity;
|
import com.aiteacher.document.FigureEntity;
|
||||||
import com.aiteacher.document.SectionEntity;
|
import com.aiteacher.document.SectionEntity;
|
||||||
|
import com.aiteacher.retrieval.CitationValidatorService;
|
||||||
|
import com.aiteacher.retrieval.LabelledContext;
|
||||||
import com.aiteacher.retrieval.NeurosurgeryRetriever;
|
import com.aiteacher.retrieval.NeurosurgeryRetriever;
|
||||||
|
import com.aiteacher.retrieval.QueryExpansionService;
|
||||||
import com.aiteacher.retrieval.RetrievalResult;
|
import com.aiteacher.retrieval.RetrievalResult;
|
||||||
import org.slf4j.Logger;
|
|
||||||
import org.slf4j.LoggerFactory;
|
|
||||||
import org.springframework.ai.chat.client.ChatClient;
|
import org.springframework.ai.chat.client.ChatClient;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
@@ -17,8 +18,6 @@ import java.util.*;
|
|||||||
@Service
|
@Service
|
||||||
public class ChatService {
|
public class ChatService {
|
||||||
|
|
||||||
private static final Logger log = LoggerFactory.getLogger(ChatService.class);
|
|
||||||
|
|
||||||
private static final String SYSTEM_PROMPT = """
|
private static final String SYSTEM_PROMPT = """
|
||||||
You are an expert neurosurgery educator assistant. Answer questions using the
|
You are an expert neurosurgery educator assistant. Answer questions using the
|
||||||
medical textbook content provided to you as context.
|
medical textbook content provided to you as context.
|
||||||
@@ -29,8 +28,8 @@ public class ChatService {
|
|||||||
- Build answers from what is present: procedures, conditions, techniques, and descriptions all contribute; combine them into a rich, structured response
|
- Build answers from what is present: procedures, conditions, techniques, and descriptions all contribute; combine them into a rich, structured response
|
||||||
- Use clear structure: headings, bullet points, or numbered steps where appropriate to maximize clarity
|
- Use clear structure: headings, bullet points, or numbered steps where appropriate to maximize clarity
|
||||||
- Only say you cannot answer if the context is entirely unrelated to the question
|
- Only say you cannot answer if the context is entirely unrelated to the question
|
||||||
- Cite sources for each major point (book title and page number from the context)
|
- Cite sources for each major claim using the reference labels from the context (e.g. [S1], [F2]). Prefer these labels over inventing page numbers, but you may also describe the source naturally if needed.
|
||||||
- When referencing diagrams or figures, cite them as [Fig. X, p.N]
|
- When referencing diagrams or figures, prefer their label from the context (e.g. [F1])
|
||||||
- Maintain continuity with the conversation history
|
- Maintain continuity with the conversation history
|
||||||
- Never fabricate clinical information not present in the context
|
- Never fabricate clinical information not present in the context
|
||||||
""";
|
""";
|
||||||
@@ -40,17 +39,23 @@ public class ChatService {
|
|||||||
private final ChatSessionRepository sessionRepository;
|
private final ChatSessionRepository sessionRepository;
|
||||||
private final MessageRepository messageRepository;
|
private final MessageRepository messageRepository;
|
||||||
private final NeurosurgeryRetriever retriever;
|
private final NeurosurgeryRetriever retriever;
|
||||||
|
private final QueryExpansionService queryExpansionService;
|
||||||
|
private final CitationValidatorService citationValidatorService;
|
||||||
|
|
||||||
public ChatService(ChatClient chatClient,
|
public ChatService(ChatClient chatClient,
|
||||||
BookRepository bookRepository,
|
BookRepository bookRepository,
|
||||||
ChatSessionRepository sessionRepository,
|
ChatSessionRepository sessionRepository,
|
||||||
MessageRepository messageRepository,
|
MessageRepository messageRepository,
|
||||||
NeurosurgeryRetriever retriever) {
|
NeurosurgeryRetriever retriever,
|
||||||
|
QueryExpansionService queryExpansionService,
|
||||||
|
CitationValidatorService citationValidatorService) {
|
||||||
this.chatClient = chatClient;
|
this.chatClient = chatClient;
|
||||||
this.bookRepository = bookRepository;
|
this.bookRepository = bookRepository;
|
||||||
this.sessionRepository = sessionRepository;
|
this.sessionRepository = sessionRepository;
|
||||||
this.messageRepository = messageRepository;
|
this.messageRepository = messageRepository;
|
||||||
this.retriever = retriever;
|
this.retriever = retriever;
|
||||||
|
this.queryExpansionService = queryExpansionService;
|
||||||
|
this.citationValidatorService = citationValidatorService;
|
||||||
}
|
}
|
||||||
|
|
||||||
public ChatSession createSession(String topicId) {
|
public ChatSession createSession(String topicId) {
|
||||||
@@ -85,25 +90,34 @@ public class ChatService {
|
|||||||
List<Message> history = messageRepository.findBySessionIdOrderByCreatedAtAsc(sessionId);
|
List<Message> history = messageRepository.findBySessionIdOrderByCreatedAtAsc(sessionId);
|
||||||
String fullQuestion = buildQuestionWithHistory(history, userContent, session.getTopicId());
|
String fullQuestion = buildQuestionWithHistory(history, userContent, session.getTopicId());
|
||||||
|
|
||||||
// Retrieve context from all ready books (aggregate across books)
|
// Expand only the current user question to clinical terminology for retrieval (US1).
|
||||||
|
// fullQuestion (which includes conversation history) is used for the LLM context prompt,
|
||||||
|
// but retrieval should be driven by a concise clinical rewrite of the actual question.
|
||||||
|
String retrievalQuery = queryExpansionService.expand(userContent).rewritten();
|
||||||
|
|
||||||
|
// Retrieve context from all ready books using the expanded query
|
||||||
List<SectionEntity> allSections = new ArrayList<>();
|
List<SectionEntity> allSections = new ArrayList<>();
|
||||||
List<FigureEntity> allFigures = new ArrayList<>();
|
List<FigureEntity> allFigures = new ArrayList<>();
|
||||||
for (com.aiteacher.book.Book book : readyBooks) {
|
for (com.aiteacher.book.Book book : readyBooks) {
|
||||||
RetrievalResult result = retriever.retrieve(fullQuestion, book.getId());
|
RetrievalResult result = retriever.retrieve(retrievalQuery, book.getId());
|
||||||
allSections.addAll(result.parentSections());
|
allSections.addAll(result.parentSections());
|
||||||
allFigures.addAll(result.figures());
|
allFigures.addAll(result.figures());
|
||||||
}
|
}
|
||||||
|
|
||||||
// Build LLM prompt with section full texts and figure references
|
// Build labelled context prompt (US2): assigns [S1]/[F1] labels to each source
|
||||||
String contextPrompt = buildContextPrompt(fullQuestion, allSections, allFigures);
|
LabelledContext ctx = buildContextPrompt(fullQuestion, allSections, allFigures);
|
||||||
|
|
||||||
String assistantContent = chatClient.prompt()
|
// Generate answer
|
||||||
|
String rawContent = chatClient.prompt()
|
||||||
.system(SYSTEM_PROMPT)
|
.system(SYSTEM_PROMPT)
|
||||||
.user(contextPrompt)
|
.user(ctx.promptText())
|
||||||
.call()
|
.call()
|
||||||
.content();
|
.content();
|
||||||
|
|
||||||
// Build sources list with TEXT and FIGURE entries
|
// Strip any citation labels not present in the retrieved context (US2)
|
||||||
|
String assistantContent = citationValidatorService.validate(rawContent, ctx.allLabels());
|
||||||
|
|
||||||
|
// Attach sources with their ref-labels for frontend traceability
|
||||||
List<Map<String, Object>> sources = buildSources(allSections, allFigures);
|
List<Map<String, Object>> sources = buildSources(allSections, allFigures);
|
||||||
|
|
||||||
Message assistantMessage = new Message(sessionId, MessageRole.ASSISTANT, assistantContent);
|
Message assistantMessage = new Message(sessionId, MessageRole.ASSISTANT, assistantContent);
|
||||||
@@ -126,51 +140,71 @@ public class ChatService {
|
|||||||
// Private helpers
|
// Private helpers
|
||||||
// -------------------------------------------------------------------------
|
// -------------------------------------------------------------------------
|
||||||
|
|
||||||
private String buildContextPrompt(String question,
|
/**
|
||||||
List<SectionEntity> sections,
|
* Builds the LLM context prompt, tagging each section as [S1], [S2]… and
|
||||||
List<FigureEntity> figures) {
|
* each figure as [F1], [F2]… so the model can cite only known sources.
|
||||||
|
*/
|
||||||
|
private LabelledContext buildContextPrompt(String question,
|
||||||
|
List<SectionEntity> sections,
|
||||||
|
List<FigureEntity> figures) {
|
||||||
|
Map<String, SectionEntity> sectionLabels = new LinkedHashMap<>();
|
||||||
|
Map<String, FigureEntity> figureLabels = new LinkedHashMap<>();
|
||||||
StringBuilder sb = new StringBuilder();
|
StringBuilder sb = new StringBuilder();
|
||||||
|
|
||||||
if (!sections.isEmpty()) {
|
if (!sections.isEmpty()) {
|
||||||
sb.append("CONTEXT:\n\n");
|
sb.append("CONTEXT:\n\n");
|
||||||
for (SectionEntity section : sections) {
|
for (int i = 0; i < sections.size(); i++) {
|
||||||
sb.append("[").append(section.getTitle())
|
SectionEntity section = sections.get(i);
|
||||||
.append(", p.").append(section.getPageStart()).append("]\n");
|
String label = "S" + (i + 1);
|
||||||
|
sectionLabels.put(label, section);
|
||||||
|
sb.append("[").append(label).append("] ")
|
||||||
|
.append(section.getTitle())
|
||||||
|
.append(", p.").append(section.getPageStart()).append("\n");
|
||||||
sb.append(section.getFullText()).append("\n\n");
|
sb.append(section.getFullText()).append("\n\n");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!figures.isEmpty()) {
|
if (!figures.isEmpty()) {
|
||||||
sb.append("AVAILABLE FIGURES:\n");
|
sb.append("AVAILABLE FIGURES:\n");
|
||||||
for (FigureEntity figure : figures) {
|
for (int i = 0; i < figures.size(); i++) {
|
||||||
sb.append("- ").append(figure.getLabel() != null ? figure.getLabel() : "Figure")
|
FigureEntity figure = figures.get(i);
|
||||||
|
String label = "F" + (i + 1);
|
||||||
|
figureLabels.put(label, figure);
|
||||||
|
sb.append("[").append(label).append("] ")
|
||||||
|
.append(figure.getLabel() != null ? figure.getLabel() : "Figure")
|
||||||
.append(" (p.").append(figure.getPage()).append("): ")
|
.append(" (p.").append(figure.getPage()).append("): ")
|
||||||
.append(figure.getCaption() != null ? figure.getCaption() : "")
|
.append(figure.getCaption() != null ? figure.getCaption() : "")
|
||||||
.append("\n");
|
.append("\n");
|
||||||
}
|
}
|
||||||
sb.append("\nWhen referencing diagrams, cite them as [Fig. X, p.N].\n\n");
|
sb.append("\nWhen referencing diagrams, use their label from the context (e.g. [F1]).\n\n");
|
||||||
}
|
}
|
||||||
|
|
||||||
sb.append("QUESTION:\n").append(question);
|
sb.append("QUESTION:\n").append(question);
|
||||||
return sb.toString();
|
return new LabelledContext(sectionLabels, figureLabels, sb.toString());
|
||||||
}
|
}
|
||||||
|
|
||||||
private List<Map<String, Object>> buildSources(List<SectionEntity> sections,
|
private List<Map<String, Object>> buildSources(List<SectionEntity> sections,
|
||||||
List<FigureEntity> figures) {
|
List<FigureEntity> figures) {
|
||||||
List<Map<String, Object>> sources = new ArrayList<>();
|
List<Map<String, Object>> sources = new ArrayList<>();
|
||||||
|
|
||||||
for (SectionEntity section : sections) {
|
for (int i = 0; i < sections.size(); i++) {
|
||||||
|
SectionEntity section = sections.get(i);
|
||||||
Map<String, Object> source = new LinkedHashMap<>();
|
Map<String, Object> source = new LinkedHashMap<>();
|
||||||
source.put("type", "TEXT");
|
source.put("type", "TEXT");
|
||||||
|
source.put("refLabel", "S" + (i + 1));
|
||||||
|
source.put("bookId", section.getBookId());
|
||||||
source.put("bookTitle", deriveTitleFromSection(section));
|
source.put("bookTitle", deriveTitleFromSection(section));
|
||||||
source.put("page", section.getPageStart());
|
source.put("page", section.getPageStart());
|
||||||
source.put("chunkText", truncate(section.getFullText(), 500));
|
source.put("chunkText", truncate(section.getFullText(), 500));
|
||||||
sources.add(source);
|
sources.add(source);
|
||||||
}
|
}
|
||||||
|
|
||||||
for (FigureEntity figure : figures) {
|
for (int i = 0; i < figures.size(); i++) {
|
||||||
|
FigureEntity figure = figures.get(i);
|
||||||
Map<String, Object> source = new LinkedHashMap<>();
|
Map<String, Object> source = new LinkedHashMap<>();
|
||||||
source.put("type", "FIGURE");
|
source.put("type", "FIGURE");
|
||||||
|
source.put("refLabel", "F" + (i + 1));
|
||||||
|
source.put("bookId", figure.getBookId());
|
||||||
source.put("bookTitle", bookRepository.findById(figure.getBookId())
|
source.put("bookTitle", bookRepository.findById(figure.getBookId())
|
||||||
.map(com.aiteacher.book.Book::getTitle).orElse("Book"));
|
.map(com.aiteacher.book.Book::getTitle).orElse("Book"));
|
||||||
source.put("page", figure.getPage());
|
source.put("page", figure.getPage());
|
||||||
@@ -178,7 +212,6 @@ public class ChatService {
|
|||||||
source.put("label", figure.getLabel() != null ? figure.getLabel() : "");
|
source.put("label", figure.getLabel() != null ? figure.getLabel() : "");
|
||||||
source.put("caption", figure.getCaption() != null ? figure.getCaption() : "");
|
source.put("caption", figure.getCaption() != null ? figure.getCaption() : "");
|
||||||
source.put("figureType", figure.getFigureType().name());
|
source.put("figureType", figure.getFigureType().name());
|
||||||
// imageUrl assembled from relative path: figures/{bookId}/{filename}
|
|
||||||
String filename = figure.getImagePath().substring(
|
String filename = figure.getImagePath().substring(
|
||||||
figure.getImagePath().lastIndexOf('/') + 1);
|
figure.getImagePath().lastIndexOf('/') + 1);
|
||||||
source.put("imageUrl", "/api/v1/figures/" + figure.getBookId() + "/" + filename);
|
source.put("imageUrl", "/api/v1/figures/" + figure.getBookId() + "/" + filename);
|
||||||
|
|||||||
@@ -1,25 +1,37 @@
|
|||||||
package com.aiteacher.config;
|
package com.aiteacher.config;
|
||||||
|
|
||||||
import org.springframework.beans.factory.annotation.Value;
|
import com.aiteacher.figure.FigureStorageService;
|
||||||
import org.springframework.context.annotation.Configuration;
|
import org.springframework.http.HttpStatus;
|
||||||
import org.springframework.web.servlet.config.annotation.ResourceHandlerRegistry;
|
import org.springframework.web.bind.annotation.*;
|
||||||
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
|
import org.springframework.web.server.ResponseStatusException;
|
||||||
|
|
||||||
import java.nio.file.Paths;
|
import jakarta.servlet.http.HttpServletResponse;
|
||||||
|
import java.io.IOException;
|
||||||
|
|
||||||
@Configuration
|
/**
|
||||||
public class FigureStorageConfig implements WebMvcConfigurer {
|
* Serves figure images by redirecting to a presigned S3 URL.
|
||||||
|
* The key stored in DB is the full S3 object key, e.g. "figures/{bookId}/{figureId}.png".
|
||||||
|
*/
|
||||||
|
@RestController
|
||||||
|
@RequestMapping("/api/v1/figures")
|
||||||
|
public class FigureStorageConfig {
|
||||||
|
|
||||||
private final String basePath;
|
private final FigureStorageService figureStorageService;
|
||||||
|
|
||||||
public FigureStorageConfig(@Value("${app.figure-storage.base-path:./uploads}") String basePath) {
|
public FigureStorageConfig(FigureStorageService figureStorageService) {
|
||||||
this.basePath = Paths.get(basePath).toAbsolutePath().normalize().toString();
|
this.figureStorageService = figureStorageService;
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@GetMapping("/{bookId}/{filename}")
|
||||||
public void addResourceHandlers(ResourceHandlerRegistry registry) {
|
public void serve(@PathVariable String bookId,
|
||||||
// Serve GET /api/v1/figures/** from the local file store
|
@PathVariable String filename,
|
||||||
registry.addResourceHandler("/api/v1/figures/**")
|
HttpServletResponse response) throws IOException {
|
||||||
.addResourceLocations("file:" + basePath + "/figures/");
|
String key = "figures/" + bookId + "/" + filename;
|
||||||
|
try {
|
||||||
|
String url = figureStorageService.presignedUrl(key);
|
||||||
|
response.sendRedirect(url);
|
||||||
|
} catch (Exception ex) {
|
||||||
|
throw new ResponseStatusException(HttpStatus.NOT_FOUND, "Figure not found: " + key);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,30 @@
|
|||||||
|
package com.aiteacher.config;
|
||||||
|
|
||||||
|
import org.springframework.beans.factory.annotation.Value;
|
||||||
|
import org.springframework.context.annotation.Bean;
|
||||||
|
import org.springframework.context.annotation.Configuration;
|
||||||
|
import org.springframework.http.client.JdkClientHttpRequestFactory;
|
||||||
|
import org.springframework.web.client.RestClient;
|
||||||
|
|
||||||
|
import java.net.http.HttpClient;
|
||||||
|
|
||||||
|
@Configuration
|
||||||
|
public class MarkerConfig {
|
||||||
|
|
||||||
|
@Value("${app.marker.base-url:http://localhost:8000}")
|
||||||
|
private String markerBaseUrl;
|
||||||
|
|
||||||
|
@Bean
|
||||||
|
RestClient markerRestClient() {
|
||||||
|
// Use the JDK HTTP client with no timeout — Marker conversions can take several minutes.
|
||||||
|
HttpClient httpClient = HttpClient.newBuilder()
|
||||||
|
.build();
|
||||||
|
JdkClientHttpRequestFactory factory = new JdkClientHttpRequestFactory(httpClient);
|
||||||
|
// No read timeout set: JDK HTTP client defaults to no deadline.
|
||||||
|
|
||||||
|
return RestClient.builder()
|
||||||
|
.baseUrl(markerBaseUrl)
|
||||||
|
.requestFactory(factory)
|
||||||
|
.build();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
package com.aiteacher.config;
|
||||||
|
|
||||||
|
import org.springframework.aot.hint.MemberCategory;
|
||||||
|
import org.springframework.aot.hint.RuntimeHints;
|
||||||
|
import org.springframework.aot.hint.RuntimeHintsRegistrar;
|
||||||
|
import org.springframework.aot.hint.TypeReference;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* GraalVM native-image runtime hints for third-party libraries that use reflection
|
||||||
|
* or classpath resource scanning not covered by Spring Boot's AOT processor.
|
||||||
|
*
|
||||||
|
* Registered via @ImportRuntimeHints on AiTeacherApplication.
|
||||||
|
*/
|
||||||
|
public class NativeHintsConfig implements RuntimeHintsRegistrar {
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void registerHints(RuntimeHints hints, ClassLoader classLoader) {
|
||||||
|
// PDFBox — font and encoding resources loaded via classpath scanning at runtime
|
||||||
|
hints.resources().registerPattern("org/apache/pdfbox/resources/*");
|
||||||
|
hints.resources().registerPattern("org/apache/pdfbox/resources/afm/*");
|
||||||
|
hints.resources().registerPattern("org/apache/pdfbox/resources/cmap/*");
|
||||||
|
hints.resources().registerPattern("org/apache/pdfbox/resources/glyphlist/*");
|
||||||
|
hints.resources().registerPattern("org/apache/pdfbox/resources/icc/*");
|
||||||
|
hints.resources().registerPattern("org/apache/pdfbox/resources/ttf/*");
|
||||||
|
hints.resources().registerPattern("org/apache/pdfbox/resources/version.properties");
|
||||||
|
|
||||||
|
// PDFBox — font encoding classes instantiated via reflection
|
||||||
|
hints.reflection().registerType(
|
||||||
|
org.apache.pdfbox.pdmodel.font.encoding.GlyphList.class,
|
||||||
|
MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS,
|
||||||
|
MemberCategory.INVOKE_PUBLIC_METHODS
|
||||||
|
);
|
||||||
|
hints.reflection().registerType(
|
||||||
|
org.apache.pdfbox.pdmodel.font.encoding.WinAnsiEncoding.class,
|
||||||
|
MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS
|
||||||
|
);
|
||||||
|
hints.reflection().registerType(
|
||||||
|
org.apache.pdfbox.pdmodel.font.encoding.MacRomanEncoding.class,
|
||||||
|
MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS
|
||||||
|
);
|
||||||
|
hints.reflection().registerType(
|
||||||
|
org.apache.pdfbox.pdmodel.font.encoding.MacExpertEncoding.class,
|
||||||
|
MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS
|
||||||
|
);
|
||||||
|
hints.reflection().registerType(
|
||||||
|
org.apache.pdfbox.pdmodel.font.encoding.StandardEncoding.class,
|
||||||
|
MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS
|
||||||
|
);
|
||||||
|
|
||||||
|
// JPA / Hibernate — array types used in entity mappings
|
||||||
|
hints.reflection().registerType(java.util.UUID[].class, MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS);
|
||||||
|
|
||||||
|
// JBoss Logging — message logger implementations generated by annotation processor.
|
||||||
|
// JBoss Logging uses reflection to look up the generated *_$logger class by name.
|
||||||
|
registerJBossLogger(hints, "org.hibernate.jpa.internal.JpaLogger_$logger");
|
||||||
|
registerJBossLogger(hints, "org.hibernate.internal.CoreMessageLogger_$logger");
|
||||||
|
registerJBossLogger(hints, "org.hibernate.internal.EntityManagerMessageLogger_$logger");
|
||||||
|
|
||||||
|
// AWS SDK v2 — HTTP client and SdkPojo serialization
|
||||||
|
hints.resources().registerPattern("software/amazon/awssdk/global/handlers/execution.interceptors");
|
||||||
|
hints.resources().registerPattern("software/amazon/awssdk/services/s3/execution.interceptors");
|
||||||
|
hints.resources().registerPattern("codegen-resources/s3/*");
|
||||||
|
hints.reflection().registerType(
|
||||||
|
software.amazon.awssdk.services.s3.S3Client.class,
|
||||||
|
MemberCategory.INVOKE_PUBLIC_METHODS
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void registerJBossLogger(RuntimeHints hints, String className) {
|
||||||
|
hints.reflection().registerType(
|
||||||
|
TypeReference.of(className),
|
||||||
|
MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS,
|
||||||
|
MemberCategory.INVOKE_PUBLIC_METHODS
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -20,7 +20,9 @@ public class SecurityConfig {
|
|||||||
@Bean
|
@Bean
|
||||||
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
|
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
|
||||||
http
|
http
|
||||||
.authorizeHttpRequests(auth -> auth.anyRequest().authenticated())
|
.authorizeHttpRequests(auth -> auth
|
||||||
|
.requestMatchers("/api/v1/figures/**").permitAll()
|
||||||
|
.anyRequest().authenticated())
|
||||||
.httpBasic(Customizer.withDefaults())
|
.httpBasic(Customizer.withDefaults())
|
||||||
.csrf(AbstractHttpConfigurer::disable);
|
.csrf(AbstractHttpConfigurer::disable);
|
||||||
return http.build();
|
return http.build();
|
||||||
@@ -28,9 +30,10 @@ public class SecurityConfig {
|
|||||||
|
|
||||||
@Bean
|
@Bean
|
||||||
public UserDetailsService userDetailsService(
|
public UserDetailsService userDetailsService(
|
||||||
|
@Value("${app.auth.username}") String username,
|
||||||
@Value("${app.auth.password}") String password) {
|
@Value("${app.auth.password}") String password) {
|
||||||
UserDetails user = User.builder()
|
UserDetails user = User.builder()
|
||||||
.username("neurosurgeon")
|
.username(username)
|
||||||
.password("{noop}" + password)
|
.password("{noop}" + password)
|
||||||
.roles("USER")
|
.roles("USER")
|
||||||
.build();
|
.build();
|
||||||
|
|||||||
@@ -1,43 +1,43 @@
|
|||||||
package com.aiteacher.document;
|
package com.aiteacher.document;
|
||||||
|
|
||||||
import com.aiteacher.figure.FigureStorageService;
|
import com.aiteacher.figure.FigureStorageService;
|
||||||
import org.apache.pdfbox.Loader;
|
|
||||||
import org.apache.pdfbox.cos.COSName;
|
|
||||||
import org.apache.pdfbox.pdmodel.PDDocument;
|
|
||||||
import org.apache.pdfbox.pdmodel.PDPage;
|
|
||||||
import org.apache.pdfbox.pdmodel.graphics.PDXObject;
|
|
||||||
import org.apache.pdfbox.pdmodel.graphics.image.PDImageXObject;
|
|
||||||
import org.slf4j.Logger;
|
import org.slf4j.Logger;
|
||||||
import org.slf4j.LoggerFactory;
|
import org.slf4j.LoggerFactory;
|
||||||
import org.springframework.beans.factory.annotation.Value;
|
import org.springframework.beans.factory.annotation.Value;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
import javax.imageio.ImageIO;
|
||||||
import java.awt.image.BufferedImage;
|
import java.awt.image.BufferedImage;
|
||||||
|
import java.io.ByteArrayInputStream;
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
import java.nio.file.Path;
|
|
||||||
import java.util.ArrayList;
|
import java.util.ArrayList;
|
||||||
|
import java.util.HashMap;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
import java.util.UUID;
|
import java.util.UUID;
|
||||||
import java.util.regex.Matcher;
|
import java.util.regex.Matcher;
|
||||||
import java.util.regex.Pattern;
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Extracts images from each PDF page using PDFBox.
|
* Extracts figure images from {@link PageResult.FigureData} entries produced by
|
||||||
* Images below the configured minimum size are skipped.
|
* {@link MarkerPageParser}.
|
||||||
* Caption is detected by the "Fig." pattern in page text.
|
*
|
||||||
|
* <p>Marker returns pre-cropped PNG bytes for each detected figure, so no PDFBox
|
||||||
|
* page rendering or bounding-box cropping is needed. This service:
|
||||||
|
* <ol>
|
||||||
|
* <li>Decodes the PNG bytes to check dimensions (skip images below min size)</li>
|
||||||
|
* <li>Classifies the figure type from caption and surrounding text keywords</li>
|
||||||
|
* <li>Persists the image via {@link FigureStorageService}</li>
|
||||||
|
* <li>Persists a {@link FigureEntity} to the database</li>
|
||||||
|
* </ol>
|
||||||
*/
|
*/
|
||||||
@Service
|
@Service
|
||||||
public class FigureExtractionService {
|
public class FigureExtractionService {
|
||||||
|
|
||||||
private static final Logger log = LoggerFactory.getLogger(FigureExtractionService.class);
|
private static final Logger log = LoggerFactory.getLogger(FigureExtractionService.class);
|
||||||
|
|
||||||
// Caption: line starting with "Fig." or "Figure" followed by a number
|
|
||||||
private static final Pattern CAPTION_PATTERN =
|
|
||||||
Pattern.compile("(?m)^(Fig\\.?\\s*\\d+[\\-.]?\\d*[^\\n]*)", Pattern.CASE_INSENSITIVE);
|
|
||||||
|
|
||||||
// Figure label: "Fig. 12-4" or "Fig. 12.4"
|
|
||||||
private static final Pattern LABEL_PATTERN =
|
private static final Pattern LABEL_PATTERN =
|
||||||
Pattern.compile("(?i)Fig\\.?\\s*(\\d+[\\-.\\d]*)");
|
Pattern.compile("(?i)Fig\\.?\\s*(\\d+[\\-.\\d]*)");
|
||||||
|
|
||||||
private final FigureStorageService storageService;
|
private final FigureStorageService storageService;
|
||||||
private final FigureRepository figureRepository;
|
private final FigureRepository figureRepository;
|
||||||
@@ -52,65 +52,77 @@ public class FigureExtractionService {
|
|||||||
this.minImageSizePx = minImageSizePx;
|
this.minImageSizePx = minImageSizePx;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Holds the extraction output: persisted figures and a Marker blockId → DB figureId map. */
|
||||||
|
public record ExtractionResult(List<FigureEntity> figures, Map<String, String> blockIdToFigureId) {}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Extracts all qualifying images from the PDF for the given book.
|
* Extracts and persists figures for all pages described by {@code pageResults}.
|
||||||
* Returns persisted FigureEntity list (without vision descriptions — set later).
|
*
|
||||||
|
* @param bookId owning book
|
||||||
|
* @param chapterId chapter bucket for these sections
|
||||||
|
* @param pageResults Marker parse output — each entry's {@code figures} list
|
||||||
|
* carries pre-cropped PNG bytes for that page
|
||||||
|
* @return {@link ExtractionResult} with persisted figures and blockId→figureId map
|
||||||
|
* (used to resolve markdown image placeholders)
|
||||||
*/
|
*/
|
||||||
public List<FigureEntity> extract(UUID bookId, String chapterId,
|
public ExtractionResult extract(UUID bookId, String chapterId,
|
||||||
List<SectionEntity> sections, Path pdfPath) {
|
List<PageResult> pageResults) {
|
||||||
List<FigureEntity> figures = new ArrayList<>();
|
List<FigureEntity> figures = new ArrayList<>();
|
||||||
|
Map<String, String> blockIdToFigureId = new HashMap<>();
|
||||||
int figureCounter = 0;
|
int figureCounter = 0;
|
||||||
|
|
||||||
try (PDDocument doc = Loader.loadPDF(pdfPath.toFile())) {
|
for (PageResult page : pageResults) {
|
||||||
for (SectionEntity section : sections) {
|
if (page.figures().isEmpty()) continue;
|
||||||
int pageIndex = section.getPageStart() - 1; // 0-based
|
|
||||||
if (pageIndex < 0 || pageIndex >= doc.getNumberOfPages()) continue;
|
|
||||||
|
|
||||||
PDPage page = doc.getPage(pageIndex);
|
|
||||||
String pageText = section.getFullText();
|
|
||||||
|
|
||||||
|
for (PageResult.FigureData figureData : page.figures()) {
|
||||||
try {
|
try {
|
||||||
for (COSName name : page.getResources().getXObjectNames()) {
|
BufferedImage image = decodeImage(figureData.imageBytes());
|
||||||
PDXObject xObject = page.getResources().getXObject(name);
|
if (image == null) {
|
||||||
if (!(xObject instanceof PDImageXObject image)) continue;
|
log.debug("Could not decode image on page {} of book {} (block {})",
|
||||||
|
page.pageNumber(), bookId, figureData.blockId());
|
||||||
BufferedImage bufferedImage = image.getImage();
|
continue;
|
||||||
if (bufferedImage.getWidth() < minImageSizePx
|
|
||||||
|| bufferedImage.getHeight() < minImageSizePx) {
|
|
||||||
continue; // skip decorative images
|
|
||||||
}
|
|
||||||
|
|
||||||
figureCounter++;
|
|
||||||
String figureId = bookId + "-fig-" + pageIndex + "-" + figureCounter;
|
|
||||||
String caption = detectCaption(pageText);
|
|
||||||
String label = detectLabel(caption, figureCounter);
|
|
||||||
FigureType type = classifyType(caption, pageText);
|
|
||||||
|
|
||||||
String imagePath = storageService.save(bookId, figureId, bufferedImage);
|
|
||||||
|
|
||||||
FigureEntity figure = new FigureEntity(
|
|
||||||
figureId, bookId, section.getId(), chapterId,
|
|
||||||
label, caption, type, section.getPageStart(), imagePath
|
|
||||||
);
|
|
||||||
figures.add(figureRepository.save(figure));
|
|
||||||
}
|
}
|
||||||
} catch (IOException ex) {
|
if (image.getWidth() < minImageSizePx || image.getHeight() < minImageSizePx) {
|
||||||
log.warn("Failed to extract images from page {} of book {}: {}",
|
log.debug("Skipping small figure on page {} ({}×{})",
|
||||||
section.getPageStart(), bookId, ex.getMessage());
|
page.pageNumber(), image.getWidth(), image.getHeight());
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
figureCounter++;
|
||||||
|
String figureId = bookId + "-fig-" + page.pageNumber() + "-" + figureCounter;
|
||||||
|
String caption = figureData.nearestCaption();
|
||||||
|
String label = detectLabel(caption, figureCounter);
|
||||||
|
FigureType type = classifyType(caption, page.orderedText());
|
||||||
|
|
||||||
|
String sectionId = bookId + "-p" + page.pageNumber();
|
||||||
|
String imagePath = storageService.save(bookId, figureId, image);
|
||||||
|
|
||||||
|
FigureEntity figure = new FigureEntity(
|
||||||
|
figureId, bookId, sectionId, chapterId,
|
||||||
|
label, caption, type, page.pageNumber(), imagePath);
|
||||||
|
figures.add(figureRepository.save(figure));
|
||||||
|
blockIdToFigureId.put(figureData.blockId(), figureId);
|
||||||
|
|
||||||
|
} catch (Exception ex) {
|
||||||
|
log.warn("Failed to extract figure on page {} of book {}: {}",
|
||||||
|
page.pageNumber(), bookId, ex.getMessage());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} catch (IOException ex) {
|
|
||||||
log.error("Could not open PDF for image extraction, book {}", bookId, ex);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
log.info("Extracted {} figures for book {}", figures.size(), bookId);
|
log.info("Extracted {} figures for book {}", figures.size(), bookId);
|
||||||
return figures;
|
return new ExtractionResult(figures, blockIdToFigureId);
|
||||||
}
|
}
|
||||||
|
|
||||||
private String detectCaption(String pageText) {
|
// --- Private helpers ---
|
||||||
if (pageText == null) return null;
|
|
||||||
Matcher m = CAPTION_PATTERN.matcher(pageText);
|
private BufferedImage decodeImage(byte[] imageBytes) {
|
||||||
return m.find() ? m.group(1).trim() : null;
|
if (imageBytes == null || imageBytes.length == 0) return null;
|
||||||
|
try {
|
||||||
|
return ImageIO.read(new ByteArrayInputStream(imageBytes));
|
||||||
|
} catch (IOException ex) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private String detectLabel(String caption, int counter) {
|
private String detectLabel(String caption, int counter) {
|
||||||
@@ -122,14 +134,18 @@ public class FigureExtractionService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private FigureType classifyType(String caption, String pageText) {
|
private FigureType classifyType(String caption, String pageText) {
|
||||||
String combined = ((caption != null ? caption : "") + " " + (pageText != null ? pageText : "")).toLowerCase();
|
String combined = ((caption != null ? caption : "") + " " +
|
||||||
|
(pageText != null ? pageText : "")).toLowerCase();
|
||||||
if (combined.contains("mri") || combined.contains("ct ") || combined.contains("magnetic")
|
if (combined.contains("mri") || combined.contains("ct ") || combined.contains("magnetic")
|
||||||
|| combined.contains("tomography")) return FigureType.MRI_CT_SCAN;
|
|| combined.contains("tomography")) return FigureType.MRI_CT_SCAN;
|
||||||
if (combined.contains("intraoperative") || combined.contains("intra-op")) return FigureType.INTRAOPERATIVE_IMAGE;
|
if (combined.contains("intraoperative") || combined.contains("intra-op"))
|
||||||
if (caption != null && caption.toLowerCase().startsWith("table")) return FigureType.TABLE;
|
return FigureType.INTRAOPERATIVE_IMAGE;
|
||||||
|
if (caption != null && caption.toLowerCase().startsWith("table"))
|
||||||
|
return FigureType.TABLE;
|
||||||
if (combined.contains("chart") || combined.contains("histogram") || combined.contains("graph"))
|
if (combined.contains("chart") || combined.contains("histogram") || combined.contains("graph"))
|
||||||
return FigureType.CHART;
|
return FigureType.CHART;
|
||||||
if (combined.contains("photograph") || combined.contains("photo")) return FigureType.SURGICAL_PHOTOGRAPH;
|
if (combined.contains("photograph") || combined.contains("photo"))
|
||||||
|
return FigureType.SURGICAL_PHOTOGRAPH;
|
||||||
return FigureType.ANATOMICAL_DIAGRAM;
|
return FigureType.ANATOMICAL_DIAGRAM;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,14 @@
|
|||||||
|
package com.aiteacher.document;
|
||||||
|
|
||||||
|
import java.util.UUID;
|
||||||
|
|
||||||
|
public interface MarkdownStorageService {
|
||||||
|
/** Uploads the markdown content and returns the S3 key. */
|
||||||
|
String save(UUID bookId, int pageNumber, String markdown);
|
||||||
|
|
||||||
|
/** Downloads and returns the markdown content for the given book and page. */
|
||||||
|
String getText(UUID bookId, int pageNumber);
|
||||||
|
|
||||||
|
/** Deletes all markdown files for the given book. */
|
||||||
|
void deleteAll(UUID bookId);
|
||||||
|
}
|
||||||
@@ -0,0 +1,335 @@
|
|||||||
|
package com.aiteacher.document;
|
||||||
|
|
||||||
|
import tools.jackson.databind.JsonNode;
|
||||||
|
import tools.jackson.databind.ObjectMapper;
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
import org.springframework.beans.factory.annotation.Qualifier;
|
||||||
|
import org.springframework.core.io.FileSystemResource;
|
||||||
|
import org.springframework.http.MediaType;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
import org.springframework.util.LinkedMultiValueMap;
|
||||||
|
import org.springframework.util.MultiValueMap;
|
||||||
|
import org.springframework.web.client.RestClient;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.util.*;
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parses a PDF with a single call to the Marker server using {@code output_format=json}.
|
||||||
|
*
|
||||||
|
* <p>The JSON response contains an {@code output} field that is itself a JSON string with a
|
||||||
|
* tree structure: the root has a {@code children} array where each item is a {@code Page} block.
|
||||||
|
* Each block carries an {@code html} field with {@code <content-ref src='blockId'>} placeholders
|
||||||
|
* that reference its {@code children} by ID.
|
||||||
|
*
|
||||||
|
* <p>{@link #jsonToHtml} mirrors the Marker Python {@code json_to_html} utility: it walks the
|
||||||
|
* tree recursively and resolves every {@code content-ref} with the rendered HTML of the
|
||||||
|
* referenced child block.
|
||||||
|
*
|
||||||
|
* <p>Returns a {@link ParsedBook} with:
|
||||||
|
* <ul>
|
||||||
|
* <li>{@code pages} — one {@link PageResult} per non-empty page (drives embeddings)</li>
|
||||||
|
* <li>{@code htmlByPage} — full resolved HTML per page (saved to S3 for the reader)</li>
|
||||||
|
* </ul>
|
||||||
|
*/
|
||||||
|
@Service
|
||||||
|
public class MarkerPageParser {
|
||||||
|
|
||||||
|
private static final Logger log = LoggerFactory.getLogger(MarkerPageParser.class);
|
||||||
|
|
||||||
|
private static final Set<String> TEXT_BLOCK_TYPES = Set.of(
|
||||||
|
"Text", "TextInlineMath", "ListItem", "Table", "TableOfContents", "Code", "Equation",
|
||||||
|
"Footnote", "Caption", "PageHeader", "PageFooter", "Handwriting"
|
||||||
|
);
|
||||||
|
private static final Set<String> FIGURE_BLOCK_TYPES = Set.of("Figure", "Picture", "FigureGroup", "PictureGroup");
|
||||||
|
|
||||||
|
private static final int CHUNK_SIZE = 100;
|
||||||
|
|
||||||
|
private static final ObjectMapper MAPPER = new ObjectMapper();
|
||||||
|
|
||||||
|
private final RestClient restClient;
|
||||||
|
private final PdfSplitterService pdfSplitterService;
|
||||||
|
|
||||||
|
public MarkerPageParser(@Qualifier("markerRestClient") RestClient restClient,
|
||||||
|
PdfSplitterService pdfSplitterService) {
|
||||||
|
this.restClient = restClient;
|
||||||
|
this.pdfSplitterService = pdfSplitterService;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parses a PDF by splitting it into {@value #CHUNK_SIZE}-page chunks, submitting each
|
||||||
|
* chunk to Marker individually, and merging the results into a single {@link ParsedBook}.
|
||||||
|
* Page numbers in the merged result are absolute (1-based across the whole document).
|
||||||
|
*/
|
||||||
|
public ParsedBook parse(Path pdfPath) throws IOException {
|
||||||
|
List<PdfSplitterService.PdfChunk> chunks = pdfSplitterService.split(pdfPath, CHUNK_SIZE);
|
||||||
|
log.info("Processing {} chunk(s) for {}", chunks.size(), pdfPath.getFileName());
|
||||||
|
|
||||||
|
List<PageResult> allPages = new ArrayList<>();
|
||||||
|
Map<Integer, String> allHtml = new LinkedHashMap<>();
|
||||||
|
|
||||||
|
try {
|
||||||
|
for (int c = 0; c < chunks.size(); c++) {
|
||||||
|
PdfSplitterService.PdfChunk chunk = chunks.get(c);
|
||||||
|
log.info("Submitting chunk {}/{} to Marker (page offset {})", c + 1, chunks.size(), chunk.pageOffset());
|
||||||
|
|
||||||
|
ParsedBook chunkResult = submitChunk(chunk.tempFile());
|
||||||
|
|
||||||
|
// Rebase page numbers from chunk-relative to document-absolute
|
||||||
|
for (PageResult page : chunkResult.pages()) {
|
||||||
|
int absolutePage = chunk.pageOffset() + page.pageNumber();
|
||||||
|
allPages.add(new PageResult(absolutePage, page.orderedText(), page.headingTitle(), page.figures()));
|
||||||
|
}
|
||||||
|
chunkResult.htmlByPage().forEach((chunkPage, html) ->
|
||||||
|
allHtml.put(chunk.pageOffset() + chunkPage, html));
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
// Delete temporary chunk files (skip if the chunk is the original PDF)
|
||||||
|
for (PdfSplitterService.PdfChunk chunk : chunks) {
|
||||||
|
if (!chunk.tempFile().equals(pdfPath)) {
|
||||||
|
try { Files.deleteIfExists(chunk.tempFile()); }
|
||||||
|
catch (IOException e) { log.warn("Could not delete temp chunk {}", chunk.tempFile()); }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
log.info("Marker produced {} non-empty pages from {} chunk(s) of {}",
|
||||||
|
allPages.size(), chunks.size(), pdfPath.getFileName());
|
||||||
|
return new ParsedBook(allPages, allHtml);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Submits a single PDF file to Marker and returns the parsed result with chunk-relative page numbers. */
|
||||||
|
private ParsedBook submitChunk(Path chunkPath) {
|
||||||
|
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
|
||||||
|
body.add("file", new FileSystemResource(chunkPath));
|
||||||
|
body.add("output_format", "json");
|
||||||
|
|
||||||
|
JsonNode response = restClient.post()
|
||||||
|
.uri("/marker/upload")
|
||||||
|
.contentType(MediaType.MULTIPART_FORM_DATA)
|
||||||
|
.body(body)
|
||||||
|
.retrieve()
|
||||||
|
.body(JsonNode.class);
|
||||||
|
|
||||||
|
try {
|
||||||
|
Files.writeString(Path.of("/tmp/marker-response-json.json"), response.toPrettyString());
|
||||||
|
} catch (IOException e) {
|
||||||
|
log.warn("Could not save Marker response to /tmp/marker-response-json.json", e);
|
||||||
|
}
|
||||||
|
|
||||||
|
List<JsonNode> pageNodes = extractPages(response);
|
||||||
|
if (pageNodes.isEmpty()) {
|
||||||
|
log.warn("Marker returned no pages for chunk {}", chunkPath.getFileName());
|
||||||
|
return new ParsedBook(List.of(), Map.of());
|
||||||
|
}
|
||||||
|
|
||||||
|
List<PageResult> pages = new ArrayList<>();
|
||||||
|
Map<Integer, String> htmlByPage = new LinkedHashMap<>();
|
||||||
|
|
||||||
|
for (int i = 0; i < pageNodes.size(); i++) {
|
||||||
|
JsonNode pageNode = pageNodes.get(i);
|
||||||
|
int pageNumber = i + 1; // 1-based, chunk-relative
|
||||||
|
|
||||||
|
PageResult result = buildPageResult(pageNode, pageNumber);
|
||||||
|
String html = jsonToHtml(pageNode);
|
||||||
|
|
||||||
|
// Always save HTML so the reader can navigate to every page
|
||||||
|
htmlByPage.put(pageNumber, html);
|
||||||
|
|
||||||
|
// Only queue for embedding if the page has extractable content
|
||||||
|
if (!result.orderedText().isBlank() || !result.figures().isEmpty()) {
|
||||||
|
pages.add(result);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return new ParsedBook(pages, htmlByPage);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Page extraction ───────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parses the {@code output} JSON string and returns the list of page nodes
|
||||||
|
* (the top-level {@code children} of the document root).
|
||||||
|
*/
|
||||||
|
private List<JsonNode> extractPages(JsonNode response) {
|
||||||
|
if (response == null) return List.of();
|
||||||
|
JsonNode outputNode = response.path("output");
|
||||||
|
if (outputNode.isMissingNode()) {
|
||||||
|
log.warn("Marker response has no 'output' field");
|
||||||
|
return List.of();
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
JsonNode root = MAPPER.readTree(outputNode.stringValue());
|
||||||
|
JsonNode children = root.path("children");
|
||||||
|
if (children.isMissingNode() || !children.isArray()) {
|
||||||
|
log.warn("Marker output root has no 'children' array");
|
||||||
|
return List.of();
|
||||||
|
}
|
||||||
|
List<JsonNode> result = new ArrayList<>();
|
||||||
|
children.forEach(result::add);
|
||||||
|
return result;
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.warn("Could not parse Marker 'output' string as JSON: {}", e.getMessage());
|
||||||
|
return List.of();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── HTML rendering ────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Java equivalent of the Marker Python {@code json_to_html} utility.
|
||||||
|
*
|
||||||
|
* <p>Algorithm:
|
||||||
|
* <ol>
|
||||||
|
* <li>If the block has no children, return its {@code html} as-is (leaf node).</li>
|
||||||
|
* <li>Otherwise recursively render each child, then replace every
|
||||||
|
* {@code <content-ref src='childId'>} placeholder in the block's own {@code html}
|
||||||
|
* with the rendered child HTML.</li>
|
||||||
|
* </ol>
|
||||||
|
*/
|
||||||
|
String jsonToHtml(JsonNode block) {
|
||||||
|
String html = str(block.path("html"));
|
||||||
|
|
||||||
|
// If the block carries image data, inject <img> data-URI tags.
|
||||||
|
// Marker stores base64 image bytes in block.images keyed by block ID.
|
||||||
|
// Picture/Figure leaf blocks have empty html, so this is the only way to
|
||||||
|
// get the image into the rendered output.
|
||||||
|
JsonNode images = block.path("images");
|
||||||
|
if (!images.isMissingNode() && !images.isNull() && !images.isEmpty()) {
|
||||||
|
StringBuilder imgTags = new StringBuilder();
|
||||||
|
images.properties().forEach(entry -> {
|
||||||
|
String base64 = str(entry.getValue());
|
||||||
|
if (!base64.isEmpty()) {
|
||||||
|
String mime = detectImageMime(base64);
|
||||||
|
imgTags.append("<img src=\"data:").append(mime)
|
||||||
|
.append(";base64,").append(base64).append("\">");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
if (!imgTags.isEmpty()) {
|
||||||
|
html = html + imgTags;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
JsonNode children = block.path("children");
|
||||||
|
if (children.isMissingNode() || children.isNull() || !children.isArray() || children.isEmpty()) {
|
||||||
|
return html; // leaf node
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build id → rendered-html map for all direct children
|
||||||
|
Map<String, String> childHtml = new LinkedHashMap<>();
|
||||||
|
for (JsonNode child : children) {
|
||||||
|
String id = str(child.path("id"));
|
||||||
|
childHtml.put(id, jsonToHtml(child));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Replace every <content-ref src='id'></content-ref> with the child's HTML
|
||||||
|
for (Map.Entry<String, String> entry : childHtml.entrySet()) {
|
||||||
|
String ref = "<content-ref src='" + entry.getKey() + "'></content-ref>";
|
||||||
|
html = html.replace(ref, entry.getValue());
|
||||||
|
}
|
||||||
|
|
||||||
|
return html;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── PageResult (text + figures for embeddings) ────────────────────────────
|
||||||
|
|
||||||
|
private PageResult buildPageResult(JsonNode pageBlock, int pageNumber) {
|
||||||
|
StringBuilder text = new StringBuilder();
|
||||||
|
String[] headingTitle = {null};
|
||||||
|
List<PageResult.FigureData> figures = new ArrayList<>();
|
||||||
|
|
||||||
|
walkBlock(pageBlock, text, headingTitle, figures);
|
||||||
|
return new PageResult(pageNumber, text.toString().strip(), headingTitle[0], figures);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Recursively walks the block tree, collecting text and figures in reading order. */
|
||||||
|
private void walkBlock(JsonNode block, StringBuilder text, String[] headingTitle,
|
||||||
|
List<PageResult.FigureData> figures) {
|
||||||
|
String type = str(block.path("block_type"));
|
||||||
|
|
||||||
|
if ("SectionHeader".equals(type)) {
|
||||||
|
String heading = stripHtml(str(block.path("html"))).strip();
|
||||||
|
if (!heading.isEmpty() && headingTitle[0] == null) headingTitle[0] = heading;
|
||||||
|
appendText(text, heading);
|
||||||
|
|
||||||
|
} else if (TEXT_BLOCK_TYPES.contains(type)) {
|
||||||
|
appendText(text, stripHtml(str(block.path("html"))));
|
||||||
|
|
||||||
|
} else if (FIGURE_BLOCK_TYPES.contains(type)) {
|
||||||
|
String caption = findCaption(block);
|
||||||
|
extractFigures(block, caption, figures);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Recurse into children (content-ref ordering is implicit via tree order)
|
||||||
|
JsonNode children = block.path("children");
|
||||||
|
if (!children.isMissingNode() && !children.isNull() && children.isArray()) {
|
||||||
|
for (JsonNode child : children) {
|
||||||
|
walkBlock(child, text, headingTitle, figures);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Finds the first Caption child inside a figure block, if any. */
|
||||||
|
private String findCaption(JsonNode figureBlock) {
|
||||||
|
JsonNode children = figureBlock.path("children");
|
||||||
|
if (children.isMissingNode() || !children.isArray()) return null;
|
||||||
|
for (JsonNode child : children) {
|
||||||
|
if ("Caption".equals(str(child.path("block_type")))) {
|
||||||
|
String caption = stripHtml(str(child.path("html"))).strip();
|
||||||
|
return caption.isEmpty() ? null : caption;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void extractFigures(JsonNode block, String caption, List<PageResult.FigureData> out) {
|
||||||
|
JsonNode images = block.path("images");
|
||||||
|
if (images.isMissingNode() || images.isEmpty()) return;
|
||||||
|
|
||||||
|
images.properties().forEach(entry -> {
|
||||||
|
String blockId = entry.getKey();
|
||||||
|
String base64 = str(entry.getValue());
|
||||||
|
if (base64.isEmpty()) return;
|
||||||
|
try {
|
||||||
|
byte[] bytes = Base64.getDecoder().decode(base64);
|
||||||
|
out.add(new PageResult.FigureData(bytes, caption, blockId));
|
||||||
|
} catch (IllegalArgumentException ex) {
|
||||||
|
log.warn("Could not decode base64 image for block {}: {}", blockId, ex.getMessage());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Utilities ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
private void appendText(StringBuilder sb, String text) {
|
||||||
|
if (text == null) return;
|
||||||
|
String stripped = text.strip();
|
||||||
|
if (stripped.isEmpty()) return;
|
||||||
|
if (sb.length() > 0) sb.append("\n\n");
|
||||||
|
sb.append(stripped);
|
||||||
|
}
|
||||||
|
|
||||||
|
private String stripHtml(String html) {
|
||||||
|
if (html == null || html.isEmpty()) return "";
|
||||||
|
return html.replaceAll("<[^>]*>", "").replaceAll("\\s{2,}", " ").strip();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Detects MIME type from the first characters of a base64-encoded image. */
|
||||||
|
private static String detectImageMime(String base64) {
|
||||||
|
if (base64.startsWith("/9j/")) return "image/jpeg";
|
||||||
|
if (base64.startsWith("iVBOR")) return "image/png";
|
||||||
|
if (base64.startsWith("R0lGO")) return "image/gif";
|
||||||
|
if (base64.startsWith("UklGR")) return "image/webp";
|
||||||
|
return "image/png"; // safe fallback
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Null-safe string extraction from a JsonNode (Jackson 3: stringValue() returns null for non-strings). */
|
||||||
|
private static String str(JsonNode node) {
|
||||||
|
String v = node.stringValue();
|
||||||
|
return v != null ? v : "";
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
package com.aiteacher.document;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Internal DTO produced by MarkerPageParser for one PDF page.
|
||||||
|
* Decouples the Marker HTTP API from downstream services.
|
||||||
|
*/
|
||||||
|
public record PageResult(
|
||||||
|
int pageNumber, // 1-based, derived from Marker page block index
|
||||||
|
String orderedText, // full page text in correct reading order (blocks joined by \n\n)
|
||||||
|
String headingTitle, // first SectionHeader block on page, or null
|
||||||
|
List<FigureData> figures // extracted figure images (may be empty)
|
||||||
|
) {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A figure extracted from the page.
|
||||||
|
* Image bytes are PNG data decoded from the Marker JSON {@code images} map.
|
||||||
|
*/
|
||||||
|
public record FigureData(
|
||||||
|
byte[] imageBytes, // PNG image data (base64-decoded from Marker response)
|
||||||
|
String nearestCaption, // text of the adjacent Caption block, or null
|
||||||
|
String blockId // Marker block ID (e.g. "/page/0/Figure/2") for traceability
|
||||||
|
) {}
|
||||||
|
}
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
package com.aiteacher.document;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Result of a full Marker parse: structured page data (from JSON) plus
|
||||||
|
* native per-page markdown (from the separate Markdown API call).
|
||||||
|
*
|
||||||
|
* @param pages one entry per non-empty page, derived from the chunks response
|
||||||
|
* @param htmlByPage concatenated block HTML keyed by 1-based page number
|
||||||
|
*/
|
||||||
|
public record ParsedBook(
|
||||||
|
List<PageResult> pages,
|
||||||
|
Map<Integer, String> htmlByPage
|
||||||
|
) {}
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
package com.aiteacher.document;
|
||||||
|
|
||||||
|
import org.apache.pdfbox.io.RandomAccessReadBufferedFile;
|
||||||
|
import org.apache.pdfbox.multipdf.Splitter;
|
||||||
|
import org.apache.pdfbox.pdfparser.PDFParser;
|
||||||
|
import org.apache.pdfbox.pdmodel.PDDocument;
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Splits a PDF file into fixed-size chunks using PDFBox.
|
||||||
|
* Each chunk is saved as a temporary file so it can be submitted independently to Marker.
|
||||||
|
*/
|
||||||
|
@Service
|
||||||
|
public class PdfSplitterService {
|
||||||
|
|
||||||
|
private static final Logger log = LoggerFactory.getLogger(PdfSplitterService.class);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A chunk of a split PDF.
|
||||||
|
*
|
||||||
|
* @param tempFile path to the temporary PDF file (caller must delete when done)
|
||||||
|
* @param pageOffset 0-based index of the first page in this chunk within the original document
|
||||||
|
*/
|
||||||
|
public record PdfChunk(Path tempFile, int pageOffset) {}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Splits {@code pdfPath} into chunks of at most {@code maxPagesPerChunk} pages.
|
||||||
|
* Returns a single-element list when the document fits in one chunk.
|
||||||
|
*
|
||||||
|
* @param pdfPath source PDF
|
||||||
|
* @param maxPagesPerChunk maximum pages per chunk
|
||||||
|
* @return ordered list of chunks; caller is responsible for deleting {@code tempFile}s
|
||||||
|
*/
|
||||||
|
public List<PdfChunk> split(Path pdfPath, int maxPagesPerChunk) throws IOException {
|
||||||
|
try (PDDocument doc = new PDFParser(new RandomAccessReadBufferedFile(pdfPath.toFile())).parse()) {
|
||||||
|
int totalPages = doc.getNumberOfPages();
|
||||||
|
log.info("PDF {} has {} pages, splitting into chunks of {}", pdfPath.getFileName(), totalPages, maxPagesPerChunk);
|
||||||
|
|
||||||
|
if (totalPages <= maxPagesPerChunk) {
|
||||||
|
// No split needed — return the original file as a single virtual chunk
|
||||||
|
return List.of(new PdfChunk(pdfPath, 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
Splitter splitter = new Splitter();
|
||||||
|
splitter.setSplitAtPage(maxPagesPerChunk);
|
||||||
|
List<PDDocument> parts = splitter.split(doc);
|
||||||
|
|
||||||
|
List<PdfChunk> chunks = new ArrayList<>(parts.size());
|
||||||
|
int offset = 0;
|
||||||
|
for (PDDocument part : parts) {
|
||||||
|
try {
|
||||||
|
Path tmp = Files.createTempFile("marker-chunk-", ".pdf");
|
||||||
|
part.save(tmp.toFile());
|
||||||
|
chunks.add(new PdfChunk(tmp, offset));
|
||||||
|
log.debug("Created chunk at {} (page offset {})", tmp, offset);
|
||||||
|
offset += part.getNumberOfPages();
|
||||||
|
} finally {
|
||||||
|
part.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return chunks;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,13 +1,17 @@
|
|||||||
package com.aiteacher.document;
|
package com.aiteacher.document;
|
||||||
|
|
||||||
|
import org.apache.pdfbox.Loader;
|
||||||
|
import org.apache.pdfbox.pdmodel.PDDocument;
|
||||||
|
import org.apache.pdfbox.pdmodel.PDPage;
|
||||||
|
import org.apache.pdfbox.pdmodel.common.PDRectangle;
|
||||||
|
import org.apache.pdfbox.text.PDFTextStripperByArea;
|
||||||
import org.slf4j.Logger;
|
import org.slf4j.Logger;
|
||||||
import org.slf4j.LoggerFactory;
|
import org.slf4j.LoggerFactory;
|
||||||
import org.springframework.ai.reader.pdf.PagePdfDocumentReader;
|
|
||||||
import org.springframework.ai.reader.pdf.config.PdfDocumentReaderConfig;
|
|
||||||
import org.springframework.core.io.FileSystemResource;
|
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
import org.springframework.transaction.annotation.Transactional;
|
import org.springframework.transaction.annotation.Transactional;
|
||||||
|
|
||||||
|
import java.awt.Rectangle;
|
||||||
|
import java.io.IOException;
|
||||||
import java.nio.file.Path;
|
import java.nio.file.Path;
|
||||||
import java.util.ArrayList;
|
import java.util.ArrayList;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
@@ -15,13 +19,18 @@ import java.util.UUID;
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Parses a PDF into page-level SectionEntity records stored in Postgres.
|
* Parses a PDF into page-level SectionEntity records stored in Postgres.
|
||||||
* Each page becomes one section, grouped under a single chapter per book.
|
* Uses column-aware extraction via PDFTextStripperByArea: for two-column pages,
|
||||||
|
* left column is extracted first then right, preserving correct reading order.
|
||||||
|
* Text is also normalized (collapsed whitespace) before storage.
|
||||||
*/
|
*/
|
||||||
@Service
|
@Service
|
||||||
public class PdfStructureParser {
|
public class PdfStructureParser {
|
||||||
|
|
||||||
private static final Logger log = LoggerFactory.getLogger(PdfStructureParser.class);
|
private static final Logger log = LoggerFactory.getLogger(PdfStructureParser.class);
|
||||||
|
|
||||||
|
// Right column is considered empty (single-column page) if it has < 20% of left column's content
|
||||||
|
private static final double TWO_COLUMN_THRESHOLD = 0.2;
|
||||||
|
|
||||||
private final ChapterRepository chapterRepository;
|
private final ChapterRepository chapterRepository;
|
||||||
private final SectionRepository sectionRepository;
|
private final SectionRepository sectionRepository;
|
||||||
|
|
||||||
@@ -35,37 +44,71 @@ public class PdfStructureParser {
|
|||||||
public List<SectionEntity> parse(UUID bookId, String bookTitle, Path pdfPath) {
|
public List<SectionEntity> parse(UUID bookId, String bookTitle, Path pdfPath) {
|
||||||
log.info("Parsing PDF structure for book {}", bookId);
|
log.info("Parsing PDF structure for book {}", bookId);
|
||||||
|
|
||||||
// One chapter per book
|
|
||||||
String chapterId = bookId + "-ch1";
|
String chapterId = bookId + "-ch1";
|
||||||
ChapterEntity chapter = new ChapterEntity(chapterId, bookId, 1, bookTitle, 1);
|
ChapterEntity chapter = new ChapterEntity(chapterId, bookId, 1, bookTitle, 1);
|
||||||
chapterRepository.save(chapter);
|
chapterRepository.save(chapter);
|
||||||
|
|
||||||
// One section per page
|
|
||||||
PagePdfDocumentReader reader = new PagePdfDocumentReader(
|
|
||||||
new FileSystemResource(pdfPath.toFile()),
|
|
||||||
PdfDocumentReaderConfig.builder().withPagesPerDocument(1).build()
|
|
||||||
);
|
|
||||||
|
|
||||||
List<org.springframework.ai.document.Document> pages = reader.get();
|
|
||||||
List<SectionEntity> sections = new ArrayList<>();
|
List<SectionEntity> sections = new ArrayList<>();
|
||||||
|
|
||||||
for (int i = 0; i < pages.size(); i++) {
|
try (PDDocument doc = Loader.loadPDF(pdfPath.toFile())) {
|
||||||
int pageNum = i + 1;
|
List<PDPage> pages = new ArrayList<>();
|
||||||
String text = pages.get(i).getText();
|
doc.getPages().forEach(pages::add);
|
||||||
if (text == null || text.isBlank()) continue;
|
|
||||||
|
|
||||||
String sectionId = bookId + "-p" + pageNum;
|
for (int i = 0; i < 25; i++) {
|
||||||
SectionEntity section = new SectionEntity(
|
int pageNum = i + 1;
|
||||||
sectionId, chapterId, bookId,
|
String text = normalizeWhitespace(extractPageText(pages.get(i)));
|
||||||
String.valueOf(pageNum),
|
if (text.isBlank()) continue;
|
||||||
"Page " + pageNum,
|
|
||||||
pageNum, pageNum,
|
String sectionId = bookId + "-p" + pageNum;
|
||||||
text
|
SectionEntity section = new SectionEntity(
|
||||||
);
|
sectionId, chapterId, bookId,
|
||||||
sections.add(sectionRepository.save(section));
|
String.valueOf(pageNum),
|
||||||
|
"Page " + pageNum,
|
||||||
|
pageNum, pageNum,
|
||||||
|
text
|
||||||
|
);
|
||||||
|
sections.add(sectionRepository.save(section));
|
||||||
|
}
|
||||||
|
} catch (IOException e) {
|
||||||
|
throw new RuntimeException("Failed to parse PDF for book " + bookId, e);
|
||||||
}
|
}
|
||||||
|
|
||||||
log.info("Parsed {} sections for book {}", sections.size(), bookId);
|
log.info("Parsed {} sections for book {}", sections.size(), bookId);
|
||||||
return sections;
|
return sections;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Extracts text from a single page using column-aware region extraction.
|
||||||
|
* Splits the page at the horizontal midpoint. If the right region has fewer
|
||||||
|
* than 20% of the characters of the left region, treats the page as single-column.
|
||||||
|
*/
|
||||||
|
private String extractPageText(PDPage page) throws IOException {
|
||||||
|
PDRectangle mediaBox = page.getMediaBox();
|
||||||
|
int width = (int) mediaBox.getWidth();
|
||||||
|
int height = (int) mediaBox.getHeight();
|
||||||
|
int mid = width / 2;
|
||||||
|
|
||||||
|
PDFTextStripperByArea stripper = new PDFTextStripperByArea();
|
||||||
|
stripper.setSortByPosition(true);
|
||||||
|
stripper.addRegion("left", new Rectangle(0, 0, mid, height));
|
||||||
|
stripper.addRegion("right", new Rectangle(mid, 0, width - mid, height));
|
||||||
|
stripper.extractRegions(page);
|
||||||
|
|
||||||
|
String left = stripper.getTextForRegion("left").strip();
|
||||||
|
String right = stripper.getTextForRegion("right").strip();
|
||||||
|
|
||||||
|
if (right.length() < left.length() * TWO_COLUMN_THRESHOLD) {
|
||||||
|
// Single-column page — left holds all (or nearly all) content
|
||||||
|
return left.isEmpty() ? right : left;
|
||||||
|
}
|
||||||
|
return left + "\n\n" + right;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Collapses multi-space/tab runs and excessive blank lines. */
|
||||||
|
private String normalizeWhitespace(String text) {
|
||||||
|
return text
|
||||||
|
.replaceAll("[ \t]{2,}", " ")
|
||||||
|
.replaceAll("\n{3,}", "\n\n")
|
||||||
|
.trim();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,97 @@
|
|||||||
|
package com.aiteacher.document;
|
||||||
|
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
import org.springframework.beans.factory.annotation.Value;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
|
||||||
|
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
|
||||||
|
import software.amazon.awssdk.core.sync.RequestBody;
|
||||||
|
import software.amazon.awssdk.regions.Region;
|
||||||
|
import software.amazon.awssdk.services.s3.S3Client;
|
||||||
|
import software.amazon.awssdk.services.s3.S3Configuration;
|
||||||
|
import software.amazon.awssdk.services.s3.model.*;
|
||||||
|
|
||||||
|
import java.net.URI;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.UUID;
|
||||||
|
|
||||||
|
@Service
|
||||||
|
public class S3MarkdownStorageService implements MarkdownStorageService {
|
||||||
|
|
||||||
|
private static final Logger log = LoggerFactory.getLogger(S3MarkdownStorageService.class);
|
||||||
|
|
||||||
|
private final S3Client s3;
|
||||||
|
private final String bucket;
|
||||||
|
|
||||||
|
public S3MarkdownStorageService(
|
||||||
|
@Value("${app.figure-storage.endpoint}") String endpoint,
|
||||||
|
@Value("${app.figure-storage.region}") String region,
|
||||||
|
@Value("${app.figure-storage.bucket}") String bucket,
|
||||||
|
@Value("${app.figure-storage.access-key-id}") String accessKeyId,
|
||||||
|
@Value("${app.figure-storage.secret-access-key}") String secretKey) {
|
||||||
|
this.bucket = bucket;
|
||||||
|
URI endpointUri = URI.create(endpoint);
|
||||||
|
StaticCredentialsProvider credentials = StaticCredentialsProvider.create(
|
||||||
|
AwsBasicCredentials.create(accessKeyId, secretKey));
|
||||||
|
Region awsRegion = Region.of(region);
|
||||||
|
S3Configuration s3Config = S3Configuration.builder().pathStyleAccessEnabled(true).build();
|
||||||
|
|
||||||
|
this.s3 = S3Client.builder()
|
||||||
|
.endpointOverride(endpointUri)
|
||||||
|
.region(awsRegion)
|
||||||
|
.credentialsProvider(credentials)
|
||||||
|
.serviceConfiguration(s3Config)
|
||||||
|
.build();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public String save(UUID bookId, int pageNumber, String markdown) {
|
||||||
|
String key = key(bookId, pageNumber);
|
||||||
|
byte[] bytes = markdown.getBytes(StandardCharsets.UTF_8);
|
||||||
|
s3.putObject(
|
||||||
|
PutObjectRequest.builder().bucket(bucket).key(key)
|
||||||
|
.contentType("text/html; charset=utf-8")
|
||||||
|
.contentLength((long) bytes.length).build(),
|
||||||
|
RequestBody.fromBytes(bytes));
|
||||||
|
return key;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public String getText(UUID bookId, int pageNumber) {
|
||||||
|
byte[] bytes = s3.getObjectAsBytes(
|
||||||
|
GetObjectRequest.builder().bucket(bucket).key(key(bookId, pageNumber)).build()
|
||||||
|
).asByteArray();
|
||||||
|
return new String(bytes, StandardCharsets.UTF_8);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void deleteAll(UUID bookId) {
|
||||||
|
String prefix = "html/" + bookId + "/";
|
||||||
|
try {
|
||||||
|
List<ObjectIdentifier> toDelete = new ArrayList<>();
|
||||||
|
s3.listObjectsV2Paginator(ListObjectsV2Request.builder()
|
||||||
|
.bucket(bucket).prefix(prefix).build()).stream()
|
||||||
|
.flatMap(page -> page.contents().stream())
|
||||||
|
.map(S3Object::key)
|
||||||
|
.map(k -> ObjectIdentifier.builder().key(k).build())
|
||||||
|
.forEach(toDelete::add);
|
||||||
|
|
||||||
|
if (toDelete.isEmpty()) return;
|
||||||
|
|
||||||
|
s3.deleteObjects(DeleteObjectsRequest.builder()
|
||||||
|
.bucket(bucket)
|
||||||
|
.delete(Delete.builder().objects(toDelete).build())
|
||||||
|
.build());
|
||||||
|
log.info("Deleted {} markdown files from S3 for book {}", toDelete.size(), bookId);
|
||||||
|
} catch (S3Exception ex) {
|
||||||
|
log.warn("Could not fully delete markdown for book {} from S3: {}", bookId, ex.getMessage());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static String key(UUID bookId, int pageNumber) {
|
||||||
|
return "html/" + bookId + "/page-" + pageNumber + ".html";
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,6 +1,8 @@
|
|||||||
package com.aiteacher.document;
|
package com.aiteacher.document;
|
||||||
|
|
||||||
import org.springframework.data.jpa.repository.JpaRepository;
|
import org.springframework.data.jpa.repository.JpaRepository;
|
||||||
|
import org.springframework.data.jpa.repository.Query;
|
||||||
|
import org.springframework.data.repository.query.Param;
|
||||||
|
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.UUID;
|
import java.util.UUID;
|
||||||
@@ -8,4 +10,10 @@ import java.util.UUID;
|
|||||||
public interface SectionRepository extends JpaRepository<SectionEntity, String> {
|
public interface SectionRepository extends JpaRepository<SectionEntity, String> {
|
||||||
List<SectionEntity> findAllByBookId(UUID bookId);
|
List<SectionEntity> findAllByBookId(UUID bookId);
|
||||||
void deleteAllByBookId(UUID bookId);
|
void deleteAllByBookId(UUID bookId);
|
||||||
|
|
||||||
|
@Query("SELECT s FROM SectionEntity s WHERE s.bookId = :bookId AND s.pageStart <= :windowEnd AND s.pageEnd >= :windowStart ORDER BY s.pageStart")
|
||||||
|
List<SectionEntity> findByBookIdAndPageOverlap(
|
||||||
|
@Param("bookId") UUID bookId,
|
||||||
|
@Param("windowStart") int windowStart,
|
||||||
|
@Param("windowEnd") int windowEnd);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -38,14 +38,52 @@ public class TextChunkingService {
|
|||||||
List<String> windows = new ArrayList<>();
|
List<String> windows = new ArrayList<>();
|
||||||
int start = 0;
|
int start = 0;
|
||||||
while (start < text.length()) {
|
while (start < text.length()) {
|
||||||
int end = Math.min(start + TARGET_CHARS, text.length());
|
int hardEnd = Math.min(start + TARGET_CHARS, text.length());
|
||||||
windows.add(text.substring(start, end));
|
if (hardEnd == text.length()) {
|
||||||
if (end == text.length()) break;
|
String last = text.substring(start).strip();
|
||||||
start = end - OVERLAP_CHARS;
|
if (!last.isEmpty()) windows.add(last);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
int splitAt = findSplitPoint(text, start, hardEnd);
|
||||||
|
String chunk = text.substring(start, splitAt).strip();
|
||||||
|
if (!chunk.isEmpty()) windows.add(chunk);
|
||||||
|
// Overlap: back up from split point, align to a word start
|
||||||
|
int overlapStart = Math.max(start + 1, splitAt - OVERLAP_CHARS);
|
||||||
|
while (overlapStart < splitAt && text.charAt(overlapStart) != ' ') overlapStart++;
|
||||||
|
start = overlapStart < splitAt ? overlapStart + 1 : splitAt;
|
||||||
}
|
}
|
||||||
return windows;
|
return windows;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Finds the best split point at or before hardEnd, preferring (in order):
|
||||||
|
* paragraph boundary, sentence boundary, word boundary, hard cut.
|
||||||
|
*/
|
||||||
|
private int findSplitPoint(String text, int start, int hardEnd) {
|
||||||
|
int lookback = Math.min(400, (hardEnd - start) / 2);
|
||||||
|
|
||||||
|
// 1. Paragraph boundary
|
||||||
|
int paraIdx = text.lastIndexOf("\n\n", hardEnd);
|
||||||
|
if (paraIdx > hardEnd - lookback && paraIdx > start) return paraIdx + 2;
|
||||||
|
|
||||||
|
// 2. Sentence boundary (. ! ?) followed by space or newline
|
||||||
|
for (int i = hardEnd - 1; i > hardEnd - lookback && i > start; i--) {
|
||||||
|
char c = text.charAt(i);
|
||||||
|
if ((c == '.' || c == '!' || c == '?') && i + 1 < text.length()) {
|
||||||
|
char next = text.charAt(i + 1);
|
||||||
|
if (next == ' ' || next == '\n') return i + 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3. Word boundary
|
||||||
|
for (int i = hardEnd - 1; i > hardEnd - 100 && i > start; i--) {
|
||||||
|
if (text.charAt(i) == ' ') return i + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4. Hard cut
|
||||||
|
return hardEnd;
|
||||||
|
}
|
||||||
|
|
||||||
private Map<String, Object> buildMetadata(SectionEntity section, String bookTitle,
|
private Map<String, Object> buildMetadata(SectionEntity section, String bookTitle,
|
||||||
int index, int total, String chunkId) {
|
int index, int total, String chunkId) {
|
||||||
Map<String, Object> m = new HashMap<>();
|
Map<String, Object> m = new HashMap<>();
|
||||||
|
|||||||
@@ -3,47 +3,106 @@ package com.aiteacher.document;
|
|||||||
import org.slf4j.Logger;
|
import org.slf4j.Logger;
|
||||||
import org.slf4j.LoggerFactory;
|
import org.slf4j.LoggerFactory;
|
||||||
import org.springframework.ai.chat.client.ChatClient;
|
import org.springframework.ai.chat.client.ChatClient;
|
||||||
import org.springframework.core.io.FileSystemResource;
|
import org.springframework.beans.factory.annotation.Value;
|
||||||
|
import org.springframework.core.io.ByteArrayResource;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
import org.springframework.util.MimeTypeUtils;
|
import org.springframework.util.MimeTypeUtils;
|
||||||
|
|
||||||
import java.nio.file.Path;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Generates a clinical text description for an extracted figure image
|
* Analyses an extracted figure image using the OpenAI vision model.
|
||||||
* using the OpenAI vision model via Spring AI ChatClient.
|
*
|
||||||
|
* <p>Returns an {@link ImageAnalysis} record containing:
|
||||||
|
* <ul>
|
||||||
|
* <li>{@code description} — 2-3 sentence clinical description of the image</li>
|
||||||
|
* <li>{@code imageText} — all visible text, labels, and annotations copied verbatim
|
||||||
|
* from the image (empty string when none present)</li>
|
||||||
|
* </ul>
|
||||||
|
*
|
||||||
|
* <p>Both fields are stored: {@code description} drives the embedding; {@code imageText}
|
||||||
|
* is added to chunk metadata so queries can match exact labels (e.g., "Circle of Willis").
|
||||||
*/
|
*/
|
||||||
@Service
|
@Service
|
||||||
public class VisionDescriptionService {
|
public class VisionDescriptionService {
|
||||||
|
|
||||||
private static final Logger log = LoggerFactory.getLogger(VisionDescriptionService.class);
|
private static final Logger log = LoggerFactory.getLogger(VisionDescriptionService.class);
|
||||||
|
|
||||||
private static final String PROMPT =
|
private static final String PROMPT = """
|
||||||
"You are a neurosurgery educator. Provide a brief 2-3 sentence clinical description of " +
|
You are a neurosurgery educator analysing a medical image.
|
||||||
"this image. Focus on anatomical structures, surgical landmarks, labels, and clinical " +
|
Respond in EXACTLY this format — no other text, no markdown:
|
||||||
"significance. If text or labels are visible, include them verbatim.";
|
DESCRIPTION: <2-3 sentence clinical description focusing on anatomical structures, surgical landmarks, and clinical significance>
|
||||||
|
IMAGE_TEXT: <all visible text, labels, measurements, and annotations copied verbatim, comma-separated; write NONE if no text visible>
|
||||||
|
""";
|
||||||
|
|
||||||
|
/** Minimum ms between vision API calls. Configurable via app.vision.min-interval-ms. */
|
||||||
|
private final long minIntervalMs;
|
||||||
private final ChatClient chatClient;
|
private final ChatClient chatClient;
|
||||||
|
private volatile long lastCallAt = 0;
|
||||||
|
|
||||||
public VisionDescriptionService(ChatClient chatClient) {
|
public VisionDescriptionService(
|
||||||
|
ChatClient chatClient,
|
||||||
|
@Value("${app.vision.min-interval-ms:2000}") long minIntervalMs) {
|
||||||
this.chatClient = chatClient;
|
this.chatClient = chatClient;
|
||||||
|
this.minIntervalMs = minIntervalMs;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Returns a description string. Falls back to the provided caption if vision fails.
|
* Holds the structured output of a vision model call on one figure image.
|
||||||
|
*
|
||||||
|
* @param description clinical description of the image content
|
||||||
|
* @param imageText verbatim text visible inside the image; empty string if none
|
||||||
*/
|
*/
|
||||||
public String describe(Path imagePath, String captionFallback) {
|
public record ImageAnalysis(String description, String imageText) {}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Analyses the image bytes and returns an {@link ImageAnalysis}.
|
||||||
|
* Falls back gracefully: if the vision call fails, the caption is used as description
|
||||||
|
* and imageText is left empty.
|
||||||
|
*
|
||||||
|
* @param imageBytes PNG bytes of the extracted figure
|
||||||
|
* @param captionFallback caption detected from surrounding text, may be null
|
||||||
|
*/
|
||||||
|
public ImageAnalysis analyze(byte[] imageBytes, String captionFallback) {
|
||||||
|
throttle();
|
||||||
try {
|
try {
|
||||||
return chatClient.prompt()
|
String raw = chatClient.prompt()
|
||||||
.user(u -> u
|
.user(u -> u
|
||||||
.text(PROMPT)
|
.text(PROMPT)
|
||||||
.media(MimeTypeUtils.IMAGE_PNG, new FileSystemResource(imagePath.toFile())))
|
.media(MimeTypeUtils.IMAGE_PNG, new ByteArrayResource(imageBytes)))
|
||||||
.call()
|
.call()
|
||||||
.content();
|
.content();
|
||||||
|
return parse(raw, captionFallback);
|
||||||
} catch (Exception ex) {
|
} catch (Exception ex) {
|
||||||
log.warn("Vision description failed for {}: {} — using caption as fallback",
|
log.warn("Vision analysis failed: {} — using caption as fallback", ex.getMessage());
|
||||||
imagePath.getFileName(), ex.getMessage());
|
return new ImageAnalysis(
|
||||||
return captionFallback != null ? captionFallback : "Figure";
|
captionFallback != null ? captionFallback : "Figure",
|
||||||
|
"");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private synchronized void throttle() {
|
||||||
|
long now = System.currentTimeMillis();
|
||||||
|
long wait = minIntervalMs - (now - lastCallAt);
|
||||||
|
if (wait > 0) {
|
||||||
|
try { Thread.sleep(wait); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
|
||||||
|
}
|
||||||
|
lastCallAt = System.currentTimeMillis();
|
||||||
|
}
|
||||||
|
|
||||||
|
private ImageAnalysis parse(String raw, String captionFallback) {
|
||||||
|
String description = captionFallback != null ? captionFallback : "Figure";
|
||||||
|
String imageText = "";
|
||||||
|
|
||||||
|
if (raw != null) {
|
||||||
|
for (String line : raw.split("\n")) {
|
||||||
|
if (line.startsWith("DESCRIPTION:")) {
|
||||||
|
String val = line.substring("DESCRIPTION:".length()).strip();
|
||||||
|
if (!val.isEmpty()) description = val;
|
||||||
|
} else if (line.startsWith("IMAGE_TEXT:")) {
|
||||||
|
String val = line.substring("IMAGE_TEXT:".length()).strip();
|
||||||
|
if (!val.isEmpty() && !"NONE".equalsIgnoreCase(val)) imageText = val;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return new ImageAnalysis(description, imageText);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,24 +1,27 @@
|
|||||||
package com.aiteacher.figure;
|
package com.aiteacher.figure;
|
||||||
|
|
||||||
import java.awt.image.BufferedImage;
|
import java.awt.image.BufferedImage;
|
||||||
import java.nio.file.Path;
|
|
||||||
import java.util.UUID;
|
import java.util.UUID;
|
||||||
|
|
||||||
public interface FigureStorageService {
|
public interface FigureStorageService {
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Saves an extracted image to the figure store and returns the relative path
|
* Saves an extracted image to S3 and returns the object key stored in the database.
|
||||||
* (relative to the configured base-path) stored in the database.
|
|
||||||
*/
|
*/
|
||||||
String save(UUID bookId, String figureId, BufferedImage image);
|
String save(UUID bookId, String figureId, BufferedImage image);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Resolves a stored relative path to an absolute filesystem path.
|
* Downloads the image bytes for the given S3 object key.
|
||||||
*/
|
*/
|
||||||
Path resolve(String relativePath);
|
byte[] getBytes(String key);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Deletes all figure files for the given book.
|
* Returns a presigned GET URL valid for 1 hour for the given S3 object key.
|
||||||
|
*/
|
||||||
|
String presignedUrl(String key);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Deletes all figure objects for the given book.
|
||||||
*/
|
*/
|
||||||
void deleteAll(UUID bookId);
|
void deleteAll(UUID bookId);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,59 +0,0 @@
|
|||||||
package com.aiteacher.figure;
|
|
||||||
|
|
||||||
import org.slf4j.Logger;
|
|
||||||
import org.slf4j.LoggerFactory;
|
|
||||||
import org.springframework.beans.factory.annotation.Value;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
import javax.imageio.ImageIO;
|
|
||||||
import java.awt.image.BufferedImage;
|
|
||||||
import java.io.IOException;
|
|
||||||
import java.nio.file.Files;
|
|
||||||
import java.nio.file.Path;
|
|
||||||
import java.nio.file.Paths;
|
|
||||||
import java.util.UUID;
|
|
||||||
|
|
||||||
@Service
|
|
||||||
public class LocalFigureStorageService implements FigureStorageService {
|
|
||||||
|
|
||||||
private static final Logger log = LoggerFactory.getLogger(LocalFigureStorageService.class);
|
|
||||||
|
|
||||||
private final Path basePath;
|
|
||||||
|
|
||||||
public LocalFigureStorageService(@Value("${app.figure-storage.base-path:./uploads}") String basePath) {
|
|
||||||
this.basePath = Paths.get(basePath).toAbsolutePath().normalize();
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public String save(UUID bookId, String figureId, BufferedImage image) {
|
|
||||||
try {
|
|
||||||
Path dir = basePath.resolve("figures").resolve(bookId.toString());
|
|
||||||
Files.createDirectories(dir);
|
|
||||||
String filename = figureId + ".png";
|
|
||||||
Path file = dir.resolve(filename);
|
|
||||||
ImageIO.write(image, "PNG", file.toFile());
|
|
||||||
// Return relative path for storage in DB
|
|
||||||
return "figures/" + bookId + "/" + filename;
|
|
||||||
} catch (IOException ex) {
|
|
||||||
throw new RuntimeException("Failed to save figure " + figureId, ex);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public Path resolve(String relativePath) {
|
|
||||||
return basePath.resolve(relativePath);
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public void deleteAll(UUID bookId) {
|
|
||||||
Path dir = basePath.resolve("figures").resolve(bookId.toString());
|
|
||||||
if (!Files.exists(dir)) return;
|
|
||||||
try (var walk = Files.walk(dir)) {
|
|
||||||
walk.sorted(java.util.Comparator.reverseOrder())
|
|
||||||
.map(Path::toFile)
|
|
||||||
.forEach(java.io.File::delete);
|
|
||||||
} catch (IOException ex) {
|
|
||||||
log.warn("Could not fully delete figures for book {}: {}", bookId, ex.getMessage());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,132 @@
|
|||||||
|
package com.aiteacher.figure;
|
||||||
|
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
import org.springframework.beans.factory.annotation.Value;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
|
||||||
|
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
|
||||||
|
import software.amazon.awssdk.core.sync.RequestBody;
|
||||||
|
import software.amazon.awssdk.regions.Region;
|
||||||
|
import software.amazon.awssdk.services.s3.S3Client;
|
||||||
|
import software.amazon.awssdk.services.s3.S3Configuration;
|
||||||
|
import software.amazon.awssdk.services.s3.model.*;
|
||||||
|
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
|
||||||
|
import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
|
||||||
|
import software.amazon.awssdk.services.s3.model.S3Object;
|
||||||
|
|
||||||
|
import javax.imageio.ImageIO;
|
||||||
|
import java.awt.image.BufferedImage;
|
||||||
|
import java.io.ByteArrayOutputStream;
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.net.URI;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.UUID;
|
||||||
|
|
||||||
|
@Service
|
||||||
|
public class S3FigureStorageService implements FigureStorageService {
|
||||||
|
|
||||||
|
private static final Logger log = LoggerFactory.getLogger(S3FigureStorageService.class);
|
||||||
|
|
||||||
|
private final S3Client s3;
|
||||||
|
private final S3Presigner presigner;
|
||||||
|
private final String bucket;
|
||||||
|
|
||||||
|
public S3FigureStorageService(
|
||||||
|
@Value("${app.figure-storage.endpoint}") String endpoint,
|
||||||
|
@Value("${app.figure-storage.region}") String region,
|
||||||
|
@Value("${app.figure-storage.bucket}") String bucket,
|
||||||
|
@Value("${app.figure-storage.access-key-id}") String accessKeyId,
|
||||||
|
@Value("${app.figure-storage.secret-access-key}") String secretKey) {
|
||||||
|
this.bucket = bucket;
|
||||||
|
URI endpointUri = URI.create(endpoint);
|
||||||
|
StaticCredentialsProvider credentials = StaticCredentialsProvider.create(
|
||||||
|
AwsBasicCredentials.create(accessKeyId, secretKey));
|
||||||
|
Region awsRegion = Region.of(region);
|
||||||
|
|
||||||
|
S3Configuration s3Config = S3Configuration.builder()
|
||||||
|
.pathStyleAccessEnabled(true)
|
||||||
|
.build();
|
||||||
|
|
||||||
|
this.s3 = S3Client.builder()
|
||||||
|
.endpointOverride(endpointUri)
|
||||||
|
.region(awsRegion)
|
||||||
|
.credentialsProvider(credentials)
|
||||||
|
.serviceConfiguration(s3Config)
|
||||||
|
.build();
|
||||||
|
|
||||||
|
this.presigner = S3Presigner.builder()
|
||||||
|
.endpointOverride(endpointUri)
|
||||||
|
.region(awsRegion)
|
||||||
|
.credentialsProvider(credentials)
|
||||||
|
.serviceConfiguration(s3Config)
|
||||||
|
.build();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public String save(UUID bookId, String figureId, BufferedImage image) {
|
||||||
|
String key = "figures/" + bookId + "/" + figureId + ".png";
|
||||||
|
try {
|
||||||
|
ByteArrayOutputStream out = new ByteArrayOutputStream();
|
||||||
|
ImageIO.write(image, "PNG", out);
|
||||||
|
byte[] bytes = out.toByteArray();
|
||||||
|
|
||||||
|
s3.putObject(
|
||||||
|
PutObjectRequest.builder().bucket(bucket).key(key)
|
||||||
|
.contentType("image/png").contentLength((long) bytes.length).build(),
|
||||||
|
RequestBody.fromBytes(bytes));
|
||||||
|
return key;
|
||||||
|
} catch (IOException ex) {
|
||||||
|
throw new RuntimeException("Failed to encode figure " + figureId, ex);
|
||||||
|
} catch (S3Exception ex) {
|
||||||
|
throw new RuntimeException("Failed to upload figure " + figureId + " to S3", ex);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public byte[] getBytes(String key) {
|
||||||
|
try {
|
||||||
|
return s3.getObjectAsBytes(
|
||||||
|
GetObjectRequest.builder().bucket(bucket).key(key).build()).asByteArray();
|
||||||
|
} catch (S3Exception ex) {
|
||||||
|
throw new RuntimeException("Failed to download figure from S3: " + key, ex);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public String presignedUrl(String key) {
|
||||||
|
GetObjectPresignRequest request = GetObjectPresignRequest.builder()
|
||||||
|
.signatureDuration(Duration.ofHours(1))
|
||||||
|
.getObjectRequest(r -> r.bucket(bucket).key(key))
|
||||||
|
.build();
|
||||||
|
return presigner.presignGetObject(request).url().toString();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void deleteAll(UUID bookId) {
|
||||||
|
String prefix = "figures/" + bookId + "/";
|
||||||
|
try {
|
||||||
|
List<ObjectIdentifier> toDelete = new ArrayList<>();
|
||||||
|
ListObjectsV2Request listRequest = ListObjectsV2Request.builder()
|
||||||
|
.bucket(bucket).prefix(prefix).build();
|
||||||
|
|
||||||
|
s3.listObjectsV2Paginator(listRequest).stream()
|
||||||
|
.flatMap(page -> page.contents().stream())
|
||||||
|
.map(S3Object::key)
|
||||||
|
.map(k -> ObjectIdentifier.builder().key(k).build())
|
||||||
|
.forEach(toDelete::add);
|
||||||
|
|
||||||
|
if (toDelete.isEmpty()) return;
|
||||||
|
|
||||||
|
s3.deleteObjects(DeleteObjectsRequest.builder()
|
||||||
|
.bucket(bucket)
|
||||||
|
.delete(Delete.builder().objects(toDelete).build())
|
||||||
|
.build());
|
||||||
|
log.info("Deleted {} figures from S3 for book {}", toDelete.size(), bookId);
|
||||||
|
} catch (S3Exception ex) {
|
||||||
|
log.warn("Could not fully delete figures for book {} from S3: {}", bookId, ex.getMessage());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
package com.aiteacher.retrieval;
|
||||||
|
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Set;
|
||||||
|
import java.util.regex.Matcher;
|
||||||
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Post-processes generated answers to strip citation labels that do not
|
||||||
|
* correspond to any passage retrieved for the current query, preventing
|
||||||
|
* hallucinated source references from reaching the user.
|
||||||
|
*/
|
||||||
|
@Service
|
||||||
|
public class CitationValidatorService {
|
||||||
|
|
||||||
|
private static final Logger log = LoggerFactory.getLogger(CitationValidatorService.class);
|
||||||
|
|
||||||
|
/** Matches citation labels of the form [S1], [F2], [S12], etc. */
|
||||||
|
private static final Pattern CITATION_PATTERN = Pattern.compile("\\[(S|F)\\d+\\]");
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Removes any {@code [Sx]} / {@code [Fx]} citation in {@code generatedAnswer}
|
||||||
|
* whose label is not contained in {@code validLabels}.
|
||||||
|
*
|
||||||
|
* @param generatedAnswer raw model output
|
||||||
|
* @param validLabels set of labels present in the retrieved context
|
||||||
|
* @return cleaned answer text with hallucinated citations removed
|
||||||
|
*/
|
||||||
|
public String validate(String generatedAnswer, Set<String> validLabels) {
|
||||||
|
if (generatedAnswer == null) return "";
|
||||||
|
|
||||||
|
Matcher matcher = CITATION_PATTERN.matcher(generatedAnswer);
|
||||||
|
List<String> removed = new ArrayList<>();
|
||||||
|
StringBuffer sb = new StringBuffer();
|
||||||
|
|
||||||
|
while (matcher.find()) {
|
||||||
|
String label = matcher.group();
|
||||||
|
String inner = label.substring(1, label.length() - 1); // strip [ ]
|
||||||
|
if (validLabels.contains(inner)) {
|
||||||
|
matcher.appendReplacement(sb, Matcher.quoteReplacement(label));
|
||||||
|
} else {
|
||||||
|
removed.add(inner);
|
||||||
|
matcher.appendReplacement(sb, "");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
matcher.appendTail(sb);
|
||||||
|
|
||||||
|
if (!removed.isEmpty()) {
|
||||||
|
log.warn("Stripped hallucinated citations: {}", removed);
|
||||||
|
}
|
||||||
|
|
||||||
|
return sb.toString();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
package com.aiteacher.retrieval;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Value object holding the original user query alongside its clinically
|
||||||
|
* rewritten variant used for vector-store retrieval.
|
||||||
|
*/
|
||||||
|
public record ExpandedQuery(String original, String rewritten) {}
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
package com.aiteacher.retrieval;
|
||||||
|
|
||||||
|
import com.aiteacher.document.FigureEntity;
|
||||||
|
import com.aiteacher.document.SectionEntity;
|
||||||
|
|
||||||
|
import java.util.HashSet;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Set;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Value object produced when building the LLM context prompt.
|
||||||
|
* Maps short ref-labels (S1, S2… / F1, F2…) to their source entities
|
||||||
|
* and carries the fully formatted prompt text.
|
||||||
|
*/
|
||||||
|
public record LabelledContext(
|
||||||
|
Map<String, SectionEntity> sectionLabels,
|
||||||
|
Map<String, FigureEntity> figureLabels,
|
||||||
|
String promptText) {
|
||||||
|
|
||||||
|
/** Returns the union of all valid citation labels for this context. */
|
||||||
|
public Set<String> allLabels() {
|
||||||
|
Set<String> labels = new HashSet<>();
|
||||||
|
labels.addAll(sectionLabels.keySet());
|
||||||
|
labels.addAll(figureLabels.keySet());
|
||||||
|
return labels;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
package com.aiteacher.retrieval;
|
||||||
|
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
import org.springframework.ai.chat.client.ChatClient;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Rewrites a user query into precise clinical/surgical terminology so that
|
||||||
|
* vector-store retrieval can match textbook language even when the user's
|
||||||
|
* phrasing differs from the documentation vocabulary.
|
||||||
|
*/
|
||||||
|
@Service
|
||||||
|
public class QueryExpansionService {
|
||||||
|
|
||||||
|
private static final Logger log = LoggerFactory.getLogger(QueryExpansionService.class);
|
||||||
|
|
||||||
|
private static final String EXPANSION_PROMPT = """
|
||||||
|
Rewrite the following question using precise medical and surgical terminology \
|
||||||
|
as it would appear in a neurosurgery textbook index. \
|
||||||
|
Output only the rewritten question, nothing else.
|
||||||
|
Question: %s""";
|
||||||
|
|
||||||
|
private final ChatClient chatClient;
|
||||||
|
|
||||||
|
public QueryExpansionService(ChatClient chatClient) {
|
||||||
|
this.chatClient = chatClient;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Returns an {@link ExpandedQuery} whose {@code rewritten} field contains
|
||||||
|
* the clinically rephrased version of {@code query}.
|
||||||
|
*/
|
||||||
|
public ExpandedQuery expand(String query) {
|
||||||
|
String rewritten = chatClient.prompt()
|
||||||
|
.user(EXPANSION_PROMPT.formatted(query))
|
||||||
|
.call()
|
||||||
|
.content();
|
||||||
|
|
||||||
|
if (rewritten == null || rewritten.isBlank()) {
|
||||||
|
rewritten = query;
|
||||||
|
}
|
||||||
|
|
||||||
|
log.debug("Query expanded: '{}' → '{}'", query, rewritten);
|
||||||
|
return new ExpandedQuery(query, rewritten);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
package com.aiteacher.topic;
|
||||||
|
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.util.UUID;
|
||||||
|
|
||||||
|
public record SavedSummaryItem(UUID id, int summaryNumber, Instant generatedAt) {
|
||||||
|
}
|
||||||
@@ -5,6 +5,7 @@ import org.springframework.web.bind.annotation.*;
|
|||||||
|
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.NoSuchElementException;
|
import java.util.NoSuchElementException;
|
||||||
|
import java.util.UUID;
|
||||||
|
|
||||||
@RestController
|
@RestController
|
||||||
@RequestMapping("/api/v1/topics")
|
@RequestMapping("/api/v1/topics")
|
||||||
@@ -32,4 +33,21 @@ public class TopicController {
|
|||||||
TopicSummaryResponse response = topicSummaryService.generateSummary(topic);
|
TopicSummaryResponse response = topicSummaryService.generateSummary(topic);
|
||||||
return ResponseEntity.ok(response);
|
return ResponseEntity.ok(response);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@GetMapping("/{id}/summaries")
|
||||||
|
public ResponseEntity<List<SavedSummaryItem>> listSummaries(@PathVariable String id) {
|
||||||
|
topicRepository.findById(id)
|
||||||
|
.orElseThrow(() -> new NoSuchElementException("Topic not found."));
|
||||||
|
|
||||||
|
return ResponseEntity.ok(topicSummaryService.listSummaries(id));
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("/{id}/summaries/{summaryId}")
|
||||||
|
public ResponseEntity<TopicSummaryResponse> getSummary(@PathVariable String id,
|
||||||
|
@PathVariable UUID summaryId) {
|
||||||
|
topicRepository.findById(id)
|
||||||
|
.orElseThrow(() -> new NoSuchElementException("Topic not found."));
|
||||||
|
|
||||||
|
return ResponseEntity.ok(topicSummaryService.getSummary(summaryId));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
package com.aiteacher.topic;
|
||||||
|
|
||||||
|
import jakarta.persistence.Column;
|
||||||
|
import jakarta.persistence.Entity;
|
||||||
|
import jakarta.persistence.GeneratedValue;
|
||||||
|
import jakarta.persistence.GenerationType;
|
||||||
|
import jakarta.persistence.Id;
|
||||||
|
import jakarta.persistence.Table;
|
||||||
|
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.util.UUID;
|
||||||
|
|
||||||
|
@Entity
|
||||||
|
@Table(name = "topic_summary")
|
||||||
|
public class TopicSummaryEntity {
|
||||||
|
|
||||||
|
@Id
|
||||||
|
@GeneratedValue(strategy = GenerationType.UUID)
|
||||||
|
private UUID id;
|
||||||
|
|
||||||
|
@Column(name = "topic_id", nullable = false)
|
||||||
|
private String topicId;
|
||||||
|
|
||||||
|
@Column(name = "summary_number", nullable = false)
|
||||||
|
private int summaryNumber;
|
||||||
|
|
||||||
|
@Column(nullable = false, columnDefinition = "TEXT")
|
||||||
|
private String summary;
|
||||||
|
|
||||||
|
@Column(name = "sources_json", nullable = false, columnDefinition = "TEXT")
|
||||||
|
private String sourcesJson;
|
||||||
|
|
||||||
|
@Column(name = "generated_at", nullable = false)
|
||||||
|
private Instant generatedAt;
|
||||||
|
|
||||||
|
protected TopicSummaryEntity() {}
|
||||||
|
|
||||||
|
public TopicSummaryEntity(String topicId, int summaryNumber, String summary,
|
||||||
|
String sourcesJson, Instant generatedAt) {
|
||||||
|
this.topicId = topicId;
|
||||||
|
this.summaryNumber = summaryNumber;
|
||||||
|
this.summary = summary;
|
||||||
|
this.sourcesJson = sourcesJson;
|
||||||
|
this.generatedAt = generatedAt;
|
||||||
|
}
|
||||||
|
|
||||||
|
public UUID getId() { return id; }
|
||||||
|
public String getTopicId() { return topicId; }
|
||||||
|
public int getSummaryNumber() { return summaryNumber; }
|
||||||
|
public String getSummary() { return summary; }
|
||||||
|
public String getSourcesJson() { return sourcesJson; }
|
||||||
|
public Instant getGeneratedAt() { return generatedAt; }
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
package com.aiteacher.topic;
|
||||||
|
|
||||||
|
import org.springframework.data.jpa.repository.JpaRepository;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.UUID;
|
||||||
|
|
||||||
|
public interface TopicSummaryRepository extends JpaRepository<TopicSummaryEntity, UUID> {
|
||||||
|
|
||||||
|
List<TopicSummaryEntity> findByTopicIdOrderBySummaryNumberAsc(String topicId);
|
||||||
|
|
||||||
|
long countByTopicId(String topicId);
|
||||||
|
}
|
||||||
@@ -2,8 +2,11 @@ package com.aiteacher.topic;
|
|||||||
|
|
||||||
import java.time.Instant;
|
import java.time.Instant;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
|
import java.util.UUID;
|
||||||
|
|
||||||
public record TopicSummaryResponse(
|
public record TopicSummaryResponse(
|
||||||
|
UUID id,
|
||||||
|
int summaryNumber,
|
||||||
String topicId,
|
String topicId,
|
||||||
String topicName,
|
String topicName,
|
||||||
String summary,
|
String summary,
|
||||||
@@ -11,6 +14,7 @@ public record TopicSummaryResponse(
|
|||||||
Instant generatedAt
|
Instant generatedAt
|
||||||
) {
|
) {
|
||||||
public record SourceReference(
|
public record SourceReference(
|
||||||
|
String bookId,
|
||||||
String bookTitle,
|
String bookTitle,
|
||||||
Integer page
|
Integer page
|
||||||
) {
|
) {
|
||||||
|
|||||||
@@ -1,21 +1,25 @@
|
|||||||
package com.aiteacher.topic;
|
package com.aiteacher.topic;
|
||||||
|
|
||||||
|
import com.aiteacher.book.Book;
|
||||||
import com.aiteacher.book.BookRepository;
|
import com.aiteacher.book.BookRepository;
|
||||||
import com.aiteacher.book.BookStatus;
|
import com.aiteacher.book.BookStatus;
|
||||||
import com.aiteacher.book.NoKnowledgeSourceException;
|
import com.aiteacher.book.NoKnowledgeSourceException;
|
||||||
|
import com.aiteacher.document.FigureEntity;
|
||||||
|
import com.aiteacher.document.SectionEntity;
|
||||||
|
import com.aiteacher.retrieval.NeurosurgeryRetriever;
|
||||||
|
import com.aiteacher.retrieval.RetrievalResult;
|
||||||
|
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||||
|
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||||
import org.slf4j.Logger;
|
import org.slf4j.Logger;
|
||||||
import org.slf4j.LoggerFactory;
|
import org.slf4j.LoggerFactory;
|
||||||
import org.springframework.ai.chat.client.ChatClient;
|
import org.springframework.ai.chat.client.ChatClient;
|
||||||
import org.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;
|
|
||||||
import org.springframework.ai.chat.model.ChatResponse;
|
|
||||||
import org.springframework.ai.document.Document;
|
|
||||||
import org.springframework.ai.vectorstore.VectorStore;
|
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
import java.time.Instant;
|
import java.time.Instant;
|
||||||
import java.util.ArrayList;
|
import java.util.ArrayList;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.Map;
|
import java.util.NoSuchElementException;
|
||||||
|
import java.util.UUID;
|
||||||
|
|
||||||
@Service
|
@Service
|
||||||
public class TopicSummaryService {
|
public class TopicSummaryService {
|
||||||
@@ -29,80 +33,190 @@ public class TopicSummaryService {
|
|||||||
|
|
||||||
When answering:
|
When answering:
|
||||||
- Structure your response clearly with key points
|
- Structure your response clearly with key points
|
||||||
- If the context mentions specific book titles and page numbers, reference them
|
- Cite claims using ONLY the reference labels provided in the context (e.g. [S1], [F2]).
|
||||||
|
Do not invent page numbers, section titles, or labels not present in the CONTEXT block.
|
||||||
- If the retrieved context does not contain sufficient information on the topic,
|
- If the retrieved context does not contain sufficient information on the topic,
|
||||||
explicitly state: "The uploaded books do not contain sufficient information on this topic."
|
explicitly state: "The uploaded books do not contain sufficient information on this topic."
|
||||||
- Never hallucinate or fabricate clinical information
|
- Never hallucinate or fabricate clinical information
|
||||||
""";
|
""";
|
||||||
|
|
||||||
private final ChatClient chatClient;
|
private final ChatClient chatClient;
|
||||||
private final VectorStore vectorStore;
|
|
||||||
private final BookRepository bookRepository;
|
private final BookRepository bookRepository;
|
||||||
|
private final NeurosurgeryRetriever retriever;
|
||||||
|
private final TopicSummaryRepository summaryRepository;
|
||||||
|
private final ObjectMapper objectMapper;
|
||||||
|
|
||||||
public TopicSummaryService(ChatClient chatClient, VectorStore vectorStore,
|
public TopicSummaryService(ChatClient chatClient,
|
||||||
BookRepository bookRepository) {
|
BookRepository bookRepository,
|
||||||
|
NeurosurgeryRetriever retriever,
|
||||||
|
TopicSummaryRepository summaryRepository,
|
||||||
|
ObjectMapper objectMapper) {
|
||||||
this.chatClient = chatClient;
|
this.chatClient = chatClient;
|
||||||
this.vectorStore = vectorStore;
|
|
||||||
this.bookRepository = bookRepository;
|
this.bookRepository = bookRepository;
|
||||||
|
this.retriever = retriever;
|
||||||
|
this.summaryRepository = summaryRepository;
|
||||||
|
this.objectMapper = objectMapper;
|
||||||
}
|
}
|
||||||
|
|
||||||
public TopicSummaryResponse generateSummary(Topic topic) {
|
public TopicSummaryResponse generateSummary(Topic topic) {
|
||||||
if (!bookRepository.existsByStatus(BookStatus.READY)) {
|
List<Book> readyBooks = bookRepository.findAll().stream()
|
||||||
|
.filter(b -> b.getStatus() == BookStatus.READY)
|
||||||
|
.toList();
|
||||||
|
|
||||||
|
if (readyBooks.isEmpty()) {
|
||||||
throw new NoKnowledgeSourceException(
|
throw new NoKnowledgeSourceException(
|
||||||
"No books are available as knowledge sources. Please upload and process at least one book.");
|
"No books are available as knowledge sources. Please upload and process at least one book.");
|
||||||
}
|
}
|
||||||
|
|
||||||
String question = buildQuestion(topic);
|
String question = buildQuestion(topic);
|
||||||
|
|
||||||
ChatResponse response = chatClient.prompt()
|
List<SectionEntity> allSections = new ArrayList<>();
|
||||||
.system(SYSTEM_PROMPT)
|
List<FigureEntity> allFigures = new ArrayList<>();
|
||||||
.advisors(QuestionAnswerAdvisor.builder(vectorStore).build())
|
for (Book book : readyBooks) {
|
||||||
.user(question)
|
RetrievalResult result = retriever.retrieve(question, book.getId());
|
||||||
.call()
|
allSections.addAll(result.parentSections());
|
||||||
.chatResponse();
|
allFigures.addAll(result.figures());
|
||||||
|
}
|
||||||
|
|
||||||
String summary = response.getResult().getOutput().getText();
|
log.debug("Topic summary for '{}': {} sections, {} figures retrieved",
|
||||||
List<TopicSummaryResponse.SourceReference> sources = extractSources(response);
|
topic.getName(), allSections.size(), allFigures.size());
|
||||||
|
|
||||||
|
String contextPrompt = buildContextPrompt(question, allSections, allFigures);
|
||||||
|
String summary = chatClient.prompt()
|
||||||
|
.system(SYSTEM_PROMPT)
|
||||||
|
.user(contextPrompt)
|
||||||
|
.call()
|
||||||
|
.content();
|
||||||
|
|
||||||
|
List<TopicSummaryResponse.SourceReference> sources = buildSources(allSections, allFigures, readyBooks);
|
||||||
|
Instant generatedAt = Instant.now();
|
||||||
|
|
||||||
|
int summaryNumber = (int) summaryRepository.countByTopicId(topic.getId()) + 1;
|
||||||
|
String sourcesJson = serializeSources(sources);
|
||||||
|
TopicSummaryEntity entity = new TopicSummaryEntity(
|
||||||
|
topic.getId(), summaryNumber, summary, sourcesJson, generatedAt);
|
||||||
|
entity = summaryRepository.save(entity);
|
||||||
|
|
||||||
return new TopicSummaryResponse(
|
return new TopicSummaryResponse(
|
||||||
|
entity.getId(),
|
||||||
|
summaryNumber,
|
||||||
topic.getId(),
|
topic.getId(),
|
||||||
topic.getName(),
|
topic.getName(),
|
||||||
summary,
|
summary,
|
||||||
sources,
|
sources,
|
||||||
Instant.now()
|
generatedAt
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public List<SavedSummaryItem> listSummaries(String topicId) {
|
||||||
|
return summaryRepository.findByTopicIdOrderBySummaryNumberAsc(topicId).stream()
|
||||||
|
.map(e -> new SavedSummaryItem(e.getId(), e.getSummaryNumber(), e.getGeneratedAt()))
|
||||||
|
.toList();
|
||||||
|
}
|
||||||
|
|
||||||
|
public TopicSummaryResponse getSummary(UUID summaryId) {
|
||||||
|
TopicSummaryEntity entity = summaryRepository.findById(summaryId)
|
||||||
|
.orElseThrow(() -> new NoSuchElementException("Summary not found."));
|
||||||
|
|
||||||
|
List<TopicSummaryResponse.SourceReference> sources = deserializeSources(entity.getSourcesJson());
|
||||||
|
|
||||||
|
return new TopicSummaryResponse(
|
||||||
|
entity.getId(),
|
||||||
|
entity.getSummaryNumber(),
|
||||||
|
entity.getTopicId(),
|
||||||
|
entity.getTopicId(),
|
||||||
|
entity.getSummary(),
|
||||||
|
sources,
|
||||||
|
entity.getGeneratedAt()
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
private String buildQuestion(Topic topic) {
|
private String buildQuestion(Topic topic) {
|
||||||
return String.format(
|
return String.format(
|
||||||
"Please provide a comprehensive educational summary of the following neurosurgery topic: " +
|
"Provide a comprehensive educational summary of the following neurosurgery topic: " +
|
||||||
"%s. Topic description: %s. " +
|
"%s. Topic description: %s. " +
|
||||||
"Include key concepts, clinical considerations, and important details that a neurosurgeon should know.",
|
"Include key concepts, clinical considerations, and important details that a neurosurgeon should know.",
|
||||||
topic.getName(), topic.getDescription()
|
topic.getName(), topic.getDescription()
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
private List<TopicSummaryResponse.SourceReference> extractSources(ChatResponse response) {
|
private String buildContextPrompt(String question,
|
||||||
List<TopicSummaryResponse.SourceReference> sources = new ArrayList<>();
|
List<SectionEntity> sections,
|
||||||
|
List<FigureEntity> figures) {
|
||||||
|
StringBuilder sb = new StringBuilder();
|
||||||
|
|
||||||
if (response.getMetadata() != null) {
|
if (!sections.isEmpty()) {
|
||||||
Object retrieved = response.getMetadata().get(QuestionAnswerAdvisor.RETRIEVED_DOCUMENTS);
|
sb.append("CONTEXT:\n\n");
|
||||||
if (retrieved instanceof List<?> docs) {
|
for (int i = 0; i < sections.size(); i++) {
|
||||||
for (Object docObj : docs) {
|
SectionEntity s = sections.get(i);
|
||||||
if (docObj instanceof Document doc) {
|
sb.append("[S").append(i + 1).append("] ")
|
||||||
Map<String, Object> metadata = doc.getMetadata();
|
.append(s.getTitle()).append(", p.").append(s.getPageStart()).append("\n");
|
||||||
String bookTitle = (String) metadata.get("book_title");
|
sb.append(s.getFullText()).append("\n\n");
|
||||||
Object pageObj = metadata.get("page_number");
|
|
||||||
Integer page = pageObj instanceof Number n ? n.intValue() : null;
|
|
||||||
if (bookTitle != null) {
|
|
||||||
sources.add(new TopicSummaryResponse.SourceReference(bookTitle, page));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Deduplicate by bookTitle + page
|
if (!figures.isEmpty()) {
|
||||||
|
sb.append("AVAILABLE FIGURES:\n");
|
||||||
|
for (int i = 0; i < figures.size(); i++) {
|
||||||
|
FigureEntity f = figures.get(i);
|
||||||
|
sb.append("[F").append(i + 1).append("] ")
|
||||||
|
.append(f.getLabel() != null ? f.getLabel() : "Figure")
|
||||||
|
.append(" (p.").append(f.getPage()).append("): ")
|
||||||
|
.append(f.getCaption() != null ? f.getCaption() : "")
|
||||||
|
.append("\n");
|
||||||
|
}
|
||||||
|
sb.append("\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
sb.append("QUESTION:\n").append(question);
|
||||||
|
return sb.toString();
|
||||||
|
}
|
||||||
|
|
||||||
|
private List<TopicSummaryResponse.SourceReference> buildSources(List<SectionEntity> sections,
|
||||||
|
List<FigureEntity> figures,
|
||||||
|
List<Book> readyBooks) {
|
||||||
|
List<TopicSummaryResponse.SourceReference> sources = new ArrayList<>();
|
||||||
|
|
||||||
|
for (SectionEntity s : sections) {
|
||||||
|
Book book = readyBooks.stream()
|
||||||
|
.filter(b -> b.getId().equals(s.getBookId()))
|
||||||
|
.findFirst()
|
||||||
|
.orElse(null);
|
||||||
|
String title = book != null ? book.getTitle() : "Book";
|
||||||
|
String bookId = book != null ? book.getId().toString() : null;
|
||||||
|
sources.add(new TopicSummaryResponse.SourceReference(bookId, title, s.getPageStart()));
|
||||||
|
}
|
||||||
|
|
||||||
|
for (FigureEntity f : figures) {
|
||||||
|
Book book = readyBooks.stream()
|
||||||
|
.filter(b -> b.getId().equals(f.getBookId()))
|
||||||
|
.findFirst()
|
||||||
|
.orElse(null);
|
||||||
|
String title = book != null ? book.getTitle() : "Book";
|
||||||
|
String bookId = book != null ? book.getId().toString() : null;
|
||||||
|
sources.add(new TopicSummaryResponse.SourceReference(bookId, title, f.getPage()));
|
||||||
|
}
|
||||||
|
|
||||||
return sources.stream().distinct().toList();
|
return sources.stream().distinct().toList();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private String serializeSources(List<TopicSummaryResponse.SourceReference> sources) {
|
||||||
|
try {
|
||||||
|
return objectMapper.writeValueAsString(sources);
|
||||||
|
} catch (JsonProcessingException e) {
|
||||||
|
log.warn("Failed to serialize sources, storing empty array", e);
|
||||||
|
return "[]";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private List<TopicSummaryResponse.SourceReference> deserializeSources(String json) {
|
||||||
|
try {
|
||||||
|
return objectMapper.readValue(json,
|
||||||
|
objectMapper.getTypeFactory().constructCollectionType(
|
||||||
|
List.class, TopicSummaryResponse.SourceReference.class));
|
||||||
|
} catch (JsonProcessingException e) {
|
||||||
|
log.warn("Failed to deserialize sources from stored JSON", e);
|
||||||
|
return List.of();
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -27,10 +27,10 @@ spring:
|
|||||||
index-type: HNSW
|
index-type: HNSW
|
||||||
initialize-schema: false
|
initialize-schema: false
|
||||||
openai:
|
openai:
|
||||||
api-key: ${OPENAI_API_KEY}
|
api-key: ${OPENAI_API_KEY:}
|
||||||
chat:
|
chat:
|
||||||
options:
|
options:
|
||||||
model: gpt-4o
|
model: gpt-4o-mini
|
||||||
embedding:
|
embedding:
|
||||||
options:
|
options:
|
||||||
model: "text-embedding-3-small"
|
model: "text-embedding-3-small"
|
||||||
@@ -52,11 +52,24 @@ logging:
|
|||||||
"[org.apache.pdfbox]": ERROR
|
"[org.apache.pdfbox]": ERROR
|
||||||
|
|
||||||
app:
|
app:
|
||||||
|
features:
|
||||||
|
upload-enabled: ${UPLOAD_ENABLED:true}
|
||||||
|
delete-enabled: ${DELETE_ENABLED:true}
|
||||||
auth:
|
auth:
|
||||||
|
username: ${APP_AUTH_USERNAME:neurosurgeon}
|
||||||
password: ${APP_PASSWORD:changeme}
|
password: ${APP_PASSWORD:changeme}
|
||||||
figure-storage:
|
figure-storage:
|
||||||
base-path: ${FIGURE_STORAGE_PATH:./uploads}
|
endpoint: ${S3_ENDPOINT:https://s3.immich-ad.ovh}
|
||||||
|
region: ${S3_REGION:garage}
|
||||||
|
bucket: ${S3_BUCKET:aiteacher}
|
||||||
|
access-key-id: ${S3_ACCESS_KEY_ID:}
|
||||||
|
secret-access-key: ${S3_SECRET_ACCESS_KEY:}
|
||||||
min-image-size-px: 100
|
min-image-size-px: 100
|
||||||
embedding:
|
embedding:
|
||||||
batch-size: 20
|
batch-size: 20
|
||||||
batch-delay-ms: 2000
|
batch-delay-ms: 2000
|
||||||
|
skip-embedding: false
|
||||||
|
marker:
|
||||||
|
base-url: ${MARKER_BASE_URL:http://192.168.1.105:8000}
|
||||||
|
vision:
|
||||||
|
min-interval-ms: ${VISION_MIN_INTERVAL_MS:2000}
|
||||||
|
|||||||
@@ -0,0 +1,10 @@
|
|||||||
|
CREATE TABLE topic_summary (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
topic_id VARCHAR(100) NOT NULL,
|
||||||
|
summary_number INT NOT NULL,
|
||||||
|
summary TEXT NOT NULL,
|
||||||
|
sources_json TEXT NOT NULL,
|
||||||
|
generated_at TIMESTAMPTZ NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX idx_topic_summary_topic_id ON topic_summary(topic_id, summary_number);
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
version: '3.9'
|
||||||
|
|
||||||
|
services:
|
||||||
|
postgres:
|
||||||
|
image: pgvector/pgvector:pg16
|
||||||
|
container_name: aiteacher-postgres-native
|
||||||
|
environment:
|
||||||
|
POSTGRES_DB: aiteacher
|
||||||
|
POSTGRES_USER: aiteacher
|
||||||
|
POSTGRES_PASSWORD: aiteacher
|
||||||
|
ports:
|
||||||
|
- "5432:5432"
|
||||||
|
volumes:
|
||||||
|
- pgdata_native:/var/lib/postgresql/data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U aiteacher -d aiteacher"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 5
|
||||||
|
|
||||||
|
backend:
|
||||||
|
image: ai-teacher-backend:latest
|
||||||
|
container_name: aiteacher-backend-native
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
|
environment:
|
||||||
|
SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/aiteacher
|
||||||
|
SPRING_DATASOURCE_USERNAME: aiteacher
|
||||||
|
SPRING_DATASOURCE_PASSWORD: aiteacher
|
||||||
|
ports:
|
||||||
|
- "8080:8080"
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
pgdata_native:
|
||||||
@@ -3,5 +3,12 @@
|
|||||||
# In production point it directly at the backend, e.g. https://api.example.com/api/v1
|
# In production point it directly at the backend, e.g. https://api.example.com/api/v1
|
||||||
VITE_API_URL=/api/v1
|
VITE_API_URL=/api/v1
|
||||||
|
|
||||||
# Shared password for HTTP Basic auth (must match APP_PASSWORD on the backend).
|
# Credentials are no longer configured here. Users enter their username and
|
||||||
VITE_APP_PASSWORD=changeme
|
# password via the login form. The backend validates them via HTTP Basic Auth.
|
||||||
|
# Configure the backend credentials with APP_AUTH_USERNAME and APP_PASSWORD.
|
||||||
|
|
||||||
|
# Set to 'false' to hide the upload UI (frontend). Also set UPLOAD_ENABLED=false on the backend to block the endpoint.
|
||||||
|
VITE_UPLOAD_ENABLED=true
|
||||||
|
|
||||||
|
# Set to 'false' to hide the delete button (frontend). Also set DELETE_ENABLED=false on the backend to block the endpoint.
|
||||||
|
VITE_DELETE_ENABLED=true
|
||||||
|
|||||||
+5
-3
@@ -1,5 +1,5 @@
|
|||||||
# ---- Build stage ----
|
# ---- Build stage ----
|
||||||
FROM node:20-alpine AS build
|
FROM docker.io/library/node:20-alpine AS build
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
COPY package*.json ./
|
COPY package*.json ./
|
||||||
RUN npm ci
|
RUN npm ci
|
||||||
@@ -7,8 +7,10 @@ COPY . .
|
|||||||
RUN npm run build
|
RUN npm run build
|
||||||
|
|
||||||
# ---- Runtime stage (nginx) ----
|
# ---- Runtime stage (nginx) ----
|
||||||
FROM nginx:alpine
|
FROM docker.io/library/nginx:alpine
|
||||||
COPY --from=build /app/dist /usr/share/nginx/html
|
COPY --from=build /app/dist /usr/share/nginx/html
|
||||||
COPY nginx.conf /etc/nginx/conf.d/default.conf
|
COPY nginx.conf /etc/nginx/conf.d/default.conf
|
||||||
|
COPY docker-entrypoint.sh /docker-entrypoint.sh
|
||||||
|
RUN chmod +x /docker-entrypoint.sh
|
||||||
EXPOSE 80
|
EXPOSE 80
|
||||||
CMD ["nginx", "-g", "daemon off;"]
|
ENTRYPOINT ["/docker-entrypoint.sh"]
|
||||||
|
|||||||
@@ -0,0 +1,16 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Write runtime env vars into a JS file loaded before the app bundle.
|
||||||
|
# Any VITE_* variable passed via `docker run -e` will be available as
|
||||||
|
# window.__env__.VITE_* inside the browser.
|
||||||
|
cat > /usr/share/nginx/html/env-config.js <<EOF
|
||||||
|
window.__env__ = {
|
||||||
|
VITE_API_URL: "${VITE_API_URL:-}",
|
||||||
|
VITE_APP_PASSWORD: "${VITE_APP_PASSWORD:-}",
|
||||||
|
VITE_UPLOAD_ENABLED: "${VITE_UPLOAD_ENABLED:-}",
|
||||||
|
VITE_DELETE_ENABLED: "${VITE_DELETE_ENABLED:-}"
|
||||||
|
};
|
||||||
|
EOF
|
||||||
|
|
||||||
|
exec nginx -g "daemon off;"
|
||||||
@@ -8,6 +8,7 @@
|
|||||||
</head>
|
</head>
|
||||||
<body>
|
<body>
|
||||||
<div id="app"></div>
|
<div id="app"></div>
|
||||||
|
<script src="/env-config.js"></script>
|
||||||
<script type="module" src="/src/main.ts"></script>
|
<script type="module" src="/src/main.ts"></script>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|||||||
+156
-21
@@ -6,23 +6,31 @@
|
|||||||
<span class="brand-name">AI Teacher</span>
|
<span class="brand-name">AI Teacher</span>
|
||||||
<span class="brand-subtitle">Neurosurgeon Learning Platform</span>
|
<span class="brand-subtitle">Neurosurgeon Learning Platform</span>
|
||||||
</div>
|
</div>
|
||||||
<ul class="navbar-links">
|
<template v-if="authStore.isAuthenticated">
|
||||||
<li>
|
<button class="burger" :class="{ open: menuOpen }" @click="menuOpen = !menuOpen" aria-label="Menu">
|
||||||
<RouterLink to="/" :class="{ active: $route.path === '/' }">
|
<span></span><span></span><span></span>
|
||||||
<span class="nav-icon">📚</span> Library
|
</button>
|
||||||
</RouterLink>
|
<div class="nav-drawer" :class="{ open: menuOpen }" @click="menuOpen = false">
|
||||||
</li>
|
<ul class="navbar-links">
|
||||||
<li>
|
<li>
|
||||||
<RouterLink to="/topics" :class="{ active: $route.path === '/topics' }">
|
<RouterLink to="/" :class="{ active: $route.path === '/' }">
|
||||||
<span class="nav-icon">🗂</span> Topics
|
<span class="nav-icon">📚</span> Library
|
||||||
</RouterLink>
|
</RouterLink>
|
||||||
</li>
|
</li>
|
||||||
<li>
|
<li>
|
||||||
<RouterLink to="/chat" :class="{ active: $route.path === '/chat' }">
|
<RouterLink to="/topics" :class="{ active: $route.path === '/topics' }">
|
||||||
<span class="nav-icon">💬</span> Chat
|
<span class="nav-icon">🗂</span> Topics
|
||||||
</RouterLink>
|
</RouterLink>
|
||||||
</li>
|
</li>
|
||||||
</ul>
|
<li>
|
||||||
|
<RouterLink to="/chat" :class="{ active: $route.path === '/chat' }">
|
||||||
|
<span class="nav-icon">💬</span> Chat
|
||||||
|
</RouterLink>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
<button class="btn btn-logout" @click.stop="logout">Sign out</button>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
</nav>
|
</nav>
|
||||||
|
|
||||||
<main class="main-content">
|
<main class="main-content">
|
||||||
@@ -35,12 +43,26 @@
|
|||||||
</template>
|
</template>
|
||||||
|
|
||||||
<script setup lang="ts">
|
<script setup lang="ts">
|
||||||
import { ref, provide } from 'vue'
|
import { ref, provide, watch } from 'vue'
|
||||||
import { RouterLink, RouterView } from 'vue-router'
|
import { RouterLink, RouterView, useRouter, useRoute } from 'vue-router'
|
||||||
|
import { useAuthStore } from '@/stores/authStore'
|
||||||
|
|
||||||
|
const authStore = useAuthStore()
|
||||||
|
const router = useRouter()
|
||||||
|
const route = useRoute()
|
||||||
|
|
||||||
|
const menuOpen = ref(false)
|
||||||
const toastMessage = ref('')
|
const toastMessage = ref('')
|
||||||
const toastType = ref<'toast-error' | 'toast-success'>('toast-error')
|
const toastType = ref<'toast-error' | 'toast-success'>('toast-error')
|
||||||
|
|
||||||
|
// Close menu on navigation
|
||||||
|
watch(() => route.path, () => { menuOpen.value = false })
|
||||||
|
|
||||||
|
function logout() {
|
||||||
|
authStore.clearCredentials()
|
||||||
|
router.push({ name: 'login' })
|
||||||
|
}
|
||||||
|
|
||||||
function showToast(message: string, type: 'error' | 'success' = 'error') {
|
function showToast(message: string, type: 'error' | 'success' = 'error') {
|
||||||
toastMessage.value = message
|
toastMessage.value = message
|
||||||
toastType.value = type === 'error' ? 'toast-error' : 'toast-success'
|
toastType.value = type === 'error' ? 'toast-error' : 'toast-success'
|
||||||
@@ -64,11 +86,11 @@ body {
|
|||||||
Ubuntu, Cantarell, 'Fira Sans', 'Droid Sans', 'Helvetica Neue', sans-serif;
|
Ubuntu, Cantarell, 'Fira Sans', 'Droid Sans', 'Helvetica Neue', sans-serif;
|
||||||
background: #f0f4f8;
|
background: #f0f4f8;
|
||||||
color: #2d3748;
|
color: #2d3748;
|
||||||
min-height: 100vh;
|
height: 100vh;
|
||||||
}
|
}
|
||||||
|
|
||||||
#app {
|
#app {
|
||||||
min-height: 100vh;
|
height: 100vh;
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
}
|
}
|
||||||
@@ -82,6 +104,9 @@ body {
|
|||||||
justify-content: space-between;
|
justify-content: space-between;
|
||||||
height: 64px;
|
height: 64px;
|
||||||
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.3);
|
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.3);
|
||||||
|
position: sticky;
|
||||||
|
top: 0;
|
||||||
|
z-index: 100;
|
||||||
}
|
}
|
||||||
|
|
||||||
.navbar-brand {
|
.navbar-brand {
|
||||||
@@ -106,6 +131,13 @@ body {
|
|||||||
margin-left: 0.25rem;
|
margin-left: 0.25rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Desktop: links inline */
|
||||||
|
.nav-drawer {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
.navbar-links {
|
.navbar-links {
|
||||||
list-style: none;
|
list-style: none;
|
||||||
display: flex;
|
display: flex;
|
||||||
@@ -131,8 +163,38 @@ body {
|
|||||||
color: white;
|
color: white;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Burger button — hidden on desktop */
|
||||||
|
.burger {
|
||||||
|
display: none;
|
||||||
|
flex-direction: column;
|
||||||
|
justify-content: center;
|
||||||
|
gap: 5px;
|
||||||
|
width: 36px;
|
||||||
|
height: 36px;
|
||||||
|
background: transparent;
|
||||||
|
border: none;
|
||||||
|
cursor: pointer;
|
||||||
|
padding: 4px;
|
||||||
|
border-radius: 6px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.burger span {
|
||||||
|
display: block;
|
||||||
|
height: 2px;
|
||||||
|
background: #bee3f8;
|
||||||
|
border-radius: 2px;
|
||||||
|
transition: transform 0.2s, opacity 0.2s;
|
||||||
|
}
|
||||||
|
|
||||||
|
.burger.open span:nth-child(1) { transform: translateY(7px) rotate(45deg); }
|
||||||
|
.burger.open span:nth-child(2) { opacity: 0; }
|
||||||
|
.burger.open span:nth-child(3) { transform: translateY(-7px) rotate(-45deg); }
|
||||||
|
|
||||||
.main-content {
|
.main-content {
|
||||||
flex: 1;
|
flex: 1;
|
||||||
|
min-height: 0;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
padding: 2rem;
|
padding: 2rem;
|
||||||
max-width: 1200px;
|
max-width: 1200px;
|
||||||
margin: 0 auto;
|
margin: 0 auto;
|
||||||
@@ -224,6 +286,20 @@ body {
|
|||||||
background: #cbd5e0;
|
background: #cbd5e0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.btn-logout {
|
||||||
|
background: transparent;
|
||||||
|
color: #bee3f8;
|
||||||
|
border: 1px solid #4a90b8;
|
||||||
|
font-size: 0.85rem;
|
||||||
|
padding: 0.4rem 0.9rem;
|
||||||
|
margin-left: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-logout:hover {
|
||||||
|
background: #2b6cb0;
|
||||||
|
color: white;
|
||||||
|
}
|
||||||
|
|
||||||
.spinner {
|
.spinner {
|
||||||
display: inline-block;
|
display: inline-block;
|
||||||
width: 20px;
|
width: 20px;
|
||||||
@@ -284,4 +360,63 @@ body {
|
|||||||
font-size: 0.9rem;
|
font-size: 0.9rem;
|
||||||
margin-top: 0.5rem;
|
margin-top: 0.5rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@media (max-width: 768px) {
|
||||||
|
.navbar {
|
||||||
|
padding: 0 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.brand-subtitle {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Show burger, hide desktop drawer */
|
||||||
|
.burger {
|
||||||
|
display: flex;
|
||||||
|
}
|
||||||
|
|
||||||
|
.nav-drawer {
|
||||||
|
display: none;
|
||||||
|
position: absolute;
|
||||||
|
top: 64px;
|
||||||
|
right: 0;
|
||||||
|
left: 0;
|
||||||
|
background: #1a365d;
|
||||||
|
flex-direction: column;
|
||||||
|
align-items: stretch;
|
||||||
|
padding: 0.5rem 0 1rem;
|
||||||
|
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.3);
|
||||||
|
z-index: 99;
|
||||||
|
}
|
||||||
|
|
||||||
|
.nav-drawer.open {
|
||||||
|
display: flex;
|
||||||
|
}
|
||||||
|
|
||||||
|
.navbar-links {
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.navbar-links a {
|
||||||
|
padding: 0.85rem 1.5rem;
|
||||||
|
border-radius: 0;
|
||||||
|
font-size: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.navbar-links a:hover,
|
||||||
|
.navbar-links a.active {
|
||||||
|
background: #2b6cb0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-logout {
|
||||||
|
margin: 0.5rem 1.5rem 0;
|
||||||
|
width: calc(100% - 3rem);
|
||||||
|
justify-content: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
.main-content {
|
||||||
|
padding: 1rem;
|
||||||
|
}
|
||||||
|
}
|
||||||
</style>
|
</style>
|
||||||
|
|||||||
@@ -33,7 +33,15 @@
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="book-actions">
|
<div class="book-actions">
|
||||||
|
<router-link
|
||||||
|
v-if="book.status === 'READY'"
|
||||||
|
:to="{ name: 'book-reader', params: { id: book.id } }"
|
||||||
|
class="btn btn-secondary"
|
||||||
|
>
|
||||||
|
Read
|
||||||
|
</router-link>
|
||||||
<button
|
<button
|
||||||
|
v-if="deleteEnabled"
|
||||||
class="btn btn-danger"
|
class="btn btn-danger"
|
||||||
:disabled="book.status === 'PROCESSING' || deleting"
|
:disabled="book.status === 'PROCESSING' || deleting"
|
||||||
@click="$emit('delete', book.id)"
|
@click="$emit('delete', book.id)"
|
||||||
@@ -52,6 +60,7 @@ import type { Book } from '@/stores/bookStore'
|
|||||||
const props = defineProps<{
|
const props = defineProps<{
|
||||||
book: Book
|
book: Book
|
||||||
deleting?: boolean
|
deleting?: boolean
|
||||||
|
deleteEnabled?: boolean
|
||||||
}>()
|
}>()
|
||||||
|
|
||||||
defineEmits<{
|
defineEmits<{
|
||||||
@@ -181,6 +190,7 @@ function formatDate(iso: string): string {
|
|||||||
.book-actions {
|
.book-actions {
|
||||||
display: flex;
|
display: flex;
|
||||||
justify-content: flex-end;
|
justify-content: flex-end;
|
||||||
|
gap: 0.5rem;
|
||||||
margin-top: 0.25rem;
|
margin-top: 0.25rem;
|
||||||
}
|
}
|
||||||
</style>
|
</style>
|
||||||
|
|||||||
@@ -0,0 +1,239 @@
|
|||||||
|
<template>
|
||||||
|
<div class="book-panel">
|
||||||
|
<div class="book-panel-header">
|
||||||
|
<span class="book-panel-title">{{ bookTitle || 'Book' }} — p. {{ page }}</span>
|
||||||
|
<div class="book-panel-nav">
|
||||||
|
<button class="nav-btn" :disabled="page <= 1" @click="emit('navigate', page - 1)">←</button>
|
||||||
|
<button class="nav-btn" @click="emit('navigate', page + 1)">→</button>
|
||||||
|
</div>
|
||||||
|
<button class="close-btn" @click="emit('close')" title="Close">✕</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="book-panel-body">
|
||||||
|
<div v-if="loading" class="panel-loading">
|
||||||
|
<div class="spinner spinner-dark" style="width:24px;height:24px;margin:0 auto 0.5rem;"></div>
|
||||||
|
<p>Loading page {{ page }}…</p>
|
||||||
|
</div>
|
||||||
|
<div v-else-if="error" class="panel-error">{{ error }}</div>
|
||||||
|
<div v-else class="markdown-body" v-html="renderedHtml"></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<script setup lang="ts">
|
||||||
|
import { ref, watch, onMounted, onUnmounted } from 'vue'
|
||||||
|
import { api } from '@/services/api'
|
||||||
|
|
||||||
|
const props = defineProps<{
|
||||||
|
bookId: string
|
||||||
|
page: number
|
||||||
|
bookTitle?: string
|
||||||
|
}>()
|
||||||
|
|
||||||
|
const emit = defineEmits<{
|
||||||
|
close: []
|
||||||
|
navigate: [page: number]
|
||||||
|
}>()
|
||||||
|
|
||||||
|
const loading = ref(false)
|
||||||
|
const error = ref<string | null>(null)
|
||||||
|
const renderedHtml = ref('')
|
||||||
|
let activeBlobUrls: string[] = []
|
||||||
|
|
||||||
|
onMounted(() => loadPage(props.page))
|
||||||
|
|
||||||
|
watch(() => [props.bookId, props.page], () => loadPage(props.page))
|
||||||
|
|
||||||
|
onUnmounted(() => {
|
||||||
|
activeBlobUrls.forEach(u => URL.revokeObjectURL(u))
|
||||||
|
})
|
||||||
|
|
||||||
|
async function loadPage(page: number) {
|
||||||
|
loading.value = true
|
||||||
|
error.value = null
|
||||||
|
renderedHtml.value = ''
|
||||||
|
activeBlobUrls.forEach(u => URL.revokeObjectURL(u))
|
||||||
|
activeBlobUrls = []
|
||||||
|
|
||||||
|
try {
|
||||||
|
const res = await api.get<string>(`/books/${props.bookId}/pages/${page}/html`, {
|
||||||
|
headers: { Accept: 'text/html' },
|
||||||
|
responseType: 'text'
|
||||||
|
})
|
||||||
|
renderedHtml.value = await resolveImages(res.data)
|
||||||
|
} catch (e: any) {
|
||||||
|
error.value = e.message ?? 'Failed to load page.'
|
||||||
|
} finally {
|
||||||
|
loading.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function resolveImages(html: string): Promise<string> {
|
||||||
|
const srcPattern = /src="(\/api\/v1\/figures\/[^"]+)"/g
|
||||||
|
const matches = [...html.matchAll(srcPattern)]
|
||||||
|
if (matches.length === 0) return html
|
||||||
|
|
||||||
|
const unique = [...new Set(matches.map(m => m[1]))]
|
||||||
|
const blobMap: Record<string, string> = {}
|
||||||
|
|
||||||
|
await Promise.all(
|
||||||
|
unique.map(async (src) => {
|
||||||
|
try {
|
||||||
|
const res = await api.get(src.replace(/^\/api\/v1/, ''), { responseType: 'blob' })
|
||||||
|
const blobUrl = URL.createObjectURL(res.data)
|
||||||
|
activeBlobUrls.push(blobUrl)
|
||||||
|
blobMap[src] = blobUrl
|
||||||
|
} catch {
|
||||||
|
// leave original src
|
||||||
|
}
|
||||||
|
})
|
||||||
|
)
|
||||||
|
|
||||||
|
return html.replace(/src="(\/api\/v1\/figures\/[^"]+)"/g, (_, src) =>
|
||||||
|
blobMap[src] ? `src="${blobMap[src]}"` : `src="${src}"`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<style scoped>
|
||||||
|
.book-panel {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
height: 100%;
|
||||||
|
background: white;
|
||||||
|
border-left: 1px solid #e2e8f0;
|
||||||
|
border-radius: 0 10px 10px 0;
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
|
||||||
|
.book-panel-header {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
padding: 0.6rem 0.75rem;
|
||||||
|
background: #f7fafc;
|
||||||
|
border-bottom: 1px solid #e2e8f0;
|
||||||
|
flex-shrink: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.book-panel-title {
|
||||||
|
flex: 1;
|
||||||
|
font-size: 0.8rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: #2b6cb0;
|
||||||
|
white-space: nowrap;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
}
|
||||||
|
|
||||||
|
.book-panel-nav {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.25rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.nav-btn {
|
||||||
|
width: 1.75rem;
|
||||||
|
height: 1.75rem;
|
||||||
|
border: 1px solid #cbd5e0;
|
||||||
|
border-radius: 5px;
|
||||||
|
background: white;
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 0.85rem;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
transition: background 0.15s;
|
||||||
|
}
|
||||||
|
.nav-btn:hover:not(:disabled) { background: #ebf8ff; border-color: #3182ce; }
|
||||||
|
.nav-btn:disabled { opacity: 0.4; cursor: not-allowed; }
|
||||||
|
|
||||||
|
.close-btn {
|
||||||
|
width: 1.75rem;
|
||||||
|
height: 1.75rem;
|
||||||
|
border: none;
|
||||||
|
border-radius: 5px;
|
||||||
|
background: none;
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 1rem;
|
||||||
|
color: #718096;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
transition: background 0.15s, color 0.15s;
|
||||||
|
}
|
||||||
|
.close-btn:hover { background: #fed7d7; color: #742a2a; }
|
||||||
|
|
||||||
|
.book-panel-body {
|
||||||
|
flex: 1;
|
||||||
|
overflow-y: auto;
|
||||||
|
padding: 1rem 1.25rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.panel-loading {
|
||||||
|
text-align: center;
|
||||||
|
padding: 2rem;
|
||||||
|
color: #718096;
|
||||||
|
font-size: 0.875rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.panel-error {
|
||||||
|
padding: 1rem;
|
||||||
|
background: #fff5f5;
|
||||||
|
border: 1px solid #fed7d7;
|
||||||
|
color: #742a2a;
|
||||||
|
border-radius: 6px;
|
||||||
|
font-size: 0.875rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.markdown-body {
|
||||||
|
font-size: 0.9rem;
|
||||||
|
line-height: 1.75;
|
||||||
|
color: #2d3748;
|
||||||
|
}
|
||||||
|
|
||||||
|
.markdown-body :deep(h1),
|
||||||
|
.markdown-body :deep(h2),
|
||||||
|
.markdown-body :deep(h3) {
|
||||||
|
color: #1a365d;
|
||||||
|
font-weight: 600;
|
||||||
|
margin: 1.25rem 0 0.5rem;
|
||||||
|
}
|
||||||
|
.markdown-body :deep(h2) { font-size: 1.05rem; border-bottom: 1px solid #e2e8f0; padding-bottom: 0.3rem; }
|
||||||
|
.markdown-body :deep(h3) { font-size: 0.95rem; }
|
||||||
|
.markdown-body :deep(p) { margin: 0.6rem 0; }
|
||||||
|
.markdown-body :deep(img) {
|
||||||
|
max-width: 100%;
|
||||||
|
border-radius: 6px;
|
||||||
|
display: block;
|
||||||
|
margin: 0.75rem auto;
|
||||||
|
box-shadow: 0 1px 4px rgba(0,0,0,0.12);
|
||||||
|
}
|
||||||
|
.markdown-body :deep(ul),
|
||||||
|
.markdown-body :deep(ol) { padding-left: 1.4rem; margin: 0.5rem 0; }
|
||||||
|
.markdown-body :deep(code) {
|
||||||
|
background: #f7fafc;
|
||||||
|
border: 1px solid #e2e8f0;
|
||||||
|
border-radius: 3px;
|
||||||
|
padding: 0.1em 0.3em;
|
||||||
|
font-size: 0.85em;
|
||||||
|
}
|
||||||
|
.markdown-body :deep(blockquote) {
|
||||||
|
border-left: 3px solid #3182ce;
|
||||||
|
padding-left: 0.75rem;
|
||||||
|
color: #4a5568;
|
||||||
|
margin: 0.5rem 0;
|
||||||
|
}
|
||||||
|
.markdown-body :deep(table) {
|
||||||
|
width: 100%;
|
||||||
|
border-collapse: collapse;
|
||||||
|
font-size: 0.875em;
|
||||||
|
margin: 0.75rem 0;
|
||||||
|
}
|
||||||
|
.markdown-body :deep(th),
|
||||||
|
.markdown-body :deep(td) {
|
||||||
|
border: 1px solid #e2e8f0;
|
||||||
|
padding: 0.35rem 0.6rem;
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
.markdown-body :deep(th) { background: #f7fafc; font-weight: 600; }
|
||||||
|
</style>
|
||||||
@@ -3,22 +3,30 @@
|
|||||||
<div class="message-bubble" :class="isUser ? 'bubble-user' : 'bubble-assistant'">
|
<div class="message-bubble" :class="isUser ? 'bubble-user' : 'bubble-assistant'">
|
||||||
<div class="message-role">{{ isUser ? 'You' : 'AI Teacher' }}</div>
|
<div class="message-role">{{ isUser ? 'You' : 'AI Teacher' }}</div>
|
||||||
<div v-if="isUser" class="message-content">{{ message.content }}</div>
|
<div v-if="isUser" class="message-content">{{ message.content }}</div>
|
||||||
<div v-else class="message-content message-content--markdown" v-html="renderedContent"></div>
|
<div v-else class="message-content message-content--markdown" v-html="renderedWithBadges" @click="onContentClick"></div>
|
||||||
|
|
||||||
<!-- Sources for assistant messages -->
|
<!-- Sources for assistant messages -->
|
||||||
<div v-if="!isUser && message.sources && message.sources.length > 0" class="message-sources">
|
<div v-if="!isUser && message.sources && message.sources.length > 0" class="message-sources">
|
||||||
<div class="sources-label">Sources:</div>
|
<div class="sources-label">Sources:</div>
|
||||||
<div class="source-list">
|
<div class="source-list" ref="sourceListEl">
|
||||||
<!-- TEXT sources -->
|
<!-- TEXT sources -->
|
||||||
<div
|
<div
|
||||||
v-for="(source, idx) in textSources"
|
v-for="(source, idx) in textSources"
|
||||||
:key="'text-' + idx"
|
:key="'text-' + idx"
|
||||||
class="source-item"
|
class="source-item"
|
||||||
|
:class="{ 'source-item--active': activeRef === source.refLabel }"
|
||||||
|
:data-ref-label="source.refLabel"
|
||||||
>
|
>
|
||||||
<div class="source-chip source-chip--text">
|
<div
|
||||||
|
class="source-chip source-chip--text"
|
||||||
|
:class="{ 'source-chip--clickable': source.bookId && source.page }"
|
||||||
|
@click="source.bookId && source.page ? emit('open-source', source.bookId, source.page) : undefined"
|
||||||
|
>
|
||||||
<span class="source-icon">📖</span>
|
<span class="source-icon">📖</span>
|
||||||
|
<span v-if="source.refLabel" class="source-ref-label">{{ source.refLabel }}</span>
|
||||||
<span class="source-book-title">{{ source.bookTitle }}</span>
|
<span class="source-book-title">{{ source.bookTitle }}</span>
|
||||||
<span v-if="source.page" class="source-page">p. {{ source.page }}</span>
|
<span v-if="source.page" class="source-page">p. {{ source.page }}</span>
|
||||||
|
<span v-if="source.bookId && source.page" class="source-open-hint">↗</span>
|
||||||
</div>
|
</div>
|
||||||
<div v-if="source.chunkText" class="source-chunk">{{ source.chunkText }}</div>
|
<div v-if="source.chunkText" class="source-chunk">{{ source.chunkText }}</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -28,12 +36,20 @@
|
|||||||
v-for="(source, idx) in figureSources"
|
v-for="(source, idx) in figureSources"
|
||||||
:key="'fig-' + idx"
|
:key="'fig-' + idx"
|
||||||
class="source-item source-item--figure"
|
class="source-item source-item--figure"
|
||||||
|
:class="{ 'source-item--active': activeRef === source.refLabel }"
|
||||||
|
:data-ref-label="source.refLabel"
|
||||||
>
|
>
|
||||||
<div class="source-chip source-chip--figure">
|
<div
|
||||||
|
class="source-chip source-chip--figure"
|
||||||
|
:class="{ 'source-chip--clickable': source.bookId && source.page }"
|
||||||
|
@click="source.bookId && source.page ? emit('open-source', source.bookId, source.page) : undefined"
|
||||||
|
>
|
||||||
<span class="source-icon">🖼️</span>
|
<span class="source-icon">🖼️</span>
|
||||||
|
<span v-if="source.refLabel" class="source-ref-label source-ref-label--figure">{{ source.refLabel }}</span>
|
||||||
<span class="source-figure-label">{{ source.label || 'Figure' }}</span>
|
<span class="source-figure-label">{{ source.label || 'Figure' }}</span>
|
||||||
<span v-if="source.page" class="source-page">p. {{ source.page }}</span>
|
<span v-if="source.page" class="source-page">p. {{ source.page }}</span>
|
||||||
<span v-if="source.figureType" class="source-figure-type">{{ formatFigureType(source.figureType) }}</span>
|
<span v-if="source.figureType" class="source-figure-type">{{ formatFigureType(source.figureType) }}</span>
|
||||||
|
<span v-if="source.bookId && source.page" class="source-open-hint">↗</span>
|
||||||
</div>
|
</div>
|
||||||
<div v-if="source.caption" class="source-caption">{{ source.caption }}</div>
|
<div v-if="source.caption" class="source-caption">{{ source.caption }}</div>
|
||||||
<div class="source-figure-image">
|
<div class="source-figure-image">
|
||||||
@@ -55,7 +71,7 @@
|
|||||||
</template>
|
</template>
|
||||||
|
|
||||||
<script setup lang="ts">
|
<script setup lang="ts">
|
||||||
import { computed } from 'vue'
|
import { computed, ref } from 'vue'
|
||||||
import { marked } from 'marked'
|
import { marked } from 'marked'
|
||||||
import type { ChatMessage, ChatSource } from '@/stores/chatStore'
|
import type { ChatMessage, ChatSource } from '@/stores/chatStore'
|
||||||
|
|
||||||
@@ -63,8 +79,43 @@ const props = defineProps<{
|
|||||||
message: ChatMessage
|
message: ChatMessage
|
||||||
}>()
|
}>()
|
||||||
|
|
||||||
|
const emit = defineEmits<{
|
||||||
|
'open-source': [bookId: string, page: number]
|
||||||
|
}>()
|
||||||
|
|
||||||
const isUser = computed(() => props.message.role === 'USER')
|
const isUser = computed(() => props.message.role === 'USER')
|
||||||
const renderedContent = computed(() => marked.parse(props.message.content) as string)
|
const activeRef = ref<string | null>(null)
|
||||||
|
const sourceListEl = ref<HTMLElement | null>(null)
|
||||||
|
|
||||||
|
/** Replaces [S1]/[F1]-style labels in the rendered HTML with clickable badges. */
|
||||||
|
const renderedWithBadges = computed(() => {
|
||||||
|
const html = marked.parse(props.message.content) as string
|
||||||
|
return html.replace(/\[(S|F)\d+\]/g, (match) => {
|
||||||
|
const inner = match.slice(1, -1) // e.g. "S1"
|
||||||
|
return `<span class="citation-badge" data-ref="${inner}" title="Jump to source ${inner}">${match}</span>`
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
function onContentClick(e: MouseEvent) {
|
||||||
|
const target = e.target as HTMLElement
|
||||||
|
if (!target.classList.contains('citation-badge')) return
|
||||||
|
|
||||||
|
const label = target.getAttribute('data-ref') // e.g. "S1" or "F1"
|
||||||
|
if (!label) return
|
||||||
|
|
||||||
|
activeRef.value = activeRef.value === label ? null : label
|
||||||
|
|
||||||
|
// Scroll to the matching source chip
|
||||||
|
const sourceEl = sourceListEl.value?.querySelector(`[data-ref-label="${label}"]`) as HTMLElement | null
|
||||||
|
sourceEl?.scrollIntoView({ behavior: 'smooth', block: 'start' })
|
||||||
|
|
||||||
|
// Open the book at the referenced page
|
||||||
|
const allSources = props.message.sources ?? []
|
||||||
|
const source = allSources.find((s: ChatSource) => s.refLabel === label)
|
||||||
|
if (source?.bookId && source.page) {
|
||||||
|
emit('open-source', source.bookId, source.page)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
const textSources = computed(() =>
|
const textSources = computed(() =>
|
||||||
(props.message.sources ?? []).filter((s: ChatSource) => s.type === 'TEXT' || !s.type)
|
(props.message.sources ?? []).filter((s: ChatSource) => s.type === 'TEXT' || !s.type)
|
||||||
@@ -255,6 +306,22 @@ function formatTime(iso: string): string {
|
|||||||
border: 1px solid #bee3f8;
|
border: 1px solid #bee3f8;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.source-chip--clickable {
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background 0.15s, border-color 0.15s;
|
||||||
|
}
|
||||||
|
|
||||||
|
.source-chip--clickable:hover {
|
||||||
|
background: #bee3f8;
|
||||||
|
border-color: #90cdf4;
|
||||||
|
}
|
||||||
|
|
||||||
|
.source-open-hint {
|
||||||
|
font-size: 0.75rem;
|
||||||
|
color: #3182ce;
|
||||||
|
margin-left: 0.1rem;
|
||||||
|
}
|
||||||
|
|
||||||
.source-chip--figure {
|
.source-chip--figure {
|
||||||
background: #f0fff4;
|
background: #f0fff4;
|
||||||
border: 1px solid #9ae6b4;
|
border: 1px solid #9ae6b4;
|
||||||
@@ -322,6 +389,43 @@ function formatTime(iso: string): string {
|
|||||||
font-style: italic;
|
font-style: italic;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.message-content--markdown :deep(.citation-badge) {
|
||||||
|
display: inline-block;
|
||||||
|
background: #ebf8ff;
|
||||||
|
border: 1px solid #90cdf4;
|
||||||
|
border-radius: 3px;
|
||||||
|
padding: 0 0.3em;
|
||||||
|
font-size: 0.78em;
|
||||||
|
font-weight: 600;
|
||||||
|
color: #2b6cb0;
|
||||||
|
cursor: pointer;
|
||||||
|
user-select: none;
|
||||||
|
transition: background 0.15s;
|
||||||
|
}
|
||||||
|
|
||||||
|
.message-content--markdown :deep(.citation-badge:hover) {
|
||||||
|
background: #bee3f8;
|
||||||
|
}
|
||||||
|
|
||||||
|
.source-item--active {
|
||||||
|
outline: 2px solid #4299e1;
|
||||||
|
border-radius: 6px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.source-ref-label {
|
||||||
|
font-size: 0.72rem;
|
||||||
|
font-weight: 700;
|
||||||
|
background: #bee3f8;
|
||||||
|
color: #2b6cb0;
|
||||||
|
border-radius: 3px;
|
||||||
|
padding: 0 0.3rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.source-ref-label--figure {
|
||||||
|
background: #9ae6b4;
|
||||||
|
color: #276749;
|
||||||
|
}
|
||||||
|
|
||||||
.message-timestamp {
|
.message-timestamp {
|
||||||
font-size: 0.7rem;
|
font-size: 0.7rem;
|
||||||
opacity: 0.6;
|
opacity: 0.6;
|
||||||
|
|||||||
Vendored
+2
@@ -3,6 +3,8 @@
|
|||||||
interface ImportMetaEnv {
|
interface ImportMetaEnv {
|
||||||
readonly VITE_API_URL: string
|
readonly VITE_API_URL: string
|
||||||
readonly VITE_APP_PASSWORD: string
|
readonly VITE_APP_PASSWORD: string
|
||||||
|
readonly VITE_UPLOAD_ENABLED: string
|
||||||
|
readonly VITE_DELETE_ENABLED: string
|
||||||
}
|
}
|
||||||
|
|
||||||
interface ImportMeta {
|
interface ImportMeta {
|
||||||
|
|||||||
@@ -0,0 +1,10 @@
|
|||||||
|
/**
|
||||||
|
* Read a VITE_ env variable.
|
||||||
|
* At runtime in Docker, values come from window.__env__ (injected by docker-entrypoint.sh).
|
||||||
|
* At build time (dev / CI), values come from import.meta.env.
|
||||||
|
*/
|
||||||
|
export function env(key: string): string | undefined {
|
||||||
|
const runtime = (window as Record<string, any>).__env__?.[key]
|
||||||
|
if (runtime) return runtime
|
||||||
|
return (import.meta as any).env?.[key]
|
||||||
|
}
|
||||||
+16
-1
@@ -4,6 +4,21 @@ import App from './App.vue'
|
|||||||
import router from './router'
|
import router from './router'
|
||||||
|
|
||||||
const app = createApp(App)
|
const app = createApp(App)
|
||||||
app.use(createPinia())
|
const pinia = createPinia()
|
||||||
|
app.use(pinia)
|
||||||
app.use(router)
|
app.use(router)
|
||||||
|
|
||||||
|
// Verify any session restored from sessionStorage is still valid.
|
||||||
|
// If the backend rejects the credentials (e.g. password changed), clear them
|
||||||
|
// before the router guard fires so the user lands on /login cleanly.
|
||||||
|
import { useAuthStore } from '@/stores/authStore'
|
||||||
|
import { api } from '@/services/api'
|
||||||
|
|
||||||
|
const auth = useAuthStore()
|
||||||
|
if (auth.isAuthenticated) {
|
||||||
|
api.get('/auth/check').catch(() => {
|
||||||
|
auth.clearCredentials()
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
app.mount('#app')
|
app.mount('#app')
|
||||||
|
|||||||
@@ -1,11 +1,19 @@
|
|||||||
import { createRouter, createWebHistory } from 'vue-router'
|
import { createRouter, createWebHistory } from 'vue-router'
|
||||||
|
import { useAuthStore } from '@/stores/authStore'
|
||||||
|
import LoginView from '@/views/LoginView.vue'
|
||||||
import UploadView from '@/views/UploadView.vue'
|
import UploadView from '@/views/UploadView.vue'
|
||||||
import TopicsView from '@/views/TopicsView.vue'
|
import TopicsView from '@/views/TopicsView.vue'
|
||||||
import ChatView from '@/views/ChatView.vue'
|
import ChatView from '@/views/ChatView.vue'
|
||||||
|
import BookReaderView from '@/views/BookReaderView.vue'
|
||||||
|
|
||||||
const router = createRouter({
|
const router = createRouter({
|
||||||
history: createWebHistory(import.meta.env.BASE_URL),
|
history: createWebHistory(import.meta.env.BASE_URL),
|
||||||
routes: [
|
routes: [
|
||||||
|
{
|
||||||
|
path: '/login',
|
||||||
|
name: 'login',
|
||||||
|
component: LoginView
|
||||||
|
},
|
||||||
{
|
{
|
||||||
path: '/',
|
path: '/',
|
||||||
name: 'upload',
|
name: 'upload',
|
||||||
@@ -20,8 +28,20 @@ const router = createRouter({
|
|||||||
path: '/chat',
|
path: '/chat',
|
||||||
name: 'chat',
|
name: 'chat',
|
||||||
component: ChatView
|
component: ChatView
|
||||||
|
},
|
||||||
|
{
|
||||||
|
path: '/books/:id/read',
|
||||||
|
name: 'book-reader',
|
||||||
|
component: BookReaderView
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
})
|
})
|
||||||
|
|
||||||
|
router.beforeEach((to) => {
|
||||||
|
const auth = useAuthStore()
|
||||||
|
if (to.name !== 'login' && !auth.isAuthenticated) {
|
||||||
|
return { name: 'login' }
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
export default router
|
export default router
|
||||||
|
|||||||
@@ -1,20 +1,30 @@
|
|||||||
import axios from 'axios'
|
import axios from 'axios'
|
||||||
|
import { useAuthStore } from '@/stores/authStore'
|
||||||
|
import { env } from '@/env'
|
||||||
|
|
||||||
export const api = axios.create({
|
export const api = axios.create({
|
||||||
baseURL: import.meta.env.VITE_API_URL ?? '/api/v1',
|
baseURL: env('VITE_API_URL') ?? '/api/v1',
|
||||||
auth: {
|
|
||||||
username: 'neurosurgeon',
|
|
||||||
password: import.meta.env.VITE_APP_PASSWORD ?? 'changeme'
|
|
||||||
},
|
|
||||||
headers: {
|
headers: {
|
||||||
'Content-Type': 'application/json'
|
'Content-Type': 'application/json'
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
// Response interceptor for error normalisation
|
api.interceptors.request.use((config) => {
|
||||||
|
const auth = useAuthStore()
|
||||||
|
if (auth.username && auth.password) {
|
||||||
|
config.auth = { username: auth.username, password: auth.password }
|
||||||
|
}
|
||||||
|
return config
|
||||||
|
})
|
||||||
|
|
||||||
api.interceptors.response.use(
|
api.interceptors.response.use(
|
||||||
(response) => response,
|
(response) => response,
|
||||||
(error) => {
|
(error) => {
|
||||||
|
if (error.response?.status === 401) {
|
||||||
|
useAuthStore().clearCredentials()
|
||||||
|
window.location.href = '/login'
|
||||||
|
return Promise.reject(new Error('Session expired. Please sign in again.'))
|
||||||
|
}
|
||||||
const message =
|
const message =
|
||||||
error.response?.data?.error ??
|
error.response?.data?.error ??
|
||||||
error.message ??
|
error.message ??
|
||||||
|
|||||||
@@ -0,0 +1,28 @@
|
|||||||
|
import { defineStore } from 'pinia'
|
||||||
|
import { ref, computed } from 'vue'
|
||||||
|
|
||||||
|
const SESSION_KEY = 'auth'
|
||||||
|
|
||||||
|
export const useAuthStore = defineStore('auth', () => {
|
||||||
|
const stored = sessionStorage.getItem(SESSION_KEY)
|
||||||
|
const parsed = stored ? (JSON.parse(stored) as { username: string; password: string }) : null
|
||||||
|
|
||||||
|
const username = ref<string | null>(parsed?.username ?? null)
|
||||||
|
const password = ref<string | null>(parsed?.password ?? null)
|
||||||
|
|
||||||
|
const isAuthenticated = computed(() => !!username.value && !!password.value)
|
||||||
|
|
||||||
|
function setCredentials(u: string, p: string) {
|
||||||
|
username.value = u
|
||||||
|
password.value = p
|
||||||
|
sessionStorage.setItem(SESSION_KEY, JSON.stringify({ username: u, password: p }))
|
||||||
|
}
|
||||||
|
|
||||||
|
function clearCredentials() {
|
||||||
|
username.value = null
|
||||||
|
password.value = null
|
||||||
|
sessionStorage.removeItem(SESSION_KEY)
|
||||||
|
}
|
||||||
|
|
||||||
|
return { username, password, isAuthenticated, setCredentials, clearCredentials }
|
||||||
|
})
|
||||||
@@ -4,8 +4,10 @@ import { api } from '@/services/api'
|
|||||||
|
|
||||||
export interface ChatSource {
|
export interface ChatSource {
|
||||||
type: 'TEXT' | 'FIGURE'
|
type: 'TEXT' | 'FIGURE'
|
||||||
|
bookId?: string
|
||||||
bookTitle: string
|
bookTitle: string
|
||||||
page: number | null
|
page: number | null
|
||||||
|
refLabel?: string
|
||||||
// TEXT-specific
|
// TEXT-specific
|
||||||
chunkText?: string
|
chunkText?: string
|
||||||
// FIGURE-specific
|
// FIGURE-specific
|
||||||
|
|||||||
@@ -10,11 +10,14 @@ export interface Topic {
|
|||||||
}
|
}
|
||||||
|
|
||||||
export interface SourceReference {
|
export interface SourceReference {
|
||||||
|
bookId: string | null
|
||||||
bookTitle: string
|
bookTitle: string
|
||||||
page: number | null
|
page: number | null
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface TopicSummary {
|
export interface TopicSummary {
|
||||||
|
id: string
|
||||||
|
summaryNumber: number
|
||||||
topicId: string
|
topicId: string
|
||||||
topicName: string
|
topicName: string
|
||||||
summary: string
|
summary: string
|
||||||
@@ -22,12 +25,20 @@ export interface TopicSummary {
|
|||||||
generatedAt: string
|
generatedAt: string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export interface SavedSummaryItem {
|
||||||
|
id: string
|
||||||
|
summaryNumber: number
|
||||||
|
generatedAt: string
|
||||||
|
}
|
||||||
|
|
||||||
export const useTopicStore = defineStore('topics', () => {
|
export const useTopicStore = defineStore('topics', () => {
|
||||||
const topics = ref<Topic[]>([])
|
const topics = ref<Topic[]>([])
|
||||||
const activeSummary = ref<TopicSummary | null>(null)
|
const activeSummary = ref<TopicSummary | null>(null)
|
||||||
const activeSummaryTopicId = ref<string | null>(null)
|
const activeSummaryTopicId = ref<string | null>(null)
|
||||||
|
const summaryList = ref<SavedSummaryItem[]>([])
|
||||||
const loading = ref(false)
|
const loading = ref(false)
|
||||||
const summaryLoading = ref(false)
|
const summaryLoading = ref(false)
|
||||||
|
const summaryListLoading = ref(false)
|
||||||
const error = ref<string | null>(null)
|
const error = ref<string | null>(null)
|
||||||
|
|
||||||
async function fetchTopics() {
|
async function fetchTopics() {
|
||||||
@@ -43,6 +54,36 @@ export const useTopicStore = defineStore('topics', () => {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function fetchSummaries(topicId: string) {
|
||||||
|
summaryListLoading.value = true
|
||||||
|
summaryList.value = []
|
||||||
|
error.value = null
|
||||||
|
try {
|
||||||
|
const response = await api.get<SavedSummaryItem[]>(`/topics/${topicId}/summaries`)
|
||||||
|
summaryList.value = response.data
|
||||||
|
} catch (err: any) {
|
||||||
|
error.value = err.message
|
||||||
|
} finally {
|
||||||
|
summaryListLoading.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchSummaryDetail(topicId: string, summaryId: string): Promise<TopicSummary | null> {
|
||||||
|
summaryLoading.value = true
|
||||||
|
activeSummary.value = null
|
||||||
|
error.value = null
|
||||||
|
try {
|
||||||
|
const response = await api.get<TopicSummary>(`/topics/${topicId}/summaries/${summaryId}`)
|
||||||
|
activeSummary.value = response.data
|
||||||
|
return response.data
|
||||||
|
} catch (err: any) {
|
||||||
|
error.value = err.message
|
||||||
|
return null
|
||||||
|
} finally {
|
||||||
|
summaryLoading.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async function generateSummary(topicId: string): Promise<TopicSummary | null> {
|
async function generateSummary(topicId: string): Promise<TopicSummary | null> {
|
||||||
summaryLoading.value = true
|
summaryLoading.value = true
|
||||||
activeSummaryTopicId.value = topicId
|
activeSummaryTopicId.value = topicId
|
||||||
@@ -65,10 +106,14 @@ export const useTopicStore = defineStore('topics', () => {
|
|||||||
topics,
|
topics,
|
||||||
activeSummary,
|
activeSummary,
|
||||||
activeSummaryTopicId,
|
activeSummaryTopicId,
|
||||||
|
summaryList,
|
||||||
loading,
|
loading,
|
||||||
summaryLoading,
|
summaryLoading,
|
||||||
|
summaryListLoading,
|
||||||
error,
|
error,
|
||||||
fetchTopics,
|
fetchTopics,
|
||||||
|
fetchSummaries,
|
||||||
|
fetchSummaryDetail,
|
||||||
generateSummary
|
generateSummary
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -0,0 +1,335 @@
|
|||||||
|
<template>
|
||||||
|
<div class="reader-view">
|
||||||
|
<!-- Header -->
|
||||||
|
<div class="reader-header">
|
||||||
|
<router-link to="/" class="back-link">← Library</router-link>
|
||||||
|
<div class="reader-title">
|
||||||
|
<h1 class="book-title">{{ book?.title ?? 'Loading…' }}</h1>
|
||||||
|
</div>
|
||||||
|
<div class="page-nav">
|
||||||
|
<button class="nav-btn" :disabled="currentPage <= 1" @click="goTo(currentPage - 1)">←</button>
|
||||||
|
<form class="page-jump" @submit.prevent="onJump">
|
||||||
|
<input
|
||||||
|
v-model.number="jumpInput"
|
||||||
|
type="number"
|
||||||
|
:min="1"
|
||||||
|
:max="book?.pageCount ?? 1"
|
||||||
|
class="page-input"
|
||||||
|
/>
|
||||||
|
<span class="page-sep">/ {{ book?.pageCount ?? '…' }}</span>
|
||||||
|
</form>
|
||||||
|
<button class="nav-btn" :disabled="!book || currentPage >= book.pageCount!" @click="goTo(currentPage + 1)">→</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Content -->
|
||||||
|
<div class="reader-body">
|
||||||
|
<div v-if="loading" class="reader-loading">
|
||||||
|
<div class="spinner spinner-dark" style="width:28px;height:28px;margin:0 auto 0.75rem;"></div>
|
||||||
|
<p>Loading page {{ currentPage }}…</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-else-if="error" class="reader-error card">
|
||||||
|
<strong>Could not load page {{ currentPage }}</strong><br />
|
||||||
|
{{ error }}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-else class="reader-content card">
|
||||||
|
<div class="markdown-body" v-html="renderedHtml"></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<script setup lang="ts">
|
||||||
|
import { ref, watch, onMounted } from 'vue'
|
||||||
|
import { useRoute } from 'vue-router'
|
||||||
|
import { api } from '@/services/api'
|
||||||
|
import { useBookStore } from '@/stores/bookStore'
|
||||||
|
import type { Book } from '@/stores/bookStore'
|
||||||
|
|
||||||
|
const route = useRoute()
|
||||||
|
const bookStore = useBookStore()
|
||||||
|
|
||||||
|
const bookId = route.params.id as string
|
||||||
|
const book = ref<Book | null>(null)
|
||||||
|
const currentPage = ref(1)
|
||||||
|
const jumpInput = ref(1)
|
||||||
|
const loading = ref(false)
|
||||||
|
const error = ref<string | null>(null)
|
||||||
|
const renderedHtml = ref('')
|
||||||
|
|
||||||
|
// Blob URLs created this session — revoked on next page load
|
||||||
|
let activeBlobUrls: string[] = []
|
||||||
|
|
||||||
|
onMounted(async () => {
|
||||||
|
book.value = bookStore.books.find(b => b.id === bookId) ?? null
|
||||||
|
if (!book.value) {
|
||||||
|
try {
|
||||||
|
const res = await api.get<Book>(`/books/${bookId}`)
|
||||||
|
book.value = res.data
|
||||||
|
} catch {
|
||||||
|
error.value = 'Book not found.'
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
await loadPage(1)
|
||||||
|
})
|
||||||
|
|
||||||
|
watch(currentPage, (page) => {
|
||||||
|
jumpInput.value = page
|
||||||
|
loadPage(page)
|
||||||
|
})
|
||||||
|
|
||||||
|
async function goTo(page: number) {
|
||||||
|
if (!book.value) return
|
||||||
|
const clamped = Math.max(1, Math.min(page, book.value.pageCount ?? 1))
|
||||||
|
if (clamped !== currentPage.value) {
|
||||||
|
currentPage.value = clamped
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function onJump() {
|
||||||
|
goTo(jumpInput.value)
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadPage(page: number) {
|
||||||
|
loading.value = true
|
||||||
|
error.value = null
|
||||||
|
renderedHtml.value = ''
|
||||||
|
|
||||||
|
// Revoke previous blob URLs to free memory
|
||||||
|
activeBlobUrls.forEach(u => URL.revokeObjectURL(u))
|
||||||
|
activeBlobUrls = []
|
||||||
|
|
||||||
|
try {
|
||||||
|
const res = await api.get<string>(`/books/${bookId}/pages/${page}/html`, {
|
||||||
|
headers: { Accept: 'text/html' },
|
||||||
|
responseType: 'text'
|
||||||
|
})
|
||||||
|
let html = await resolveImages(res.data)
|
||||||
|
renderedHtml.value = html
|
||||||
|
} catch (e: any) {
|
||||||
|
error.value = e.message ?? 'Failed to load page.'
|
||||||
|
} finally {
|
||||||
|
loading.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Finds <img src="/api/v1/figures/..."> in the HTML, fetches each image
|
||||||
|
* through the authenticated axios instance, and replaces the src with a
|
||||||
|
* temporary blob URL so the browser can render it without re-authenticating.
|
||||||
|
*/
|
||||||
|
async function resolveImages(html: string): Promise<string> {
|
||||||
|
const srcPattern = /src="(\/api\/v1\/figures\/[^"]+)"/g
|
||||||
|
const matches = [...html.matchAll(srcPattern)]
|
||||||
|
if (matches.length === 0) return html
|
||||||
|
|
||||||
|
const unique = [...new Set(matches.map(m => m[1]))]
|
||||||
|
const blobMap: Record<string, string> = {}
|
||||||
|
|
||||||
|
await Promise.all(
|
||||||
|
unique.map(async (src) => {
|
||||||
|
try {
|
||||||
|
const res = await api.get(src.replace(/^\/api\/v1/, ''), { responseType: 'blob' })
|
||||||
|
const blobUrl = URL.createObjectURL(res.data)
|
||||||
|
activeBlobUrls.push(blobUrl)
|
||||||
|
blobMap[src] = blobUrl
|
||||||
|
} catch {
|
||||||
|
// leave original src — browser will attempt (and likely fail silently)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
)
|
||||||
|
|
||||||
|
return html.replace(/src="(\/api\/v1\/figures\/[^"]+)"/g, (_, src) =>
|
||||||
|
blobMap[src] ? `src="${blobMap[src]}"` : `src="${src}"`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<style scoped>
|
||||||
|
.reader-view {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 1rem;
|
||||||
|
max-width: 860px;
|
||||||
|
margin: 0 auto;
|
||||||
|
flex: 1;
|
||||||
|
min-height: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.reader-header {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 1rem;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.back-link {
|
||||||
|
color: #3182ce;
|
||||||
|
text-decoration: none;
|
||||||
|
font-size: 0.9rem;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
.back-link:hover { text-decoration: underline; }
|
||||||
|
|
||||||
|
.reader-title {
|
||||||
|
flex: 1;
|
||||||
|
min-width: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.book-title {
|
||||||
|
font-size: 1.1rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: #1a365d;
|
||||||
|
white-space: nowrap;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-nav {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.nav-btn {
|
||||||
|
width: 2rem;
|
||||||
|
height: 2rem;
|
||||||
|
border: 1px solid #cbd5e0;
|
||||||
|
border-radius: 6px;
|
||||||
|
background: #fff;
|
||||||
|
cursor: pointer;
|
||||||
|
font-size: 1rem;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
transition: background 0.15s;
|
||||||
|
}
|
||||||
|
.nav-btn:hover:not(:disabled) { background: #ebf8ff; border-color: #3182ce; }
|
||||||
|
.nav-btn:disabled { opacity: 0.4; cursor: not-allowed; }
|
||||||
|
|
||||||
|
.page-jump {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.35rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-input {
|
||||||
|
width: 3.5rem;
|
||||||
|
text-align: center;
|
||||||
|
border: 1px solid #cbd5e0;
|
||||||
|
border-radius: 6px;
|
||||||
|
padding: 0.25rem 0.4rem;
|
||||||
|
font-size: 0.9rem;
|
||||||
|
color: #2d3748;
|
||||||
|
}
|
||||||
|
.page-input:focus { outline: none; border-color: #3182ce; }
|
||||||
|
|
||||||
|
.page-sep {
|
||||||
|
font-size: 0.85rem;
|
||||||
|
color: #718096;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.reader-body {
|
||||||
|
flex: 1;
|
||||||
|
min-height: 0;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
}
|
||||||
|
|
||||||
|
.reader-loading {
|
||||||
|
text-align: center;
|
||||||
|
padding: 3rem;
|
||||||
|
color: #718096;
|
||||||
|
}
|
||||||
|
|
||||||
|
.reader-error {
|
||||||
|
padding: 1.25rem;
|
||||||
|
background: #fff5f5;
|
||||||
|
border: 1px solid #fed7d7;
|
||||||
|
color: #742a2a;
|
||||||
|
border-radius: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.reader-content {
|
||||||
|
flex: 1;
|
||||||
|
min-height: 0;
|
||||||
|
overflow-y: auto;
|
||||||
|
padding: 2rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Markdown rendering */
|
||||||
|
.markdown-body {
|
||||||
|
font-size: 0.95rem;
|
||||||
|
line-height: 1.75;
|
||||||
|
color: #2d3748;
|
||||||
|
}
|
||||||
|
|
||||||
|
.markdown-body :deep(h1),
|
||||||
|
.markdown-body :deep(h2),
|
||||||
|
.markdown-body :deep(h3) {
|
||||||
|
color: #1a365d;
|
||||||
|
font-weight: 600;
|
||||||
|
margin: 1.5rem 0 0.75rem;
|
||||||
|
}
|
||||||
|
.markdown-body :deep(h2) { font-size: 1.15rem; border-bottom: 1px solid #e2e8f0; padding-bottom: 0.4rem; }
|
||||||
|
.markdown-body :deep(h3) { font-size: 1rem; }
|
||||||
|
|
||||||
|
.markdown-body :deep(p) { margin: 0.75rem 0; }
|
||||||
|
|
||||||
|
.markdown-body :deep(img) {
|
||||||
|
max-width: 100%;
|
||||||
|
border-radius: 6px;
|
||||||
|
display: block;
|
||||||
|
margin: 1rem auto;
|
||||||
|
box-shadow: 0 1px 4px rgba(0,0,0,0.12);
|
||||||
|
}
|
||||||
|
|
||||||
|
.markdown-body :deep(ul),
|
||||||
|
.markdown-body :deep(ol) {
|
||||||
|
padding-left: 1.5rem;
|
||||||
|
margin: 0.75rem 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.markdown-body :deep(code) {
|
||||||
|
background: #f7fafc;
|
||||||
|
border: 1px solid #e2e8f0;
|
||||||
|
border-radius: 3px;
|
||||||
|
padding: 0.1em 0.35em;
|
||||||
|
font-size: 0.88em;
|
||||||
|
}
|
||||||
|
|
||||||
|
.markdown-body :deep(blockquote) {
|
||||||
|
border-left: 3px solid #3182ce;
|
||||||
|
padding-left: 1rem;
|
||||||
|
color: #4a5568;
|
||||||
|
margin: 0.75rem 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.markdown-body :deep(table) {
|
||||||
|
width: 100%;
|
||||||
|
border-collapse: collapse;
|
||||||
|
font-size: 0.9em;
|
||||||
|
margin: 1rem 0;
|
||||||
|
}
|
||||||
|
.markdown-body :deep(th),
|
||||||
|
.markdown-body :deep(td) {
|
||||||
|
border: 1px solid #e2e8f0;
|
||||||
|
padding: 0.4rem 0.75rem;
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
.markdown-body :deep(th) { background: #f7fafc; font-weight: 600; }
|
||||||
|
|
||||||
|
@media (max-width: 768px) {
|
||||||
|
.reader-view {
|
||||||
|
max-width: 100%;
|
||||||
|
}
|
||||||
|
|
||||||
|
.reader-content {
|
||||||
|
padding: 1rem;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</style>
|
||||||
+125
-123
@@ -3,27 +3,10 @@
|
|||||||
<h1 class="page-title">Knowledge Chat</h1>
|
<h1 class="page-title">Knowledge Chat</h1>
|
||||||
<p class="page-subtitle">Ask questions grounded in your uploaded medical textbooks.</p>
|
<p class="page-subtitle">Ask questions grounded in your uploaded medical textbooks.</p>
|
||||||
|
|
||||||
<!-- Step 1: Topic Selection -->
|
<!-- Session selection -->
|
||||||
<div v-if="!chatStore.session && !selectedTopic" class="topic-selection">
|
<div v-if="!chatStore.session" class="session-setup card">
|
||||||
<h2 class="section-title">Select a Topic</h2>
|
|
||||||
<div class="topic-grid">
|
|
||||||
<button
|
|
||||||
v-for="topic in topicStore.topics"
|
|
||||||
:key="topic.id"
|
|
||||||
:class="['topic-tile', { 'topic-tile-freeform': topic.id === 'free-form' }]"
|
|
||||||
@click="handleTopicSelect(topic)"
|
|
||||||
>
|
|
||||||
<span class="topic-tile-name">{{ topic.name }}</span>
|
|
||||||
<span v-if="topic.id === 'free-form'" class="topic-tile-hint">Any neurosurgery question</span>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Step 2: Topic selected — previous sessions + new chat -->
|
|
||||||
<div v-else-if="!chatStore.session && selectedTopic" class="session-setup card">
|
|
||||||
<div class="setup-header">
|
<div class="setup-header">
|
||||||
<button class="btn-back" @click="handleBack">← Topics</button>
|
<h2 class="section-title">Free-form Chat</h2>
|
||||||
<h2 class="section-title">{{ selectedTopic.name }}</h2>
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div v-if="chatStore.error" class="error-banner">{{ chatStore.error }}</div>
|
<div v-if="chatStore.error" class="error-banner">{{ chatStore.error }}</div>
|
||||||
@@ -71,56 +54,74 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- Messages Area -->
|
<!-- Chat + Reader split -->
|
||||||
<div class="messages-container" ref="messagesContainer">
|
<div class="chat-reader-split">
|
||||||
<div v-if="chatStore.loading && chatStore.messages.length === 0" class="empty-state">
|
<!-- Messages + Input -->
|
||||||
<div class="spinner spinner-dark" style="width:32px;height:32px;margin:0 auto 1rem;"></div>
|
<div class="chat-column">
|
||||||
<p class="empty-state-text">Loading messages...</p>
|
<!-- Messages Area -->
|
||||||
</div>
|
<div class="messages-container" ref="messagesContainer">
|
||||||
|
<div v-if="chatStore.loading && chatStore.messages.length === 0" class="empty-state">
|
||||||
|
<div class="spinner spinner-dark" style="width:32px;height:32px;margin:0 auto 1rem;"></div>
|
||||||
|
<p class="empty-state-text">Loading messages...</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
<div v-else-if="chatStore.messages.length === 0" class="empty-state">
|
<div v-else-if="chatStore.messages.length === 0" class="empty-state">
|
||||||
<div class="empty-state-icon">💬</div>
|
<div class="empty-state-icon">💬</div>
|
||||||
<p class="empty-state-text">No messages yet</p>
|
<p class="empty-state-text">No messages yet</p>
|
||||||
<p class="empty-state-hint">Ask a question about the uploaded books below.</p>
|
<p class="empty-state-hint">Ask a question about the uploaded books below.</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div v-else class="messages-list">
|
<div v-else class="messages-list">
|
||||||
<ChatMessage
|
<ChatMessage
|
||||||
v-for="message in chatStore.messages"
|
v-for="message in chatStore.messages"
|
||||||
:key="message.id"
|
:key="message.id"
|
||||||
:message="message"
|
:message="message"
|
||||||
/>
|
@open-source="handleOpenSource"
|
||||||
<div v-if="chatStore.sending" class="typing-indicator">
|
/>
|
||||||
<div class="typing-bubble">
|
<div v-if="chatStore.sending" class="typing-indicator">
|
||||||
<span></span><span></span><span></span>
|
<div class="typing-bubble">
|
||||||
|
<span></span><span></span><span></span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Input Area -->
|
<!-- Input Area -->
|
||||||
<div class="input-area card">
|
<div class="input-area card">
|
||||||
<div v-if="chatStore.error" class="error-banner">{{ chatStore.error }}</div>
|
<div v-if="chatStore.error" class="error-banner">{{ chatStore.error }}</div>
|
||||||
<div class="input-row">
|
<div class="input-row">
|
||||||
<textarea
|
<textarea
|
||||||
v-model="inputText"
|
v-model="inputText"
|
||||||
class="message-input"
|
class="message-input"
|
||||||
placeholder="Ask a question about your uploaded books..."
|
placeholder="Ask a question about your uploaded books..."
|
||||||
rows="2"
|
rows="2"
|
||||||
:disabled="chatStore.sending"
|
:disabled="chatStore.sending"
|
||||||
@keydown.enter.exact.prevent="handleSend"
|
@keydown.enter.exact.prevent="handleSend"
|
||||||
@keydown.enter.shift.exact="inputText += '\n'"
|
@keydown.enter.shift.exact="inputText += '\n'"
|
||||||
></textarea>
|
></textarea>
|
||||||
<button
|
<button
|
||||||
class="btn btn-primary send-btn"
|
class="btn btn-primary send-btn"
|
||||||
:disabled="!inputText.trim() || chatStore.sending"
|
:disabled="!inputText.trim() || chatStore.sending"
|
||||||
@click="handleSend"
|
@click="handleSend"
|
||||||
>
|
>
|
||||||
<span v-if="chatStore.sending" class="spinner"></span>
|
<span v-if="chatStore.sending" class="spinner"></span>
|
||||||
<span v-else>Send</span>
|
<span v-else>Send</span>
|
||||||
</button>
|
</button>
|
||||||
|
</div>
|
||||||
|
<p class="input-hint">Press Enter to send, Shift+Enter for new line.</p>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<p class="input-hint">Press Enter to send, Shift+Enter for new line.</p>
|
|
||||||
|
<!-- Inline book reader panel -->
|
||||||
|
<BookPagePanel
|
||||||
|
v-if="readerPanel"
|
||||||
|
:book-id="readerPanel.bookId"
|
||||||
|
:page="readerPanel.page"
|
||||||
|
:book-title="readerPanel.bookTitle"
|
||||||
|
class="reader-panel"
|
||||||
|
@close="readerPanel = null"
|
||||||
|
@navigate="(p) => readerPanel && (readerPanel.page = p)"
|
||||||
|
/>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -130,8 +131,10 @@
|
|||||||
import { ref, nextTick, onMounted, watch, inject } from 'vue'
|
import { ref, nextTick, onMounted, watch, inject } from 'vue'
|
||||||
import { useChatStore } from '@/stores/chatStore'
|
import { useChatStore } from '@/stores/chatStore'
|
||||||
import { useTopicStore } from '@/stores/topicStore'
|
import { useTopicStore } from '@/stores/topicStore'
|
||||||
|
import { useBookStore } from '@/stores/bookStore'
|
||||||
import type { ChatSession } from '@/stores/chatStore'
|
import type { ChatSession } from '@/stores/chatStore'
|
||||||
import ChatMessage from '@/components/ChatMessage.vue'
|
import ChatMessage from '@/components/ChatMessage.vue'
|
||||||
|
import BookPagePanel from '@/components/BookPagePanel.vue'
|
||||||
|
|
||||||
interface Topic {
|
interface Topic {
|
||||||
id: string
|
id: string
|
||||||
@@ -142,6 +145,7 @@ interface Topic {
|
|||||||
|
|
||||||
const chatStore = useChatStore()
|
const chatStore = useChatStore()
|
||||||
const topicStore = useTopicStore()
|
const topicStore = useTopicStore()
|
||||||
|
const bookStore = useBookStore()
|
||||||
const showToast = inject<(msg: string, type?: 'error' | 'success') => void>('showToast')
|
const showToast = inject<(msg: string, type?: 'error' | 'success') => void>('showToast')
|
||||||
|
|
||||||
const selectedTopic = ref<Topic | null>(null)
|
const selectedTopic = ref<Topic | null>(null)
|
||||||
@@ -150,10 +154,22 @@ const loadingTopicSessions = ref(false)
|
|||||||
const inputText = ref('')
|
const inputText = ref('')
|
||||||
const messagesContainer = ref<HTMLElement | null>(null)
|
const messagesContainer = ref<HTMLElement | null>(null)
|
||||||
|
|
||||||
|
interface ReaderPanel { bookId: string; page: number; bookTitle?: string }
|
||||||
|
const readerPanel = ref<ReaderPanel | null>(null)
|
||||||
|
|
||||||
|
function handleOpenSource(bookId: string, page: number) {
|
||||||
|
const book = bookStore.books.find(b => b.id === bookId)
|
||||||
|
readerPanel.value = { bookId, page, bookTitle: book?.title }
|
||||||
|
}
|
||||||
|
|
||||||
onMounted(async () => {
|
onMounted(async () => {
|
||||||
if (topicStore.topics.length === 0) {
|
if (topicStore.topics.length === 0) {
|
||||||
await topicStore.fetchTopics()
|
await topicStore.fetchTopics()
|
||||||
}
|
}
|
||||||
|
const freeForm = topicStore.topics.find((t) => t.id === 'free-form')
|
||||||
|
if (freeForm) {
|
||||||
|
await handleTopicSelect(freeForm)
|
||||||
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
watch(
|
watch(
|
||||||
@@ -189,11 +205,6 @@ async function handleTopicSelect(topic: Topic) {
|
|||||||
loadingTopicSessions.value = false
|
loadingTopicSessions.value = false
|
||||||
}
|
}
|
||||||
|
|
||||||
function handleBack() {
|
|
||||||
selectedTopic.value = null
|
|
||||||
topicSessions.value = []
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleNewChat() {
|
async function handleNewChat() {
|
||||||
const ok = await chatStore.createSession(selectedTopic.value!.id)
|
const ok = await chatStore.createSession(selectedTopic.value!.id)
|
||||||
if (!ok) {
|
if (!ok) {
|
||||||
@@ -207,9 +218,7 @@ async function handleResumeSession(session: ChatSession) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function handleLeaveSession() {
|
function handleLeaveSession() {
|
||||||
// Leave without deleting — session stays in DB and will appear in "Previous Chats"
|
|
||||||
chatStore.leaveSession()
|
chatStore.leaveSession()
|
||||||
// Refresh the sessions list for the current topic
|
|
||||||
if (selectedTopic.value) {
|
if (selectedTopic.value) {
|
||||||
loadingTopicSessions.value = true
|
loadingTopicSessions.value = true
|
||||||
chatStore.fetchSessionsByTopic(selectedTopic.value.id).then((sessions) => {
|
chatStore.fetchSessionsByTopic(selectedTopic.value.id).then((sessions) => {
|
||||||
@@ -231,12 +240,6 @@ async function handleSend() {
|
|||||||
</script>
|
</script>
|
||||||
|
|
||||||
<style scoped>
|
<style scoped>
|
||||||
.topic-selection {
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
gap: 1.25rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.section-title {
|
.section-title {
|
||||||
font-size: 1.1rem;
|
font-size: 1.1rem;
|
||||||
font-weight: 600;
|
font-weight: 600;
|
||||||
@@ -244,52 +247,6 @@ async function handleSend() {
|
|||||||
margin: 0;
|
margin: 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
.topic-grid {
|
|
||||||
display: grid;
|
|
||||||
grid-template-columns: repeat(auto-fill, minmax(220px, 1fr));
|
|
||||||
gap: 0.75rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.topic-tile {
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
align-items: flex-start;
|
|
||||||
gap: 0.25rem;
|
|
||||||
padding: 1rem 1.1rem;
|
|
||||||
background: white;
|
|
||||||
border: 1px solid #e2e8f0;
|
|
||||||
border-radius: 8px;
|
|
||||||
cursor: pointer;
|
|
||||||
text-align: left;
|
|
||||||
transition: border-color 0.15s, box-shadow 0.15s;
|
|
||||||
}
|
|
||||||
|
|
||||||
.topic-tile:hover {
|
|
||||||
border-color: #3182ce;
|
|
||||||
box-shadow: 0 2px 8px rgba(49, 130, 206, 0.15);
|
|
||||||
}
|
|
||||||
|
|
||||||
.topic-tile-freeform {
|
|
||||||
border-style: dashed;
|
|
||||||
border-color: #a0aec0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.topic-tile-freeform:hover {
|
|
||||||
border-color: #718096;
|
|
||||||
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.08);
|
|
||||||
}
|
|
||||||
|
|
||||||
.topic-tile-name {
|
|
||||||
font-size: 0.9rem;
|
|
||||||
font-weight: 600;
|
|
||||||
color: #2d3748;
|
|
||||||
}
|
|
||||||
|
|
||||||
.topic-tile-hint {
|
|
||||||
font-size: 0.78rem;
|
|
||||||
color: #a0aec0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.session-setup {
|
.session-setup {
|
||||||
max-width: 540px;
|
max-width: 540px;
|
||||||
}
|
}
|
||||||
@@ -381,6 +338,29 @@ async function handleSend() {
|
|||||||
min-height: 500px;
|
min-height: 500px;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.chat-reader-split {
|
||||||
|
display: flex;
|
||||||
|
flex: 1;
|
||||||
|
min-height: 0;
|
||||||
|
gap: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.chat-column {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
flex: 1;
|
||||||
|
min-width: 0;
|
||||||
|
gap: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.reader-panel {
|
||||||
|
width: 420px;
|
||||||
|
flex-shrink: 0;
|
||||||
|
border-radius: 10px;
|
||||||
|
margin-left: 1rem;
|
||||||
|
box-shadow: -2px 0 8px rgba(0, 0, 0, 0.07);
|
||||||
|
}
|
||||||
|
|
||||||
.session-bar {
|
.session-bar {
|
||||||
display: flex;
|
display: flex;
|
||||||
align-items: center;
|
align-items: center;
|
||||||
@@ -505,4 +485,26 @@ async function handleSend() {
|
|||||||
font-size: 0.875rem;
|
font-size: 0.875rem;
|
||||||
margin-bottom: 0.75rem;
|
margin-bottom: 0.75rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@media (max-width: 768px) {
|
||||||
|
.chat-layout {
|
||||||
|
height: auto;
|
||||||
|
min-height: unset;
|
||||||
|
}
|
||||||
|
|
||||||
|
.chat-reader-split {
|
||||||
|
flex-direction: column;
|
||||||
|
}
|
||||||
|
|
||||||
|
.chat-column {
|
||||||
|
min-height: 60vh;
|
||||||
|
}
|
||||||
|
|
||||||
|
.reader-panel {
|
||||||
|
width: 100%;
|
||||||
|
margin-left: 0;
|
||||||
|
margin-top: 1rem;
|
||||||
|
box-shadow: none;
|
||||||
|
}
|
||||||
|
}
|
||||||
</style>
|
</style>
|
||||||
|
|||||||
@@ -0,0 +1,183 @@
|
|||||||
|
<template>
|
||||||
|
<div class="login-wrapper">
|
||||||
|
<div class="login-card card">
|
||||||
|
<div class="login-header">
|
||||||
|
<span class="login-icon">🧠</span>
|
||||||
|
<h1 class="login-title">AI Teacher</h1>
|
||||||
|
<p class="login-subtitle">Neurosurgeon Learning Platform</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<form class="login-form" @submit.prevent="handleSubmit">
|
||||||
|
<div class="form-group">
|
||||||
|
<label for="username">Username</label>
|
||||||
|
<input
|
||||||
|
id="username"
|
||||||
|
v-model="username"
|
||||||
|
type="text"
|
||||||
|
autocomplete="username"
|
||||||
|
required
|
||||||
|
:disabled="loading"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="form-group">
|
||||||
|
<label for="password">Password</label>
|
||||||
|
<input
|
||||||
|
id="password"
|
||||||
|
v-model="password"
|
||||||
|
type="password"
|
||||||
|
autocomplete="current-password"
|
||||||
|
required
|
||||||
|
:disabled="loading"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-if="errorMessage" class="login-error">
|
||||||
|
{{ errorMessage }}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<button type="submit" class="btn btn-primary login-btn" :disabled="loading || !username || !password">
|
||||||
|
<span v-if="loading" class="spinner"></span>
|
||||||
|
<span v-else>Sign in</span>
|
||||||
|
</button>
|
||||||
|
</form>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<script setup lang="ts">
|
||||||
|
import { ref } from 'vue'
|
||||||
|
import { useRouter } from 'vue-router'
|
||||||
|
import { useAuthStore } from '@/stores/authStore'
|
||||||
|
import { api } from '@/services/api'
|
||||||
|
|
||||||
|
const router = useRouter()
|
||||||
|
const authStore = useAuthStore()
|
||||||
|
|
||||||
|
const username = ref('')
|
||||||
|
const password = ref('')
|
||||||
|
const loading = ref(false)
|
||||||
|
const errorMessage = ref('')
|
||||||
|
|
||||||
|
async function handleSubmit() {
|
||||||
|
errorMessage.value = ''
|
||||||
|
loading.value = true
|
||||||
|
|
||||||
|
authStore.setCredentials(username.value, password.value)
|
||||||
|
|
||||||
|
try {
|
||||||
|
await api.get('/auth/check')
|
||||||
|
router.push('/')
|
||||||
|
} catch {
|
||||||
|
authStore.clearCredentials()
|
||||||
|
errorMessage.value = 'Invalid username or password.'
|
||||||
|
} finally {
|
||||||
|
loading.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<style scoped>
|
||||||
|
.login-wrapper {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
min-height: 100vh;
|
||||||
|
background: #f0f4f8;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-card {
|
||||||
|
width: 100%;
|
||||||
|
max-width: 380px;
|
||||||
|
padding: 2rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-header {
|
||||||
|
text-align: center;
|
||||||
|
margin-bottom: 1.75rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-icon {
|
||||||
|
font-size: 2.5rem;
|
||||||
|
display: block;
|
||||||
|
margin-bottom: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-title {
|
||||||
|
font-size: 1.5rem;
|
||||||
|
font-weight: 700;
|
||||||
|
color: #1a365d;
|
||||||
|
margin-bottom: 0.25rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-subtitle {
|
||||||
|
font-size: 0.85rem;
|
||||||
|
color: #718096;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-form {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.form-group {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.35rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.form-group label {
|
||||||
|
font-size: 0.875rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: #4a5568;
|
||||||
|
}
|
||||||
|
|
||||||
|
.form-group input {
|
||||||
|
padding: 0.6rem 0.75rem;
|
||||||
|
border: 1px solid #cbd5e0;
|
||||||
|
border-radius: 6px;
|
||||||
|
font-size: 0.95rem;
|
||||||
|
outline: none;
|
||||||
|
transition: border-color 0.15s;
|
||||||
|
}
|
||||||
|
|
||||||
|
.form-group input:focus {
|
||||||
|
border-color: #3182ce;
|
||||||
|
box-shadow: 0 0 0 3px rgba(49, 130, 206, 0.15);
|
||||||
|
}
|
||||||
|
|
||||||
|
.form-group input:disabled {
|
||||||
|
background: #f7fafc;
|
||||||
|
color: #a0aec0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-error {
|
||||||
|
padding: 0.6rem 0.75rem;
|
||||||
|
background: #fed7d7;
|
||||||
|
color: #c53030;
|
||||||
|
border: 1px solid #fc8181;
|
||||||
|
border-radius: 6px;
|
||||||
|
font-size: 0.875rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-btn {
|
||||||
|
width: 100%;
|
||||||
|
justify-content: center;
|
||||||
|
padding: 0.7rem;
|
||||||
|
font-size: 0.95rem;
|
||||||
|
margin-top: 0.25rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (max-width: 768px) {
|
||||||
|
.login-wrapper {
|
||||||
|
align-items: flex-start;
|
||||||
|
padding-top: 2rem;
|
||||||
|
min-height: unset;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-card {
|
||||||
|
max-width: 100%;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</style>
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
<template>
|
<template>
|
||||||
<div class="topics-view">
|
<div class="topics-view">
|
||||||
<h1 class="page-title">Topics</h1>
|
<h1 class="page-title">Topics</h1>
|
||||||
<p class="page-subtitle">Select a topic to generate an AI-powered summary from uploaded books.</p>
|
<p class="page-subtitle">Select a topic to view or generate an AI-powered summary from uploaded books.</p>
|
||||||
|
|
||||||
<!-- Loading state -->
|
<!-- Loading state -->
|
||||||
<div v-if="topicStore.loading" class="empty-state">
|
<div v-if="topicStore.loading" class="empty-state">
|
||||||
@@ -18,83 +18,198 @@
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div v-else class="topics-layout">
|
<div v-else class="topics-layout">
|
||||||
<!-- Topic Grid -->
|
<div class="topics-main">
|
||||||
<div class="topic-grid">
|
|
||||||
<TopicCard
|
|
||||||
v-for="topic in topicStore.topics"
|
|
||||||
:key="topic.id"
|
|
||||||
:topic="topic"
|
|
||||||
:is-generating="topicStore.activeSummaryTopicId === topic.id"
|
|
||||||
@generate="handleGenerate"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Summary Panel -->
|
<!-- Summary history list -->
|
||||||
<div v-if="topicStore.summaryLoading" class="summary-panel card">
|
<div v-if="selectedTopicId" class="history-panel card">
|
||||||
<div class="summary-loading">
|
<div class="history-header">
|
||||||
<div class="spinner spinner-dark" style="width:36px;height:36px;margin:0 auto 1rem;"></div>
|
<span class="history-title">Saved summaries</span>
|
||||||
<p class="summary-loading-text">Generating summary from uploaded books...</p>
|
<button class="btn btn-primary btn-sm" :disabled="topicStore.summaryLoading" @click="handleGenerate(selectedTopicId!)">
|
||||||
<p class="summary-loading-hint">This may take up to 30 seconds.</p>
|
<span v-if="topicStore.summaryLoading" class="spinner" style="width:14px;height:14px;display:inline-block;vertical-align:middle;margin-right:4px;"></span>
|
||||||
</div>
|
Generate New
|
||||||
</div>
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
<div v-else-if="summaryError" class="summary-panel card summary-error">
|
<div v-if="topicStore.summaryListLoading" class="history-loading">
|
||||||
<h2 class="summary-topic-name">Summary Error</h2>
|
<div class="spinner spinner-dark" style="width:20px;height:20px;margin-right:8px;display:inline-block;vertical-align:middle;"></div>
|
||||||
<p class="error-text">{{ summaryError }}</p>
|
Loading...
|
||||||
<p v-if="isNoBooks" class="no-books-hint">
|
</div>
|
||||||
Please
|
|
||||||
<RouterLink to="/">upload and process at least one book</RouterLink>
|
|
||||||
first.
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div v-else-if="topicStore.activeSummary" class="summary-panel card">
|
<div v-else-if="topicStore.summaryList.length === 0" class="history-empty">
|
||||||
<div class="summary-header">
|
No summaries yet. Click "Generate New" to create one.
|
||||||
<h2 class="summary-topic-name">{{ topicStore.activeSummary.topicName }}</h2>
|
</div>
|
||||||
<span class="summary-timestamp">{{ formatDate(topicStore.activeSummary.generatedAt) }}</span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="summary-text">{{ topicStore.activeSummary.summary }}</div>
|
<div v-else class="history-list">
|
||||||
|
<button
|
||||||
<div v-if="topicStore.activeSummary.sources.length > 0" class="sources-section">
|
v-for="item in topicStore.summaryList"
|
||||||
<button class="sources-toggle" @click="showSources = !showSources">
|
:key="item.id"
|
||||||
Sources ({{ topicStore.activeSummary.sources.length }})
|
class="history-chip"
|
||||||
<span>{{ showSources ? '▲' : '▼' }}</span>
|
:class="{ 'history-chip--active': topicStore.activeSummary?.id === item.id }"
|
||||||
</button>
|
@click="handleLoadSummary(item)"
|
||||||
<div v-if="showSources" class="sources-list">
|
|
||||||
<div
|
|
||||||
v-for="(source, idx) in topicStore.activeSummary.sources"
|
|
||||||
:key="idx"
|
|
||||||
class="source-chip"
|
|
||||||
>
|
>
|
||||||
<span class="source-book">{{ source.bookTitle }}</span>
|
Summary #{{ item.summaryNumber }}
|
||||||
<span v-if="source.page" class="source-page">p. {{ source.page }}</span>
|
<span class="history-chip-date">· {{ formatDateShort(item.generatedAt) }}</span>
|
||||||
</div>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div v-else class="no-sources">
|
|
||||||
No source citations available for this summary.
|
<!-- Summary Panel -->
|
||||||
|
<div v-if="topicStore.summaryLoading" class="summary-panel card">
|
||||||
|
<div class="summary-loading">
|
||||||
|
<div class="spinner spinner-dark" style="width:36px;height:36px;margin:0 auto 1rem;"></div>
|
||||||
|
<p class="summary-loading-text">Generating summary from uploaded books...</p>
|
||||||
|
<p class="summary-loading-hint">This may take up to 30 seconds.</p>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
|
||||||
|
<div v-else-if="summaryError" class="summary-panel card summary-error">
|
||||||
|
<h2 class="summary-topic-name">Summary Error</h2>
|
||||||
|
<p class="error-text">{{ summaryError }}</p>
|
||||||
|
<p v-if="isNoBooks" class="no-books-hint">
|
||||||
|
Please
|
||||||
|
<RouterLink to="/">upload and process at least one book</RouterLink>
|
||||||
|
first.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-else-if="!topicStore.activeSummary" class="summary-panel card summary-placeholder">
|
||||||
|
<p class="summary-placeholder-text">
|
||||||
|
{{ selectedTopicId ? 'Select a saved summary or generate a new one.' : 'Select a topic to get started.' }}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-else class="summary-panel card">
|
||||||
|
<div class="summary-header">
|
||||||
|
<h2 class="summary-topic-name">{{ topicStore.activeSummary.topicName }}</h2>
|
||||||
|
<div class="summary-meta">
|
||||||
|
<span v-if="topicStore.activeSummary.summaryNumber" class="summary-number">
|
||||||
|
Summary #{{ topicStore.activeSummary.summaryNumber }}
|
||||||
|
</span>
|
||||||
|
<span class="summary-timestamp">{{ formatDate(topicStore.activeSummary.generatedAt) }}</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="summary-text summary-text--markdown" v-html="renderedSummary" @click="handleSummaryClick"></div>
|
||||||
|
|
||||||
|
<div ref="sourcesSection" v-if="topicStore.activeSummary.sources.length > 0" class="sources-section">
|
||||||
|
<button class="sources-toggle" @click="showSources = !showSources">
|
||||||
|
Sources ({{ topicStore.activeSummary.sources.length }})
|
||||||
|
<span>{{ showSources ? '▲' : '▼' }}</span>
|
||||||
|
</button>
|
||||||
|
<div v-if="showSources" class="sources-list">
|
||||||
|
<div
|
||||||
|
v-for="(source, idx) in topicStore.activeSummary.sources"
|
||||||
|
:key="idx"
|
||||||
|
class="source-chip"
|
||||||
|
:class="{ 'source-chip--clickable': source.bookId && source.page }"
|
||||||
|
@click="source.bookId && source.page ? handleOpenSource(source.bookId, source.page) : undefined"
|
||||||
|
>
|
||||||
|
<span class="source-icon">📖</span>
|
||||||
|
<span class="source-book">{{ source.bookTitle }}</span>
|
||||||
|
<span v-if="source.page" class="source-page">p. {{ source.page }}</span>
|
||||||
|
<span v-if="source.bookId && source.page" class="source-open-hint">↗</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<BookPagePanel
|
||||||
|
v-if="readerPanel"
|
||||||
|
:book-id="readerPanel.bookId"
|
||||||
|
:page="readerPanel.page"
|
||||||
|
:book-title="readerPanel.bookTitle"
|
||||||
|
class="reader-panel"
|
||||||
|
@close="readerPanel = null"
|
||||||
|
@navigate="(p) => readerPanel && (readerPanel.page = p)"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<div v-else class="no-sources">
|
||||||
|
No source citations available for this summary.
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Topic Grid -->
|
||||||
|
<div class="topic-grid">
|
||||||
|
<TopicCard
|
||||||
|
v-for="topic in summaryTopics"
|
||||||
|
:key="topic.id"
|
||||||
|
:topic="topic"
|
||||||
|
:is-generating="topicStore.activeSummaryTopicId === topic.id"
|
||||||
|
:is-selected="selectedTopicId === topic.id"
|
||||||
|
@generate="handleTopicClick"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</div><!-- end topics-main -->
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</template>
|
</template>
|
||||||
|
|
||||||
<script setup lang="ts">
|
<script setup lang="ts">
|
||||||
import { ref, onMounted, inject } from 'vue'
|
import { ref, computed, onMounted, inject } from 'vue'
|
||||||
|
import { marked } from 'marked'
|
||||||
import { RouterLink } from 'vue-router'
|
import { RouterLink } from 'vue-router'
|
||||||
import { useTopicStore } from '@/stores/topicStore'
|
import { useTopicStore, type SavedSummaryItem } from '@/stores/topicStore'
|
||||||
|
import { useBookStore } from '@/stores/bookStore'
|
||||||
import TopicCard from '@/components/TopicCard.vue'
|
import TopicCard from '@/components/TopicCard.vue'
|
||||||
|
import BookPagePanel from '@/components/BookPagePanel.vue'
|
||||||
|
|
||||||
const topicStore = useTopicStore()
|
const topicStore = useTopicStore()
|
||||||
|
const bookStore = useBookStore()
|
||||||
const showToast = inject<(msg: string, type?: 'error' | 'success') => void>('showToast')
|
const showToast = inject<(msg: string, type?: 'error' | 'success') => void>('showToast')
|
||||||
|
|
||||||
const showSources = ref(true)
|
const showSources = ref(true)
|
||||||
const summaryError = ref<string | null>(null)
|
const summaryError = ref<string | null>(null)
|
||||||
const isNoBooks = ref(false)
|
const isNoBooks = ref(false)
|
||||||
|
const sourcesSection = ref<HTMLElement | null>(null)
|
||||||
|
const selectedTopicId = ref<string | null>(null)
|
||||||
|
|
||||||
|
interface ReaderPanel { bookId: string; page: number; bookTitle?: string }
|
||||||
|
const readerPanel = ref<ReaderPanel | null>(null)
|
||||||
|
|
||||||
|
const summaryTopics = computed(() => topicStore.topics.filter(t => t.id !== 'free-form'))
|
||||||
|
|
||||||
|
const renderedSummary = computed(() => {
|
||||||
|
if (!topicStore.activeSummary) return ''
|
||||||
|
const html = marked.parse(topicStore.activeSummary.summary) as string
|
||||||
|
return html.replace(/\[S(\d+)\]/g, '<span class="source-ref">[S$1]</span>')
|
||||||
|
})
|
||||||
|
|
||||||
|
function handleSummaryClick(e: MouseEvent) {
|
||||||
|
if ((e.target as HTMLElement).classList.contains('source-ref')) {
|
||||||
|
showSources.value = true
|
||||||
|
sourcesSection.value?.scrollIntoView({ behavior: 'smooth', block: 'start' })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function handleOpenSource(bookId: string, page: number) {
|
||||||
|
const book = bookStore.books.find(b => b.id === bookId)
|
||||||
|
readerPanel.value = { bookId, page, bookTitle: book?.title }
|
||||||
|
showSources.value = true
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleTopicClick(topicId: string) {
|
||||||
|
if (selectedTopicId.value !== topicId) {
|
||||||
|
selectedTopicId.value = topicId
|
||||||
|
topicStore.activeSummary = null
|
||||||
|
summaryError.value = null
|
||||||
|
await topicStore.fetchSummaries(topicId)
|
||||||
|
// Auto-load the latest summary if any exist
|
||||||
|
const list = topicStore.summaryList
|
||||||
|
if (list.length > 0) {
|
||||||
|
await topicStore.fetchSummaryDetail(topicId, list[list.length - 1].id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleLoadSummary(item: SavedSummaryItem) {
|
||||||
|
if (!selectedTopicId.value) return
|
||||||
|
summaryError.value = null
|
||||||
|
await topicStore.fetchSummaryDetail(selectedTopicId.value, item.id)
|
||||||
|
}
|
||||||
|
|
||||||
onMounted(async () => {
|
onMounted(async () => {
|
||||||
await topicStore.fetchTopics()
|
await topicStore.fetchTopics()
|
||||||
|
if (bookStore.books.length === 0) {
|
||||||
|
await bookStore.fetchBooks()
|
||||||
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
async function handleGenerate(topicId: string) {
|
async function handleGenerate(topicId: string) {
|
||||||
@@ -109,27 +224,122 @@ async function handleGenerate(topicId: string) {
|
|||||||
summaryError.value.toLowerCase().includes('no books') ||
|
summaryError.value.toLowerCase().includes('no books') ||
|
||||||
summaryError.value.toLowerCase().includes('knowledge source')
|
summaryError.value.toLowerCase().includes('knowledge source')
|
||||||
showToast?.(summaryError.value, 'error')
|
showToast?.(summaryError.value, 'error')
|
||||||
|
} else {
|
||||||
|
// Refresh the history list to include the newly saved summary
|
||||||
|
await topicStore.fetchSummaries(topicId)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function formatDate(iso: string): string {
|
function formatDate(iso: string): string {
|
||||||
return new Date(iso).toLocaleString()
|
return new Date(iso).toLocaleString()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function formatDateShort(iso: string): string {
|
||||||
|
return new Date(iso).toLocaleDateString(undefined, { month: 'short', day: 'numeric' })
|
||||||
|
}
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
<style scoped>
|
<style scoped>
|
||||||
.topics-layout {
|
.topics-layout {
|
||||||
|
display: flex;
|
||||||
|
gap: 2rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.topics-main {
|
||||||
|
flex: 1;
|
||||||
|
min-width: 0;
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
gap: 2rem;
|
gap: 2rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.reader-panel {
|
||||||
|
margin-top: 1rem;
|
||||||
|
height: 600px;
|
||||||
|
min-height: 400px;
|
||||||
|
border-radius: 10px;
|
||||||
|
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.07);
|
||||||
|
}
|
||||||
|
|
||||||
.topic-grid {
|
.topic-grid {
|
||||||
display: grid;
|
display: grid;
|
||||||
grid-template-columns: repeat(auto-fill, minmax(280px, 1fr));
|
grid-template-columns: repeat(auto-fill, minmax(280px, 1fr));
|
||||||
gap: 1rem;
|
gap: 1rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* History panel */
|
||||||
|
.history-panel {
|
||||||
|
border-top: 3px solid #805ad5;
|
||||||
|
padding: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.history-header {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: space-between;
|
||||||
|
margin-bottom: 0.75rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.history-title {
|
||||||
|
font-size: 0.875rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: #553c9a;
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-sm {
|
||||||
|
font-size: 0.8rem;
|
||||||
|
padding: 0.3rem 0.75rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.history-loading {
|
||||||
|
font-size: 0.85rem;
|
||||||
|
color: #718096;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
.history-empty {
|
||||||
|
font-size: 0.85rem;
|
||||||
|
color: #a0aec0;
|
||||||
|
font-style: italic;
|
||||||
|
}
|
||||||
|
|
||||||
|
.history-list {
|
||||||
|
display: flex;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.history-chip {
|
||||||
|
background: #faf5ff;
|
||||||
|
border: 1px solid #d6bcfa;
|
||||||
|
border-radius: 6px;
|
||||||
|
padding: 0.3rem 0.75rem;
|
||||||
|
font-size: 0.8rem;
|
||||||
|
font-weight: 500;
|
||||||
|
color: #553c9a;
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background 0.15s, border-color 0.15s;
|
||||||
|
}
|
||||||
|
|
||||||
|
.history-chip:hover {
|
||||||
|
background: #e9d8fd;
|
||||||
|
border-color: #b794f4;
|
||||||
|
}
|
||||||
|
|
||||||
|
.history-chip--active {
|
||||||
|
background: #805ad5;
|
||||||
|
border-color: #805ad5;
|
||||||
|
color: #fff;
|
||||||
|
}
|
||||||
|
|
||||||
|
.history-chip-date {
|
||||||
|
font-weight: 400;
|
||||||
|
color: inherit;
|
||||||
|
opacity: 0.75;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Summary panel */
|
||||||
.summary-panel {
|
.summary-panel {
|
||||||
border-top: 3px solid #3182ce;
|
border-top: 3px solid #3182ce;
|
||||||
}
|
}
|
||||||
@@ -170,6 +380,22 @@ function formatDate(iso: string): string {
|
|||||||
color: #1a365d;
|
color: #1a365d;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.summary-meta {
|
||||||
|
display: flex;
|
||||||
|
align-items: baseline;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-number {
|
||||||
|
font-size: 0.8rem;
|
||||||
|
font-weight: 600;
|
||||||
|
color: #805ad5;
|
||||||
|
background: #faf5ff;
|
||||||
|
border: 1px solid #d6bcfa;
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 0.1rem 0.4rem;
|
||||||
|
}
|
||||||
|
|
||||||
.summary-timestamp {
|
.summary-timestamp {
|
||||||
font-size: 0.8rem;
|
font-size: 0.8rem;
|
||||||
color: #a0aec0;
|
color: #a0aec0;
|
||||||
@@ -179,10 +405,60 @@ function formatDate(iso: string): string {
|
|||||||
font-size: 0.95rem;
|
font-size: 0.95rem;
|
||||||
line-height: 1.7;
|
line-height: 1.7;
|
||||||
color: #2d3748;
|
color: #2d3748;
|
||||||
white-space: pre-wrap;
|
|
||||||
margin-bottom: 1rem;
|
margin-bottom: 1rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.summary-text--markdown {
|
||||||
|
white-space: normal;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(h1),
|
||||||
|
.summary-text--markdown :deep(h2),
|
||||||
|
.summary-text--markdown :deep(h3),
|
||||||
|
.summary-text--markdown :deep(h4) {
|
||||||
|
font-weight: 700;
|
||||||
|
margin: 0.75rem 0 0.35rem;
|
||||||
|
line-height: 1.3;
|
||||||
|
color: #1a202c;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(h1) { font-size: 1.15rem; }
|
||||||
|
.summary-text--markdown :deep(h2) { font-size: 1.05rem; }
|
||||||
|
.summary-text--markdown :deep(h3) { font-size: 0.975rem; }
|
||||||
|
.summary-text--markdown :deep(h4) { font-size: 0.925rem; }
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(p) { margin: 0.4rem 0; }
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(ul),
|
||||||
|
.summary-text--markdown :deep(ol) {
|
||||||
|
padding-left: 1.4rem;
|
||||||
|
margin: 0.4rem 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(li) { margin: 0.2rem 0; }
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(strong) {
|
||||||
|
font-weight: 700;
|
||||||
|
color: #1a202c;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(em) { font-style: italic; }
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(code) {
|
||||||
|
background: #edf2f7;
|
||||||
|
border-radius: 3px;
|
||||||
|
padding: 0.1em 0.35em;
|
||||||
|
font-size: 0.87em;
|
||||||
|
font-family: monospace;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(blockquote) {
|
||||||
|
border-left: 3px solid #bee3f8;
|
||||||
|
margin: 0.5rem 0;
|
||||||
|
padding: 0.25rem 0.75rem;
|
||||||
|
color: #4a5568;
|
||||||
|
}
|
||||||
|
|
||||||
.sources-section {
|
.sources-section {
|
||||||
border-top: 1px solid #e2e8f0;
|
border-top: 1px solid #e2e8f0;
|
||||||
padding-top: 0.75rem;
|
padding-top: 0.75rem;
|
||||||
@@ -223,6 +499,20 @@ function formatDate(iso: string): string {
|
|||||||
font-size: 0.8rem;
|
font-size: 0.8rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.source-chip--clickable {
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background 0.15s, border-color 0.15s;
|
||||||
|
}
|
||||||
|
|
||||||
|
.source-chip--clickable:hover {
|
||||||
|
background: #bee3f8;
|
||||||
|
border-color: #90cdf4;
|
||||||
|
}
|
||||||
|
|
||||||
|
.source-icon {
|
||||||
|
font-size: 0.8rem;
|
||||||
|
}
|
||||||
|
|
||||||
.source-book {
|
.source-book {
|
||||||
color: #2b6cb0;
|
color: #2b6cb0;
|
||||||
font-weight: 500;
|
font-weight: 500;
|
||||||
@@ -232,6 +522,12 @@ function formatDate(iso: string): string {
|
|||||||
color: #718096;
|
color: #718096;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.source-open-hint {
|
||||||
|
font-size: 0.75rem;
|
||||||
|
color: #3182ce;
|
||||||
|
margin-left: 0.1rem;
|
||||||
|
}
|
||||||
|
|
||||||
.no-sources {
|
.no-sources {
|
||||||
font-size: 0.85rem;
|
font-size: 0.85rem;
|
||||||
color: #a0aec0;
|
color: #a0aec0;
|
||||||
@@ -252,4 +548,31 @@ function formatDate(iso: string): string {
|
|||||||
color: #3182ce;
|
color: #3182ce;
|
||||||
text-decoration: underline;
|
text-decoration: underline;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.summary-placeholder {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
min-height: 6rem;
|
||||||
|
border-top-color: #cbd5e0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-placeholder-text {
|
||||||
|
font-size: 0.95rem;
|
||||||
|
color: #a0aec0;
|
||||||
|
font-style: italic;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(.source-ref) {
|
||||||
|
color: #3182ce;
|
||||||
|
font-weight: 600;
|
||||||
|
cursor: pointer;
|
||||||
|
border-radius: 3px;
|
||||||
|
padding: 0 0.15em;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-text--markdown :deep(.source-ref:hover) {
|
||||||
|
background: #ebf8ff;
|
||||||
|
text-decoration: underline;
|
||||||
|
}
|
||||||
</style>
|
</style>
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
<template>
|
<template>
|
||||||
<div class="upload-view">
|
<div class="upload-view">
|
||||||
<h1 class="page-title">Book Library</h1>
|
<h1 class="page-title">Book Library</h1>
|
||||||
<p class="page-subtitle">Upload medical textbooks (PDF) to build the knowledge base.</p>
|
<p v-if="uploadEnabled" class="page-subtitle">Upload medical textbooks (PDF) to build the knowledge base.</p>
|
||||||
|
|
||||||
<!-- Upload Section -->
|
<!-- Upload Section -->
|
||||||
<div class="upload-section card">
|
<div v-if="uploadEnabled" class="upload-section card">
|
||||||
<h2 class="section-title">Upload a Book</h2>
|
<h2 class="section-title">Upload a Book</h2>
|
||||||
|
|
||||||
<div
|
<div
|
||||||
@@ -87,6 +87,7 @@
|
|||||||
:key="book.id"
|
:key="book.id"
|
||||||
:book="book"
|
:book="book"
|
||||||
:deleting="deletingId === book.id"
|
:deleting="deletingId === book.id"
|
||||||
|
:delete-enabled="deleteEnabled"
|
||||||
@delete="handleDelete"
|
@delete="handleDelete"
|
||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
@@ -98,6 +99,10 @@
|
|||||||
import { ref, onMounted, onUnmounted, inject } from 'vue'
|
import { ref, onMounted, onUnmounted, inject } from 'vue'
|
||||||
import { useBookStore } from '@/stores/bookStore'
|
import { useBookStore } from '@/stores/bookStore'
|
||||||
import BookCard from '@/components/BookCard.vue'
|
import BookCard from '@/components/BookCard.vue'
|
||||||
|
import { env } from '@/env'
|
||||||
|
|
||||||
|
const uploadEnabled = env('VITE_UPLOAD_ENABLED') !== 'false'
|
||||||
|
const deleteEnabled = env('VITE_DELETE_ENABLED') !== 'false'
|
||||||
|
|
||||||
const bookStore = useBookStore()
|
const bookStore = useBookStore()
|
||||||
const showToast = inject<(msg: string, type?: 'error' | 'success') => void>('showToast')
|
const showToast = inject<(msg: string, type?: 'error' | 'success') => void>('showToast')
|
||||||
|
|||||||
@@ -0,0 +1,79 @@
|
|||||||
|
# Internal Contract: DocumentAiPageParser → FigureExtractionService
|
||||||
|
|
||||||
|
**Branch**: `002-image-aware-embedding` | **Date**: 2026-04-04
|
||||||
|
**Type**: Internal Java DTO (not an HTTP contract)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
`PageResult` is the internal data transfer object produced by `DocumentAiPageParser` for each
|
||||||
|
PDF page. It decouples the Google Document AI SDK types from the rest of the pipeline so that
|
||||||
|
`PdfStructureParser` can be replaced without cascading changes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Java Record
|
||||||
|
|
||||||
|
```java
|
||||||
|
package com.aiteacher.document;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Internal DTO produced by DocumentAiPageParser for one PDF page.
|
||||||
|
* Decouples the Document AI SDK types from downstream services.
|
||||||
|
*/
|
||||||
|
public record PageResult(
|
||||||
|
int pageNumber, // 1-based, matches Document.Page.getPageNumber()
|
||||||
|
String orderedText, // full page text in correct reading order (blocks joined by \n\n)
|
||||||
|
String headingTitle, // first HEADING block on page, or null
|
||||||
|
List<FigureBbox> figures // detected figure regions (may be empty)
|
||||||
|
) {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Normalized bounding box for a detected figure region.
|
||||||
|
* Coordinates are in the [0.0, 1.0] range relative to page dimensions.
|
||||||
|
*/
|
||||||
|
public record FigureBbox(
|
||||||
|
float x, // left edge (normalized)
|
||||||
|
float y, // top edge (normalized)
|
||||||
|
float width, // width (normalized)
|
||||||
|
float height, // height (normalized)
|
||||||
|
String nearestCaption // text of adjacent paragraph block, or null
|
||||||
|
) {}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Production Rules
|
||||||
|
|
||||||
|
| Field | Rule |
|
||||||
|
|-------|------|
|
||||||
|
| `orderedText` | Concatenation of all `PARAGRAPH` and `HEADING_*` blocks, joined with `\n\n`. Tables are represented as tab-separated text. |
|
||||||
|
| `headingTitle` | First block whose `blockType` is `HEADING_1` through `HEADING_6`. `null` if no heading detected. |
|
||||||
|
| `figures` | One entry per `VisualElement` with `type == "figure"` and `confidence ≥ 0.5`. Sorted top-to-bottom by `y`. |
|
||||||
|
| `nearestCaption` | The `PARAGRAPH` block immediately following the figure bbox (by Y coordinate). May be `null` if no paragraph follows within 10% of page height. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Mapping from Document AI Proto
|
||||||
|
|
||||||
|
```
|
||||||
|
Document.Page.Block → orderedText (concatenated)
|
||||||
|
Document.Page.Block (HEADING_*) → headingTitle (first match)
|
||||||
|
Document.Page.VisualElement → FigureBbox
|
||||||
|
└─ layout.bounding_poly.normalized_vertices[0] → (x, y) top-left
|
||||||
|
└─ normalized_vertices[2] → (x+w, y+h) bottom-right
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Consumers
|
||||||
|
|
||||||
|
| Consumer | What It Uses |
|
||||||
|
|----------|-------------|
|
||||||
|
| `BookEmbeddingService` | `orderedText` → `SectionEntity.fullText`; `headingTitle` → `SectionEntity.title` |
|
||||||
|
| `FigureExtractionService` | `figures` list → renders page via PDFBox, crops each bbox to `BufferedImage` |
|
||||||
|
| `TextChunkingService` | Receives `SectionEntity` (indirectly uses `orderedText`) — **unchanged** |
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
# Internal Contract: MarkerPageParser → FigureExtractionService / BookEmbeddingService
|
||||||
|
|
||||||
|
**Branch**: `002-image-aware-embedding` | **Date**: 2026-04-04
|
||||||
|
**Type**: Internal Java DTO (not an HTTP contract)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
`PageResult` is the internal data transfer object produced by `MarkerPageParser` for each
|
||||||
|
PDF page. It decouples the Marker HTTP API from the rest of the pipeline. Downstream consumers
|
||||||
|
(`BookEmbeddingService`, `FigureExtractionService`, `TextChunkingService`) are unaware of
|
||||||
|
Marker and depend only on this DTO.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Java Record
|
||||||
|
|
||||||
|
```java
|
||||||
|
package com.aiteacher.document;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Internal DTO produced by MarkerPageParser for one PDF page.
|
||||||
|
* Decouples the Marker HTTP API from downstream services.
|
||||||
|
*/
|
||||||
|
public record PageResult(
|
||||||
|
int pageNumber, // 1-based, derived from Marker page block index
|
||||||
|
String orderedText, // full page text in correct reading order (blocks joined by \n\n)
|
||||||
|
String headingTitle, // first SectionHeader block on page, or null
|
||||||
|
List<FigureData> figures // extracted figure images (may be empty)
|
||||||
|
) {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A figure extracted from the page.
|
||||||
|
* Image bytes are PNG data decoded from the Marker JSON `images` map.
|
||||||
|
*/
|
||||||
|
public record FigureData(
|
||||||
|
byte[] imageBytes, // PNG image data (base64-decoded from Marker response)
|
||||||
|
String nearestCaption, // text of the adjacent Caption block, or null
|
||||||
|
String blockId // Marker block ID (e.g. "/page/0/Figure/2") for traceability
|
||||||
|
) {}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Production Rules
|
||||||
|
|
||||||
|
| Field | Rule |
|
||||||
|
|-------|------|
|
||||||
|
| `pageNumber` | 1-based index derived from the Marker page block's position in the `children` array (index + 1). |
|
||||||
|
| `orderedText` | HTML-stripped text from all `Text`, `TextInlineMath`, `SectionHeader`, `ListItem`, and `Table` blocks, joined with `\n\n`. Marker already returns them in reading order. |
|
||||||
|
| `headingTitle` | Plain text of the first `SectionHeader` block on the page. `null` if no heading detected. |
|
||||||
|
| `figures` | One `FigureData` per `Figure` or `Picture` block that has a non-empty `images` entry. Blocks with no image data are skipped. |
|
||||||
|
| `imageBytes` | Base64-decoded bytes from `block.images[blockId]`. Marker returns PNG. |
|
||||||
|
| `nearestCaption` | Plain text of the first `Caption` block that is a sibling appearing immediately after the figure block. `null` if absent. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Mapping from Marker JSON
|
||||||
|
|
||||||
|
```
|
||||||
|
Marker JSON → PageResult
|
||||||
|
|
||||||
|
Page block ("/page/N/Page/M") → PageResult(pageNumber = N + 1)
|
||||||
|
SectionHeader child → headingTitle (first match, HTML-stripped)
|
||||||
|
Text / TextInlineMath children → orderedText (HTML-stripped, joined \n\n)
|
||||||
|
Figure / Picture child → FigureData
|
||||||
|
images[blockId] → FigureData.imageBytes (base64-decoded)
|
||||||
|
next Caption sibling → FigureData.nearestCaption (HTML-stripped)
|
||||||
|
blockId → FigureData.blockId
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Consumers
|
||||||
|
|
||||||
|
| Consumer | What It Uses |
|
||||||
|
|----------|-------------|
|
||||||
|
| `BookEmbeddingService` | `orderedText` → `SectionEntity.fullText`; `headingTitle` → `SectionEntity.title` |
|
||||||
|
| `FigureExtractionService` | `figures` list → decodes `imageBytes`, checks min size, saves to S3 |
|
||||||
|
| `TextChunkingService` | Receives `SectionEntity` (uses `orderedText` indirectly) — **unchanged** |
|
||||||
@@ -1,40 +1,42 @@
|
|||||||
# Implementation Plan: Enhanced Embedding with Image Parsing and Metadata
|
# Implementation Plan: Enhanced Embedding with Image Parsing and Metadata
|
||||||
|
|
||||||
**Branch**: `002-image-aware-embedding` | **Date**: 2026-04-03 | **Spec**: [spec.md](spec.md)
|
**Branch**: `002-image-aware-embedding` | **Date**: 2026-04-04 | **Spec**: [spec.md](spec.md)
|
||||||
**Input**: Feature specification from `/specs/002-image-aware-embedding/spec.md`
|
**Input**: Feature specification from `/specs/002-image-aware-embedding/spec.md`
|
||||||
|
|
||||||
## Summary
|
## Summary
|
||||||
|
|
||||||
Enhance the book embedding pipeline to extract images from every PDF page, generate descriptive
|
Enhance the PDF embedding pipeline to extract figures and generate AI descriptions for them,
|
||||||
text for each image, and store all content (text chunks + figure captions) with rich, consistent
|
making image content semantically searchable alongside text. PDF parsing and figure extraction
|
||||||
metadata in the vector store. A new document hierarchy (Book → Chapter → Section → TextChunk +
|
are delegated to a local **Marker** server (`http://localhost:8000/marker/upload`), which
|
||||||
Figure) is introduced. Postgres holds the full-text sections and figure metadata; the vector
|
returns reading-order text and pre-cropped figure images (base64) in a single JSON response,
|
||||||
store holds chunk and figure caption embeddings; the local file store holds extracted image files.
|
eliminating the need for PDFBox column heuristics and figure bbox rendering.
|
||||||
At query time, both the text-chunk store and figure-caption store are searched in parallel and
|
|
||||||
results are merged before being sent to the LLM.
|
|
||||||
|
|
||||||
## Technical Context
|
## Technical Context
|
||||||
|
|
||||||
**Language/Version**: Java 25 (backend), TypeScript / Node 20 (frontend)
|
**Language/Version**: Java 25 (backend), TypeScript / Node 20 (frontend)
|
||||||
**Primary Dependencies**: Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API (embeddings + chat), PDFBox (via Spring AI PDF reader dependency)
|
**Primary Dependencies**: Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API (embeddings +
|
||||||
**Storage**: PostgreSQL (JPA + Flyway), pgvector (Spring AI `VectorStore`), local file system (extracted images — `/uploads/figures/`)
|
GPT-4o vision), PDFBox 3.0.3 (via `spring-ai-pdf-document-reader` — retained transitively,
|
||||||
**Testing**: Spring Boot Test, JUnit 5, Mockito
|
no longer used directly), Marker local HTTP API (`http://localhost:8000/marker/upload`)
|
||||||
**Target Platform**: Linux server (Docker Compose)
|
**Storage**: PostgreSQL (JPA + Flyway), pgvector (Spring AI `VectorStore`), S3-compatible
|
||||||
**Project Type**: Web application — backend REST API + Vue 3 frontend
|
object store (figure images via `FigureStorageService`)
|
||||||
**Performance Goals**: Full book (up to 500 pages with images) processed in ≤ 30 minutes; query response unchanged from existing baseline
|
**Testing**: Maven / JUnit 5 (`spring-boot-starter-test`)
|
||||||
**Constraints**: No new deployable units; all changes within the existing `backend/` module; image storage on local disk (S3 migration is a future concern, behind an interface)
|
**Target Platform**: Linux server
|
||||||
**Scale/Scope**: POC — <10 concurrent users; single shared book library
|
**Project Type**: Web application (backend API + frontend client)
|
||||||
|
**Performance Goals**: SC-003 — book processing time ≤ 3× text-only for ≤ 500 pages
|
||||||
|
**Constraints**: REST API only (Constitution III); Marker server must be running locally;
|
||||||
|
S3-compatible storage configured via env vars
|
||||||
|
**Scale/Scope**: POC — handful of books, <10 users
|
||||||
|
|
||||||
## Constitution Check
|
## Constitution Check
|
||||||
|
|
||||||
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
*GATE: Must pass before Phase 0 research. Re-checked after Phase 1 design.*
|
||||||
|
|
||||||
| Principle | Status | Notes |
|
| Principle | Status | Notes |
|
||||||
|-----------|--------|-------|
|
|-----------|--------|-------|
|
||||||
| I — KISS | ⚠️ Justified violation — see Complexity Tracking | Hierarchical model + dual search adds complexity; justified by precision requirement |
|
| **I. KISS** | ✅ Justified | Marker replaces a bespoke PDFBox column heuristic + Google Cloud SDK with one HTTP call. Net complexity reduction vs. the Document AI approach. |
|
||||||
| II — Easy to Change | ✅ | Figure storage wrapped behind `FigureStorageService` interface; can swap local disk for S3 |
|
| **II. Easy to Change** | ✅ | `MarkerPageParser` is the only class that knows about Marker; swap the implementation to replace Marker with any other parser. `PageResult` DTO remains unchanged. |
|
||||||
| III — Web-First | ✅ | All new capabilities exposed via existing REST API; no new deployable units |
|
| **III. Web-First** | ✅ | Internal pipeline change; no public API contract change. |
|
||||||
| IV — Docs as Architecture | ⚠️ Required | README Mermaid diagram MUST be updated in this PR to show new storage tiers |
|
| **IV. Documentation** | ✅ | README must be updated to show Marker as a local external service. |
|
||||||
|
|
||||||
## Project Structure
|
## Project Structure
|
||||||
|
|
||||||
@@ -46,60 +48,38 @@ specs/002-image-aware-embedding/
|
|||||||
├── research.md # Phase 0 output
|
├── research.md # Phase 0 output
|
||||||
├── data-model.md # Phase 1 output
|
├── data-model.md # Phase 1 output
|
||||||
├── quickstart.md # Phase 1 output
|
├── quickstart.md # Phase 1 output
|
||||||
├── contracts/ # Phase 1 output
|
├── contracts/
|
||||||
└── tasks.md # Phase 2 output (/speckit.tasks)
|
│ ├── api.md # HTTP API contracts (unchanged from initial plan)
|
||||||
|
│ └── marker-page-result.md # Internal DTO contract (MarkerPageParser → downstream)
|
||||||
|
└── tasks.md # Phase 2 output (/speckit.tasks — not created here)
|
||||||
```
|
```
|
||||||
|
|
||||||
### Source Code (repository root)
|
### Source Code
|
||||||
|
|
||||||
```text
|
```text
|
||||||
backend/
|
backend/
|
||||||
├── src/main/java/com/aiteacher/
|
├── src/main/java/com/aiteacher/
|
||||||
|
│ ├── config/
|
||||||
|
│ │ └── MarkerConfig.java # NEW: RestClient bean + base-url property
|
||||||
|
│ ├── document/
|
||||||
|
│ │ ├── MarkerPageParser.java # NEW: replaces DocumentAiPageParser + PdfStructureParser
|
||||||
|
│ │ ├── PageResult.java # UPDATED: FigureBbox → FigureData (bytes not bbox)
|
||||||
|
│ │ ├── FigureExtractionService.java # UPDATED: no PDFBox render; decode bytes directly
|
||||||
|
│ │ ├── TextChunkingService.java # UNCHANGED
|
||||||
|
│ │ ├── VisionDescriptionService.java # UNCHANGED
|
||||||
|
│ │ └── [removed] DocumentAiPageParser.java
|
||||||
│ ├── book/
|
│ ├── book/
|
||||||
│ │ ├── Book.java (existing)
|
│ │ └── BookEmbeddingService.java # MINOR UPDATE: inject MarkerPageParser, drop DocumentAiPageParser
|
||||||
│ │ ├── BookController.java (existing)
|
│ └── [removed] config/DocumentAiConfig.java
|
||||||
│ │ ├── BookService.java (existing)
|
├── src/main/resources/
|
||||||
│ │ ├── BookRepository.java (existing)
|
│ └── application.yaml # UPDATED: remove document-ai.*, add marker.base-url
|
||||||
│ │ ├── BookStatus.java (existing)
|
└── pom.xml # UPDATED: remove google-cloud-document-ai
|
||||||
│ │ ├── BookEmbeddingService.java (existing — enhanced)
|
|
||||||
│ │ └── NoKnowledgeSourceException.java (existing)
|
|
||||||
│ ├── document/ (new package)
|
|
||||||
│ │ ├── BookNode.java
|
|
||||||
│ │ ├── ChapterNode.java
|
|
||||||
│ │ ├── SectionNode.java
|
|
||||||
│ │ ├── SectionRepository.java
|
|
||||||
│ │ ├── TextChunkNode.java
|
|
||||||
│ │ ├── FigureNode.java
|
|
||||||
│ │ ├── FigureRepository.java
|
|
||||||
│ │ ├── FigureType.java
|
|
||||||
│ │ ├── ChunkFigureRef.java
|
|
||||||
│ │ └── ChunkFigureRefRepository.java
|
|
||||||
│ ├── figure/ (new package)
|
|
||||||
│ │ ├── FigureStorageService.java (interface)
|
|
||||||
│ │ └── LocalFigureStorageService.java (implementation)
|
|
||||||
│ ├── retrieval/ (new package)
|
|
||||||
│ │ └── NeurosurgeryRetriever.java
|
|
||||||
│ ├── chat/
|
|
||||||
│ │ └── ChatService.java (updated — uses NeurosurgeryRetriever)
|
|
||||||
│ └── config/
|
|
||||||
│ └── FigureStorageConfig.java (new — configures upload dir)
|
|
||||||
└── src/main/resources/
|
|
||||||
└── db/migration/
|
|
||||||
├── V4__document_hierarchy.sql (new)
|
|
||||||
└── V5__figures_and_refs.sql (new)
|
|
||||||
|
|
||||||
uploads/
|
|
||||||
└── figures/ (runtime — extracted images; gitignored)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
**Structure Decision**: Option 2 (Web Application) confirmed. All backend changes stay within
|
**Structure Decision**: Option 2 (backend + frontend) per constitution Technology Constraints.
|
||||||
`backend/`. Two new packages (`document/`, `retrieval/`) plus one interface package (`figure/`)
|
Frontend changes are display-only (render figure citations inline).
|
||||||
keep concerns separated without adding a deployable unit.
|
|
||||||
|
|
||||||
## Complexity Tracking
|
## Complexity Tracking
|
||||||
|
|
||||||
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
> No constitution violations — Marker reduces complexity compared to the previous
|
||||||
|-----------|------------|-------------------------------------|
|
> Google Document AI approach (fewer dependencies, no GCP credentials, no 15-page batching).
|
||||||
| Document hierarchy (BookNode → ChapterNode → SectionNode) | Parent-child retrieval: chunks reference their parent section so the LLM receives full section context, not just the matching fragment. This is the established solution for RAG precision. | Flat page-per-doc model (current) loses inter-sentence context; chunk-only retrieval produces incomplete answers for multi-paragraph clinical questions |
|
|
||||||
| Dual vector search (text chunks + figure captions) | Figure captions must be independently searchable — a query about "cavernous sinus anatomy" must surface the diagram even if no text chunk scores highly | Single vector store search would miss figures whose captions don't happen to be the highest-similarity hit; this is the core deliverable of the feature |
|
|
||||||
| Third storage tier (local file store for images) | Extracted images cannot live in Postgres (binary blobs degrade query performance) or the vector store (only vectors). A file-per-image approach is standard. | Storing images as base64 in Postgres JSONB would bloat the DB and complicate backup/restore; the `FigureStorageService` interface keeps the implementation swappable |
|
|
||||||
|
|||||||
@@ -1,34 +1,67 @@
|
|||||||
# Quickstart: Enhanced Embedding with Image Parsing and Metadata
|
# Quickstart: Enhanced Embedding with Image Parsing and Metadata
|
||||||
|
|
||||||
**Branch**: `002-image-aware-embedding` | **Date**: 2026-04-03
|
**Branch**: `002-image-aware-embedding` | **Date**: 2026-04-04 (updated: Marker replaces Google Document AI)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
- Docker Compose running (PostgreSQL + pgvector)
|
- Docker Compose running (PostgreSQL + pgvector)
|
||||||
- OpenAI API key set in `backend/src/main/resources/application.properties` or as env var `OPENAI_API_KEY`
|
- OpenAI API key set as env var `OPENAI_API_KEY`
|
||||||
- Java 25 + Maven on PATH
|
- Java 25 + Maven on PATH
|
||||||
|
- **Marker server running** on `http://localhost:8000` (see setup below)
|
||||||
|
- S3-compatible bucket configured (existing setup)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## New Configuration
|
## Marker Server Setup (one-time)
|
||||||
|
|
||||||
Add to `backend/src/main/resources/application.properties`:
|
Marker is a local Python service — no cloud credentials required.
|
||||||
|
|
||||||
```properties
|
```bash
|
||||||
# Figure storage
|
# Install (Python 3.10+ required)
|
||||||
app.figure-storage.base-path=./uploads
|
pip install marker-pdf
|
||||||
app.figure-storage.min-image-size-px=100
|
|
||||||
|
# Start the server on port 8000
|
||||||
|
marker_server --port 8000
|
||||||
```
|
```
|
||||||
|
|
||||||
The `uploads/figures/` directory is created automatically on first use. Add it to `.gitignore`.
|
The server is ready when you see:
|
||||||
|
```
|
||||||
|
INFO: Uvicorn running on http://0.0.0.0:8000
|
||||||
|
```
|
||||||
|
|
||||||
|
Keep the server running in the background (or use a process manager like `systemd` or `screen`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Backend Configuration
|
||||||
|
|
||||||
|
Add or update `backend/src/main/resources/application.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
app:
|
||||||
|
figure-storage:
|
||||||
|
endpoint: https://your-s3-endpoint
|
||||||
|
region: your-region
|
||||||
|
bucket: ${S3_BUCKET:aiteacher}
|
||||||
|
access-key-id: ${S3_ACCESS_KEY_ID}
|
||||||
|
secret-access-key: ${S3_SECRET_ACCESS_KEY}
|
||||||
|
min-image-size-px: 100 # skip decorative images smaller than 100×100 px
|
||||||
|
marker:
|
||||||
|
base-url: ${MARKER_BASE_URL:http://localhost:8000}
|
||||||
|
embedding:
|
||||||
|
batch-size: 20
|
||||||
|
batch-delay-ms: 2000
|
||||||
|
```
|
||||||
|
|
||||||
|
No GCP credentials or project IDs are needed.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Database Migration
|
## Database Migration
|
||||||
|
|
||||||
Two new Flyway migrations run automatically on startup:
|
Two Flyway migrations run automatically on startup:
|
||||||
|
|
||||||
- `V4__document_hierarchy.sql` — adds `chapter` and `section` tables
|
- `V4__document_hierarchy.sql` — adds `chapter` and `section` tables
|
||||||
- `V5__figures_and_refs.sql` — adds `figure` and `chunk_figure_ref` tables
|
- `V5__figures_and_refs.sql` — adds `figure` and `chunk_figure_ref` tables
|
||||||
@@ -54,10 +87,11 @@ image-aware pipeline runs. Status can be polled via `GET /api/v1/books`.
|
|||||||
|
|
||||||
## Verifying Image Extraction
|
## Verifying Image Extraction
|
||||||
|
|
||||||
1. Upload a PDF with diagrams: `POST /api/v1/books/upload`
|
1. Ensure Marker is running: `curl http://localhost:8000` should respond.
|
||||||
2. Wait for `status: "READY"` via `GET /api/v1/books`
|
2. Upload a PDF with diagrams: `POST /api/v1/books/upload`
|
||||||
3. List figures: `GET /api/v1/books/{id}/figures` — should return at least one entry per image page
|
3. Wait for `status: "READY"` via `GET /api/v1/books`
|
||||||
4. Ask a diagram-specific question in chat — response `sources` should include a `type: "FIGURE"` entry
|
4. List figures: `GET /api/v1/books/{id}/figures` — should return at least one entry per image page
|
||||||
|
5. Ask a diagram-specific question in chat — response `sources` should include a `type: "FIGURE"` entry
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -80,7 +114,8 @@ mvn test
|
|||||||
```
|
```
|
||||||
|
|
||||||
Key new test classes:
|
Key new test classes:
|
||||||
- `FigureExtractionServiceTest` — unit tests for image extraction and classification
|
- `MarkerPageParserTest` — unit tests for JSON parsing and block-to-PageResult mapping
|
||||||
|
- `FigureExtractionServiceTest` — unit tests for base64 decode, size filtering, classification
|
||||||
- `NeurosurgeryRetrieverTest` — unit tests for dual-search merge and deduplication
|
- `NeurosurgeryRetrieverTest` — unit tests for dual-search merge and deduplication
|
||||||
- `BookEmbeddingServiceIntegrationTest` — integration test: upload PDF with known figures,
|
- `BookEmbeddingServiceIntegrationTest` — integration test: upload PDF with known figures,
|
||||||
verify figures appear in `GET /api/v1/books/{id}/figures`
|
verify figures appear in `GET /api/v1/books/{id}/figures`
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
# Research: Enhanced Embedding with Image Parsing and Metadata
|
# Research: Enhanced Embedding with Image Parsing and Metadata
|
||||||
|
|
||||||
**Branch**: `002-image-aware-embedding` | **Date**: 2026-04-03
|
**Branch**: `002-image-aware-embedding` | **Date**: 2026-04-04 (updated: Marker replaces Google Document AI)
|
||||||
|
|
||||||
This document resolves all technical unknowns identified during planning. The primary source for
|
This document resolves all technical unknowns identified during planning. Decisions 1–10 cover
|
||||||
decisions is the detailed architecture provided directly by the project owner, supplemented by
|
the core pipeline. The **Marker Study** section at the bottom explains why Marker was chosen
|
||||||
Spring AI 2.0.0-M4 API specifics.
|
over Google Document AI to drive PDF parsing and figure extraction.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -28,19 +28,29 @@ association explicit and queryable.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Decision 2: Image Extraction Strategy
|
## Decision 2: Document Parsing Strategy
|
||||||
|
|
||||||
**Decision**: Use PDFBox (already on classpath via `spring-ai-pdf-document-reader`) to extract
|
**Decision**: Use **Marker** (local HTTP server, `http://localhost:8000/marker/upload`) as the
|
||||||
images per page. Each image is tagged with `page`, `figure_id` (derived from caption, e.g.
|
single entry point for PDF parsing. A single `POST` with `output_format=json` returns:
|
||||||
"Fig. 12-4"), and the parent `sectionId`. Images are saved to local disk under
|
- Reading-order text blocks (headings, paragraphs) — no column-split heuristic needed
|
||||||
`/uploads/figures/{bookId}/`.
|
- Pre-cropped figure images as base64-encoded PNG in the `images` map of each `Figure` block
|
||||||
|
- Table, equation, and code blocks as structured HTML
|
||||||
|
|
||||||
**Rationale**: PDFBox is already present (Spring AI bundles it). No new dependency needed.
|
`MarkerPageParser` translates the Marker JSON response into `List<PageResult>`, which is the
|
||||||
Per-page extraction ensures every image is captured regardless of PDF structure.
|
same internal DTO used by the rest of the pipeline.
|
||||||
|
|
||||||
|
**Rationale**: Marker handles column reordering, scanned-page OCR, and figure cropping in one
|
||||||
|
call, eliminating the PDFBox column heuristic (`PdfStructureParser`) and the PDFBox
|
||||||
|
render+crop loop in `FigureExtractionService`. Net result: fewer classes, no cloud dependency,
|
||||||
|
no GCP credentials.
|
||||||
|
|
||||||
**Alternatives considered**:
|
**Alternatives considered**:
|
||||||
- iText / iText7 → additional commercial dependency; overkill for extraction
|
- PDFBox column heuristic (previous approach) → rejected: 50/50 split fails on asymmetric
|
||||||
- Screenshot each page as PNG, then OCR → far slower; loses vector quality
|
columns and scanned pages
|
||||||
|
- Google Document AI Layout Parser → rejected: adds GCP credentials, per-page billing, 15-page
|
||||||
|
batch limit, and still requires PDFBox to render+crop figure regions from bounding boxes.
|
||||||
|
See Marker Study below for detailed comparison.
|
||||||
|
- Screenshot each page + OCR → far slower; loses digital text quality
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -103,18 +113,19 @@ search. This is the higher-recall path; dual search (Decision 4) is the higher-p
|
|||||||
|
|
||||||
## Decision 6: Image Storage
|
## Decision 6: Image Storage
|
||||||
|
|
||||||
**Decision**: Extracted images are saved as PNG files to a local directory
|
**Decision**: Marker returns figure images as base64-encoded PNG bytes in the JSON response.
|
||||||
(`${app.figure-storage.base-path}`, defaults to `./uploads/figures/{bookId}/`). The path is
|
`FigureExtractionService` decodes these bytes and passes them to `FigureStorageService`, which
|
||||||
stored in `figure.image_path` in Postgres. A `FigureStorageService` interface wraps all disk
|
persists them to an S3-compatible bucket (`${app.figure-storage.bucket}`). The image path/URL
|
||||||
I/O so the implementation can be swapped to S3 or another object store without changing
|
is stored in `figure.image_path` in Postgres.
|
||||||
callers.
|
|
||||||
|
|
||||||
**Rationale**: Local disk is the simplest viable option for a POC with <10 users. The interface
|
The `FigureStorageService` interface is unchanged; only the caller changes (from PDFBox crop
|
||||||
boundary satisfies Constitution Principle II (Easy to Change).
|
to base64 decode).
|
||||||
|
|
||||||
|
**Rationale**: Marker's pre-cropped images remove the need for PDFBox rendering.
|
||||||
|
`FigureStorageService` interface boundary satisfies Constitution Principle II (Easy to Change).
|
||||||
|
|
||||||
**Alternatives considered**:
|
**Alternatives considered**:
|
||||||
- S3 from day 1 → operational overhead not justified at POC scale
|
- Store base64 in Postgres JSONB → bloats DB; complicates backup; query performance degrades
|
||||||
- Base64 in Postgres JSONB → bloats DB; complicates backup; query performance degrades
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -123,7 +134,8 @@ boundary satisfies Constitution Principle II (Easy to Change).
|
|||||||
**Decision**: Use the enum `FigureType { ANATOMICAL_DIAGRAM, SURGICAL_PHOTOGRAPH, MRI_CT_SCAN,
|
**Decision**: Use the enum `FigureType { ANATOMICAL_DIAGRAM, SURGICAL_PHOTOGRAPH, MRI_CT_SCAN,
|
||||||
TABLE, CHART, INTRAOPERATIVE_IMAGE }`. Classification is derived from:
|
TABLE, CHART, INTRAOPERATIVE_IMAGE }`. Classification is derived from:
|
||||||
1. Caption keywords ("MRI", "CT", "Fig.", "Table") — heuristic, no model needed
|
1. Caption keywords ("MRI", "CT", "Fig.", "Table") — heuristic, no model needed
|
||||||
2. Fall back to `ANATOMICAL_DIAGRAM` if unclassifiable
|
2. Marker `block_type` hint (`"Table"` → TABLE, `"Figure"` / `"Picture"` → ANATOMICAL_DIAGRAM default)
|
||||||
|
3. Fall back to `ANATOMICAL_DIAGRAM` if unclassifiable
|
||||||
|
|
||||||
**Rationale**: Allows the frontend to render different icon/label per type (e.g., "MRI" badge).
|
**Rationale**: Allows the frontend to render different icon/label per type (e.g., "MRI" badge).
|
||||||
Heuristic classification avoids a separate model call per image at extraction time.
|
Heuristic classification avoids a separate model call per image at extraction time.
|
||||||
@@ -175,14 +187,225 @@ the process fails mid-way. An explicit, idempotent trigger is safer and more obs
|
|||||||
|
|
||||||
## Decision 10: Minimum Image Size Threshold
|
## Decision 10: Minimum Image Size Threshold
|
||||||
|
|
||||||
**Decision**: Images smaller than 100×100 pixels are discarded and no chunk is created. This
|
**Decision**: Images smaller than 100×100 pixels are discarded and no chunk is created. Marker
|
||||||
threshold filters out decorative elements (bullets, dividers, publisher logos) without a
|
returns PNG bytes; `FigureExtractionService` decodes to `BufferedImage` solely to check
|
||||||
classification model.
|
dimensions. This threshold filters out decorative elements without a classification model.
|
||||||
|
|
||||||
**Rationale**: Neurosurgery textbook diagrams and MRI scans are never smaller than 100×100 px.
|
**Rationale**: Neurosurgery textbook diagrams and MRI scans are never smaller than 100×100 px.
|
||||||
The threshold is configurable via `app.figure-storage.min-image-size-px` in
|
The threshold is configurable via `app.figure-storage.min-image-size-px`.
|
||||||
`application.properties`.
|
|
||||||
|
|
||||||
**Alternatives considered**:
|
**Alternatives considered**:
|
||||||
- No threshold → decorative icons pollute the figure index
|
- No threshold → decorative icons pollute the figure index
|
||||||
- ML-based classification → accurate but adds model dependency; not needed at POC scale
|
- ML-based classification → accurate but adds model dependency; not needed at POC scale
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Marker Study — Why Marker Replaces Google Document AI
|
||||||
|
|
||||||
|
*Added 2026-04-04.*
|
||||||
|
|
||||||
|
## What Marker Offers
|
||||||
|
|
||||||
|
Marker is an open-source, locally-runnable PDF-to-structured-content converter that uses a
|
||||||
|
pipeline of deep-learning models (surya for OCR + layout detection, texify for equations).
|
||||||
|
Key capabilities relevant to this project:
|
||||||
|
|
||||||
|
| Capability | Marker | Google Document AI |
|
||||||
|
|-----------|--------|--------------------|
|
||||||
|
| Multi-column reading order | ✅ | ✅ |
|
||||||
|
| OCR on scanned pages | ✅ | ✅ |
|
||||||
|
| Figure detection | ✅ returns pre-cropped images | ⚠️ returns bbox only; PDFBox still needed |
|
||||||
|
| Table extraction | ✅ HTML tables | ✅ |
|
||||||
|
| JSON output with image bytes | ✅ base64 in `images` map | ❌ |
|
||||||
|
| No cloud credentials | ✅ | ❌ GCP service account required |
|
||||||
|
| No per-page billing | ✅ | ❌ ~$10/1,000 pages |
|
||||||
|
| Batch size limits | None (local) | 15 pages / 20 MB per sync call |
|
||||||
|
| Setup | `pip install marker-pdf && marker_server` | GCP project + processor + IAM |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Does Marker Solve the Current Pain Points?
|
||||||
|
|
||||||
|
### Pain Point 1: Naive 50/50 Column Split
|
||||||
|
|
||||||
|
**Answer: Yes, Marker fixes this completely.**
|
||||||
|
|
||||||
|
`PdfStructureParser.extractPageText()` splits pages at the horizontal midpoint with a 20%
|
||||||
|
threshold. This fails on asymmetric columns and scanned pages. Marker's surya layout model
|
||||||
|
returns blocks in natural reading order — no heuristic needed.
|
||||||
|
|
||||||
|
### Pain Point 2: Figure Detection Misses Rasterized Figures
|
||||||
|
|
||||||
|
**Answer: Yes, Marker fixes this for most cases.**
|
||||||
|
|
||||||
|
`FigureExtractionService` previously iterated PDF XObjects (only finds embedded XObject images,
|
||||||
|
misses rasterized figures and vector-path drawings). Marker's layout model detects visual
|
||||||
|
elements by type and returns the cropped image bytes directly — no PDFBox page rendering needed.
|
||||||
|
|
||||||
|
### Pain Point 3: OCR on Scanned Pages
|
||||||
|
|
||||||
|
**Answer: Yes, Marker handles scanned pages transparently via surya OCR.**
|
||||||
|
|
||||||
|
### Pain Point 4: Caption Detection
|
||||||
|
|
||||||
|
**Answer: Improved — Marker groups caption blocks with their figure block.**
|
||||||
|
|
||||||
|
The `block_type = "Caption"` block appears as a sibling or child adjacent to the `"Figure"`
|
||||||
|
block in the Marker JSON, making caption association structural rather than regex-based.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Marker API Integration
|
||||||
|
|
||||||
|
### Local Server Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install marker-pdf
|
||||||
|
marker_server --port 8000
|
||||||
|
```
|
||||||
|
|
||||||
|
The server exposes `POST /marker/upload` (the user's configured endpoint).
|
||||||
|
|
||||||
|
### Request
|
||||||
|
|
||||||
|
```
|
||||||
|
POST http://localhost:8000/marker/upload
|
||||||
|
Content-Type: multipart/form-data
|
||||||
|
|
||||||
|
file=@document.pdf
|
||||||
|
output_format=json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Response (abbreviated)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"output_format": "json",
|
||||||
|
"output": {
|
||||||
|
"block_type": "Document",
|
||||||
|
"children": [
|
||||||
|
{
|
||||||
|
"block_type": "Page",
|
||||||
|
"id": "/page/0/Page/0",
|
||||||
|
"children": [
|
||||||
|
{
|
||||||
|
"block_type": "SectionHeader",
|
||||||
|
"id": "/page/0/SectionHeader/0",
|
||||||
|
"html": "<h1>Cavernous Sinus Anatomy</h1>"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"block_type": "Text",
|
||||||
|
"id": "/page/0/Text/1",
|
||||||
|
"html": "<p>The cavernous sinus contains...</p>"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"block_type": "Figure",
|
||||||
|
"id": "/page/0/Figure/2",
|
||||||
|
"html": "<figure><img src='/page/0/Figure/2'/></figure>",
|
||||||
|
"images": {
|
||||||
|
"/page/0/Figure/2": "iVBORw0KGgo..."
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"block_type": "Caption",
|
||||||
|
"id": "/page/0/Caption/3",
|
||||||
|
"html": "<p>Fig. 12-4. Coronal cross-section...</p>"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"metadata": { "page_stats": [...] }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Java Integration Pattern
|
||||||
|
|
||||||
|
```java
|
||||||
|
// MarkerPageParser — core call
|
||||||
|
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
|
||||||
|
body.add("file", new FileSystemResource(pdfPath));
|
||||||
|
body.add("output_format", "json");
|
||||||
|
|
||||||
|
JsonNode response = restClient.post()
|
||||||
|
.uri(baseUrl + "/marker/upload")
|
||||||
|
.contentType(MediaType.MULTIPART_FORM_DATA)
|
||||||
|
.body(body)
|
||||||
|
.retrieve()
|
||||||
|
.body(JsonNode.class);
|
||||||
|
|
||||||
|
JsonNode document = response.get("output");
|
||||||
|
```
|
||||||
|
|
||||||
|
### Mapping Marker Blocks to PageResult
|
||||||
|
|
||||||
|
```
|
||||||
|
Page block (id "/page/N/Page/M") → PageResult(pageNumber = N+1)
|
||||||
|
SectionHeader children → headingTitle (first match)
|
||||||
|
Text, TextInlineMath children → orderedText (HTML stripped, joined \n\n)
|
||||||
|
Figure children with images map → FigureData(imageBytes = base64decode(images[id]))
|
||||||
|
Caption sibling of Figure → FigureData.nearestCaption
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture Change
|
||||||
|
|
||||||
|
```
|
||||||
|
Before (Document AI — removed):
|
||||||
|
DocumentAiPageParser
|
||||||
|
→ Google Document AI API (GCP, 15-page batches, credentials)
|
||||||
|
→ returns text blocks + figure bboxes
|
||||||
|
PdfStructureParser (PDFBox column heuristic)
|
||||||
|
FigureExtractionService
|
||||||
|
→ renders page via PDFBox at 150 DPI
|
||||||
|
→ crops bbox region
|
||||||
|
|
||||||
|
After (Marker):
|
||||||
|
MarkerPageParser
|
||||||
|
→ POST PDF to http://localhost:8000/marker/upload (output_format=json)
|
||||||
|
→ returns text blocks (correct reading order) + Figure blocks with base64 images
|
||||||
|
→ produces List<PageResult> (same DTO, FigureData carries bytes not bbox)
|
||||||
|
FigureExtractionService (simplified)
|
||||||
|
→ base64-decodes image bytes from PageResult.FigureData
|
||||||
|
→ checks min size (ImageIO.read → getWidth/getHeight)
|
||||||
|
→ saves to S3 via FigureStorageService (UNCHANGED)
|
||||||
|
VisionDescriptionService (UNCHANGED)
|
||||||
|
BookEmbeddingService orchestration (MINOR: inject MarkerPageParser)
|
||||||
|
```
|
||||||
|
|
||||||
|
**What is removed**:
|
||||||
|
- `DocumentAiPageParser` — replaced by `MarkerPageParser`
|
||||||
|
- `DocumentAiConfig` — replaced by `MarkerConfig`
|
||||||
|
- `PdfStructureParser` — Marker handles reading order
|
||||||
|
- `google-cloud-document-ai` Maven dependency
|
||||||
|
- `app.document-ai.*` configuration properties
|
||||||
|
|
||||||
|
**What stays the same**:
|
||||||
|
- `PageResult` DTO structure (fields renamed, not restructured)
|
||||||
|
- `FigureExtractionService` public interface
|
||||||
|
- `TextChunkingService`, `VisionDescriptionService`, `BookEmbeddingService` orchestration
|
||||||
|
- All JPA entities, repositories, vector store, S3 storage
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Constitution Compliance
|
||||||
|
|
||||||
|
| Principle | Assessment |
|
||||||
|
|-----------|------------|
|
||||||
|
| **I. KISS** | ✅ Simpler than Document AI — one HTTP call replaces GCP SDK + PDFBox render loop. No new dependency beyond an HTTP client (Spring RestClient, already available). |
|
||||||
|
| **II. Easy to Change** | ✅ `MarkerPageParser` is the only Marker-aware class. Swap it to use any other parser. `PageResult` DTO unchanged in contract. |
|
||||||
|
| **III. Web-First** | ✅ Internal pipeline change; no API contract change. |
|
||||||
|
| **IV. Documentation** | ✅ README must show Marker as a local external service dependency. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Risks & Mitigations
|
||||||
|
|
||||||
|
| Risk | Likelihood | Mitigation |
|
||||||
|
|------|-----------|------------|
|
||||||
|
| Marker server not running when book is uploaded | Medium | `BookEmbeddingService` catches exception from `MarkerPageParser`, marks book as `FAILED`, logs full error. |
|
||||||
|
| Marker misses some figures (complex PDFs) | Medium | `app.figure-storage.min-image-size-px` threshold can be tuned. Add fallback: if Marker returns 0 figures for a page with known images, log a warning. |
|
||||||
|
| SC-003 (≤ 3× processing time) violated | Low | Marker runs locally (no network latency to cloud). Benchmark with a real 500-page book early. |
|
||||||
|
| Large PDF upload to Marker (>100MB) | Low | Marker server handles the full file; no batching needed. Multipart upload limit configurable. |
|
||||||
|
| Marker image quality vs PDFBox crop | Low | Marker crops at native resolution; quality is equivalent or better than 150 DPI PDFBox render. |
|
||||||
|
|||||||
@@ -48,12 +48,13 @@
|
|||||||
|
|
||||||
**Independent Test**: Upload a PDF containing at least one page with a labelled anatomical diagram. After status shows `READY`, call `GET /api/v1/books/{id}/figures` — response must contain at least one entry with `figureType`, `caption`, `page`, and `imageUrl` populated. Verify the PNG file exists at the path in `imagePath`.
|
**Independent Test**: Upload a PDF containing at least one page with a labelled anatomical diagram. After status shows `READY`, call `GET /api/v1/books/{id}/figures` — response must contain at least one entry with `figureType`, `caption`, `page`, and `imageUrl` populated. Verify the PNG file exists at the path in `imagePath`.
|
||||||
|
|
||||||
- [X] T013 [US2] Create `PdfStructureParser` service in `backend/src/main/java/com/aiteacher/document/PdfStructureParser.java` — uses Spring AI's `PagePdfDocumentReader` to extract per-page text; groups pages into `SectionEntity` records using heading-detection heuristics (lines matching `^\d+(\.\d+)*\s+[A-Z]`); groups sections into `ChapterEntity` records; persists both to Postgres via `ChapterRepository` and `SectionRepository`; returns `List<SectionEntity>` for the book
|
- [X] T013 [US2] ~~Create `PdfStructureParser`~~ → **SUPERSEDED**: PDF parsing is handled by `MarkerPageParser` (see T013b). `PdfStructureParser` exists but is not wired into the pipeline.
|
||||||
- [X] T014 [US2] Create `FigureExtractionService` in `backend/src/main/java/com/aiteacher/document/FigureExtractionService.java` — opens PDF with PDFBox `PDDocument`; iterates pages; extracts `PDImageXObject` instances; skips images whose width or height are below `min-image-size-px`; classifies `FigureType` using the keyword-matching table from data-model.md §FigureType; parses caption from the nearest text line matching `CAPTION_PATTERN`; saves PNG via `FigureStorageService`; persists `FigureEntity` to `FigureRepository`; returns `List<FigureEntity>` per book
|
- [X] T013b [US2] Create `MarkerPageParser` in `backend/src/main/java/com/aiteacher/document/MarkerPageParser.java` — POSTs PDF to `http://localhost:8000/marker/upload?output_format=json` via Spring `RestClient`; parses JSON response into `List<PageResult>` (one per page block); extracts heading, ordered text, and pre-cropped figure PNG bytes per page
|
||||||
|
- [X] T014 [US2] Update `FigureExtractionService` in `backend/src/main/java/com/aiteacher/document/FigureExtractionService.java` — **Marker migration**: removed PDFBox rendering + bbox-crop loop; decodes PNG bytes from `PageResult.FigureData` via `ImageIO.read()`; skips images below `min-image-size-px`; classifies `FigureType`; saves via `FigureStorageService`; persists `FigureEntity`
|
||||||
- [X] T015 [US2] Create `VisionDescriptionService` in `backend/src/main/java/com/aiteacher/document/VisionDescriptionService.java` — accepts a `Path` to a PNG and a caption String; calls the OpenAI vision model (via Spring AI `ChatClient` with image media type) to generate a 2–4 sentence clinical description; returns the generated description string; handles API failures by returning the caption as fallback
|
- [X] T015 [US2] Create `VisionDescriptionService` in `backend/src/main/java/com/aiteacher/document/VisionDescriptionService.java` — accepts a `Path` to a PNG and a caption String; calls the OpenAI vision model (via Spring AI `ChatClient` with image media type) to generate a 2–4 sentence clinical description; returns the generated description string; handles API failures by returning the caption as fallback
|
||||||
- [X] T016 [US2] Create `TextChunkingService` in `backend/src/main/java/com/aiteacher/document/TextChunkingService.java` — accepts a `SectionEntity`; splits `fullText` into overlapping 400–600 token windows (20-token overlap); wraps each window in a Spring AI `Document` with the flat metadata map defined in data-model.md §Text chunk document; returns `List<Document>`
|
- [X] T016 [US2] Create `TextChunkingService` in `backend/src/main/java/com/aiteacher/document/TextChunkingService.java` — accepts a `SectionEntity`; splits `fullText` into overlapping 400–600 token windows (20-token overlap); wraps each window in a Spring AI `Document` with the flat metadata map defined in data-model.md §Text chunk document; returns `List<Document>`
|
||||||
- [X] T017 [US2] Create `ChunkFigureRefService` in `backend/src/main/java/com/aiteacher/document/ChunkFigureRefService.java` — accepts a Spring AI `Document` (with its `id` as `chunkId`) and a `List<FigureEntity>` for the book; scans chunk text for patterns `Fig\.\s*\d+[\-\.]\d+` and `Figure\s+\d+[\-\.]\d+`; matches against figure labels; persists `ChunkFigureRefEntity` rows via `ChunkFigureRefRepository`
|
- [X] T017 [US2] Create `ChunkFigureRefService` in `backend/src/main/java/com/aiteacher/document/ChunkFigureRefService.java` — accepts a Spring AI `Document` (with its `id` as `chunkId`) and a `List<FigureEntity>` for the book; scans chunk text for patterns `Fig\.\s*\d+[\-\.]\d+` and `Figure\s+\d+[\-\.]\d+`; matches against figure labels; persists `ChunkFigureRefEntity` rows via `ChunkFigureRefRepository`
|
||||||
- [X] T018 [US2] Rewrite `BookEmbeddingService.embedBook()` in `backend/src/main/java/com/aiteacher/book/BookEmbeddingService.java` to orchestrate the full pipeline: (1) `PdfStructureParser` → sections; (2) parallel: `FigureExtractionService` + `TextChunkingService` for each section; (3) `VisionDescriptionService` for each figure; (4) embed figure captions+descriptions as `Document`s (metadata per data-model.md §Figure caption document) into `vectorStore`; (5) embed text chunks into `vectorStore`; (6) `ChunkFigureRefService` for each chunk; update `captionEmbeddingId` on `FigureEntity` after embedding
|
- [X] T018 [US2] Update `BookEmbeddingService.embedBook()` — **Marker migration**: injected `MarkerPageParser` replacing `DocumentAiPageParser`; updated `figureExtractionService.extract()` call (removed `pdfPath` arg); updated log message. Pipeline: (1) `MarkerPageParser` → `List<PageResult>`; (2) `buildAndSaveSections()` → sections; (3) `TextChunkingService` → chunks → embed; (4) `FigureExtractionService.extract()` → figures; (5) `VisionDescriptionService` → embed figure chunks; (6) `ChunkFigureRefService` → refs
|
||||||
- [X] T019 [US2] Extend `BookEmbeddingService.deleteBookChunks()` to also delete: all `ChunkFigureRefEntity` rows (via `findByFigureIdIn`), all `FigureEntity` rows (via `deleteAllByBookId`), all figure PNG files (via `FigureStorageService.delete(bookId)`), all `SectionEntity` and `ChapterEntity` rows for the book
|
- [X] T019 [US2] Extend `BookEmbeddingService.deleteBookChunks()` to also delete: all `ChunkFigureRefEntity` rows (via `findByFigureIdIn`), all `FigureEntity` rows (via `deleteAllByBookId`), all figure PNG files (via `FigureStorageService.delete(bookId)`), all `SectionEntity` and `ChapterEntity` rows for the book
|
||||||
- [X] T020 [US2] Add `POST /api/v1/books/{id}/reembed` endpoint to `BookController` in `backend/src/main/java/com/aiteacher/book/BookController.java` — returns `202` with `{ bookId, status: "PROCESSING" }`; returns `404` if not found; returns `409` if already `PROCESSING`; calls `deleteBookChunks()` then `embedBook()` asynchronously
|
- [X] T020 [US2] Add `POST /api/v1/books/{id}/reembed` endpoint to `BookController` in `backend/src/main/java/com/aiteacher/book/BookController.java` — returns `202` with `{ bookId, status: "PROCESSING" }`; returns `404` if not found; returns `409` if already `PROCESSING`; calls `deleteBookChunks()` then `embedBook()` asynchronously
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,35 @@
|
|||||||
|
# Specification Quality Checklist: Basic Login Protection
|
||||||
|
|
||||||
|
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||||
|
**Created**: 2026-04-06
|
||||||
|
**Feature**: [spec.md](../spec.md)
|
||||||
|
|
||||||
|
## Content Quality
|
||||||
|
|
||||||
|
- [x] No implementation details (languages, frameworks, APIs)
|
||||||
|
- [x] Focused on user value and business needs
|
||||||
|
- [x] Written for non-technical stakeholders
|
||||||
|
- [x] All mandatory sections completed
|
||||||
|
|
||||||
|
## Requirement Completeness
|
||||||
|
|
||||||
|
- [x] No [NEEDS CLARIFICATION] markers remain
|
||||||
|
- [x] Requirements are testable and unambiguous
|
||||||
|
- [x] Success criteria are measurable
|
||||||
|
- [x] Success criteria are technology-agnostic (no implementation details)
|
||||||
|
- [x] All acceptance scenarios are defined
|
||||||
|
- [x] Edge cases are identified
|
||||||
|
- [x] Scope is clearly bounded
|
||||||
|
- [x] Dependencies and assumptions identified
|
||||||
|
|
||||||
|
## Feature Readiness
|
||||||
|
|
||||||
|
- [x] All functional requirements have clear acceptance criteria
|
||||||
|
- [x] User scenarios cover primary flows
|
||||||
|
- [x] Feature meets measurable outcomes defined in Success Criteria
|
||||||
|
- [x] No implementation details leak into specification
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- All items pass. Spec is complete and ready for planning.
|
||||||
|
- FR-012 resolved: credentials are managed via environment variables / config file (no in-app user management UI).
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
# API Contract: Auth
|
||||||
|
|
||||||
|
**Base path**: `/api/v1/auth`
|
||||||
|
**Authentication**: HTTP Basic (all endpoints in this group require valid credentials)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## GET /api/v1/auth/check
|
||||||
|
|
||||||
|
Verifies that the supplied HTTP Basic credentials are valid. Used by the frontend after a page refresh to confirm stored credentials are still accepted before rendering the app.
|
||||||
|
|
||||||
|
### Request
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /api/v1/auth/check
|
||||||
|
Authorization: Basic <base64(username:password)>
|
||||||
|
```
|
||||||
|
|
||||||
|
No request body.
|
||||||
|
|
||||||
|
### Response — 200 OK
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"username": "neurosurgeon"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Type | Description |
|
||||||
|
|-------|------|-------------|
|
||||||
|
| `username` | string | The authenticated username |
|
||||||
|
|
||||||
|
### Response — 401 Unauthorized
|
||||||
|
|
||||||
|
Spring Security returns a standard 401 with `WWW-Authenticate: Basic realm="Realm"` header. No JSON body.
|
||||||
|
|
||||||
|
### Behaviour
|
||||||
|
|
||||||
|
- Returns `200` with the authenticated username if credentials are valid.
|
||||||
|
- Returns `401` if credentials are absent or incorrect.
|
||||||
|
- No side effects (idempotent, read-only).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- All other existing endpoints (`/api/v1/books`, `/api/v1/chat`, etc.) continue to require HTTP Basic Auth as before.
|
||||||
|
- The frontend sends `Authorization: Basic ...` on every request via the axios request interceptor.
|
||||||
|
- A global axios response interceptor detects `401` responses and redirects the user to `/login`.
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
# Data Model: Basic Login Protection
|
||||||
|
|
||||||
|
**Feature**: 003-basic-login
|
||||||
|
**Date**: 2026-04-06
|
||||||
|
|
||||||
|
## No Backend Schema Changes
|
||||||
|
|
||||||
|
This feature introduces no new database tables or Flyway migrations. The user account is defined entirely in the Spring Security in-memory configuration (`SecurityConfig.java`) backed by environment variables.
|
||||||
|
|
||||||
|
## Frontend: Auth Store State
|
||||||
|
|
||||||
|
The Pinia `authStore` is the single source of truth for authentication state in the frontend.
|
||||||
|
|
||||||
|
```
|
||||||
|
AuthState
|
||||||
|
├── username: string | null — entered username, null if not logged in
|
||||||
|
├── password: string | null — entered password, null if not logged in
|
||||||
|
└── isAuthenticated: boolean — derived: true when both username and password are non-null
|
||||||
|
|
||||||
|
Actions
|
||||||
|
├── login(username, password) — validates credentials via /api/v1/auth/check, stores in sessionStorage on success
|
||||||
|
├── logout() — clears username, password, sessionStorage; redirects to /login
|
||||||
|
└── restoreSession() — reads credentials from sessionStorage on app start; calls /api/v1/auth/check to verify still valid
|
||||||
|
```
|
||||||
|
|
||||||
|
## Backend: Application Properties
|
||||||
|
|
||||||
|
Two properties configure the single allowed user account:
|
||||||
|
|
||||||
|
| Property | Default | Source | Example |
|
||||||
|
|----------|---------|--------|---------|
|
||||||
|
| `app.auth.username` | `neurosurgeon` | `application.yaml` / env var `APP_AUTH_USERNAME` | `admin` |
|
||||||
|
| `app.auth.password` | (required) | env var `APP_AUTH_PASSWORD` | `s3cret` |
|
||||||
|
|
||||||
|
No hashing is applied in the current `SecurityConfig` (`{noop}` prefix). The spec (FR-011) requires passwords not to be stored in plaintext — this refers to the backend config/env var pattern, which is acceptable as env vars are not persisted in the codebase. If hashing is required later, the `{noop}` prefix can be replaced with `{bcrypt}` without other code changes.
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
# Implementation Plan: Basic Login Protection
|
||||||
|
|
||||||
|
**Branch**: `003-basic-login` | **Date**: 2026-04-06 | **Spec**: [spec.md](./spec.md)
|
||||||
|
**Input**: Feature specification from `/specs/003-basic-login/spec.md`
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Add a login page to the Vue frontend so users must enter a username and password before accessing any route. The backend already has Spring Security with HTTP Basic Auth fully configured; credentials are validated on every API call. The implementation introduces a Pinia auth store that holds the entered credentials in `sessionStorage`, an axios interceptor that injects them on every request, a `/login` route with a login form, router guards that redirect unauthenticated users, and a logout button in the navbar. A lightweight `/api/v1/auth/check` endpoint is added to the backend to allow the frontend to verify credentials without side effects. Username is made configurable in the backend (currently hardcoded as "neurosurgeon").
|
||||||
|
|
||||||
|
## Technical Context
|
||||||
|
|
||||||
|
**Language/Version**: Java 21 (backend) / TypeScript + Node 20 (frontend)
|
||||||
|
**Primary Dependencies**: Spring Boot 4.0.5, Spring Security (already included), Vue 3.4, Vue Router 4.3, Pinia 2.1, Axios 1.7
|
||||||
|
**Storage**: No new storage — credentials held in browser `sessionStorage` (frontend only)
|
||||||
|
**Testing**: Spring Boot Test (backend), Vitest (not yet set up — out of scope for this feature)
|
||||||
|
**Target Platform**: Web (SPA + REST API)
|
||||||
|
**Project Type**: Web application (backend API + Vue frontend client)
|
||||||
|
**Performance Goals**: Login response within 1 second under normal load
|
||||||
|
**Constraints**: No new backend dependencies; no database changes; must not break existing API surface
|
||||||
|
**Scale/Scope**: Small team (POC), single user role
|
||||||
|
|
||||||
|
## Constitution Check
|
||||||
|
|
||||||
|
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||||
|
|
||||||
|
| Principle | Status | Notes |
|
||||||
|
|-----------|--------|-------|
|
||||||
|
| I. KISS | PASS | HTTP Basic Auth is reused; no new auth protocol, no new dependencies. Frontend uses sessionStorage — no JWT, no refresh tokens. |
|
||||||
|
| II. Easy to Change | PASS | Auth store is a single Pinia store; swapping the auth mechanism later only requires updating the store and the SecurityConfig. |
|
||||||
|
| III. Web-First | PASS | Backend exposes REST endpoint; frontend is standalone SPA client. No server-side rendering added. |
|
||||||
|
| IV. Documentation as Architecture | PASS | README must be updated to show the login flow in the architecture diagram (same PR). |
|
||||||
|
| Technology Constraints | PASS | Still two deployable units (backend + frontend). No new service added. |
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
### Documentation (this feature)
|
||||||
|
|
||||||
|
```text
|
||||||
|
specs/003-basic-login/
|
||||||
|
├── plan.md # This file
|
||||||
|
├── research.md # Phase 0 output
|
||||||
|
├── data-model.md # Phase 1 output
|
||||||
|
├── quickstart.md # Phase 1 output
|
||||||
|
├── contracts/ # Phase 1 output
|
||||||
|
│ └── auth.md
|
||||||
|
└── tasks.md # Phase 2 output (/speckit.tasks — NOT created by /speckit.plan)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Source Code (repository root)
|
||||||
|
|
||||||
|
```text
|
||||||
|
backend/
|
||||||
|
├── src/main/java/com/aiteacher/
|
||||||
|
│ ├── config/
|
||||||
|
│ │ └── SecurityConfig.java # MODIFY: make username configurable
|
||||||
|
│ └── auth/
|
||||||
|
│ └── AuthController.java # ADD: GET /api/v1/auth/check endpoint
|
||||||
|
|
||||||
|
frontend/
|
||||||
|
├── src/
|
||||||
|
│ ├── stores/
|
||||||
|
│ │ └── authStore.ts # ADD: Pinia store for credentials + session
|
||||||
|
│ ├── views/
|
||||||
|
│ │ └── LoginView.vue # ADD: login form UI
|
||||||
|
│ ├── services/
|
||||||
|
│ │ └── api.ts # MODIFY: read credentials from authStore
|
||||||
|
│ ├── router/
|
||||||
|
│ │ └── index.ts # MODIFY: add /login route + navigation guard
|
||||||
|
│ └── App.vue # MODIFY: add logout button to navbar
|
||||||
|
```
|
||||||
|
|
||||||
|
**Structure Decision**: Option 2 (web application). Existing `backend/` and `frontend/` layout used; no new projects or packages.
|
||||||
|
|
||||||
|
## Complexity Tracking
|
||||||
|
|
||||||
|
> No constitution violations. Table left empty.
|
||||||
@@ -0,0 +1,198 @@
|
|||||||
|
# Quickstart: Basic Login Protection
|
||||||
|
|
||||||
|
**Feature**: 003-basic-login
|
||||||
|
**Date**: 2026-04-06
|
||||||
|
|
||||||
|
## What Changes
|
||||||
|
|
||||||
|
| Component | Change |
|
||||||
|
|-----------|--------|
|
||||||
|
| `SecurityConfig.java` | Username made configurable via `app.auth.username` property |
|
||||||
|
| `AuthController.java` | New: `GET /api/v1/auth/check` endpoint |
|
||||||
|
| `authStore.ts` | New: Pinia store managing credentials + sessionStorage |
|
||||||
|
| `LoginView.vue` | New: login form page |
|
||||||
|
| `api.ts` | Replace hardcoded Basic Auth with dynamic interceptor |
|
||||||
|
| `router/index.ts` | Add `/login` route + `beforeEach` navigation guard |
|
||||||
|
| `App.vue` | Add logout button to navbar |
|
||||||
|
| `application.yaml` | Add `app.auth.username` property with default |
|
||||||
|
|
||||||
|
## Backend Setup
|
||||||
|
|
||||||
|
### 1. Add username to application.yaml
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
app:
|
||||||
|
auth:
|
||||||
|
username: ${APP_AUTH_USERNAME:neurosurgeon}
|
||||||
|
password: ${APP_AUTH_PASSWORD} # already present
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Update SecurityConfig.java
|
||||||
|
|
||||||
|
Inject both username and password:
|
||||||
|
|
||||||
|
```java
|
||||||
|
@Bean
|
||||||
|
public UserDetailsService userDetailsService(
|
||||||
|
@Value("${app.auth.username}") String username,
|
||||||
|
@Value("${app.auth.password}") String password) {
|
||||||
|
UserDetails user = User.builder()
|
||||||
|
.username(username)
|
||||||
|
.password("{noop}" + password)
|
||||||
|
.roles("USER")
|
||||||
|
.build();
|
||||||
|
return new InMemoryUserDetailsManager(user);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Add AuthController.java
|
||||||
|
|
||||||
|
```java
|
||||||
|
@RestController
|
||||||
|
@RequestMapping("/api/v1/auth")
|
||||||
|
public class AuthController {
|
||||||
|
@GetMapping("/check")
|
||||||
|
public ResponseEntity<Map<String, String>> check(Principal principal) {
|
||||||
|
return ResponseEntity.ok(Map.of("username", principal.getName()));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Frontend Setup
|
||||||
|
|
||||||
|
### 1. Create authStore.ts
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// src/stores/authStore.ts
|
||||||
|
import { defineStore } from 'pinia'
|
||||||
|
import { ref, computed } from 'vue'
|
||||||
|
|
||||||
|
const SESSION_KEY = 'auth'
|
||||||
|
|
||||||
|
export const useAuthStore = defineStore('auth', () => {
|
||||||
|
const stored = sessionStorage.getItem(SESSION_KEY)
|
||||||
|
const parsed = stored ? JSON.parse(stored) : null
|
||||||
|
|
||||||
|
const username = ref<string | null>(parsed?.username ?? null)
|
||||||
|
const password = ref<string | null>(parsed?.password ?? null)
|
||||||
|
|
||||||
|
const isAuthenticated = computed(() => !!username.value && !!password.value)
|
||||||
|
|
||||||
|
function setCredentials(u: string, p: string) {
|
||||||
|
username.value = u
|
||||||
|
password.value = p
|
||||||
|
sessionStorage.setItem(SESSION_KEY, JSON.stringify({ username: u, password: p }))
|
||||||
|
}
|
||||||
|
|
||||||
|
function clearCredentials() {
|
||||||
|
username.value = null
|
||||||
|
password.value = null
|
||||||
|
sessionStorage.removeItem(SESSION_KEY)
|
||||||
|
}
|
||||||
|
|
||||||
|
return { username, password, isAuthenticated, setCredentials, clearCredentials }
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Update api.ts
|
||||||
|
|
||||||
|
Replace hardcoded `auth` with a request interceptor:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import axios from 'axios'
|
||||||
|
import { useAuthStore } from '@/stores/authStore'
|
||||||
|
|
||||||
|
export const api = axios.create({
|
||||||
|
baseURL: import.meta.env.VITE_API_URL ?? '/api/v1',
|
||||||
|
headers: { 'Content-Type': 'application/json' }
|
||||||
|
})
|
||||||
|
|
||||||
|
api.interceptors.request.use((config) => {
|
||||||
|
const auth = useAuthStore()
|
||||||
|
if (auth.username && auth.password) {
|
||||||
|
config.auth = { username: auth.username, password: auth.password }
|
||||||
|
}
|
||||||
|
return config
|
||||||
|
})
|
||||||
|
|
||||||
|
api.interceptors.response.use(
|
||||||
|
(response) => response,
|
||||||
|
(error) => {
|
||||||
|
if (error.response?.status === 401) {
|
||||||
|
useAuthStore().clearCredentials()
|
||||||
|
window.location.href = '/login'
|
||||||
|
}
|
||||||
|
const message = error.response?.data?.error ?? error.message ?? 'An unexpected error occurred.'
|
||||||
|
return Promise.reject(new Error(message))
|
||||||
|
}
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Update router/index.ts
|
||||||
|
|
||||||
|
Add `/login` route and guard:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import LoginView from '@/views/LoginView.vue'
|
||||||
|
import { useAuthStore } from '@/stores/authStore'
|
||||||
|
|
||||||
|
// add to routes array:
|
||||||
|
{ path: '/login', name: 'login', component: LoginView }
|
||||||
|
|
||||||
|
// add global guard:
|
||||||
|
router.beforeEach((to) => {
|
||||||
|
const auth = useAuthStore()
|
||||||
|
if (to.name !== 'login' && !auth.isAuthenticated) {
|
||||||
|
return { name: 'login' }
|
||||||
|
}
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Create LoginView.vue
|
||||||
|
|
||||||
|
A simple centered form with username and password fields. On submit:
|
||||||
|
1. Store credentials tentatively in the auth store
|
||||||
|
2. Call `GET /api/v1/auth/check`
|
||||||
|
3. If 200 → navigate to `/`
|
||||||
|
4. If 401 → clear credentials, show error message
|
||||||
|
|
||||||
|
### 5. Add logout to App.vue navbar
|
||||||
|
|
||||||
|
```html
|
||||||
|
<button class="btn btn-secondary" @click="logout">Sign out</button>
|
||||||
|
```
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { useAuthStore } from '@/stores/authStore'
|
||||||
|
import { useRouter } from 'vue-router'
|
||||||
|
const auth = useAuthStore()
|
||||||
|
const router = useRouter()
|
||||||
|
function logout() {
|
||||||
|
auth.clearCredentials()
|
||||||
|
router.push({ name: 'login' })
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Environment Variables
|
||||||
|
|
||||||
|
### Backend (.env / docker-compose environment)
|
||||||
|
|
||||||
|
```
|
||||||
|
APP_AUTH_USERNAME=neurosurgeon # optional, defaults to neurosurgeon
|
||||||
|
APP_AUTH_PASSWORD=your-secret
|
||||||
|
```
|
||||||
|
|
||||||
|
### Frontend (.env)
|
||||||
|
|
||||||
|
```
|
||||||
|
VITE_API_URL=/api/v1
|
||||||
|
# VITE_APP_PASSWORD is no longer needed and should be removed
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing the Login Flow
|
||||||
|
|
||||||
|
1. Open the app in an incognito window — should redirect to `/login`
|
||||||
|
2. Enter wrong credentials → error message, stay on login
|
||||||
|
3. Enter correct credentials → redirect to `/` (Library)
|
||||||
|
4. Refresh the page → stay logged in
|
||||||
|
5. Click "Sign out" → redirect to `/login`; back button shows login again (no cached page access)
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
# Research: Basic Login Protection
|
||||||
|
|
||||||
|
**Feature**: 003-basic-login
|
||||||
|
**Date**: 2026-04-06
|
||||||
|
|
||||||
|
## Finding 1: Backend Auth Mechanism — Already Implemented
|
||||||
|
|
||||||
|
**Decision**: Keep existing HTTP Basic Auth (Spring Security, `SecurityConfig.java`).
|
||||||
|
**Rationale**: Spring Security with HTTP Basic is already configured and working. The backend validates credentials on every API request. There is nothing to add except making the username configurable and adding a credential-check endpoint.
|
||||||
|
**Alternatives considered**: Form-based login with server-side sessions — rejected because it adds session management complexity on the backend that is unnecessary for an SPA using HTTP Basic.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Finding 2: Frontend Credential Storage — sessionStorage
|
||||||
|
|
||||||
|
**Decision**: Store entered username and password in browser `sessionStorage` via a Pinia store.
|
||||||
|
**Rationale**:
|
||||||
|
- `sessionStorage` persists across page refreshes (same tab) but is cleared when the tab is closed — this matches the expected session behavior (SC-004) without needing a server-side session or JWT.
|
||||||
|
- Simpler than `localStorage` (no explicit logout needed to clear on browser close).
|
||||||
|
- No additional dependencies required.
|
||||||
|
|
||||||
|
**Alternatives considered**:
|
||||||
|
- `localStorage` — rejected: credentials would persist indefinitely across browser sessions, which is unexpected for a "login" flow.
|
||||||
|
- In-memory (reactive ref only) — rejected: credentials lost on page refresh, violating SC-004.
|
||||||
|
- Cookie-based session (server-side) — rejected: requires CSRF protection, session store, and more backend complexity; violates KISS.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Finding 3: Credential Verification — Lightweight Backend Endpoint
|
||||||
|
|
||||||
|
**Decision**: Add `GET /api/v1/auth/check` that returns `200 OK` with `{"username": "..."}` for authenticated requests.
|
||||||
|
**Rationale**: The frontend needs a way to verify that stored credentials are valid when the app loads (e.g., after a refresh). Without this, the first real API call would fail with a 401 and force a re-login on every refresh if credentials changed. This endpoint is protected by Spring Security like all others — no special logic needed.
|
||||||
|
**Alternatives considered**:
|
||||||
|
- Re-use any existing GET endpoint (e.g., `GET /api/v1/books`) — rejected: couples auth verification to a business endpoint; semantically wrong and fragile.
|
||||||
|
- Intercept 401s globally and redirect to login — used as a fallback but not sufficient alone: the user would see a flash of the main UI before being redirected.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Finding 4: Axios Integration — Request Interceptor
|
||||||
|
|
||||||
|
**Decision**: Replace the hardcoded `auth` field in `api.ts` with a dynamic request interceptor that reads credentials from the Pinia auth store at request time.
|
||||||
|
**Rationale**: The current `api.ts` sets `auth: { username, password }` once at module initialisation from env vars. This must change so the login form's entered credentials are used. A request interceptor reads the store on every call, enabling logout (clear store → next request gets no credentials → 401 → redirect to login).
|
||||||
|
**Alternatives considered**:
|
||||||
|
- Recreate the axios instance after login — rejected: all existing services import the singleton `api`; recreating would require updating every import.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Finding 5: Backend Username Configurability
|
||||||
|
|
||||||
|
**Decision**: Read username from `${app.auth.username:neurosurgeon}` in `SecurityConfig.java` (with "neurosurgeon" as default).
|
||||||
|
**Rationale**: The spec (FR-012) requires credentials to be configurable. Currently the password is configurable via env var but the username is hardcoded. Adding a `@Value`-injected username field is a one-line change.
|
||||||
|
**Alternatives considered**: None — this is the Spring Boot idiomatic approach already used for the password.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Summary of Unknowns Resolved
|
||||||
|
|
||||||
|
| Unknown | Resolution |
|
||||||
|
|---------|-----------|
|
||||||
|
| Where to store credentials on the frontend | `sessionStorage` via Pinia |
|
||||||
|
| How to verify credentials after page refresh | `GET /api/v1/auth/check` endpoint |
|
||||||
|
| How to inject credentials into axios | Request interceptor in `api.ts` |
|
||||||
|
| How to handle 401s globally | Response interceptor → redirect to `/login` |
|
||||||
|
| Backend username configurability | `@Value("${app.auth.username:neurosurgeon}")` |
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
# Feature Specification: Basic Login Protection
|
||||||
|
|
||||||
|
**Feature Branch**: `003-basic-login`
|
||||||
|
**Created**: 2026-04-06
|
||||||
|
**Status**: Draft
|
||||||
|
**Input**: User description: "Add simple and basic login (username and password) to protect the app."
|
||||||
|
|
||||||
|
## User Scenarios & Testing *(mandatory)*
|
||||||
|
|
||||||
|
### User Story 1 - Authenticate to Access the App (Priority: P1)
|
||||||
|
|
||||||
|
A user opens the application and is presented with a login screen. They enter their username and password and, upon successful authentication, gain access to the full application. Without logging in, no part of the application is accessible.
|
||||||
|
|
||||||
|
**Why this priority**: This is the core feature — all other functionality depends on this gate being in place.
|
||||||
|
|
||||||
|
**Independent Test**: Can be fully tested by navigating to any page without credentials (should redirect to login), then logging in with valid credentials (should grant access) — this alone delivers the full MVP value.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** an unauthenticated user, **When** they navigate to any page of the app, **Then** they are redirected to the login screen
|
||||||
|
2. **Given** the login screen, **When** the user enters valid credentials and submits, **Then** they are redirected to the application home/dashboard
|
||||||
|
3. **Given** the login screen, **When** the user enters invalid credentials and submits, **Then** an error message is displayed and they remain on the login screen
|
||||||
|
4. **Given** an authenticated user, **When** they navigate directly to a protected page, **Then** they can access it without re-authenticating
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### User Story 2 - Log Out of the App (Priority: P2)
|
||||||
|
|
||||||
|
An authenticated user can explicitly log out of the application, terminating their session. After logging out, they are redirected to the login screen and must re-authenticate to access the app.
|
||||||
|
|
||||||
|
**Why this priority**: Logout is essential for security — especially on shared machines — but the app is still protected even without explicit logout (session expires).
|
||||||
|
|
||||||
|
**Independent Test**: Can be fully tested by logging in, clicking logout, and confirming that the protected pages are no longer accessible.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** an authenticated user, **When** they click the logout button, **Then** their session is terminated and they are redirected to the login screen
|
||||||
|
2. **Given** a user who has logged out, **When** they navigate to a protected page, **Then** they are redirected to the login screen
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### User Story 3 - Session Persistence Across Browser Refresh (Priority: P3)
|
||||||
|
|
||||||
|
An authenticated user refreshes the page or reopens the browser tab and remains logged in without having to re-enter credentials, as long as their session has not expired.
|
||||||
|
|
||||||
|
**Why this priority**: Improves usability — users should not be forced to log in after every page refresh during normal use.
|
||||||
|
|
||||||
|
**Independent Test**: Can be tested by logging in, refreshing the page, and confirming the user is still authenticated.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** an authenticated user, **When** they refresh the browser, **Then** they remain logged in and on the same page
|
||||||
|
2. **Given** a session that has expired, **When** the user tries to access a protected page, **Then** they are redirected to the login screen
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Edge Cases
|
||||||
|
|
||||||
|
- What happens when the login form is submitted with empty username or password fields?
|
||||||
|
- How does the system handle a user whose credentials are removed/disabled while they have an active session?
|
||||||
|
- What happens if the user attempts to access the login page while already authenticated?
|
||||||
|
- How does the system behave if the session store becomes unavailable?
|
||||||
|
|
||||||
|
## Requirements *(mandatory)*
|
||||||
|
|
||||||
|
### Functional Requirements
|
||||||
|
|
||||||
|
- **FR-001**: System MUST display a login screen (username + password fields and submit button) to unauthenticated users
|
||||||
|
- **FR-002**: System MUST redirect unauthenticated users attempting to access any protected page to the login screen
|
||||||
|
- **FR-003**: System MUST validate submitted credentials against configured or stored credentials
|
||||||
|
- **FR-004**: System MUST create an authenticated session upon successful login
|
||||||
|
- **FR-005**: System MUST display a clear, user-friendly error message when credentials are incorrect (without revealing which field is wrong)
|
||||||
|
- **FR-006**: System MUST provide a logout action that terminates the active session
|
||||||
|
- **FR-007**: System MUST redirect users to the login screen after logout
|
||||||
|
- **FR-008**: System MUST prevent login form submission when username or password is empty
|
||||||
|
- **FR-009**: System MUST automatically expire sessions after a reasonable inactivity period
|
||||||
|
- **FR-010**: System MUST redirect users to the login page if their session has expired
|
||||||
|
- **FR-011**: Credentials MUST be stored securely (passwords hashed, not stored in plaintext)
|
||||||
|
- **FR-012**: System MUST allow at least one user account to be configured via environment variables or a configuration file; credential changes take effect on restart
|
||||||
|
|
||||||
|
### Key Entities
|
||||||
|
|
||||||
|
- **User Account**: Represents a person who can authenticate; has a username (unique identifier) and a hashed password
|
||||||
|
- **Session**: Represents an active authenticated context; linked to a user account, has an expiry time
|
||||||
|
|
||||||
|
## Success Criteria *(mandatory)*
|
||||||
|
|
||||||
|
### Measurable Outcomes
|
||||||
|
|
||||||
|
- **SC-001**: Unauthenticated users cannot access any protected page — 100% of protected routes redirect to login
|
||||||
|
- **SC-002**: Users can complete the login flow (enter credentials, submit, land on the app) in under 30 seconds under normal conditions
|
||||||
|
- **SC-003**: Invalid login attempts display an error within 3 seconds and do not reveal which field was wrong
|
||||||
|
- **SC-004**: Authenticated sessions persist across page refreshes for the configured session duration without requiring re-authentication
|
||||||
|
- **SC-005**: Logout terminates the session immediately — any subsequent request to a protected page results in a redirect to login
|
||||||
|
|
||||||
|
## Assumptions
|
||||||
|
|
||||||
|
- The target users are a small, known group — there is no public self-registration for new accounts
|
||||||
|
- A single set of credentials (or a small number of pre-configured accounts) is sufficient for this initial version
|
||||||
|
- Mobile/responsive design for the login form is expected but full mobile optimization is not the focus of this feature
|
||||||
|
- The app currently has no authentication layer, so this will be added globally
|
||||||
|
- Session duration defaults to a reasonable inactivity timeout (e.g., 30 minutes of inactivity), configurable if needed
|
||||||
|
- A "remember me" / persistent login cookie is out of scope for this initial implementation
|
||||||
@@ -0,0 +1,172 @@
|
|||||||
|
# Tasks: Basic Login Protection
|
||||||
|
|
||||||
|
**Input**: Design documents from `/specs/003-basic-login/`
|
||||||
|
**Prerequisites**: plan.md ✅ spec.md ✅ research.md ✅ data-model.md ✅ contracts/ ✅ quickstart.md ✅
|
||||||
|
|
||||||
|
**Tests**: Not requested — no test tasks included.
|
||||||
|
|
||||||
|
**Organization**: Tasks grouped by user story to enable independent implementation and testing.
|
||||||
|
|
||||||
|
## Format: `[ID] [P?] [Story] Description`
|
||||||
|
|
||||||
|
- **[P]**: Can run in parallel (different files, no dependencies on incomplete tasks)
|
||||||
|
- **[Story]**: Which user story this task belongs to (US1, US2, US3)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1: Setup (Shared Infrastructure)
|
||||||
|
|
||||||
|
**Purpose**: No new project initialization needed — existing backend and frontend projects are in place. Phase 1 confirms the entry points for changes.
|
||||||
|
|
||||||
|
- [x] T001 Verify `spring-boot-starter-security` is present in `backend/pom.xml` (already included — confirm, no change needed)
|
||||||
|
- [x] T002 Verify Pinia is listed in `frontend/package.json` dependencies (already included — confirm, no change needed)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: Foundational (Blocking Prerequisites)
|
||||||
|
|
||||||
|
**Purpose**: Backend credential endpoint and frontend auth store — required by all three user stories.
|
||||||
|
|
||||||
|
**⚠️ CRITICAL**: No user story work can begin until this phase is complete.
|
||||||
|
|
||||||
|
- [x] T003 Add `app.auth.username` property to `backend/src/main/resources/application.yaml` with value `${APP_AUTH_USERNAME:neurosurgeon}` alongside the existing `app.auth.password` entry
|
||||||
|
- [x] T004 Update `backend/src/main/java/com/aiteacher/config/SecurityConfig.java` to inject `@Value("${app.auth.username}")` and pass it to `User.builder().username(username)` instead of the hardcoded string `"neurosurgeon"`
|
||||||
|
- [x] T005 Create `backend/src/main/java/com/aiteacher/auth/AuthController.java` — `@RestController` at `/api/v1/auth`, with a single `GET /check` method that accepts a `Principal` argument and returns `ResponseEntity.ok(Map.of("username", principal.getName()))`
|
||||||
|
- [x] T006 Create `frontend/src/stores/authStore.ts` — Pinia store with `username` and `password` refs (initially `null`), `isAuthenticated` computed, `setCredentials(u, p)` action, and `clearCredentials()` action (sessionStorage persistence added in Phase 5 / US3)
|
||||||
|
|
||||||
|
**Checkpoint**: Backend exposes `GET /api/v1/auth/check`; `authStore` is callable from any Vue component.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: User Story 1 - Authenticate to Access the App (Priority: P1) 🎯 MVP
|
||||||
|
|
||||||
|
**Goal**: Unauthenticated users are redirected to `/login`; successful credential entry grants full app access.
|
||||||
|
|
||||||
|
**Independent Test**: Open the app in incognito → redirected to `/login`. Enter wrong credentials → error shown. Enter correct credentials → land on Library (`/`). Refresh the page → stays on Library (credentials held in-memory for now; persistence comes in US3).
|
||||||
|
|
||||||
|
### Implementation for User Story 1
|
||||||
|
|
||||||
|
- [x] T007 [US1] Create `frontend/src/views/LoginView.vue` — centered card with username input, password input, submit button, and an error message area; on submit, call `authStore.setCredentials(u, p)`, then call `GET /api/v1/auth/check` via the `api` service; on 200 navigate to `/`; on failure call `authStore.clearCredentials()` and display the error
|
||||||
|
- [x] T008 [US1] Update `frontend/src/services/api.ts` — remove the hardcoded `auth: { username, password }` field from the axios instance; add a **request interceptor** that reads `authStore.username` and `authStore.password` and sets `config.auth` dynamically; add a **response interceptor** that on `401` calls `authStore.clearCredentials()` and redirects to `/login` (replace the existing error-normalisation interceptor rather than adding a second one — keep error normalisation intact)
|
||||||
|
- [x] T009 [US1] Update `frontend/src/router/index.ts` — add a `{ path: '/login', name: 'login', component: LoginView }` route; add a `router.beforeEach` guard that redirects to `{ name: 'login' }` when `to.name !== 'login'` and `!authStore.isAuthenticated`
|
||||||
|
|
||||||
|
**Checkpoint**: US1 fully functional — incognito flow, failed login, and successful login all work independently.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4: User Story 2 - Log Out of the App (Priority: P2)
|
||||||
|
|
||||||
|
**Goal**: Authenticated users can log out, terminating their session and returning to `/login`.
|
||||||
|
|
||||||
|
**Independent Test**: Log in → click "Sign out" in the navbar → redirected to `/login`; navigating back to any protected route redirects to `/login` again.
|
||||||
|
|
||||||
|
### Implementation for User Story 2
|
||||||
|
|
||||||
|
- [x] T010 [US2] Update `frontend/src/App.vue` — import `useAuthStore` and `useRouter`; add a "Sign out" button to the navbar (visible only when `authStore.isAuthenticated`); clicking it calls `authStore.clearCredentials()` then `router.push({ name: 'login' })`; hide the navbar links (`RouterLink` items) when on the login page by checking `$route.name !== 'login'`
|
||||||
|
|
||||||
|
**Checkpoint**: US2 fully functional — logout clears session and blocks re-entry without credentials.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 5: User Story 3 - Session Persistence Across Browser Refresh (Priority: P3)
|
||||||
|
|
||||||
|
**Goal**: Authenticated users survive a page refresh without re-logging in; expired/invalid stored credentials redirect to `/login`.
|
||||||
|
|
||||||
|
**Independent Test**: Log in → refresh the browser → remain on the same page without re-entering credentials. Manually clear `sessionStorage` and refresh → redirected to `/login`.
|
||||||
|
|
||||||
|
### Implementation for User Story 3
|
||||||
|
|
||||||
|
- [x] T011 [US3] Update `frontend/src/stores/authStore.ts` — in `setCredentials`, write `{ username, password }` to `sessionStorage` under a key (e.g. `'auth'`); in `clearCredentials`, call `sessionStorage.removeItem('auth')`; on store initialization (module load), read from `sessionStorage` and pre-populate `username` and `password` refs if present
|
||||||
|
- [x] T012 [US3] Update `frontend/src/main.ts` — after creating the app and mounting Pinia, call `authStore.restoreSession()` (or inline the check): if `authStore.isAuthenticated`, call `GET /api/v1/auth/check`; if the response is `401`, call `authStore.clearCredentials()` so stale stored credentials are evicted before the router guard fires
|
||||||
|
|
||||||
|
**Checkpoint**: US3 fully functional — refresh persists login; stale or invalidated credentials are detected on load.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 6: Polish & Cross-Cutting Concerns
|
||||||
|
|
||||||
|
**Purpose**: Documentation and cleanup that spans all user stories.
|
||||||
|
|
||||||
|
- [x] T013 [P] Update `frontend/.env.example` — remove `VITE_APP_PASSWORD` (the frontend no longer reads a password from env; add a comment explaining credentials are now entered via the login form)
|
||||||
|
- [x] T014 [P] Update `README.md` — add or update the Mermaid architecture diagram to show the login flow: browser → login form → `/api/v1/auth/check` → app; this satisfies Constitution Principle IV (diagram must be updated in the same PR as any architectural change)
|
||||||
|
|
||||||
|
**Checkpoint**: Feature complete — all three user stories functional, documentation current, obsolete env var removed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dependencies & Execution Order
|
||||||
|
|
||||||
|
### Phase Dependencies
|
||||||
|
|
||||||
|
- **Setup (Phase 1)**: No dependencies — confirm immediately
|
||||||
|
- **Foundational (Phase 2)**: Depends on Phase 1 — **BLOCKS all user stories**
|
||||||
|
- **US1 (Phase 3)**: Depends on Phase 2 completion
|
||||||
|
- **US2 (Phase 4)**: Depends on Phase 2 completion; integrates with authStore from Phase 2 and router from US1
|
||||||
|
- **US3 (Phase 5)**: Depends on Phase 2 completion; extends authStore from Phase 2
|
||||||
|
- **Polish (Phase 6)**: Depends on all desired stories complete
|
||||||
|
|
||||||
|
### User Story Dependencies
|
||||||
|
|
||||||
|
- **US1 (P1)**: Depends only on Foundational — the primary blocker for all UI work
|
||||||
|
- **US2 (P2)**: Depends on Foundational and US1 (logout button lives in App.vue which needs the router guard from US1)
|
||||||
|
- **US3 (P3)**: Depends on Foundational only — authStore persistence is independent of the login form; can be developed in parallel with US1 if desired
|
||||||
|
|
||||||
|
### Within Each User Story
|
||||||
|
|
||||||
|
- Foundational tasks (T003–T006) must all complete before US1 starts
|
||||||
|
- T007 (LoginView) and T008 (api.ts) can be developed in parallel within US1; T009 (router guard) depends on T007 existing
|
||||||
|
- US2 is a single task (T010) with no internal ordering complexity
|
||||||
|
- T012 (main.ts restore check) depends on T011 (authStore persistence) within US3
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Parallel Example: Foundational Phase
|
||||||
|
|
||||||
|
```
|
||||||
|
Parallelizable within Phase 2:
|
||||||
|
T003 — application.yaml update
|
||||||
|
T004 — SecurityConfig.java update
|
||||||
|
T005 — AuthController.java (new file)
|
||||||
|
T006 — authStore.ts (new file)
|
||||||
|
All four touch different files with no shared dependencies.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Parallel Example: User Story 1
|
||||||
|
|
||||||
|
```
|
||||||
|
Parallelizable within Phase 3:
|
||||||
|
T007 — LoginView.vue (new file)
|
||||||
|
T008 — api.ts update (different file)
|
||||||
|
Then sequentially:
|
||||||
|
T009 — router/index.ts (depends on LoginView existing)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementation Strategy
|
||||||
|
|
||||||
|
### MVP First (User Story 1 Only)
|
||||||
|
|
||||||
|
1. Complete Phase 1: Confirm existing deps
|
||||||
|
2. Complete Phase 2: Foundational — backend endpoint + auth store
|
||||||
|
3. Complete Phase 3: US1 — login page, axios interceptors, router guard
|
||||||
|
4. **STOP and VALIDATE**: Open incognito, verify redirect → login → success flow
|
||||||
|
5. Demo / merge if MVP is sufficient
|
||||||
|
|
||||||
|
### Incremental Delivery
|
||||||
|
|
||||||
|
1. Phase 1 + Phase 2 → Foundation ready
|
||||||
|
2. Phase 3 (US1) → Login gate works — **MVP** ✅
|
||||||
|
3. Phase 4 (US2) → Logout works ✅
|
||||||
|
4. Phase 5 (US3) → Session survives refresh ✅
|
||||||
|
5. Phase 6 → Documentation and cleanup ✅
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- [P] tasks touch different files and have no dependency on an incomplete sibling task in the same phase
|
||||||
|
- No tests included (not requested in the spec)
|
||||||
|
- `VITE_APP_PASSWORD` should be removed from `.env.example` once T013 is done — do **not** remove it from any local `.env` file before the login form is working (T007–T009 complete)
|
||||||
|
- The 401 response interceptor in T008 handles the edge case where stored credentials become invalid server-side — no additional handling needed
|
||||||
|
- Constitution IV requires the README Mermaid diagram to be updated in the **same PR** — T014 must not be skipped
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
# Specification Quality Checklist: RAG Retrieval Quality Improvements
|
||||||
|
|
||||||
|
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||||
|
**Created**: 2026-04-06
|
||||||
|
**Feature**: [spec.md](../spec.md)
|
||||||
|
|
||||||
|
## Content Quality
|
||||||
|
|
||||||
|
- [x] No implementation details (languages, frameworks, APIs)
|
||||||
|
- [x] Focused on user value and business needs
|
||||||
|
- [x] Written for non-technical stakeholders
|
||||||
|
- [x] All mandatory sections completed
|
||||||
|
|
||||||
|
## Requirement Completeness
|
||||||
|
|
||||||
|
- [x] No [NEEDS CLARIFICATION] markers remain
|
||||||
|
- [x] Requirements are testable and unambiguous
|
||||||
|
- [x] Success criteria are measurable
|
||||||
|
- [x] Success criteria are technology-agnostic (no implementation details)
|
||||||
|
- [x] All acceptance scenarios are defined
|
||||||
|
- [x] Edge cases are identified
|
||||||
|
- [x] Scope is clearly bounded
|
||||||
|
- [x] Dependencies and assumptions identified
|
||||||
|
|
||||||
|
## Feature Readiness
|
||||||
|
|
||||||
|
- [x] All functional requirements have clear acceptance criteria
|
||||||
|
- [x] User scenarios cover primary flows
|
||||||
|
- [x] Feature meets measurable outcomes defined in Success Criteria
|
||||||
|
- [x] No implementation details leak into specification
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- All items pass. Ready to proceed to `/speckit.clarify` or `/speckit.plan`.
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# API Contract: Chat (unchanged endpoints)
|
||||||
|
|
||||||
|
**Branch**: `004-rag-retrieval-quality` | **Date**: 2026-04-06
|
||||||
|
|
||||||
|
## No endpoint changes
|
||||||
|
|
||||||
|
This feature makes no changes to the public API surface. All existing `/api/v1/chat/...` endpoints remain identical in path, method, request body, and response shape.
|
||||||
|
|
||||||
|
## Message response — `sources` field addition
|
||||||
|
|
||||||
|
The only observable change to callers is that each source entry in `Message.sources` gains an optional `refLabel` field. This is backwards-compatible (additive only).
|
||||||
|
|
||||||
|
### Existing contract (unchanged)
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /api/v1/chat/sessions/{sessionId}/messages
|
||||||
|
Body: { "content": "..." }
|
||||||
|
Response: Message { id, sessionId, role, content, sources: [...], createdAt }
|
||||||
|
```
|
||||||
|
|
||||||
|
### Source entry schema (additive change)
|
||||||
|
|
||||||
|
Before:
|
||||||
|
```json
|
||||||
|
{ "type": "TEXT", "bookTitle": "...", "page": 142, "chunkText": "..." }
|
||||||
|
```
|
||||||
|
|
||||||
|
After (new optional field):
|
||||||
|
```json
|
||||||
|
{ "type": "TEXT", "refLabel": "S1", "bookTitle": "...", "page": 142, "chunkText": "..." }
|
||||||
|
```
|
||||||
|
|
||||||
|
Frontend consumers that ignore unknown fields are unaffected. `ChatMessage.vue` may optionally use `refLabel` for future inline citation linking.
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
# API Contract: Topics — Summary Persistence
|
||||||
|
|
||||||
|
**Base path**: `/api/v1/topics`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Existing endpoint (unchanged behaviour, extended response)
|
||||||
|
|
||||||
|
### `POST /api/v1/topics/{id}/summary`
|
||||||
|
|
||||||
|
Generates a new summary for the topic, **persists it**, and returns it.
|
||||||
|
|
||||||
|
**Path param**: `id` — topic id (e.g. `intracranial-aneurysms`)
|
||||||
|
|
||||||
|
**Response** `200 OK`:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
|
||||||
|
"summaryNumber": 3,
|
||||||
|
"topicId": "intracranial-aneurysms",
|
||||||
|
"topicName": "Intracranial Aneurysms",
|
||||||
|
"summary": "## Key Points\n...",
|
||||||
|
"sources": [
|
||||||
|
{ "bookId": "uuid", "bookTitle": "Youmans & Winn", "page": 142 }
|
||||||
|
],
|
||||||
|
"generatedAt": "2026-04-07T10:23:00Z"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Error responses**:
|
||||||
|
- `404 Not Found` — topic id does not exist
|
||||||
|
- `503 Service Unavailable` — no books ready (existing `NoKnowledgeSourceException`)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## New endpoints
|
||||||
|
|
||||||
|
### `GET /api/v1/topics/{id}/summaries`
|
||||||
|
|
||||||
|
Returns the list of saved summaries for a topic (no full text — list metadata only).
|
||||||
|
|
||||||
|
**Path param**: `id` — topic id
|
||||||
|
|
||||||
|
**Response** `200 OK`:
|
||||||
|
```json
|
||||||
|
[
|
||||||
|
{ "id": "uuid-1", "summaryNumber": 1, "generatedAt": "2026-04-06T08:00:00Z" },
|
||||||
|
{ "id": "uuid-2", "summaryNumber": 2, "generatedAt": "2026-04-06T09:15:00Z" }
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
Ordered ascending by `summaryNumber`. Empty array if no summaries saved yet.
|
||||||
|
|
||||||
|
**Error responses**:
|
||||||
|
- `404 Not Found` — topic id does not exist
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `GET /api/v1/topics/{id}/summaries/{summaryId}`
|
||||||
|
|
||||||
|
Fetches the full content of a specific saved summary.
|
||||||
|
|
||||||
|
**Path params**: `id` — topic id, `summaryId` — summary UUID
|
||||||
|
|
||||||
|
**Response** `200 OK`: same shape as `POST /summary` response above.
|
||||||
|
|
||||||
|
**Error responses**:
|
||||||
|
- `404 Not Found` — topic or summary not found
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
# Data Model: RAG Retrieval Quality + Topic Summary Persistence
|
||||||
|
|
||||||
|
**Branch**: `004-rag-retrieval-quality` | **Date**: 2026-04-07
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## New persistent entity: TopicSummaryEntity
|
||||||
|
|
||||||
|
**Table**: `topic_summary`
|
||||||
|
**Migration**: `V6__topic_summary.sql`
|
||||||
|
|
||||||
|
| Column | Type | Notes |
|
||||||
|
|--------|------|-------|
|
||||||
|
| `id` | `UUID` PK | `gen_random_uuid()` default |
|
||||||
|
| `topic_id` | `VARCHAR(100)` NOT NULL | FK to `topic.id` |
|
||||||
|
| `summary_number` | `INT` NOT NULL | Sequential per topic (1, 2, 3, …). Set at insert time: `COUNT(*) WHERE topic_id = ? + 1`. |
|
||||||
|
| `summary` | `TEXT` NOT NULL | Full markdown summary text |
|
||||||
|
| `sources_json` | `TEXT` NOT NULL | JSON array of `SourceReference` objects (same structure as `TopicSummaryResponse.sources`) |
|
||||||
|
| `generated_at` | `TIMESTAMPTZ` NOT NULL | UTC timestamp of generation |
|
||||||
|
|
||||||
|
**Constraints**: no unique constraint on `summary_number` (sequential, not concurrent-safe for POC). No FK constraint enforced at DB level (topic ids are static seed data).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## In-memory objects (new, from RAG quality work)
|
||||||
|
|
||||||
|
### ExpandedQuery (value object, not persisted)
|
||||||
|
|
||||||
|
Produced by `QueryExpansionService` for each user message.
|
||||||
|
|
||||||
|
| Field | Type | Description |
|
||||||
|
|-------|------|-------------|
|
||||||
|
| `original` | `String` | The user's literal question |
|
||||||
|
| `rewritten` | `String` | Clinically rewritten version used for vector search |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### LabelledContext (value object, not persisted)
|
||||||
|
|
||||||
|
Produced by `ChatService.buildContextPrompt()` to track the mapping from ref-labels to source entities.
|
||||||
|
|
||||||
|
| Field | Type | Description |
|
||||||
|
|-------|------|-------------|
|
||||||
|
| `sectionLabels` | `Map<String, SectionEntity>` | e.g. `{"S1" → SectionEntity, "S2" → SectionEntity}` |
|
||||||
|
| `figureLabels` | `Map<String, FigureEntity>` | e.g. `{"F1" → FigureEntity}` |
|
||||||
|
| `promptText` | `String` | The fully formatted context prompt including `[S1]`, `[F1]` tags |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## New API DTOs
|
||||||
|
|
||||||
|
### SavedSummaryItem (list view — no full text)
|
||||||
|
|
||||||
|
```java
|
||||||
|
record SavedSummaryItem(UUID id, int summaryNumber, Instant generatedAt) {}
|
||||||
|
```
|
||||||
|
|
||||||
|
Used in `GET /api/v1/topics/{id}/summaries` to show the summary history list without transmitting full text.
|
||||||
|
|
||||||
|
### TopicSummaryResponse (existing, extended)
|
||||||
|
|
||||||
|
Adds `id` (UUID) and `summaryNumber` (int) fields so the frontend knows which saved record was just created.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Existing entities (unchanged)
|
||||||
|
|
||||||
|
| Entity | Table | Change |
|
||||||
|
|--------|-------|--------|
|
||||||
|
| `SectionEntity` | `section` | None |
|
||||||
|
| `FigureEntity` | `figure` | None |
|
||||||
|
| `Message` | `message` | `sources` field gets `refLabel` key added per entry |
|
||||||
|
| `ChatSession` | `chat_session` | None |
|
||||||
|
| `Book` | `book` | None |
|
||||||
|
| `Topic` | `topic` | None |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Message.sources structure (existing, clarified)
|
||||||
|
|
||||||
|
After the RAG quality feature each entry includes `refLabel`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "TEXT",
|
||||||
|
"refLabel": "S1",
|
||||||
|
"bookTitle": "Youmans & Winn Neurological Surgery",
|
||||||
|
"page": 142,
|
||||||
|
"chunkText": "..."
|
||||||
|
}
|
||||||
|
```
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
# Implementation Plan: RAG Retrieval Quality + Topic Summary Persistence
|
||||||
|
|
||||||
|
**Branch**: `004-rag-retrieval-quality` | **Date**: 2026-04-07 | **Spec**: [spec.md](spec.md)
|
||||||
|
**Input**: Feature specification from `/specs/004-rag-retrieval-quality/spec.md` + user request: "Save summaries by topic; list previously saved summaries; button to generate new."
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Improve RAG retrieval quality by (1) expanding user queries via LLM rewrite to bridge vocabulary gaps and (2) validating generated citations against the retrieved context to eliminate hallucinated references. Additionally, persist generated topic summaries to the database so students can revisit past summaries and generate new ones on demand.
|
||||||
|
|
||||||
|
## Technical Context
|
||||||
|
|
||||||
|
**Language/Version**: Java 21 (backend), TypeScript / Node 20 (frontend)
|
||||||
|
**Primary Dependencies**: Spring Boot 4.0.5, Spring AI 2.0.0-M4, OpenAI API (chat + embeddings), Vue 3.4, Pinia 2.1, Axios 1.7
|
||||||
|
**Storage**: PostgreSQL (JPA + Flyway), pgvector (`VectorStore`)
|
||||||
|
**Testing**: JUnit 5, Spring Boot Test, Vitest
|
||||||
|
**Target Platform**: Linux server
|
||||||
|
**Project Type**: Web application (backend API + Vue frontend)
|
||||||
|
**Performance Goals**: RAG query latency remains acceptable (one additional LLM call for query expansion, ~1–2s overhead)
|
||||||
|
**Constraints**: KISS — no new architectural layers; no new external services; single deployable backend + frontend
|
||||||
|
|
||||||
|
## Constitution Check
|
||||||
|
|
||||||
|
| Principle | Status | Notes |
|
||||||
|
|-----------|--------|-------|
|
||||||
|
| I — KISS | ✅ Pass | Query expansion = one LLM call. Citation validation = string scan. Summary persistence = one new table + minimal service change. No new layers. |
|
||||||
|
| II — Easy to Change | ✅ Pass | `QueryExpansionService`, `CitationValidatorService`, `TopicSummaryRepository` are small, focused interfaces. Swappable without touching callers. |
|
||||||
|
| III — Web-First | ✅ Pass | All new functionality exposed via REST at `/api/v1/...`. Frontend-only UI changes. |
|
||||||
|
| IV — Documentation as Architecture | ⚠️ Action needed | README must be updated to reflect new `topic_summary` table and the summary history flow. Update in the same PR. |
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
### Documentation (this feature)
|
||||||
|
|
||||||
|
```text
|
||||||
|
specs/004-rag-retrieval-quality/
|
||||||
|
├── plan.md # This file
|
||||||
|
├── research.md # Phase 0 — decisions on query expansion, citation grounding, summary persistence
|
||||||
|
├── data-model.md # Phase 1 — topic_summary table, ExpandedQuery, LabelledContext DTOs
|
||||||
|
├── contracts/
|
||||||
|
│ ├── chat-api.md # Existing chat API (updated for labelled sources)
|
||||||
|
│ └── topics-api.md # New/updated topics API (summary list + detail endpoints)
|
||||||
|
└── tasks.md # Phase 2 output (/speckit.tasks command)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Source Code
|
||||||
|
|
||||||
|
```text
|
||||||
|
backend/
|
||||||
|
├── src/main/java/com/aiteacher/
|
||||||
|
│ ├── retrieval/
|
||||||
|
│ │ ├── QueryExpansionService.java (NEW) — LLM rewrite of user query
|
||||||
|
│ │ ├── ExpandedQuery.java (NEW) — value object {original, rewritten}
|
||||||
|
│ │ ├── LabelledContext.java (NEW) — value object {sectionLabels, figureLabels, promptText}
|
||||||
|
│ │ └── CitationValidatorService.java (NEW) — strip unknown [Sx]/[Fx] refs from answer text
|
||||||
|
│ └── topic/
|
||||||
|
│ ├── TopicSummaryEntity.java (NEW) — JPA entity for topic_summary table
|
||||||
|
│ ├── TopicSummaryRepository.java (NEW) — findByTopicIdOrderBySummaryNumberAsc
|
||||||
|
│ ├── SavedSummaryItem.java (NEW) — DTO for list view (id, summaryNumber, generatedAt)
|
||||||
|
│ ├── TopicSummaryResponse.java (MODIFY) — add id: UUID, summaryNumber: int fields
|
||||||
|
│ ├── TopicSummaryService.java (MODIFY) — persist after generation; add list/get methods
|
||||||
|
│ └── TopicController.java (MODIFY) — add GET /summaries and GET /summaries/{id}
|
||||||
|
├── src/main/resources/db/migration/
|
||||||
|
│ └── V6__topic_summary.sql (NEW) — create topic_summary table
|
||||||
|
└── chat/
|
||||||
|
└── ChatService.java (MODIFY) — use QueryExpansionService + CitationValidatorService
|
||||||
|
|
||||||
|
frontend/
|
||||||
|
└── src/
|
||||||
|
├── stores/
|
||||||
|
│ └── topicStore.ts (MODIFY) — add fetchSummaries(), fetchSummaryDetail(), summaryList state
|
||||||
|
└── views/
|
||||||
|
└── TopicsView.vue (MODIFY) — show summary list when topic selected, "Generate New" button
|
||||||
|
```
|
||||||
|
|
||||||
|
**Structure Decision**: Option 2 (web application). Existing `backend/` + `frontend/` directories. No new top-level directories.
|
||||||
|
|
||||||
|
## Complexity Tracking
|
||||||
|
|
||||||
|
> No constitution violations introduced.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 0: Research (complete)
|
||||||
|
|
||||||
|
See [research.md](research.md). All decisions resolved:
|
||||||
|
- Query expansion: single LLM rewrite call
|
||||||
|
- Citation grounding: ref-label tagging + post-generation string scan
|
||||||
|
- Summary persistence: new `topic_summary` table, sequential numbering at insert time
|
||||||
|
|
||||||
|
## Phase 1: Design (complete)
|
||||||
|
|
||||||
|
### Data model
|
||||||
|
See [data-model.md](data-model.md). New `topic_summary` table; two new in-memory value objects; `TopicSummaryResponse` extended with `id` + `summaryNumber`.
|
||||||
|
|
||||||
|
### API contracts
|
||||||
|
See [contracts/topics-api.md](contracts/topics-api.md):
|
||||||
|
- `POST /api/v1/topics/{id}/summary` — now persists and returns `id` + `summaryNumber`
|
||||||
|
- `GET /api/v1/topics/{id}/summaries` — list metadata (no full text)
|
||||||
|
- `GET /api/v1/topics/{id}/summaries/{summaryId}` — full detail
|
||||||
|
|
||||||
|
### Frontend behaviour
|
||||||
|
|
||||||
|
When a **topic card is clicked** (before generating):
|
||||||
|
1. Call `GET /api/v1/topics/{id}/summaries`.
|
||||||
|
2. If summaries exist → show list panel with chips: "Summary #1 · Apr 6", "Summary #2 · Apr 7", … + a "Generate New" button.
|
||||||
|
3. If no summaries → show "Generate Summary" button (current behaviour).
|
||||||
|
4. Clicking a chip → call `GET /api/v1/topics/{id}/summaries/{summaryId}` → display the saved summary.
|
||||||
|
5. Clicking "Generate New" → call `POST /api/v1/topics/{id}/summary` → display new summary, refresh list.
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# Research: RAG Retrieval Quality Improvements
|
||||||
|
|
||||||
|
**Branch**: `004-rag-retrieval-quality` | **Date**: 2026-04-06
|
||||||
|
|
||||||
|
## Decision 1: Query Expansion Strategy
|
||||||
|
|
||||||
|
**Decision**: Single LLM rewrite — ask the model to restate the user's question using clinical/technical terminology before retrieval. Use the rewritten query for vector search instead of (or in addition to) the original.
|
||||||
|
|
||||||
|
**Rationale**: The simplest approach that directly addresses vocabulary mismatch. A single extra LLM call rewrites the query into the language of the documentation (clinical terms), so the embedding similarity search has a much better chance of matching. No new dependencies, no index changes.
|
||||||
|
|
||||||
|
**Alternatives considered**:
|
||||||
|
- *HyDE (Hypothetical Document Embeddings)*: Ask the model to write a hypothetical answer, then embed that. More powerful but adds latency and the answer may itself hallucinate clinical content — rejected for POC.
|
||||||
|
- *Multi-query retrieval*: Generate 3–5 alternative queries and merge results. Effective but multiplies retrieval calls and deduplication complexity — rejected (KISS).
|
||||||
|
- *Synonym dictionary*: Pre-built medical thesaurus mapping. No new dependencies but requires maintenance and won't generalise — rejected.
|
||||||
|
- *Re-ranking*: Run a cross-encoder after retrieval. Addresses ordering, not vocabulary gap — deferred.
|
||||||
|
|
||||||
|
**Implementation note**: The rewrite prompt should be a short, focused instruction: "Rewrite the following question using precise medical/surgical terminology as it would appear in a neurosurgery textbook index. Output only the rewritten question." Use the same `ChatClient` bean; no new API client needed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Decision 2: Citation Grounding Strategy
|
||||||
|
|
||||||
|
**Decision**: Tag each retrieved context section with a short ref-label (`[S1]`, `[S2]`, …, `[Fn]` for figures) in the prompt. Instruct the model to cite only using those labels. Post-process the generated answer to detect and strip any citation that does not correspond to a known label.
|
||||||
|
|
||||||
|
**Rationale**: The simplest way to make citations verifiable — all valid citation targets are enumerated in the prompt itself. Post-processing is a pure string operation; no second LLM call needed for validation.
|
||||||
|
|
||||||
|
**Alternatives considered**:
|
||||||
|
- *Ask the model to self-check citations*: Second LLM call to verify each claim. More accurate but doubles cost and latency — rejected for POC.
|
||||||
|
- *Structured output (JSON)*: Return answer as JSON with claim/source pairs. Most precise but requires significant prompt engineering and frontend changes — deferred.
|
||||||
|
- *No citation enforcement*: Let the model cite freely and show retrieved sources separately. Already the current state — rejected because it doesn't solve citation hallucination.
|
||||||
|
|
||||||
|
**Implementation note**:
|
||||||
|
- Context sections labelled `[S1] Section Title, p.N` through `[Sk]` in `buildContextPrompt()`.
|
||||||
|
- Figures labelled `[F1]`, `[F2]` etc.
|
||||||
|
- System prompt updated: "Cite claims using ONLY the reference labels [S1]…[Sk] and [F1]…[Fj] provided in the context. Do not invent page numbers or section titles."
|
||||||
|
- `CitationValidatorService` scans generated text for `[Sx]` / `[Fx]` patterns, checks each against the known label set, and removes unknown ones.
|
||||||
|
- `Message.sources` is already a `List<Map<String,Object>>`. No schema change needed — but we store the ref-label alongside each source so the frontend can correlate.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Decision 3: Frontend Source Display
|
||||||
|
|
||||||
|
**Decision**: No frontend change needed for MVP. The backend already filters out hallucinated citations via `CitationValidatorService` before saving the message. The `sources` list attached to the message is already the retrieved set. The existing `ChatMessage.vue` source panel continues to show all retrieved sources (which is correct — they were all used as context).
|
||||||
|
|
||||||
|
**Rationale**: KISS. The critical correctness fix (no hallucinated citations in answer text) is fully backend-side. The sources panel shows context that was genuinely available to the model — that is accurate and useful. Linking specific claims to specific sources (inline highlighting) is a UX enhancement that can be a follow-on feature.
|
||||||
|
|
||||||
|
**Alternatives considered**:
|
||||||
|
- *Inline citation links*: Highlight `[S1]` in answer text and link to the source card. Better UX but requires markdown parsing and component changes — deferred.
|
||||||
|
- *Show only cited sources*: Filter `sources` to only those actually cited in the answer. Marginally more accurate but the uncited context is still legitimately retrieved — deferred.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Decision 4: Persisting Topic Summaries
|
||||||
|
|
||||||
|
**Decision**: Add a new `topic_summary` table that stores each generated summary. Summaries are numbered sequentially per topic (summary #1, #2, …). The POST endpoint continues to generate and now also persists. A new GET endpoint lists saved summaries for a topic.
|
||||||
|
|
||||||
|
**Rationale**: The simplest approach — one new table, one new repository, minimal changes to the existing service. No caching layer, no event system. The sequential number is derived at query time via `ROW_NUMBER` or counted in the repository, keeping the schema minimal.
|
||||||
|
|
||||||
|
**Alternatives considered**:
|
||||||
|
- *Store in frontend state only (session memory)*: Already the current state — summaries are lost on reload. Rejected because the user explicitly wants persistence.
|
||||||
|
- *Store summary_number as a persisted column*: Avoids a query-time count but risks gaps/duplicates on concurrent writes. For a POC with single-user use, a `SELECT COUNT(*) + 1` approach at insert time is sufficient.
|
||||||
|
- *Versioning / soft-delete*: Overkill for a POC where the user just wants to re-read old summaries. No delete endpoint needed initially.
|
||||||
|
|
||||||
|
**Implementation note**:
|
||||||
|
- New Flyway migration `V6__topic_summary.sql`.
|
||||||
|
- `TopicSummaryEntity` (JPA `@Entity` → table `topic_summary`): `id` (UUID), `topicId` (VARCHAR FK), `summaryNumber` (INT), `summary` (TEXT), `sourcesJson` (TEXT — JSON array), `generatedAt` (TIMESTAMP).
|
||||||
|
- `summaryNumber` set at insert time: `COUNT(*) WHERE topic_id = ?` + 1.
|
||||||
|
- `TopicSummaryRepository extends JpaRepository<TopicSummaryEntity, UUID>` with a `findByTopicIdOrderBySummaryNumberAsc` query.
|
||||||
|
- `TopicSummaryService.generateSummary()` saves the result before returning it.
|
||||||
|
- `TopicController` gains `GET /api/v1/topics/{id}/summaries` returning `List<SavedSummaryResponse>` (id, summaryNumber, generatedAt — no full text, for list efficiency) and `GET /api/v1/topics/{id}/summaries/{summaryId}` for the full detail.
|
||||||
|
- Frontend: when a topic card is clicked, fetch the summary list. Show "Summary #1", "Summary #2" chips + "Generate New" button. Clicking a chip loads the full summary.
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
# Feature Specification: RAG Retrieval Quality Improvements
|
||||||
|
|
||||||
|
**Feature Branch**: `004-rag-retrieval-quality`
|
||||||
|
**Created**: 2026-04-06
|
||||||
|
**Status**: Draft
|
||||||
|
**Input**: User description: "I want to enhance the current RAG system, to avoid common pitfalls like: Vocabulary mismatch in retrieval, where the user's language doesn't overlap with the documentation's terminology and Citation errors in generation, where the model cites a chunk that either wasn't retrieved or doesn't support the specific claim being made"
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The AI Teacher's question-answering system retrieves relevant content from neurosurgery documentation and generates answers. Two reliability problems reduce trust in the system:
|
||||||
|
|
||||||
|
1. **Vocabulary mismatch**: A student asking "what happens after cutting the skull?" may use everyday language while the documentation uses clinical terms like "craniotomy" — causing relevant passages to be missed entirely.
|
||||||
|
2. **Citation hallucination**: The model sometimes references a section or page in its answer that was not actually retrieved or that does not support the specific claim made, misleading students.
|
||||||
|
|
||||||
|
This feature improves both the accuracy of what gets retrieved and the integrity of what the model claims as its sources.
|
||||||
|
|
||||||
|
## User Scenarios & Testing *(mandatory)*
|
||||||
|
|
||||||
|
### User Story 1 - Accurate Retrieval Despite Different Terminology (Priority: P1)
|
||||||
|
|
||||||
|
A medical student asks a question using lay or imprecise language. The system bridges the gap between their vocabulary and the technical terminology used in the textbook, returning contextually relevant passages even when there is no word overlap between the question and the document text.
|
||||||
|
|
||||||
|
**Why this priority**: Vocabulary mismatch is the most frequent silent failure — the system returns an answer but based on wrong or empty context, so students receive incorrect information without realising it.
|
||||||
|
|
||||||
|
**Independent Test**: Ask the system a question using a common synonym or lay term for a concept that appears in the documentation only under a clinical name. Verify that the retrieved passages contain relevant content about that concept.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** a question uses a lay term (e.g., "brain swelling"), **When** the system retrieves content, **Then** it returns passages discussing the medically indexed equivalent ("cerebral edema") even though that phrase was not in the question.
|
||||||
|
2. **Given** a question uses an acronym the documentation spells out fully, **When** retrieval runs, **Then** relevant passages are still found.
|
||||||
|
3. **Given** a highly technical question that perfectly matches documentation language, **When** retrieval runs, **Then** quality does not regress compared to current behavior.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### User Story 2 - Grounded Citation in Generated Answers (Priority: P1)
|
||||||
|
|
||||||
|
When the model produces an answer and references a source (section, page, or figure), that source must have been part of the retrieved context and must genuinely support the specific claim being cited.
|
||||||
|
|
||||||
|
**Why this priority**: A hallucinated citation is worse than no citation — it gives students false confidence that a claim is documented when it is not.
|
||||||
|
|
||||||
|
**Independent Test**: Submit a query, capture the retrieved context passed to the model, and verify that every source reference in the generated answer maps to an identifier present in that context.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** retrieved context contains sections A, B, and C, **When** the model generates an answer citing "Section D", **Then** that citation is either removed or flagged before the answer is shown to the user.
|
||||||
|
2. **Given** a retrieved passage is used as context, **When** the model cites it, **Then** the cited passage actually contains the information the citation supports.
|
||||||
|
3. **Given** no relevant context was retrieved for part of a query, **When** the model responds, **Then** it acknowledges uncertainty rather than fabricating a source.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### User Story 3 - User Visibility into Retrieval Confidence (Priority: P2)
|
||||||
|
|
||||||
|
A student can see which parts of the answer are well-supported by the retrieved material and which parts carry lower confidence, allowing them to judge how much to rely on each claim.
|
||||||
|
|
||||||
|
**Why this priority**: Transparency builds appropriate trust — students should know when the system is uncertain rather than presenting all answers with equal authority.
|
||||||
|
|
||||||
|
**Independent Test**: Ask a question where only partial relevant content exists in the corpus. Verify the answer visually differentiates well-supported claims from lower-confidence statements.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** an answer contains a claim backed by a directly retrieved passage, **When** displayed, **Then** the source is shown alongside the claim.
|
||||||
|
2. **Given** an answer contains a claim not directly covered by retrieved content, **When** displayed, **Then** the system signals lower confidence or absence of a source for that claim.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Edge Cases
|
||||||
|
|
||||||
|
- What happens when query enrichment produces terms that retrieve completely unrelated passages?
|
||||||
|
- How does the system behave when the retrieved context is very short or empty?
|
||||||
|
- What if two retrieved sections contradict each other — how does citation work in that case?
|
||||||
|
- What if citation verification removes all citations from an answer (model cited nothing valid)?
|
||||||
|
- How does the system handle questions in languages other than the documentation language?
|
||||||
|
|
||||||
|
## Requirements *(mandatory)*
|
||||||
|
|
||||||
|
### Functional Requirements
|
||||||
|
|
||||||
|
- **FR-001**: The system MUST enrich user queries to bridge vocabulary gaps before retrieval, using the domain context of the book being queried.
|
||||||
|
- **FR-002**: The system MUST retrieve passages based on the enriched query, not solely on the literal user input.
|
||||||
|
- **FR-003**: The system MUST pass only verified, retrieved passage identifiers to the generation step as eligible citation targets.
|
||||||
|
- **FR-004**: The system MUST validate that every source reference in a generated answer corresponds to a passage present in the retrieved context.
|
||||||
|
- **FR-005**: The system MUST suppress or flag any citation in the generated answer that refers to a passage not present in the retrieved context.
|
||||||
|
- **FR-006**: The system MUST surface to the user which retrieved passages were used to support each claim (or indicate absence of supporting source).
|
||||||
|
- **FR-007**: When no relevant passages are retrieved, the system MUST communicate this clearly rather than generating an unsupported answer.
|
||||||
|
- **FR-008**: Query enrichment MUST be scoped to the active book to avoid introducing terminology from unrelated domains.
|
||||||
|
|
||||||
|
### Key Entities
|
||||||
|
|
||||||
|
- **Enriched Query**: The augmented version of the user's original question, including synonyms, alternate phrasings, or domain-aligned terms used for retrieval.
|
||||||
|
- **Retrieved Context**: The set of passages (sections and figures) returned by retrieval, each identified by a unique source reference, passed to generation.
|
||||||
|
- **Citation**: A reference in the generated answer to a specific source; must be traceable to a member of the Retrieved Context.
|
||||||
|
- **Citation Validation Result**: A per-citation judgment of whether the cited source was retrieved and supports the claim.
|
||||||
|
|
||||||
|
## Success Criteria *(mandatory)*
|
||||||
|
|
||||||
|
### Measurable Outcomes
|
||||||
|
|
||||||
|
- **SC-001**: Questions using common lay synonyms for clinical terms retrieve relevant passages at least 80% of the time (verified on a manually curated test set of synonym-query pairs).
|
||||||
|
- **SC-002**: Zero citations appear in generated answers that reference a source not present in the retrieved context passed to the model.
|
||||||
|
- **SC-003**: For every generated answer, 100% of displayed citations can be traced to a retrieved passage identifier.
|
||||||
|
- **SC-004**: Retrieval quality for queries that already match documentation vocabulary does not degrade (baseline score maintained or improved).
|
||||||
|
- **SC-005**: Users can identify, for each factual claim in an answer, whether a supporting source was found — without needing to ask a follow-up question.
|
||||||
|
|
||||||
|
## Assumptions
|
||||||
|
|
||||||
|
- The existing retrieval pipeline returns section and figure identifiers that can be used as citation anchors.
|
||||||
|
- Query enrichment operates at query time (not ingestion time), so no changes to stored embeddings are needed.
|
||||||
|
- The generation model can be instructed via prompt to restrict citations to a provided list of identifiers.
|
||||||
|
- Citation validation is performed after generation but before the answer is shown to the user (post-processing step).
|
||||||
|
- Mobile or offline support is out of scope for this feature.
|
||||||
|
- Multi-language support (non-English questions against English documentation) is a future concern and not addressed here.
|
||||||
@@ -0,0 +1,250 @@
|
|||||||
|
# Tasks: RAG Retrieval Quality Improvements
|
||||||
|
|
||||||
|
**Input**: Design documents from `/specs/004-rag-retrieval-quality/`
|
||||||
|
**Prerequisites**: plan.md ✅, spec.md ✅, research.md ✅, data-model.md ✅, contracts/ ✅
|
||||||
|
|
||||||
|
**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.
|
||||||
|
**Tests**: Not requested in spec — no test tasks generated.
|
||||||
|
|
||||||
|
## Format: `[ID] [P?] [Story] Description`
|
||||||
|
|
||||||
|
- **[P]**: Can run in parallel (different files, no dependencies)
|
||||||
|
- **[Story]**: Which user story this task belongs to (US1, US2, US3)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1: Setup (Shared Infrastructure)
|
||||||
|
|
||||||
|
**Purpose**: No new project structure needed — services are added to the existing `retrieval/` package. This phase is a single verification step.
|
||||||
|
|
||||||
|
- [x] T001 Verify active branch is `004-rag-retrieval-quality` and `backend/src/main/java/com/aiteacher/retrieval/` exists
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: Foundational (Blocking Prerequisites)
|
||||||
|
|
||||||
|
**Purpose**: Two lightweight value objects shared by both user stories.
|
||||||
|
|
||||||
|
**⚠️ CRITICAL**: Both user story phases depend on these records being present.
|
||||||
|
|
||||||
|
- [x] T002 Create `ExpandedQuery` record in `backend/src/main/java/com/aiteacher/retrieval/ExpandedQuery.java` with fields `String original` and `String rewritten`
|
||||||
|
- [x] T003 [P] Create `LabelledContext` record in `backend/src/main/java/com/aiteacher/retrieval/LabelledContext.java` with fields `Map<String, SectionEntity> sectionLabels`, `Map<String, FigureEntity> figureLabels`, and `String promptText`
|
||||||
|
|
||||||
|
**Checkpoint**: Foundation ready — US1 and US2 implementation can begin in parallel
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: User Story 1 — Accurate Retrieval Despite Different Terminology (Priority: P1) 🎯 MVP
|
||||||
|
|
||||||
|
**Goal**: Before each retrieval call, rewrite the user's question into clinical terminology so that vector search finds relevant sections even when the user uses lay language.
|
||||||
|
|
||||||
|
**Independent Test**: Ask "what happens after cutting the skull?" — verify retrieved sections contain content about craniotomy without that word appearing in the query.
|
||||||
|
|
||||||
|
### Implementation for User Story 1
|
||||||
|
|
||||||
|
- [x] T004 [US1] Create `QueryExpansionService` in `backend/src/main/java/com/aiteacher/retrieval/QueryExpansionService.java`:
|
||||||
|
- Constructor-inject `ChatClient`
|
||||||
|
- Method `expand(String query): ExpandedQuery`
|
||||||
|
- LLM prompt: *"Rewrite the following question using precise medical/surgical terminology as it would appear in a neurosurgery textbook index. Output only the rewritten question, nothing else. Question: {query}"*
|
||||||
|
- Return `new ExpandedQuery(query, rewrittenText)`
|
||||||
|
- Annotate with `@Service`
|
||||||
|
|
||||||
|
- [x] T005 [US1] Modify `NeurosurgeryRetriever.retrieve()` in `backend/src/main/java/com/aiteacher/retrieval/NeurosurgeryRetriever.java`:
|
||||||
|
- Change method signature from `retrieve(String query, UUID bookId)` to `retrieve(String query, UUID bookId)` — no signature change; just use `query` for vector search (already correct; no change needed here unless query is pre-expanded by caller)
|
||||||
|
- *Note*: expansion is done in ChatService before calling retrieve, so no change to NeurosurgeryRetriever is required
|
||||||
|
|
||||||
|
- [x] T006 [US1] Modify `ChatService` in `backend/src/main/java/com/aiteacher/chat/ChatService.java`:
|
||||||
|
- Constructor-inject `QueryExpansionService`
|
||||||
|
- In `sendMessage()`, call `queryExpansionService.expand(fullQuestion)` before the retrieval loop
|
||||||
|
- Pass `expandedQuery.rewritten()` to `retriever.retrieve()` instead of `fullQuestion`
|
||||||
|
- Keep passing `fullQuestion` (original) to `buildContextPrompt()` so the QUESTION block shown to the model reflects what the user actually asked
|
||||||
|
|
||||||
|
**Checkpoint**: User Story 1 fully functional — retrieval now uses clinically rewritten queries
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4: User Story 2 — Grounded Citation in Generated Answers (Priority: P1)
|
||||||
|
|
||||||
|
**Goal**: Tag all retrieved sections and figures with short ref-labels (`[S1]`, `[F1]`…) in the prompt, instruct the model to cite only those labels, then post-process the answer to strip any citation referencing a label that was not provided.
|
||||||
|
|
||||||
|
**Independent Test**: Trigger a question where only sections S1–S3 are retrieved. Verify the generated answer contains no citation outside that set, and the `sources` list in the response carries `refLabel` fields.
|
||||||
|
|
||||||
|
### Implementation for User Story 2
|
||||||
|
|
||||||
|
- [x] T007 [US2] Create `CitationValidatorService` in `backend/src/main/java/com/aiteacher/retrieval/CitationValidatorService.java`:
|
||||||
|
- Annotate with `@Service`
|
||||||
|
- Method `validate(String generatedAnswer, Set<String> validLabels): String`
|
||||||
|
- Scan `generatedAnswer` for occurrences of `[Sn]` and `[Fn]` patterns using a regex like `\[(S|F)\d+\]`
|
||||||
|
- Remove (or replace with empty string) any match whose label is not in `validLabels`
|
||||||
|
- Return the cleaned answer text
|
||||||
|
|
||||||
|
- [x] T008 [US2] Modify `ChatService.buildContextPrompt()` in `backend/src/main/java/com/aiteacher/chat/ChatService.java`:
|
||||||
|
- Change signature to return `LabelledContext` instead of `String`
|
||||||
|
- Assign sequential labels: sections get `S1`, `S2`, …; figures get `F1`, `F2`, …
|
||||||
|
- Prefix each section block with its label: `[S1] Section Title, p.N\n{fullText}\n\n`
|
||||||
|
- Prefix each figure line with its label: `[F1] Fig. X (p.N): caption`
|
||||||
|
- Populate `sectionLabels` and `figureLabels` maps in the returned `LabelledContext`
|
||||||
|
- Store the full formatted prompt in `LabelledContext.promptText()`
|
||||||
|
|
||||||
|
- [x] T009 [US2] Update system prompt constant in `backend/src/main/java/com/aiteacher/chat/ChatService.java`:
|
||||||
|
- Replace the citation rule *"Cite sources for each major point (book title and page number from the context)"* with: *"Cite claims using ONLY the reference labels provided in the context (e.g. [S1], [F2]). Do not invent page numbers, section titles, or labels not present in the CONTEXT block."*
|
||||||
|
|
||||||
|
- [x] T010 [US2] Wire `CitationValidatorService` into `ChatService.sendMessage()` in `backend/src/main/java/com/aiteacher/chat/ChatService.java`:
|
||||||
|
- Constructor-inject `CitationValidatorService`
|
||||||
|
- After the `chatClient.prompt()...call().content()` call, pass `assistantContent` and the label set from `LabelledContext` to `citationValidatorService.validate()`
|
||||||
|
- Use the validated string as `assistantContent` going forward
|
||||||
|
|
||||||
|
- [x] T011 [US2] Modify `buildSources()` in `backend/src/main/java/com/aiteacher/chat/ChatService.java`:
|
||||||
|
- Accept the `LabelledContext` (or its two maps) as an additional parameter
|
||||||
|
- Add `"refLabel"` entry to each source map: e.g. `source.put("refLabel", "S1")` for sections, `source.put("refLabel", "F1")` for figures
|
||||||
|
- Keep all other existing fields unchanged
|
||||||
|
|
||||||
|
- [x] T012 [US2] Update `sendMessage()` call chain in `backend/src/main/java/com/aiteacher/chat/ChatService.java` to thread `LabelledContext` through steps T008–T011:
|
||||||
|
- `LabelledContext ctx = buildContextPrompt(fullQuestion, allSections, allFigures)`
|
||||||
|
- Pass `ctx.promptText()` to the LLM call
|
||||||
|
- Pass `ctx` label maps to `validate()` and `buildSources()`
|
||||||
|
|
||||||
|
**Checkpoint**: User Stories 1 and 2 both fully functional — queries are expanded, citations are grounded
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 5: User Story 3 — User Visibility into Retrieval Confidence (Priority: P2)
|
||||||
|
|
||||||
|
**Goal**: The answer text contains `[S1]`-style labels (after US2). This phase exposes them in the frontend so users can see which claim maps to which source card.
|
||||||
|
|
||||||
|
**Independent Test**: Send a question, receive an answer with inline `[S1]` labels visible in the rendered text, and confirm clicking/hovering the label highlights the corresponding source card.
|
||||||
|
|
||||||
|
**Note**: Per research.md, the backend is already complete after US1+US2. US3 is a frontend-only UX enhancement.
|
||||||
|
|
||||||
|
### Implementation for User Story 3
|
||||||
|
|
||||||
|
- [x] T013 [US3] Modify `ChatMessage.vue` in `frontend/src/components/ChatMessage.vue`:
|
||||||
|
- Parse the answer text for `[Sn]` and `[Fn]` citation labels using a regex
|
||||||
|
- Render each label as a styled inline badge (e.g. `<span class="citation-badge">[S1]</span>`)
|
||||||
|
- When a badge is clicked or hovered, highlight the corresponding source card (match by `source.refLabel`)
|
||||||
|
|
||||||
|
- [x] T014 [US3] Update source card rendering in `frontend/src/components/ChatMessage.vue`:
|
||||||
|
- Add a `data-ref-label` attribute to each source card element so it can be targeted by the citation badge interaction
|
||||||
|
- Apply a visual highlight style (CSS class) when the card is active
|
||||||
|
|
||||||
|
**Checkpoint**: All three user stories functional — full end-to-end quality improvements delivered
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 6: User Story 4 — Topic Summary Persistence & History (user-requested)
|
||||||
|
|
||||||
|
**Goal**: Every generated topic summary is saved to the database. When a topic is selected the UI shows a numbered history list; the student can view any past summary or generate a new one.
|
||||||
|
|
||||||
|
**Independent Test**: Generate a summary for "Intracranial Aneurysms", reload the page, click the topic — verify "Summary #1" appears. Generate again — verify "Summary #2" appears. Click "Summary #1" — verify the original text loads without regeneration.
|
||||||
|
|
||||||
|
- [x] T018 Create Flyway migration `backend/src/main/resources/db/migration/V6__topic_summary.sql` — table `topic_summary` with columns: `id UUID PRIMARY KEY DEFAULT gen_random_uuid()`, `topic_id VARCHAR(100) NOT NULL`, `summary_number INT NOT NULL`, `summary TEXT NOT NULL`, `sources_json TEXT NOT NULL`, `generated_at TIMESTAMPTZ NOT NULL`
|
||||||
|
- [x] T019 [P] [US4] Create `TopicSummaryEntity.java` in `backend/src/main/java/com/aiteacher/topic/TopicSummaryEntity.java` — JPA `@Entity` mapped to table `topic_summary`; fields: `@Id UUID id`, `String topicId`, `int summaryNumber`, `String summary`, `String sourcesJson`, `Instant generatedAt`; no-arg + all-args constructor
|
||||||
|
- [x] T02X [P] [US4] Create `SavedSummaryItem.java` record in `backend/src/main/java/com/aiteacher/topic/SavedSummaryItem.java` — fields: `UUID id`, `int summaryNumber`, `Instant generatedAt` (list-view DTO, no full text)
|
||||||
|
- [x] T02X [US4] Create `TopicSummaryRepository.java` in `backend/src/main/java/com/aiteacher/topic/TopicSummaryRepository.java` — `extends JpaRepository<TopicSummaryEntity, UUID>`; add `List<TopicSummaryEntity> findByTopicIdOrderBySummaryNumberAsc(String topicId)` and `long countByTopicId(String topicId)`
|
||||||
|
- [x] T02X [US4] Modify `TopicSummaryResponse.java` in `backend/src/main/java/com/aiteacher/topic/TopicSummaryResponse.java` — add fields `UUID id` and `int summaryNumber` to the record components
|
||||||
|
- [x] T02X [US4] Modify `TopicSummaryService.java` in `backend/src/main/java/com/aiteacher/topic/TopicSummaryService.java` — inject `TopicSummaryRepository` and `ObjectMapper`; at end of `generateSummary()` compute `summaryNumber = (int) repository.countByTopicId(topicId) + 1`, persist a `TopicSummaryEntity` (serialise `sources` list to JSON via `objectMapper.writeValueAsString()`), and include `id` + `summaryNumber` in the returned `TopicSummaryResponse`; add `List<SavedSummaryItem> listSummaries(String topicId)` and `TopicSummaryResponse getSummary(UUID summaryId)` methods
|
||||||
|
- [x] T02X [US4] Modify `TopicController.java` in `backend/src/main/java/com/aiteacher/topic/TopicController.java` — add `@GetMapping("/{id}/summaries")` returning `List<SavedSummaryItem>` (delegates to `listSummaries`); add `@GetMapping("/{id}/summaries/{summaryId}")` returning `TopicSummaryResponse` (delegates to `getSummary`); both return 404 via `NoSuchElementException` when topic or summary not found
|
||||||
|
- [x] T02X [US4] Modify `topicStore.ts` in `frontend/src/stores/topicStore.ts` — add state `summaryList: SavedSummaryItem[]`; add `fetchSummaries(topicId)` action calling `GET /api/v1/topics/{topicId}/summaries`; add `fetchSummaryDetail(topicId, summaryId)` action calling `GET /api/v1/topics/{topicId}/summaries/{summaryId}` and setting `activeSummary`; clear `summaryList` when a different topic is selected
|
||||||
|
- [x] T02X [US4] Modify `TopicsView.vue` in `frontend/src/views/TopicsView.vue` — when a topic card is clicked: (1) call `topicStore.fetchSummaries(topicId)` first; (2) if summaries exist, display a summary history list showing chips "Summary #1 · [date]", "Summary #2 · [date]", … + a "Generate New" button; (3) clicking a chip calls `fetchSummaryDetail()` and renders the saved summary in the existing panel; (4) clicking "Generate New" calls `handleGenerate()` then re-calls `fetchSummaries()` to refresh the list; (5) if no summaries exist, show only the "Generate Summary" button (current behaviour)
|
||||||
|
|
||||||
|
**Checkpoint**: Summary persistence fully working end-to-end. US4 independently testable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 7: Polish & Cross-Cutting Concerns
|
||||||
|
|
||||||
|
**Purpose**: Constitution IV compliance and cleanup.
|
||||||
|
|
||||||
|
- [x] T027 Update `README.md` Mermaid architecture diagram to add `QueryExpansionService` and `CitationValidatorService` to the chat pipeline flow, and the `topic_summary` table to the data diagram (required by Constitution Principle IV — must be in the same PR)
|
||||||
|
- [x] T028 [P] Log the expanded query at DEBUG level in `QueryExpansionService` (e.g. `log.debug("Query expanded: '{}' → '{}'", original, rewritten)`) for observability
|
||||||
|
- [x] T029 [P] Log stripped citation labels at WARN level in `CitationValidatorService` when any labels are removed (e.g. `log.warn("Stripped hallucinated citations: {}", removedLabels)`)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dependencies & Execution Order
|
||||||
|
|
||||||
|
### Phase Dependencies
|
||||||
|
|
||||||
|
- **Phase 1 (Setup)**: No dependencies — start immediately
|
||||||
|
- **Phase 2 (Foundational)**: Depends on Phase 1 — blocks all user story phases
|
||||||
|
- **Phase 3 (US1)**: Depends on Phase 2 (needs `ExpandedQuery`)
|
||||||
|
- **Phase 4 (US2)**: Depends on Phase 2 (needs `LabelledContext`); can run in parallel with Phase 3
|
||||||
|
- **Phase 5 (US3)**: Depends on Phase 4 (needs `refLabel` in sources)
|
||||||
|
- **Phase 6 (US4)**: No dependency on Phase 2 for the migration (T018); entity/service work (T019+) depends on T018
|
||||||
|
- **Phase 7 (Polish)**: Depends on all implementation phases complete
|
||||||
|
|
||||||
|
### User Story Dependencies
|
||||||
|
|
||||||
|
- **User Story 1 (P1)**: Depends on Phase 2 only — no dependency on US2 or US3
|
||||||
|
- **User Story 2 (P1)**: Depends on Phase 2 only — can run in parallel with US1
|
||||||
|
- **User Story 3 (P2)**: Depends on US2 (needs `refLabel` in the API response)
|
||||||
|
- **User Story 4**: Independent of US1–US3 — can start immediately after T018 migration
|
||||||
|
|
||||||
|
### Within Each User Story
|
||||||
|
|
||||||
|
- T004 → T006 (QueryExpansionService must exist before ChatService wiring)
|
||||||
|
- T007 → T010 → T012 (CitationValidatorService → wire into sendMessage → thread context)
|
||||||
|
- T008 → T012 (LabelledContext must be built before threading through)
|
||||||
|
- T013 → T014 (badge rendering before card targeting)
|
||||||
|
|
||||||
|
### Parallel Opportunities
|
||||||
|
|
||||||
|
- T002 and T003 (Phase 2) can run in parallel — different files
|
||||||
|
- Phase 3 (US1) and Phase 4 (US2) can run in parallel after Phase 2 — all different files
|
||||||
|
- T015, T016, T017 (Polish) can run in parallel — different files
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Parallel Example: US1 + US2
|
||||||
|
|
||||||
|
```
|
||||||
|
After Phase 2 completes:
|
||||||
|
|
||||||
|
Track A (US1):
|
||||||
|
T004 — Create QueryExpansionService
|
||||||
|
T005 — (no change to NeurosurgeryRetriever)
|
||||||
|
T006 — Wire into ChatService
|
||||||
|
|
||||||
|
Track B (US2):
|
||||||
|
T007 — Create CitationValidatorService
|
||||||
|
T008 — Modify buildContextPrompt() → LabelledContext
|
||||||
|
T009 — Update system prompt
|
||||||
|
T010 — Wire CitationValidatorService into sendMessage()
|
||||||
|
T011 — Add refLabel to buildSources()
|
||||||
|
T012 — Thread LabelledContext through call chain
|
||||||
|
|
||||||
|
Merge point: Both tracks modify ChatService — coordinate T006 and T012
|
||||||
|
to avoid conflicts (implement T006 first or use feature branches).
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementation Strategy
|
||||||
|
|
||||||
|
### MVP First (User Stories 1 + 2 — both P1)
|
||||||
|
|
||||||
|
1. Complete Phase 1: Setup (T001)
|
||||||
|
2. Complete Phase 2: Foundational (T002, T003)
|
||||||
|
3. Complete Phase 3: US1 — query expansion (T004–T006)
|
||||||
|
4. **VALIDATE**: Ask a lay-language question; confirm relevant clinical passages are retrieved
|
||||||
|
5. Complete Phase 4: US2 — citation grounding (T007–T012)
|
||||||
|
6. **VALIDATE**: Confirm no `[Sx]` label appears in the answer that wasn't in the retrieved set
|
||||||
|
7. **STOP and DEMO**: Both P1 stories deliver the core reliability improvements
|
||||||
|
|
||||||
|
### Incremental Delivery
|
||||||
|
|
||||||
|
1. Phase 1 + 2 → infrastructure ready
|
||||||
|
2. Phase 3 → vocabulary mismatch fixed → demo-able
|
||||||
|
3. Phase 4 → citation hallucination fixed → demo-able
|
||||||
|
4. Phase 5 → citation badges in UI → UX polish
|
||||||
|
5. Phase 6 → README + logging → PR-ready
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- `ChatService` is modified by both US1 (T006) and US2 (T008–T012) — coordinate edits or implement sequentially
|
||||||
|
- `buildContextPrompt()` changes return type from `String` to `LabelledContext` (T008) — update all callers in the same task
|
||||||
|
- The system prompt change (T009) is a one-line string edit inside `ChatService`; no separate class needed
|
||||||
|
- `CitationValidatorService` operates purely on strings — no DB or AI dependency, easy to unit-test manually
|
||||||
|
- US3 frontend tasks (T013–T014) are entirely in `ChatMessage.vue` — no backend change
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# Build Contract: Native Image
|
||||||
|
|
||||||
|
**Branch**: `005-native-image-deployment` | **Date**: 2026-04-07
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This contract defines what the native build produces and how consumers (developers, CI, docker-compose)
|
||||||
|
interact with it. The REST API contract is **unchanged** — all existing endpoints remain identical.
|
||||||
|
|
||||||
|
## Build Inputs
|
||||||
|
|
||||||
|
| Input | Required | Description |
|
||||||
|
|-------|----------|-------------|
|
||||||
|
| GraalVM JDK 25 | Yes (native profile only) | `JAVA_HOME` must point to GraalVM 25 |
|
||||||
|
| `DOCKER_HOST` / Docker daemon | Optional | Required for `jib:dockerBuild`; not needed for `jib:build` |
|
||||||
|
| `jib.to.image` Maven property | Optional | Override target image name; defaults to `ai-teacher-backend` |
|
||||||
|
|
||||||
|
## Build Outputs
|
||||||
|
|
||||||
|
| Profile | Command | Output |
|
||||||
|
|---------|---------|--------|
|
||||||
|
| Default (JVM) | `mvn package` | `target/ai-teacher-backend-*.jar` |
|
||||||
|
| Native | `mvn -Pnative package` | `target/ai-teacher-backend` (native executable) |
|
||||||
|
| Native + Docker | `mvn -Pnative package jib:dockerBuild` | Local Docker image `ai-teacher-backend:latest` |
|
||||||
|
| Native + Registry | `mvn -Pnative package jib:build` | Remote Docker image (configured via properties) |
|
||||||
|
|
||||||
|
## Docker Image Contract
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|----------|-------|
|
||||||
|
| Base image | `gcr.io/distroless/base-nossl-debian12` |
|
||||||
|
| Entrypoint | `/app/ai-teacher-backend` (native executable) |
|
||||||
|
| Exposed port | `8080` |
|
||||||
|
| Architecture | `linux/amd64` (matches build host) |
|
||||||
|
| Required env vars | Same as JVM mode (`SPRING_DATASOURCE_URL`, `OPENAI_API_KEY`, etc.) |
|
||||||
|
|
||||||
|
## REST API Contract
|
||||||
|
|
||||||
|
Unchanged — the native image exposes the same HTTP API as the JVM image:
|
||||||
|
|
||||||
|
- `GET /api/v1/books`
|
||||||
|
- `POST /api/v1/books` (multipart PDF upload)
|
||||||
|
- `DELETE /api/v1/books/{id}`
|
||||||
|
- `POST /api/v1/chat`
|
||||||
|
- `GET /api/v1/chat/history`
|
||||||
|
|
||||||
|
All endpoints use the same request/response schemas.
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# Data Model: Native Image Deployment
|
||||||
|
|
||||||
|
**Branch**: `005-native-image-deployment` | **Date**: 2026-04-07
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
This feature introduces **no new data entities**. It is a build and packaging change only.
|
||||||
|
|
||||||
|
The existing data model (books, sections, figures, messages, pgvector embeddings) is unchanged.
|
||||||
|
All Flyway migrations run identically in native mode as in JVM mode.
|
||||||
|
|
||||||
|
## Impact on Existing Entities
|
||||||
|
|
||||||
|
| Entity | Change |
|
||||||
|
|--------|--------|
|
||||||
|
| Books | None |
|
||||||
|
| Sections | None |
|
||||||
|
| Figures | None |
|
||||||
|
| Messages | None |
|
||||||
|
| pgvector embeddings | None |
|
||||||
|
|
||||||
|
## Configuration Properties (new/changed)
|
||||||
|
|
||||||
|
The following new or updated configuration properties are added for native deployment:
|
||||||
|
|
||||||
|
| Property | Description | Default |
|
||||||
|
|----------|-------------|---------|
|
||||||
|
| `jib.to.image` | Target Docker image name/tag | (set at build time) |
|
||||||
|
| `jib.to.auth.username` | Docker registry username | (CI env var) |
|
||||||
|
| `jib.to.auth.password` | Docker registry password | (CI env var) |
|
||||||
|
|
||||||
|
All runtime configuration (database URL, OpenAI API key, S3 credentials) remains in
|
||||||
|
`application.properties` / environment variables — unchanged.
|
||||||
@@ -0,0 +1,213 @@
|
|||||||
|
# Implementation Plan: Native Image Deployment
|
||||||
|
|
||||||
|
**Branch**: `005-native-image-deployment` | **Date**: 2026-04-07 | **Spec**: [spec.md](spec.md)
|
||||||
|
**Input**: Feature specification from `/specs/005-native-image-deployment/spec.md`
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Add a `native` Maven profile to the backend that compiles the Spring Boot 4.0.5 application to a
|
||||||
|
GraalVM 25 native executable, then packages it into a minimal Docker image via Jib. The JVM build
|
||||||
|
(default profile + existing Dockerfile) remains unchanged. A companion `docker-compose.native.yml`
|
||||||
|
enables local full-stack testing with the native image. README.md is updated with build
|
||||||
|
instructions and an updated architecture diagram.
|
||||||
|
|
||||||
|
## Technical Context
|
||||||
|
|
||||||
|
**Language/Version**: Java 25 (backend), TypeScript / Node 20 (frontend)
|
||||||
|
**Primary Dependencies**: Spring Boot 4.0.5, Spring AI 2.0.0-M4, `native-maven-plugin` 0.10.6,
|
||||||
|
`jib-maven-plugin` 3.4.5, `jib-native-image-extension-maven` 0.1.0
|
||||||
|
**Storage**: PostgreSQL 16 + pgvector (unchanged)
|
||||||
|
**Testing**: Spring Boot Test / JUnit 5 (unchanged); native integration test via smoke-test on
|
||||||
|
produced Docker image
|
||||||
|
**Target Platform**: Linux x86_64 container (Docker), GraalVM 25 CE or Oracle GraalVM 25
|
||||||
|
**Project Type**: Web application — backend API + frontend client (Option 2 structure)
|
||||||
|
**Performance Goals**: Backend container starts in < 1 s; idle RSS < 150 MB
|
||||||
|
**Constraints**: Native profile is opt-in; JVM mode unchanged; no new runtime dependencies;
|
||||||
|
cross-compilation out of scope
|
||||||
|
**Scale/Scope**: Single backend deployable unit (1 Docker image); frontend unchanged
|
||||||
|
|
||||||
|
## Constitution Check
|
||||||
|
|
||||||
|
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||||
|
|
||||||
|
| Principle | Status | Notes |
|
||||||
|
|-----------|--------|-------|
|
||||||
|
| I. KISS | ✅ Pass | Build-time change only; no new runtime layers or abstractions. |
|
||||||
|
| II. Easy to Change | ✅ Pass | `native` profile is additive; default JVM profile unchanged. Native hints isolated in one `NativeHintsConfig` class. |
|
||||||
|
| III. Web-First Architecture | ✅ Pass | REST API contract unchanged. |
|
||||||
|
| IV. Documentation as Architecture | ⚠️ Required | README.md MUST be updated with native build instructions and updated Mermaid diagram showing the build pipeline. |
|
||||||
|
| Technology Constraints | ✅ Pass | Still 1 backend + 1 frontend deployable unit; no new services. |
|
||||||
|
|
||||||
|
**Post-design re-check**: After Phase 1 — Principle IV gate requires README update in same PR.
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
### Documentation (this feature)
|
||||||
|
|
||||||
|
```text
|
||||||
|
specs/005-native-image-deployment/
|
||||||
|
├── plan.md # This file
|
||||||
|
├── spec.md # Feature specification
|
||||||
|
├── research.md # Phase 0 — toolchain & compatibility decisions
|
||||||
|
├── data-model.md # Phase 1 — no new entities (confirmed)
|
||||||
|
├── quickstart.md # Phase 1 — developer build guide
|
||||||
|
├── contracts/
|
||||||
|
│ └── build-contract.md # Build inputs/outputs & unchanged REST API
|
||||||
|
└── tasks.md # Phase 2 output (/speckit.tasks — NOT created here)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Source Code (repository root)
|
||||||
|
|
||||||
|
```text
|
||||||
|
backend/
|
||||||
|
├── pom.xml # +native profile, +Jib plugin, +NativeHintsConfig
|
||||||
|
├── src/
|
||||||
|
│ └── main/
|
||||||
|
│ └── java/com/aiteacher/
|
||||||
|
│ └── config/
|
||||||
|
│ └── NativeHintsConfig.java # NEW — RuntimeHints for PDFBox, AWS SDK gaps
|
||||||
|
└── src/
|
||||||
|
└── main/
|
||||||
|
└── resources/
|
||||||
|
└── META-INF/
|
||||||
|
└── native-image/ # Optional: manual JSON hints if AOT gaps remain
|
||||||
|
|
||||||
|
docker-compose.yml # UNCHANGED (JVM mode)
|
||||||
|
docker-compose.native.yml # NEW — native image + postgres full stack
|
||||||
|
README.md # UPDATED — build instructions + Mermaid diagram
|
||||||
|
```
|
||||||
|
|
||||||
|
**Structure Decision**: Follows Option 2 (web application). Only backend is modified; frontend
|
||||||
|
is untouched. New files are minimal: one Java config class, one docker-compose file.
|
||||||
|
|
||||||
|
## Complexity Tracking
|
||||||
|
|
||||||
|
> No constitution violations. No entry required.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementation Phases
|
||||||
|
|
||||||
|
### Phase 1: Maven `native` Profile + Native Maven Plugin
|
||||||
|
|
||||||
|
**Goal**: `mvn -Pnative package` produces a native executable `target/ai-teacher-backend`.
|
||||||
|
|
||||||
|
**Tasks**:
|
||||||
|
|
||||||
|
1. Add `<profile id="native">` to `backend/pom.xml`:
|
||||||
|
- Wire `spring-boot-maven-plugin` `process-aot` goal to `prepare-package` phase.
|
||||||
|
- Add `native-maven-plugin` 0.10.6 with `add-reachability-metadata` and `compile-no-fork`
|
||||||
|
executions.
|
||||||
|
- Set `<imageName>ai-teacher-backend</imageName>` and add essential build args:
|
||||||
|
`-H:+ReportExceptionStackTraces`, `--initialize-at-build-time=org.slf4j`.
|
||||||
|
|
||||||
|
2. Create `backend/src/main/java/com/aiteacher/config/NativeHintsConfig.java`:
|
||||||
|
- Implements `RuntimeHintsRegistrar`.
|
||||||
|
- Registers PDFBox classpath resources: `org/apache/pdfbox/resources/**`.
|
||||||
|
- Registers PDFBox reflection entries for font/encoding classes.
|
||||||
|
- Registers AWS SDK `SdkPojo` subtypes used by S3 (discovered iteratively).
|
||||||
|
- Annotate main application class or a `@Configuration` class with
|
||||||
|
`@ImportRuntimeHints(NativeHintsConfig.class)`.
|
||||||
|
|
||||||
|
3. Validate: run `mvn -Pnative package -DskipTests` and confirm executable is produced.
|
||||||
|
|
||||||
|
**Acceptance**: `target/ai-teacher-backend` exists, `./target/ai-teacher-backend --help`
|
||||||
|
exits cleanly, basic startup reaches DB connection stage.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 2: Jib Plugin — Package Native Image into Docker
|
||||||
|
|
||||||
|
**Goal**: `mvn -Pnative package jib:dockerBuild` produces a local Docker image.
|
||||||
|
|
||||||
|
**Tasks**:
|
||||||
|
|
||||||
|
1. Add `jib-maven-plugin` 3.4.5 to the `<build><plugins>` section (outside native profile,
|
||||||
|
but configured to package the native executable when the profile is active):
|
||||||
|
- Add `jib-native-image-extension-maven` 0.1.0 as a plugin dependency.
|
||||||
|
- Configure `<from><image>gcr.io/distroless/base-nossl-debian12</image></from>`.
|
||||||
|
- Set `<to><image>ai-teacher-backend</image></to>` (overridable via `-Djib.to.image`).
|
||||||
|
- Add `<pluginExtensions>` pointing to `JibNativeImageExtension`.
|
||||||
|
- Set `<container><ports><port>8080</port></ports></container>`.
|
||||||
|
|
||||||
|
2. Verify Jib picks up the native executable from `target/ai-teacher-backend` and sets it
|
||||||
|
as entrypoint.
|
||||||
|
|
||||||
|
3. Run `mvn -Pnative package jib:dockerBuild` and confirm:
|
||||||
|
- Image exists in local Docker: `docker images | grep ai-teacher-backend`.
|
||||||
|
- Container starts: `docker run --rm -e SPRING_DATASOURCE_URL=... ai-teacher-backend`.
|
||||||
|
|
||||||
|
**Acceptance**: Image starts, logs show "Started AiTeacherApplication in < 1.0 seconds",
|
||||||
|
and `GET /api/v1/books` returns 200 (with DB connected).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 3: Integration Smoke Test
|
||||||
|
|
||||||
|
**Goal**: All existing features work in native mode.
|
||||||
|
|
||||||
|
**Tasks**:
|
||||||
|
|
||||||
|
1. Start PostgreSQL locally (or via existing docker-compose.yml).
|
||||||
|
2. Start the native container with full environment variables.
|
||||||
|
3. Verify:
|
||||||
|
- Flyway migrations run successfully at startup.
|
||||||
|
- `POST /api/v1/books` (PDF upload + embedding) succeeds.
|
||||||
|
- `POST /api/v1/chat` (RAG chat) returns a non-empty answer.
|
||||||
|
- Spring Security HTTP Basic auth is enforced.
|
||||||
|
|
||||||
|
4. Fix any `MissingResourceException` or `ClassNotFoundException` by adding entries to
|
||||||
|
`NativeHintsConfig` and rebuilding.
|
||||||
|
|
||||||
|
**Acceptance**: All 5 REST API endpoints respond correctly with native image (FR-005).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 4: Docker Compose for Native Stack
|
||||||
|
|
||||||
|
**Goal**: `docker compose -f docker-compose.native.yml up` starts the full stack.
|
||||||
|
|
||||||
|
**Tasks**:
|
||||||
|
|
||||||
|
1. Create `docker-compose.native.yml` at repo root:
|
||||||
|
- `postgres` service: same as `docker-compose.yml` (copy).
|
||||||
|
- `backend` service: uses `ai-teacher-backend:latest` (the Jib-built native image).
|
||||||
|
- Wires all required env vars via `.env` file reference or inline secrets.
|
||||||
|
- Adds `depends_on` with health check for postgres.
|
||||||
|
|
||||||
|
2. Document the `.env` file format in `quickstart.md`.
|
||||||
|
|
||||||
|
**Acceptance**: `docker compose -f docker-compose.native.yml up` → all services healthy →
|
||||||
|
app responds at `http://localhost:8080/api/v1/books`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 5: README Update (Constitution IV Gate)
|
||||||
|
|
||||||
|
**Goal**: README.md reflects the new build pipeline and updated deployment diagram.
|
||||||
|
|
||||||
|
**Tasks**:
|
||||||
|
|
||||||
|
1. Add "Native Image Build" section to README.md with:
|
||||||
|
- Prerequisite: GraalVM 25 install instructions (sdkman command).
|
||||||
|
- Build command: `mvn -Pnative package jib:dockerBuild`.
|
||||||
|
- Run command: `docker compose -f docker-compose.native.yml up`.
|
||||||
|
|
||||||
|
2. Update the Mermaid architecture diagram in README.md to show:
|
||||||
|
- Build pipeline: Maven native profile → GraalVM native-image → Jib → Docker image.
|
||||||
|
- Keep the runtime diagram unchanged (same components, just a lighter container).
|
||||||
|
|
||||||
|
**Acceptance**: README.md Mermaid diagram renders correctly on GitHub; build section
|
||||||
|
contains all commands from `quickstart.md`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Risk Log
|
||||||
|
|
||||||
|
| Risk | Likelihood | Mitigation |
|
||||||
|
|------|------------|------------|
|
||||||
|
| Spring AI 2.0.0-M4 missing native hints for some AI response types | Medium | Iterative hint discovery during Phase 3 smoke test |
|
||||||
|
| PDFBox font loading fails at runtime in native | Medium | Register full `org/apache/pdfbox/resources/**` in `NativeHintsConfig` |
|
||||||
|
| AWS SDK S3 reflection errors | Medium | Switch to `url-connection-client` (simpler); lazy-init S3Client |
|
||||||
|
| GraalVM 25 availability in CI | Low | Documented prerequisite; CI pipeline config out of scope (can be added as follow-up) |
|
||||||
|
| Native build time > 5 min | Low | Acceptable for a first build; not a blocker for POC |
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
# Quickstart: Native Image Build & Deploy
|
||||||
|
|
||||||
|
**Branch**: `005-native-image-deployment` | **Date**: 2026-04-07
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- GraalVM JDK 25 CE or Oracle GraalVM 25 installed and set as `JAVA_HOME`
|
||||||
|
- Docker daemon running (for `jib:dockerBuild`)
|
||||||
|
- Maven 3.9+
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install GraalVM 25 CE via sdkman
|
||||||
|
sdk install java 25-graalce
|
||||||
|
sdk use java 25-graalce
|
||||||
|
|
||||||
|
# Verify
|
||||||
|
java -version # should show GraalVM 25
|
||||||
|
native-image --version # should show GraalVM 25
|
||||||
|
```
|
||||||
|
|
||||||
|
## Build Native Docker Image (local)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
|
||||||
|
# Build native executable AND package into local Docker image
|
||||||
|
mvn -Pnative package jib:dockerBuild
|
||||||
|
|
||||||
|
# The image is now available locally
|
||||||
|
docker images | grep ai-teacher-backend
|
||||||
|
```
|
||||||
|
|
||||||
|
## Run the Full Stack (native)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# From repo root — starts PostgreSQL + native backend
|
||||||
|
docker compose -f docker-compose.native.yml up
|
||||||
|
```
|
||||||
|
|
||||||
|
Access the app at `http://localhost:8080`.
|
||||||
|
|
||||||
|
## Build and Push to Registry (CI)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mvn -Pnative package jib:build \
|
||||||
|
-Djib.to.image=ghcr.io/your-org/ai-teacher-backend:native-latest \
|
||||||
|
-Djib.to.auth.username=$REGISTRY_USER \
|
||||||
|
-Djib.to.auth.password=$REGISTRY_TOKEN
|
||||||
|
```
|
||||||
|
|
||||||
|
## JVM Build (unchanged)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Default profile — no GraalVM required
|
||||||
|
cd backend
|
||||||
|
mvn package -DskipTests
|
||||||
|
|
||||||
|
java -jar target/ai-teacher-backend-*.jar
|
||||||
|
```
|
||||||
|
|
||||||
|
## Environment Variables (both JVM and native)
|
||||||
|
|
||||||
|
| Variable | Description |
|
||||||
|
|----------|-------------|
|
||||||
|
| `SPRING_DATASOURCE_URL` | PostgreSQL JDBC URL |
|
||||||
|
| `SPRING_DATASOURCE_USERNAME` | DB user |
|
||||||
|
| `SPRING_DATASOURCE_PASSWORD` | DB password |
|
||||||
|
| `OPENAI_API_KEY` | OpenAI API key |
|
||||||
|
| `AWS_ACCESS_KEY_ID` | S3 access key (if S3 storage enabled) |
|
||||||
|
| `AWS_SECRET_ACCESS_KEY` | S3 secret key |
|
||||||
|
| `AWS_REGION` | S3 region |
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
**Build fails with "ImageGenerationFailed"**: Ensure `native-image` is on `PATH` and
|
||||||
|
`JAVA_HOME` points to GraalVM 25, not a regular JDK.
|
||||||
|
|
||||||
|
**Missing resource at runtime**: Add the resource pattern to `NativeHintsConfig` and rebuild.
|
||||||
|
|
||||||
|
**ClassNotFoundException at runtime**: Register the class in `NativeHintsConfig` with
|
||||||
|
`MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS` and rebuild.
|
||||||
@@ -0,0 +1,262 @@
|
|||||||
|
# Research: Native Image Deployment
|
||||||
|
|
||||||
|
**Branch**: `005-native-image-deployment` | **Date**: 2026-04-07
|
||||||
|
|
||||||
|
## Decision Log
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 1. GraalVM Version and Java 25 Support
|
||||||
|
|
||||||
|
**Decision**: Use GraalVM JDK 25 (Oracle GraalVM or GraalVM CE 25).
|
||||||
|
|
||||||
|
**Rationale**: Oracle GraalVM 25 ships with Java 25 support and includes the
|
||||||
|
`native-image` tool. GraalVM CE 25 is available via sdkman (`sdk install java 25-graalce`).
|
||||||
|
Both support the `native-maven-plugin` 0.10.x workflow.
|
||||||
|
|
||||||
|
**Alternatives considered**:
|
||||||
|
- GraalVM 21 with Java 21: would require downgrading `java.version` in pom.xml — rejected
|
||||||
|
because pom.xml already targets Java 25.
|
||||||
|
- Mandrel (Red Hat distribution): Red Hat Mandrel 25 may lag behind Oracle GraalVM 25 release;
|
||||||
|
acceptable fallback but Oracle GraalVM 25 is preferred.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 2. Native Image Build Toolchain
|
||||||
|
|
||||||
|
**Decision**: Use `native-maven-plugin` (0.10.x) inside a Maven `native` profile, combined
|
||||||
|
with Spring Boot's AOT processing via `spring-boot-maven-plugin` `process-aot` execution.
|
||||||
|
|
||||||
|
**Rationale**: This is the canonical Spring Boot 4.x native image approach:
|
||||||
|
1. `mvn -Pnative package` runs AOT → generates source under `target/spring-aot/`
|
||||||
|
2. `native:compile` compiles the AOT-generated sources + app into a native executable.
|
||||||
|
|
||||||
|
The `native-maven-plugin` coordinates with GraalVM `native-image` CLI automatically.
|
||||||
|
|
||||||
|
**Alternatives considered**:
|
||||||
|
- Paketo Buildpacks (`spring-boot:build-image -Pnative`): does not use Jib; rejected because
|
||||||
|
the user explicitly wants Jib.
|
||||||
|
- Manual `native-image` CLI: fragile, no Maven lifecycle integration — rejected.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 3. Jib + Native Image Integration
|
||||||
|
|
||||||
|
**Decision**: Use `jib-maven-plugin` (3.4.x) with the `jib-native-image-extension-maven`
|
||||||
|
extension (0.1.0).
|
||||||
|
|
||||||
|
**Rationale**: The Jib native image extension packages a pre-built native executable into a
|
||||||
|
Docker image without requiring a Docker daemon. The workflow is:
|
||||||
|
|
||||||
|
```
|
||||||
|
mvn -Pnative native:compile # step 1: build native executable
|
||||||
|
mvn -Pnative jib:dockerBuild # step 2: package into Docker image
|
||||||
|
```
|
||||||
|
|
||||||
|
Or combined (with `process-aot` wired in `native` profile):
|
||||||
|
|
||||||
|
```
|
||||||
|
mvn -Pnative package jib:dockerBuild
|
||||||
|
```
|
||||||
|
|
||||||
|
Jib uses the extension to locate the native executable under `target/` and set it as the
|
||||||
|
container entrypoint.
|
||||||
|
|
||||||
|
**Key configuration**:
|
||||||
|
```xml
|
||||||
|
<plugin>
|
||||||
|
<groupId>com.google.cloud.tools</groupId>
|
||||||
|
<artifactId>jib-maven-plugin</artifactId>
|
||||||
|
<version>3.4.5</version>
|
||||||
|
<configuration>
|
||||||
|
<from>
|
||||||
|
<!-- distroless for security; or cgr.dev/chainguard/static for even smaller -->
|
||||||
|
<image>gcr.io/distroless/base-nossl-debian12</image>
|
||||||
|
</from>
|
||||||
|
<pluginExtensions>
|
||||||
|
<pluginExtension>
|
||||||
|
<implementation>
|
||||||
|
com.google.cloud.tools.jib.maven.extension.nativeimage.JibNativeImageExtension
|
||||||
|
</implementation>
|
||||||
|
</pluginExtension>
|
||||||
|
</pluginExtensions>
|
||||||
|
</configuration>
|
||||||
|
<dependencies>
|
||||||
|
<dependency>
|
||||||
|
<groupId>com.google.cloud.tools</groupId>
|
||||||
|
<artifactId>jib-native-image-extension-maven</artifactId>
|
||||||
|
<version>0.1.0</version>
|
||||||
|
</dependency>
|
||||||
|
</dependencies>
|
||||||
|
</plugin>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Alternatives considered**:
|
||||||
|
- Custom Dockerfile for native image: more control but manual, diverges from Jib workflow —
|
||||||
|
rejected in favor of Jib as specified.
|
||||||
|
- `jib:build` (push to registry) vs `jib:dockerBuild` (local daemon): default to
|
||||||
|
`jib:dockerBuild` for dev; `jib:build` for CI with registry configured.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 4. Docker Base Image for Native Executable
|
||||||
|
|
||||||
|
**Decision**: `gcr.io/distroless/base-nossl-debian12` (Debian 12 glibc variant).
|
||||||
|
|
||||||
|
**Rationale**: GraalVM native images compiled on Linux link against glibc by default.
|
||||||
|
Distroless provides glibc without a shell, minimizing attack surface.
|
||||||
|
|
||||||
|
Important: the native executable compiled on the host must target the same glibc ABI as the
|
||||||
|
base image. Since the build is on Linux x86_64 this matches Debian 12's glibc 2.36.
|
||||||
|
|
||||||
|
**Alternatives considered**:
|
||||||
|
- `alpine` (musl libc): requires `--static` or `--libc=musl` native-image flag; more complex —
|
||||||
|
rejected for simplicity (KISS).
|
||||||
|
- `gcr.io/distroless/java-base`: includes JVM libs we don't need — rejected (pure native).
|
||||||
|
- `scratch`: too minimal; requires fully static binary with all libs included — rejected.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 5. Spring AI 2.0.0-M4 Native Image Compatibility
|
||||||
|
|
||||||
|
**Decision**: Rely on Spring AI's built-in AOT hints; add manual `RuntimeHints` only for gaps
|
||||||
|
found during build.
|
||||||
|
|
||||||
|
**Rationale**: Spring AI 2.0.0-M4 includes `@NativeHint` registrations for the OpenAI
|
||||||
|
integration (HTTP client, response DTOs) and pgvector store. Spring Boot's AOT processor
|
||||||
|
picks these up automatically. M4 milestone has known native image improvements.
|
||||||
|
|
||||||
|
**Known gaps** (require manual RuntimeHints):
|
||||||
|
- `spring-ai-pdf-document-reader`: PDFBox font/resource loading uses `Class.forName()` and
|
||||||
|
classpath resource scanning. Need to register font resources and PDFBox parser classes.
|
||||||
|
- `spring-ai-advisors-vector-store`: generally AOT-compatible via Spring AI hints.
|
||||||
|
|
||||||
|
**Approach**: Add a `NativeHintsConfig` class annotated with `@ImportRuntimeHints` that
|
||||||
|
registers PDFBox fonts and any other gaps found during a `-Ob` (quick) native build.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 6. PDFBox Reflection Hints
|
||||||
|
|
||||||
|
**Decision**: Register PDFBox font resources and parser classes via a `RuntimeHintsRegistrar`.
|
||||||
|
|
||||||
|
**Rationale**: PDFBox 3.x loads fonts from classpath resources (`/org/apache/pdfbox/resources/`)
|
||||||
|
and uses reflection for CMap/encoding classes. Without hints the native build fails at runtime
|
||||||
|
with `ClassNotFoundException` or missing resource errors.
|
||||||
|
|
||||||
|
**Minimum hints needed**:
|
||||||
|
```java
|
||||||
|
hints.resources().registerPattern("org/apache/pdfbox/resources/**");
|
||||||
|
hints.reflection().registerType(org.apache.pdfbox.pdmodel.font.encoding.GlyphList.class,
|
||||||
|
MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS);
|
||||||
|
// Additional classes as discovered during smoke tests
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 7. AWS SDK v2 Native Image Support
|
||||||
|
|
||||||
|
**Decision**: Add `software.amazon.awssdk:aws-crt-client` is NOT needed; instead add
|
||||||
|
`software.amazon.awssdk:netty-nio-client` or use URL connection client.
|
||||||
|
Add AWS SDK v2 native image support via `aws-sdk-java-v2-native-bridge` if available,
|
||||||
|
otherwise register reflection manually.
|
||||||
|
|
||||||
|
**Rationale**: AWS SDK v2 2.30.x provides native image support via:
|
||||||
|
- `software.amazon.awssdk:apache-client` or `url-connection-client` (simpler, fewer reflection
|
||||||
|
needs than netty)
|
||||||
|
- The SDK's `SdkPojo` interfaces generate reflection entries; Spring Boot AOT may not cover all.
|
||||||
|
|
||||||
|
**Minimum approach**: Switch S3 HTTP client from default (netty) to `url-connection-client`
|
||||||
|
for simplicity in native mode, register `SdkPojo` subtypes via `RuntimeHints`.
|
||||||
|
|
||||||
|
**Alternative**: If S3 is used infrequently and only for figure uploads, mark S3Client as a
|
||||||
|
conditional bean initialized lazily — reflection issues then surface only on first use (not
|
||||||
|
startup), making native hints easier to discover iteratively.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 8. Flyway in Native Mode
|
||||||
|
|
||||||
|
**Decision**: No special configuration needed; Flyway 10.x has native image support.
|
||||||
|
|
||||||
|
**Rationale**: Spring Boot 4.x AOT includes Flyway native hints. PostgreSQL JDBC driver
|
||||||
|
and Flyway 10 have been tested with GraalVM native. No manual hints required.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 9. Spring Security in Native Mode
|
||||||
|
|
||||||
|
**Decision**: No special configuration needed.
|
||||||
|
|
||||||
|
**Rationale**: Spring Security 7.x (bundled with Spring Boot 4.x) includes AOT-compatible
|
||||||
|
configuration. HTTP Basic auth does not require dynamic proxies at runtime.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 10. Build Command Summary
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install GraalVM 25
|
||||||
|
sdk install java 25-graalce
|
||||||
|
|
||||||
|
# Build native executable + Docker image (local daemon)
|
||||||
|
cd backend
|
||||||
|
mvn -Pnative package jib:dockerBuild
|
||||||
|
|
||||||
|
# Or push to registry (CI)
|
||||||
|
mvn -Pnative package jib:build \
|
||||||
|
-Djib.to.image=ghcr.io/your-org/ai-teacher-backend:native-latest \
|
||||||
|
-Djib.to.auth.username=$CI_USER \
|
||||||
|
-Djib.to.auth.password=$CI_TOKEN
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 11. Maven Profile Structure
|
||||||
|
|
||||||
|
```xml
|
||||||
|
<profiles>
|
||||||
|
<profile>
|
||||||
|
<id>native</id>
|
||||||
|
<build>
|
||||||
|
<plugins>
|
||||||
|
<!-- AOT processing -->
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||||
|
<executions>
|
||||||
|
<execution>
|
||||||
|
<id>process-aot</id>
|
||||||
|
<goals><goal>process-aot</goal></goals>
|
||||||
|
</execution>
|
||||||
|
</executions>
|
||||||
|
</plugin>
|
||||||
|
<!-- Native compile -->
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.graalvm.buildtools</groupId>
|
||||||
|
<artifactId>native-maven-plugin</artifactId>
|
||||||
|
<version>0.10.6</version>
|
||||||
|
<executions>
|
||||||
|
<execution>
|
||||||
|
<id>add-reachability-metadata</id>
|
||||||
|
<goals><goal>add-reachability-metadata</goal></goals>
|
||||||
|
</execution>
|
||||||
|
<execution>
|
||||||
|
<id>compile</id>
|
||||||
|
<goals><goal>compile-no-fork</goal></goals>
|
||||||
|
<phase>package</phase>
|
||||||
|
</execution>
|
||||||
|
</executions>
|
||||||
|
<configuration>
|
||||||
|
<imageName>ai-teacher-backend</imageName>
|
||||||
|
<buildArgs>
|
||||||
|
<buildArg>--initialize-at-build-time=org.slf4j</buildArg>
|
||||||
|
<buildArg>-H:+ReportExceptionStackTraces</buildArg>
|
||||||
|
</buildArgs>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
|
</plugins>
|
||||||
|
</build>
|
||||||
|
</profile>
|
||||||
|
</profiles>
|
||||||
|
```
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
# Feature Specification: Native Image Deployment
|
||||||
|
|
||||||
|
**Feature Branch**: `005-native-image-deployment`
|
||||||
|
**Created**: 2026-04-07
|
||||||
|
**Status**: Draft
|
||||||
|
**Input**: Prepare the application for deployment using GraalVM native image for the backend, built into a Docker image via Jib.
|
||||||
|
|
||||||
|
## User Scenarios & Testing *(mandatory)*
|
||||||
|
|
||||||
|
### User Story 1 - Build Native Docker Image (Priority: P1)
|
||||||
|
|
||||||
|
A developer or CI pipeline runs a single Maven command to produce a Docker image containing the
|
||||||
|
native-compiled backend binary. The image starts in under one second and uses significantly less
|
||||||
|
memory than the JVM fat-jar image.
|
||||||
|
|
||||||
|
**Why this priority**: This is the core deliverable — without a working native Docker image the
|
||||||
|
entire feature has no value.
|
||||||
|
|
||||||
|
**Independent Test**: Run `mvn -Pnative jib:dockerBuild` (or equivalent), start the resulting
|
||||||
|
container, hit `GET /api/v1/books`, and receive a valid response. Startup log must show the server
|
||||||
|
is ready in under 1 second.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** a clean checkout with GraalVM installed, **When** the developer runs the native build
|
||||||
|
command, **Then** a Docker image is produced without manual steps.
|
||||||
|
2. **Given** the produced Docker image, **When** the container is started with the required
|
||||||
|
environment variables, **Then** the backend starts, connects to PostgreSQL, and serves HTTP
|
||||||
|
requests correctly.
|
||||||
|
3. **Given** the produced Docker image, **When** a RAG chat request is made, **Then** it responds
|
||||||
|
correctly (OpenAI call, pgvector retrieval, and Flyway migrations all work in native mode).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### User Story 2 - JVM Mode Preserved (Priority: P2)
|
||||||
|
|
||||||
|
A developer who has not installed GraalVM can still build and run the backend the same way as
|
||||||
|
today (fat-jar / existing Dockerfile), without any change to their workflow.
|
||||||
|
|
||||||
|
**Why this priority**: Preserving the developer experience for non-native builds ensures the team
|
||||||
|
can keep iterating without a GraalVM prerequisite.
|
||||||
|
|
||||||
|
**Independent Test**: Run `mvn package -DskipTests` and start the resulting jar with `java -jar` —
|
||||||
|
the application starts and serves requests identically to before this feature.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** a standard JDK (no GraalVM), **When** `mvn package` is run, **Then** a fat-jar is
|
||||||
|
produced and runs normally.
|
||||||
|
2. **Given** the existing Dockerfile, **When** `docker build` is run, **Then** a JVM-mode image
|
||||||
|
is produced and works exactly as before.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### User Story 3 - Docker Compose Full Stack (Priority: P3)
|
||||||
|
|
||||||
|
A developer can bring up the complete stack (PostgreSQL + native backend) with a single
|
||||||
|
`docker compose up` command for local integration testing.
|
||||||
|
|
||||||
|
**Why this priority**: Makes it easy to validate the native image against the real database before
|
||||||
|
pushing to any environment.
|
||||||
|
|
||||||
|
**Independent Test**: Run `docker compose up` (with native image tag), wait for healthy status,
|
||||||
|
make a RAG request — all services communicate correctly.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** the docker-compose.yml, **When** `docker compose up` is run with the native backend
|
||||||
|
image, **Then** all services start, health checks pass, and the application is reachable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Edge Cases
|
||||||
|
|
||||||
|
- What happens when a reflection-heavy library (PDFBox, AWS SDK) is called at runtime without
|
||||||
|
proper native hints? → Must be caught at build time via native-image test or detected early
|
||||||
|
via integration smoke test.
|
||||||
|
- What if GraalVM is not installed in CI? → Build must fail with a clear error message, not
|
||||||
|
silently produce a JVM image.
|
||||||
|
- What if an environment variable required at runtime is missing? → Container must fail fast with
|
||||||
|
a meaningful error (not a NullPointerException from missing reflection).
|
||||||
|
|
||||||
|
## Requirements *(mandatory)*
|
||||||
|
|
||||||
|
### Functional Requirements
|
||||||
|
|
||||||
|
- **FR-001**: The project MUST provide a `native` Maven profile that compiles the backend to a
|
||||||
|
standalone native executable using GraalVM.
|
||||||
|
- **FR-002**: The native profile MUST include Spring Boot AOT processing to generate the
|
||||||
|
reflection, serialization, and proxy hints required at compile time.
|
||||||
|
- **FR-003**: The Jib Maven plugin MUST be configured to package the native executable into a
|
||||||
|
Docker image without requiring Docker daemon access during the build.
|
||||||
|
- **FR-004**: The resulting Docker image MUST start the backend in under 1 second on modern
|
||||||
|
hardware (2+ CPU cores, 2 GB RAM).
|
||||||
|
- **FR-005**: All existing REST endpoints MUST function correctly in native mode (book upload,
|
||||||
|
RAG chat, Flyway migrations, pgvector retrieval, Spring Security).
|
||||||
|
- **FR-006**: The JVM fat-jar build (default profile) MUST continue to work unchanged.
|
||||||
|
- **FR-007**: The README.md MUST be updated with native build instructions and updated
|
||||||
|
architecture/deployment diagram.
|
||||||
|
- **FR-008**: The `docker-compose.yml` MUST be updated (or a companion file added) to support
|
||||||
|
running the native Docker image alongside PostgreSQL.
|
||||||
|
|
||||||
|
### Key Entities
|
||||||
|
|
||||||
|
- **Native Profile**: Maven build profile (`native`) that activates AOT processing and
|
||||||
|
GraalVM native compilation.
|
||||||
|
- **Docker Image**: OCI image produced by Jib, containing the native executable and a minimal
|
||||||
|
base OS layer.
|
||||||
|
- **AOT Hints**: Compile-time metadata (reflection, serialization, proxy) that replace runtime
|
||||||
|
reflection for third-party libraries.
|
||||||
|
|
||||||
|
## Success Criteria *(mandatory)*
|
||||||
|
|
||||||
|
### Measurable Outcomes
|
||||||
|
|
||||||
|
- **SC-001**: The native Docker image starts and serves its first HTTP request in under 1 second
|
||||||
|
on a machine with 2+ CPU cores and 2 GB RAM.
|
||||||
|
- **SC-002**: The native container's idle memory footprint is at least 40% lower than the
|
||||||
|
equivalent JVM container running the same application.
|
||||||
|
- **SC-003**: All existing API endpoints return correct responses in native mode (0 regressions).
|
||||||
|
- **SC-004**: A developer with GraalVM installed can build the native Docker image with a single
|
||||||
|
command documented in README.md.
|
||||||
|
- **SC-005**: The default JVM build and Dockerfile continue to work without any changes to the
|
||||||
|
developer workflow.
|
||||||
|
|
||||||
|
## Assumptions
|
||||||
|
|
||||||
|
- GraalVM 25 (CE or Oracle) will be available in CI and on developer machines that want to build
|
||||||
|
native images.
|
||||||
|
- The native build is performed on Linux x86_64; cross-compilation for other architectures is out
|
||||||
|
of scope for this feature.
|
||||||
|
- Spring Boot 4.0.5 and Spring AI 2.0.0-M4 have sufficient native image support for the features
|
||||||
|
used (OpenAI chat/embedding, pgvector, Flyway, Spring Security, Spring Data JPA).
|
||||||
|
- The Docker registry target and image name/tag will be configured via Maven properties or
|
||||||
|
environment variables, not hardcoded.
|
||||||
|
- Frontend deployment is out of scope for this feature; only the backend native image is addressed.
|
||||||
|
- The existing Dockerfile is kept as a fallback JVM image build; the Jib-built native image is
|
||||||
|
the new primary artifact.
|
||||||
@@ -0,0 +1,189 @@
|
|||||||
|
# Tasks: Native Image Deployment
|
||||||
|
|
||||||
|
**Input**: Design documents from `/specs/005-native-image-deployment/`
|
||||||
|
**Prerequisites**: plan.md ✅, spec.md ✅, research.md ✅, data-model.md ✅, contracts/ ✅
|
||||||
|
|
||||||
|
**Tests**: No test tasks — this feature has no unit/integration test requirements in the spec.
|
||||||
|
Smoke testing is part of US1 implementation (T010, T011).
|
||||||
|
|
||||||
|
**Organization**: Tasks grouped by user story for independent implementation and verification.
|
||||||
|
|
||||||
|
## Format: `[ID] [P?] [Story] Description`
|
||||||
|
|
||||||
|
- **[P]**: Can run in parallel (different files, no incomplete-task dependencies)
|
||||||
|
- **[Story]**: Which user story this task belongs to (US1, US2, US3)
|
||||||
|
- Paths use `backend/` and repo root (web app structure per constitution)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1: Setup (Shared Infrastructure)
|
||||||
|
|
||||||
|
**Purpose**: Verify prerequisites and baseline before touching any build files.
|
||||||
|
|
||||||
|
- [ ] T001 Verify GraalVM 25 CE or Oracle GraalVM 25 is installed: run `native-image --version` and confirm output shows GraalVM 25; document install command (`sdk install java 25-graalce`) if missing
|
||||||
|
- [ ] T002 [P] Verify baseline JVM build passes: run `mvn package -DskipTests` inside `backend/` and confirm `target/ai-teacher-backend-*.jar` is produced
|
||||||
|
|
||||||
|
**Checkpoint**: GraalVM available, existing JVM build green — safe to modify build files
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: Foundational (Blocking Prerequisites)
|
||||||
|
|
||||||
|
**Purpose**: Add AOT processing and native compilation support to the Maven build. These changes
|
||||||
|
are prerequisites for all user stories.
|
||||||
|
|
||||||
|
**⚠️ CRITICAL**: US1 cannot start until T007 confirms the native executable is produced.
|
||||||
|
|
||||||
|
- [ ] T003 Add `<profile id="native">` skeleton to `backend/pom.xml` and wire `spring-boot-maven-plugin` `process-aot` goal to the `prepare-package` phase inside that profile
|
||||||
|
- [ ] T004 Add `native-maven-plugin` 0.10.6 to the `native` profile in `backend/pom.xml` with executions `add-reachability-metadata` and `compile-no-fork` (phase `package`), image name `ai-teacher-backend`, and build args `--initialize-at-build-time=org.slf4j` and `-H:+ReportExceptionStackTraces`
|
||||||
|
- [ ] T005 [P] Create `backend/src/main/java/com/aiteacher/config/NativeHintsConfig.java` implementing `RuntimeHintsRegistrar`: register resource pattern `org/apache/pdfbox/resources/**` and reflection for `org.apache.pdfbox.pdmodel.font.encoding.GlyphList` with `MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS`
|
||||||
|
- [ ] T006 Add `@ImportRuntimeHints(NativeHintsConfig.class)` annotation to the main `@SpringBootApplication` class in `backend/src/main/java/com/aiteacher/`
|
||||||
|
- [ ] T007 Run `mvn -Pnative package -DskipTests` inside `backend/` and confirm `target/ai-teacher-backend` native executable is produced; fix any AOT compilation errors before proceeding
|
||||||
|
|
||||||
|
**Checkpoint**: `target/ai-teacher-backend` exists and exits cleanly — US1 implementation can begin
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: User Story 1 — Build Native Docker Image (Priority: P1) 🎯 MVP
|
||||||
|
|
||||||
|
**Goal**: One command (`mvn -Pnative package jib:dockerBuild`) produces a local Docker image
|
||||||
|
containing the native backend. Container starts in < 1 s and all REST endpoints work.
|
||||||
|
|
||||||
|
**Independent Test**: Start native container with DB env vars → `GET /api/v1/books` returns 200 →
|
||||||
|
startup log shows ready in under 1 second.
|
||||||
|
|
||||||
|
### Implementation for User Story 1
|
||||||
|
|
||||||
|
- [ ] T008 [US1] Add `jib-maven-plugin` 3.4.5 to `backend/pom.xml` `<build><plugins>` section (default build, not inside `native` profile): configure `<from><image>gcr.io/distroless/base-nossl-debian12</image></from>`, `<to><image>ai-teacher-backend</image></to>`, exposed port `8080`, and `<pluginExtensions>` referencing `com.google.cloud.tools.jib.maven.extension.nativeimage.JibNativeImageExtension`; add `jib-native-image-extension-maven` 0.1.0 as plugin `<dependency>`
|
||||||
|
- [ ] T009 [US1] Run `mvn -Pnative package jib:dockerBuild -DskipTests` inside `backend/` and confirm `docker images | grep ai-teacher-backend` shows the produced image
|
||||||
|
- [ ] T010 [US1] Start the native container with required env vars (`SPRING_DATASOURCE_URL`, `SPRING_DATASOURCE_USERNAME`, `SPRING_DATASOURCE_PASSWORD`, `OPENAI_API_KEY`) pointing to a local PostgreSQL; verify startup log shows "Started … in < 1.0 seconds" and `GET /api/v1/books` returns HTTP 200
|
||||||
|
- [ ] T011 [US1] Run full smoke test against the running native container: `POST /api/v1/books` (PDF upload), `POST /api/v1/chat` (RAG query), `GET /api/v1/chat/history`, and HTTP Basic auth enforcement; for each `MissingResourceException` or `ClassNotFoundException` found, add the corresponding entry to `backend/src/main/java/com/aiteacher/config/NativeHintsConfig.java` and rebuild (T009 → T010 loop) until all pass
|
||||||
|
|
||||||
|
**Checkpoint**: Native Docker image starts < 1 s, all 5 endpoints work — US1 complete ✅
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4: User Story 2 — JVM Mode Preserved (Priority: P2)
|
||||||
|
|
||||||
|
**Goal**: Default Maven build and existing Dockerfile continue to work with no changes required
|
||||||
|
from developers who do not have GraalVM installed.
|
||||||
|
|
||||||
|
**Independent Test**: `mvn package -DskipTests` produces a fat-jar; `docker build` using
|
||||||
|
`backend/Dockerfile` produces a working JVM container.
|
||||||
|
|
||||||
|
### Implementation for User Story 2
|
||||||
|
|
||||||
|
- [ ] T012 [P] [US2] Verify `mvn package -DskipTests` (no `-Pnative`) inside `backend/` still produces `target/ai-teacher-backend-*.jar`; confirm the jar starts correctly with `java -jar`
|
||||||
|
- [ ] T013 [P] [US2] Verify `docker build -t ai-teacher-backend-jvm backend/` using the existing `backend/Dockerfile` succeeds and the resulting JVM container starts and serves `GET /api/v1/books` correctly
|
||||||
|
|
||||||
|
**Checkpoint**: JVM path fully unchanged — US2 complete ✅
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 5: User Story 3 — Docker Compose Full Stack (Priority: P3)
|
||||||
|
|
||||||
|
**Goal**: `docker compose -f docker-compose.native.yml up` starts PostgreSQL + native backend
|
||||||
|
together so developers can run the complete native stack locally with a single command.
|
||||||
|
|
||||||
|
**Independent Test**: `docker compose -f docker-compose.native.yml up` → all services healthy →
|
||||||
|
`GET http://localhost:8080/api/v1/books` returns 200.
|
||||||
|
|
||||||
|
### Implementation for User Story 3
|
||||||
|
|
||||||
|
- [ ] T014 [US3] Create `docker-compose.native.yml` at repo root: include `postgres` service (same image and config as `docker-compose.yml`), add `backend` service using image `ai-teacher-backend:latest` with all required env vars sourced from `.env`, `depends_on` postgres with health-check condition, and port mapping `8080:8080`
|
||||||
|
- [ ] T015 [P] [US3] Create `.env.example` at repo root listing all env vars needed by the native stack (`SPRING_DATASOURCE_URL`, `SPRING_DATASOURCE_USERNAME`, `SPRING_DATASOURCE_PASSWORD`, `OPENAI_API_KEY`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`) with placeholder values and inline comments
|
||||||
|
|
||||||
|
**Checkpoint**: Full native stack starts via docker compose — US3 complete ✅
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 6: Polish & Cross-Cutting Concerns
|
||||||
|
|
||||||
|
**Purpose**: Constitution IV gate (README must be updated in the same PR as architectural change)
|
||||||
|
and final documentation alignment.
|
||||||
|
|
||||||
|
- [ ] T016 Update `README.md` at repo root: add a "Native Image Build" section with GraalVM install command (`sdk install java 25-graalce`), build command (`mvn -Pnative package jib:dockerBuild`), and run command (`docker compose -f docker-compose.native.yml up`); update the Mermaid architecture diagram to show the native build pipeline (Maven native profile → GraalVM native-image → Jib → Docker image)
|
||||||
|
- [ ] T017 [P] Update `specs/005-native-image-deployment/quickstart.md` with any corrections or additions discovered during implementation (env var names, exact commands, troubleshooting entries)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dependencies & Execution Order
|
||||||
|
|
||||||
|
### Phase Dependencies
|
||||||
|
|
||||||
|
- **Setup (Phase 1)**: No dependencies — start immediately
|
||||||
|
- **Foundational (Phase 2)**: Requires Phase 1 completion — **BLOCKS all user stories**
|
||||||
|
- **US1 (Phase 3)**: Requires Phase 2 (T007 must pass) — primary deliverable
|
||||||
|
- **US2 (Phase 4)**: Requires Phase 2 (T003–T006 must not break JVM path) — can run in parallel with US1 after Phase 2
|
||||||
|
- **US3 (Phase 5)**: Requires US1 completion (T009 must produce the Docker image)
|
||||||
|
- **Polish (Phase 6)**: Requires US1, US2, US3 all complete
|
||||||
|
|
||||||
|
### User Story Dependencies
|
||||||
|
|
||||||
|
- **US1 (P1)**: Unblocked after Foundational — critical path
|
||||||
|
- **US2 (P2)**: Unblocked after Foundational — can run in parallel with US1 (different verification commands, no file conflicts)
|
||||||
|
- **US3 (P3)**: Depends on US1 (needs the Docker image to exist for docker-compose)
|
||||||
|
|
||||||
|
### Within US1
|
||||||
|
|
||||||
|
T008 (Jib config) → T009 (build image) → T010 (verify startup) → T011 (smoke test + hint fixes, may loop back to T009)
|
||||||
|
|
||||||
|
### Parallel Opportunities
|
||||||
|
|
||||||
|
- T001 and T002 (Phase 1): both can run in parallel
|
||||||
|
- T003/T004 (pom.xml edits) are sequential (same file); T005 (new Java file) can run in parallel with T003/T004
|
||||||
|
- T012 and T013 (US2 verification): parallel, different commands
|
||||||
|
- T014 and T015 (US3 docker-compose + .env.example): parallel, different files
|
||||||
|
- T016 and T017 (Polish): parallel, different files
|
||||||
|
- US1 (Phase 3) and US2 (Phase 4) can run in parallel after Foundational completes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Parallel Example: Foundational Phase
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# In parallel — different files, no dependencies between them:
|
||||||
|
Task T005: "Create NativeHintsConfig.java in backend/src/main/java/com/aiteacher/config/"
|
||||||
|
|
||||||
|
# Sequential — same file (pom.xml):
|
||||||
|
Task T003: "Add native profile skeleton with spring-boot-maven-plugin process-aot"
|
||||||
|
Task T004: "Add native-maven-plugin 0.10.6 to native profile" # after T003
|
||||||
|
```
|
||||||
|
|
||||||
|
## Parallel Example: After Foundational Completes
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# US1 and US2 can start simultaneously:
|
||||||
|
Developer A → Phase 3 (US1): T008 → T009 → T010 → T011
|
||||||
|
Developer B → Phase 4 (US2): T012, T013 (parallel)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementation Strategy
|
||||||
|
|
||||||
|
### MVP First (User Story 1 Only)
|
||||||
|
|
||||||
|
1. Complete Phase 1: Setup (T001, T002)
|
||||||
|
2. Complete Phase 2: Foundational (T003–T007) — **CRITICAL, do not skip T007**
|
||||||
|
3. Complete Phase 3: US1 (T008–T011)
|
||||||
|
4. **STOP and VALIDATE**: Native image starts < 1 s, all endpoints work
|
||||||
|
5. This alone satisfies FR-001 through FR-006 and SC-001 through SC-003
|
||||||
|
|
||||||
|
### Incremental Delivery
|
||||||
|
|
||||||
|
1. Phase 1 + Phase 2 → build toolchain ready
|
||||||
|
2. Phase 3 (US1) → native Docker image working (**MVP**)
|
||||||
|
3. Phase 4 (US2) → JVM fallback confirmed
|
||||||
|
4. Phase 5 (US3) → full native stack via docker compose
|
||||||
|
5. Phase 6 (Polish) → README updated (constitution IV gate — **required before merge**)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- [P] tasks operate on different files or independent commands — safe to run concurrently
|
||||||
|
- US2 (T012, T013) can be verified at any time after Phase 2; they are quick sanity checks
|
||||||
|
- The hint-fix loop in T011 is the most likely source of iteration — budget extra time
|
||||||
|
- Constitution IV gate (README update, T016) is **mandatory** — PR cannot merge without it
|
||||||
|
- `.env.example` (T015) should be committed; actual `.env` must be in `.gitignore`
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
# Implementation Plan: Mobile-Responsive UI
|
||||||
|
|
||||||
|
**Branch**: `006-mobile-responsive-ui` | **Date**: 2026-04-10 | **Spec**: [spec.md](spec.md)
|
||||||
|
**Input**: Feature specification from `/specs/006-mobile-responsive-ui/spec.md`
|
||||||
|
|
||||||
|
**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/plan-template.md` for the execution workflow.
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Add CSS media queries to make the frontend usable on small screens (≤ 768px). Three targeted fixes:
|
||||||
|
1. Make the navbar sticky and its link bar horizontally scrollable
|
||||||
|
2. Prevent horizontal overflow in the book reader view
|
||||||
|
3. Align the login card toward the top of the viewport instead of vertically centered
|
||||||
|
|
||||||
|
No new dependencies, no backend changes, no new components. Pure CSS additions scoped to existing Vue SFCs.
|
||||||
|
|
||||||
|
## Technical Context
|
||||||
|
|
||||||
|
**Language/Version**: TypeScript / Node 20 (frontend only)
|
||||||
|
**Primary Dependencies**: Vue 3.4, Vue Router 4.3, Pinia 2.1 — no changes
|
||||||
|
**Storage**: N/A (frontend-only change)
|
||||||
|
**Testing**: Manual browser testing at 375px viewport; `npm run lint` for TypeScript
|
||||||
|
**Target Platform**: Modern mobile browsers (iOS Safari, Chrome Android); desktop unchanged
|
||||||
|
**Project Type**: Web application (frontend client only for this feature)
|
||||||
|
**Performance Goals**: No regression — changes are CSS-only, zero runtime cost
|
||||||
|
**Constraints**: No new dependencies; no changes to desktop layout; no hamburger menu
|
||||||
|
**Scale/Scope**: 5 files touched (App.vue, LoginView.vue, BookReaderView.vue, possibly UploadView.vue, BookCard.vue)
|
||||||
|
|
||||||
|
## Constitution Check
|
||||||
|
|
||||||
|
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||||
|
|
||||||
|
| Principle | Status | Notes |
|
||||||
|
|-----------|--------|-------|
|
||||||
|
| I. KISS | ✅ PASS | Pure CSS media queries — simplest viable solution. No new abstractions. |
|
||||||
|
| II. Easy to Change | ✅ PASS | Changes are scoped CSS within existing SFCs; easy to revert or extend. |
|
||||||
|
| III. Web-First Architecture | ✅ PASS | Frontend-only change; no API contract impact. |
|
||||||
|
| IV. Documentation as Architecture | ✅ PASS | No architectural change → README diagram unchanged. |
|
||||||
|
| Technology Constraints | ✅ PASS | Remains `backend/` + `frontend/`; no new deployable unit. |
|
||||||
|
|
||||||
|
**Verdict**: No violations. Complexity Tracking table not required.
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
### Documentation (this feature)
|
||||||
|
|
||||||
|
```text
|
||||||
|
specs/006-mobile-responsive-ui/
|
||||||
|
├── plan.md # This file
|
||||||
|
├── research.md # Phase 0 output
|
||||||
|
├── data-model.md # N/A (no data model changes)
|
||||||
|
├── quickstart.md # Phase 1 output
|
||||||
|
├── contracts/ # N/A (no API contract changes)
|
||||||
|
└── tasks.md # Phase 2 output (/speckit.tasks command)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Source Code (repository root)
|
||||||
|
|
||||||
|
```text
|
||||||
|
frontend/
|
||||||
|
├── src/
|
||||||
|
│ ├── App.vue # Navbar sticky + link bar scroll
|
||||||
|
│ ├── views/
|
||||||
|
│ │ ├── LoginView.vue # Login card top-aligned on mobile
|
||||||
|
│ │ ├── BookReaderView.vue # Reader fits screen width on mobile
|
||||||
|
│ │ └── UploadView.vue # Book grid min-width reduction (if needed)
|
||||||
|
│ └── components/
|
||||||
|
│ └── BookCard.vue # Card min-width reduction (if needed)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Structure Decision**: Option 2 (Web application) per constitution. This feature touches only `frontend/`; `backend/` is untouched.
|
||||||
|
|
||||||
|
## Complexity Tracking
|
||||||
|
|
||||||
|
> No violations — table not required for this feature.
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# Quickstart: Mobile-Responsive UI
|
||||||
|
|
||||||
|
**Feature**: 006-mobile-responsive-ui
|
||||||
|
|
||||||
|
## What this feature does
|
||||||
|
|
||||||
|
Adds CSS media queries to 2–3 Vue SFCs so the app is usable on phone screens (≤ 768px):
|
||||||
|
- Navbar sticks to top; link bar scrolls horizontally
|
||||||
|
- Book reader stacks vertically (no horizontal overflow)
|
||||||
|
- Login card appears near the top of the screen
|
||||||
|
|
||||||
|
## Files to change
|
||||||
|
|
||||||
|
| File | Change |
|
||||||
|
|------|--------|
|
||||||
|
| `frontend/src/App.vue` | Add `position: sticky; top: 0; z-index: 100` to `.navbar`. Add `@media (max-width: 768px)` block: make `.navbar-links` horizontally scrollable, reduce `.navbar` padding |
|
||||||
|
| `frontend/src/views/LoginView.vue` | Add `@media (max-width: 768px)`: change `.login-wrapper` to `align-items: flex-start; padding-top: 2rem` |
|
||||||
|
| `frontend/src/views/BookReaderView.vue` | Add `@media (max-width: 768px)`: set `.chat-reader-split` to `flex-direction: column`, remove fixed width on `.reader-panel`, set `.reader-panel` to `width: 100%` |
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
1. Open browser DevTools → toggle device toolbar → select "iPhone SE" (375 × 667)
|
||||||
|
2. Navigate to each page:
|
||||||
|
- **Any page**: navbar should be visible and sticky; links should be scrollable horizontally within the bar
|
||||||
|
- **Login** (`/`): login card should appear near the top, not vertically centered
|
||||||
|
- **Book reader** (`/books/:id`): content should fill the width, no horizontal page scrollbar
|
||||||
|
|
||||||
|
## No backend changes. No new dependencies. No new files.
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# Research: Mobile-Responsive UI
|
||||||
|
|
||||||
|
**Feature**: 006-mobile-responsive-ui
|
||||||
|
**Date**: 2026-04-10
|
||||||
|
|
||||||
|
## Decision 1: Navbar Sticky vs Fixed
|
||||||
|
|
||||||
|
- **Decision**: Use `position: sticky; top: 0` on `.navbar` (currently unstyled for position)
|
||||||
|
- **Rationale**: `sticky` keeps the element in normal flow until it hits the scroll threshold, avoiding the need to add `padding-top` to `<main>` to compensate for a `fixed` element that's removed from flow. Simpler with fewer side effects.
|
||||||
|
- **Alternatives considered**: `position: fixed` — works but requires matching `padding-top` on `.main-content` to prevent content from hiding under the navbar. More coupled.
|
||||||
|
|
||||||
|
## Decision 2: Navbar Links — Horizontal Scroll vs Hamburger Menu
|
||||||
|
|
||||||
|
- **Decision**: Horizontal scroll on `.navbar-links` using `overflow-x: auto; white-space: nowrap; -webkit-overflow-scrolling: touch`
|
||||||
|
- **Rationale**: User explicitly requested horizontal scroll ("button on the right need horizontal scroll to be accessed"). This is the simplest implementation — one CSS rule. A hamburger menu would require a toggle button, JavaScript state, and an open/close animation — unjustified complexity for a POC.
|
||||||
|
- **Alternatives considered**: Hamburger/drawer menu — rejected per user request and KISS principle.
|
||||||
|
|
||||||
|
## Decision 3: Book Reader Layout on Mobile
|
||||||
|
|
||||||
|
- **Decision**: Stack reader and chat panels vertically on mobile (≤ 768px) using `flex-direction: column` on `.chat-reader-split`, remove fixed width on `.reader-panel`
|
||||||
|
- **Rationale**: The current layout in BookReaderView.vue uses a flex row with `.reader-panel` at a fixed 420px. On a 375px screen this immediately overflows. Stacking vertically is the simplest fix — one media query, two CSS rules.
|
||||||
|
- **Alternatives considered**: Tabs (reader/chat toggle) — more complex, requires JS state; rejected per KISS.
|
||||||
|
|
||||||
|
## Decision 4: Login Card Top Alignment on Mobile
|
||||||
|
|
||||||
|
- **Decision**: On ≤ 768px, change `.login-wrapper` from `align-items: center` to `align-items: flex-start` and add `padding-top: 2rem`
|
||||||
|
- **Rationale**: The wrapper is a full-height flex column (`min-height: 100vh`). Centering vertically on mobile pushes the form to ~50% of viewport height. Switching to `flex-start` with a small top padding keeps the form near the top without changing desktop behavior.
|
||||||
|
- **Alternatives considered**: Removing `min-height: 100vh` — would break the background fill; rejected.
|
||||||
|
|
||||||
|
## Decision 5: Breakpoint
|
||||||
|
|
||||||
|
- **Decision**: Single breakpoint at `max-width: 768px`
|
||||||
|
- **Rationale**: Covers all common phone widths (320px–430px) while leaving tablet/desktop untouched. Consistent with common mobile-first conventions. Adding a second breakpoint (e.g., 480px) would be premature for a POC.
|
||||||
|
- **Alternatives considered**: 480px only — too narrow, misses larger phones; 1024px — too wide, affects tablets unnecessarily.
|
||||||
|
|
||||||
|
## No NEEDS CLARIFICATION items remain.
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
# Feature Specification: Mobile-Responsive UI
|
||||||
|
|
||||||
|
**Feature Branch**: `006-mobile-responsive-ui`
|
||||||
|
**Created**: 2026-04-10
|
||||||
|
**Status**: Draft
|
||||||
|
**Input**: User description: "I want the frontend to be usable in phone (small screen): Fix the nav bar on top, button on the right need horizontal scroll to be accessed. When read the books (from library section) the book section should fit the screen (width). The login section should be closer to the top of the screen"
|
||||||
|
|
||||||
|
## User Scenarios & Testing *(mandatory)*
|
||||||
|
|
||||||
|
### User Story 1 - Sticky Navbar with Scrollable Links (Priority: P1)
|
||||||
|
|
||||||
|
On a phone, the user opens the app. The navbar stays visible at the top as they scroll. All navigation links are reachable by horizontally scrolling the link bar — no links are clipped or hidden.
|
||||||
|
|
||||||
|
**Why this priority**: Without a usable nav, the user cannot navigate between sections at all. This is the most fundamental usability blocker on mobile.
|
||||||
|
|
||||||
|
**Independent Test**: Can be tested by opening the app on a ~375px-wide viewport (iPhone SE) and verifying the navbar is fixed/sticky, all nav links are scrollable horizontally, and the page content below scrolls independently.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** a 375px-wide viewport, **When** the page loads, **Then** the navbar is visible and stuck to the top of the screen
|
||||||
|
2. **Given** a 375px-wide viewport, **When** there are multiple nav links that overflow horizontally, **Then** the links are accessible by swiping/scrolling the link bar without triggering page scroll
|
||||||
|
3. **Given** a 375px-wide viewport, **When** the user scrolls the page content down, **Then** the navbar remains fixed at the top
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### User Story 2 - Book Reader Fits Screen Width (Priority: P2)
|
||||||
|
|
||||||
|
A user opens a book from the library on their phone. The book content area fits the full screen width — no horizontal page scroll is required to read the text.
|
||||||
|
|
||||||
|
**Why this priority**: The book reader is the primary value-delivery surface of the app. If it overflows the screen, the app is unusable for its main purpose.
|
||||||
|
|
||||||
|
**Independent Test**: Can be tested by navigating to the BookReaderView on a 375px viewport and verifying no horizontal scrollbar appears and text is readable without zooming.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** a 375px-wide viewport, **When** the user opens a book, **Then** the book content panel fills the screen width without overflow
|
||||||
|
2. **Given** a 375px-wide viewport, **When** the reader and chat panels are side-by-side on desktop, **Then** on mobile they stack vertically (reader on top, chat below) or the layout adapts to prevent overflow
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### User Story 3 - Login Form Near Top of Screen (Priority: P3)
|
||||||
|
|
||||||
|
A user lands on the login page on their phone. The login form appears near the top of the visible screen rather than perfectly centered vertically, so they do not need to scroll or zoom to reach the form.
|
||||||
|
|
||||||
|
**Why this priority**: Vertical centering that places a form at the mid-screen on desktop pushes it below the keyboard fold on mobile. Fixing this improves first-contact UX.
|
||||||
|
|
||||||
|
**Independent Test**: Can be tested by opening the login page on a 375px-wide, 667px-tall viewport (iPhone SE) and verifying the login card starts within the top 40% of the viewport.
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** a 375px-wide viewport, **When** the login page loads, **Then** the login card is positioned toward the top of the screen (not perfectly vertically centered)
|
||||||
|
2. **Given** a phone with virtual keyboard open, **When** the user taps an input field, **Then** the form remains accessible without scrolling behind the keyboard
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Edge Cases
|
||||||
|
|
||||||
|
- What happens when a very long book title wraps in the navbar or book card?
|
||||||
|
- How does the nav handle exactly the boundary between "fits" and "needs scroll" (e.g., 3 vs 4 links)?
|
||||||
|
- What if the user rotates to landscape on a phone (wider but shorter viewport)?
|
||||||
|
|
||||||
|
## Requirements *(mandatory)*
|
||||||
|
|
||||||
|
### Functional Requirements
|
||||||
|
|
||||||
|
- **FR-001**: The navbar MUST be sticky/fixed at the top of the viewport on all screen sizes
|
||||||
|
- **FR-002**: The navbar link area MUST be horizontally scrollable on small screens to access all navigation links
|
||||||
|
- **FR-003**: Horizontal page-level scroll MUST NOT occur due to navbar overflow
|
||||||
|
- **FR-004**: The book reader content area MUST NOT exceed the viewport width on screens ≤ 768px
|
||||||
|
- **FR-005**: The login card MUST be positioned toward the top of the viewport (not vertically centered) on screens ≤ 768px
|
||||||
|
|
||||||
|
### Key Entities *(include if feature involves data)*
|
||||||
|
|
||||||
|
- N/A — this is a pure frontend CSS/layout change with no data model impact
|
||||||
|
|
||||||
|
## Success Criteria *(mandatory)*
|
||||||
|
|
||||||
|
### Measurable Outcomes
|
||||||
|
|
||||||
|
- **SC-001**: On a 375px-wide viewport, the navbar is visible and fixed; all links are reachable by horizontal scroll within the navbar
|
||||||
|
- **SC-002**: On a 375px-wide viewport, the BookReaderView shows content without a horizontal scrollbar at the page level
|
||||||
|
- **SC-003**: On a 375px-wide viewport, the login card top edge is within the top 150px of the viewport
|
||||||
|
|
||||||
|
## Assumptions
|
||||||
|
|
||||||
|
- Target breakpoint for "small screen / phone" is ≤ 768px width
|
||||||
|
- No hamburger menu is required — horizontal scroll on the link bar is acceptable per user request
|
||||||
|
- The existing Vue 3 + plain CSS stack is retained (no CSS framework added)
|
||||||
|
- Desktop layout is unchanged
|
||||||
|
- No backend changes required
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
# Tasks: Mobile-Responsive UI
|
||||||
|
|
||||||
|
**Input**: Design documents from `/specs/006-mobile-responsive-ui/`
|
||||||
|
**Prerequisites**: plan.md ✅, spec.md ✅, research.md ✅, quickstart.md ✅
|
||||||
|
|
||||||
|
No foundational phase needed — all changes are isolated CSS additions within existing SFCs.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1: User Story 1 - Sticky Navbar with Scrollable Links (Priority: P1) 🎯 MVP
|
||||||
|
|
||||||
|
**Goal**: Navbar stays fixed at top; all links are reachable by horizontal scroll on mobile.
|
||||||
|
|
||||||
|
**Independent Test**: Open DevTools → iPhone SE (375px) → verify navbar is sticky and links scroll horizontally.
|
||||||
|
|
||||||
|
- [x] T001 [US1] Add `position: sticky; top: 0; z-index: 100` to `.navbar` in `frontend/src/App.vue`
|
||||||
|
- [x] T002 [US1] Add `@media (max-width: 768px)` block to `frontend/src/App.vue`: reduce navbar padding, make `.navbar-links` horizontally scrollable (`overflow-x: auto; white-space: nowrap; -webkit-overflow-scrolling: touch; flex-shrink: 0`)
|
||||||
|
|
||||||
|
**Checkpoint**: Navbar is sticky and links are horizontally scrollable on 375px viewport.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: User Story 2 - Book Reader Fits Screen Width (Priority: P2)
|
||||||
|
|
||||||
|
**Goal**: Book reader content fills phone screen width without page-level horizontal overflow.
|
||||||
|
|
||||||
|
**Independent Test**: Open DevTools → iPhone SE → navigate to a book → verify no horizontal scrollbar, content fills width.
|
||||||
|
|
||||||
|
- [x] T003 [US2] Add `@media (max-width: 768px)` block to `frontend/src/views/BookReaderView.vue`: set `.chat-reader-split` to `flex-direction: column`, set `.reader-panel` to `width: 100%; min-width: unset; max-width: 100%`
|
||||||
|
|
||||||
|
**Checkpoint**: BookReaderView stacks vertically with no horizontal overflow on 375px viewport.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: User Story 3 - Login Form Near Top of Screen (Priority: P3)
|
||||||
|
|
||||||
|
**Goal**: Login card appears near top of viewport on mobile instead of vertically centered.
|
||||||
|
|
||||||
|
**Independent Test**: Open DevTools → iPhone SE → navigate to login → verify card top is within top 150px of viewport.
|
||||||
|
|
||||||
|
- [x] T004 [US3] Add `@media (max-width: 768px)` block to `frontend/src/views/LoginView.vue`: change `.login-wrapper` to `align-items: flex-start; padding-top: 2rem`
|
||||||
|
|
||||||
|
**Checkpoint**: Login card appears near top of screen on 375px viewport.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4: Polish
|
||||||
|
|
||||||
|
- [x] T005 [P] Run `npm run lint` in `frontend/` and fix any lint errors (ESLint not configured — pre-existing, unrelated to this feature)
|
||||||
|
- [x] T006 [P] Verify `.main-content` padding is reduced on mobile in `frontend/src/App.vue` (covered in T002 media query block)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dependencies & Execution Order
|
||||||
|
|
||||||
|
- T001 → T002 (same file, sequential)
|
||||||
|
- T003, T004 independent of each other and of T001/T002 (different files)
|
||||||
|
- T005, T006 after all implementation tasks
|
||||||
Reference in New Issue
Block a user