Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,15 @@ on:
branches: [main]
pull_request:

permissions:
contents: read

jobs:
python-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
with:
python-version: "3.12"
- name: Install dependencies
Expand All @@ -28,10 +31,10 @@ jobs:
run:
working-directory: frontend
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: "20.19.0"
node-version: '24.22.1'
cache: npm
cache-dependency-path: frontend/package-lock.json
- name: Install frontend dependencies
Expand Down Expand Up @@ -60,8 +63,8 @@ jobs:
env:
RATE_LIMIT_TEST_DATABASE_URL: postgres://postgres:postgres@localhost:5432/postgres
steps:
- uses: actions/checkout@v4
- uses: denoland/setup-deno@v2
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: denoland/setup-deno@22d081ff2d3a40755e97629de92e3bcbfa7cf2ed # v2
with:
deno-version: v2.4.5
- name: Type-check API and tests
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@ jobs:
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4

- name: Configure GitHub Pages
uses: actions/configure-pages@v5
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5

- name: Install frontend dependencies
working-directory: frontend
Expand All @@ -45,10 +45,10 @@ jobs:
touch frontend/dist/.nojekyll

- name: Upload Pages artifact
uses: actions/upload-pages-artifact@v3
uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa # v3
with:
path: frontend/dist

- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4
4 changes: 2 additions & 2 deletions .github/workflows/supabase-functions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ jobs:
SUPABASE_PROJECT_REF: ${{ vars.SUPABASE_PROJECT_REF }}
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4

- name: Install Supabase CLI
uses: supabase/setup-cli@v1
uses: supabase/setup-cli@1dedf2c611547ede7232d26866dd3c56ab903bbb # v1
with:
version: latest

Expand Down
3 changes: 2 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,14 @@ ENV PYTHONUNBUFFERED=1
ENV PYTHONPATH=/app/src

COPY requirements.txt .
COPY constraints-security.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY src ./src
COPY docs ./docs
COPY static ./static

RUN mkdir -p /app/data /app/data/chroma
RUN mkdir -p /app/data

EXPOSE 8000

Expand Down
71 changes: 71 additions & 0 deletions constraints-security.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Python 3.12 security resolution; platform-only dependencies remain conditional.
PyYAML==6.0.3
Pygments==2.21.0
SQLAlchemy==2.1.6
annotated-doc==0.0.5
annotated-types==0.8.0
anyio==4.15.1
certifi==2026.10.23
cffi==2.1.1
charset-normalizer==3.5.2
click==8.5.0
colorama==0.4.6
cryptography==50.0.2
distro==1.9.0
fastapi==0.162.0
filetype==1.2.0
google-auth==2.62.0
google-genai==2.33.0
h11==0.16.0
httpcore2==2.16.0
httpcore==1.0.9
httptools==0.9.0
httpx2==2.16.0
httpx==0.28.1
idna==3.20
iniconfig==2.3.1
jiter==0.18.0
jsonpatch==1.36
jsonpointer==3.4
langchain-core==1.7.0
langchain-google-genai==4.4.2
langchain-ollama==1.1.0
langchain-openai==1.7.2
langchain-protocol==0.0.20
langsmith==0.14.12
numpy==2.5.4
ollama==0.6.3
openai==3.37.0
opentelemetry-api==1.45.1
orjson==3.13.0
packaging==26.3
pluggy==1.7.0
psycopg-binary==3.3.6
psycopg==3.3.6
pyasn1==0.6.4
pyasn1_modules==0.4.2
pycparser==3.12
pydantic-settings==2.16.1
pydantic==2.14.1
pydantic_core==2.50.1
pytest-asyncio==1.4.0
pytest==9.1.1
python-dotenv==1.2.4
regex==2026.9.29
requests-toolbelt==1.0.0
requests==2.34.2
sniffio==1.3.1
starlette==1.10.0
tenacity==9.1.4
tiktoken==0.14.0
truststore==0.10.5
typing-inspection==0.4.4
typing_extensions==4.16.0
tzdata==2026.5
urllib3==2.8.0
uuid_utils==0.18.0
uvicorn==0.54.0
watchfiles==1.3.0
websockets==17.3
xxhash==4.0.2
zstandard==0.25.0
10 changes: 8 additions & 2 deletions docs/02-langchain.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,21 @@
流程:

1. 筆記寫入 SQLite
2. `RagService.sync_note()` 把內容切成 Document 存入 Chroma
3. `POST /ask` 用 Retriever 找 top-k 片段
2. `POST /ask` 從 SQLite 筆記重建 LangChain Core 的 `InMemoryVectorStore`
3. 在該次查詢的記憶體索引找 top-k 片段,同一批片段供 prompt 與 sources 使用
4. Prompt + Chat Model 產生答案

對應程式:

- `src/app/services/langchain_rag.py`
- `src/app/services/llm_factory.py`

### 本機索引與資料保留(2026-10-25)

Chroma 及其整合套件已移除。SQLite 是筆記唯一持久資料來源,新增/更新/刪除只寫 SQLite;每次問答重建索引,因此重啟、更新與刪除後不會讀到舊向量。舊 `CHROMA_DIR` 設定仍可解析,但不讀寫或清除該目錄,不需搬移既有筆記。

代價是每次問答都要為現有筆記產生向量,適合小型本機教學,不能直接外推大量資料或遠端模型延遲。預設 mock 使用固定的合成向量,方便重現;不是語意檢索或答案品質證明。實際模型與資料集品質另驗,測試不呼叫付費服務。

## LLM 模式

| LLM_PROVIDER | 需要 API Key | 用途 |
Expand Down
4 changes: 4 additions & 0 deletions docs/03-system-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,10 @@ flowchart LR

## 5. ADR

### ADR-006:本機 RAG 安全替換(2026-10-25 已核准)

Chroma 改為 LangChain Core 記憶體索引,由既有 SQLite 筆記於每次查詢重建,保留舊索引目錄。契約、代價、量測前效能門檻與驗收見 [ADR-006](adr-006-memory-index-security.md)。正式雲端 API 不因本次本機改動而重啟。

### ADR-001:正式前端使用 GitHub Pages + Supabase

- 狀態:accepted
Expand Down
12 changes: 12 additions & 0 deletions docs/adr-006-memory-index-security.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# ADR-006:以 SQLite 重建本機記憶體索引

- 日期:2026-10-25;狀態:使用者已核准替換 Chroma、驗證後合併。
- Task:SEC-P2-20261025;輸入為既有 SQLite notes schema、原問答 API 與合成筆記;交付為無 Chroma 的本機 RAG 路線及去敏驗證證據。
- 背景:Chroma 1.5.9 有4個無修補版公告,嵌入模式路由不可達不等於套件已修補。
- 決策:使用既有 LangChain Core 的 InMemoryVectorStore;SQLite 是唯一持久來源。每次 ask 重建該次索引,只檢索一次,使答案上下文與 sources 一致;保留來源排序。CRUD 不建立第二份持久索引。
- 保留:既有 SQLite 資料與舊 Chroma 目錄不讀寫、不移位、不刪除;CHROMA_DIR 僅相容舊設定。
- 取捨:每次查詢需重建向量,避免全域快取失效與跨程序舊索引。這是本機小型教材範圍,遠端 provider 的花費與延遲不能由此證據推論;本次只用免費 mock/注入向量fixture。
- Out of scope:重啟已停用雲端後端、實際 LLM 品質、搬移私人資料、啟用付費 API 或大型向量服務。
- 驗收:SQLite 真實讀寫後跨新RagService查詢、更新/刪除/空資料、來源順序與上下文一致、舊目錄位元組保留;全套既有API與前端煙霧;乾淨環境安裝不含Chroma且解析依賴公告0。
- 效能門檻(量測前固定):同硬體Python3.12、384維固定mock,100與1000合成筆記,各2次暖身及20次問答(包含SQLite讀取、重建、檢索及mock回答);暖p95分別<500ms/<2000ms,1000筆冷重建<5000ms。記錄舊Chroma基線及新索引結果,不把測試耗時當正式API延遲,也不以合成題庫評真實語意品質。
- 證據:security/reports/2026-10-25-p2-remediation.md/.json;本機原始量測由跨專案安全工作保留,不提交私人路徑或資料。
Loading
Loading