Observer KB: new app for ChatGPT/Claude PDF knowledge extraction

- New branch: observer-kb
- PRD: apps/observer-kb/docs/PRD.md
- ChatGPT PDF parser: extracts conversation turns (user/assistant)
- Upload portal: pure HTML/JS (no WASM), drag & drop, progress bars
- Browse: view ingested datasets with preview
- Search: semantic search across embedded conversations (HNSW)
- Served on :3400 via systemd (observer-kb.service)
- Uses Lakehouse core for storage, query, vectors, AI

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-03-27 22:23:09 -05:00
co-authored by Claude Opus 4.6
parent 84407eeb51
commit c49b30053b
5 changed files with 580 additions and 1 deletions
+7 -1
View File
@@ -43,7 +43,13 @@ pub async fn ingest_file(
let (schema, batches) = match file_type {
FileType::Csv => csv_ingest::parse_csv(content)?,
FileType::Json | FileType::NdJson => json_ingest::parse_json(content)?,
FileType::Pdf => pdf_ingest::parse_pdf(content, filename)?,
FileType::Pdf => {
// Try ChatGPT parser first, fall back to generic PDF
match crate::chatgpt_parser::parse_chatgpt_pdf(content, filename) {
Ok(result) => result,
Err(_) => pdf_ingest::parse_pdf(content, filename)?,
}
}
FileType::Text => parse_text_file(content, filename)?,
FileType::Unknown => return Err(format!("unknown file type for '{filename}'")),
};