Next experiments · In planning
Building a private AI journal archive
An experiment in making years of handwritten and digital journals searchable, with answers that point back to the original pages. I want to do it with local AI rather than send private writing to a cloud service.
Where it stands: this is a plan. I haven’t run the full pipeline yet, so nothing here is a result. Where I say what the design should do, I mean it as a goal I still have to test.
The problem
Years of journals hold decisions, changes of mind and patterns I can’t see from one page. They are hard to search, and harder to compare across time. I’d like to be able to ask:
- When have I felt this way about work before?
- What helped during similar periods in the past?
- How has my thinking about music or career direction changed?
- What patterns seem to repeat, and which entries support that?
The journals stay the source of truth. The AI’s job is to find relevant passages and help me read them. It shouldn’t be treated as an authority on my life, or as a memory I can trust without checking.
Design principles
- Local first. This writing doesn’t go to a cloud model.
- Originals are never overwritten. Every photo and PDF is kept as it was.
- Plain Markdown is the archive. It stays readable, editable, portable and backed up on its own.
- Answers show their sources. Dates and page references, with quotes, summaries and interpretation kept apart.
- Uncertainty stays visible. An unclear word is marked
[unclear], and an unsure date is flagged, not guessed.
Proposed design
From a page to a sourced answer.
- Capture. Photograph or scan the physical journals in batches, and import my existing reMarkable and ViWoods PDFs.
- Transcribe locally. A vision model on my own computer reads each page. Qwen3-VL 8B is one candidate. I haven’t chosen one.
- Write dated Markdown. One file per page or entry, with its source page, the model used and a review status.
- Keep it in a separate Obsidian vault. It’s a new vault, apart from the one my cloud-connected assistants use.
- Index it with AnythingLLM. It uses local embeddings, a local index and a local language model. It’s the search and chat layer, not the archive and not the transcriber.
- Answer with sources. Dates, passages and page references. Possibly reachable from my iPhone over a private network later. That part is only an idea.
A diagram of a proposed design, not a working system.
The privacy boundary
Privacy is the point of the project. The intended design keeps transcription, embeddings, the index and the answering model on my own machines, with no cloud model, cloud embedding service or web search connected to the journal workspace. If I add iPhone access, it would use a private network and not a public address.
Those are design goals. Local doesn’t mean secure by default. I’d still need to check the whole data flow, who can log in, the backups and the network setup. A local model can also get things wrong.
I’d start on my MacBook Pro (M1 Max, 32 GB). That’s a starting point for testing, not proof that every model will run well. I might move it to the ThinkPad or the Raspberry Pi later, but I haven’t tested either for this and I’m not planning to buy anything.
What I need to prove
The first test is small on purpose: about 20 representative pages, including clear and messy handwriting, different pens, dense and short entries, photographs of the physical journals and pages from my digital PDFs. I’d run each candidate model on the same pages and choose the fastest one that is accurate enough, not the biggest.
For each one I’d record:
- How accurate the transcription is against a page I’ve checked by hand, and how many corrections each page needs.
- How long a page takes and what a realistic batch looks like, including memory use and stability.
- How often dates are captured correctly, and how well doubtful words are flagged.
- Whether every Markdown file traces back to its original page.
- Whether answers find the right passages and cite them accurately.
- Whether the journal text really stays out of cloud services.
I’m not planning to fine-tune a model. Retrieval keeps the archive editable and citable, and I can add or correct entries without retraining anything.
Results so far
None yet. I’ll add what I measure, how I tested it and where it failed once I have it, and I’ll label a small sample as early.
This follows on from my earlier notes on reading text from images with Ollama, LM Studio and AnythingLLM and local AI for sensitive documents.