AI Diary
An intelligent voice & photo journaling system — record your day on iPhone or snap a photo, let AI transcribe, analyze, and remember it for you. All data stays home on a Raspberry Pi.
What it does
AI Diary turns your voice into a searchable, queryable personal archive. Record an audio note on your iPhone, drop it via Shortcuts to the FastAPI backend running on a Raspberry Pi, and within moments the entry is transcribed, analyzed for mood, events, people, and topics — then stored locally where you can chat with it in natural language.
The project integrates with Open WebUI as a Pipe function so your diary lives right next to your other AI tools. A Telegram bot adds a second interface — send voice notes, text queries, or even photos straight from the Telegram app. Snap a picture and a vision model writes a Swedish description, attaching the image to the day's entry. Monthly and yearly summaries are generated automatically, and a hierarchical retrieval system ensures the chat interface fetches only the granularity it needs — keeping token costs low and answers accurate.
Privacy-first by design. All diary data lives on your local network. Only processed text is sent to an AI model for analysis — raw audio never leaves your home.
How an entry is processed
From voice or photo to searchable memory — all running automatically in the background.
iPhone Shortcut → Audio Upload
An iPhone Shortcut records m4a audio and POSTs it to the FastAPI endpoint. The HTTP response is instant — heavy processing happens in a background task.
Transcription via OpenRouter
Audio is sent to a configurable OpenRouter transcription model, returning clean Swedish-language text.
Photo & Vision Analysis
Send a photo via Telegram and a vision model writes a Swedish description, attaching the image to the day's entry. The description is stored locally so the raw image is only sent to the model once.
LLM Analysis
A language model extracts a summary, mood indicators, events, mentioned people, topics, and action items from the transcription.
Daily Entry Merge
Multiple recordings from the same day are automatically merged into a single cohesive daily entry.
SQLite FTS5 Storage
Transcription, structured analysis, and rolled-up summaries are stored with full-text indexing across all fields for fast retrieval.
Health Data via iPhone Shortcut
A second Shortcut sends a daily Apple Health snapshot — steps, distance, active energy, flights climbed — either as a direct API call or as a JSON message dropped into the Telegram bot. Field names are normalised against a list of common aliases so it isn't brittle to how the Shortcut happens to be built, then upserted as a single row per date.
Chat & Reports via Open WebUI
A Pipe function exposes the diary in Open WebUI. Query intent classification decides whether to fetch quick summaries or full transcriptions before responding — and any photos tied to the referenced dates are returned alongside the answer.
Dagboksradion — Audio Summaries
On request, an LLM writes a TTS-friendly Swedish script for a day, month, year, or year-to-date. OpenRouter renders it to MP3 and both script and audio are cached, so the spoken summary is delivered as a playable file in chat or Telegram.
Telegram Bot Interface
An alternative access layer built on the same pipeline. Send voice notes or text messages via Telegram to record entries or query the diary — conversation history is kept per chat for context-aware replies.
Dagboksradion — audio summaries
Turn your diary into a podcast. Dagboksradion generates a radio-style spoken summary for a single day, a month, a full year, or the year so far. A language model writes a TTS-friendly script in Swedish, OpenRouter renders it to an MP3, and both the script and audio are cached so repeat requests are instant.
It's triggerable straight from a conversation — no menus, no buttons. Ask in Open WebUI ("Ge mig en ljudsammanfattning för idag", "Gör en podcast av juni") or message the Telegram bot in natural language or with a slash command like /summary 2026. The bot replies with a playable audio file titled for the period it covers.
Health data, logged straight from your iPhone
A second iPhone Shortcut pulls a daily snapshot from Apple Health — steps, distance, active energy, and flights climbed — and delivers it to AI Diary either as a direct API call or as a plain JSON message dropped into the Telegram bot. Both paths funnel through the same upsert logic, so a day's health row is created once and quietly updated as new numbers roll in.
Different Shortcuts export field names inconsistently, so the parser normalises keys and matches them against a list of common aliases (steps, step_count, stepcount all resolve to the same field) rather than depending on one exact schema. Health rows are stored per calendar date alongside the diary entries, and the Telegram bot replies with a short Swedish confirmation summarising what was saved or updated — steg, kilometer, kcal, and våningar.
Key capabilities
- Voice Capture via iPhone Shortcuts — Upload m4a recordings directly from your phone. Background processing keeps the API response instant.
- Smart Transcription — Converts audio to text using configurable models through the OpenRouter API, with Swedish language support.
- Photo Journaling with Vision AI — Send photos through Telegram and a vision model writes a Swedish description, attaching the image to that day's entry. Photos are served from a dedicated endpoint and surface in chat answers — inline base64 for web clients, server URLs for Telegram.
- Intelligent Analysis — LLM-powered extraction of summaries, mood indicators, events, people mentions, topics, and action items.
- Unified Daily Entries — Multiple recordings from the same day are automatically merged into a single cohesive entry.
- Full-Text Search — SQLite FTS5 indexes transcriptions, summaries, topics, and mentioned individuals for fast, flexible querying.
- Contextual Chat Interface — Natural language Q&A that classifies query intent and fetches either summaries or full transcriptions depending on what the question needs.
- Temporal Summaries — Automatically generates and stores condensed monthly and yearly overviews for long-term reflection.
- Apple Health Logging — A dedicated iPhone Shortcut sends a daily Apple Health snapshot (steps, distance, active energy, flights climbed) via API call or as a Telegram JSON message. Field names are normalised against common aliases and upserted per date, with a Swedish confirmation reply in Telegram.
- Dagboksradion — Audio Summaries — Podcast/radio-style spoken summaries for a day, month, full year, or year-to-date. An LLM writes a TTS-friendly Swedish script, OpenRouter renders it to MP3, and both script and audio are cached. Trigger it conversationally in Open WebUI or via natural language /
/summary <period>in Telegram. - Telegram Bot — Record voice notes or ask questions directly in Telegram. The bot processes audio through the same transcription pipeline and keeps per-chat conversation history for context-aware, natural language responses. Access is restricted to an allowlist of user IDs.
- Privacy-First Design — All diary data lives locally on the Raspberry Pi. Only processed text is sent to the AI model — raw audio never leaves your home network.
Technologies used
Async Python backend with background tasks
Local storage with full-text search
Transcription & language models
Swedish photo descriptions
Cached Swedish audio summaries (MP3)
Always-on home server
Voice capture & upload automation
Daily steps, distance & energy logging
Chat interface integration
Voice & text diary access