Preserving Grandma's Recipes & Stories: Building a 100% Private, Source-Grounded Family Cookbook with Open-Source AI

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

Every family has a treasure trove of secret recipes, culinary tips, and cherished memories. But for most of us, these aren't written down in neat cookbooks or standardized digital apps—they are hidden inside informal WhatsApp voice messages, spoken anecdotes over dinner, and cassette tapes passed down across generations.

When my grandmother cooks, she doesn't measure in grams or follow 10-minute timers. She cooks by feel and memory:

"Take a handful of fresh cumin seeds, roast them until fragrant like my mother used to do on Sunday afternoons in Jaipur, then add potatoes until soft..."

Traditional recipe apps fail here because they demand rigid structured inputs. On the flip side, commercial cloud AI services present two severe flaws:

1. Privacy & Sensitivity: Sending intimate family voice recordings, personal names, and vocal biometrics to third-party cloud LLM APIs surrenders private heritage to remote servers.
2. AI Hallucinations & Distortion: Standard AI generators tend to "fix" recipes by inventing missing quantities (e.g. inserting 1 tsp salt or 10 mins prep time) when Grandma never specified them, mangling family tradition into generic generic web recipes.

To solve this, I built Grandma's Kitchen—a 100% local, privacy-first, AI-powered family cookbook and memory archive.

What Grandma's Kitchen Does:

• Voice-to-Recipe Extraction: Ingests raw audio recordings or live browser voice notes and transcribes them locally via Whisper STT.
• Strict Source Grounding & Anti-Hallucination: Extracts ingredients, steps, and timings into structured recipe cards without ever inventing missing details. Unspecified quantities are marked as null with confidence: "unknown".
• Memory vs. Cooking Separation: Separates personal anecdotes ("My mother made this every Diwali...") from step-by-step cooking instructions, preserving family memories in a dedicated memory archive tab.
• Interactive Provenance Inspector: Every extracted ingredient, step, and story maintains line-by-line provenance linking directly back to original audio transcript quotes.
• Local Semantic Search: Allows family members to search by natural questions ("What did Grandma make on cold rainy days?") using local vector embeddings.

Demo

Watch the complete project overview and live walkthrough below:

📺 Video Link: Watch Grandma's Kitchen Demo on YouTube

Key Features Highlighted in the Demo:

• 🎙️ Live Audio Ingestion: Browser recording with waveform display and instant file upload support for .wav, .mp3, .m4a, and .ogg.
• 🔍 Source Provenance Modal: Clicking on any ingredient or cooking step opens an evidence drawer showing the exact transcript excerpt and audio context supporting that fact.
• 📖 Family Memory Archive: View captured stories, relatives mentioned, and cultural occasions tied to each dish.
• 🖨️ Printable Family Cookbook: Clean, print-ready CSS stylesheet for exporting physical family heirloom cookbooks.

Code

The entire codebase is open-source and available on GitHub:

🐙 GitHub Repository: scha54/Hactoberfest-Weekend-Challenge

grandmas-kitchen/
├── backend/
│   ├── app/
│   │   ├── api/             # FastAPI REST endpoints (recordings, recipes, search)
│   │   ├── core/            # Config, database initialization, security
│   │   ├── db/              # SQLite database session & models
│   │   ├── schemas/         # Pydantic data schemas & JSON specs
│   │   └── services/
│   │       ├── extraction/  # Ollama LLM provider & strict source-grounding prompts
│   │       ├── transcription/# OpenAI Whisper local audio transcription engine
│   │       ├── embeddings/  # Local vector embedding generation
│   │       └── search/      # Local cosine similarity vector search
│   └── tests/               # Pytest suite for API endpoints & extraction logic
├── frontend/
│   ├── src/
│   │   ├── components/      # RecipeSteps, SourceEvidenceModal, AudioRecorder, etc.
│   │   ├── pages/           # UploadPage, CookbookPage, SearchPage, RecipeDetailPage
│   │   └── services/        # API client & audio streaming utilities
│   └── public/              # Static assets & sample recordings
├── data/                    # 100% local persistent storage (audio, SQLite DB, vectors)
└── scripts/                 # Cross-platform 1-click startup scripts (.ps1 & .sh)

How I Built It

Grandma's Kitchen was designed from the ground up around local, open-source AI models and a resilient provider abstraction architecture.

┌─────────────────────────────────────────────────────────────┐
│                       Browser Web UI                        │
│             React + TypeScript + Vite + Tailwind            │
└──────────────────────────────┬──────────────────────────────┘
                               │ HTTP / JSON / Audio Stream
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                    FastAPI Local Backend                    │
└──────┬───────────────────────┼──────────────────────┬───────┘
       │                       │                      │
       ▼                       ▼                      ▼
┌──────────────┐       ┌──────────────┐       ┌──────────────┐
│ Local STT    │       │ Local LLM    │       │ Embeddings   │
│ Whisper      │       │ Ollama       │       │ Local Vector │
└──────┬───────┘       └──────┬───────┘       └──────┬───────┘
       │                       │                      │
       └───────────────────────┼──────────────────────┘
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                        Storage Layer                        │
│         SQLite (Metadata/Data) + Local Disk (Audio)         │
└─────────────────────────────────────────────────────────────┘

1. Local Speech-to-Text (Open-Weight Whisper)

• Model: OpenAI Whisper open-weight models running locally via Python (whisper / torch).
• Function: Audio files uploaded from the React frontend are written to data/audio/ and transcribed locally without leaving the computer.
• Fallback: Includes a native local Python fallback parser to ensure 100% testability even on resource-constrained systems without GPU acceleration.

2. Local LLM Extraction & Anti-Hallucination Framework

• Model: Ollama running open-weight LLMs (Llama 3, Mistral, or Qwen) via local HTTP endpoints (http://localhost:11434).
• Prompt Engineering & System Constraints: We engineered a strict 7-point System Prompt enforcing strict source grounding:
  • Never invent missing quantities: Unmentioned measurements become quantity: null and confidence: "unknown".
  • Never assume culinary constants: Vague instructions ("until brown") are kept verbatim rather than converted to guessed minutes.
  • Categorize storytelling: Anecdotes and mentions of relatives are placed into stories[], keeping cooking steps clean.
  • Require provenance: For every extracted ingredient, step, and story, the LLM must output source_text containing the exact transcript quote.

3. Local Vector Search & Embeddings

• Model: Local vector embedding engine calculating semantic dense vector representations.
• Search Mechanics: Cosine similarity search over recipe titles, ingredient vectors, and family memory stories, returning ranked results with matching context snippets.

4. Full-Stack Local Technology Stack

• Frontend: React 18, TypeScript, Vite, Tailwind CSS, Lucide Icons, and Web Audio API for in-browser audio capture.
• Backend: Python 3.10+, FastAPI, Pydantic v2, SQLite (for zero-config relational storage), Pytest.
• Deployment: Zero external dependencies. Controlled with 1-line startup scripts (.\scripts\start.ps1 or ./scripts/start.sh).

Why Does Open Innovation Matter?

Open innovation is not just a developer preference for Grandma's Kitchen—it is a fundamental requirement. Here is why open-source AI was irreplaceable for this project:

1. Privacy & Data Sovereignty for Intimate Family Heritage

Voice messages are uniquely personal. They contain non-verbal emotional nuances, family names, home locations, and vocal biometrics. Handing over decades of intimate family voice recordings to proprietary cloud APIs exposes personal heritage to remote server logs, third-party data breaches, and AI training pipelines. Open-weight models (Whisper and Llama) allow families to retain 100% local control over their data.

2. Preserving Qualitative & Cultural Cooking Nuances

Proprietary cloud LLMs are heavily RLHF-tuned toward standard Western commercial outputs. When given informal spoken text, closed models tend to "autocorrect" cultural recipes into standard cookbook formulas—converting a pinch into 1/4 tsp or dropping regional term names. Open models allow custom prompts and open-source fine-tuning that respect qualitative cooking language, family idioms, and regional traditions.

3. Zero Cost & Decade-Long Permanence

Family heirlooms should last for generations. Relying on closed SaaS APIs introduces monthly subscription fees, API pricing changes, and vendor shutdown risks. By building on open-source AI running on local hardware, Grandma's Kitchen costs $0/month to run and will remain functional 10 or 20 years from now, completely independent of cloud service lifecycles.

4. Reproducibility & Community Empowerment

Because every component of this stack relies on open weights and open-source software, any family or developer worldwide can clone our GitHub repository, run a single setup script, and immediately build a private memory vault for their own loved ones.

Open innovation turns technology from a locked walled garden into a living tool for human connection and cultural preservation.

Thank you for reading! Feel free to check out the GitHub Repository and share how you preserve your family's recipes and stories.