I Built a Multi-Agent AI Team to Run My Father's Business — Here's the Architecture

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I built

I built an AI team that runs my father's business. Not a chatbot, not a single agent, but a team of specialized AI agents that handle operations, content, customer support, code, and analytics. All coordinated through a central task system. All running on open-source AI.

The "friend" in this challenge is my father. He runs a small business and was drowning in repetitive work: answering the same customer questions, managing social media, updating product listings, tracking orders, reviewing comments on Google Business Profile. The work wasn't hard. It was just relentless.

So I built him a team of AI agents to do it for him.

Meet the team:

• Kodi: the dispatcher. Assigns tasks, tracks progress, detects blockers, handles handoffs, and only nudges me when something needs human attention.
• Stark: the engineer. Manages Git projects, the portfolio website, side projects, deployments, and PR reviews.
• Edith: the marketer. Handles Amazon storefront, Google Business Profile, social media content, and ad campaigns.
• Friday: the support agent. Maintains the analytics dashboard, handles customer queries, and keeps the system lean.

They don't just respond to prompts. They proactively pick up work, coordinate with each other, and report back. Like a real team, but without the meetings.

Demo

The system runs on a local server. Here's what it looks like in action:

→ New customer inquiry detected
→ Friday picks it up, drafts a response
→ Kodi assigns Stark to review the technical details
→ Stark approves, Friday sends the response
→ Edith logs the interaction for follow-up
→ All of this happens in under 30 seconds

The dashboard shows live agent status, task queues, and a real time activity feed. It's like having a Jira board, but the workers are AI.

How I built it

The Core: Open-Source Agent Framework

The whole system runs on Hermes Agent, an open-source AI agent framework. Each agent is a separate instance with its own:

• System prompt: personality, role, constraints
• Tool access: what it can do (GitHub, databases, APIs, file system)
• Memory: persistent context across sessions
• Skills: reusable procedures it can invoke

No closed APIs. No black boxes. Every piece is inspectable, modifiable, and runs locally.

The Coordination Layer: SQLite + MCP

Agents don't talk to each other directly. They communicate through a shared SQLite database, a single source of truth for:

• Task assignments and status
• Agent availability and current work
• Approval workflows (who can approve what)
• Event history (append-only audit trail)

This was a deliberate choice. I wanted something I could inspect with any SQLite browser, debug with simple queries, and back up with cp. No message broker, no Redis, no Kafka cluster. Just a database.

The database is exposed via MCP (Model Context Protocol), an open standard for giving AI agents access to external tools. Each agent has an MCP client that lets it:

# Agent picks up a task
tasks = mcp.get_tasks(status="available", agent="stark")
mcp.acknowledge_task(tasks[0].id, agent="stark")

# Agent reports progress
mcp.log_progress(tasks[0].id, agent="stark",
                 summary="Started implementation",
                 step="Writing tests first")

# Agent submits for review
mcp.submit_completion(tasks[0].id, agent="stark",
                      result="Feature implemented",
                      evidence="Tests passing, PR opened")

The Approval System: Decisions Agents Can Take

Not every decision needs a human. I built an approval system where each agent has a scope of autonomy:

• Friday can approve responses to common customer questions
• Stark can merge PRs that pass all checks
• Edith can publish content that matches the brand voice
• Kodi can reassign tasks when someone is blocked

When an agent hits something outside its scope, it escalates. Kodi decides whether to handle it or nudge me. I only get notified when something broke, needs my attention, or requires an approval. PR merges, escalations, and blockers. That's it.

This was the key to making the system actually useful. Without it, I'd be approving every little thing. With it, the team runs itself and I only step in when it matters.

The Memory System: Mnemosyne

Agents forget things. It's annoying. So I built a persistent memory layer called Mnemosyne that gives every agent:

• Working memory: current session context
• Episodic memory: past experiences, lessons learned
• Semantic memory: facts, relationships, preferences
• Persona memory: stable identity and preferences that never get evicted

The memory system uses a three-tier architecture:

1. Hot tier: in-memory, always available, small
2. Warm tier: vector search (FAISS) for semantic recall
3. Cold tier: full-text search (SQLite FTS5) for exact matches

When an agent needs to remember something, it queries Mnemosyne. The system returns relevant memories ranked by a hybrid score: 50% vector similarity, 30% text match, 20% importance.

This means when I ask Stark to "fix the login bug from last week," it doesn't just search for "login bug." It semantically understands what I mean and pulls up the relevant context from a week ago.

The RAG System: Hybrid Retrieval

The agents need access to a lot of knowledge: documentation, past decisions, code patterns, customer FAQs, product catalogs, competitor data. I built a Retrieval-Augmented Generation (RAG) system that indexes:

• 8,500+ chunks from our knowledge base
• 180+ agent skills and procedures
• 22 shared retro lessons from past projects

The retrieval uses a hybrid approach:

• FAISS for vector similarity (semantic search)
• BM25 for keyword matching (exact search)
• α = 0.6 blending factor (slightly favoring semantic)

Why hybrid? Because pure vector search misses exact matches ("What's the API key for Stripe?"), and pure keyword search misses semantic matches ("How do I handle payment failures?"). The hybrid gets both.

The RAG system is what lets agents answer questions they've never seen before. Instead of hardcoding every response, they retrieve relevant context and generate an answer. It's like giving the agents a library card.

The Dashboard: Agent HQ

I built a web dashboard that shows:

• Live agent status: who's working, who's idle, who's blocked
• Task queue: prioritized by urgency and dependencies
• Activity feed: real time stream of agent actions
• Kodi's hologram: a visual indicator of the dispatcher's state

The dashboard is a single-page app that polls the SQLite database via a lightweight Python server. No framework, no build step. Just HTML, CSS, and JavaScript talking to a REST API.

What Each Agent Actually Does

Edith: The Marketing Machine

Edith handles the entire marketing operation:

• Amazon storefront: creating and managing product listings, competitor search, SEO keyword optimization, generating product images
• Google Business Profile: automating posts, responding to comments, updating business information
• Social media: drafting content, scheduling posts, tracking engagement
• Ad campaigns: managing Meta Ads, A/B testing headlines, optimizing for conversions

The next step is fully automatic posting on Instagram, Facebook, and Meta Ads. Edith will generate the content, review it, publish it, and track performance. No human intervention.

Stark: The Engineer

Stark handles all technical work:

• Git projects: managing repositories, reviewing PRs, merging code
• Portfolio website: updating content, fixing bugs, deploying changes
• Side projects: building features, writing tests, managing infrastructure
• Deployments: staging and production, rollback on failure

Stark is the agent I'd want on my team if I were building a startup. It writes clean code, reviews thoroughly, and never breaks production (well, almost never).

Friday: The Support and Analytics Agent

Friday maintains the analytics dashboard on Agent HQ:

• Daily order status: how many orders, how many cancelled, revenue
• Social media metrics: IG followers, engagement rate, reach
• Google Business Profile: comments, ratings, response rate
• Skills learning: flagging and deleting unused reports, keeping the system lean and efficient

Friday also handles customer queries, drafts responses, and escalates technical issues to Stark. It's patient, thorough, and never loses its temper (even when customers do).

Kodi: The Dispatcher

Kodi runs the whole system. It:

• Assigns tasks to the right agent
• Detects blockers and escalates them
• Handles handoffs between agents
• Runs opportunity scans to find new work
• Triggers self-healing when something breaks
• Only nudges me when I need to know something

I mostly interact with only Kodi. It handles everything else. If a task is taking too long, Kodi reassigns it. If an agent is blocked, Kodi escalates it. If something needs my attention, Kodi tells me. Otherwise, I don't hear from the team.

Opportunity Scans and Self-Healing

The system doesn't just wait for tasks. Kodi runs regular opportunity scans to find new work:

• New customer inquiries that need responses
• Social media mentions that need engagement
• Product listings that need optimization
• Competitor changes that need analysis
• Analytics anomalies that need investigation

When something breaks, the system self-heals. If an agent crashes, Kodi restarts it. If a task fails, Kodi retries with a different approach. If a tool is unavailable, Kodi finds an alternative. The system is designed to keep running, even when pieces fail.

Cron Cost Cutdown

Running AI agents is expensive. Every token counts. I managed to cut cron costs significantly using deterministic steps:

• Instead of running expensive AI queries every minute, the system uses cheap deterministic checks first
• Only when a deterministic check flags something does the system invoke the AI
• This reduced our token usage by 70% while keeping the system responsive

The trick was to make the AI the exception, not the rule. Most of the time, the system can handle things with simple if/else logic. Only when it can't does it ask the AI for help.

Why does open innovation matter?

Every piece of this system is open-source:

• Hermes Agent: the agent framework
• FAISS: vector similarity search
• SQLite: the coordination database
• MCP: the tool protocol
• MiniLM-L6-v2: the embedding model

If I'd used closed APIs, I'd be:

• Locked in: if the API changes or shuts down, the whole system breaks
• Blind: I couldn't inspect why the agent made a decision
• Limited: I couldn't customize the behavior for my specific needs
• Paying per token: costs scale linearly with usage

With open source, I can:

• Run everything locally: no data leaves my machine
• Swap components: try a different embedding model in minutes
• Debug everything: every layer is inspectable
• Extend freely: add new agents, new tools, new capabilities

The system cost me $0 in API calls to build. The only cost was my time, and honestly, it was a fun weekend.

Future roadmap

This is just the beginning. Here's what's next:

Phase 2: Autonomous Content Pipeline

Edith will automatically:

• Generate content calendars based on trending topics
• Draft, review, and publish posts across platforms
• A/B test headlines and optimize engagement
• Learn from performance data to improve future content
• Auto-post on Instagram, Facebook, and Meta Ads

Phase 3: Self-Improving Agents

Agents will:

• Analyze their own performance and identify bottlenecks
• Propose improvements to their own skills and procedures
• Share learnings across the team automatically
• Run chaos engineering tests on their own infrastructure

Phase 4: Multi-Business Support

The system will:

• Support multiple isolated business contexts
• Share learnings across businesses (with permission)
• Provide cross-business analytics and insights
• Enable new businesses to spin up their own agent team in minutes

Phase 5: Open Source Release

I plan to open-source the entire system so other builders can:

• Deploy their own multi-agent team
• Customize agents for their domain
• Contribute new capabilities and integrations
• Learn from our architecture and patterns

What I learned

Building this taught me that the hardest part of multi-agent systems isn't the AI. It's the coordination. Getting agents to work together without stepping on each other, without duplicating work, without losing context. That's the real engineering challenge.

The SQLite database was the key insight. By making the coordination layer simple, inspectable, and debuggable, I could focus on the fun part: giving agents personality, skills, and autonomy.

If you're thinking about building something similar, start small. One agent, one task, one database table. Then add complexity only when you feel the pain of not having it.

What would you build with a team of AI agents? I'd love to hear your ideas in the comments.

Follow the Journey

This system isn't static. It evolves every week. I write weekly LinkedIn posts about what I'm building, how I'm improving the system, what problems I'm facing, and how I'm solving them. If you want to see how this architecture grows over time, follow me on LinkedIn and read the weekly posts. You'll see the decisions, the failures, the breakthroughs, and the lessons learned along the way.

Thanks for reading this and let me know your thoughts in the comments below !