AI Product Engineering: What I Learned Shipping AI Products People Actually Use

"AI product engineering" is the job title on every hiring board and the theme of every enterprise blog this year. Most of those posts are strategy decks with no shipped product behind them. I run three AI products in production right now: an IvanOnTech advisor grounded in his daily streams, an Alex Hormozi advisor grounded in his business talks and a mirror of myself built from my own notes.

The pattern repeats across all three. The model behind them is a commodity I swap when a better one ships. The product is the system around it: where the knowledge comes from, the rules that keep answers honest, the loop that keeps it fresh and the memory of who the user is. That system is engineering work. It is where the value lives.

This post walks through those four layers with the concrete decisions behind them. If you are building an AI product, an agent or a support bot, the same layers apply. The checklist at the end is what I would hand any team starting out.

---

The Three AI Products I Run in Production

One is an advisor that answers as IvanOnTech, grounded in real transcripts of his daily livestreams. A refresh script pulls the newest streams, replaces the old seeds and the persona follows his current views. Another answers business questions as Alex Hormozi, with his business talks as the ground truth. The third is a mirror of myself, built from my own Obsidian vault, my notes and my daily entries.

None of these are demos. They run on schedule, refresh automatically and answer questions from people who come back week after week. That last detail is the metric that matters: whether the user comes back a month later and remembers the first answer. A chatbot demo gets one question. An AI product gets the second one.

That difference drives every engineering decision below.

The Four Layers of an AI Product

When I look at any AI product now, I break it into four layers:

• Source: where the product's knowledge comes from
• Rules: the constraints that keep answers grounded
• Refresh: how the knowledge stays current
• State: what the product remembers about the user

Most teams build only the middle two. They write a system prompt, add guardrails and ship. The products that survive are the ones that treat all four as first-class engineering work. Chip Huyen makes the same point in her book AI Engineering. Anthropic's "Building Effective Agents" describes the shift from model quality to system design.

The Source Layer: Your Data Is the Moat

The first decision is where the product gets its knowledge. For my personas that means transcripts of real streams, not summaries and not encyclopedia entries. Real material carries the voice, the phrasing and the repeated lines. Summaries flatten it into something any expert could have said.

Two source lessons cost me real time.

Recency matters more than popularity. YouTube's videos tab sorts by popularity, which once handed me three-month-old streams as "latest". The RSS feed is the source of truth. Whatever your product is, define what "current" means and source against that definition, not against what looks popular.

Off-script material carries the real value. The streams where something went wrong, a red day, a rant, a vulnerable moment, hold the emotional register that makes a persona feel human. If you only seed the best days, the product never learns how the persona behaves when things go wrong. The same applies to any product with a personality: honest material builds trust, polished material builds distance.

The Rules Layer: Grounding Beats Prompting

The second layer is the rule set that keeps answers honest. My grounding rules are short and they do the heavy lifting:

• Read all sources before answering
• Never invent facts the sources did not say
• When sources conflict, the newest one wins
• If the answer is not in the sources, say so in the persona's voice

That last rule separates a product from a hallucination machine. A market advisor handles price questions with a refusal in its own voice. It says "I follow the trend, I do not predict" and redirects to the framework. The refusal is part of the persona, which makes it feel honest instead of evasive.

This is context engineering at the system level. It matches Anthropic's agent guidance. Hand the system the right material and clear rules, and the model does the rest. Prompt tricks fail under pressure. Rules plus sourced material hold.

The Refresh Loop: AI Products Rot

A persona grounded in last month's streams is a persona from last month. Markets move and opinions shift. A product that answers with stale views loses trust in one conversation. The user remembers the last thing the persona said. The product has to remember it too.

My refresh runs as a script on a schedule, no agent in the loop. It pulls the newest streams, checks they are live streams within the window, overwrites the old seeds and stays quiet unless something changed. Zero tokens, zero attention, automatic. This is the same pattern I described in my post about automating my content pipeline with Claude Code skills. The recurring work belongs in scripts, not in agent runs.

If your AI product has knowledge that can go stale, you do not have a one-time build. You have a service that needs a refresh loop. That loop needs to be engineered like any other pipeline.

The State Layer: From Chatbot to Companion

The layer most AI products skip is state. A stateless bot answers the same question the same way every time. A product with memory knows what you asked last week, what you are working on and what it told you before.

For my personas that means tracking the relationship: what the user asked, where they are in their journey and the callbacks the persona would naturally make. The effect shows up fast. A user who gets "you asked about this last time, did you follow through?" treats the product as a person, not as a search box. One conversation with memory is worth ten without it.

This is where AI product engineering is heading. The models are good enough now. The differentiator is whether the product remembers.

The Checklist for Your AI Product

If you are building one now, here is the order I would work in:

• Define the source and what "current" means for it. Build the sourcing before the prompt.
• Write the grounding rules as a short list. Include the refusal rule and test the refusals first. They are the trust builders.
• Build the refresh loop on a schedule. Scripts, not agents, for the recurring work.
• Add state last but do not skip it. Start with the last three things the user asked.
• Ship and watch for the second question. That is the metric.

The personas run as agent skills, a pattern I covered in Stop Writing Prompts. Start Writing Skills.. Skills are the packaging that makes this repeatable. The sources, the rules and the refresh live in one place that any agent can load.

Conclusion

AI product engineering is source, rules, refresh and state around a commodity model. I have seen the difference this system makes across three products that people use every week. The pattern transfers to any AI project.

The work is sourcing real material, writing short rule lists, scheduling scripts and keeping state. Skip those parts and you ship a demo that answers once. Build them and people come back.

If you are building AI products or automating your workflow with agents, I document the full systems behind the personas and the pipelines in The Agentic Architect Lab on Skool. Senior Dev Energy + AI Speed = Ship More Together.

Sources

• Anthropic, "Building Effective Agents": https://www.anthropic.com/engineering/building-effective-agents
• Chip Huyen, "AI Engineering" (book, companion resources): https://github.com/chiphuyen/aie-book
• addyosmani/agent-skills (GitHub): https://github.com/addyosmani/agent-skills