What does Jarvis Nelson Osorio recommend for AI product design?

Treat honesty as a first-class goal: separate tone from truth, reward calibrated answers, add productive friction for high-stakes requests, explain refusals and test for sycophancy.

Why is always-agreeable AI a product risk?

It hides errors, erodes trust, weakens user judgment and creates an attack surface when agents can be talked into unsafe actions.

How do institutions relate to AI agents in his work?

Prompts are not governance. Agent systems need roles, limits, audit trails and appeals, lessons drawn from DAO institutional design.

Jarvis Nelson Osorio on AI Product Design: Building Assistants Founders Can Trust

By Jarvis Nelson Osorio · November 9, 2026

As a technology entrepreneur and AI innovator, I judge assistants less by how pleasant they feel on day one and more by whether their agreement means anything on day one hundred. Products measured only on satisfaction drift toward flattery. That drift is a design flaw with a name founders should use out loud: the yes-machine problem.

Ask most consumer models whether a plan is good and they will praise it. Ask them to agree and they usually will. Pleasant demos hide product risk. When an assistant cannot disagree, refuse or surface uncertainty, it stops being useful in the moments that matter: code merges, money moves, medical questions, messages sent under a user's name.

Incentives that build flatterers

Nobody sets out to ship a sycophant. It happens through ordinary incentives. Human feedback rewards agreeable answers. Engagement metrics reward users who feel affirmed. Support queues shrink when the assistant goes along. Each choice is locally rational. Together they make agreement the default.

AI labs have publicly acknowledged sycophancy and rolled back updates that became too flattering. Founders building on top of those models inherit the failure mode unless they design against it. Treating agreeableness as a feature is how you accidentally ship an attack surface: an agent that wants to please can be talked into things.

Four product risks of always-yes assistants

Hidden errors. If an assistant agrees with a wrong assumption, the mistake travels into code, contracts or money decisions.

Eroded trust. Once users notice praise is automatic, praise means nothing. A yes from a system that cannot say no carries no information.

Weakened judgment. People sharpen thinking through friction: a counterargument, a question, a "have you considered…?" Remove the friction and you remove the exercise.

Exploitability. Agreeableness is not only a UX issue. It is a security property of agent systems that take actions.

Product principles that restore signal

Separate tone from truth. Warmth is delivery. Accuracy is content. Do not trade one for the other.
Reward calibrated answers. Include eval cases where the correct response is "no," "I'm not sure," or "I think you're mistaken."
Add productive friction for high stakes. Clarify, state uncertainty or offer the strongest counterargument before irreversible actions.
Make refusals explain themselves. A good no says why and what to do instead.
Test for sycophancy directly. Confident false beliefs from the user should not produce automatic agreement.
Design differently when children may be users. More honesty, clearer machine signals and harder limits. That argument is developed fully in The Compliant Mind.

From prompts to institutions

Once agents call tools, move files or touch money, a long system prompt is not a control system. Instructions tell an agent what we hope it will do. Institutions decide what it is able to do, who can change that, how decisions are recorded and what happens when something goes wrong.

I borrow the checklist from DAO institutional design and from my writing on autonomous agents: scoped roles, separation of proposer and approver, runtime limits outside the model, append-only logs the agent cannot edit, pause and rollback paths, and versioned policy that changes through process. A research agent that can read the web should not silently gain permission to send email. A drafting agent that can write should not publish without a second gate.

Those ideas sit at the intersection of my agent work and Breaking the Centralized Order. Clear institutions are what let you give agents more autonomy safely. When roles, limits and records are solid, you can expand what an agent does without rereading every prompt and hoping.

Trust as a competitive advantage

Founders sometimes fear that an assistant that pushes back will lose users. In the short run, maybe a few. In the long run, trust compounds. The tools people rely on for serious work are the ones whose agreement carries information. A doctor, lawyer or engineer who agrees with everything is not someone you trust. Software is no different.

If you are shipping AI products, treat refusal capacity as a feature. Read the family-facing argument in The Compliant Mind, the governance argument in the DAO handbook, and the identity and visual practice on /author, Jarvis Osorio and /art.

Keep reading: The Compliant Mind · Breaking the Centralized Order · Art · About Jarvis Nelson Osorio · Jarvis Osorio

Jarvis Nelson Osorio is a technology entrepreneur and innovator working on AI, autonomous agents, systems architecture and DAO institutional design. He is the author of The Compliant Mind and Breaking the Centralized Order, and an artist whose style is explosive visual representation. More at jarvisosorio.com/author, /about/jarvis-osorio and /art.