Big Technology Podcast

Vijoy Pandey on why every multi-agent system fails silently

Vijoy Pandey· SVP & GM, Outshift at Cisco
·~37 min·English·Big Technology
AgentsAI SafetyAI InfrastructureOpen Source
TL;DR

Outshift's Vijoy Pandey says connected agents fail silently: bad assumptions compound until three cognitive failures crater measured accuracy from 98% to 16%, and he argues the same probabilistic shift is remaking computing all the way down to quantum.

01The State of Agents

Isolated, to Islands, to Connected

Agents raced from isolated, to islands trapped inside one vendor's harness, to connected across vendors, clouds and companies — but Pandey says connectivity was only the first layer, and it turned out to be the easy part.

Agents can connect together without thinking together today.

— Vijoy Pandey, Big Technology
Key Insight
The plumbing problem — discovering agents, giving them scoped identity, letting them message and be observed — is largely solved by open protocols. What is not solved is shared cognition, so “connected” is a milestone, not the destination.
02The Measurement

Every Multi-Agent System Failed

When Outshift measured production systems, every single one hit at least one cognitive failure; one failure left 98% accuracy, two dropped it to about 80%, and three collapsed it to 16% — no better than picking a random number.

100% of multi-agent systems had at least one cognitive failure.

— Vijoy Pandey, Big Technology
Key Insight
The collapse is non-linear: failures do not add, they multiply. A system can look 98% reliable after one slip and be near-worthless after three, which is why the silent accumulation of small errors — not any single dramatic one — is the real risk.
03The Mechanism

Bad Assumptions Spread Like Wildfire

One agent introduces a bad assumption; the next agent treats it as context, the one after that as evidence, and the error compounds down the chain until even the verifier inherits the same false premise.

What's happening is that bad assumptions are spreading like wildfire across these systems.

— Vijoy Pandey, Big Technology
Key Insight
Because every hop raises the apparent confidence, the system grows more internally consistent as it grows more wrong. Consistency is not correctness, and a verifier sitting downstream of the error cannot catch what it was handed as truth.
04The Blind Spot

Dashboards Green, Cognition Red

Messages flow perfectly and every observability dashboard reads green, yet the agents can be marching toward a completely wrong shared understanding — the health signals watch traffic, not meaning.

from a dashboard perspective, your dashboards are all green, but your shared understanding and your cognitive understanding is completely red.

— Vijoy Pandey, Big Technology
Key Insight
Classic observability measures whether a message was delivered, not whether two agents understood it the same way. The louder the green dashboards, the more dangerous the blind spot — operators end up trusting exactly the signal that is blind to the failure.
05Single vs. Multi

The Answer Is Not Fewer Agents

Dropping to a single agent to dodge the failures misses the point — a real $2M deal already flows through sales, legal, finance, security and coding agents from different vendors, so Pandey argues the fix is better machinery, not fewer agents.

The answer is not fewer agents. The answer is better machinery, better infrastructure that allows agents, just like human beings, to then collaborate as subject matter experts to solve for a business outcome.

— Vijoy Pandey, Big Technology
Key Insight
His deeper design rule comes from hyperscaler resilience: build your checker, creator and security agents on different model stacks so they do not fail the same way at the same time. In a multi-agent system, homogeneity is the hidden single point of failure.
06The Fix

The Cognitive Layer Agents Are Missing

Language triggered humanity's cognitive revolution — shared intent, then shared context, then shared reasoning; agents still lack that layer, so Outshift watches for semantic drift and, like lane-keeping assist, nudges agents back toward the global intent before they veer off.

about 70,000 years ago, language was invented. And language caused what is now known as the cognitive revolution, where humans moved from isolated individual intelligence to collective intelligence.

— Vijoy Pandey, Big Technology
Key Insight
The design choice is restraint. The layer does not seize control like a full self-driving stack; it intervenes only on measured drift. That keeps agents autonomous while making the failure mode — silent divergence of meaning — observable and correctable where it actually happens.
07The Bigger Shift

Determinism Is Dead

For 70 years computing obeyed one rule — same input, same output; agentic workloads broke it, because asking for 'the best drug' is an optimization, not a lookup, so both the question and the answer are now probabilistic.

That fundamental computing rule of determinism is now shattered. That axiom is dead.

— Vijoy Pandey, Big Technology
Key Insight
The unfinished half of the shift is hardware. Today's probabilistic workloads run on deterministic CPUs and GPUs that only approximate randomness with matrix math and pseudo-random numbers. Pandey's bet is that the substrate itself changes next — which is where quantum enters.
08The Clock

Q-Day Is 2029

Pandey dates 'Q-Day' — when quantum breaks today's encryption — to 2029, because the same three scaling axes cut the algorithm's requirement from a million qubits to 10,000 while hardware climbed to 1,000, and a quantum network can link the rest.

I'm convinced that Q-Day, which is the day that encryption is broken, is 2029. So, it's 2 years away.

— Vijoy Pandey, Big Technology
Key Insight
The uncomfortable implication is that the deadline has already arrived for anything that must stay secret for years. 'Harvest now, decrypt later' means an adversary needs only patience and storage today — and because NIST's post-quantum standards already exist, the defensive clock started before the offensive one.