The Decision I Made Twice: Adopting a Framework, and Then Not

On Technology, AI & Leadership

Every technical leader gets asked some version of this: why aren't we using the standard thing? It is a good question and it deserves a better answer than preference.

The specific version I got recently was about LangChain. What follows is the reasoning, because the reasoning generalizes and the tool does not.

The question behind the question

When someone asks why you built rather than adopted, they are usually asking one of three different things, and answering the wrong one wastes everybody's time.

Are you reinventing something solved? Fair. Sometimes yes.

Are you creating a dependency on yourself? Also fair, and the sharper version.

Do you actually understand the tradeoff, or did you default? That is the one worth answering, because a default in either direction is the real failure. Building everything and adopting everything are both ways of not deciding.

What made this one concrete

I run a verification pipeline where one model drafts a finding and two models from different vendors independently assess it. A finding is published only if the first supports it and the second does not contradict it. Three vendors, deliberately, so no single company's bad day passes unchallenged.

It had been running on two vendors for weeks.

One of the models had quietly changed behavior — given a small budget it now spends the entire allowance on internal reasoning and returns nothing. My fallback logic did exactly what it was designed to do and handed the work to the backup. No error was raised. Response times were normal. The output was plausible.

The system was architecturally broken while every dashboard said it was healthy.

I caught it because I own that code path and log which model actually responded. Behind a framework's abstraction, the same symptom reads as a generic provider hiccup, and diagnosing it means reading someone else's source.

The judgment, stated so it transfers

Adopt a framework when it removes work you understand. Refuse it when it hides work you need to see.

Undifferentiated work belongs to vendors who will always be better at it than I will: payments, identity, deliverability, edge compute. I do not run my own anything in those categories.

But the verification layer is the product. An abstraction there would have bought me convenience and cost me the visibility that caught a silent failure in the one mechanism customers are actually paying for.

The part I will not pretend away

There is a genuine gap on my side. I have no systematic tracing across model calls — no standing answer to how often does a model fall through to a backup, or what did the model actually see on the run that produced a bad result. I found this failure by reasoning about a symptom, not by looking at a dashboard, which means a subtler one would still be running.

That is a real deficiency, and the honest response is to close it rather than argue about it. The answer happens to be a managed gateway rather than a framework, but the point is that I would rather name the gap than defend the position.

Why this matters beyond one library

Most technology decisions that go wrong do not go wrong at the moment of choosing. They go wrong later, when something breaks in the part of the system nobody can see into anymore, and the team discovers that convenience was purchased with visibility.

The discipline is not build or buy. It is knowing, before you decide, which parts of your system you must be able to see into — and refusing to trade those away, however good the sales pitch.

The silent demotion, in twenty seconds.

This piece first appeared on jessemyers.ai.