A workflow that handles forty requests a day and one that handles four thousand are not the same system with a bigger number attached. Rate limits, token costs and queue depth all move from footnotes to the main problem.
What to Measure Before You Change Anything
Baseline first. Without a measurement taken before the change, any improvement claim afterwards is a story rather than a result.
Two weeks of honest observation beats a quarter of instrumented guesswork, provided the two weeks are representative rather than cherry-picked.
| Metric | Why it matters |
|---|---|
| Cycle time end to end | The number the customer actually experiences |
| Touches per request | Proxy for how much human attention each unit consumes |
| Exception rate | Tells you how much of the process resists automation |
The Part That Should Stay Human
Judgment under ambiguity stays human. If a step requires weighing two things that cannot both be satisfied, it is a decision and not a task.
Automating a decision does not remove the decision. It moves it earlier, into whoever wrote the rules, and hides it from the people living with the consequences.
- Anything where being wrong is expensive and hard to reverse
- Anything a customer would be upset to learn was handled by a machine
- Anything whose correct answer changes with context the system cannot see
Building for the Failure Case
Production automation is mostly error handling. The happy path is the small part, and it is the part that gets all the attention during the build.
Every external call can fail, time out, or return something unexpected. A workflow that assumes otherwise will work until precisely the moment it matters.
| Failure | Minimum acceptable handling |
|---|---|
| API timeout | Retry with backoff, then alert a human |
| Unexpected payload shape | Fail loudly, never silently continue |
| Credential expiry | Alert before expiry, not after the break |
Ownership Beats Documentation
Documentation rots. A named owner who feels responsible when something breaks does not, at least not as fast.
The practical test is simple: if this workflow failed tonight, who would notice, and would they know what to do? If either answer is unclear, that is the actual risk.
- One named owner per workflow, not a team
- A one-line description of purpose, written at build time
- An alert that reaches a person, not just a log file
Cost, Honestly Accounted
The build cost is visible and the maintenance cost is not, which is why automation is consistently underestimated by the people commissioning it.
Assume ongoing maintenance runs a meaningful fraction of the original build effort every year, and budget for it explicitly rather than pretending it is zero.
Conclusion
The tooling is rarely the hard part. Scope it narrowly, measure the baseline first, name an owner, and build for the failure case rather than the demo.
Frequently Asked Questions
How long before this pays for itself?
Most well-scoped automations pay back within a few months, but only when the baseline was measured beforehand. Without a baseline you cannot honestly answer this question at all.
Do we need a dedicated person to run this?
Not dedicated, but named. The failure mode is shared ownership, where everyone assumes someone else is watching.
What if the underlying model or API changes?
Assume it will. Anything that depends on an exact output format should validate that format explicitly rather than trusting it.