I used to hate bureaucracy. Maybe that's partly why I worked for startups earlier. And I hated all that ceremony nonsense. Scrum, meetings, whatever. I just didn’t think they helped much. I always thought about hiring the best engineers and trusting them to get things done. I still believe in hiring the best but I have been questioning my earlier assumption for a while now.

The fundamental flaw in my earlier thinking was assuming that large organizations consist of a particular kind of engineer who is happy to fill gaps, intrinsically motivated, capable of managing themselves, comfortable with ambiguity and technically very competent. I still think these are good qualities to have but not everyone has all of them at the same time to the degree needed to move the organization towards its goals.

My earlier way of thinking still works in small teams with exceptional engineers, who distil information, understand the lay of the land and eventually deliver what is needed with a good level of quality. They are autonomous enough so they don’t need a lot of ceremony because they know what needs to be done and when, so they decide on their own.

I used to describe my leadership style as "fire and forget". Give people direction, trust them to figure things out, and step away. I still like that approach, but I think I was expecting too many people to work the way I preferred to work myself.

Bureaucracy, in its natural habitatBureaucracy, in its natural habitat

Organization with Bards

I have always liked small teams. Even if the team gets larger, I tend to create subteams. That’s fine and functions well with enough bureaucracy. When some standards aren’t there, then it becomes harder to get predictable and consistent work across the organization. My earlier self required bards of different kinds who sang different songs to make sure we do things right.

Some bards are quality gatekeepers. Some resolve context gaps between multiple teams. Some bards remember everyone’s contributions and propose fair promotion candidates. Some bards insist on making sure new joiners or existing people have enough resources to get started, find information and onboard quickly. The problem is that you may not have these bards, but also not every human has the same degree of competence, motivation, judgement, and willingness to take responsibility.

You can still have those bards but the org needs to be able to function without them or when they leave. So, the organization's capability can’t depend inadvertently on the bards who sing songs to make everyone better. There have to be standards defined for everyone to do the right thing even when there’s no bard.

Even if I somehow managed to hire only exceptional engineers, I would still have a problem. Two great engineers can make perfectly reasonable decisions for their own teams while creating problems for each other. They have different information and priorities. Hiring better engineers doesn't necessarily make those differences disappear.

What kinda bothers me looking back is how easily I could mistake the contributions of these individuals for the capability of the organization itself. Quality looks good because someone insists on reviewing everything. Cross-team work gets done because someone knows everyone, remembers the invariants. Then one of them leaves, and we discover how much of the organization was living in their head.

Why Bureaucracy Exists

It sounds unreal to me that I'm now about to advocate for bureaucracy. I said in the past that enough bureaucracy got things right, but what’s enough? In order to answer that question, let’s dig into a bit of political philosophy.

Max Weber explored different forms of authority, including patrimonial and legal-rational authority. In the past, an official's power could depend on inheritance, personal relationships or loyalty to a ruler. Bureaucracy brought an interesting dimension to this through impersonality and abstraction. You no longer conformed to a person in charge but started following rules established by the institution. Authority became attached to an office, with responsibilities and limits that remained even when the person occupying it changed. So, impersonalization provided an abstract sense of power through these institutions. That's sort of what we gained in the corporate world, not persons but positions. This creates continuity despite different people.

I find that quite interesting. A manager can leave, but their successor inherits the same responsibilities. As an employee, I shouldn't have to renegotiate expectations every time my manager changes. I wanted my bards to do the right thing naturally, but I hadn't thought enough about what would preserve those behaviours after they left.

That's why you can’t scale an organization very far without impersonality. That means at some point, you need to come up with an abstract sense of rules and authority. So, everyone conforms to a defined rulebook because individuals don’t always have the same priorities.

Without formal rules, people eventually figure out whom to ask. It might be the staff engineer everyone respects or a manager who knows the right people. Then, work starts moving through personal relationships and influence. That might work quite well for the people who know everyone, but what about those who don't?

Constraining power is another angle. You again want decisions to be challenged against the right criteria. If someone is underperforming, you want the manager to follow a proper performance process, including a PIP when appropriate, because their judgement alone might not be fair. So, you want the manager to do the right thing. If you let them fire someone right away without a consistent process, that would mean the exercise of power could become arbitrary.

I've seen a similar problem with incident reviews. We can advocate for blameless postmortems all we want, but I've seen senior leaders use those reviews to assign blame. A written policy gives engineers something to stand on when that happens. Of course, it only works if the policy applies to the directors and VPs as well. Otherwise, we're asking people to trust the same individuals the policy is supposed to constrain.

Another reason is predictability. When roles are clear and people follow through, then it's easier to know what to do. Say, if you have a concept of DRI and they need to make sure they groom their backlog, log tickets for upcoming work, talk with other DRIs or external people to move their work forward. And if the role itself is clear, changing the DRI wouldn’t really change as much.

The funny thing is, I've been doing this myself for years. I create subteams, assign DRIs, ask people to maintain project pages and write down how they should coordinate. I never called that bureaucracy. I just wanted my teams to work without me having to get involved in every detail.

Earlier, I thought these rules could be easily figured out or people would do the rational thing, but it really doesn't happen as often as I wanted it to. If rules or roles aren’t written, people won’t necessarily follow unwritten ways of doing things. Worse, they will adopt their own ways of working, sometimes without a good standard.

Suppose you need to change the daily processing job to hourly. Your team owns that, but if there isn't an architectural discussion requiring the DRI to work with upstream and downstream dependencies, then it can go sideways. The upstream system might still produce data daily, or downstream consumers might have capacity assumptions based on the existing schedule. The problem is that they sometimes skip it. So, if you want people to act autonomously, you want rules and roles that make sense for them to move the needle as opposed to having to ask whether it's okay to change to hourly. The absence of rules makes it even harder to act autonomously because there’s no prescribed way to do so. That’s one reason platform engineering uses the concept of paved roads.

I used to think a rule would slow me down. Now I see that knowing who owns what, and when other teams need to be involved, can save everyone from chasing people for permission. Perhaps some bureaucracy is necessary to give people the autonomy I always wanted them to have.

When Bureaucracy Becomes the Problem

I think most of the issues stem from what happens when we stop trusting people to make decisions within their own roles. If you have a manager and they can’t decide for their own teams within defined rules, then this becomes an issue. For instance, if a team wants to deploy a change and it needs to be rubber-stamped by higher-ups who don't understand the change itself, it doesn’t serve anyone other than the procedure.

The problem with these kinds of issues is that bureaucracy can hide an incompetent manager or IC. A capable manager or IC might log a decision to discuss alternatives and see what’s the best way to go about the deployment. If you can’t trust people to do the job, then you move the permissions higher up the chain, which not only creates bad bureaucracy but also inefficiency. Ideally, you should have good rules of thumb in place so that escalations are not business as usual.

Of course, some restrictions are justified. I wouldn't want even my best engineer bypassing production access controls simply because I trust them. The possible consequences are too serious. I think the question is whether the rule prevents a real problem or simply moves responsibility to someone else.

There's another danger I hadn't considered enough. The standards created by yesterday's bards can survive long after the reasons behind them disappear. Robert K. Merton called this goal displacement, where following the procedure becomes more important than what the procedure was supposed to achieve. I've seen how a rule can outlive the incident that originally justified it. Sometimes everyone keeps following it because nobody knows whether it's safe to stop. My younger self would have hated that, and he'd have a point.

Designing Bureaucracy for Ordinary Humans

Now, organizations need to challenge, reorganize themselves, and update their rulebook. Any process needs to have an identifiable benefit and an accountable owner. Sometimes the benefit is preventing something that rarely happens, but I still want someone to explain why the process exists and when we should reconsider it. If the rules just exist for the sake of existing and nobody can explain what they're protecting, then those rules shouldn’t be part of the bureaucracy. When that happens, it becomes coercive. People obey rules not because of their value but because they have to.

Paul Adler and Bryan Borys described the difference between enabling and coercive bureaucracy. I wish I'd made that distinction earlier.

NeedWorth havingWorth questioning
Code qualityAutomated checks, ownership, shared standardsEvery change requires a senior architect's approval
ArchitectureClear ownership, recorded rationale, exception mechanismsCommittees that enforce standards without understanding them
Career progressionTransparent levels, evidence, independent calibrationOpaque committees without meaningful feedback
Production reliabilityRunbooks, incident authority, rehearsed recoveryMultiple managerial approvals during an outage
Knowledge sharingDocumented contracts and operational decisionsDocumentation maintained solely for compliance

This is one of the reasons why exceptional managers or engineers should have autonomy through these rules and why the rules should help them exercise judgement more quickly and efficiently, not the other way around. A good bureaucracy supports autonomy with well-defined ways of working.

I don't want to design everything around the least capable engineer, either. The bards still have a place. I want them to challenge standards, improve how we work and notice things the rest of us missed. But I'd rather have them teach what good looks like instead of taking that knowledge with them.

The Organization After You Leave

As an engineer, I valued freedom a lot and wanted to make decisions autonomously. Now, after years of managing, I acknowledge the limits of freedom when it comes to sustaining an organization across different engineers, managers and varying capabilities and personalities. A question comes to my mind worth answering. Suppose the best engineer and the best manager leave the organization tomorrow, what remains?

That’s what I find myself trying to answer. Do people raise quality issues? Do people contact neighboring teams? Do people promote the right behavior? If you can answer with a solid yes to all of them, then that’s a sustainable organization that has a way to survive key departures. I used to believe that great engineers needed less bureaucracy. I now think great organizations need to function even when great people aren't around.

P.S. The bards metaphor was inspired by Blind Guardian's The Bard's Song. I love how the whole audience sings along together. There's something wonderful about that.