Third and final part of a series on the scaling tensions inside modern data platforms. Part 1, “Scale the Team or Scale the System?”, is here. Part 2, “Why Your Data Mesh Stalled at the First Domain”, is here.

Part 2 closed on a rule: decentralize ownership only as fast as the platform can carry it. That rule is useless without the second half, which is the question every CTO asks next — what does the platform actually have to do?

The short answer is the one that makes platform teams uncomfortable: the platform is a product, its customer is the domain team, and governance is one of its features. Not a policy. Not a board. A feature.

Your customer is the domain team

A central data team that becomes a platform team usually keeps the old posture: it reviews, it approves, it is consulted. That posture is why the bottleneck comes back wearing a different hat. If a domain team has to ask you for something, you are still in the critical path — you have just moved from doing the work to authorizing it, which is slower and less useful.

The test is simple and it is not about architecture. Can a domain team take a dataset from “we have this” to “it is published, governed, discoverable, and someone else is querying it” without a conversation with your team? If the answer is no, you don’t have a platform. You have a service desk with an API.

Which means the platform gets the things products get: a roadmap, adoption metrics, SLAs, and a real answer to “who asked for this?” A platform team that can’t name its top three consumer complaints is not running a product.

What self-serve actually has to include

Part 2 named the failure mode — eight domains quietly rebuilding the same plumbing eight times. The way you prevent it is not a document telling them not to. It’s making the paved path cheaper than the alternative. Concretely, the platform has to own five surfaces end to end, or domains will rebuild the missing ones:

Publishing. A domain can register a dataset as a data product — schema, owner, description, access policy — through one path, without provisioning infrastructure. This is the one that determines volume. In the transformation I led, 135+ data products landed in the first six months. That number is not a sign of unusual enthusiasm; it is arithmetic. Nobody publishes 135 data products through a ticket queue.

Access and security. Grants are requested and granted against the data product, not against buckets, tables, or accounts. The domain decides who may consume; the platform makes that decision enforceable and auditable without the domain writing policy by hand. A federated catalog — in our case SageMaker Unified Studio with Lake Formation behind it — is what makes discovery and access the same motion rather than two disconnected ones.

Quality checks. Tests ship with the data product and run on every load, declared by the domain in the same place the product is defined. If quality is a separate system that someone has to remember to wire up, it will be wired up for the products that were already fine.

Observability. Freshness, volume, schema drift, and lineage, available to the consumer — not just to the producer. Consumers who can see why yesterday’s number moved stop filing tickets about it. Consumers who can’t will route around your platform and copy the data.

Cost attribution. Every data product’s compute and storage lands on the owning domain’s line. This is the least glamorous item and the one that changes behavior fastest. A domain that pays for its own full-table refresh every fifteen minutes will fix it without being asked.

Anything on that list you don’t provide, a domain will build for itself — worse, in isolation, and eight times over.

Contracts are the unit of governance

In Part 2 I defined a data product by four things: an owner you can name, a documented contract, a quality SLA, and a consumer who would complain. The word doing the load-bearing work there is documented, and it’s the weakest of the four, because a contract in a wiki is a description of intent, not a constraint.

Make it machine-checkable and it becomes governance. The contract declares the schema, the semantics of each field, freshness and availability targets, PII classification, and retention. It lives with the code that produces the data, it is versioned, and the platform validates every publish against it.

The interesting design question is what happens on breach, and the honest answer is: it depends on which side breaks.

  • A producer breaks the schema. The publish fails. The last good version stays live. Consumers see stale data with a clear reason, which is recoverable — instead of fresh data in a shape their queries no longer match, which is not.
  • A producer misses the freshness SLA. The consumer is notified through the platform, automatically, not by the producer remembering to send a message. Escalation goes to the owner named in the contract.
  • A consumer depends on something the contract never promised — a column that was always incidental, a load-order coincidence. The contract is what makes this the consumer’s problem rather than a permanent obligation on the producer. This is the clause that keeps a data product changeable, and it is the one people skip.

Breaking changes then work the way they do in any API: a new version, both live, a deprecation window, and a list of consumers you can actually enumerate because access runs through the platform. None of this requires a meeting.

Federated governance without a governance board

“Centralized governance, decentralized ownership” is easy to say and the whole difficulty is in the split. My version, and the part I’d defend hardest:

Central owns the invariants — the rules that are not negotiable per domain, because a single domain opting out damages everyone. Identity and authentication. PII classification and handling. Retention and deletion. Encryption and network boundaries. Lineage capture. Audit logging. Cost attribution.

Domains own the semantics — the decisions where domain knowledge beats central consistency. What an entity means. Which fields exist and what they represent. Freshness and availability targets, and what the product is worth committing to. Who gets access. When to break a version.

Read those two lists again and notice the pattern: the central list is entirely things a platform can enforce in code, and the domain list is entirely things it can’t. That is not a coincidence — it’s the selection criterion. If an invariant can only be upheld by a human checking, it isn’t an invariant; it’s a preference with a review meeting attached, and it will be routed around under deadline.

So there is no governance board approving data products, because there is nothing for it to approve. Non-compliant products don’t reach a review — they don’t publish. What is left for humans is the genuinely hard, genuinely rare work: adding a new invariant, arbitrating a semantic conflict between two domains, and deciding when an invariant is wrong. That is a handful of decisions a quarter, not a standing weekly.

A governance model a domain team has to attend gets routed around. One it inherits by using the platform does not.

The platform team is the next bottleneck

The failure mode of this whole design is not technical, and it’s worth naming plainly because it’s the one I’d watch for. It’s a platform team that reads “we own the invariants” as “we are the gatekeepers.” The symptoms are recognizable: a request queue for anything not yet paved, exceptions granted case by case, a roadmap made of internal preferences, and — the tell — domain teams building their own path anyway and not mentioning it.

The instrumentation for this is not complicated:

  • Time from “we have a dataset” to “it is a published, governed product.” If it is measured in weeks, you are the bottleneck. In the transformation I led, end-to-end delivery went from about ten weeks to four days — a 94% reduction — and effectively all of that came out of removing the central team from the path, not from making any single step faster.
  • Share of data products published without platform-team involvement. If it isn’t close to all of them, self-serve is aspirational.
  • Exceptions outstanding. A growing count means the paved path doesn’t cover real needs, and each one is a permanent manual obligation.
  • Where your own engineers’ time goes. Ours came out the other side as 4 engineers redirected off data operations and onto revenue work. If platform headcount instead scales with the number of domains, you have rebuilt the centralized team with extra steps.

The platform team’s job is to make itself unnecessary for the common case, and to be genuinely good at the rare one. That is a product discipline, not an operational one.

The through-line

Three posts, one argument.

Part 1: don’t scale the team, scale the system — headcount doesn’t move a bottleneck that is structural. Part 2: you can’t declare ownership faster than the platform can afford it, or it re-centralizes within eighteen months. Part 3: the thing that affords it is a platform treated as a product, with governance compiled into it rather than convened around it.

Which puts the sequencing where I think it belongs. Most organizations start with the org chart — draw the domains, assign the owners, then work out the tooling. It is the wrong order, and it is the order almost everyone picks, because the org chart is the artifact that’s easy to change on a Tuesday.

Build the platform that makes ownership possible, and the domains form around what they can now actually do. The system you scale is the platform. The org chart follows it, not the other way round.

Part of a series drawn from hands-on experience building and scaling data platforms. Details are anonymized; no client or employer is identified.