In-house · consumer marketplace platform

The quota limit was the symptom. The absence of attribution was the problem.

Hitting a warehouse query quota threatened reporting. Mitigating it was an afternoon; the durable fix was making it possible to attribute cost to the thing that caused it.

Per-application service accountsAttribution
PreventedRecurrence
Cost monitoring documented for the whole teamVisibility

The situation

The warehouse hit a query quota limit, putting reporting at risk. The immediate incident was straightforward to mitigate. The underlying problem was that no one could say which application or workload had consumed the quota, which meant the same incident would recur.

What I did

  1. Handled the immediate mitigation to restore reporting.
  2. Split warehouse access into per-application service accounts, so every query could be attributed to the system that ran it.
  3. Documented cost monitoring so the whole team, not just one person, could see consumption and act on it.
  4. Converted a one-off incident response into standing governance rather than closing the ticket.

What it means for you

Warehouse cost is unmanageable until it is attributable. Splitting access by application is cheap to do early and expensive to reconstruct later, and it turns cost from a monthly finance conversation into something the team that creates it can see.

Context

These are from eight years owning the data warehouse and reporting function of a consumer marketplace platform, in-house rather than as an outside consultant. The employer and the internal system names are withheld; the numbers are the real ones.

Tell me what is broken.

A short call is usually enough to tell whether this is a two-week fix or a two-month one — and whether I am the right person for it.