Data platform build

A data platform your team can run without me.

The platform is not the point. Being able to answer a new question next quarter without a project is the point. Everything below is built backwards from that.

When this is the right call

These are the sentences that usually precede the enquiry:

  • Analysis still runs off spreadsheet exports and one person’s laptop.
  • Each new data source is a bespoke integration with its own failure mode.
  • There is no single place where a number is defined once.
  • The team is scared to change anything because nothing is tested.

What is different afterwards

  • One warehouse holding modelled, documented, tested data.
  • Ingestion that recovers from failure on its own and tells you when it cannot.
  • A transformation layer under version control with tests that run on every change.
  • A team that can add a source or a metric without calling an outside consultant.

What you receive

  • Warehouse design: layering, naming, access model and retention.
  • Ingestion for the agreed sources, with monitoring and alerting.
  • Transformation project in dbt, version controlled, tested and documented.
  • Orchestration with dependency-aware scheduling and failure notification.
  • Runbook and handover sessions covering operation, extension and recovery.

How it runs

  1. Decide what it is forThe decisions the platform must support, named explicitly. This is what stops the build drifting into generic infrastructure.
  2. DesignSources, layers, models, orchestration and access, written down and agreed before anything is built.
  3. Build the first vertical sliceOne source, end to end, into one working report. It proves the design and gives the business something usable early.
  4. WidenRemaining sources and models, in the order the business needs them rather than the order that is technically tidiest.
  5. Hand overDocumentation, runbooks and working sessions until your team is comfortable operating it unaided.

Who this suits

  • Companies moving off spreadsheets and production database replicas for the first time.
  • Teams replacing a platform that grew organically and now blocks them.
  • Businesses with a small or no in-house data team who need something maintainable.

Questions

Before you get in touch

Which warehouse should we use?

Usually the one that fits your existing cloud, team skills and data volume — most commonly BigQuery or Snowflake, and Postgres where volumes are modest. The choice is made against your constraints, and the reasoning is written down so a future team can revisit it.

Do you build or buy the ingestion layer?

Buy wherever a managed connector exists and is cheap enough, build where the source is unusual or the licence cost outgrows the effort. Custom connectors are a maintenance liability, so the default is to avoid them.

What happens when the project ends?

You hold every credential, every repository and every document, and your team has operated the platform under supervision before I leave. Ongoing support is available but it is never a dependency created on purpose.

Can you work alongside our existing engineers?

Yes, and that is usually the better outcome. Building with the team that inherits the platform produces something they will maintain rather than something they will quietly replace.

Enquire about data platform build

Describe the situation in a few lines. You will get a straight answer about fit, timing and rough shape of the work.

The current situation and what "solved" looks like is enough. Detail can wait for the call.

Tell me what is broken.

A short call is usually enough to tell whether this is a two-week fix or a two-month one — and whether I am the right person for it.