Technology
Data & AnalyticsBigQuery, Snowflake, dbt
The data stack we reach for, the trade-offs we name, and when we use something else — BigQuery or Snowflake for the warehouse, dbt for the transformation, the right BI tool for the audience, with the pipeline as the architecture and the dashboard as the surface.
What we use it for
BigQuery or Snowflake for the warehouse — the right answer for the data volume, the query pattern and the team. dbt for the transformation — the right answer for the SQL-based transformation, the testing, the documentation. Airflow for the orchestration — the right answer for the pipeline scheduling and the dependencies.
We extend the stack with the BI tool the audience needs — Looker, Metabase, Superset, Power BI. The choice is made against the data, the team and the audience.
When we choose it over the alternative
BigQuery is right when the workload is on GCP, the data is structured, and the team is comfortable with SQL. Snowflake is right when the workload is multi-cloud, the data is structured, and the team needs the separation of compute and storage. dbt is right when the transformation is SQL-based, the team is comfortable with the framework, and the documentation is a leverage point.
The right answer depends on the data, the team and the audience. We will say so on the call, with a written rationale for the choice and the trade-offs named.
When we do not choose it
We do not choose BigQuery when the data is unstructured and the workload is search. We do not choose Snowflake when the cost is prohibitive and a smaller warehouse is the right answer. We do not choose dbt when the transformation is not SQL-based and the framework does not fit.
Frequently asked
The questions the team asks
- BigQuery or Snowflake?
- It depends on the workload, the team and the cost. BigQuery is the right default for GCP-native workloads, with the SQL-based transformation. Snowflake is the right answer for multi-cloud workloads, with the separation of compute and storage. We will say so on the call.
- How do you handle the pipeline?
- A written pipeline with named owners, retries, idempotency and monitoring. The pipeline is observable, recoverable and tested. The pipeline is part of the architecture, not a script, and the data is the deliverable.
- How do you handle the dashboard?
- The dashboard is the surface, the data is the product. The metrics are on a dashboard, the SLOs are written, and the alerts fire on the SLO budget. The dashboard is what the team can act on, not a chart that nobody opens.
Have a project in mind?Let’s scope it together.
Tell us the outcome you need. You get an approach, a rough timeline and next steps within one business day.