FDE PulseFDE jobs open 441New in 7 days 29Companies hiring 47Remote-friendly 24%Median US pay $216kTop hirer Databricks 125
VI

The newspaper of the Forward Deployed Engineer

Tools

Cancelling a Redshift Serverless query still costs money: FDEs must set guardrails in advance

Snowflake, BigQuery and Redshift each charge in a different way. BigQuery can reject a query that exceeds its limit before it costs anything. The other two need limits put in place ahead of time.

In brief

  • Snowflake bills compute in credits, BigQuery on-demand bills by bytes processed, and Redshift Serverless bills RPU-hours per second with a 60-second minimum.
  • BigQuery can reject a query that exceeds its limit before it runs, at no charge. Redshift Serverless still bills for the time a cancelled query has already run.
  • Only ACCOUNTADMIN can create a Snowflake resource monitor, so FDEs must request access or ask the client's admin from the start.
ShareLinkedInFacebookX
GraphicThree warehouses, three billing models, three guardrails
Billing unitCost guardrail
SnowflakeCompute billed in creditsResource monitor caps credits; only ACCOUNTADMIN can create one
BigQueryOn-demand by bytes processed, or capacity by slotsDry run, maximum bytes billed, daily quota
Redshift ServerlessRPU-hours metered per second, 60-second minimumUsage limit: log, SNS alert or turn off queries

BigQuery can block an over-limit query before it is charged; Snowflake and Redshift Serverless need limits set in advance to stop usage.

Graphic: FDE Times

AWS documentation for Redshift Serverless is explicit: if you run a query and cancel it partway through, you are still charged for the time it ran.

The Cancel button does not remove that time from the bill. At a client site, that can be the difference between a routine data exploration session and an awkward call with the finance team.

On day one at a client, an FDE is usually given read access to the data warehouse and starts exploring: counting tables, checking distributions, trying joins across a few large tables. Snowflake, BigQuery and Redshift charge for this work in different ways. To set the right guardrail, you need to know what the platform is counting.

Each platform counts something different

Snowflake splits costs into three parts: compute, storage and data transfer. Compute is billed in credits, consumed whenever a warehouse is running. Storage is billed at a flat rate per terabyte per month.

Snowflake does not charge for bringing data into an account, but it does charge for data going out (egress). FDEs should remember that pushing a large export to another system is also a cost.

BigQuery has two models. Under on-demand pricing, the client pays for the bytes processed by each query. Under capacity pricing, the client provisions processing capacity as slots for each workload, with optional autoscaling.

So the first question on any BigQuery project is which model the client uses. It determines whether your queries cost money by the volume they scan, or take a share of capacity that other teams are also using.

With Redshift, billing depends on the deployment type. Provisioned clusters are billed by the hour. Redshift Serverless is billed in RPU-hours, metered per second, with a 60-second minimum. If a query reads external data through Redshift Spectrum, the client also pays for the bytes Spectrum scans.

BigQuery rejects queries before they cost money

Of the three platforms, BigQuery has the easiest guardrail to reason about, because it acts before the query runs. Google recommends a dry run (or preview) to estimate cost first. Maximum bytes billed is a hard block: if the estimated bytes exceed the limit, the query fails and is not charged.

Consider an example. You set maximum bytes billed to 100 GB for exploratory queries. A colleague writes a query joining two event tables, and the dry run estimates 1.5 TB. The query is rejected, the bill does not change, and you get the chance to add a date filter and run it again.

At project level, Google recommends setting a custom daily query quota to cap the data processed each day. That gives you two layers: maximum bytes billed stops individual queries, and the daily quota caps total usage for the day, even if someone runs a loop that fires a query hundreds of times.

Note that both layers work on bytes, so they affect the bill directly mainly when the client is on on-demand pricing. If the client uses capacity pricing, ask the admin which slots your queries run on, so you do not take capacity from other workloads.

Snowflake can stop spending, but only the right person can set it up

Snowflake’s guardrail is the resource monitor, which caps the credits warehouses can consume. At its strictest, a monitor can immediately suspend every standard warehouse assigned to it and cancel statements that are running.

Only ACCOUNTADMIN can create a resource monitor, so an FDE with read-only access cannot build this guardrail alone. The task for week one is to book time with the client’s admin and agree a credit limit for the project’s warehouse, along with how the system should respond when it is reached.

Redshift: cancelling a query does not refund the time already run

On Redshift Serverless, cancelling a query only stops it running further. AWS recommends setting a maximum number of RPU-hours as a cost guardrail. Usage limits have three actions: log, send an alert via SNS, or turn off user queries and send a notification.

A sensible order is to enable the SNS alert first, then set the query shut-off slightly higher. The alert gives you time to respond; the shut-off keeps the bill within the agreed threshold.

Three common traps in week one

The first trap is sharing a warehouse with the client’s production pipelines. On Snowflake, a hard stop also cuts off jobs that are mid-run, so a monitor set too tight can break other people’s work. The safer approach is to ask for a dedicated warehouse for your work and assign the monitor to that.

The second trap is sending alerts to a channel nobody reads. An SNS alert is only useful if it arrives somewhere that both you and the client’s owner check every day.

The third trap is treating the Cancel button as a safety net. On Redshift Serverless, time already run is still billed, so you are only truly protected once a limit is in place.

What to learn first

If you have only a week to prepare, start with BigQuery. Dry runs and maximum bytes billed show you the estimated cost and how the guardrail works straight away.

Then read the documentation on Snowflake resource monitors and Redshift Serverless usage limits, to learn where limits are set and how each system responds when they are reached.

When a job description lists “Snowflake/BigQuery/Redshift”, do not just write “proficient in SQL” on your CV. A line such as “set up maximum bytes billed and daily quotas on a client’s BigQuery project before data exploration” shows an interviewer you understand that at a client site, cost is the FDE’s responsibility.

Clients may forget a polished dashboard, but they rarely forget a month when the bill spiked because of an outsider’s queries.

Was this article useful?

Use with your AI assistantAsk Claude ↗Ask ChatGPT ↗
7 sources
Read next on the roadmap · Stage 5: DeploymentOllama and LM Studio: demoing open-weight models on a laptop when the client bans data egressWhen a client's legal team rules out every cloud API, the demo depends on a single laptop. Whether it works comes down to how you set that laptop up before you reach the client's office.