Transit at the Depot: Pricing GPU Inference Bandwidth in 2026

November 17, 2026 · 10 MIN READ

At IDACORE Depots, inference bandwidth is IP transit from our own network, AS36824, billed on committed bandwidth at the 95th percentile. It isn't per-GB egress. Tiers run 100M at $170/month, 1G at $600, 10G at $2,000, and 100G at $11,050. Depot 01 in Northeast Washington is live with transit delivered on-site.

How Much Bandwidth Does GPU Inference Use?

It depends on what comes out of the GPU. Most bandwidth budgets go wrong on exactly this point.

Text inference is light per request. A 1,000-token completion is roughly 4KB of text. Streaming changes that. With per-token server-sent events wrapped in JSON, the same response can run 100KB or more on the wire. At 10 million requests a day, that's around 1TB/day, or about 30TB a month, from a chatbot that "barely uses any bandwidth."

Other workloads are heavier:

  • Image generation: a 1024×1024 PNG runs about 1.5MB. Two million images a day comes to roughly 90TB a month.
  • Video generation: clips run several megabytes per second of output, and bandwidth climbs with every product win.
  • Embeddings: a 1,536-dimension vector serialized as JSON decimals is often 15–20KB. Batch embedding jobs return millions of them.
  • GPU marketplace hosting: renters pull datasets in and push checkpoints out on their own schedule. You can't forecast it.

Then there's inbound. A 70B-parameter model at FP16 is about 140GB of weights. Over a 10G handoff, that pull takes around two minutes. Most hyperscalers don't charge for ingress, so nobody budgets for it. On transit, a two-minute spike is irrelevant, for reasons we'll cover below.

The pattern is simple. Inference traffic is outbound-heavy, it grows with usage, and it's spiky. Per-GB billing punishes all three of those traits.

Why Does Per-GB Cloud Egress Get Expensive So Fast?

Hyperscaler internet egress is metered by the gigabyte on a declining tier schedule. Typical published list rates start around $0.09/GB for the first 10TB and step down to roughly $0.05/GB past 150TB. Every response your model sends is a line item, so your network bill scales with your success.

Transit works differently. You buy a pipe sized to your traffic, and the price doesn't change with how many bytes cross it. Here's the comparison if you ran each tier flat-out for a 30-day month:

Sustained traffic Monthly transfer Typical hyperscaler egress (tiered list rates, approx.) IDACORE transit commit
100 Mbps ~32 TB ~$2,800 $170
1 Gbps ~324 TB ~$20,000 $600
10 Gbps ~3,240 TB ~$166,000 $2,000

Two caveats, because you'll check. First, this assumes a flat line at the commit rate, which is the worst case for egress. Real traffic has a daily shape, and the scenario below models that. Second, accounts pushing petabytes negotiate private egress pricing. Even with a steep discount, though, the gap at 1G and 10G is an order of magnitude, not a rounding error. At 100G, our commit is $11,050/month.

The bigger problem with egress is that you can't forecast it. A feature that doubles image resolution quadruples your network bill. Transit cost only moves when you deliberately change tiers.

How Does 95th Percentile Billing Treat Bursty Inference Traffic?

95th-percentile billing samples your port every five minutes for the month. That's 8,640 samples in a 30-day month. The samples are sorted, the top 5% are thrown out, and you're billed on the highest remaining value. The discarded 5% is 432 samples, or 36 hours of your busiest traffic that doesn't count.

You can run the math yourself from your own SNMP or flow data:

# samples: 5-minute average Mbps readings for the billing month
def billable_p95(samples):
    s = sorted(samples)
    cutoff = int(len(s) * 0.95) - 1   # 8,640 samples -> index 8207
    return s[cutoff]

# Standard practice measures inbound and outbound separately
# and bills the higher. Inference is outbound-heavy.

Here's what that means for inference specifically. A model weight pull, a launch-day spike, and a batch embedding job that saturates the link for an afternoon all fit inside the 36 hours that get discarded. You're billed on your sustained busy-hour rate, not your worst five minutes.

A worked example: image generation API

Take an API serving 2 million images a day at about 1.5MB each. That's 3TB/day, or 90TB/month.

  • Average outbound rate: about 280 Mbps
  • Busy-hour peak: roughly 2.5× average, around 700 Mbps
  • 95th percentile: around 600 Mbps, comfortably inside a 1G commit

At typical tiered hyperscaler list rates, 90TB of internet egress runs about $7,100 a month. On a 1G commit with IDACORE, it's $600. The team also gets headroom to roughly 2.5× its current traffic before the bill changes. At that point, moving to the next tier is a decision you make, not a surprise you find on an invoice.

What Does a Depot Deployment Cost With Transit Included?

Let's price a realistic inference deployment at Depot 01: three 8-GPU B200-class servers in a single 50kW air-cooled rack, with 40kW of capacity reserved.

Colocation: $175/kW/month NNN on a 36-month term, covering space, cooling, and facility infrastructure. That's 40 × $175 = $7,000/month.

Power at cost: we meter IT load, multiply by PUE, and bill at the utility rate with no markup. If the servers average 32kW across the month:

32 kW × 730 hours = 23,360 kWh IT load
23,360 kWh × 1.15 PUE = 26,864 kWh billed at the utility rate

The Depot standard targets a PUE of about 1.1–1.2. That comes from cross-flow free-air cooling with MERV 13 filtration and a direct evaporative wet wall for summer, with no compressors and no chillers. Less overhead means fewer billed kilowatt-hours.

Transit: a 1G commit at $600/month. Handoffs are 1G, 10G, or 100G. We run BGP with you over IPv4 and IPv6, and billing is burstable 95th percentile.

Everything else: you power-cycle your own gear from the IDACORE portal. When you need hands on hardware, remote hands is $150/hr during business hours in 15-minute increments, or $250/hr after hours with a 1-hour minimum. There's no minimum deployment size. Deposit and first month are due at signing.

Notice the proportions. Transit is a small line next to compute and power. That's how it should be. In the cloud, a heavy-output inference workload can see egress rival the GPU bill itself.

Who runs the network?

We do. AS36824 is our ASN, with Lumen and Cogent upstreams. The people building it founded and ran an ISP, ran their own BGP, and peered at the Seattle Internet Exchange. When a route goes sideways, you're talking to engineers who can read the routing table and change policy. You won't wait on a reseller to open a ticket with somebody else's NOC.

Why Run Inference Across Many Small Depots?

A Depot is a small compute site: 5MW or less, typically about 1MW of IT load, built from roughly 500kW Compute Modules. We site each one where the utility already has power. That's why Depot 01 can have racks powered in under 2 weeks from signing, while large campus projects wait years for substations.

The design philosophy is cattle, not pets. No single site is gold-plated. The base power offering is utility-fed, which keeps it at the lowest-cost tier, and you pick the resilience you need on top:

  • UPS plus generator at the site, available as an add-on
  • Battery energy storage (BESS), available as an add-on
  • Site-level redundancy by spreading your fleet across multiple Depots

Inference suits the third option. Stateless model servers behind anycast or health-checked DNS don't care which site answers. If one site drops, traffic shifts. That beats paying for a single facility engineered never to fail.

The next Depots are planned for the Treasure Valley and Eastern Oregon, connected back to our Boise interconnect hub over metro fiber. Each Depot gets transit from the same ASN, so your routing policy stays consistent across sites. Dark fiber between Depots is on our long-term roadmap. It doesn't exist today, and we won't pretend it does.

Small sites also make for better neighbors. A Depot doesn't need a new substation and doesn't come with tax-break asks. It brings modern infrastructure and local jobs to small towns without straining the grid. We also bring training, events, and community contributions to the places we build, and we already host IdahoMesh MeshCore repeaters. That's how a site gets welcomed instead of fought.

Frequently Asked Questions

How much does IP transit cost at an IDACORE Depot?
IDACORE prices IP transit on committed bandwidth with 95th-percentile billing: 100M at $170/month ($1.70/Mbps), 1G at $600/month ($0.60/Mbps), 10G at $2,000/month ($0.20/Mbps), and 100G at $11,050/month ($0.111/Mbps). Transit comes from our own ASN, AS36824, with Lumen and Cogent upstreams. It's delivered on-site at Depot 01 on 1G, 10G, or 100G handoffs and is billed separately from the $175/kW/month NNN colocation rate.

Is 95th percentile IP transit cheaper than cloud egress for AI inference?
For any inference workload pushing meaningful output, yes, by a wide margin. An image-generation API sending about 90TB a month would pay roughly $7,100 at typical tiered hyperscaler list rates for internet egress. That traffic averages under 300 Mbps, so its 95th percentile fits inside a 1G commit at IDACORE for $600/month. Predictability matters just as much. Your transit cost tracks your commit tier, not every gigabyte your users download.

What happens if my inference traffic bursts above my bandwidth commit?
Billing uses the 95th percentile of 5-minute samples, so the busiest 5% of the month, about 36 hours, is discarded before the bill is calculated. Launch spikes, model weight pulls, and short traffic surges inside that window don't set your billable rate. If your 95th-percentile figure lands above your commit month after month, that's the signal to move up a tier, which still costs a fraction of equivalent per-GB egress. Tell us before a known launch.

Is IDACORE Depot 01 available for GPU colocation now?
Yes. Depot 01, the pilot site in Northeast Washington, is live and energized. It offers air-cooled capacity at up to 50kW per rack, enough for three 8-GPU B200-class servers. Racks are powered in under 2 weeks from signing, the site carries a 99.9% uptime SLA, and IP transit from AS36824 is delivered on-site. There's no minimum deployment size. Check idacore.com/depots for current availability before you plan a deployment.

What does GPU colocation at an IDACORE Depot cost in total?
Colocation is $175/kW/month NNN on a 36-month term, covering space, cooling, and facility infrastructure. Power bills on top at cost: metered IT load x PUE at the utility rate, with no markup. The design-target PUE is about 1.1 to 1.2. IP transit, backup power, transport, and remote hands bill separately. Remote hands is $150/hr during business hours and $250/hr after hours. 12- and 24-month terms are available at a higher rate. Deposit and first month are due at signing.

If your egress line grows every time your product gets more popular, it's time to move your inference fleet onto a pipe you size yourself. At Depot 01, transit from our own ASN is delivered on-site next to your racks, billed on the 95th percentile with no per-GB meter. Get a Depot 01 rack and transit quote for your inference traffic. Send us your current monthly egress, and we'll show you which commit tier fits.

Ready to Implement These Strategies?

Our team of experts can help you apply these bandwidth economics techniques to your infrastructure. Contact us for personalized guidance and support.

Get Expert Help