If Networking Is 90% of Agentic Infrastructure, the Network Should Be the Platform
How F5 BIG-IP Next for Kubernetes runs a GPU cloud or AI factory as one programmable platform, from picking the site down to picking the GPU.
If Networking Is 90% of Agentic Infrastructure, the Network Should Be the Platform
How F5 BIG-IP Next for Kubernetes Runs a GPU Cloud as One Programmable Platform
Michael Levan opens his post with "networking is 90% of what you need to get right" in agentic infrastructure. I agree. And if networking is 90% of the job, the network shouldn't be a pipe agents pass through. It should be the platform the AI factory runs on.
His post, "Agent Substrate + Agentgateway: Your Secure Egress Path to Models", is a clear hands-on walkthrough of one slice of that: getting a sandboxed agent's traffic to a model. Read it first. This post zooms out to the platform that slice sits in, BIG-IP Next for Kubernetes (BNK), and to what a GPU cloud builds on it.
What you'll get from this post:
- The seven decisions BNK makes for one inference request, at one address.
- Why the architecture holds up: separate control and request paths, standard objects, one engine.
- The tiers of the platform and the F5 services that plug into each one.
- What a GPU cloud can sell on it, and where an agent sandbox fits.
One Request, Seven Decisions, One Address
Start with what a customer of a GPU cloud actually calls: one OpenAI-compatible address in front of the GPUs they rent. Here's everything that happens to a chat request at that address on BNK.
All seven run in TMM, the traffic engine BNK takes from BIG-IP. The firewall goes first, at the network layer. Steps 1 to 6 are one policy iRule per endpoint, built from whichever policies the provider switched on. The iRule makes plain sideband calls to the key service, to F5 AI Guardrails and to the usage store, and asks the Endpoint Picker for a GPU over gRPC ext_proc.
The order matters. A request with a bad source, a bad key, a harmful prompt or an empty budget is refused before a GPU sees it, so refusals cost the provider no GPU time. And because metering reads the token count from the model's own answer, the bill matches what the customer's SDK reported.
An Architecture Built from Standard Parts
A control path and a request path that never mix
control: provider API -> adapter -> Kubernetes objects -> BNK controller -> TMM
request: client -> TMM address -> the picker's GPU choice -> GPU
Creating, changing and deleting an endpoint goes down the first line. Every prompt and every streamed token goes down the second. The provider's console and API are never in the request path; TMM only calls the key service and sends usage records.
Every endpoint is a set of ordinary objects
| Object | What it does for the endpoint |
|---|---|
| Gateway | The endpoint's address and listeners |
| HTTPRoute | Sends /v1 to the model pool |
| InferencePool + F5EPP | The pool of model servers and the cache-aware picker in front of it |
| iRule + policy | The per-request logic for steps 1 to 6 |
| F5BigFwPolicy | The endpoint's own network firewall |
Each object is labelled with the endpoint it belongs to, so a platform team reads the whole thing with kubectl and the tools it already has. Delete the endpoint and exactly its objects go.
The provider keeps its own API
Customers keep using the provider's console and API. A small adapter turns those API calls into BNK objects, and it needs nothing beyond the Kubernetes API and the BNK, Gateway API and Inference Extension CRDs. It ships as one Helm chart.
One engine, on the node or on the DPU
The same TMM carries ingress, policy, routing and metering, and it runs on Kubernetes nodes or on BlueField DPUs. A provider starts in host mode and moves the work onto DPUs when it wants enforcement the tenant's host can't reach.
The Platform, Tier by Tier
That endpoint is one slice of a bigger platform. BNK covers the whole path, from choosing the site down to choosing the GPU.
Tier 1 decides which site or factory gets the request, and BNK does that itself with GSLB and DNS. A GSLB wide IP answers for the service name, data centers and servers describe the sites, monitors track their health, and topology records steer clients by where they are. DNS zones, views and a DNS cache run alongside. Put BNK in each factory and you have the shape of AI Grid: distributed sites run as one inference service.
Tier 2 is for providers who rent bare metal to strangers. In NVIDIA's zero-trust model BNK runs on BlueField DPUs as provider fabric ingress (f5-ingress), managed out of band through NVIDIA DPF and NICo, NVIDIA's open-source infrastructure controller. It's never managed through the tenant's host, and tenants are kept apart by per-tenant fabric VRFs, so isolation lives on hardware the tenant doesn't control. In this mode BNK's job is the fabric edge: ingress, L4 to L7 services and security. The security services are declared like everything else, with kinds such as F5BigFwPolicy for firewall rules and F5BigDdosProfile for DDoS vectors and behavioral detection.
Tiers 3 and 4 are where agent traffic lives. In tier 3, BNK is the VTEP on a BGP EVPN-VXLAN fabric, so each tenant is a VRF with its own VNI, and VRFs can be set to strict isolation. Each tenant gets its own egress and SNAT pool, and the AI Gateway runs on the same tenant gateway. In tier 4, TMM asks the F5 Endpoint Picker over gRPC ext_proc which GPU worker should take the request.
Every F5 function plugs in
The services in that list are the same F5 engine BIG-IP customers already run, declared as Kubernetes resources: GSLB and DNS, network firewall, DDoS protection, intrusion prevention, web application security, IPsec, carrier-grade NAT, access policy and IP intelligence. For a telco putting AI factories at the network edge, the service provider toolkit is there too: deep packet inspection, policy enforcement and zero-rating, with Diameter, GTP, SIP, NGAP and RADIUS support. When a provider needs logic none of these cover, it writes an iRule, and the platform grows with it.
What a GPU Cloud Sells on It
Renting GPUs is the base business. On BNK the provider can also charge for what sits in front of them: inference endpoints, tokens and policies. A firewall per endpoint, with matches streamed to the customer's SIEM. Guardrails on every prompt. Per-user token budgets. Frontier models behind the same address the customer already calls.
Cache-aware routing adds to that. When the Endpoint Picker sends a prompt to the GPU that already holds its prefix, that GPU skips the prefill it would otherwise redo, so the same GPUs answer more requests.
Tenancy lines up across layers: a VRF, a BNK gateway with its own address and SNAT range, AI Gateway keys and budgets, and a GPU queue in the scheduler. NVIDIA's cloud partner reference architecture leaves the tenant access network to partners, and this is how BNK fills that slot. Token and request logs stream over HSL to the provider's billing pipeline, and an OpenTelemetry collector handles telemetry.
Per-tenant logic is an iRule away. This one takes request priority away from the caller before the Endpoint Picker sees it; a controller fills the lookup tables.
when HTTP_REQUEST {
# the caller never sets its own priority
HTTP::header remove "x-gateway-inference-objective"
set caller ""
if { [HTTP::header exists "x-caller-key"] } {
set h [b64encode [sha256 [HTTP::header value "x-caller-key"]]]
HTTP::header remove "x-caller-key"
if { [info exists static::callers($h)] } { set caller $static::callers($h) }
}
# unknown or suspect callers are shed first when the GPU pool is saturated
if { $caller eq "" || [info exists static::suspect($caller)] } {
HTTP::header insert "x-gateway-inference-objective" "suspect-traffic"
}
}
The Essentials in Kubernetes
You need BNK installed and licensed, the Gateway API and Inference Extension CRDs, and a model server pool. Names are placeholders.
Turn the AI features on
The AI Gateway and intelligent load balancing are switched on in the CNEInstance, BNK's top-level resource. This is a fragment of a larger resource.
apiVersion: k8s.f5.com/v1
kind: CNEInstance
metadata:
name: f5-bnk-f5-cne-controller
namespace: f5-bnk
spec:
aiGateway:
enabled: true # AI Gateway console and admin API
intelligentLB:
enabled: true # deploys the Analyzer
eppNamespaces: [models] # namespaces where F5EPP resources live
Put the model pool behind the Endpoint Picker
A standard InferencePool, selected by labels, points at F5's Endpoint Picker (F5EPP). With failureMode: FailClose the request fails if the picker is down, so nothing silently falls back to round robin.
apiVersion: inference.k8s.f5.com/v1alpha1
kind: F5EPP
metadata: {name: model-pool-epp, namespace: models}
spec:
poolRef: {name: model-pool}
engine: dynamo # reads Dynamo KV-cache events from NATS
natsURL: nats://dynamo-nats.dynamo-system.svc:4222 # your Dynamo NATS service
---
apiVersion: inference.networking.k8s.io/v1
kind: InferencePool
metadata: {name: model-pool, namespace: models}
spec:
selector:
matchLabels:
app: llm-decode
trust.example.com/state: trusted # used by the runtime pattern below
targetPorts: [{number: 8000}]
endpointPickerRef: {name: model-pool-epp, port: {number: 9002}, failureMode: FailClose}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: {name: model-route, namespace: models}
spec:
parentRefs: [{name: inference-gateway, namespace: f5-bnk}]
rules:
- matches: [{path: {type: PathPrefix, value: /v1}}]
backendRefs:
- {group: inference.networking.k8s.io, kind: InferencePool, name: model-pool, port: 8000}
That trusted label is also a security hook. A controller fed by Tetragon's eBPF events and the sandbox's audit trail can relabel a GPU worker that misbehaves, and the Endpoint Picker stops choosing it within seconds while the pod stays up for forensics. The same controller can have BNK answer a compromised caller with a 403 and add its address to the firewall.
Call it
The client sees an OpenAI-compatible endpoint; the Anthropic Messages API works too.
curl https://ai-gw.example.com/v1/chat/completions \
-H "Authorization: Bearer $VIRTUAL_KEY" -H 'Content-Type: application/json' \
-d '{"model":"auto","messages":[{"role":"user","content":"Write a two-line poem about BGP"}]}'
Where the Sandbox Plugs In
Isolating the agent's process is the sandbox's job. The design I'd recommend pairs a sandbox, Agent Substrate or NVIDIA OpenShell, with BNK as the tenant gateway behind it.
OpenShell is worth describing precisely. Each sandbox gets its own supervisor, which in Kubernetes runs as its own pod (os-supervisor-<sandbox>), paired with the sandbox workload by the openshell.ai/boundary-pair label. The supervisor enforces that sandbox's policy (OPA/Rego) on configured HTTP, GraphQL and MCP traffic, writes an OCSF audit trail per sandbox, and its provider profile can inject a credential for a named destination. NVIDIA's condition for "no network path except through the supervisor" is that the cluster CNI enforces NetworkPolicy ingress and egress in the sandbox namespace. So OpenShell works per sandbox, at process level. It isn't a shared egress point for many pods or for a tenant.
Point the sandbox's allowed destination at the tenant's AI Gateway, by FQDN for Substrate, since its egress policy can't name an IP. An OpenShell provider profile can inject the virtual key for that destination. With Substrate, atenet-egress injects the actor JWT, and since the AI Gateway validates JWTs from your issuer, that JWT is checked at BNK. From there the tenant's VRF, SNAT range and gateway are BNK's.
The Slice Levan Covers
Levan's post walks one path through tier 3: a sandboxed agent calling a hosted model. In Agent Substrate, agents run as Actors in sandboxes, and all their outbound traffic goes through atenet-egress. For model concerns he adds a dedicated agentgateway LLM gateway:
actor -> atunnel -> atenet-egress (policy, injects actor JWT)
-> agentgateway LLM gateway -> OpenAI / Anthropic
agentgateway does model routing and failover, budgets, rate limits and prompt guarding, and it implements the endpoint picker protocol too. Splitting egress and model concerns across components is the layout his article chose. On BNK the same slice runs through one engine, inside the tiers above.
What Comes Next
Here's where I'd take the platform. Agent identity and per-task authorization belong in policy, with a declarative layer next to iRules, CEL evaluated over an ext_authz callout.
Egress should follow the task: open only to what the task may reach, and closed when it's done. Every decision should land in a signed record per tenant. And a quarantine should come with a provable stop, so you can show a workload sent nothing after it was stopped.
Across sites, AI Grid takes the same model up a level: GSLB picks the factory on GPU-aware signals, and BNK in each one picks the model and the GPU.
Try It
Start with the BNK documentation for the CNEInstance and enable intelligentLB and the AI Gateway. Build an InferencePool that points at the F5 Endpoint Picker, attach a policy iRule and an F5BigFwPolicy to its Gateway, and put a sandbox in front, Substrate or OpenShell, with egress allowed only to that address (by FQDN for Substrate).
Agent egress to a model is one slice of an AI factory. Build the platform first, and the slice comes with it.