Skip to main content

Networking

This layer provides platform teams with common networking resources. It is managed as code in pt-corpus and provides a consistent foundation for all workloads.

  • Shared VPC: A single host project VPC is shared across team service projects, centralizing network control while allowing teams to run workloads in isolated projects
  • Subnets: Per-zone node subnets and master subnets with secondary ranges for GKE pods and services, with Private Google Access enabled; IAM bindings grant GKE service accounts the network user role on each subnet
  • Cloud NAT: Regional Cloud NAT gateways provide outbound internet access for private cluster nodes without exposing them to inbound traffic
  • DNS zones: A public osinfra.io zone, a private zone for internal resolution, and per-team subdomain zones with NS delegation records in the parent zone
  • Private service networking: A reserved IP range and service networking connection enable private connectivity to managed services (e.g., Cloud SQL)
Architecture Decision Records

This page includes Architecture Decision Records documenting the key design decisions.

IP Address Management​

The 10.0.0.0/8 RFC 1918 space is divided into four /10 blocks, each supporting up to 30 isolated GKE clusters. VPCs use the same address space across sandbox, non-production, and production environments, with each environment isolated to its own project. All CIDR ranges are defined in pt-logos and flow through pt-corpus to pt-pneuma.

VPC: standard-shared

IPAM calculator configuration

We break up the 10.0.0.0/10 CIDR block with the GKE IPAM calculator using the following inputs:

{
  "network": "10.0.0.0",
  "netmask": 10,
  "nodeNetmask": 20,
  "clusterNetmask": 15,
  "serviceNetmask": 20,
  "nodePodNetmask": "24",
  "masterNetwork": "UNIQUE",
  "locationType": "REGIONAL",
  "extraZones": 1
}

pt-pneuma-us-east1-b

Primary
10.60.0.0/20
Pods
10.0.0.0/15
Services
10.61.224.0/20
Master
10.63.192.0/28

pt-pneuma-us-east1-c

Primary
10.60.16.0/20
Pods
10.2.0.0/15
Services
10.61.240.0/20
Master
10.63.192.16/28

pt-pneuma-us-east1-d

Primary
10.60.32.0/20
Pods
10.4.0.0/15
Services
10.62.0.0/20
Master
10.63.192.32/28

pt-pneuma-us-east4-a

Primary
10.60.48.0/20
Pods
10.6.0.0/15
Services
10.62.16.0/20
Master
10.63.192.48/28

pt-pneuma-us-east4-b

Primary
10.60.64.0/20
Pods
10.8.0.0/15
Services
10.62.32.0/20
Master
10.63.192.64/28

pt-pneuma-us-east4-c

Primary
10.60.80.0/20
Pods
10.10.0.0/15
Services
10.62.48.0/20
Master
10.63.192.80/28

pt-kryptos-us-east1-b

Primary
10.60.96.0/20
Pods
10.12.0.0/15
Services
10.62.64.0/20
Master
10.63.192.96/28

pt-kryptos-us-east4-a

Primary
10.60.112.0/20
Pods
10.14.0.0/15
Services
10.62.80.0/20
Master
10.63.192.112/28

Consumer Contract​

A new GKE cluster is always guaranteed a pre-allocated CIDR slot — no IP conflict is possible because every range is defined in pt-logos before any cluster is created. Each environment has its own isolated VPC, so the plan supports up to 30 clusters per environment in the active /10 block, expandable to 120 per environment across all four /10 blocks. When all 30 slots in the active block are claimed, the Nomos Agent's IPAM sequences must be extended to cover the next /10 block before any additional clusters can be provisioned.

To provision a new cluster, use the Nomos Agent in pt-logos. The agent reads all existing CIDR allocations from teams/*.tfvars, assigns the next available slot automatically, and opens the necessary pull requests — no manual CIDR selection required.

Components​

ComponentDescription
shared-vpcA VPC host project network shared across all team projects in an environment
subnetA regional subnetwork with primary node CIDR, pod secondary range, and service secondary range
dns-zoneA Cloud DNS managed zone for a platform domain (e.g., osinfra.io)
cloud-natManaged NAT for private nodes to reach the internet without public IPs

Core Invariant​

Every platform-managed project is a Shared VPC service project with subnet access granted at creation.

Architecture Decision Records​

GKE IP Address Management for Shared VPC​

StatusDateDeciders
Accepted ✅April 2026Corpus

Context and Problem Statement​

GKE VPC-native clusters using Shared VPC require all IP address ranges — primary node subnets, pod secondary ranges, service secondary ranges, and control plane ranges — to be pre-created and user-managed. GKE cannot auto-manage these ranges in a Shared VPC. This forces upfront address planning before any cluster can be created and means every range must be tracked centrally.

Each cluster is a regional cluster with a single zonal node pool (e.g., pt-pneuma-us-east1-b has a regional control plane and nodes pinned to us-east1-b), so the number of clusters scales with zone coverage rather than workload size. Address space must accommodate growth across multiple regions and zones without requiring re-addressing.

The guiding principle throughout: use GKE defaults unless there is a clear reason not to.

Decisions​

  1. Carve 10.0.0.0/8 into four /10 blocks. Each /10 is large enough for 30 clusters. Four blocks give the platform room to grow across independent address spaces without redesign.

  2. Follow GKE defaults for all subnet sizes where possible.

    • /20 primary node range — GKE default
    • /24 per-node pod alias range — GKE default, supports 110 pods per node
    • /20 service range — GKE default, supports 4,096 services per cluster
    • /28 control plane range — GKE default for private cluster masters

  3. Use /15 for the cluster-level pod secondary range. This is the one deliberate deviation from defaults. A /14 would roughly double per-cluster node capacity (~1,022 nodes) but would halve the number of clusters per /10 (~15 vs 30) with nearly identical total node capacity across the block. Regional clusters with zonal node pools benefit more from cluster count than cluster size, so /15 was chosen.

  4. Use the GKE IPAM calculator for address planning. This ensures ranges are correctly sized, non-overlapping, and documented reproducibly.

  5. Define all ranges centrally in pt-logos. All CIDRs — primary, pod, service, and master — are defined in the google_subnets map in pt-logos and flow through pt-corpus to pt-pneuma. This keeps all network addressing consolidated and visible in one place.

  6. Use the same address space across sandbox, non-production, and production. Each environment has its own project for isolation. Keeping ranges consistent across environments reduces cognitive overhead and eliminates environment-specific address planning.

Alternatives Considered​

  • /14 cluster pod CIDR — ~1,022 nodes per cluster, but only ~15 clusters per /10. Total node capacity across the block is nearly identical. Rejected in favour of more clusters at a still-generous 510 nodes each.
  • Auto-managed secondary ranges — Not possible with Shared VPC. GKE requires user-managed secondary ranges when using a host project.
  • Separate address spaces per environment — Rejected. Project-level isolation is sufficient; duplicating the address plan per environment adds complexity with no benefit.

Consequences​

  • 30 cluster slots available per environment in the active /10 block
  • 510 nodes per cluster maximum
  • 4,096 services per cluster maximum
  • 110 pods per node maximum
  • All IP address ranges must be defined in pt-logos before any cluster can be created
  • Adding a new cluster requires claiming an available slot from the IPAM plan