Why I Split Terraform in Two Instead of Fighting One Apply

Putting all infrastructure in one Terraform root is attractive.

One directory. One terraform apply. VPC, GKE, node pool, namespaces, Secrets, ConfigMaps—done.

That was my first instinct for CairnOps too.

I ended up splitting it into two stages.

“Created” and “ready” were not always the same moment

The first stage uses the Google provider to create the physical infrastructure.

The second uses the Kubernetes provider to configure resources inside the new cluster.

The awkward bit is that the Kubernetes provider needs a working Kubernetes API endpoint before it can do anything useful.

In practice, I sometimes hit a small timing gap between GKE reporting that the cluster had been created and the Kubernetes API being ready to accept connections reliably.

It was not a constant failure. That made it worse, not better. Intermittent infrastructure failures are hard to trust.

So I made the dependency explicit in the directory structure:

infra/terraform/
├── 01-infrastructure/
│   ├── VPC
│   ├── GKE cluster
│   ├── node pool
│   ├── Neon
│   ├── Upstash
│   └── Qdrant
└── 02-kubernetes-bootstrap/
    ├── Namespace
    ├── Secret
    ├── ConfigMap
    └── ServiceAccount

The workflow became boring, which is exactly what I want from infrastructure:

cd 01-infrastructure
terraform apply

gcloud container clusters get-credentials cairnops ...
kubectl get nodes

cd ../02-kubernetes-bootstrap
terraform apply

get-credentials sets up access. kubectl get nodes is the check that tells me the cluster is actually responding before Stage 2 starts.

Remote state became the interface

Once the two stages were separate, Stage 2 still needed values from Stage 1: cluster endpoint, CA certificate, database URLs, Redis URL, Qdrant URL, and so on.

I used separate GCS-backed state and exposed only the outputs Stage 2 needed.

Then Stage 2 read them through terraform_remote_state.

What I like about this setup is that Stage 2 does not care how Stage 1 creates a cluster. It cares about the values Stage 1 promises to expose.

That makes the outputs feel less like “Terraform plumbing” and more like an interface between two infrastructure layers.

The destroy path convinced me to keep it

The split became even more useful when tearing the environment down.

I destroy in reverse order:

cd 02-kubernetes-bootstrap
terraform destroy

cd ../01-infrastructure
terraform destroy

If I delete the GKE cluster first, the Kubernetes provider loses the API it needs to remove resources from inside that cluster.

Separating the state makes the lifecycle obvious.

It also narrows the blast radius of normal changes. Updating a Namespace or Secret does not need to touch the VPC state. Changing the cluster does not automatically mix itself with every bootstrap resource.

I originally introduced two stages to avoid a timing problem.

I kept them because they gave the infrastructure two clear lifecycles.

That turned out to be more valuable than having one impressive terraform apply.