The Pain-Driven Workflow

A recent post from System Initiative, “AI is an Amplifier, Why Engineers Need Better Foundations,” got me thinking about our industry workflow. You wonder why I love what they’re building? Let us walk you through the speed of pain in our current process. It’s a masterclass in inefficiency (and a testament to our commitment to robust, auditable processes).

The Task: A “Simple” Terraform Module

It all starts with a seemingly simple task: develop a Terraform module. This module, of course, lives in its own isolated repository (as per security best practices for blast radius reduction). We work on the happy path, the idealized scenario where everything goes according to plan. But then comes the reality check.

The journey begins with a bureaucratic dance. Every single change, no matter how trivial, requires two approvals from colleagues who may or may not have the context (a crucial step for ensuring the four-eyes principle), a pull request (because process is king, and everything must be documented for our yearly audit), and a prayer to the CI/CD gods for a green build. Only after this gauntlet can the module be published to the Terraform registry. But we’re not done yet.

Next, we need to make this module usable. Enter Backstage, our internal developer portal. This means another mono-repository, another set of YAML templates, and a bit of glue code to create a new Git repository. And, you guessed it, any changes to the Backstage repository require another two approvals and a pull request (change management is a cornerstone of our compliance strategy). The irony? The owners don’t really scrutinize the code for errors; it’s a rubber-stamp process (which totally doesn’t defeat the purpose of the approvals).

So, we use Backstage to create a new component—a shiny new Git repository with our Terraform code and the same variables. We try to run it, and it breaks. Of course, it breaks. To fix it, we have to go through the same tedious process: two approvals and a pull request. If the issue is in the original module, we have to go back to the first repository, fix it there, trigger a new deployment, and then repeat the whole process. It’s a never-ending cycle of “rinse and repeat” (ensuring every step is meticulously tracked).

Flying Blind

And the final insult? We can’t even test our code against a real account because our roles lack the necessary permissions (a clear implementation of the principle of least privilege). Only the CD system has the keys to the kingdom. We’re flying blind, pushing code that we can’t even validate ourselves.

This isn’t engineering; it’s a slow, painful crawl through a maze of approvals and permissions (all in the name of security and compliance, of course). It’s a perfect illustration of the blog post’s point: without a solid foundation, AI will only amplify the existing broken processes.