IaC at the Management Level: Using metaprojects
In the DevOps practice the first stage shall always be the Infrastructure as Code (IaC) pattern. This usually affects only the Development and Operation teams: Terraform, CloudFormation, Cloud Development Kits, Ansible, Puppet… are tools well known in the IT world.
But what about Project Management? Do they create all the resources associated to a project using this approach? The answer is no. And here it is where DevOps methodology should emerge as a conglomerate of culture, practices and tools to improve it.
(btw, this article does not pretend to force the PMs and TLs to write code, we are just trying to export DevOps patterns to the leadership roles)
At Devgurus (a DMI company), we work in a microservices based, API first, cloud and Headless oriented environment (read more about MACH here), the perfect scenario to implement this vision.
So, let’s get started: What are the common resources we are going to need in a modern software project?
Repositories, cloud provider projects, SaaS services (such as databases, email platforms), base templates, roles… All these things can be created with Terraform.
Starting with the repositories, for every project we choose a Git provider (Bitbucket, Github or Gitlab), create a project and assign users and roles.
Instead of doing this manually, we can have a metaproject (or admin project) for the selected provider and follow IaC pattern.
The meta Gitlab project is comprised of the following files:
· Users: We can import or create users to assign it to the proper project.
· Groups and permission: We establish a root repositories folder (also called group, project, organization…) to set common properties.
· Repositories: Here we configure all the repositories that will forme the project. Fine grained permissions and base configuration can be easily tracked
· Backend and pipeline configuration: The backstage configuration that makes everything work. There’s a good guide in this link. In our implementation, we store the tfstate in a Gitlab managed HTTP storage, so that we don’t have to worry about buckets and permission, the state is stored in the same meta project repositories with a CI token.
There are more advanced resources available like variables, hooks, review rules… therefore, the leaderships can established whatever company standards need to be followed (such as merge request policies, secret management, guest access, etc.).
With the project repository structured with code easily engaged in a pipeline, further project changes can be done with ease.
Next stop: Environments
Any typical cloud project has also some manual tasks that need to be initialized: creating the GCP project itself, assigning a billing, enabling the API, creating the first buckets to stores states, assigning users and roles… this can be achieved in a tidy manner using code.
Most of our projects are hosted in Google Cloud, so we have an administration project that manages the organization projects as explained above. Google has also a nice guide here.
Have in mind that normally we have to replicate the cloud resources per every environment. 3 environments means that we have to repeat it three times. IaC is a great improvement.
Our GCP admin project has bit different structure (for the sake of clarity we have 1 Terraform file per software project) but same approach: A God project will create the initial structure with the following components:
· Project specific configuration:
We have as well advance options such as configurable budgets, retention policies, alerts…
In summary, the Infrastructure as Code (IaC) pattern is also a good approach to bootstrap the initials tasks of a new project from the Management standpoint. This ensures that everything is documented, repeatable and standardized across the company.
As a DevOps engineers we can setup this infrastructure and guide the management roles. We have the tools, the culture and the procedure. Don’t let anyone, not even PMs or TMs, to create anything manually.
