Terraform is targeting to provision multi-hosting-provider provisioning. If your infrastructure is based only on AWS or on only 1 other hosting provider - Terraform is not the tool for you.
Why this post exists?
I do want to share my experience with Terraform and hopefully, save some of your time.
I did try Terraform with this blog. It worked really well. In couple hours I had the whole stack as the code. I was able to provision whole new blog with 1 command. And was really happy about it.
Then I gave a shot for it to provision one of the microservices at work. The level of details raised drastically, I ended up using modules, structuring the code and etc.
It is still relatively simple: ASG, Solr Slave/Master ASG, Redis (AWS ElasticCache), ELB, Route53, S3 etc… All in HA.
Everything was cool until… I needed Redis MultiAZ configuration. And that was a bummer. Before starting I did check, that there was AWS Redis module, and there were appropriate settings, and I couldn’t even think, that this setting could be missing.
Ok, I did hit the problem on day 2, when I actually had almost everything. That was the instant blocker.
While I was using the tool I noticed, that it is not really stable, and there are sometimes situations, when Terraform is not waiting enough time for components to provision and it would fail.
Provisioning failures in Terraform is another story. It means you need manual action. Which is really inconvenient, if I want just to spawn new testing environment, and for some reason, it fails. Executing the command second time it fixes the issues. Strange right? You think it is about dependencies being not set between components. Nope, it is not.
So, relying on this 3rd party did cost time and effort to migrate from it away.
Terraform. When it actually works with AWS.
- You are not familiar with AWS.
- You are scared of AWS Cloudformation. Especially documentation is not very friendly, it takes an effort to get familiar.
- Your infrastructure is really simple. P.S. I just want to provision this S3 website.
- AWS is one the hosting platforms you use and Terraform can actually glue these infrastructures. Here your infrastructure needs to be simple enough. I do not see how complex architectures can be covered by Terraform.
Terraform. The good example
Use it to provision multi-hosting infrastructure. And on separate providers use other specialized tools:
- do provision AWS with specialized AWS Cloudformation tool . Saltstack can provision external template - “AWS_CLOUDFORMATION_STACK”
- do provision on DigitalOcean or Saltstack/Puppet and etc.
This way Terraform would do what it does the best - manage multi-hosting platform configuration. And single platform specialized tools can be used to provision with latest features support.
It is developed relatively good pace - last month 195 merged pull requests, 153 closed issues. But.
What matters the most, that it will be always behind what hosting platform can provide. This is the problem for all unofficial tools, which are trying to replace, improve and or simplify official tools.
For example, to have MultiAZ replication available for Redis it took around 8 months. And still, it is not clear how stable it is:
- first raised issue https://github.com/hashicorp/terraform/issues/4361
- PR which solves it https://github.com/hashicorp/terraform/pull/8275
It is hard to rely on 3rd parties when it not clear when to appropriate AWS modules will be updated. And how many bugs it will add up on top?
- Use official provisioning tool. It is supported by AWS and tested by time and huge configurations.
- Spend more time on documentation pages for checking key points of the problem you are solving. After confirmation that required functionality, check how hard it is to build a prototype.
- 3rd parties tools come with its own:
- AWS limitations
- versioning handling
This only complicates your infrastructure provisioning from your CI.
Migration from Terraform to AWS Cloudformation
It is something for another blog post. But heads up - it took exactly 1 day to have the whole infrastructure as code in AWS Cloudformation. And without limitations found on Terraform.
- Do not use Terraform if your infrastructure based on single hosting platform only.
- Use it for multi-hosting setups only to cover the connection between them.
- Use platform-specific tools if available.