Misadventures in securing IaC

•

I’ve had an Infrastructure-as-Code repo since around 2005. That used to be Puppet code, and included config for bind, apache/nginx, dovecot IMAP, and postfix. There were a few years of apathy in which I hosted DNS elsewhere: this cascaded into all sorts of pain.

I’ve since pulled nearly everything back to AWS so I can manage it all with Terraform in a GitHub repo. That raises a question: how secure are my AWS resources? Inspecting stuff with AWS is possible, with tools like AWS Security Hub. It would be nice to know that I were making secure Terraform code before I pushed it and applied. I chose Checkov. There are other tools out there, like Snyk IaC.

I installed the Checkov package from PiPi using Pipenv, and then invoked pipenv shell to call the checkov script from a Makefile. Here’s the kind of output that Checkov gives you:

(terraform) bash-3.2$ checkov --quiet -d .; exit

       _               _
   ___| |__   ___  ___| | _______   __
  / __| '_ \ / _ \/ __| |/ / _ \ \ / /
 | (__| | | |  __/ (__|   < (_) \ V /
  \___|_| |_|\___|\___|_|\_\___/ \_/

By bridgecrew.io | version: 2.0.338

terraform scan results:

Passed checks: 91, Failed checks: 3, Skipped checks: 43

Check: CKV_AWS_19: "Ensure all data stored in the S3 bucket is securely encrypted at rest"
	FAILED for resource: aws_s3_bucket.media-build-doctor-com
	File: /build-doctor.com.tf:84-90
	Guide: https://docs.bridgecrew.io/docs/s3_14-data-encrypted-at-rest

		84 | resource "aws_s3_bucket" "media-build-doctor-com" {
		88 |   bucket = "foo.build-doctor.com"
		89 |   acl = "private"
		90 | }

Here’s what I learned:

Refactor for a better time

Checkov will tell you in no uncertain terms that your Terraform code is insecure; you’ll end up questioning your life choices. It was especially bad for my poorly composed Terraform code - I had lot of similar resources in my code, each of which picked up many warnings.

Doing the right thing and extracting modules for my static websites was a good thing - it drove consistency and reduced the howls of outage from Checkov.

Apply and test often

The age old struggle between convenience and security played out again - I spent a couple of hours happily smashing security vulnerabilities that Checkov reported, before finding that I’d completely broken the sites, with no access from S3 buckets to Cloudfront. The log buckets were so tightly locked down that I couldn’t even see what happened. After quite some time kicking tyres, I rolled the changes back.

I made some some really simple tests using curl, that I can easily run from make. This gives me confidence that the websites still work. I try to get early feedback by applying the changes in Terraform and then testing afterward.

Suppress the false positives

The many findings about S3 security make a great deal of sense in many contexts - the way we use S3 has changed during the lifetime of the S3 service. Nobody wants to win a Bucket Negligence Award. In my case, where I host some very basic static websites, I’m happy to suppress those:

resource "aws_s3_bucket" "bucket" {
  #checkov:skip=CKV2_AWS_6:We have a public access block
  #checkov:skip=CKV_AWS_18:this kind of content doesn't need encryption
  #checkov:skip=CKV_AWS_19:this kind of content doesn't need encryption
  #checkov:skip=CKV_AWS_20=Public ACLs are fine for this
  #checkov:skip=CKV_AWS_52:MFA delete hard to apply via TF
  #checkov:skip=CKV_AWS_144:Cross Region Replication not important
  #checkov:skip=CKV_AWS_145:We're not going to encrypt public website stuff

Suppression helps me to focus on the things that matter in my context.

DevOps New Zealand