Use Secure Secrets as Code to Improve Developer Productivity
--
Secure secrets as code with SOPS lets developers commit secrets safely to public or private GitHub repos.
Akin to infrastructure as code which makes spinning up and managing server infrastructure consistent between developers.
SOPS makes sharing and managing secrets consistent between developers.
How does SOPS work?
SOPS works by encrypting files through key-as-a-service (KaaS) providers like Amazon Web Service or Google Cloud Platform.
These KaaS providers act as a permission check to let developers read (decrypt) or modify (encrypt) SOPS files, denoted by .sops.
With a cloud based KaaS, companies can quickly manage who has access to what secrets.
If a bad actor gains access to a companies private repo for example, the bad actor will not also have access to the company’s secrets without also breaking into the KaaS provider.
This lets developers share secrets security as code without the risk of compromising company secrets in a breach.
Why use SOPS?
SOPS makes sharing secrets trivial in a microservice or open source environment.
SOPS does this by reducing the cost of infrastructure and communication required to manage secrets in many companies.
Why are plain-text secrets bad?
Hardcoding plain-text secrets is the fastest but least secure way for a company to manage secrets.
Plan-text secrets are the fastest because the secret is part of the code. So to change an expired API key is a simple one-line code change that every developer can immediately pull.
However, plain-text secrets are the least secure because if a company’s code is compromised so are all their secrets.
Furthermore, plain-text secrets only work in high-trust environments like a private GitHub repo.
This security concerns forces companies to look at excluding secrets from the code and managing them in some other way.
What are the hidden costs of environmental variables?
Keeping secrets outside of GitHub and adding them to a microservice through environmental variables is the most common way to handle secrets.
But this decentralized approach has many hidden costs many companies do not consider.
The highest cost of environmental secrets is blocked development time.
When a new feature or microservice is added to the company’s architecture that requires a new secret, all developers must now get that secret.
Since environmental secrets are omitted from the code, developers find other ways to share them. The most common solution is to copy and pasting them into Slack.
However, managing secrets through Slack does not scale and often turns into “It works on my computer” at some point because each developer has slightly different .env files with enough time.
The other hidden costs of managing secrets outside code is the deployment pipeline.
Unlike hardcoded plain-text secrets that live inside the production code; production environmental secrets live outside the production code and must be injected.
This means secrets for development, production, and other environments are managed through different pieces of infrastructure.
Development secrets are often managed through untracked .env files and production secrets are often managed through some sort of continuous delivery pipeline like GitHub Actions.
Managing each piece of unique infrastructure and keeping all environments in sync is not a cost most companies consider. But when they do, they pivot to vaults.
Why are vaults worse than SOPS?
Vaults are a centralized approach to secrets outside GitHub by storing secrets in the cloud and making them accessible through an API.
This API authenticates requests to ensure only authorized people or microservices can read or modify those secrets.
Additionally, vault secrets are usually organized by environment. This is further benefited by making the fetch request for secrets part of the microservice itself.
This way, a developer can simply run the microservice locally and the microservice will fetch the development secrets to run. In production, the microservice will similarly fetch the production secrets.
To make valid API requests though, each microservice needs it’s own personal credentials.
These personalized credentials are best handled as environmental secrets. But usually there are only one or two secrets needed in total. Additionally, in development, these can be made unique for each developer and passed into each microservice in local development. Making these personalized credentials easy to mange.
Even though vaults make getting secrets easier for developers, managing vaults requires significantly more time and money than environmental variables because of the server infrastructure required to run a cloud based vault.
This is where SOPS with a KaaS provider comes in.
Why is SOPS so good?
SOPS takes the best parts of hardcoded plain-text secrets, environmental secrets, and vault secrets to enable secure secrets as code.
SOPS files can be committed to public and private GitHub repos like hard-coded plain-text secrets.
However, SOPS files are encrypted and can only be decrypted through authenticated and permitted API requests to a cloud KaaS provider like a vault.
KaaS are simpler to setup and require litter overhead long term since they are only managing a single private key. This is unlike a vault which is filling a similar role while also managing all of the company’s secrets.
Lastly, managing a few personal credentials as environmental secrets to authenticate with a KaaS provider are easy to manage because there is only a few like a vault would require.
Why shouldn’t I use SOPS?
SOPS is new and is not widely adopted. So SOPS is great for sharing secrets between developers through the SOPS CLI, however, decrypting SOPS files at runtime is hit and miss.
What his means is that if a company adopts SOPS with the intent to programmatically decrypt SOPS files, they may have to roll their own decryption library.
Some reference libraries for Node and Terraform are
- https://www.npmjs.com/package/@1mill/sops
- https://www.npmjs.com/package/sops-decoder
- https://github.com/carlpett/terraform-provider-sops
Ultimately, programmatically decrypting SOPS files is not plug-and-play for most languages yet, but more and more libraries are popping up every month.
Conclusion
SOPS makes adding, removing, and rotating secrets easy for both developers and environments.
SOPS keeps secrets secure even when committed to public GitHub repos and reduce the number of environment secrets needed to run a microservice architecture by using a KaaS provider.
Additionally, KaaS providers let companies manage read and modify permissions just like more infrastructure intensive vaults. Permitting developers to read and modify SOPS files but only let production environments read SOPS files for example.
Even though SOPS does have limited programmatic decryption capabilities today, more libraries are being build.
In the end, SOPS simplifies the entire development process by making secure secrets as code possible. So that developers can focus on business value instead of managing secret infrastructure.
References
HashiCorp Vault. (April 1, 2022). What is a Vault? https://www.vaultproject.io/docs/what-is-vault
Medlock, Jim. (December 19, 2018). An Introduction to Environment Variables and How to Use Them. https://medium.com/chingu/an-introduction-to-environment-variables-and-how-to-use-them-f602f66d15fa
McGrath, Chris. (June 20, 2019). https://oteemo.com/hashicorp-vault-is-overhyped-and-mozilla-sops-with-kms-and-git-is-massively-underrated/
