DevSecOps: Automation for Assurance

This is Part 3 in our Four Part DevSecOps Blog Series

In Part 2 of this blog series, DevSecOps – Security at the Speed of Business, we explained the “what” and “why” of our security guardrails and the Agile Hackathon method used to develop these guardrails, which enable the cloud offer teams to confidently build and deploy their offers more quickly. In Part 3, we will cover “how-to” automate security guardrails.

Addressing security early on in the development cycle reduces security risks and shortens the time to remediate them. While providing the guardrails can help the hundreds of development teams to comply with the security requirements, automating these guardrails saves these team’s time and effort in addition to providing security posture assurance. This is in keeping with our DevSecOps Manifesto #4 “Automate and Continuous Security Over Manual and Point in Time Security” discussed in Part 1 of our blog DevSecOps series.

Automating and delivering security as Code

The DevSecOps practice was built on Amazon’s Web Services (AWS) platform as our first target environment. We had two main objectives with the automation of Guardrails for AWS: 1) Embed Guardrails at the time of new AWS account provisioning and 2) Continuous Security Validation of guardrail auditing checks so we can validate security posture of each AWS account.

Security Guardrail Automation

We wanted to provide an easy way for Cisco teams to apply these guardrails. The best opportunity to do so is at the time of initial AWS account provisioning. We developed code to enable the secure configurations and integrated it with Cisco’s enterprise account provisioning process called eStore. This required coordination with multiple teams within Cisco – InfoSec, IT, Supply Chain, Procurement, and Cloud business units – keeping with our Manifesto #1: “Collaboration To Securely Enable Business Over Mandates”. As of today, all new accounts provisioned via eStore have guardrails automatically installed ensuring critical security requirements are met. Teams that had their AWS accounts prior to the eStore enablement, can also run these scripts to configure their guardrails.

Guardrails automated at provisioning time:

Strong Identity – SSO (Single Sign On) using Cisco Enterprise Identity

Enable Security Logging and Monitoring and Setup Security Monitoring Role

Setup Vulnerability Scanning

Setup Encryption Key

Configure Resource Tagging

Setup Security Audit Role

Continuous Security Validation

Now that the guardrails are enabled in each AWS account, we needed to ensure they are still up to date with any new resources or changes done by the teams. We do this with continuous security validation along with real time monitoring of the security logs. We achieved it using two separate AWS accounts, one for running continuous validation and the other to do log monitoring and incident detection. We designed the solution leveraging AWS service offering called Lambda. It is a server-less solution that runs continuously on Cisco AWS accounts performing 100+ checks on each account in keeping with our DevSecOps Manifesto #6 “Consumable Security Services with APIs over Mandated Security Controls and Paperwork.”

Continuous security validation performed on:

Identity accounts and policies

Checking for improperly Exposed Buckets (Storage)

Bastion Host Configurations

Network ACLs, VPCs and Security Groups

OS and AWS Hardening Benchmarks

Security Monitoring Plays (checks designed for log analysis)

Vulnerability Analysis of external EC2 hosts

Analyze AWS Trusted Advisor Metrics

Proper tagging and classification of data (S3 buckets and EC2 instances)

All the findings from our validation are being captured on a security health report which we send to Cisco tenants on a daily basis.  With the possession of a security report, tenants have visibility into the security health of their own account and can now address any critical security findings in a timely manner. We provide a security grade to each account (A-F) based on a weighted scoring of the different validation checks.

This is the continuous security we had envisioned in DevOps models whereby as the teams are continuously integrating and deploying code, they also get continuous security assurance – the holy grail of Security. The teams are empowered to self-provision, self-verify and self-remediate demonstrating many key principles in our Manifesto #2 “Security Metrics and Data over Uncertainty and Assumption” and Manifesto #5 “Trust and Transparency over Secrecy and Obfuscation.”

Choice of Automation Technologies – Continuous Security Buddy

We achieved the security automation via our own tool called Continuous Security Buddy (CSB). The tooling is based on several AWS services like Lambda, CloudTrail, DynamoDB, Trusted Advisor etc. as depicted in the diagram below.

We continue to improve on the Continuous Security Buddy capabilities in a DevOps model and are integrating some of Cisco’s security products like Stealthwatch Cloud and Umbrella.

In Part3 of this blog series, we covered our journey of automation to help us scale security across hundreds of development teams.  In the next and final blog, we will share our success factors and lessons learned along our DevSecOps journey. Please stay tuned and in the meantime we welcome your feedback.