Compliance focused organizations are required to not only monitor but also demonstrate and report on the nature of cybersecurity attacks they are facing. The ability to use advanced machine learning and threat monitoring and detection is critical to help ensure the confidentiality, integrity and availability of their data assets. However, most organizations struggle with implementing effective threat monitoring and alerting capabilities due to cost and complexity involved. Their challenges are compounded by the need to meet strong compliance requirements specifically around the Audit and Accountability (AU) controls. The release of Amazon GuardDuty changes that!
Introducing Amazon GuardDuty for Continuous Threat Monitoring
When Amazon announced Amazon GuardDuty, the team at stackArmor were excited to have threat data as an integrated service, and GuardDuty didn’t disappoint! Amazon GuardDuty is an intelligent threat detection service that provides customers with an accurate and easy way to continuously monitor and protect their AWS accounts and workloads. GuardDuty analyzes AWS CloudTrail (AWS user and API activity in your accounts), Amazon VPC Flow Logs (network traffic data), and DNS Logs (name query patterns). Amazon GuardDuty consumes and analyzes the data logs provided by AWS CloudTrail, Amazon VPCFlow Logs and DNS Logs to provide threat findings. The primary finding categories include:
Reconnaissance – Findings related to reconnaissance by an attacker, such as unusual API activity, intra-VPC port scanning, unusual patterns of failed login requests, or unblocked port probing from a known bad IP.
Instance compromise – Findings indicating an instance compromise, such as cryptocurrency mining, malware using domain generation algorithms (DGA), outbound denial of service activity, unusually high volume of network traffic, unusual network protocols, outbound instance communication with a known malicious IP, temporary Amazon EC2 credentials used by an external IP address, and data exfiltration using DNS.
Account compromise – Findings related to account compromise including API calls from an unusual geolocation or anonymizing proxy, attempts to disable AWS CloudTrail logging, unusual instance or infrastructure launches, infrastructure deployments in an unusual region, and API calls from known malicious IP addresses.
The data provided is very informative and useful with only one challenge: How to get the data out of its tabular-only UI and into a format that is less noisy to the decision-makers that should see it. Currently, the data is visible from the AWS Management Console.
Screenshot of Amazon GuardDuty in Console with list of findings and events
However, most users need the ability to access the information and rapidly analyze the threats through customized queries.
Using Amazon QuickSight, Athena and Lambda for a simple monitoring pipeline
Luckily, Amazon GuardDuty provides the ability to access the data through Amazon CloudWatch Logs. By using existing AWS services, one can easily pipe the data to Amazon QuickSight and begin to plot maps and create customized charts. The diagram below provides an overview of the solution.
In order to more easily access the data, using the AWS Command Line Interface, we created a CloudWatch rule that calls a Lambda function each time a GuardDuty event occurs. The event data passed is in JSON, so the lambda code saves the JSON as a unique file in a S3 bucket.
Screenshot of JSON object generated by GuardDuty
Once the data is available in the S3 bucket, we use Amazon Athena and QuickSight to provide our clients with the data in rich informative ways through a dashboard. The screenshot below shows the GuardDuty displayed based on severity and action types.
To the country of origin in a chart:
And even a map!
With basic knowledge of AWS Lambda supported languages and the available AWS services, the stackArmor team was able to take Amazon GuardDuty data provided in a useful but raw format and show it from different angles, making it even more useful and informative to customers. The ability to help customers protect their data and deploy intelligent threat intelligence and detection capability as an integrated service without the need to acquire third-party services helps customers save money and simplify their security monitoring architecture.
Meeting Public Sector Compliance Requirements with Amazon QuickSight and Athena
The security operations and analyst teams at most public-sector organizations are generally overloaded with work and face resource constraints. The ability to use cloud-native services avoiding the need for expensive third-party “bolt-ons” provides an easy and effective way to provide better security as well as comply with compliance requirements. Using the combination of Amazon GuardDuty, Athena and QuickSight, public sector security and IT teams can more easily meet their Audit and Accountability (AU) security requirements. The AU requirements specifically mandate the monitoring, analysis, investigation, and reporting of unlawful,
unauthorized, or inappropriate IT system activity. Public sector organizations like District of Columbia’s Health Benefits Exchange (HBX) have already incorporated Amazon GuardDuty into their security monitoring and reporting processes.
About stackArmor
stackArmor is a public sector focused Advanced APN Partner stackArmor specializing in secure and compliance oriented IT solutions to regulated industries in Government, Financial Services, Healthcare, Non-profits and Energy. stackArmor is recognized by Amazon Web Services (AWS) for strong Public Sector and Security competencies on the AWS platform and was 1 of 10 inaugural launch partner firms globally for the AWS Security Competency. Learn more by visiting www.stackArmor.com
