Feb 3, 2012 - Email Address & Password ... Risks to Creators of Public or Shared AMIs. Failure to properly .... Clou
“Get Off of My Cloud”: Cloud Credential Compromise and Exposure Jeff Jarmoc - Sr. Security Researcher Dell SecureWorks Counter Threat Unit℠
Our Work • Understanding of AWS credential types and their “order of precedence” • Understanding of common mistakes & pitfalls • Develop tools to detect credential exposure within images • Large-scale testing of public EC2 Data for exposure • Experiment to quantify scale of potential victims of a malicious AMI • Findings disclosed to Amazon • Consistent with our reading of the “Amazon Web Services Customer Agreement” and the “Amazon Web Services Terms of Use” 2
AWS Security Credentials • Access Credentials – Access Keys – X.509 Certificates – Amazon EC2 Key Pairs – Amazon CloudFront Key Pairs
• Sign-In Credentials – Email Address & Password – AWS Multi-Factor Authentication Device (optional) – AWS Virtual MFA (optional – Recommended!)
• Account Identifiers – AWS Account ID – Canonical User ID
3
Risks to Creators of Public or Shared AMIs Failure to properly review AMI contents leads to data leakage • AMI filesystem – AWS Cert + Private Key – SSH Key Pairs – SSL certs and private keys
• Bash history files containing environmental variable exports or command-line usage of credentials (e.g., Secret Access Key) • Bash profile (e.g., .bashrc, .bash_profile) containing environment variable exports • Sensitive contents in .viminfo files 4
Risks to users of Public or Shared AMIs Malicious AMIs can include • SSH authorized keys • Rootkits • Trojaned binaries (e.g., sshd) • Open sockets (e.g., reverse shell / connect back) • Trojaned kernel • Anything! 5
AMI Exposed tool • An extensible framework for scanning AMIs for common credential leakage and security problems • Uses Amazon's APIs to automate – Generation of a list of images within scope – Launch instances of each image – Run tests via SSH session – Record findings to a database
Released under GPL at • http://www.secureworks.com/research/tools/ • https://github.com/jjarmoc/AMI-Exposed/
6
AMI Exposed: Test Modules • Ten test modules at release • Presence of SSH authorized_keys – Potential backdoor
• Presence of SSH identity keys
– Can be used to gain illicit access to other hosts
• Presence of AWS x.509 certificate (.pem) files – Can be used to tamper with publisher's EC2 account
• Active connections to other hosts – Potential backdoors
• SSH Password authentication enabled
– Potential backdoor via default passwords – Potential exposure via weak passwords
7
Scope of Testing with AMI Exposed • All US-East images
– Removed Windows images from scope – 5515 remaining in scope
• Total Non-Win AMIs 5515 – These were all processed by AMIExposed
– Skipped 771 paid images (4744 remaining) – 2767 Failed testing › › › › ›
Boot volume not found Kernel panic on boot Missing manifest files SSHD didn’t start Mostly broken and unusable images
• 1977 AMIs Completed Tests Successfully • 580 AMIs w/ findings • Findings shared only with Amazon’s security team.
– No verification or testing of recovered credentials attempted.
8
Summary of findings – SSH Keys 19.52% of tested AMIs had unknown SSH Keys
# of Findings
SSH Keys - Tested AMIs
597
761 386 1591
SSH Keys
Other
Unauthorized SSH Keys were 44.96% of all findings Found
9
Not Found
10
% of Occurrences
% of AMIs w/ findings
597 342 181 90 27 19 47 11 9 11 10 10 2 2 1358
386 336 127 65 24 18 15 11 9 3 2 2 2 2 580
43.96% 25.18% 13.33% 6.63% 1.99% 1.40% 3.46% 0.81% 0.66% 0.81% 0.74% 0.74% 0.15% 0.15%
66.55% 57.93% 21.90% 11.21% 4.14% 3.10% 2.59% 1.90% 1.55% 0.52% 0.34% 0.34% 0.34% 0.34% 29.34%
% of AMIs Completed
Unique AMIs
Finding Name Unauthorized SSH Key Exists History File(s) Exist Found Possible AWS environment variables Found Possible AWS Access Keys Found Possible AWS Account ID SSH identity key File(s) Exist Found AWS Access Keys Var AWS Certificate File(s) Exist AWS Key File(s) Exist Found Possible Canonical ID Found Variable refering to AWS Cert Found Variable refering to AWS Private Key SSH Password Authentication enabled* Vim info file(s) exist Overall
Occurrences
Summary of Findings - Overall
19.52% 17.00% 6.42% 3.29% 1.21% 0.91% 0.76% 0.56% 0.46% 0.15% 0.10% 0.10% 0.10% 0.10%
Diving in to costs
• Total month’s AWS bill $1333.16 • $.67 per AMI scanned
• $.98 per finding • $15.68 per AWS credential
• Malicious scanning would only incur costs until a valid credential is found.
11
Risks to AMI Consumers
Empirical Testing of Security Group Practices • Unofficial EC2 AMIs running BackTrack 5 released publicly on June 23rd, 2011 • Announced via • Twitter • Backtrack forums • IRC • SANS GPWN email list
• Useful to Pen Testers • Security practitioners • Students • Also useful for gathering data and statistics 12
Backtrack 5 – Phone Home Stats • 95 phone homes received as of 07/31/2011 – 69 unique Instance IDs
• When receiving a phone home, we pull an SSH banner
Port 22 checks
27
– No attempt is made to login (nor are we able to) only a banner grab
68
• 68 successful connections met with an SSH banner – 71.5% – 50 unique instances yield an SSH banner – 72% SSH Banner
13
No Connection
Phone Home Lessons • EC2 Security groups are widely configured to allow SSH access from unknown sources. • Despite targeting security minded folks, our success rate is concerning.
• A simple SSH key left on the image likely would’ve been discovered by some, but would still have had a high success rate. • General populace is likely in worse shape. • Sophisticated backdoors compiled into applications, or the kernel itself, would be extremely difficult to detect.
• When you use a public AMI, you’re placing a lot of trust in the publisher. Consider the source, and whether that trust is warranted. 14
New Guidance from Amazon • Public AMI Publishing: Hardening and Clean-up Requirements
– Published 09/07/2011 – https://aws.amazon.com/articles/9001172542712674 – Collects previously scattered recommendations into a single reference › › › › ›
Secure deletion of sensitive files (shred, srm, SDelete, etc.) Pubkey SSH Auth _ONLY_ Remove/disable passwords Remove credentials, key materials, etc. Watch for exposed services
• ec2-bundle-vol tool excludes sensitive files by default!
– '*.sw', '*.swo', '*.swp', '*.pem', '*.priv', '*.gpg', '*.jks', '*/.ssh/authorized_keys', '*/.bash_history'. – Older AMIs bundled with other tools may remain vulnerable. – Can be overridden by advanced users – http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReferenc e/CLTRG-ami-bundle-vol.html
15
Obtaining Trustworthy Amazon Machine Images?
• Amazon Web Services provides supported and maintained images – Support available with subscription to AWS Premium Support service – Security updates available via AWS package repositories – Predictable and documented Product Lifecycle and AMI updates – http://aws.amazon.com/amis
• A number of 3rd party vendors also provide their own images
• Organizations can use AWS supported and maintained images as a foundation for their own customized images 16
AWS Security Team
http://aws.amazon.com/security/
17
Prior & Related Research • Cloud Computing Models and Vulnerabilities: Raining on the Trendy New Parade
Alex Stamos, Andrew Becherer, Nathan Wilcox, Black Hat USA 2009 / DEF CON 17 https://www.blackhat.com/html/bh-usa-09/bh-usa-09speakers.html#Stamos
• Cloud Security: Amazon's EC2 serves up 'certified preowned' server images
Alan Puzic, TippingPoint DVLabs http://dvlabs.tippingpoint.com/blog/2011/04/11/cloud-security-amazonsec2-serves-up-certified-pre-owned-server-images
• CASED scientists find sensitive data of Amazon Web Services users Center for Advanced Security Research Darmstadt http://trust.cased.de/AMID
• A Security Analysis of Amazon’s EC2 Service
Eurecom technology institute, Northeastern University, and SecludIT http://www.forbes.com/sites/andygreenberg/2011/11/08/researchersfind-amazon-cloud-servers-teeming-with-backdoors-and-other-peoplesdata/
18
Thanks!
I value your feedback.
A longer version of this talk was presented at DEF CON 19 and Derbycon Videos available at; https://www.defcon.org http://www.derbycon.com AMI-Exposed source code at; https://github.com/jjarmoc/AMI-Exposed/ 19
