CrowdStrike recently conducted an investigation for a client operating in the healthcare sector that was subject to an ongoing phishing scam focused on harvesting credentials for cloud email providers such as Google (Gmail), Yahoo, and Hotmail. This is the second post in a two-part series focusing on evidence mechanisms used to investigate the activity outlined in “More than just your eSignature.” From a technical perspective the campaign was unsophisticated, however, when analyzing tradecraft, the attackers architected a phishing campaign with an enduring strategy that evolved faster than domain registrars and defenders could track.
Investigating cloud resources adds yet another layer of complexity in understanding the intricacies of an intrusion. Third party cloud services often facilitate trivial implementation, but from a security perspective it means data must be fused across multiple networks in order to paint a comprehensive timeline of events. Individually, each evidence source might provide circumstantial insight into an event, but corroborating multiple data points around the same event takes the probability of evil from circumstantial to high confidence.
In this case, the attacker used each victim’s inbox as a platform for propagating their attack and achieving their objectives. Once the adversary logged into a victim email account, they covered their tracks by erasing forensic evidence of the phishing campaign in an attempt to curb suspicion from the victim. Erased data included the original phishing email, trash, sent items, and indicators from Google mail that would have alerted the user to a suspicious login from an unrecognized device.
Separating the Wheat from the Chaff
With an attacker determined to obfuscate his activity through anti-forensics techniques and using legitimate credentials to access the victim account two investigative methodologies were critical separating legitimate user activity with attacker activity: pattern-of-life analysis and least frequency of occurrence.
Pattern-of-life analysis is a method of surveillance specifically used for documenting or understanding a user’s (or many users’) habits. This information can then be used to predict future actions or observe anomalies deviating from normal user activity. Because the attacker logged into the victims account using legitimate credentials, pattern-of life analysis was used to separate attacker from legitimate user activity. Knowing that our victim was located on the US east coast and analyzing their habitual routine over a period of time, CrowdStrike was able to detect anomalies in account activity that was not legitimate user activity. Specifically, logon times were used to formulate conclusions about what was or was not legitimate user activity. During the investigation we noticed a grouping of successful logons occurring at approximately 0100 to 0200 US EST time into the victims account. While this seems like an easy win, it is not that uncommon for devices like smart phones or iPads to sync to an inbox in the middle of the night. With 6 months of log data stored by Google, we were able to determine what normal user activity looked like over a period of time and support that analysis with least frequency of occurrence. Using least frequency of occurrence analysis of connected IP addresses, browsers, and devices we were able to better draw conclusions about anomalous artifacts. Drawing conclusions based on behavior required the right data points. Two log sources stored by Google proved particularly useful for the investigation: administrative auditing and individual account logs.
Google’s admin console makes it easy to examine potential security risks, measure user collaboration, track who signs in and when, analyze administrator activity, and much more. The reports have interactive graphics and tables that show broad, domain-level data alongside granular user-level details. These reports can be accessed by any super-user level administrator and are a great source of log evidence. Table 1 shows individual logging stores and the approximate storage duration.
|Name of Log or Report||Retention Time|
|Audit Log||6 months|
|Calendar audit log||6 months|
|OAuth Token audit log||6 months|
|Drive audit log (Google Apps Unlimited)||6 months|
|Account activity reports||6 months|
|Security reports||6 months|
|Groups audit log||6 months|
|API audit data||6 months|
|API reporting data||15 months|
Armed with the knowledge that some of the evidence in the affected account had been removed, we next tested the hypothesis that the attacker may have logged into the victim account directly to remove artifacts. Google’s login report records this type of activity. Metadata captured is the username, event name, source IP address, and date/ timestamp of each login. It also records events such as failed and successful logons, reasons, and account logouts. This information is similar to data located in Windows security event logs, specifically event IDs “4624” and “4625.” Analysis of login events showed one suspicious login from 01:18 to 01:50, however, this evidence was circumstantial and we needed more data points to increase our confidence level.To investigate the intrusion, we reviewed three administrative logs that provided insight into attacker activity. Since the account in question during this investigation belonged to a super-user administrator, the first log source reviewed was Google’s administrator audit logs. Compromising a super-user account not only gives the attacker access to one account but permissions to access, change, and enforce policies across all applications and users in the Google enterprise account. The metadata provided by this data source includes the event name, event description, administrator account name, date/ timestamp, and IP address for each login. Some of the available events tracked are: email log searches, user creation, temporary password viewed, role assigned, two step verification, etc. In this case, there was no evidence in these logs that the attacker made any changes at an enterprise level.
CrowdStrike’s assessment of the campaign showed that the attacker used the victim’s contact list to propagate their campaign as well as to siphon financial data. When examining the contents of the victim’s inbox, trash, and sent items, we discovered no evidence of any sent or delivered emails during the suspicious login – unfortunately the attacker had already cleaned these folders. To fill in the gaps, we turned to Google’s email trace logs, part of Google’s audit logging feature. Metadata captured by this source is the subject line, date, sender, recipient, status, and direction (received, sent, mixed). The email trace logs capture all emails sent and received across users in the enterprise. After analyzing the trace logs, we discovered that an email was sent by Google at 01:18 to the victim account alerting them that an unrecognized device had been used to login. This evidence further corroborated our findings of the suspicious logon. In addition, we were able to report good news to the client: no other emails were sent or received by the victim during that time period.
The hypothesis that an attacker accessed the account from 01:18 to 01:50 with a logon from an unrecognized device was starting to become more certain. It was time to examine log data associated with the affected individual account to see if some device metadata could be recovered. CrowdStrike turned to Google’s account security logs located at “security.google.com.” Metadata captured by this source is a list of devices currently signed in to the account, or that have accessed an account in the last 28 days. The log tracks details about the last activity from each device, including the last time, location, and browser type used to access the account. The graphic in Figure 1 shows an example of device logging.
By reviewing recent device metadata, we determined that the login occurred from a location where the client was not present, using a device and browser not utilized by the client workstation build. Putting all of the information together, we determined that an attacker logged into the account from 01:18 to 01:50, conducted preliminary cleanup of the account, and did not use the account to propagate the attack by sending or receiving emails
Phishing campaigns like the one discussed in this post are commonplace. Adversaries motivated by financial gain augment their tradecraft staying one step ahead of investigators. Luckily, when analyzing accounts hosted by cloud email providers, multiple log sources are available to investigate account activity. Even after data has been manipulated by an attacker, information can often be fused from multiple sources to build a high confidence timeline of events.