Reversing complex software quickly is challenging due to the lack of professional tools that support collaborative analysis. The CrowdRE project aims to fill this gap. Rather than using a live distribution of changes to all clients, which has proven to fail in the past, it leverages the architecture that is being used with success to organize source code repositories: a system that manages a history of changesets as commit messages. The central component is a cloud based server that keeps track of commits in a database. Each commit covers one or more functions of an analyzed binary and contains information like annotations, comments, prototype, struct and enum definitions and the like. Clients can search the database for commits of functions by constructing a query of the analyzed binary’s hash and the function offset. Different concurring commits for a function are possible; in such cases it is up to the user to decide which commit is better.
This basic concept is sufficient for a collaborative workflow on a per-function basis for a shared binary. One exciting feature is a similarity hashing scheme that considers the basic block boundaries of a function. Each function is mapped on a similarity preserving hash of fixed size. A database query for such a functions similarity hash returns a set of functions sorted by their similarity value, and the analyst can choose amongst them. This is extremely helpful when analyzing variants based on the same code or generations of a malware family, for example.
Another interesting feature is the synchronization of used types. It is customary for reversers to document C++ class structures and vtables in IDA structs. This also greatly enhances the Hex-Rays decompiler output. CrowdRE will automatically identify all referenced types from a function (being it function parameter types or local variable types). Those types are then bundled with your commit. When annotations are pulled from the cloud, it will automatically also offer to pull the referenced types of that specific commit (with the definition at the time of the commit). If you already have a local type with a colliding name and its definition differs, CrowdRE will offer you various conflict resolution strategies.
The CrowdRE client is now freely available as an IDA Pro plugin. CrowdStrike maintains a central cloud for the community to share their commits amongst each other. It is our goal to help build a public database of known, well annotated functions to speed up the analysis of standard components, somewhat similar to what BinCrowd (which is offline nowadays) offered but with support for multiple co-existing commits for the same function. We will also support list-based commit visibility to give users control over who else can see and import their contributions. In the coming days we will release a series of how-to blog posts and videos to explain different use cases of CrowdRE. CrowdRE continues to be used heavily by the CrowdStrike Intelligence and Services Teams and we look forward to sharing out our commits to help the community reverse as a crowd! Please contact CrowdRE@crowdstrike.com for more information or to provide us feedback. This is an alpha version and we’d love your comments on how we can improve upon it!
We originally developed CrowdRE for internal use but decided to release it for free when we realized that the broader security community can benefit from it as well. In addition, we think it’s important to encourage information sharing and collaboration among the security industry. Our adversaries are collaborating, shouldn’t the security community do the same?