______                     _ _                     _____
        | ___ \                   (_| |                   /  ___|
        | |_/ /___ _ __   ___  ___ _| |_ ___  _ __ _   _  \ `--.  ___ __ _ _ __  _ __   ___ _ __
        |    // _ | '_ \ / _ \/ __| | __/ _ \| '__| | | |  `--. \/ __/ _` | '_ \| '_ \ / _ | '__|
        | |\ |  __| |_) | (_) \__ | | || (_) | |  | |_| | /\__/ | (_| (_| | | | | | | |  __| |
        \_| \_\___| .__/ \___/|___|_|\__\___/|_|   \__, | \____/ \___\__,_|_| |_|_| |_|\___|_|
                  | |                               __/ |
                  |_|                              |___/

Repository Scanner

The Repository Scanner (RESC) is a tool used to detect secrets in source code management and version control systems (e.g. GitHub, BitBucket, or Azure DevOps). Among the types of secrets that the Repository Scanner detects are credentials, passwords, tokens, API keys, and certificates. The tool is maintained and updated by the ABN AMRO Bank to match the constantly changing cyber security landscape.

The Repository Scanner was created to prevent that credentials and other sensitive information are left unprotected in code repositories. Exposing sensitive information in such a way can have severe consequences for the security posture of an organization. An attacker can use the data to compromise the organization's network. This can be prevented by scanning a repository with the RESC tool. It marks all the instances of exposed sensitive information in the source code.

📒 Table of contents

• Links
• Versions
• Technical information
• Getting started
• Contributing guidelines
• Contact
• License
• Acknowledgments

🔗 Links

Throughout the process of open sourcing this project, the ABN AMRO Bank created a series of articles that describe the capabilities of the Repository Scanner (RESC) tool, the architectural decisions behind it, and the road to open sourcing RESC. With the articles, users can look "behind the scenes" and gain a deeper understanding of the tool.

⚙️ Versions

The SemVer numbering is used for the releases of the Repository Scanner tool. A version consists of a MAJOR.MINOR.PATCH number (e.g. v1.2.4):

• The MAJOR part is reserved for updates that:

  • alter the flow and behavior of the tool.
  • noticeably impact the user experience.

• The MINOR part of the number focuses on:

  • big new features.
  • feature updates.
  • other newsworthy changes in the tool.

• The PATCH part captures all other changes:

  • small content edits.
  • bug fixes.
  • information corrections.
  • updated broken links.

Major releases

Every notable release of the Repository Scanner tool, the changes that come with the release, and the release date are listed below. The full list of all releases can be found on the Releases page:

• v1.0.0 – Initial release of the Repository Scanner:
  • Able to scan for secrets in repositories.
  • Complete overview of the found secrets through the User Interface with a direct link to the issue.
  • Able to sort the findings by secret type (tokens, certificates, passwords, etc.).
  • A statistics page that gives an overview of findings and how they increase/decrease monthly along with the amount of times a finding appears.
  • The ability to import your own rulepack with custom rules to fit your personal means.

🛠️ Technical information

The technologies that the Repository Scanner Tool is built on is listed below. There is also a list with direct links to the individual components of RESC.

• 
• 
• 
• 
• 
• 

VCS Scanner Worker flow diagram

The flow diagram below shows the different stages that a VCS Scanner Worker goes through and the choices it is confronted with to come to the desired result. It first picks up a branch from the queue where it is decided, in conjunction with user input, what type of scan to run. If it is a base scan, a full scan of all commits will be carried out to look for secrets. Possible findings are stored inside the database along with the last scanned commit hash of the branch. An incremental scan, where the branch was scanned earlier, only looks at the commits that were made after the last scanned commit hash. The process of finding secrets and storing them in the database is similar as previously described.

RESC high-level overview

The diagram below gives a high-level overview of the Repository Scanner tool. All the different components of the tool and the technologies that it utilizes are explained in detail here. As shown in the diagram, all the components mentioned are run as Docker containers in a Kubernetes ecosystem.

• RESC-Frontend: The RESC-Frontend is a fully responsive dashboard application developed using Vue.js 2 and the BootstrapVue framework. It has screens for Analytics, Repositories, Scan Findings, Rule Analytics, and Rule Pack.
• RESC-Backend: The RESC-Backend is the backend of the Repository Scanner tool. The RESC-Backend consists of RabbitMQ users and queue creation, Database models, the RESC Web service, and Alembic scripts for database migration. The RESC Web service is created using FASTAPI.
• RESC-VCS-Scanner: RESC-VCS-Scanner, which runs as a celery worker, gathers repositories from the repositories queue and carries out a secret scan. Gitleaks is used as the scanner to find secrets.
• RESC-VCS-Scraper: All projects and repositories from supported VCS providers such as Bitbucket, Azure Repos, and GitHub are gathered by the RESC-VCS-SCRAPER. This component contains the VCS-Scraper-Projects and VCS-Scraper-Repositories as its primary modules.

🛠️ Getting started

The instructions below help you to get a copy of the project up and running on your local machine for development and testing purposes.

• Running RESC
  Deployment through helm

• Running individual components
  RESC-BACKEND
  RESC-VCS-SCRAPER
  RESC-VCS-SCANNER
  RESC-FRONTEND

💁🏽 Contributing guidelines

We believe that innovating together can lead to the most incredible results and developments. Contributions to the Repository Scanner tool are therefore highly encouraged. We have created guidelines that we expect contributors to the project to follow. By contributing to the project you also agree with our Code of Conduct.

📧 Contact

If you need to get in touch with the maintainers of the Repository Scanner tool, please use the following e-mail address: resc@nl.abnamro.com.

⚖️ License

The Repository Scanner (RESC) Tool is licensed under the MIT License.

🎉 Acknowledgements

Since the Repository Scanner (RESC) makes use of GitLeaks, we want to give Zachary Rice credits for creating and maintaining GitLeaks. GitLeaks has helped many organizations in securing their codebases for any leaked secrets.