Organizations are feeling an increased sense of urgency to ensure the security of their built applications by putting in place cybersecurity protocols. However, as they enable security analyzers on source code and related assets, they find that the amount of data they are getting in terms of potential vulnerabilities is overwhelming to development teams. This article provides seven steps to take to implement scans without dramatically slowing down developer velocity. This tutorial is based on my work with customers who have experienced this dilemma.

The need for increased application security

Application security has become a greater a focus for organizations in part because of the following reasons:

• Recent high-profile cyber attacks have had major business impacts, including long outage windows, stolen data, and ransoms being paid.
• The U.S. government released a series of executive orders concentrated on software supply chain security and now requires that vendors supply a software bill of materials (SBOM) that lists the ingredients in terms of open source libraries that make up a software application.
• Compliance standards such as SOC2, which mandate companies to validate the security of their applications, are becoming more common in many industries.

One way to meet this demand for increased application security is by enabling security scans, but that can be an overwhelming proposition. Consider, for example, a technology startup with Series A funding building a SaaS application. Security scans are crucial for compliance, but so is optimizing the velocity of new feature development. These goals can seem contradictory, at times. Organizations in this situation are often uncertain about what approach to take to ensure compliance standards are met and their applications are as secure as possible without bringing developer velocity to a halt.

How to keep development fast and secure

Here are the steps to take to ensure that your development can keep pace while still meeting compliance and security requirements.

Step 1: Assess the current state of development

Your starting point with security scans is going to be strongly dependent on the details of the applications you build, in terms of both the industries and customer base you serve, and the frameworks, languages, and cloud technologies used to build and deploy the application. A company producing an embedded device, for example, will have a completely different set of concerns than a company producing a SaaS application. It's important to bring together your technology leaders, cybersecurity experts, business executives, and others with the appropriate business and technology expertise to understand:

• the compliance standards that apply to the applications you're producing
• the types potential vulnerabilities that present the biggest threat based on industry research and your application and network architecture

Knowing these characteristics will assist in your development of your threat landscape: your application's identified and potential cyberthreats.

If you have a variety of applications, start small and test your scanning and vulnerability management program with a few pilot applications. Choose the applications that are the most critical and/or will provide the best validation and feedback to inform enhancements to your application security strategy.

This assessment will help identify what security scans you want to enable for these pilot applications.

Step 2: Enable security scans

Next, you'll want to conduct initial scans of the pilot applications you identified in Step 1. The initial scans should be done outside of development pipelines so that you do not distract the developers who are implementing new features and bug fixes. In GitLab, you can use scan execution policies to enable scheduled scans of the default branch.

You should only enable the highest priority scans based on the threat landscape you identified in Step 1. While every customer will be slightly different, GitLab customers commonly start with secrets detection and dependency scanning at a bare minimum.

Step 3: Evaluate scan results

Once you have the results of initial scans for pilot applications, you will want to analyze the results.

Are there vulnerabilities that are false positives based on your specific application architecture? One example of this might be a dependency scan finding vulnerabilities in libraries that are only used in a development environment as part of a test framework but are not used for the build of the production application. These vulnerabilities can be safely ignored.

Are there vulnerabilities that are just “noise” and not really important? Many static application security testing (SAST) scanner default rule configurations will find hundreds of low severity vulnerabilities in source code that you might decide are not the most important thing to focus on right now.

Finally, based on your knowledge of the applications, and the risk assessments performed, are there areas of potential vulnerabilities that have not been uncovered with the existing scans? You’ll want to understand where gaps lie that might need additional scanners or manual assessment.

Step 4: Adjust scan analyzers and rules used

Use the evaluation of the initial scan results to define processes for vulnerability triaging and remediation. Then adjust the scanner rules to only focus on what’s most important. This is a crucial step to ensure that development velocity does not dramatically slow down due to development teams having many low-priority vulnerabilities to sort through as they implement code changes. For example, a SAST analyzer rule might find critical vulnerabilities in a code that runs on an IoT device. That device, however, has other security controls that effectively mitigate exploitability of a latent vulnerability in the embedded code. In this case, thevulnerabilities are not of concern and you can safely disable the rule.

Similarly, if you find that your testers commonly use personally identifiable information in test data that sometimes inadvertently gets committed to the code repository, then you will want to create a custom secrets detection rule that looks for the appropriate string patterns for this PII data.

GitLab allows you to customize rulesets for SAST and secrets detection. As you implement custom rulesets, be sure to clearly document rule customizations and rationale for each and review them periodically.

Step 5: Prioritize initial vulnerabilities for remediation

Business, product, and development leaders should agree on the importance of remediating high priority vulnerabilities and convey that importance to development teams. It is crucial for development teams to plan to work on the highest priority vulnerability remediation along with feature work. Vulnerabilities with a clear and practical attack vector should be prioritized and added to the backlog. Vulnerabilities that are not a priority for remediation can be left open in the confirmed state or be dismissed as an acceptable risk or a false positive. Add notes to capture the dismissal reason for auditing purposes. This prioritization strategy is one more step that can prevent slowdowns in developer velocity.

Step 6: Enable scans in development pipelines

After completing the five steps above, you now have enough information to formalize your security scan program and bring the developers for the pilot applications into the fold. You'll want to start to drive developer participation in application security sooner rather than later, but take an iterative approach. Start small by introducing the highest priority scans, with any custom configurations identified in the initial scans, into development pipelines via triggered jobs defined in scan execution policies. Be sure to configure scan jobs to run in parallel with other CI jobs you've already defined for your application.

Monitor pipeline times to ensure scans are not dramatically slowing down time to completion, and configure any extremely slow scans such as dynamic applications security testing to run on a scheduled basis instead of being triggered on commits to a feature branch. This will ensure that developers are still getting fast feedback as they are working on code changes for new features and bug fixes.

As much as possible, you will want to start with providing developer visibility of found vulnerabilities first without adding any enforcement or blocking of the merge of code changes. The GitLab merge request widget shows new potential vulnerabilities found in the feature branch to provide visibility to developers of the security impact of their code changes. If certain critical vulnerabilities are becoming commonplace, consider enabling merge request approval policies only for newly found critical vulnerabilities to put some guardrails in place without requiring too much extra process that slows down developers.

At the same time, educate the developers to understand what the security vulnerabilities mean and how to remediate them. GitLab integrates with the security training providers Secure Code Warrior, Kontra, and SecureFlag to help your developers learn how to fix vulnerabilities, providing links on the vulnerability details page to the appropriate training resources that match a vulnerability identifier.

Lastly, drive a collaborative environment between development, security, and operations. Application security should be a joint initiative with clear priority across all roles. By helping developers understand potential vulnerabilities early in the software development lifecycle and remediate them when warranted, they will spend much less effort than having to remediate them at later stages, and that will allow teams to continue to have capacity for new feature development.

Step 7: Rinse and repeat

Congratulations! You now have a security scan and vulnerability remediation program enabled for a set of pilot teams. But the job is not done. You will want to focus on continuous improvement – incorporate what you learn from the pilot teams and iterate. Then, enable a second wave of teams, following all of the defined steps. You'll want to continue with small sets of teams until security scans have been enabled across all relevant applications.

Read more

• Getting started with application security documentation
• How to tailor SAST and secret detection to your application context with custom rulesets
• Webinar: Starting a Vulnerability Remediation Program
• Video tutorial: Creating security policies - the basics (3:06)

Watch the on-demand "Vulnerability Management Strategies" webinar and review the accompanying slide deck.