Blog

Software failure, such as the recent outage caused by the failed CrowdStrike Falcon update, represents a significant risk, not only to the operational efficiency of organizations but also to human safety and the economy. The root cause of the CrowdStrike incident was an out-of-bounds memory read caused by the Falcon Sensor processing 20 inputs instead of the expected 21, leading to a Windows kernel crash. CrowdStrike now faces a long list of victims threatening to sue the company including its own investors and businesses affected by the IT outage such as Delta Air Lines, and possibly others. This underscores the need for software vendors to thoroughly test their product before deploying it. 

The impact of software failure can range from critical vulnerabilities that expose customers to ransomware, to bugs that can result in system crashes and operational downtime. When software is integrated into critical infrastructure, the consequences of failure escalate dramatically, potentially leading to severe bodily harm, death, or catastrophic environmental or economic damage.

Furthermore, the reputational damage from such incidents can erode customer trust, leading to significant financial losses and long-term brand damage. Put simply, software vendors want to avoid pushing buggy software out to their customers. Therefore it's crucial for software vendors to mitigate risks and enhance the reliability and security of their products. This is primarily done through various types of Application Security Testing such as web-application, mobile app, and API security testing. 

In this article we will review an extensive list of Application Security Testing models to ensure that software vendors have a solid grasp of the available scope. Along the way, we will also provide links to some tools commonly used in each type of testing model.

Key Software Security Testing Models

Application Security Testing models are essential for identifying vulnerabilities and ensuring the robustness of software systems. Many software developers are familiar with SAST (Static Application Security Testing) and DAST (Dynamic Application Security Testing), which we have discussed before on the Packetlabs blog.  These two types of testing are fundamental to secure development operations, or DevSecOps. However, these broad types of application testing can be further divided into subcategories, each tailored to meet specific testing objectives.

Below is a list of various models and methodologies commonly used in software security testing:

• Unit Testing: Unit testing tests individual components or pieces of code such as functions, classes, and scripts to ensure they are functional, secure, and free from edge case bugs. It helps ensure that each part of the system works as designed in isolation before integrating them into larger systems. While developers can write their own tools for unit testing, some popular open-source tools include: JUnit (Java), NUnit (.NET), TestNG (Java), xUnit (.NET), PHPUnit (PHP), PyUnit (Python), and Google Test (C++).

• Interactive Application Security Testing (IAST): IAST combines elements of SAST and DAST by allowing a human to analyze code for vulnerabilities by interacting with the application while it is running.

• Threat Modeling: Threat modeling is a proactive security process used to identify, assess, and mitigate potential security threats and vulnerabilities during the design phase. Threat modeling involves analyzing the architecture, components, and data flows to pinpoint potential security weaknesses. Based on threat modeling assessments, security experts design defenses to reduce risk. Some common threat modeling tools include: Microsoft Threat Modeling Tool, OWASP Threat Dragon, and PASTA (Process for Attack Simulation and Threat Analysis).

• Fuzz Testing (Fuzzing): Fuzzing submits invalid, unexpected, or random data inputs to a program to identify crashes and vulnerabilities. Fuzzing may be conducted as part of a comprehensive unit test. By submitting random data to a program, function, or API, developers can gain assurance that it can handle malformed data and edge cases. Some of the most common fuzzing tools include: AFL (American Fuzzy Lop), Peach Fuzzer, Google OSS-Fuzz.

• Software Composition Analysis (SCA): SCA is essentially a static code analysis scan that seeks to identify its Software Bill Of Materials (SBOM). SBOM is a list of components and libraries within the software. Once the components are all known, they can be cross referenced to detect known vulnerabilities. Some common SCA tools include: OWASP Dependency-Check, Snyk, OWASP Dependency Track, and FOSSA.

• Configuration Management Testing: Configuration management testing ensures that software and systems are configured securely and according to best practices. Some popular open-source configuration management testing tools include: Ansible, Chef InSpec, Puppet, and CIS-CAT.

• Container Security Testing: Container Security Testing identifies vulnerabilities in containerized environments such as scanning the container's contents for known weaknesses, or assessing the container's deployment script for best practices.  Some open-source tools for implementing Container Security Testing include:  Docker Bench for Security, Anchor Engine, and Clair.

• Database Security Testing: Database Security Testing focuses on identifying security vulnerabilities such as SQL Injection attacks that could allow an unauthorized attacker to steal or modify a database's contents. Also, to be compliant with the EU's GDPR and other compliance standards such as SOC-2, PCI-DSS, and ISO-27001, database administrators must also verify that certain database tables and columns are encrypted by default to protect the personally-identifiable information (PII) of users. Common tools for database testing include SQLMap and PostgreSQL pgTap.

• Regression Testing: Regression Testing is used to ensure that recent code changes have not negatively affected existing functionalities. It is essentially a change-management testing model that is crucial for maintaining stability in software after updates or enhancements. Tools include Selenium (web-apps) and JUnit (Java) among others.

• Phased Rollout / Staged Rollout Testing: This software testing technique is built into an update deployment strategy. The update is released to a limited audience in controlled stages allowing developers to mitigate risk by gathering data on the update’s performance, catch potential errors, and assess user feedback incrementally.

• Stress Testing: Stress Testing assesses software's ability to use resources effectively, especially under extreme use conditions. It helps identify the breaking points and the maximum operational capacity of the application. Some tools for stress testing include: LoadRunner, Apache JMeter, Gatling, and Artillery.

• Zero Trust (ZT) Security Testing: ZT is a theoretical authentication framework for designing applications such that no part of the system is trusted by default, and every access request is verified. OpenZiTi is an open-source framework for building ZT network architecture.

• Mobile Application Security Testing (MAST): MAST focuses on identifying vulnerabilities in mobile applications. Tools include OWASP Mobile Security Testing Guide (MSTG), MobSF, Drozer, and QARK (Quick Android Review Kit).

• API Security Testing: Identifies vulnerabilities in APIs (Application Programming Interfaces). Typically this includes testing authentication and authorization to ensure sensitive API requests are properly protected, fuzzing to ensure the API can properly handle malformed data and edge cases, as well as SQL Injection testing. The most common tools include Postman, SoapUI, OWASP ZAP for APIs.

• DevSecOps Integration Testing: This form of testing integrates security testing into the DevOps pipeline to ensure continuous security assessment. It aims to identify and mitigate vulnerabilities early in the software development life cycle, enhancing the overall security posture of the application before deployment. Some tools include Jenkins, GitLab CI/CD, and SCA tools such as Snyk.

Conclusion

Application Security Testing is a critical role in safeguarding software vendors from reputational risks associated with bugs and vulnerabilities as well as their downstream clients from operating vulnerable software that can lead to a breach or operational downtime.

By reviewing a range of testing models—from unit testing to stress testing—and highlighting specific tools for each, software vendors can better understand the range of testing models available and implement more robust security measures.