Quality assessment of static analyzers for containerized applications

Modern software development and operations heavily depend on container technologies. With reduced overhead and faster deployment, containerization presents new challenges regarding its security and the need for in-depth defence against cyber-attacks. Vulnerability detection is one of the components that can provide additional security and prevent system compromise.In this study, we evaluate the quality of static analysis tools in the context of containerized applications. We query more than 3,000 images from Docker Hub and analyse them using various scanning tools, including open-source (Trivy, Clair, Anchore Grype and Snyk) and commercial (Docker Scout), as well as cloud-based ones (Google Container Registry). To compare the accuracy of the vulnerability reports generated by each scanner, we introduce quality assessment metrics including precision and recall. We also attempt to explain discrepancies in the results and explore the distribution of detected vulnerabilities among the packages and images.To the best of our knowledge, previous studies in the field of static vulnerability detection for containers have failed to analyse a sufficient number of images, and mainly focus on no more than a few open-source scanners. Our findings may be useful for security professionals as well as developers of static analysis tools.