A framework for web crawling and security scanning

Wallenberg Scholar Andrei Sabelfeld sets out with his research group to develop a new paradigm of input-aware and database-sensitive web exploration that increases both code coverage and vulnerability detection compared to today's best crawlers and scanners.

The web is a key enabler for today's ever-more digital world. Our society increasingly relies on the web to support the financial, governmental, and military infrastructure. At the same time, web applications are complex systems under attack from powerful, resourceful and motivated adversaries. The complexity of modern systems and ever-powerful adversaries make securing web applications a grand challenge.

"The biggest challenge is the complexity of modern web applications," says Andrei Sabelfeld. "Traditional web applications used to be much more static, with easily recognizable structure. Modern web applications are much more dynamic and highly interactive. They rely on code from third parties and load this code on demand, depending on the features used, where some features when triggered enable further new features. This poses a challenge for analyzing such a complex behavior and detecting vulnerabilities."

"We are in a unique position to break new ground in the area of web crawling and security scanning," Andrei continues. "Thanks to the scale of the project we will be able to achieve what is not feasible in a conventional research project, which is typically of a smaller scale. Because of the scale of the project, we will be able to apply a range of techniques from program analysis to machine learning to develop a comprehensive framework for web crawling and security scanning."