Welcome to the VB2021 conference!

When malware changed its mind: an empirical study of variable program behaviours in the real world

Erin Avllazagaj (University of Maryland, College Park), Ziyun Zhu (Facebook), Leyla Bilge (NortonLifeLock Research Group), Davide Balzarotti (EURECOM) & Tudor Dumitras (University of Maryland, College Park)
partner message

QI-ANXIN Technology Group Inc. Leader of New Generation Cybersecurity


QI-ANXIN Technology Group Inc. offers next generation enterprise-class cybersecurity products

and services to government and businesses. QI-ANXIN is the Official Cyber Security Services and

Anti-Virus Software Sponsor of the Olympic and Paralympic Winter Games Beijing 2022.

partner message

Farsight Security DNSDB® is the world's largest real-time and historical database of DNS resolutions


DNSDB 2.0 introduces Flexible Search support, unlocking both Regular Expressions and Globbing syntaxes for more granular and accurate search results. Get your free DNSDB API key and use it in our newly updated

web GUI, DNSDB Scout as well as your own existing environments. Contextualize everything that is DNS related with one API key - DNSDB.

partner message

IoT security begins with your Smart TV


CHOMAR Smart TV Security.

Protect your Android Smart TV against malicious activities and use your IoT devices without any worries.

partner message

VirusTotal: Actionable crowdsourced threat intelligence


Comprehensive context and cutting-edge functionality to proactively protect from cybersecurity threats.

partner message

Ransomware prevention starts with zero


Ransomware attacks are increasing 500% year-over-year.

Learn how Zscaler's Zero Trust Exchange helps minimize exposure, damage, and risk

at every stage of a ransomware attack.

partner message

Stay ahead of threats with VirusTotal


Stay ahead of the next generation of threats and get relevant insights to solve

the most critical security challenges.

partner message

F5 helps find malware hiding in plain sight


Encrypted malware is becoming increasingly common, and daisy-chaining security devices is neither

cost-effective, nor efficient. Detecting and stopping malware doesn’t have to be overwhelming with

F5’s innovative products.

partner message

Threat Intelligence and Cyber Resilience


Join the VB2021 Threat Intelligence Practitioners' Summit (TIPS), sponsored by the Cyber Threat Alliance,

and learn how investment in threat intelligence builds cyber resilience, allowing you to be more effective

when addressing today's dynamic threat landscape.

partner message

Do you like doing work that matters to you… and really frustrates the bad guys?


At Talos, our mission is to make the internet a safer place and fight the good fight for our customers

and users. If you think you have the expertise and attitude to help lead the world in cutting-edge security,

we’d like to talk.

partner message

Calling all Hackers!


We are hiring mid-senior-principal level hackers!

Remote option • Flex schedule • Unlimited vacation • Opportunities for research and publication

partner message

Looking for performance validation for your product?


Get an edge over your competitors with Virus Bulletin’s anti-malware & email security certification

programmes, supported by 30+ years of experience. Or take advantage of our bespoke testing service

offering valuable performance feedback for R&D. Email [email protected].

partner message

Amazon Information Security - come build the future with us!


Do you want to work on privacy and security challenges at unprecedented scale?

We have Privacy and Information Security opportunities available now across

the United States, Dublin, Ireland, and Sydney, Australia.

partner message

We don’t just talk about sharing. We do it every day.


Learn how to collaborate with the Cyber Threat Alliance to improve your overall cyber resilience.

We are a greater team when we work together; our collective efforts magnifies our success and

ensures that we are and remain cyber resilient.

Behavioural program analysis is widely used for understanding malware behaviour, for creating rule-based detectors, and for clustering samples into malware families. However, this approach is ineffective when the behaviour of individual samples changes across different executions, owing to environment sensitivity, evasive techniques or time variability. While the inability to observe the complete behaviour of a program is a well-known limitation of dynamic analysis, the prevalence of this behaviour variability in the wild, and the behaviour components that are most affected by it, are still unknown. As the behavioural traces are typically collected by executing the samples in a controlled environment, the models created and tested using such traces do not account for the broad range of behaviours observed in the wild, and may result in a false sense of security.

In this paper we conduct the first quantitative analysis of behavioural variability in Windows malware, PUP and benign samples, using a novel dataset of 7.6M execution traces, recorded in 5.4M real hosts from 113 countries. We analyse program behaviours at multiple granularities, and we show how they change across hosts and across time. We then analyse the invariant parts of the malware behaviours, and we show how this affects the effectiveness of malware detection using a common class of behavioural rules. Our findings have actionable implications for malware clustering and detection, and they emphasize that program behaviour in the wild depends on a subtle interplay of factors that may only be observed at scale, by monitoring malware on real hosts.

Got a question about this presentation? To get in touch with the speakers, contact Erin Avllazagaj on Twitter at @albocoder.
Erin Avllazagaj
University of Maryland, College Park

Erin is a third-year Ph.D. student in the Department of Electrical and Computer Engineering at the University of Maryland in College Park, advised by Prof. Tudor Dumitraș. He received his B.S. in computer science from Bilkent University (Ankara, Turkey) in 2018. Erin's broad research interests cover data-driven malware analysis and automatic exploit generation. Specifically, in his recent Ph.D. work he has analysed executions of malware traces in the real world to derive guidelines for creating effective behaviour-based detection systems. Erin is currently interested in automatic exploit generation for heap-based exploits. His participation in various CTF competitions and his internship work have been major influences in this new research direction.

Ziyun Zhu

Ziyun Zhu is a Research scientist at Facebook in Greater New York Area.

Leyla Bilge
NortonLifeLock Research

Leyla Bilge is technical director and leads the branch of the research team that resides in Europe.

Davide Balzarotti

Davide Balzarotti is a Professor (Professeur des université) at the EURECOM Graduate School and Research Center, located in Sophia Antipolis on the French riviera. His research interests include most aspects of system security and in particular the areas of binary and malware analysis, reverse engineering, computer forensics, and web security. Davide is a recipient of an ERC Consolidator Grant which focuses on the analysis of compromised systems. Davide is a member of the Order of the Overflow – the team which organizes the DefCon Capture the Flag competition. Before that, he was one of the founding members of the Shellphish hacking group, with whom he participated in ten DEFCON CTF finals in Vegas (winning in 2005). When he was a post-doc in the security group at UCSB, he also helped to organize several early editions of the iCTF competition. When not in front of his computer, Davide likes to climb rocks, surf waves, hike trails, and take pictures along the way.

Tudor Dumitras
University of Maryland, College Park

Tudor works on data-driven security. His research objective is to provide an evidence-based foundation for security, by building defences grounded in a rigorous understanding of real-world adversaries. Tudor conducts empirical studies of adversary behaviour, builds machine learning systems for detecting malware and attacks, and studies the security of machine learning in adversarial environments. He also has a good knowledge of the security industry, having worked for 2.5 years at Symantec Research Labs. There,he built WINE, one of the first platforms for sharing field data collected by the security industry with academic researchers. In his most cited paper he measured how long zero-day attacks go undiscovered in the wild; this measurement was made possible, for the first time, by the WINE platform. Tudor's research has been featured in the Research Highlights of the Communications of the ACM and has been widely cited in the media, for example in The Economist, the MIT Technology Review, Forbes, and The Register. He also enjoys giving TED-style talks, to explain his work to broad audiences. Tudor has a Ph.D. from Carnegie Mellon University and undergraduate degrees from the Ecole Polytechnique and the “Politehnica” University, Bucharest.