Electrical Engineering
      and Computer Sciences

Electrical Engineering and Computer Sciences


UC Berkeley


2008 Research Summary

Security of Adaptive Systems (SecML)

View Current Project Information

Marco Antonio Barreno, Blaine Alan Nelson, Benjamin I. P. Rubinstein, Anthony D. Joseph, Doug Tygar, Udam Saini, Kai Xia and Jack Chi

Machine learning is becoming prevalent in the systems domain as a detection and analysis tool for problems amenable to adaptive techniques. However, the adaptivity and flexibility that are machine learning's biggest assets are also qualities that an attacker might exploit. Thus, it is important to study the security of learning systems [1,2].

We are investigating the vulnerability of real-world learning systems, primarily in the context of the open-source, naive-Bayes-based spam filter SpamBayes [3]; we are exploring how various avenues of attack might exploit the learning algorithm's adaptability to compromise the system.

In [4], the authors use PCA for detecting anomalous point-to-point flows based on link volume data. We are investigating the effect an adversary can have on the normal subspace of link volume vectors learned under various realistic models of control.

We also investigate security as a property of families of learners. Universal sequence prediction [5] considers the loss of a learner in the presence of an adversary. This approach is appropriate for security, as the adversary is modeled in a general way. Robust statistics is another appropriate framework, which quantifies the effect of outliers. For security, it is important to quantify the cost of an attack, possibly in the presence of non-adversarial data.

M. Barreno, B. Nelson, R. Sears, A. D. Joseph, and J. D. Tygar, "Can Machine Learning Be Secure?" ACM Symp. Information, Computer and Communications Security, 2006.
B. Nelson and A. D. Joseph, "Bounding an Attack's Complexity for a Simple Learning Model," First Workshop on Tackling Computer Systems Problems with Machine Learning Techniques, 2006.
T. A. Meyer and B. Whateley, "SpamBayes: Effective Open-Source, Bayesian Based, Email Classification System," Conf. Email and Anti-Spam, 2004.
A. Lakhina, M. Crovella, and C. Diot, "Diagnosing Network-Wide Traffic Anomalies," ACM SIGCOMM Computer Communication Review, Vol. 34, No. 4, 2004.
N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games, Cambridge University Press, 2006.