Recent Advance of Two-sample Testing and Its Application in AI Security

Two-sample tests ask, “given samples from each, are these two populations the same?” For instance, one might wish to know whether a treatment and control group differ. With very low-dimensional data and/or strong parametric assumptions, methods such as t-tests or Kolmogorov-Smirnov tests are widespread. Recent work in statistics and machine learning has sought tests that cover situations not well-handled by these classic methods, providing tools useful in machine learning for domain adaptation, causal discovery, generative modeling, fairness, adversarial learning, and more. In this talk, I will introduce one advance in the two-sample testing field: Two-sample testing under high dimensionality. I also present how to use advanced two-sample tests to defend against the adversarial attacks, which justified the significance of two-sample testing in the AI security area.