Drawing tickets from a canvas

Ongoing Project

Abstract

In this work, we analyze more efficient way of drawing winning tickets using importance sampling in an anytime learning regime. We further aim to do conclusive studies on the uniqueness of the winning tickets discovered by the algorithm in comparison to conventional pruning and LTH techniques.

References:

[1] Frankle, Jonathan, and Michael Carbin. “The lottery ticket hypothesis: Finding sparse, trainable neural networks.” arXiv preprint arXiv:1803.03635 (2018).

[2] Paul, Mansheej, Surya Ganguli, and Gintare Karolina Dziugaite. “Deep Learning on a Data Diet: Finding Important Examples Early in Training.” Advances in Neural Information Processing Systems 34 (2021).

[3] Caccia, Lucas, Jing Xu, Myle Ott, Marc’Aurelio Ranzato, and Ludovic Denoyer. “On Anytime Learning at Macroscale.” arXiv preprint arXiv:2106.09563 (2021).

Participants