The Thieves on Sesame Street are Polyglots Extracting Multilingual Models from Monolingual APIs

Pre-training in natural language processing makes it easier for an adversary with only query access to a victim model to reconstruct a local copy of the victim by training with gibberish input data paired with the victim’s labels for that data. We discover that this extraction process extends to local copies initialized from a pre-trained...More