Скачать с ютуб The Lottery Ticket Hypothesis and pruning in PyTorch в хорошем качестве

The Lottery Ticket Hypothesis and pruning in PyTorch

In this video, we are going to explain how one can do pruning in PyTorch. We will then use this knowledge to implement a paper called "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks". The paper states that feedforward neural networks have subnetworks (winning tickets) inside of them that perform as good as (or even better than) the original network. It also proposes a recipe how to find them. Paper: https://arxiv.org/abs/1803.03635 Official code: https://github.com/facebookresearch/o... Code from this video: https://github.com/jankrepl/mildlyove... 00:00 Intro 00:50 Paper overview: Hypothesis (diagram) [slides] 01:33 Paper overview: Hypothesis (formal) [slides] 02:15 Paper overview: Finding winning tickets [slides] 03:44 Paper overview: Our setup [slides] 05:08 Pruning 101 in PyTorch [code] 10:29 Data - MNIST [code] 12:18 Multilayer perceptron [code] 14:05 Pruning: Linear + MLP [code] 16:45 Randomly initializing: Linear + MLP [code] 18:24 Weight copying: Linear + MLP [code] 19:51 Computing statistics [code] 20:53 Training functions [code] 24:38 CLI and training preparation [code] 27:13 Train-prune loop [code] 30:04 Grid search script [code] 31:01 Results: Actual vs desired pruning [no code] 32:47 Results: Winning tickets (parallel coordinate plots) [no code] 36:02 Results: Winning tickets (standard plots) [no code] 37:12 Outro If you have any video suggestions or you just wanna chat feel free to join the discord server:   / discord   Twitter:   / moverfitted   Credits logo animation Title: Conjungation · Author: Uncle Milk · Source:   / unclemilk   · License: https://creativecommons.org/licenses/... · Download (9MB): https://auboutdufil.com/?id=600