Deep Learning has created too much hype. Lots of people talk about it but how many do exactly understand how it works?

In this post, I will share some resources that helped me better understand and explain how neural networks work under the hood.

1 — Some training slides ??‍??

This is a deck of slides I created for a deep learning training session I made for my colleagues.

It’s meant to be practical since it covers lots of things in ~3 hours. It’s a good starting point if you’re new to the field and want to have a general overview of the subject matter.

Here’s, as a sneak peek, a small portion of it in a video format that illustrates the forward and backward passes in a neural network.
Do not hesitate to pause it or replay it to understand what happens.

Now here’s what I’ll cover in the slides:

• Some Deep Learning applications through different industries
• The concept of the artificial neuron
• The perceptron
• The multi-layer perceptron
• Activation functions
• Loss functions
• The Gradient Descent algorithm
• How to formalize the training pipeline of a neural network
• Tips and tricks to avoid overfitting

You can view the slides here:

2 — Code and notebooks ??

The last section of the previous slides is hands-on. If you follow it, you’ll build a fully functioning trainable neural network from scratch. In pure python code only, with no frameworks involved.

Once you’re done, you will test your baby Neural Network on some complex non-linear classification problems and thanks to an interactive visualization you’ll find in the notebooks, you will see it learning and improving live, in front of you!

All the code is documented in my Github repository:

I encourage you to pull it or fork it, deep dive in it, understand it and reproduce it by yourself.

If you have any questions regarding it, do not hesitate to submit an issue.

If you want to pimp it up by adding a new feature, feel free to send me a pull request.

3 — Some other kickass resources ????

Deep learning is a fascinating subject. While learning about it, I came across very good papers, tools, and blog posts.

Here’s my non-exhaustive list for you:

• Neural networks and deep learning

Inarguably the most detailed and precise introduction to neural networks. It’s a very well written online book that has everything demonstrated.

• Andrej Karpathy lectures’ notes about deep learning at Stanford

Great material. Detailed and straight to the point. The notes about neural networks are split into 3 parts.

A general introduction to neural networks with an analogy to biology

Notes about data pre-processing, regularization and loss functions

Notes about efficient training and hyperparameter optimization

• Tensorflow Playground

Tensorflow Playground is a web application that runs a Neural Network on your browser. It allows you to quickly set the parameters of the network (the number of layers, the number of neurons, activation functions, etc.) as well as the hyperparameters (learning rate, regularization rate, etc.) before launching a training.

This will eventually help you build an intuition about the role of each parameter and how they all interact with each other.

• The Deep Learning Book by Ian Goodfellow, Yoshua Bengio and Aaron Courville

This is most certainly the holy bible of every deep learning practitioner.

It’s certainly math-heavy but it’s got everything you need if you want to dive into the mathematical demonstrations. It’s the best resource to come back to in time of doubt.

You can find the PDF chapters of this book here.

• MIT 6.S191 Introduction to Deep Learning

This is MIT’s official introductory course on deep learning methods with applications in gameplay, robotics, and more!

Thank you ??

I hope you’ll find these resources valuable to start your deep learning journey!

There are certainly tons of other great papers and blogs out there but these should keep you busy, at least for the beginning.

Do not hesitate to share some other links to complete the list!

Originally published at https://www.ahmedbesbes.com.