Deep learning is an extension of neural network. When we use multi-layer neural network, it is called Deep Neural Network (DNN). And when DNN is used to train the neural network, we call it Deep Learning.
So Deep Learning is a Machine Learning technique that employs multi-layer neural network. The following figure illustrates the concept of Deep Learning.
Although it seems very simple, it took 30 years to implement additional layer to single layer neural network and 20 more years to enhance the performance to a practical level. So in total, the development phase took 50 years! The first impediment of Deep Learning was learning rule. There was no appropriate way to train hidden layers at the beginning. In 1986, the back propagation algorithm was introduced and it became possible to train hidden layers. However, the performance couldn’t meet the expectation. That is why researchers eventually stopped working with Deep Learning. However, by the mid-20s, DNN comes into spotlight again. Nowadays, Deep Learning is the most promising machine learning technique.
