Backpropagation of Simple Expression
Understanding Backward Pass with Backpropagation of Simple Expression.
This blog is 1 of the 3 blog series for the understanding of Backpropagation in detail.
- It is simply a method for calculating derivatives and with the help of these derivatives, we can reduce the loss of the model and maximize the accuracy. As we know backward pass is mainly termed Backpropagation.
- Backpropagation is simply a derivative of the final output with respect to its leaf nodes. concerningerm leaf node, I mean all those parameters which are not fixed from start. Example — Weights.
- The intuition behind the manual creation of forward pass and backward pass came from Andrej Karpathy's explanation regarding micrograd.
One more very important point, the whole point of this blog is not to grasp code or get scared by the code, as these codes are much longer than we use with the help of libraries. But using this basic approach will make the understanding of the concept crystal clear for all of us.
Manual Backpropagation of Simple Expression
Value Class (Data Structure)
- This is the basic data structure that is being created using normal operations without even using any libraries.
- Do check the code once for a better understanding.
- Here I also have used Graph Visualizer using Digraph
Now let’s do the forward pass using simple basic operations
Now if we want to check our forward pass using Digraph then we can simply do it using the function that we have already created.
After this code, we will get a very easy-to-understand graph, even with just the below anyone can easily understand what is happening below the table.
Now, as we are done with forward pass here comes the turn of the backward pass. Before that let’s add a backward function into our Digraph code and again visualize the graph.
Let’s visualize the graph again
As soon as we hear the word backward pass then our mind already knew that we are going to deal with backpropagation.
Before all the steps I already had written some lines of explanation regarding that step. Read the above snippet you will be able to understand the backward pass very easily. The one and the only requirement for the understanding of backward pass is simply the Chain Rule of the Calculus.
Now let’s visualize it using Digraph.
With all this, let’s end this blog as this is it of Backpropagation of simple expression. If you will find any issue regarding concept or code, you can message me on my Twitter or LinkedIn. The next blog of this series will get published on 4 October 2022.
Some words about me
I’m Mohit.❤️ You can also call me Chessman. I’m a Machine learning Developer and a competitive programmer. Most of my time is spent staring at a computer screen. During the day, I am usually programming, working to derive insight from large datasets. My skills include Data Analysis, Data Visualization, Machine learning, and Deep Learning. I have developed a strong acumen for problem-solving, and I enjoy occasional challenges.
