Backpropagation of Simple Expression

Understanding Backward Pass with Backpropagation of Simple Expression.

This blog is 1 of the 3 blog series for the understanding of Backpropagation in detail.

  • It is simply a method for calculating derivatives and with the help of these derivatives, we can reduce the loss of the model and maximize the accuracy. As we know backward pass is mainly termed Backpropagation.
  • Backpropagation is simply a derivative of the final output with respect to its leaf nodes. concerningerm leaf node, I mean all those parameters which are not fixed from start. Example — Weights.
  • The intuition behind the manual creation of forward pass and backward pass came from Andrej Karpathy's explanation regarding micrograd.
Image is taken from 3Blue1Brown Youtube Channel

One more very important point, the whole point of this blog is not to grasp code or get scared by the code, as these codes are much longer than we use with the help of libraries. But using this basic approach will make the understanding of the concept crystal clear for all of us.

Manual Backpropagation of Simple Expression

Value Class (Data Structure)

  • This is the basic data structure that is being created using normal operations without even using any libraries.
  • Do check the code once for a better understanding.
Value Data Structure
  • Here I also have used Graph Visualizer using Digraph
Digraph Code for Graph Visualization

Now let’s do the forward pass using simple basic operations

Forward Pass

Now if we want to check our forward pass using Digraph then we can simply do it using the function that we have already created.

After this code, we will get a very easy-to-understand graph, even with just the below anyone can easily understand what is happening below the table.

Graph of Forward Pass

Now, as we are done with forward pass here comes the turn of the backward pass. Before that let’s add a backward function into our Digraph code and again visualize the graph.

Digraph Code with Grad unit

Let’s visualize the graph again

Graph of Forward Pass with Grad

As soon as we hear the word backward pass then our mind already knew that we are going to deal with backpropagation.

Before all the steps I already had written some lines of explanation regarding that step. Read the above snippet you will be able to understand the backward pass very easily. The one and the only requirement for the understanding of backward pass is simply the Chain Rule of the Calculus.

Now let’s visualize it using Digraph.

Graph with Backward Pass

With all this, let’s end this blog as this is it of Backpropagation of simple expression. If you will find any issue regarding concept or code, you can message me on my Twitter or LinkedIn. The next blog of this series will get published on 4 October 2022.

Some words about me

I’m Mohit.❤️ You can also call me Chessman. I’m a Machine learning Developer and a competitive programmer. Most of my time is spent staring at a computer screen. During the day, I am usually programming, working to derive insight from large datasets. My skills include Data Analysis, Data Visualization, Machine learning, and Deep Learning. I have developed a strong acumen for problem-solving, and I enjoy occasional challenges.

My Portfolio and Github.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Mohit Mishra

My skills include Data Analysis, Data Visualization, Machine learning, and Deep Learning. I have developed a strong acumen for problem-solving, and I enjoy ML.