Linear Regression Using Stochastic Gradient Descent in Python

Updated: Mar 15

As Artificial Intelligence is becoming more popular, there are more people trying to understand neural networks and how they work. To illustrate, neural networks are computer systems that are designed to learn and improve, somewhat correlating to the human brain.


In this blog, I will show you guys an example of using Linear Regression in Python (Code at https://github.com/anaypant/LinearRegression/tree/master)


An example of a convolutional neural network. Each neural network takes a certain amount of inputs and returns a certain amount of outputs. In this neural network, there are 3 columns of nodes in the middle. Each column is called a hidden layer. There is no exact answer to how many layers a neural network needs to have. To learn more about neural networks, see this video by 3Blue1Brown to learn more:


In machine learning, there are many different types of algorithms that are used for different tasks. The one I will show you guys today is what is known as Linear Regression. In Linear Regression, the expected input(s) is/are a number, as well for the outputs.

One common example of Linear Regression in everyday life is finding an appropriate value for housing value prices. In this network, there would be multiple inputs (different features of the house), and there would be only one output(the estimated price). Some inputs could be the square feet of the house, the number of bedrooms, number of bathrooms, age in days of the house, etc. The output would then be the appropriate value of the house.

First things first, how will we use the weights (synapses) of our model to find an approximated output? We will do this by using the simple formula:


This formula is a simple yet effective one. For anyone who didn’t know, in this equation, the output(y) is equal to the input(x) times the slope(m) + a bias or y-intercept(b).

For further understanding, feel free to visit this website: http://www.math.com/school/subject2/lessons/S2U4L2GL.html#:~:text=The%20equation%20of%20any%20straight,line%20crosses%20the%20y%20axis.

For our Linear Regression model, we will fine-tune two parameters in this equation: the slope, and the bias.



Coding a Linear Regression Model


Now, we will get to see our Linear Regression model in Python!

— — Editor’s Note: — —

You will NOT need any external libraries/packages for this model. However, this is a very simplistic version, so feel free to add your own twist to the code.

Let’s start with the code: