Linear Regression From Scratch: Generating and Understanding Dummy Data

 

Scatter plot of 20 random data points with a fitted linear regression line showing noisy relationship between x and y.

In this blog we will learn about Linear Regression Model in PyTorch using a Dummy data to understand how the training works Step by Step and understand the basics of Machine Learning.

Machine Learning is not a magic or something complicated. At its core, its just a Mathematics and logic. Instead of guessing, we use clear step to let the model learn patterns from data.

Step 1: Creating a Dummy data

First we will create a Random 20 x values, roughly between -5 to 15.

we will use NumPy for this. It's random module makes it easy to generate numbers within a range.

Array of 20 random numbers between -5 to 15

With this snippet, we generate a one-dimension array stored in variable x.

Now we will create y values using a simple line.

y = mx + c / wx + b

The real equation is:

y = 0.5x - 1 with a slope as 0.5 and the Intercept -1.

Adding some normal distribution in this equation or making it noisy. Why? because real world data is never clean and this is not some random noise these are Gaussian Distribution ranging from -∞ to +∞ and stay close to zero.

Visual Representation of Data scattered in graph

Step 2: Creating a model

 We will create a simple PyTorch Linear model:

Simple Linear Model with one input and one output.

This creates a line:

y = wx+b

where,

• w = Slope
• b = Intercept 

At the beginning, PyTorch assigns random values to w and b. Because of this, the line will not match our data.

That’s completely normal.

The model starts wrong — and then learns.

Step 3: Defining Loss and Optimizer

Loss And Optimizer

We will use Mean Squared Error for loss calculation.

MSE or Mean Squared Error primarily used in regression problems

If you are wondering why squared the error? Squaring ensured our error stays positive so the negative and positive doesn't cancel out. 

For data such as this it will not affect too much if we only use mean error as loss but for data with millions parameters one cancel out can be fatal to the accuracy of the model.

For optimizer we used SGD ( Stochastic Gradient Descent ) 

Stochastic Gradient Descent

As the name suggest Gradient Descent it slowly descent on the curve by each round it update its weight (w) and bias (b) to find the the line of best fit.

Think of it like walking down a mountain, you start randomly and slowly start walking downward.

With each step it will update the weight(w) and bias(b) moving toward global cost minimum the point where our model is more accurate.

Step 4: Preparing data for PyTorch

Input and Target for PyTorch

PyTorch works with Tensors, not numpy arrays.

PyTorch expects inputs in specific format so we reshape the data (N Rows and 1 Columns) first then convert it into the Tensors.

Note:- Numpy create arrays with the data type float64.

however, PyTorch expects the data to be in float32.

Step 5: Training the Model

Training The Regression Model

Lets break it down:

• Epoch is just the number of round or step model will take to learn its a Hyper-parameter means you can decide the number of loops.
• Losses is an empty list that will store the losses after each loops.

Now our loop starts

• zero_grad() clears the previous gradient after one loop ends. By default PyTorch adds some gradients. If we don't clear them they will keep accumulating and gives a wrong updates.
• After that we pass the input into the model to get an output. and compute the loss between target and output using criterion(MSELoss) we decided.
• Loss stored into our empty list.
• Loss.backward() compute gradient of loss with respect to model parameters.
• Finally optimizer took another step and repeat the same process till the specified number of epochs.

What Happens During Training?

But as training continues:

The loss slowly decreases.

That means the model is learning.

The curve initially started with a value above 2 and after that it started decreasing slowly.

In middle zigzag patterns shows that model started increasing the loss but later corrected himself and move smoothly toward global minimum.

Step 6: Making Final Prediction

Predicting the result.

If you are wondering why we use model(input) why not use the output?

Because output was only temporary inside the loop and after training we want final output the model has learned.

.detach() is used to remove gradient tracking so we can convert it into numpy for plotting.

line of best fit

The line fitted perfectly

The predicted weight and biases are:

Weight: 0.4764383

Bias: -0.91488725

are closed to our actual weight and biases

Actual_Weight = 0.5

Actual_Bias = -1

as y = 0.5x - 1

 Conclusion

In this blog, we built a Linear Regression model in PyTorch from scratch using dummy data.

We started by generating random input values and creating output values using a simple equation with added noise. Then we created a linear model, defined the loss function and optimizer, prepared the data properly, and trained the model step by step.

At the beginning, the model started with random weights and high loss. But as the training continued, the loss gradually decreased. This shows that the model was learning.

Finally, when we made predictions, the learned weight and bias were very close to the actual values:

• Actual weight = 0.5

• Learned weight ≈ 0.476

• Actual bias = -1

• Learned bias ≈ -0.914

This small difference is expected because we added noise to the data.

The most important lesson from this experiment is that machine learning is not magic. It is simply a process of:

Data → Prediction → Error → Update → Repeat

Linear regression may look simple, but it forms the foundation of deeper models and neural networks. Understanding this clearly makes advanced topics much easier later.

This is just the beginning of untangling neural networks.