# Linear regression example

In this example, we want to approximate the following scatter plot with a single layer neural network. Blue points are the training set given by an input $$x_i$$ and an expected output $$y'_i$$. The red line is the output of the network $$y=f(x)$$ after training. The following perceptron will be used for the single layer network: $$x$$ is the input. The activation function is given by $$f(x)=x$$.

### Update rule

As explained on the previous page, the weights will be updated according to this formula:

$$w_i'= w_i + \eta(y'-y)x_i$$

Let's detail for each weight $$w_1$$ and $$w_2$$:

$$w_1'= w_1 + \eta(y'-y)x$$ $$w_2'= w_2 + \eta(y'-y)$$

### Source code

Click one of the language below to display the source code of this example:

### Matlab

close all;
clear all;
clc;

%% Parameters
% Model parameters
a=0.6;
b=-2;
NoiseStd=0.5;
% Training set size
N=1000;
% Learning rate
Eta=0.003;

%% Create trainning set
% X is the input
X=20*rand(N,1)-10;
% Y_ is the expected output (+ noise)
Y_=a*X+b +normrnd(0,NoiseStd,N,1);

%% Initialize weight
W=[0;0];

%% Trainig loop
for i = 1:numel(X)
% Forward
Y=W'*[X(i);1];
W=W+Eta*(Y_(i) - Y)*[X(i);1];
end

%% Plots
% Training set
plot (X,Y_,'.');
hold on;

% Network output
Xt=[-10:10];
Yt=W(1)*Xt + W(2);
plot (Xt,Yt,'r','LineWidth',4);
xlabel ('X');
ylabel ('Y');
title ('Single Layer perceptron');
axis square equal;
grid on;

%% Display results
Model=[a,b]
Weights=W'


#### Output :

Model =

0.6000   -2.0000

Weights =

0.5954   -1.8923


### C / C++

#include <iostream>

// Parameters of the line (y=ax+b)
#define     a       0.6
#define     b       -2

// Size fo training set
#define     N       1000

// Learning rate
#define     ETA     0.03

// Generate random double
double rand_double(double fMin, double fMax)
{
double f = (double)rand() / RAND_MAX;
return fMin + f * (fMax - fMin);
}

int main(void)
{
// Create training set
double X[N],Y_[N];
for (int i=0;i\$lt;N;i++)
{
X[i]=rand_double(-10,10);
Y_[i]=a*X[i]+b;
}

// Training
double w1=0,w2=0;
for (int i=0;i<N;i++)
{
// Forward
double Y=w1*X[i]+w2;
// Update weights (backward)
w1=w1+ETA*(Y_[i]-Y)*X[i];
w2=w2+ETA*(Y_[i]-Y);
}

std::cout << "::: Results :::" << std::endl;

// Display a and b
std::cout << std::endl;
std::cout << "** Model **" << std::endl;
std::cout << "a=" << a << std::endl;
std::cout << "b=" << b << std::endl;

// Display weights (w1 and w2)
std::cout << std::endl;
std::cout << "** Network weights **" << std::endl;
std::cout << "w1=" << w1 << std::endl;
std::cout << "w2=" << w2 << std::endl;

return 0;
}



#### Output :

::: Results :::

** Model **
a=0.6
b=-2

** Network weights **
w1=0.6
w2=-2