Iterative Methods for Linear Systems:

We want to solve a Linear System

as an iterative method. The idea is to decompose

being M an invertible matrix. Then

where

and

. In general if we consider a matrix A, we can decompose

, being its lower, diagonal and upper parts.

Jacobi Method:

Gauss-Seidel Method:

Solution of x in Ax=b using Iterative Methods:

%A = [10 1 1 1; 1 10 1 1 ; 1 1 10 1 ;  1 1 1 10]
%A=rand(4)+eye(4);
A=[
1.183907788282420, 0.902716109915281, 0.337719409821377, 0.780252068321138;
0.239952525664903, 1.944787189721650, 0.900053846417662, 0.389738836961253;
0.417267069084370, 0.490864092468080, 1.369246781120220, 0.241691285913833;
0.049654430325742, 0.489252638400019, 0.111202755293787, 1.403912145588110
]
A = 4×4
    1.1839    0.9027    0.3377    0.7803
    0.2400    1.9448    0.9001    0.3897
    0.4173    0.4909    1.3692    0.2417
    0.0497    0.4893    0.1112    1.4039
b = [13; 13; 13; 13] 
b = 4×1
    13
    13
    13
    13
n = size(A,1);
xini = rand(n,1)
xini = 4×1
    0.7802
    0.0811
    0.9294
    0.7757
AU = triu(A,1);
AL = tril(A,-1);
AD = diag(diag(A));
tol = 1e-5; %tolerance for the result 
maxItr = 200; %maximum of iterations allowed

Jacobi Method

Solution of x in Ax=b using Jacobi Method

x = xini;
n = size(x,1);
normDif = Inf; 
itrJacobi = 0;
evolJacobi = [];
ADinv = inv(AD);
R = -ADinv*(AL+AU);
c = ADinv*b;
while (normDif>tol && itrJacobi < maxItr)
    xold = x;
    x = R*xold+c;
    normDif = norm(x-xold);
    evolJacobi = [evolJacobi, normDif];
    itrJacobi = itrJacobi+1;
end
fprintf('Solution of the system is : \n%f, %f, %f, %f in %d iterations',x,itrJacobi);
Solution of the system is : 
2.490017, 1.651172, 6.720104, 8.064054 in 96 iterations

Gauss-Seidel Method

Solution of x in Ax=b using Gauss-Seidel Method

x = xini;
n = size(x,1);
normDif = Inf; 
itrGS = 0;
evolGS = [];
ADLinv = inv(AD+AL);
R = -ADLinv*AU;
c = ADLinv*b;
while (normDif>tol && itrGS < maxItr)
    xold = x;
    x = R*xold+c;
    normDif = norm(x-xold);
    evolGS = [evolGS, normDif];
    itrGS = itrGS+1;
end
fprintf('Solution of the system is : \n%f, %f, %f, %f in %d iterations',x,itrGS);
Solution of the system is : 
2.490019, 1.651174, 6.720106, 8.064054 in 15 iterations

Comparison: Gauss Seidel Method Vs Jacobi Method

figure

hold on

step=2;

plot(1:step:itrGS,evolGS(1:step:itrGS),'LineWidth',1)

plot(1:step:itrJacobi,evolJacobi(1:step:itrJacobi),'LineWidth',1)

legend('Gauss Seidel Method','Jacobi Method')

ylabel('Error Value')

xlabel('Number of iterations')

title('Gauss Seidel Method Vs Jacobi Method')

hold off