Dtyjlui

Having the matrix in something approaching row echelon form is definitely the right way to go, and you are nearly done once you have it. Unfortunately I think you did your final row operation incorrectly; my row echelon form matrix looks like this:
$$
begin{bmatrix}1 & 1 & -1 & 2\0 & 1 & 2 & 1\ 0 & 0 & a^2-4 & a-2end{bmatrix}
$$
Is this enough to show you the way?

Hint:

A non-homogeneous linear system has solutions if and only if the matrix of the homogeneous side and the augmented matrix have the same rank. Furthermore, this rank is the codimension of the affine space of solutions.

Thus here, there is a unique solution if and only if the matrix $;smash{begin{bmatrix}1&1&-1\ 1&2&1\1&1&a^2-5end{bmatrix}}$ has rank $3$. You can obtain its rank with row reduction.

The standard start:

The matrix is

$begin{bmatrix}
1 & 1 & -1 & 2\
1 & 2 & 1 & 3\
1 & 1 & a^2-5 & a
end{bmatrix}
$

The 3x3 determinant is
$2(a^2-5)+1-1-(1+(a^2-5)-2)
=a^2-4
$
so,
if $a^2 ne 4$,
there is a unique solution.

note: in the original version of the question the answer for part b was subtuly wrong, it should say $a neq pm 2$

I would start by getting rid of x by subtracting one of the cases (I went for the first one) from the other two.

$$ (2y + z)-(y-z) = 3 - 2 $$
$$ (y + (a^2 -5)z) - (y-z) = a - 2 $$

Multiplying out and simplifying.

$$ y + 2z = 1 $$

$$ (a^2 -4)z = a - 2 $$

So how do you solve an equation like that? you would want to divide it through by $(a^2 -4)$ but it is only valid to do that if $(a^2 -4) neq 0$. Some elementary knowlage of quadratics tells us that $(a^2 -4) neq 0$ when $a neq pm 2$. Therefore when $a neq pm 2$ our equation gives a single value for z which in turn gives a single value for y which in turn gives a single value for x.

Ok what about when $a = pm 2$ well since we only have to cases left we can just substitute them in.

$$ 0z = -2 - 2 = -4 $$

$$ 0z = 2 - 2 = 0 $$

The first equation clearly has no soloutions, the second clearly has an infinite soloution set.

our equations: (a^2 means a * a)

x+y−z=2

x+2y+z=3

x+y+(a^2−5)z=a

general equation form:

a1*x+b1*y+c1*z = d1

a2*x+b2*y+c2*z = d2

a3*x+b3*y+c3*z = d3

matrix A:

a1 b1 c1

a2 b2 c2

a3 b3 c3

matrix B:

d1 b1 c1

d2 b2 c2

d2 b3 c3

matrix C:

a1 d1 c1

a2 d2 c2

a3 d3 c3

matrix D:

a1 b1 d1

a2 b2 d2

a3 b3 d3

x = det(B) / det(A);

y = det(C) / det(A);

z = det(D) / det(A)

matrix A general format:

a11 a12 a13

a21 a22 a23

a31 a32 a33

det(A) = a11*a22*a33 + a12*a23*a31 + a13*a21*32 - a13*a22*a31 - a12*a21*a33 - a11*a23*a32

det(B) = d1*b2*c3 + b1*c2*d3 + c1*d2*b3 - c1*b2*d3 - b1*d2*c3 - d1*c2*b3

det(C) = a1*d2*c3 + d1*c2*a3 + c1*a2*d3 - c1*d2*a3 - d1*a2*c3 - a1*c2*d3

det(D) = a1*b2*d3 + b1*d2*a3+ d1*a2*b3 - d1*b2*a3- b1*a2*d3 - a1*d2*b3

assuming if my math below is correct, we get:

det(A) = a^2 - 4

det(B) = a^2+3a-10

det(C) = a^2-2a

det(D) = a - 2

x = (a^2+3a-10) / (a^2 - 4)

y = (a^2+3a-10) / (a^2 - 4)

z = (a-2)/(a^2 - 4)

If a^2 = 4, then the equation has no solution b/c we can't divide by zero

So, we can't have a = -2 or a = 2

Otherwise, the equation has the unique solution.