Invariant points can be defined as singular points on a graph that do not move after a linear transformations. Linear transformations include, but are not limited to rotations, reflections and enlargements.
The most clear and obvious example of this is the point \((0, 0) \), the origin. Because the origin can be described in the 2d vector \(\begin{bmatrix}0\\0\end{bmatrix}\). Whatever you multiply it by, it will always give you \(\begin{bmatrix}0\\0\end{bmatrix}\). We know this, because all linear transformations can be described as a 2x2 matrix multiplication!
$$ \begin{bmatrix}a & b\\c & d\end{bmatrix} \begin{bmatrix}0\\0\end{bmatrix} = \begin{bmatrix}0\\0\end{bmatrix} $$
Therefore, we can make the following statement: "For any linear transformation, the origin is an invariant point."
In order to calculate invariant points, which aren't necessarily the origin, one needs to form a formula similar to what we have just written! Simply substitute the origin for \(\begin{bmatrix}x\\y\end{bmatrix}\).
$$ \begin{bmatrix}a & b\\c & d\end{bmatrix} \begin{bmatrix}x\\y\end{bmatrix} = \begin{bmatrix}x\\y\end{bmatrix} $$
Now let's multiply these matrices algebraically (if you can't remember how, I'd recommend reading this):
$$ \begin{gather} \begin{bmatrix}a \times x + b \times x\\c \times y + d \times y\end{bmatrix} = \begin{bmatrix}x\\y\end{bmatrix} \end{gather} $$
From this, we can then form a set of simultaneous equations:
$$ \begin{gather} ax + bx = x \\ cy + dy = y \end{gather} $$
It is probably easiest to show this in an actual example!
Let's make the matrix (\(\boldsymbol{M}\)) a rotation of \(90^o\) anticlockwise:
$$ \begin{gather} \boldsymbol{M} = \begin{bmatrix}cos \theta & -sin \theta \\sin \theta & cos \theta\end{bmatrix}\\ \text{where } \theta = 90 \\ \boldsymbol{M} = \begin{bmatrix}cos 90 & -sin 90 \\sin 90 & cos 90\end{bmatrix} \\ \boldsymbol{M} = \begin{bmatrix}0 & -1 \\1 & 0\end{bmatrix} \end{gather} $$
Now let's calculate the invariant points of this rotation!
$$ \begin{gather} \begin{bmatrix}0 & -1 \\1 & 0\end{bmatrix} \begin{bmatrix}x\\y\end{bmatrix} = \begin{bmatrix}x\\y\end{bmatrix} \\ \therefore \\ -y = x \\ x = y \end{gather} $$
Substituting the formula \(x=y\) into \(-y=x\) gives \(-x=x\). The only time \(-x=x\) is when \(x=0\). Therefore, the only point for this rotation that's invariant is the origin (0, 0). This is true for all rotations (other than full rotations, i.e. rotation by 360 or 0).
Invariant Lines are slightly different. The points on the line are allowed to move, but they always end up on some other point on the line. Therefore, after a linear transformation, the line remains the same.
The way we can tackle this, is by forming another set of simultaneous equations, but instead of saying that the vector \(\begin{bmatrix}x\\y\end{bmatrix}\) will remain the same. We want the line \(y=mx+c\) to remain the same.
$$ \begin{gather} \begin{bmatrix}a & b\\c & d\end{bmatrix} \begin{bmatrix}x\\mx+c\end{bmatrix} = \begin{bmatrix}x^` \\mx^` + c\end{bmatrix} \end{gather} $$
The little tick (`) above the "x" means that the two values of x are different, we could have used any new variable here, but keeping it as x is easier.
I will now jump straight into an example of using this method:
$$ \begin{bmatrix}2 & 1\\2 & 3\end{bmatrix} \begin{bmatrix}x\\mx+c\end{bmatrix} = \begin{bmatrix}x^` \\mx^` + c\end{bmatrix} $$
Now we will multiply the left-hand side:
$$ \begin{bmatrix}2x + 1(mx+c)\\2(x) + 3(mx+c)\end{bmatrix} = \begin{bmatrix}x^` \\mx^` + c\end{bmatrix} $$
We can then separate these into two simultaneous equations:
$$ \begin{gather} 2x + mx + c = x^` \\ 2x + 3(mx+c) = mx^` + c \end{gather} $$
Brace yourself! This is about to get into algebraic hell territory! But don't fear, we'll make it through!
Because we have defined a value for \(x^`\) (that being \(2x + mx + c\)), we can substitute it's value into the second equation to form a new equation with no \(x^`\).
\begin{align} 2x + 3(mx+c) &= m(2x + mx + c) + c \\ 2x + 3mx + 3c &= 2mx + m^2x + mc + c \\ 2x + 3mx + 2c &= 2mx + m^2x + mc \\ 2x + 2c &= 2mx - 3mx + m^2x + mc \\ 2x + 2c &= -1mx + m^2x + mc \\ -mc + 2c &= -1mx + m^2x + 2x \\ c(-m + 2) &= x(-1m + m^2 + 2) \\ c(-m + 2) &= x(m^2 - 1m + 2) \\ c(-m + 2) &= x((m -2 )(m + 1)) \\ c(-m + 2) &= x(m -2 )(m + 1) \\ 0 &= x(m - 2)(m + 1) - c(-m + 2) \\ 0 &= x(m - 2)(m + 1) + c(m - 2) \\ 0 &= (m - 2)(x(m + 1) + c) \\ \end{align}
Now that we have this equation, we can solve it just like a quadratic. We can say the formula is equal to 0, so when \(m = 2\).
Therefore there is the invariant line \(y = 2x\)
But, there is another! Let's look at the second to last formula and use that. Let's let m = -1:
\begin{gather} x(m - 2)(m + 1) + c(m - 2) \\ x(-1 - 2)(-1 + 1) + c(-1 - 2) \\ -3c \end{gather}
Can you see how we have a "c" variable left over, we can then say for all lines that follow the pattern: \(y = -x + c\) are invariant lines!