ClearAll["Global`*"]

Eigenvalues, Eigenvectors, and Tensors

Suppose I have been working on a Moment of Inertia problem, using some convenient coordinate system xyz, and I discover that the tensor for the Moment of Inertia in this coordinate system comes out to be:

M1=(({{3, -2, 4}, {-2, 1, 2}, {4, 2, 3}}));

The eigenvalues are determined by finding the determinant of this matrix, but subtracting λ from each diagonal:

λ = λ/.Solve[Det[(({{3-λ, -2, 4}, {-2, 1-λ, 2}, {4, 2, 3-λ}}))]==0,λ]

Clearly, the eigenvalues are λ = -3, 3, and 7, in no particular order. 
Similarly, Mathematica can just tell you the Eigenvalues:

Eigenvalues[M1]

Hmmm... it seems that this time, Mathematica decided to list them in a different order than before. 
In any case, this problem has 3 unique eigenvalues. We could check these values by looking at some determinants. 
Here are a) the original determinant, and b), c) and d) the determinants when each λ is subtracted from the diagonals:

Print["Original Determinant = ",Det[(({{3, -2, 4}, {-2, 1, 2}, {4, 2, 3}}))]]
Print["Determinant using λ = -3 = ",Det[(({{3-(-3), -2, 4}, {-2, 1-(-3), 2}, {4, 2, 3-(-3)}}))]]
Print["Determinant using λ = +3 = ",Det[(({{3-3, -2, 4}, {-2, 1-3, 2}, {4, 2, 3-3}}))]]
Print["Determinant using λ = 7  = ",Det[(({{3-7, -2, 4}, {-2, 1-7, 2}, {4, 2, 3-7}}))]]

Eigenvectors
Let’s find the Eigenvectors for this same matrix. I need to use the following process three times; once for each eigenvalue. I’ll start with λ = -3.
I want (({{3-(-3), -2, 4}, {-2, 1-(-3), 2}, {4, 2, 3-(-3)}})).(({{x}, {y}, {z}}))=0.
This is really three equations. They are:
  6x -2y+4z = 0,
  -2x +4y +2z = 0, and 
  4x+2y+6z = 0.
These don’t look so bad. We did this for a 2x2 matrix on the board last time. 
I’ll use algebra on the first two equations to eliminate z (that is, I solve EQ 2 for z, and plug that into EQ 1). The result is:
  10x - 10y = 0, or more simply, x = y. 
 Then,  I’ll plug this into EQ 3:
  4x + 2(x) + 6z =0, or more simply, z = -x. 
  So, I just found my first Eigenvector: (Subscript[E, 1]) = x(OverHat[i]) + y(OverHat[j]) + z(OverHat[k]) = x(OverHat[i]) + x(OverHat[j]) - x(OverHat[k]).
  When normalized to make it a unit vector, I get: : (Subscript[E, 1]) =((x(OverHat[i])+x(OverHat[j])-x(OverHat[k]))/(Sqrt[(x^2)+(x^2)+(x^2)])). Every “x” cancels out, and my final, normalized eigenvector is:
  (Subscript[E, 1]) = 0.5774(OverHat[i]) + 0.5774(OverHat[j]) - 0.5774(OverHat[k]) .
  I could follow this process again by hand to learn the other two normalized Eigenvectors.
  For λ = 7, I get (Subscript[E, 2]) = 0.70711(OverHat[i]) + 0(OverHat[j]) + 0.70711(OverHat[k]) , and 
  For λ = 3, I get (Subscript[E, 3]) = -0.4083(OverHat[i]) + 0.8165(OverHat[j]) + 0.4083(OverHat[k]).
  
  Of course, why should we do it by hand, when Mathematica can do it for us?

Eigenvectors[M1]

Hey! These aren’t even normalized! We could have Mathematica normalize them:

E1 =1.0 Eigenvectors[M1][[1]]//Normalize  (* Goes with λ = 7. The 1.0 is multiplied in front because I want to see decimal results *)
E2=1.0 Eigenvectors[M1][[2]]//Normalize (* Goes with λ = -3.  *)
E3=1.0 Eigenvectors[M1][[3]]//Normalize (* Goes with λ = 3.  *)

You’ll notice that these are in a different order than ours, and are also sometimes in the opposite direction of ours.
For example, Mathematica’s E2 is negative one times my E1.
At least, the order of the answers from the Eigenvector function matches the order of the eigenvalues from the Eigenvalue function.
So, that matches what we found by hand.
There’s nothing we can do about the direction; neither one is truly "more correct" than the other. 
We could do some other checking, too. With or without normalization, these should all be perpendicular. Let’s do some dot products:

Print["Dot product of (Subscript[E, 1]) and (Subscript[E, 2]) = ",E1.E2 ]
Print["Dot product of (Subscript[E, 2]) and (Subscript[E, 3]) = ",E2.E3 ]
Print["Dot product of (Subscript[E, 3]) and (Subscript[E, 1]) = ",E3.E1 ]

Similarly, the cross product of any two eigenvectors should equal the other one. So if (Subscript[E, 1]) = (Subscript[E, 3]) x (Subscript[E, 2]), then (Subscript[E, 3]) x (Subscript[E, 2]) - (Subscript[E, 1]) should be zero..
I got the "cross product" operator from the Basic Math Assistant/Advanced palette:

Print["Testing cross product of (Subscript[E, 3]) and (Subscript[E, 2]) : ",E3 \[Cross] E2 -E1, "  Note computer rounding of result!"]
Print["Testing cross product of (Subscript[E, 3]) and (Subscript[E, 2]) : ",E1 \[Cross] E3 -E2]
Print["Testing cross product of (Subscript[E, 3]) and (Subscript[E, 2]) : ",E2 \[Cross] E1 -E3, "  Note computer rounding of result!"]

These are all zero except for rounding errors, so E3 × E2 = E1. Being perpendicular, these three eigenvectors always form a new coordinate system.
You’ll note that strictly speaking, in the order given by Mathematica this time, these eigenvectors form a left-handed coordinate system, because it should instead be true that E2 × E3 = E1. Lesson: don't trust Mathematica with signs of Eigenvectors!

Tensor Rotations

One of many interesting facts is that in the coordinate system described by the eigenvectors, the components of the tensor are the eigenvalues. For example, in the eigenvector coordinate system:
(Subscript[M, coords described by Eigenvectors])=(({{7, 0, 0}, {0, -3, 0}, {0, 0, 3}})).
But of course, we can even find the components of the tensor in any arbitrary coordinate system.
There are a couple of ways we might conceive the new coordinate system.
One way is to just name any three unit vectors to be the new x, y, and z axes. Of course, these would all have to be perpendicular.

Another way would be to think of the new coordinate system as being the result of some rotation of the old coordinate system by some angle or angles.
Actually, in this way of thinking, every coordinate system you can think of can be described by a two step process:
   1. rotate the original coordinate system by some angle (Subscript[θ, z]) about the z axis to obtain some intermediate coordinate system.
   2. rotate the resulting intermediate coordinate system by some other angle (Subscript[θ, y]) about the y’ axis to obtain the final coordinate axes.
Let’s try this both ways: I’m thinking of a new coordinate system whose unit vectors, with respect to
the original coordinate system, are:

u1=(({{0.866}, {0.500}, {0.000}}));u2=(({{-0.500}, {0.866}, {0.000}}));u3=(({{0.000}, {0.000}, {1.000}}));

You could easily verify that these are all unit vectors perpendicular to each other. In fact, this is just the coordinate system obtained by rotating the original coordinate system about the z axis by 30°, followed by no rotation (Subscript[θ, y]) .
Now that we have the axes of the new coordinate system, we make a matrix “C” whose columns are these new unit vectors:

C1=(({{0.866, -0.500, 0.000}, {0.500, 0.866, 0.000}, {0.000, 0.000, 1.000}})); (* made by jamming together all the unit vectors describing the new coordinate system *)

The components of M in the new coordinate system are found by a product of three matrices.
Note that this is VERY DIFFERENT from matrices used to rotate vectors!!:

M4=Inverse[C1].(M1.C1);  (* you see that there are TWO dot products in here, and C1 is used twice! *)
M4 //MatrixForm

This is the same M1 tensor, with the same physical meaning, but having components expressed with respect to the new coordinate system.
Let’s look at the eigenvalues of M4 and compare them to those of M1:

Print["the Eigenvectors of M1 are: ",Eigenvalues[M1],"; the Eigenvectors of M4 are: ",Eigenvalues[M4]]

In other words, Eigenvalues are independent of coordinate system... they are “true” or “invariant” through all coordinate systems.

Or, you could take the original Tensor M1 and rotate it using a (passive) rotation matrix.
Suppose we want to rotate M1 about the z axis by 30°. The rotation matrix R1 is just:

θz=π/6;
R1=(({{Cos[θz], -Sin[θz], 0}, {Sin[θz], Cos[θz], 0}, {0, 0, 1}}));
M5=Inverse[R1].(M1.R1); (* note that R1 is used TWICE in this expression! *)
N[M5] //MatrixForm (* this is the components of M1 in the new coordinate system *)

You’ll notice that this result is the same as M4 (the method above that used the new unit vectors to generate the rotation matrix, instead of using the angle θz).