I'm assuming 3d. By "rotation Matrix" I'm also assuming you mean Basis. Also, it's not clear whether you'd like me to teach you the linear algebra or just point to the methods Godot offers to do this for you.
The basic Basis is the Identity which is a matrix made up of three unit vectors (length of 1) pointing right, up and forward (x, y, z)
Basis(Vector3(1, 0, 0), Vector3(0, 1, 0), Vector3(0, 0, 1))
This is often used for rotation but in reality it's more basic than that: it's defining what right, up and forward mean. Want to flip a model upsidedown without changing every Vector3 in your mesh? Then definite the y component of the vector as being negative.
Basis(Vector3(1, 0, 0), Vector3(0, -1, 0), Vector3(0, 0, 1))
Like the identity Basis this Basis is still orthonormal: unit vectors at right angles so the model isn't distorted or stretched. If I wanted to stretch twice on the y then:
Basis(Vector3(1, 0, 0), Vector3(0, 2, 0), Vector3(0, 0, 1))
Think of this as defining the axes on a graph, albeit a 3d one. It's in reference to this that your mesh's vertices vectors are plotted. (that an the "origin", the starting point)
Let's pick a random Vector3. Say Vector3(12, 5, 10)
and let's normalise it. In code that's Vector3(12, 5, 10).normalized()
in maths it's this vector divided the length (which is just Pythagoras) sqrt(12*12+5*5+10*10)
so Vector3(0.73, 0.3, 0.61). Let's say this is forward; the y vector.
So now we need to work out the orthonormal vectors to get a useful Basis. Well, let's use a placeholder vector for UP of Vector3(0, 1, 0) and do the cross product as you say. The cross product gives you a right angle vector. Honestly, this is already getting long and explaining the maths behind the cross product would take a while. Khan Academy has 144 videos going through it which I recommend.
Suffice to say, it's the linear algebra way of taking the sine, whereas the dot product is the same as the cosine. In Godot it's just Vector3(0.73, 0.3, 0.61).cross(Vector3(0, 1, 0)
. It output the right angle vector, in this case the x.
But we need to now work out the y - because we just had a placeholder. So, let's cross the Vectors we know. Vector3(0.73, 0.3, 0.61).cross(Vector3(-0.61, 0, 0.73)).
So our orthonormal basis is pointing to y is:
Basis(Vector3(0.73, 0.3, 0.61), Vector3(0.219, -0.9, 0.18), Vector3(-0.61, 0, 0.73))
Oops, that's upside down, reverse one of the crosses to put it right way up.
Or... Just put look_at(Vector3(12, 5, 10), Vector3.UP)
To rotate it with angles then you can just use a matrix with sine and cosine. In 3d you need 3 sets of 3x3 matrices. Sounds complicated but it's just really basic trig and a bit of matrix multiplication. This is for Euler rotations (like a gimbal).
If gimbal locking is an issue then you can use quaternions but this is more advanced so ignore the maths and just use the inbuilt functions.
Honestly, if all this maths is too much then you have loads of inbuilt methods where you can ignore the maths. It's good to know what the Basis means fundamentally so I hoped this helped a little.