I'm trying to do a simple hit test on a triangle and return the first triangle hit. In my mesh class I have a public function which iterates through all the faces in the triangle, fetches their vertices based on the indexes, transforms the vertices to world space and then passes that to the glm hit algorithm for processing:
bool Mesh::findIntersection(glm::mat4 rayMatrix, glm::vec3& foundPoint) {
glm::vec3 rayOrigin;
glm::vec3 rayDirection;
rayOrigin = glm::vec3(rayMatrix[3]); //pick up position of ray from the matrix
rayDirection = glm::vec3(0,0,-1)*glm::mat3(rayMatrix); //direction of ray
glm::vec3 v0,v1,v2; //three vertices of the triangle to be tested
float epsilon = 0.000001f; //used by a previous algorithm i tried
//iterate through all faces
for (std::vector<Face>::size_type i=0;i < faces.size(); i++) {
glm::vec3 intersection;
v0 = vertices[faces[i].a]; //vertices are kept in another vector object
v1 = vertices[faces[i].b]; //I fetch them using indices
v2 = vertices[faces[i].c];
v0 = glm::mat3(modelMatrix)*v0; //first rotate the vertex
v0 = v0+glm::vec3(modelMatrix[3]); //then translate it
v1 = glm::mat3(modelMatrix)*v1;
v1 = v1+glm::vec3(modelMatrix[3]);
v2 = glm::mat3(modelMatrix)*v2;
v2 = v2+glm::vec3(modelMatrix[3]);
if (glm::intersectLineTriangle(rayOrigin,rayDirection,v0,v1,v2,intersection)) {
foundPoint = intersection;
return true;
}
}
return false;
}
The problem is that I never get a hit. The rayMatrix that I pass contains the translation of the ray's origin and the direction it's facing (the -z axis, I'm casting using a 3D tracked wii-mote). I suspect the problem to be with the way I'm transforming the vertices to world space before I pass them to the testing function.
