Week 6 – Algorithm for IR Tracking Part 3
Yuvan S -
Last week, I wrote the algorithm to find where the cameras are located in 3D space. Now, using this data, I am going to write the algorithm to find the LED’s location in 3D space. Remember that we already know the 2D location of the LED in the camera frame. So for example, a LED in the room would appear to camera 1 in (100, 200) and (300, 200) for camera 2.
Step one is to find something called the epipolar line. It takes a 2D point on one camera and draws a line on the second camera that this 2D point probably lies somewhere on that line. This allows us to relate point 1 from camera 1 to the same point on camera 2, creating correspondences. Correspondences just tell me that a certain point relates to another point on the other camera. So the same point is seen from different camera views.
With the camera poses and correspondences, we can do something called the Direct Linear Transform or DLT. Essentially, there is something called the projection matrix. You take the projection matrix and multiply it by the 3D world point you get the 2D point in the image. We know the projection matrices. So, we can set up a linear system with the correspondences (containing 2D points) and the known projection matrices to solve for the 3D points. I solve for it using singular value decomposition from the numpy library in Python, which reduces the projection error (extreme oversimplification, but good enough for the blog post). Then, we run that through bundle adjustment again to optimize and reduce reprojection error (see last week’s blog post). After all of this, we get 3D points of the LED in the room.
I then also use calibration points to create a world transformation matrix that takes the 3D points from earlier, and turns them into traditional x,y,z coordinates that corresponds to the room. So, for example, moving the LED up will increase the Y value rather than decrease it.
Now that we are finally done with the IR tracking algorithm. I will get back into the drone aspect next week.