Breaking Through the 3D Barrier—Finally!
Ethan w -
Hi everyone! Welcome back to week 7 of my blog.
Last week, I was stuck in a frustrating loop with my parallel processing issues on Windows. I tried to edit the code to fix this issue but nothing seemed to work so I decided a Linux VM would completely bypass the windows file handling limitations. While I am fairly confident this will solve the issues, I didn’t get around to impkementing this and running a model since I wanted to prioritize making real progress on my extensions of label distribution smoothing and feature distribution smoothing.
First, I had to return to how these smoothing techniques would work in higher dimensions. In the original paper by Yang et al., LDS and FDS were designed for 1D regression tasks like age estimation. For 2D and 3D data, I needed to develop multivariate versions of these techniques.
For Label Distribution Smoothing in 3D, I implemented a multivariate Gaussian kernel that used a 3×3 covariance matrix that controls the smoothing bandwidth to smooth the label distribution in all three dimensions simultaneously, accounting for the spatial relationships between data points.
This mathematical formulation was very challenging, especially when it came to discretizing the continuous 3D space into bins for the kernel to use them easily. I ended up creating a 3D grid of bins and computing the kernel matrix between all bin pairs, which required careful optimization.
I also started implementing the 3D version of Feature Distribution Smoothing, which involves computing feature statistics (mean and covariance) for each 3D bin and then smoothing these statistics using the same conceptual multivariate Gaussian kernel as LDS. This was even more complex than LDS because it required handling high-dimensional feature vectors and ensuring numerical stability during the whitening and re-coloring procedure.
By the end of the week, I had a “working” implementation of 3D LDS that still needs to be really tested on a model and had made significant progress on 3D FDS.
Next week, I’ll finally implement that Linux VM so that I can start testing models with new implementations of LDS and FDS. I’ll also be finalizing the 3D FDS implementation and selecting appropriate datasets for testing. Stay tuned for more 3D adventures!
Comments:
All viewpoints are welcome but profane, threatening, disrespectful, or harassing comments will not be tolerated and are subject to moderation up to, and including, full deletion.