Shifting Focus

Taariq R -

This week, I shifted my focus towards running initial tests on my AI model after successfully setting up the development environment. With all necessary packages installed, I began training the model on a small dataset to evaluate its preliminary performance. This helped identify key areas for improvement, such as feature extraction and classification accuracy.

One major development was implementing a better loss function, which stabilized the training a lot. The model had unstable loss fluctuations before, but after I changed the loss function and other settings, I saw more stable curves. I also used data augmentation techniques like rotating images and changing contrast to allow the model to learn more from different versions of the scans. I was also able to fix the packaging issue, where my python processing version was not up to date, so I redownloaded the package with the latest version.

Despite these advances, I have been struggling with overfitting, where the model has performed ideally on training sets but not generalized to validation samples. To fix this, I might try regularization techniques. Another challenge has been trying to balance computational speed and accuracy to ensure the model is working as efficiently as it can be. 

Soon, I plan to enlarge the dataset and perform more accurate testing with more labeled scans. I also would like to tune hyperparameters and try differing structures for better performance. 

Overall, this week has been a milestone in moving from setup to actual testing, and I am happy to continue to refine the model for better early diagnosis.

More Posts

Comments:

All viewpoints are welcome but profane, threatening, disrespectful, or harassing comments will not be tolerated and are subject to moderation up to, and including, full deletion.

    aryan_r
    Hi Taariq, Great work so far! I was wondering why overfitting is a phenomenon present when training AI. If they perform ideally on training sets, but don't do well in real world examples, then what is the purpose of those training sets? Would it not be better if they mirrored what you see in the real world? Do you see any reason for that disconnect?

Leave a Reply

Your email address will not be published. Required fields are marked *