Week 9-A Supervised Approach To Training An Artificial Intelligence To Extract Relevant Genomic Data From Literature
Adam B -
Hello everyone,
This week, I continued working on the API for my AI model. This week, I managed to implement a solution that resolved the problems described last week. I would like to spend this week’s blog discussing this process.
Previously, the API’s workflow worked for a given paper by first calling the AI with a Python script . The AI would then analyze this paper for genetic extractions (per the prompt previously developed many weeks ago that is embedded in the Python script). The AI would then return its output back. Here, it would be handed off to an expansive group of Python code that would take its output of genes and other data and transform it into an Excel document or Microsoft Access framework that can be easily understood by myself and lab members for comparison to our own outputs.
The main problem that has developed is the difficulty of taking a somewhat loosely specified output and making it work with the Python code designed for clear outputs. Typically, some genes would get “lost in translation,” tables would be shifted, and a whole host of other errors would appear since the AI had an output method that 1) could not be consistent simply because I had not told it to do so and 2) could not be seen because it is immediately handed off to the Python code and not provided for me to see.
The solution for this code was to simply convert the mass of conversion-Python-code into plain English text and give it to the AI alongside the previously developed prompt. Looking back, this feels remarkably intuitive. The result is that the AI has far more consistent outputs since it is no longer confusing the Python code with its small variations I hadn’t accounted for.
I am glad it was a simple solution.
All the best,
Adam
Comments:
All viewpoints are welcome but profane, threatening, disrespectful, or harassing comments will not be tolerated and are subject to moderation up to, and including, full deletion.