EFO terms showing blank in the download file

[ljwh2]

In the last DR, Mapped trait is showing blank record in the studies and associations download files for ~126K associations (16% of total) and ~15K studies (11% of total). To the best of my knowledge, these EFO terms are displayed in the UI, but the links to OLS do not work.

472 EFOs were obsoleted in last DR, and 431 of these are showing blank where they previously had an EFO term. Examples are EFO_0005852, replaced with OBA_VT0000803; EFO_0005918, replaced with OBA_2050271.

These recently obsoleted terms account for around 60% of the studies with blank mapping.

Of the remainder, some were also blank in the previous release. Example are OBA_2050164, OBA_2050154

Note, not all OBA terms have problems, here is an example of one that appears correctly: OBA_2042195

I am still investigating the rest.

[ljwh2]

More analysis here: https://docs.google.com/document/d/10lWo7_3KzatRovyTAiKCUZ05wZN3q9ME42XKPVW1tg4/edit?usp=sharing

[ala-ebi]

https://www.ebi.ac.uk/ols4/api/ontologies/efo/terms?short_form=EFO_0600068
term_replaced_by is http://purl.obolibrary.org/obo/OBA_2050119
the replacement link doesnt work

as for the rest, the problem is that the replacement already exists in the catalog…

fixing this will require
changing all entries in ASSOCIATION_EFO_TRAIT, STUDY_EFO_TRAIT, ASSOCIATION_BKG_EFO_TRAIT, STUDY_BACKGROUND_EFO_TRAIT for the obsolete efo to the new efo in oracle
changing all study.efoTraits, study.backgroundEfoTraits, association.efoTraits, association.backgroundEfoTraits for the obsolete efo to the new efo in mongo

this obviously needs to be carefully done.. preferably programatically.
i dont know how we can prevent this from happening in the future.. we use efo1 and efo2 in the app and then EFO release says that efo2 is now replacing efo1

[ala-ebi]

discussed with @sprintell the possibility of pointing the references from the obsolete to the replacement efo, in case the replacement already exists, that would work for mongo, but once we replace the reference to the obsolete efo and replace it, once Sync runs it will attempt to delete the obsolete efo but will run into errors as there are still references to it in Oracle. what we can do is maybe flag the trait as obsolete and we add a replacement field in mongo, and the Sync will attempt to do the replacement?
we can later delete all obsolete efos or handle them properly in the rest of the apps

[ljwh2]

Full list with errors is here:
https://docs.google.com/spreadsheets/d/1ViLpRjv7QOKYp96UlZaMqEa_bXznwjB1L8pVSAfjEic/edit?usp=sharing

[ala-ebi]

fixed the majority by updating the EFO version we use

[ljwh2]

We were pointing to an old version of EFO as a fix for previous issue, now updated to use latest version