Biopragmatics

Comparing manually curated semantic mappings in SSSOM

2026-06-19T12:47:00+00:00

I am currently supporting Philip Strömert and Noura Rayya in the efforts to modernize and revitalize the Chemical Methods Ontology (CHMO) to support annotation of instrumentation used to produce experimental data captured in the Chemotion electronic laboratory notebook as part of NFDIChem. This post is about the adoption of Simple Standard for Sharing Ontological Mappings (SSSOM) to support interoperability between CHMO and other resources, and the workflow I developed to compare overlapping manual curations from different researchers.

Philip and Noura have already completed the important initial steps of assuming maintainership from the Royal Society of Chemistry, porting the ontology to use a standardized Ontology Development Kit (ODK) layout, and revising the definitions of many classes based on the IUPAC GoldBook.

Landscape of Resources

There are several other NFDI consortia including NFDI4Cat (catalysis), DAPHNE4NFDI (photon and neutron physics), and FAIRmat (materials science) that have similar goals to annotate instrumentation. While each reuse CHMO to some extent for this purpose, DAPHNE4NFDI additionally develops the Photon and Neutron Experimental Techniques (PANET) Ontology and FAIRmat develops the NeXus format and associated NeXus Ontology as part of the NOMAD materials science data management platform.

Further, there are several other resources with similar goals including the Allotrope Foundation Ontology (AFO), the deprecated Physico-chemical Methods and Properties (FIX) ontology, the deprecated Physico-chemical process (REX) ontology, IUPAC GoldBook, and Wikidata.

Establishing Interoperability

In order to establish interoperability between these many resources, we are using the Simple Standard for Sharing Ontological Mappings (SSSOM) to curate exact matches, narrow matches, and broad matches between CHMO terms and external ones in PANET, NeXuS (sort of), AFO, FIX, REX, IUPAC GoldBook, and Wikidata.

First, Philip had Ambika, a student research assistant (Hiwi, abbreviated in German), work for several months to manually curate mappings from CHMO to REX, FIX, AFO, and Wikidata (see this PR).

In parallel, I took the opportunity to spin up a new instance of a SSSOM Curator repository within the NFDI Section Metadata Working Group for Ontology Harmonization and Mappings GitHub repository, run lexical prediction to generate candidate mappings from CHMO, and efficiently manually curate the results in this PR. and this PR over the course of about an hour.

Need for Comparison

The next challenge was to efficiently triage the similarities and differences between my curations and Ambika’s. Therefore, I implemented a workflow for comparing the manually curated mappings in two SSSOM documents in cthoyt/sssom-pydantic#141. This workflow creates a Markdown file describing similarities and differences.

I chained together the following two CLI commands with sssom_pydantic to get the separate mapping files from Ambika’s branch in the NFDI4Chem fork of CHMO, merge them, then run the comparison against my own curations. Note that these won’t be reproducible after the branch is merged and deleted, and the actual results will change as more curation is done.

$ sssom_pydantic merge \
        --input https://github.com/NFDI4Chem/rsc-cmo/raw/refs/heads/Add-tsv-files/src/mappings/fix-mappings.sssom.tsv \
        --input https://github.com/NFDI4Chem/rsc-cmo/raw/refs/heads/Add-tsv-files/src/mappings/afo-mappings.sssom.tsv \
        --input https://github.com/NFDI4Chem/rsc-cmo/raw/refs/heads/Add-tsv-files/src/mappings/rex-mappings.sssom.tsv \
        --input https://github.com/NFDI4Chem/rsc-cmo/raw/refs/heads/Add-tsv-files/src/mappings/wikidata-mappings.sssom.tsv \
        --standardize \
        --output ambika.sssom.tsv
$ sssom_pydantic compare \
    ambika.sssom.tsv \
    https://github.com/nfdi-de/section-metadata-wg-onto/raw/refs/heads/main/sssom/data/positive.sssom.tsv \
    --standardize \
    --standardize-flip \
    --left-label Ambika \
    --right-label Charlie

Since the comparison workflow outputs Markdown, its results can easily be embedded in GitHub issues or my blog, which is itself written in Markdown.

Results

I am happy with the first version of the comparison workflow. Luckily, there were only a small number of discrepancies which have obvious solutions. There were also a few interesting discrepancies which were novel to either my or Ambika’s curations, which can be reviewed by a third curator (sorry Philip, more work for you).

Next Steps

I think that it can be extended to identify and report on one-to-many, many-to-one, and many-to-many mappings which arise when jointly examining two mapping sets. After Philip and others interact with the results, I’m sure we will be able to extend it with other analyses.

More generally, the implementation of the comparison workflow is part of a larger suite of workflows that I would like to describe in future posts including:

merging manually curated mappings
generating OWL ontology bridges
incorporating SSSOM into ODK builds, which I will support Damien Goutte-Gattat to document in the ODK repository and the OBOOK.
unify this analysis with my other idea for doing automated evaluation of predicted mappings, which I hope can be used to run future mapping challenges

Without further ado, here’s the comparison, copied verbatim from the output of the previous command:

Comparison between Ambika and Charlie

CHMO to FIX
CHMO to REX

CHMO to FIX

Subject Comparison

288 entities appear as subjects only in Ambika
19 entities appear as subjects only in Charlie only
138 entities appear as subjects in both

The following 6 subjects (4.3%) appearing in both have conflicting objects:

subject_id	subject_label	Ambika	both	Charlie
CHMO:0000141	diffraction method	FIX:0000004 (crystallography)	FIX:0000217 (diffraction method)
CHMO:0000164	electron scattering	FIX:0000666 (electron scattering spectroscopy)	FIX:0000401 (electron scattering)
CHMO:0000255	flame atomic emission spectroscopy	FIX:0000935 (spark method)	FIX:0000928 (flame atomic emission spectroscopy)
CHMO:0000307	X-ray emission spectroscopy	FIX:0000673 (X-ray fluorescence spectroscopy)	FIX:0000100 (X-ray emission spectroscopy)
CHMO:0000366	electron energy loss spectroscopy	FIX:0000664 (electron impact spectroscopy)	FIX:0000663 (electron energy loss spectroscopy)
CHMO:0000570	proton transfer reaction ion trap mass spectrometry	FIX:0000919 (proton transfer reaction ion trap mass spectrometry)		FIX:0000918 (proton transfer reaction mass spectrometry)

Object Comparison

296 entities appear as objects only in Ambika
19 entities appear as objects only in Charlie
138 entities appear as objects in both

The following 2 objects (1.4%) appearing in both have conflicting subjects:

object_id	object_label	Ambika	both	Charlie
FIX:0000629	pulsed field gel electrophoresis	CHMO:0002315 (pulsed-field electrophoresis)		CHMO:0002316 (pulsed-field gel electrophoresis)
FIX:0000816	square-wave polarography	CHMO:0000040 (square-wave voltammetry)	CHMO:0000035 (square-wave polarography)

Subject-Object Pair Comparison

301 subject-object pairs only appear in Ambika
20 subject-object pairs only appear in Charlie
137 subject-object pairs appear in both

The following 1 subject-object pairs (0.7%) appearing in have conflicting predicates or predicate modifiers:

subject_id	subject_label	object_id	object_label	warning	Ambika	Charlie
CHMO:0000164	electron scattering	FIX:0000401	electron scattering	different predicate	skos:narrowMatch	skos:exactMatch

CHMO to REX

Subject Comparison

1 entities appear as subjects only in Ambika
18 entities appear as subjects only in Charlie only
0 entities appear as subjects in both

Object Comparison

1 entities appear as objects only in Ambika
18 entities appear as objects only in Charlie
0 entities appear as objects in both

Subject-Object Pair Comparison

1 subject-object pairs only appear in Ambika
18 subject-object pairs only appear in Charlie
0 subject-object pairs appear in both

A practical approach to translate OWL to SKOS

2026-06-03T10:11:00+00:00

The data model in PyOBO closely resembles the Web Ontology Language (OWL) and macros in the OBO Flat File format. However, some communities prefer the simplified nature of the SKOS data model compared to the highly precise (and sometimes burdensome) nature of OWL semantics. This post describes the workflow I implemented to downscale OWL to SKOS.

Before diving into this, I searched for previous work in mapping OWL to SKOS and was surprised to find only a few things (and they’re all old):

Exploring the Relationships between OWL and SKOS by Nor Azlinayati Abdul Manaf (2009)
Using OWL and SKOS by Sean Bechhofer and Alstair Miles (2008)
SKOS2OWL by Hepp et al. (2007)

Predicate Mappings

However, I didn’t find these resources very helpful, so I just jumped right into an implementation. First, I mapped several predicates used in OWL to predicates used in SKOS:

rdfs:label maps to skos:prefLabel
synonym predicates from OBO-flavored OWL (oboInOwl:hasExactSynonym, oboInOwl:hasNarrowSynonym, oboInOwl:hasBroadSynonym, and oboInOwl:hasRelatedSynonym) are squashed to skos:altLabel, and synonym type information is thrown away
dcterms:description maps to skos:definition
rdf:subClassOf and rdf:type are squashed to skos:broadMatch
similarly, individuals (owl:NamedIndividual) and classes (owl:Class) are both squashed to skos:Concept
predicates and relationships are omitted
root terms annotated on the ontology with IAO:0000700 (ontology has root term) get annotated on the SKOS concept scheme with skos:hasTopConcept

Hierarchical Reasoning

I introduced OMO:0003014 (has ontology hierarchical property) in information-artifact-ontology/ontology-metadata#193 to enable ontologies to explicitly annotate which predicates should be used during hierarchical browsing (in addition to rdf:subClassOf). For example, the Uber Anatomy Ontology (UBERON) contains a partonomy of anatomical features, and should therefore be browsable based on the BFO:0000050 (part of) relationship. This annotation was originally inspired by the custom configuration for adding new ontologies to the EBI’s Ontology Lookup Service (OLS) - now, the configuration for the OLS can be upstreamed into the ontologies themselves and be made more reusable.

Accordingly, I used hierarchical properties annotated on to the ontology with OMO:0003014 to translate object property constraints into broad matches. Inside PyOBO, I updated the source for the Research Organization Registry (ROR) to explicitly annotate org:subOrganizationOf as a hierarchical property. Below, the entry for the French Techniques of Informatics and Microelectronics for Integrated Systems Architecture shows the translation into broad matches for both the subclass relation to the ad hoc term for a facility and the sub-organization relations to its four parent organizations:

ror:000063q30 a skos:Concept ;
    skos:altLabel "TIMA",
        "TIMA Laboratory",
        "Techniques de l'Informatique et de la Microélectronique pour l'Architecture des Systèmes Intégrés"@fr ;
    skos:broadMatch ,
        ror:02feahw73,
        ror:02rx3b187,
        ror:04z22qz54,
        ror:05sbt2524 ;
    skos:prefLabel "Techniques of Informatics and Microelectronics for Integrated Systems Architecture" ;
    skos:scopeNote "a facility in Grenoble established in 2003" .

Inverse Hierarchical Reasoning

I extended this notion further to leverage inverse predicate information for the hierarchical property annotations. If the inverse of a hierarchical property is used in an object property constraint (or annotation property), then it is translated into a narrow match.

For example, in FamPlex, complexes are annotated with the RO:0002351 (has member) relationship to genes. This algorithm looks up the inverse relationship RO:0002350 (is member of) and annotates that as a complex has narrow match to gene triple, and simultaneously, gene has broad match to complex triple.

fplx:Sarcoglycan_complex a skos:Concept ;
    skos:broadMatch fplx:DGC ;
    skos:inScheme  ;
    skos:narrowMatch hgnc:10805,
        hgnc:10806,
        hgnc:10807,
        hgnc:10808,
        hgnc:14075 ;
    skos:prefLabel "Sarcoglycan_complex" ;
    skos:scopeNote "A family of transmembrane dystrophin-associated proteins that play a role in the membrane association of the DYSTROPHIN-ASSOCIATED PROTEIN COMPLEX." .

More or Less, Semantics

We have the goal in NFDI4Chem to map from ontology terms in the Chemical Methods Ontology (CHMO) to the IUPAC GoldBook, a compendium of chemical terminology which also covers instrumentation and experimental techniques.

While GoldBook’s data is now available in both JSON and XML, neither ascribe semantics to their fields, i.e., the XML doesn’t use namespaces and the JSON doesn’t have a linked data component. This meant that as I implemented a PyOBO source for GoldBook in biopragmatics/pyobo#436, I had to interpret what each field means, and ascribe semantics to them by assigning existing, well-known predicates to each.

However, based on discussions in preparation for the 6^th Ontologies4Chem workshop later this year, it appears IUPAC does not want to make ontological commitments for its entities, and would instead prefer to produce a SKOS vocabulary.

My work on ontologizing GoldBook was done without discussion with IUPAC, similar to most PyOBO sources. Personally, I think it’s much more valuable to ascribe precise semantics. I am less dogmatic about ontological commitments and have usually focused on what makes a data resource useful for me. In this case, it was being able to access GoldBook through an ontology-like interface such that it could be loaded in SSSOM Curator to lexically predict then manually curate CHMO-GoldBook mappings.

However, now with the SKOS exporter described earlier in this post, I can have it both ways: first starting with a more precise OWL artifact, then downscaling to a simpler SKOS artifact with entries that look like:

goldbook:08003 a skos:Concept ;
    skos:inScheme  ;
    skos:prefLabel "analytical chemistry" ;
    skos:scopeNote "Scientific discipline that develops and applies strategies, instruments, and procedures to obtain information on the composition and nature of matter in space and time." .

There are still a few parts of SKOS that I’m not familiar with, so I expect that this translation will evolve over time. For example, because SKOS is so tightly tied to RDF as a serialization, it has better support for language tags. OWL can be serialized in RDF, but the OBO Flat File Format is inherently limited in its ability to express language. I’m interested to overcome these limits in the PyOBO implementation.

Transparent provenance for derived semantic mappings in SSSOM

2026-05-29T10:11:00+00:00

The Simple Standard for Sharing Ontological Mappings (SSSOM) requires semantic mappings to annotate a justification from the Semantic Mapping Vocabulary (SEMAPV) such as semapv:ManualMappingCuration, semapv:MappingInversion, and semapv:MappingChaining. However, SSSOM did not have a mechanism for tracking which mappings were used during inference workflows like inversion or chaining. This post is about my recent SSSOM enhancement proposal to add a new field derived_from to support this.

Background

Before we get started, it’s worth defining a few terms about mappings to make sure that we stay precise.

Mapping Triple

A mapping triple is a subject, predicate, and object such as mesh:C000089, skos:exactMatch, CHEBI:28646. By semantic web conventions based on the open world assumption (OWA), triples are assumed to be true unless stated otherwise.

flowchart LR
    mesh:C000089[ammeline
mesh:C000089]
CHEBI:28646[ammeline
CHEBI:28646]

mesh:C000089-->|skos:exactMatch|CHEBI:28646

Mapping Quadruple

A mapping quadruple is a subject, predicate, object, and truthiness (a boolean). SSSOM represents the truthiness using the predicate_modifier field, where an empty value means that the mapping is true, and a value of Not means that it is false.

For example, (mesh:C000089, skos:exactMatch, CHEBI:28646, True) makes the example mapping triple from above into a mapping quadruple that explicitly states its truthiness.

flowchart LR
    mesh:C000089[ammeline
mesh:C000089]
CHEBI:28646[ammeline
CHEBI:28646]

mesh:C000089-->|"skos:exactMatch\n(true)"|CHEBI:28646

Conversely, (CHEBI:10057, skos:exactMatch, mesh:C002563, False) is false because CHEBI:10057 refers to 9H-xanthene, a small molecule, and mesh:C002563 refers to xanthan gum, a polysaccharide.

flowchart LR
    mesh:C002563[xanthan gum
mesh:C000089]
CHEBI:10057[9H-xanthene
CHEBI:10057]

mesh:C002563-->|"not skos:exactMatch"|CHEBI:10057

By convention, mapping triples are implicitly considered to refer to the corresponding true mapping quadruple.

Mapping Record

A mapping record is a subject, predicate, object, truthiness, and other metadata about the mapping that appears in the SSSOM Mapping data model. A mapping record best be visualized by reifying the triple (i.e., make into a node) to show how additional metadata is connected.

flowchart LR
    mesh:C000089[ammeline
mesh:C000089]
CHEBI:28646[ammeline
CHEBI:28646]
1181E81AD58C1B94("mesh:C000089
skos:exactMatch
CHEBI:28646
manual curation")
style 1181E81AD58C1B94 fill:#bbf
orcid:0000-0003-4423-4370[Charles Tapley Hoyt
orcid:0000-0003-4423-4370]
style orcid:0000-0003-4423-4370 fill: #bef
wikidata:Q111239110[Biomappings
wikidata:Q111239110]
style wikidata:Q111239110 fill: #feb
1181E81AD58C1B94 -->|has author|orcid:0000-0003-4423-4370
1181E81AD58C1B94-->|source|wikidata:Q111239110
mesh:C000089 -->|subject of|1181E81AD58C1B94
CHEBI:28646 -->|object of|1181E81AD58C1B94

Note that I’m playing a bit fast and loose with the semantics of the predicates relating the mapping record to other components. This is explicitly specified as part of SSSOM, but I don’t think super instructive here, so I’ve used more human-readable labels.

Motivation

I’ve been thinking about the large-scale aggregation and inference over semantic mappings for several years now. My initial work resulted in the implementation of the Semantic Mapping Reasoner and Assembler (SeMRA) and the publication of its companion article Assembly and reasoning over semantic mappings at scale for biomedical data integration.

Initially, SeMRA implemented a custom data model that was similar to SSSOM, but had a more explicit provenance model for which mapping records (e.g., from SSSOM) were used to infer/derive new mapping quadruples. As SeMRA and its applications have matured, I have been able to backport many of its good ideas to SSSOM. This post is specifically about how I’ve proposed a simple, optional, explicit provenance model that allows mapping records in SSSOM derived through inference to reference the set of mapping quadruples that were used during inference, in a new derived_from slot.

Reference the Triple, Quadruple, or Record?

This slot is deliberately under-specified, but there’s a key nuance that the values in this field should point to identifiers for mapping quadruples, not mapping triples nor mapping records for the following reasons:

Mapping triples are insufficient: without the judgment of whether a mapping is true or false, then an algorithm could accidentally conclude from A skos:exactMatch B and B (not) skos:exactMatch C that A skos:exactMatch C. This is why mapping triples are insufficient
Inference happens on the mapping quadruple level, so it matches nicely
Full mapping records are inflexible: the SSSOM data should be flexible so if additional evidence (i.e., records) for a given mapping quadruple are found, then the confidence in the inferred/derived mapping (e.g., chained or inverted) can be adjusted accordingly. This is possible because most chaining and inversion algorithms logically operate on mapping quadruples, and not on records.

This is why in the following diagrams, there are mapping quads represented in pink boxes and mapping records represented by purple boxes.

Note: the local unique identifiers used for mappings in this example are related to the proposal in https://github.com/ts4nfdi/mapping-sameness-identifier (which currently is under review). For now, the SSSOM specification isn’t prescribing how to assign identifiers to mapping quadruples.

Contribution

See the original issue mapping-commons/sssom#537 and pull request mapping-commons/sssom#548. This post is an extension of the documentation I wrote in that PR (https://mapping-commons.github.io/sssom/dev/inference).

I implemented the derived_from field in SSSOM Pydantic in cthoyt/sssom-pydantic#108. Subsequently, while preparing this post, I implemented a mechanism for generating the pretty Mermaid diagrams I’ve used throughout this post in cthoyt/sssom-pydantic#129.

Here’s how this looks in Python:

from curies import Converter, NamableReference, NamedReference
from curies.vocabulary import charlie, manual_mapping_curation, mapping_chaining, exact_match

from sssom_pydantic import SemanticMapping, hash_triple_to_reference

CONVERTER = Converter.from_prefix_map({
    "BTO": "http://purl.obolibrary.org/obo/CHEBI_",
    "CL": "http://purl.obolibrary.org/obo/CL_",
    "mesh": "http://id.nlm.nih.gov/mesh/",
    "wikidata": "http://www.wikidata.org/entity/",
    "orcid": "https://orcid.org/",
    "semapv": "https://w3id.org/semapv/vocab/",
    "skos": "http://www.w3.org/2004/02/skos/core#",

})

E1 = NamedReference.from_curie("BTO:0006078", name="pluripotent stem cell")
E2 = NamedReference.from_curie("CL:0002248", name="pluripotent stem cell")
E3 = NamedReference.from_curie("mesh:D039904", name="Pluripotent Stem Cells")
SOURCE = NamableReference.from_curie("wikidata:Q111239110", name="Biomappings")

m1 = SemanticMapping(
    subject=E1,
    predicate=exact_match,
    object=E2,
    justification=manual_mapping_curation,
    authors=[charlie],
    source=SOURCE,
)
m2 = SemanticMapping(
    subject=E2,
    predicate=exact_match,
    object=E3,
    justification=manual_mapping_curation,
    authors=[charlie],
    source=SOURCE,
)
m3 = SemanticMapping(
    subject=E1,
    predicate=exact_match,
    object=E3,
    justification=mapping_chaining,
    # This is the new part!
    derived_from=[
        hash_triple_to_reference(m1, CONVERTER),
        hash_triple_to_reference(m2, CONVERTER),
    ],
)

Damien Goutte-Gattat also implemented the derived_from in SSSOM Java in gouttegd/sssom-java#19.

Examples

To finish off this post, I’ve included five real-world examples combining manually curated mappings from Biomappings and the ChEBI ontology that show off three types of inference: inversion, chaining, and background knowledge-based mapping.

Inversion

A mapping inversion workflow inverts the mapping predicate then swaps subject and object components. For example, the mapping predicate skos:narrowMatch andskos:broadMatch are inverses and skos:exactMatch is its own inverse. More information about mapping predicates can be found in the SSSOM documentation.

The introduction of the derived_from field allows an inverted mapping record to refer back to the mapping it was derived from.

In the example below, a manually curated mapping between MeSH and ChEBI’s terms for ammeline is inverted.

flowchart LR
    mesh:C000089[ammeline
mesh:C000089]
CHEBI:28646[ammeline
CHEBI:28646]
1181E81AD58C1B94("mesh:C000089
skos:exactMatch
CHEBI:28646
manual curation")
style 1181E81AD58C1B94 fill:#bbf
orcid:0000-0003-4423-4370[Charles Tapley Hoyt
orcid:0000-0003-4423-4370]
style orcid:0000-0003-4423-4370 fill: #bef
wikidata:Q111239110[Biomappings
wikidata:Q111239110]
style wikidata:Q111239110 fill: #feb
36a1f9244ea7641a90987c82f33c25c0c13712ee8f48207b2a0825f8a4e4e26a[["mesh:C000089
skos:exactMatch
CHEBI:28646"]]
style 36a1f9244ea7641a90987c82f33c25c0c13712ee8f48207b2a0825f8a4e4e26a fill: #f9f
54D1553627DC5514("CHEBI:28646
skos:exactMatch
mesh:C000089
inversion")
style 54D1553627DC5514 fill: #bbf
faeefc6d1dd08238a9732de5a3c9dcf99388e62fa8b1caaf9ba28c7eaf6d483a[["CHEBI:28646
skos:exactMatch
mesh:C000089"]]
style faeefc6d1dd08238a9732de5a3c9dcf99388e62fa8b1caaf9ba28c7eaf6d483a fill: #f9f
1181E81AD58C1B94 -->|has author|orcid:0000-0003-4423-4370
1181E81AD58C1B94-->|source|wikidata:Q111239110
mesh:C000089 -->|subject of|1181E81AD58C1B94
CHEBI:28646 -->|object of|1181E81AD58C1B94
36a1f9244ea7641a90987c82f33c25c0c13712ee8f48207b2a0825f8a4e4e26a -->|has evidence|1181E81AD58C1B94
CHEBI:28646 -->|subject of|54D1553627DC5514
mesh:C000089 -->|object of|54D1553627DC5514
faeefc6d1dd08238a9732de5a3c9dcf99388e62fa8b1caaf9ba28c7eaf6d483a -->|has evidence|54D1553627DC5514
54D1553627DC5514 -->|derived from|36a1f9244ea7641a90987c82f33c25c0c13712ee8f48207b2a0825f8a4e4e26a

Source SSSOM TSV

Discussions at the 7th NFDI4Chem Consortium Meeting

2026-05-18T14:51:00+00:00

The 7^th NFDI4Chem consortium meeting took place last week in Jena. This post is a summary of some of the interesting discussions I had there.

Ontologies and Mappings

I’ve been working on ontologies, semantic mappings, and knowledge graphs for nearly a decade. These interests fit well within NFDI4Chem TA4 and more generally within NFDI Section Metadata’s Working Group on Ontologies and Mappings. I want to highlight a few discussions I had about these topics at the 7^th NFDI4Chem consortium meeting:

Ontology for Theoretical Chemistry

I discussed with Mario Wolter and Philip Strömert on how we could support Mario’s use case of making molecular dynamics and simulation experimental metadata more fair by constructing an ontology for theoretical chemistry.

The NFDI4Chem Ontologies Collection on the Semantic Farm (adapted from Ontologies4Chem: the landscape of ontologies in chemistry) lists the Computational Chemistry Ontology from the World Avatar project, but that resource is effectively unusable, motivating the curation of a new ontology (which when applicable can map back to the World Avatar ontology).

We plan to use the Ontology Development Kit (ODK) to begin work and will first focus on capturing basis sets used in computation. The ODK supports us in curating mappings to the World Avatar project’s ontology (when applicable) in the Simple Standard for Sharing Ontological Mappings (SSSOM). Interestingly, in the week since the conference, we identified parallel work in the NFDI on the Molecular Simulation Ontology (MOLSIM) which will need to be considered further.

Semantic Mappings for CHMO

In the weeks leading up to the consortium meeting, I have been discussing with Philip Strömert how to leverage SSSOM Curator to predict and curate semantic mappings between CHMO, FIX, REX, IUPAC OrangeBook, IUPAC GoldBook, and DAPHNE4NFDI’s PaNET ontology. While his HiWi made initial mappings between CHMO and GoldBook (among other exciting improvements) in NFDI4Chem/rsc-cmo#17, I was able to prepare mappings at scale between all resources using SSSOM Curator, then curate them relatively quickly (within a few hours) in nfdi-de/section-metadata-wg-onto#88 and nfdi-de/section-metadata-wg-onto#89. Philip and I coordinated next steps for the ontology agenda for NFDI4Chem (TA4), which will include porting several other chemistry ontologies to using the ODK (RXNO, CHEMINF) and to write manuscript describing how to produce ontology bridge files from the SSSOM semantic mapping curations.

Annotating Interfacial Electrochemistry Data

Albert Engstfeld presented echemdb, a community project for interfacial electrochemistry and local research data management. They recently ported their data model to LinkML, but weren’t aware of how ontologies could be used to annotate data. I discussed with him how their free-text descriptions of electrode types could be replaced with references to terms in the Chemical Methods Ontology (CHMO) under the CHMO:0002344 (electrode) hierarchy.

Immediately, Albert identified missing electrode terms from CHMO and places where the terms could be better organized. Luckily, NFDI4Chem is currently under the process of taking stewardship over CHMO and making a major update, so NFDI4Chem will be able to support and enable Albert to improve the ontology to better suit his data resource. In general, this shows the power of the open data, open code, open infrastructure (O3) mindset codified by the OBO Foundry Principles that influenced the original development of CHMO.

Knowledge Graphs and Graph Machine Learning

I discussed with Kenichi Endo and Felix Neubauer (U. Stuttgart) how to find ontologies to annotate their data describing reaction and process steps, and ultimately how these could be converted into knowledge graphs. I also discussed graph machine learning methods with them, particularly knowledge graph embedding models (KGEMs) and the PyKEEN graph machine learning library that I co-developed.

Better Ontological Communication

On the train ride home, I sat with Theo Bender and reflected on the discussions with Albert, Kenichi, and many others. Despite NFDI4Chem being a nexus of ontology expertise, it’s actually the case that many people don’t really know what they are, how to find relevant ontologies for their work, or how to use them. This is an opportunity for me to participate in TA5 to help improve the educational material NFDI4Chem has, and a call to give more (less technical talks) demonstrating the usage of ontologies in NFDI4Chem to motivate others to get excited (tracked in NFDI4Chem/knowledge_base#508). Further, this ties into discussions about Semantic Farm (see below) and how it’s a tool that can help NFDI4Chem consortium members find the right ontologies for their work.

Training Materials and DALIA

On the train ride home, I also discussed with Hans-Georg Weinig (GDCh, German Chemical Society) how NFDI4Chem could incorporate training materials from the German Chemical Society into DALIA, the NFDI’s search portal for training materials, as part of TA5.

Semantic Farm

The Semantic Farm (previously called the Bioregistry, now adapted to be domain-agnostic) is a registry of ontologies, controlled vocabularies, terminologies, and other resources that mint (persistent) identifiers. While it’s already been widely adopted in the biomedical domain since its creation in 2019 (e.g., by the OBO Foundry, Monarch Initiative), I’ve been working to integrate it within NFDI4Chem and other NFDI consortia via the NFDI Section Metadata Working Group for Ontology Harmonization and Mapping. I want to highlight a few discussions I had about the Semantic Farm at the 7^th NFDI4Chem consortium meeting:

Extending Semantic Farm’s Provider Data Model

For each ontology, controlled vocabulary, terminology, and other resources that mint (persistent) identifiers, the Semantic Farm keeps track of one or more websites that can provide information about entities from that resource. For example, there is a first-party provider for the Gene Ontology that shows information terms like GO:0032571 (response to vitamin K). There are also many third party providers such as through the EBI Ontology Lookup Service (OLS) and Jackson Laboratories’ browser.

I talked with Steffen Neumann about extending the Semantic Farm’s data model for providers to include more information about:

what media types (e.g., HTML, JSON, RDF) the provider returns by default
whether content negotiation is possible by sending an Accept header to tell the server which media type to return

Steffen’s use case in NFDI4Chem TA3/TA6 is the Semantic Farm entry for MassBank (https://semantic.farm/massbank). Enabling Semantic Farm to resolve in different ways will support the development of computational workflows as well as make it more useful when implementing the MassBank front-end.

Adding Semantic Web Interoperability to Semantic Farm’s Resolver

I talked with Egon Willighagen (a longtime collaborator of mine and a member of one of the NFDI4Chem advisory boards) about how to extend the “resolver” functionality of Semantic Farm to support content negotiation. The resolver redirects URLs constructed with a CURIE like https://semantic.farm/GO:0032571 to the first-party (or best) web page for human reading.

Egon suggested that if a request contains an Accept header asking for text/turtle (or any other RDF-adjacent mimetype) that it could return the list of related URIs. Importantly, this functionality is already available via the API (https://semantic.farm/api/reference/GO:0032571) and on the front-end (https://semantic.farm/reference/GO:0032571), but not possible through the resolver endpoint.

In biopragmatics/bioregistry#1954, I extended the resolver so now it’s possible to use Accept headers like in the following:

import requests

res = requests.get(
   "https://semantic.farm/GO:0032571",
   headers={"Accept": "text/turtle"},
)

I also added a way of adding query parameters to get the same results when navigating to https://semantic.farm/GO:0032571?format=turtle (note the addition of ?format=turtle). For both, the following is returned:

@prefix rdfs:  .

 rdfs:seeAlso ,
        ,
        ,
        ,
        ,
        ,
        ,
        ,
        ,
        ,
        ,
        ,
        ,
        ,
         .

Semantic Farm and Base4NFDI

I’ve been working towards proposing the Semantic Farm as a Base4NFDI service (despite the recent announcement that Base4NFDI won’t be accepting any new proposals for funding). I discussed this with Martin Reinhardt and Hannah Butz from Base4NFDI at their poster (photo below borrowed from Martin’s post on LinkedIn).

They had the exciting idea to encourage people to draw on their poster, so I added the relationships between TS4NFDI, KG4NFDI, and PID4NFDI and the Semantic Farm. It’s possible to see in the photo if you squint, but here’s the same arrows drawn again:

flowchart LR
    ts4nfdi[TS4NFDI] -->|already uses|sf[Semantic Farm]
    kgi4nfdi[KGI4NFDI] -->|should use|sf
    pid4nfdi[PID4NFDI] -->|could use|sf

We discussed how the Semantic Farm is already a key service as part of the TS4NFDI, how it should be adopted by KGI4NFDI (e.g., to make sure that all NFDI knowledge graphs use the same prefixes, CURIEs, and URIs for the same things), and how could fit in with PID4NFDI to support them in better communicating what identifiers do, where they come from, and how they make data FAIR beyond the limited number of identifier spaces (e.g., DOI, ORCiD, ROR, Handles) on which they currently focus. They also suggested I attend the Base4NFDI User Conference 2026 in Berlin to communicate these things further.

Chemotion

I’m currently leading the effort to develop a data infrastructure for the Catalaix project which will capture experimental information about polymerization and depolymerization reactions from both the literature and experiments in our laboratory. Our researchers are using Chemotion, so I discussed requirements for programmatically extracting data from Chemotion with Shashank Harivyasi (KIT), the Chemotion developer responsible for Chemotion’s API. I also discussed with Nicole Jung (KIT) our more general need to identify and reuse, or develop, an external standard for reaction information.

I’m a big fan of these meetings, especially because they are typically short and well-attended. While this post focused on bigger picture discussions, I also had lots of small talks with friends and collaborators like Kohulan and Chandu that were completely invaluable to making NFDI4Chem a great experience. I’m looking forward to our next big in-person meetup at Ontologies4Chem in Limburg in November!

International Society of Biocuration Presents: Curate This!

2026-03-03T10:13:00+00:00

While researchers typically communicate their work through poster presentations, oral presentations, and written communication, programmers often give (live) demonstrations. I’m not aware of any technical nor practical barriers for why curators couldn’t do the same, and always wished that curators did this more often. This post is about how I planned to make this a reality by starting a podcast with the International Society for Biocuration (ISB) entitled ISB Presents: Curate This!.

The key first step was to decide on the goals of the podcast and its target audience. The primary goal of Curate This! is to explicitly show the process of curation and have an informal discussion about the challenges associated with it. It should have short episodes that require as little preparatory work for both the interviewer and interviewee as possible to make it possible to scale. I also decided with the ISB that it should be hosted as an ISB podcast, not as something just from me. This better fits the message for the curation community and is overall a better governance decision to support longevity (if we’re successful).

It’s not a goal of this podcast to give a background on curation - there are plenty of resources available from the ISB that cover this. It’s also not a goal of this podcast to focus on the curator themselves, such as how they became a curator - the ISB hosts two Careers in Biocuration sessions each year, one at the in-person conference and one virtually, that cover this. The target audience for this podcast is practitioners.

Script

Curate This! interviews are split into several segments. The first and last are recorded by the interviewer after the interview is done to give an introduction and parting remarks. The bulk of the episode is contained within three segments: introduction, demonstration, and reflections.

I’ve written out the questions and the concept for each segment below to serve as a resource for potential interviewees to read ahead of time and prepare themselves as well as a resource for interviewers to follow and stay on task.

Introduction

The goal of the first segment of the interview is to describe the history, goals, and uses of your curated resource in around five minutes. We’ll loosely use the following question lists:

Basic
- What is your resource called (if not already mentioned in the introduction)?
- When was your resource established?
- What kind of information is in your resource?
- Do you develop/reuse any data standards?
Impact
- Who uses (or could use) your resource and why? How do you assess this?
- Have you seen any cool citations of your resource?
- What is the broader impact in the basic and translational research space in biomedicine (or beyond?)
Personnel
- What does your resource’s team look like?
  - How many people/groups work on your resource?
  - Is it developed and maintained by a group within your institution, as a community effort, or somewhere in between?
- If it’s a community effort, How do you do project management and communication? E.g., Slack, GitHub, Trello, etc.
- How do you onboard new curators? If there’s a difference between internal/external, what does this dichotomy look like?

Demonstration

The goal of the second segment of the interview is to demonstrate the contribution of a curation to your resource, live, in between ten and thirty minutes. Here’s what makes a satisfying live demonstration:

Show how you select what you’re going to curate.
- How do you find content? For example, if you curate text from literature or patents, do you have a search query that runs on a chronological basis?
- How do you prioritize content? For example, do you use ranking from a search system, or a more sophisticated document classifier?
What do you look for in the text?
- Do you use external ontologies, terminologies, or semantic spaces to tag named entities?
What kinds of assumptions do you make as a curator? For example, if you’re curating relationships between proteins, do you assume that authors refer to proteins using their corresponding gene names?
How do you report the confidence of your curation (and its components)?
What kind of metadata do you capture, e.g., the curator’s ORCiD, the time of curation, or anything else?

Ideally, you should prepare a curation ahead of time so you can quickly walk through the process during the live demo, rather than needing time to think and consider (though, this might be more realistic!).

Reflections

The goal of the third segment of the interview is to reflect on the demonstration and conclude the interview with parting thoughts, in around five minutes.

Next Steps
- What happens next after curation?
  - Does the data get reflected on the website immediately?
  - Does a second curator check things?
  - Are more substantial releases made periodically?
- What are some difficulties/challenges in curating your resource?
- What could authors/journals/publishers do to make it easier to curate?
  - What data should they include (that they don’t)?
  - How should data look?
  - What kinds of standards would you like to see developed?
- Contrast curating a “good” paper versus a “bad” paper.
Longevity and Sustainability
- How/where do you think that AI has a place in the curation and maintenance of your resource?
- What’s the funding situation like?
- How much do you estimate it costs to maintain this resource per year?

First episode

In our inaugural episode, I interviewed Dr. Susan (Sue) Bello, a curator for the Mouse Genome Informatics (MGI) knowledge base and Alliance of Genome Resources (AGR) who works at the Jackson Laboratory in Maine. Sue is also the ISB executive committee chair. She showed us how she curates alleles in MGI using the paper Mice deficient in TWIK-1 are more susceptible to kainic acid-induced seizures (Kim et al., 2025).

Let us Interview You

If you curate a resource and want to be featured on the podcast, please fill out this form.

Efficient Bulk Access to Citations in OpenCitations

2026-02-10T09:46:00+00:00

OpenCitations aggregates and deduplicates bibliographic information from CrossRef, Europe PubMed Central, and other sources to construct a comprehensive, open index of citations between scientific works. This post describes the opencitations-client package which wraps the OpenCitations API and implements an automated pipeline for locally downloading, caching, and accessing OpenCitations in bulk.

Background

OpenCitations both provides access via an API and bulk data downloads distributed across FigShare and Zenodo. Importantly, it publishes its data under the CC0 public domain license to democratize access to citations - previously, this data was only available through paid access to commercial databases owned by publishers.

While API access can be convenient for ad-hoc usage, it’s generally slow, rate-limited, susceptible to DDoS (e.g., from crawlers), and therefore difficult (if not impossible) to use in bulk. My solution is to write software that automates downloading, processing, and caching databases in bulk and provides fast, highly available, local access. I’ve previously written about developing standalone software packages for several large databases including DrugBank, ChEMBL, UMLS, ORCiD, and ClinicalTrials.gov. Similarly, I maintain several similar workflows in the PyOBO software package for converting resources into ontology-like data structures. I previously wrote about how this looks for HGNC.

Building on an Existing Ecosystem

I’ve been developing a software ecosystem over the last decade to support common workflows in research data management and data integration. When I start a new project, I try and reuse or improve existing components from that ecosystem wherever possible. Importantly, I try and find meaningful ways of organizing code across my ecosystem to reduce duplication, separate concerns, reduce the burden of testing, and ease maintenance.

OpenCitations publishes its bulk data dumps across several records in Figshare and Zenodo. I’ve previously written zenodo-client to interact with Zenodo’s API and orchestrates downloading and caching. zenodo-client heavily builds on pystow, which implements I/O and filesystem operations to enable reproducible, automated downloading, caching, and opening of data.

I had not previously written software to interact with Figshare, so I followed the form of zenodo-client and created a new package, figshare-client. I’m able to quickly create new high-quality packages because I’ve encoded all the wisdom and experience I’ve gained over the years in a Cookiecutter template, cookiecutter-snekpack, which I can use to set up a new project in mere minutes.

Along the way, I realized that the archives in Zenodo and Figshare were a combination of TAR and ZIP archives, each with many CSV files inside. In Python, TAR and ZIP archives have lots of weird quirks, even though they mostly do the same thing. However, rather than addressing those issues in opencitations-client, it made more sense to add utility functions in PyStow in cthoyt/pystow#125 (tar and zip archive iteration), which I was much better able to test in the PyStow archive.

A key functionality of OpenCitations is to implement graph-like queries to find incoming and outgoing citations. I considered several solutions for efficiently caching and querying graph-like data including pickles and SQLite, but these were respectively slow and disk inefficient. I found better solutions based on NumPy’s memory maps and was surprised that I couldn’t find an implementation in a popular package (e.g., SciPy). So, I had to decide where to put an implementation of disk-based cached graph. I didn’t want to put it in OpenCitations nor make a tiny package for just this one operation, so I decided to expand the scope of PyStow and add it there in cthoyt/pystow#121.

Finally, OpenCitations deals with a variety of identifier spaces including first-party OpenCitations Metadata IDs (OMIDs) and OpenCitations Citation IDs (OCIs) as well as third-party identifiers from Wikidata, OpenAlex, PubMed, DOI, and others. I’ve written the curies to handle identifiers in an explicit and transparent way. In the end, the opencitations-client relies on several components from my ecosystem, and of course, several more generic and popular packages. Here’s how the dependencies look:

flowchart LR
    opencitations-client -- depends on --> figshare-client
    opencitations-client -- depends on --> zenodo-client
    opencitations-client -- depends on --> curies
    figshare-client -- depends on --> pystow
    zenodo-client -- depends on --> pystow

Demo

It’s important for software packages to implement simple, top-level APIs that cover 99% of use cases with reasonable defaults. Most use cases for OpenCitations are to get incoming/outgoing citations for a DOI, PubMed identifiers, or OpenCitations identifiers. Here’s how this looks:

from curies import Reference
from opencitations_client import get_incoming_citations, get_outgoing_citations

# a CURIE for the DOI for the Bioregistry paper
bioregistry_curie = "doi:10.1038/s41597-022-01807-3"

# who did the Bioregistry paper cite?
outgoing: list[Reference] = get_outgoing_citations(bioregistry_curie)

# who cited the Bioregistry paper?
incoming: list[Reference] = get_incoming_citations(bioregistry_curie)

Importantly, each of these functions has a backend argument that defaults to api and can be swapped to local. Because everything is built on software that is smart about caching, loading, and data workflows, on the first time backend='local' is used, all processing happens automatically (warning, takes a few hours on a single core). This function also has a return_value argument that can be used to swap between principled curies.Reference data structures that explicitly encode identifiers, simple string local unique identifiers that match the input prefix, or full citation objects (only available through OpenCitations API).

See the opencitations-client code on GitHub (https://github.com/cthoyt/opencitations-client) and documentation on ReadTheDocs (https://opencitations-client.readthedocs.io).

While I’ve been thinking about adding citations to the bibliographic components of knowledge graph construction workflows for several years, I was finally pushed to implement opencitations-client for the Catalaix project, where we’re developing new methods for recycling and reuse of (bio)plastics. I wanted to get all seventeen laboratories’ publications, who they cited, and who cited them as a seed for information extraction and curation. Here’s a small example of a citation network from those queries:

flowchart TD
    26802344["Mechanism-specific and whole-organism ecotoxicity of mono-rhamnolipids.
Blank (2016)"]
34492827["The Green toxicology approach: Insight towards the eco-toxicologically safe development of benign catalysts.
Herres-Pawlis (2021)"]
28779508["Highly Active N,O Zinc Guanidine Catalysts for the Ring-Opening Polymerization of Lactide.
Herres-Pawlis (2017)"]
33195133["Genetic Cell-Surface Modification for Optimized Foam Fractionation.
Blank (2020)"]
32974309["Integration of Genetic and Process Engineering for Optimized Rhamnolipid Production Using
Jupke, Blank (2020)"]
30811863["New Kids in Lactide Polymerization: Highly Active and Robust Iron Guanidine Complexes as Superior Catalysts.
Pich, Herres-Pawlis (2019)"]
30758389["Tuning a robust system: N,O zinc guanidine catalysts for the ROP of lactide.
Pich, Herres-Pawlis (2019)"]
28524364["Biofunctional Microgel-Based Fertilizers for Controlled Foliar Delivery of Nutrients to Plants.
Pich, Schwaneberg (2017)"]
34865895["A plea for the integration of Green Toxicology in sustainable bioeconomy strategies - Biosurfactants and microgel-based pesticide release systems as examples.
Pich, Blank, Schwaneberg (2022)"]
32449840["Robust Guanidine Metal Catalysts for the Ring-Opening Polymerization of Lactide under Industrially Relevant Conditions.
Herres-Pawlis (2020)"]
34492827 --> 30811863
34492827 --> 30758389
34492827 --> 28779508
34492827 --> 32449840
32974309 --> 33195133
34865895 --> 26802344
34865895 --> 32974309
34865895 --> 28524364
34865895 --> 34492827

Challenges with Semantic Mappings

2026-01-20T10:42:00+00:00

There are many challenges associated with the curation, publication, acquisition, and usage of semantic mappings. This post examines their philosophical, technical, and practical implications, highlights existing solutions, and describes opportunities for next steps for the community of curators, semantic engineers, software developers, and data scientists who make and use semantic mappings.

Proliferation of Formats

The first challenge with semantic mappings is the variety of forms they can take. This both includes different data models and serializations of those models. This problem is effectively solved, but I think is worth reviewing for historical purposes (please let me know if I missed something):

Simple Knowledge Organization System (SKOS) is a data model for RDF to represent controlled vocabularies, taxonomies, dictionaries, thesauri, and other semantic artifacts. It defines several semantic mapping predicates including for broad matches, narrow matches, close matches, related matches, and exact matches.

JSKOS (JSON for Knowledge Organization Systems), a JSON-based extension of the SKOS data model. I recently wrote a post about converting between SSSOM and JSKOS.

Web Ontology Language (OWL) is primarily used for ontologies. It has first-class language support for encoding equivalences between classes, properties, or individuals. Other semantic mappings can be encoded as annotation properties on classes, properties, or individuals, e.g., using SKOS predicates.

The OBO Flat File Format is a simplified version of OWL with macros most useful for curating biomedical ontologies. It has the same abilities as OWL, but also the xref macro which corresponds to oboInOwl:hasDbXref relations, which are by nature imprecise and therefore used in a variety of ways.

The Simple Standard for Sharing Ontological Mappings (SSSOM) is a fit-for-purpose format for semantic mappings between classes, properties, or individuals. SSSOM guides curators towards inputting key metadata that are typically missing from other formalisms and is gaining wider community adoption. Importantly, SSSOM integrates into ontology curation workflows, especially for Ontology Development Kit (ODK) users.

The Expressive and Declarative Ontology Alignment Language (EDOAL) lives in a similar space to SSSOM, but IMO was much less approachable (c.f. XML + Java), and has not seen a lot of traction in the biomedical space.

OntoPortal has its own data model for semantic mappings that has low metadata precision. I recently wrote a post on converting OntoPortal to SSSOM. OntoPortal would also like to invest more in SSSOM infrastructure if it can organize funding and human resources.

Wikidata has its own data model for semantic mappings that include higher precision metadata. I recently wrote a post on mapping between the data models from SSSOM and Wikidata.

Finally, there’s a long tail of mappings that live in poorly annotated CSV, TSV, Excel, and other formats. Similarly, mappings can live in plain RDF files, e.g., encoded with SKOS predicates, but without high precision metadata.

Scattered, Partially Overlapping, and Incomplete

Semantic mappings are not centralized, meaning that multiple sources of semantic mappings often need to be integrated to map between two semantic spaces. Even then, these integrated mappings are often incomplete. Using Medical Subject Headings (MeSH) and the Human Phenotype Ontology (HPO) as an example, we can see the following:

MeSH doesn’t maintain any mappings to HPO.
HPO maintains some mappings as primary mappings.
The Unified Medical Language System (UMLS) maintains some mappings as secondary mappings. HPO suggests using UMLS as a supplementary mapping resource.
Biomappings maintains some community-curated mappings as secondary mappings.

This actually might not be the best example - it would have been better to show a pair of resources that both partially map to the other. When I first made this chart, I had to engineer the UMLS inference by hand. Eventually, the need to generalize this workflow led to the development of the Semantic Mapping Reasoner and Assembler (SeMRA) Python package which does this automatically and at scale. The fact that there were missing mappings that even UMLS inference couldn’t retrieve led to establishing the Biomappings project for prediction and semi-automated curation of semantic mappings. The underlying technology stack from Biomappings eventually got spun out to SSSOM Curator and is now fully domain-agnostic.

Different Precision or Conflicts

Another challenge with semantic mappings is when different resources have different level of precision. In the example below, OrphaNet uses low-precision mapping predicates (i.e., oboInOwl:hasDbXref) while MONDO uses high-precision mapping predicates (i.e., skos:exactMatch). It makes sense to take the highest quality mapping in this situation, but having a coherent software stack to do this at scale was the big challenge (solved by SeMRA).

This can get a bit dicier when there might be conflicting information, for example, if one resource says exact match and another says broader match. In SeMRA, I devised a confidence assessment scheme (which should get its own post later).

Common Conflations

There are three flavors of conflations that make curating and reviewing mappings difficult that I want to highlight.

Different Ontology Encodings

Classes, instances, and properties are mutually exclusive by design. This means that any semantic mappings between them are nonsense, but there are many situations where these mappings might get produced by an automated system or by a curator who is less knowledgable about the ontology aspect of semantic mappings. There’s also a much more subtle discussion about classes, instances, and metaclasses ( see this discussion) that I would set aside.

As a concrete example, the Information Artifact Ontology (IAO) has a class that represents the section of a document that contains its abstract: abstract (IAO:0000315). Schema.org has an annotation property whose range is a creative work and whose domain is the text of the abstract itself: schema:abstract. These both have the same label abstract, which means that it’s possible to conflate (i.e., accidentally map them).

Different Entity Types

The second kind of conflation is even more subtle, when two classes, instances, or properties come from similar but distinct hierarchies.

For example, there’s a subtle difference between what is a phenotype and what is a disease. Ontologies are highly apt at encoding this subtlety with axioms that can then be used by reasoners. This can become a problem for curating and reviewing semantic mappings because some diseases are named after the phenotype that it presents or that causes it. Using MeSH’s disease hierarchy and HPO’s phenotype hierarchy as an example, we can see that Staghorn Calculi (mesh:D000069856) and Staghorn calculus (HP:0033591) should not get mapped.

Many more examples can be produced (which also show there are even more subtleties here) using SSSOM Curator with the command: sssom_curator predict lexical doid hp. See the SSSOM Curator documentation for more information on the lexical matching workflow.

Different Senses

The basic formal ontology (BFO) is an upper-level ontology that is used by many ontologies, including almost the entire Open Biomedical Ontologies (OBO) Foundry. However, as Chris Mungall described in his blog post, Shadow Concepts Considered Harmful, there are many different senses in which an entity can be described, each falling under a different, mutually exclusive branch of BFO. The figure below, from Chris’s post, represents different senses in which a human heart can be described:

This problem is particularly bad in disease modeling. Here are only a few examples (of many more) that illustrate this:

the Ontology for General Medical Science (OGMS) term for disease (OGMS:0000031), the Experimental Factor Ontology (EFO) term for disease (EFO:0000408), Monarch Disease Ontology (MONDO) term for disease (MONDO:0000001) is a disposition (BFO:0000016)
the Gender, Sex, and Sexual Orientation Ontology (GSSO) term for disease (GSSO:000486) is a process (BFO:0000015)
the Human Disease Ontology (DOID) informally mentions that a disease is a disposition, but doesn’t make an ontological commitment to BFO
many more controlled vocabularies including NCIT, SNOMED-CT, and MI have their own terms for diseases but don’t use BFO as an upper-level ontology nor are constructed in a way conducive towards integration with other ontologies

Schultz et al. (2011) proposed a way to formalize the connections between the various senses for diseases in Scalable representations of diseases in biomedical ontologies. However, the OBO community has yet to resolve the long and taxing discussion on how to standardize disease modeling practices.

For semantic mappings, this becomes a problem because a reasoner will explode if diseases under two different BFO branches get marked as equivalent, because the BFO upper level terms are marked as disjoint - this is a feature, not a bug. However, while useful for creating carefully constructed, logically (self-)consistent descriptions of diseases, these modeling choices can be confusing when curating or reviewing mappings. These modeling choices might not be so important in downstream applications, such as assembling a knowledge graph to support graph machine learning, where many different knowledge sources with lower levels of accuracy and precision must be merged. In practice, I have merged triples using conflicting senses for diseases in a useful way, without issue.

Interpretation is Important

While the last few examples were cautionary tales for when things (probably) shouldn’t be mapped, the next examples are about when things (probably) should be mapped.

Definitions

Here are three vocabularies’ terms for proteins and their textual definitions (though, many more contain their own term for proteins):

Entity	Label	Description
wikidata:Q8054	protein	biomolecule or biomolecule complex largely consisting of chains of amino acid residues
SIO:010043	protein	A protein is an organic polymer that is composed of one or more linear polymers of amino acids.
PR:000000001	protein	An amino acid chain that is canonically produced de novo by ribosome-mediated translation of a genetically-encoded mRNA, and any derivatives thereof.

As semantic mapping curator, we have two options:

We can reasonably assume that the intent from all three resources was to represent the same thing, despite the definitions being quite different. This assumption can be built on our prior knowledge about what a protein is, why Wikidata, SIO, and PR exist, and then infer the intent of the term’s definition’s author
We can make a very literal reading of the definition and conclude that these three terms represent very different things

I think that the latter is really unconstructive for several reasons, but I have worked with colleagues, especially from the linguistics background, who take this approach. First, this is unconstructive because it means you’ll probably never map anything.

Second, if you want to be rigorous, use an ontology formalism with proper logical definitions. For example, the Cell Ontology (CL) exhaustively defines its cells using appropriate logical axioms. However, this also has a caveat, that to make mappings based on logical definitions, then the different modelers have to agree on the same axioms and same modeling paradigm. As far as I know, there aren’t any groups out there that use the same modeling paradigm that haven’t just combine forces to work on the same resource. So we’re stuck back at option 1 either way :)

Context Sometimes Matters

In contrast to the discussion about mapping phenotypes and diseases, there are context-dependent reasons to make semantic mappings, which can be illustrated in biomedicine using genes and proteins. Let’s start with some definitions:

SO:0000704 A gene is a region of a chromosome that encodes a transcript
PR:000000001 A protein is a chain of amino acids

The biomedical literature often uses gene symbols to discuss the proteins they encode. While this isn’t precise, it’s still useful in many cases. Therefore, when reading the COVID-19 literature, you will likely see discussion of the IL6-STAT cascade, where IL6 is the HGNC gene symbol for the Interleukin 6 protein. Most of the time, the HGNC approved gene symbol is an initialism or other abbreviation of the protein, but this isn’t always the case.

Edit: Sue Bello pointed out that most journals enforce gene names being put in italics (IL6) and proteins without italics, though this requires the author and reader to know that distinction, as well as for formatting to be preserved, which it often isn’t unless you’re reading the original PDF or publisher’s HTML.

Similar to the literature, many pathway databases that accumulate knowledge about the processes and reactions in which proteins take part actually use gene symbols (or other gene identifiers) to curate proteins.

The take-home message here is that genes and proteins are indeed not the same thing, but in some contexts, it’s useful to map between them. There’s also a compromise - the Relation Ontology (RO) has a predicate has gene product (RO:0002205) that explicitly models the relationship between IL6 and Interleukin 6, which can then be automatically inferred to mean a less precise mapping for certain scenarios (SeMRA implements this).

Outside of biomedicine, I have also heard that context-specific mappings are very important in the digital humanities. As I’m better understanding the use cases of colleagues in other NFDI Consortia that focus on the digital humanities, I will try and update this section to have alternate perspectives.

Evidence

A key challenge that motivated the development of SSSOM as a standard was to associate high-quality metadata with semantic mappings, such as the reason the mapping was produced (e.g., manual curation, lexical matching, structural matching), who produced it (e.g., a person, algorithm, agent), when, how, and more.

We developed the Semantic Mapping Vocabulary (semapv) to encode different kinds of evidence such as for manual curation of mappings, lexical matching, structural matching, and others. SSSOM is well-suited towards capturing simple evidences (blue).

Provenance for Inferences

The purple evidence from the figure in the last section requires a more detailed data model to represent provenance for inferred semantic mappings that simply doesn’t fit in the SSSOM paradigm (and it shouldn’t be hacked in, either). I proposed a more detailed data model for capturing how inference is done in Assembly and reasoning over semantic mappings at scale for biomedical data integration and provided a reference implementation in the Semantic Reasoning Mapper and Reasoner (SeMRA) Python software package. Here’s what that data model looks like, which also has a Neo4j counterpart:

Negative Semantic Mappings

SSSOM also has first-class support for encoding negative relationships, meaning that the following can be represented:

This means that SSSOM curators can keep track of non-trivial negative mappings, e.g., when curating the results of semantic mapping prediction or automated inference. In a semi-automated curation loop, this allows us to avoid re-reviewing zombie mappings over and over again.

High quality, non-trivial negative mappings also enable more accurate machine learning, as opposed to using negative sampling. For example, we have been working on developing graph machine learning-based ontology matching and merging using PyKEEN (a graph machine learning package I helped develop and maintain).

An open challenge is that we neither have support from data modeling formalisms (e.g., ontologies in OWL, knowledge graphs in RDF or Neo4j) to encode negative knowledge (in this case negative mappings) nor tooling support. This means that when we output SSSOM to RDF, we use our own formalism, which won’t be correctly recognized by any other tooling that wasn’t developed with SSSOM in mind. I’m keeping notes about this in a separate post about negative knowledge that I update periodically.

Despite the challenges, I think that the mapping world is actually getting quite mature. I am currently working with NFDI and RDA colleagues to further unify the SSSOM and JSKOS worlds, especially given that the Cocoda mapping curation tool solved many of these problems (from the digital humanities perspective) many years ago, and we simply were unaware of it.

I hope this post can continue as a living document - if I missed something, please let me know and I will update the post to include it!

Semantic Mappings Enable Automated Assembly

2026-01-16T10:42:00+00:00

Data and knowledge originating from heterogeneous sources often use heterogeneous controlled vocabularies and/or ontologies for annotating named entities. Semantic mappings are essential towards resolving these discrepancies and integrating in a coherent way. This post highlights how this looks in two scenarios: when constructing a knowledge graph for graph machine learning and when constructing a comprehensive lexica for natural language processing, text mining, and curation.

Background

Data and knowledge integration are challenging because there exist many controlled vocabularies, ontologies, taxonomies, thesauri, classifications, and other resources that mint identifiers with some degree of conceptual overlap, redundancies, and discrepancies.

Problem Statement

When integrating data and knowledge from heterogeneous sources that refer to the same concepts using different identifiers, we get non-trivial duplications and missing connections in our results.

For example, if we constructed a knowledge graph by combining the Comparative Toxicogenomics Database (CTD) and the Monarch Disease Ontology (MONDO), we would get disconnected mechanisms describing how sapropterin is used to treat phenylketonuria because the CTD uses the Medical Subject Headings (MeSH) to describe genes/proteins and MONDO uses the HUGO Gene Nomenclature Committee (HGNC).

flowchart LR
    a["sapropterin (mesh:C003402)"] -- activates --> b["Phenylalanine Hydroxylase (mesh:D010651)"]
    c["phenylalanine hydroxylase (HGNC:8582)"] -- decreases --> d["phenylketonuria (MONDO:0009861)"]
    b -...-|missing connection, these should be collapsed together| c

As a consequence, these redundancies lead to inaccurate results, for example, when making queries between drugs and diseases or when using machine learning algorithms to make predictions for new edges.

Causes

I roughly classify these redundancies into three bins (from left to right in the figure): similar domain, hierarchically related domain, and non-specific to a domain. Below, I’ll give some concrete examples from the life sciences to illustrate.

In chemistry, there are dozens of resources that assign identifiers to small molecules. Many have been constructed with unique scope or purpose such as MetaboLights for metabolites, SwissLipids for lipids, DrugBank for drugs. Some have similar scope and purpose, but have been constructed in parallel due to scientific modeling reasons, such as different disease ontologies modeling diseases with different parts of the Basic Formal Ontology (BFO). Some have similar scope and purpose, but have been constructed in parallel due to non-scientific reasons, such as PubChem and ChEMBL for small molecules with assay information. As an aside, building resources in an open and collaborative manner can help reduce proliferation, with the (major) caveat that they don’t satisfy funding bodies nor the requirements for career progression so easily.

In medicine and epidemiology, there are many resources describing diseases, transmission, response, adverse outcomes, and other facets. Particularly during the COVID-19 pandemic, many independent controlled vocabularies were constructed to model information at various levels of specificity. The figure shows the hierarchical relationships between the Disease Ontology (DOID), the Infectious Disease Ontology (IDO), the Viral Infectious Disease Ontology (VIDO), the Coronavirus Infectious Disease Ontology (CIDO), and the COVID-19 Infectious Disease Ontology (IDOCOVID19). When effectively reusing terms (as OBO Foundry Ontologies often do), this doesn’t create an issue, but in practice, many resources do not reuse terms for various reasons.

In the life sciences, there are several controlled vocabularies that cover a large number of domains such as the Medical Subject Headings (MeSH), National Cancer Institute Thesaurus (NCIT), and Unified Medical Language System (UMLS). While they give good coverage across many domains, these resources are often neither detailed, precise enough, nor curated as ontologies. Therefore, many controlled vocabularies use terms from these resources as a base and curate further. However, this causes redundancy, and in many cases, the group does not correctly cross-reference back. The Ontology for MicroRNA Target (OMIT) even imported the entirety of MeSH, but didn’t make any cross-references back to the source, creating even more redundancy.

If you were wondering why for each domain, we couldn’t just have a single resource, then please have a look at https://xkcd.com/927 :) Though, some resources that have been around for a long time basically have a monopoly. For example, nobody in their right mind in 2026 would start their own protein database to compete with UniProt.

Assembly

I want to highlight two groups for whom resolving redundancy has a high impact, but not necessarily high visibility. The first group is data scientists who consume knowledge graphs, for example, for graph machine learning. This group is often unaware of how graphs were constructed (see: any graph machine learning literature since 2013 that blindly uses FB15k and WN18).

The second group is curators, who want to use a combination of terminology services like the Ontology Lookup Service (OLS) and text mining tools to annotate the literature with controlled vocabulary terms. Curators don’t want to (and shouldn’t have to) understand the landscape of related controlled vocabularies for their domain and should just be responsible with terminology services and text mining tools to find any appropriate term for their curation.

The important point is that software should solve the problem of redundancy, and it needs to do so by consuming semantic mappings that bridge the gap illustrated above in the phenylketonuria example.

This leads to the main goal of the post, which is to describe two high-level workflows that can resolve redundancies and discrepancies when integration data and knowledge by using semantic mappings. This post isn’t about where semantic mappings come from - see my other posts on SSSOM, JSKOS, and SeMRA for more background on that.

Knowledge Graph Assembly

Resolving redundancies when constructing a knowledge graph means standardizing the subjects, predicates, and objects in triples. For example, if we have knowledge about ethanol from multiple sources and some identify it using the ChEBI identifier 16236 while others identify it using the DrugBank identifier DB000898, we will have a similar issue to the phenylketonuria approach. If we have semantic mappings that denote CHEBI:16236 and drugbank:DB000898 are equivalent, as well as the ruleset that ChEBI identifiers take precedent over DrugBank identifiers, then we can map the triples from the resource that uses DrugBank like described in the figure below:

Let’s take the ChEBI Ontology and DrugBank pharmacological data as two examples that both annotate chemical roles. Here are a few scenarios for a given DrugBank entry (assuming they both use the same rdfs:subClassOf relationship):

There are no semantic mappings linking it to a ChEBI entry. In this case, the subject doesn’t need to be mapped.
There is a semantic mapping linking it to a ChEBI entry, but there’s no semantic mapping linking the object to a ChEBI entry. For example, DrugBank annotates ethanol as Agents Causing Muscle Toxicity (drugbank.category:DBCAT003935). Therefore, the subject is mapped but the object is retained.
There is a semantic mapping linking it to a CheBI entry and a semantic mapping linking the object to the ChEBI entry. For example, ChEBI annotates ethanol as a NMDA receptor antagonist (CHEBI:60643) and DrugBank annotates ethanol as a NMDA Receptor Antagonists (drugbank.category:DBCAT002723). In this case, the DrugBank triple is fully a duplicate of the ChEBI one. However, it may have valuable metadata to keep.

As another example, DrugBank annotates ethanol as a Cytochrome P-450 CYP3A4 Inhibitors (drugbank.category:DBCAT003232). There’s a corresponding term in ChEBI EC 1.14.13.97 (taurochenodeoxycholate 6α-hydroxylase) inhibitor, (CHEBI:86501), but there isn’t a semantic mapping capturing this. This is a job for the SSSOM Curator software and a semantic mapping repository like Biomappings to store it.

There are many examples of manually constructed workflows that do this process. While these work (e.g., Hetionet was one of the best), they are brittle towards expansion to new datasets and mappings, and often hard to keep up-to-date. My goal has been to implement a fully generic and automated version of this workflow, which I did as part of the Semantic Reasoner and Assembler. However, describing how it works on a technical level will be part of a future post.

Lexicon Assembly

Controlled vocabularies often contain labels and synonyms for their terms that are useful when constructing lexical indexes (i.e., databases of labels and synonyms) that can be fed into named entity recognition (NER) and named entity normalization (NEN) workflows - crucial components of natural language processing and text mining workflows that are commonly used by curators to annotate the literature with relationships that eventually become part of databases that are used to construct knowledge graphs.

However, much like knowledge, synonyms for the same concept might be spread over multiple different resources. Therefore, semantic mappings can be used to group multiple terms together and pool all of their synonyms, which both improves recall and reduces the number of duplicate groundings that might be given for a given part of text.

I’ve already made a technical implementation of this workflow in the Biolexica project, but I’m working towards generalizing and rebranding it for use outside the biomedical domain. I previously posted a simple demonstration using the underlying NER and NEN technology stack, but just using MeSH - a future post will show how this works with a coherent lexica for diseases, genes, and other entity types.

Mapping from SSSOM to JSKOS

2026-01-15T10:42:00+00:00

JSKOS (JSON for Knowledge Organization Systems) is a JSON-based data model for representing terminologies, thesauri, classifications, and other semantic artifacts. Like the Simple Standard for Sharing Ontological Mappings (SSSOM), it can also encode semantic mappings. This post is about developing and implementing a crosswalk between them in the sssom-pydantic Python package.

Background on JSKOS

At its core, JSKOS implements the Simple Knowledge Organization System (SKOS) data model and extends it with a data model inspired by Wikidata with the following types:

JSKOS enables representing semantic mappings two ways:

using the narrower, broader, and related slots in the Concept class that correspond to SKOS relations skos:narrowMatch, skos:broadMatch, and skos:relatedMatch
using the mappings slot in the Concept class, which accepts a list of instances of the more generic Mapping class

Here’s how JSKOS represents an exact match from the Biomappings community curated mappings database between a Medical Subject Headings (MeSH) term and Chemical Entities of Biological Interest (ChEBI) ontology term for the chemical ammeline:

{
  "license": [{ "uri": "https://spdx.org/licenses/CC0-1.0" }],
  "uri": "https://w3id.org/biopragmatics/biomappings/sssom/biomappings.sssom.tsv",
  "mappings": [
    {
      "type": ["http://www.w3.org/2004/02/skos/core#exactMatch"],
      "subject_bundle": {
        "member_set": [{ "uri": "http://id.nlm.nih.gov/mesh/C000089" }]
      },
      "object_bundle": {
        "member_set": [{ "uri": "http://purl.obolibrary.org/obo/CHEBI_28646" }]
      },
      "justification": "https://w3id.org/semapv/vocab/ManualMappingCuration"
    }
  ]
}

Notably, JSKOS is baked into the Cocoda mapping editor, which is being widely adopted in the humanities consortia of the NFDI.

Interoperability between SSSOM and JSKOS

Given the overlapping ability of the Simple Standard for Sharing Ontological Mappings (SSSOM) and JSKOS to represent semantic mappings, the JSKOS and SSSOM teams developed a crosswalk between JSKOS and SSSOM. Along the way, the SSSOM and JSKOS data models evolved to incorporate good ideas from the other, for example, the addition of a mapping identifier to SSSOM records to allow for referencing the SSSOM mapping itself.

The crosswalk is not (yet) lossless, for example, JSKOS does not yet have a mechanism to express information about lexical and other automated mappings. However, lossless conversion between data models isn’t always possible, nor is it always necessary, considering the different domains for which JSKOS and SSSOM were developed. That JSKOS was developed by researchers in the digital humanities and SSSOM was developed by researchers in the life and natural sciences can contextualize some of their discrepancies.

Technical Implementation

The sssom-js JavaScript package contains the first SSSOM to JSKOS converter and has an open issue for conversion back to SSSOM (TSV). It was developed by the JSKOS team, meaning that I have high confidence that the implemenation of the crosswalk is accurate.

While it can be invoked from the command line using npx like in npx sssom-js --from tsv --to jskos --output output.json input.sssom.tsv, it can also be explored in the first-party SSSOM Validation and Transformation website.

Originally, my plan was to implement SSSOM to JSKOS export in the sssom-pydantic so it can be easily incorporated into other SSSOM-aware applications like SSSOM Curator and the Semantic Mapping Reasoner and Assembler.

I started by implementing an object model for JSKOS in Python using Pydantic in a dedicated package. This actually turned out to be very difficult to get to work in general because the JSKOS data model is hierarchical and does not always contain fields that make it possible to discriminate between which class a given arbitrary JSON object follows. This makes it difficult to use Pydantic’s nested discriminated unions feature, so I had to implement a custom solution.

Ultimately, I scrapped the idea of re-implementing the crosswalk myself (for now) and instead defer to the sssom-js implementation (while wrapping it in an idiomatic Python API). Once sssom-js implements a TSV exporter, I will have a high-quality oracle against which to test my implementation. These first steps were implemented in cthoyt/sssom-pydantic#26.

Here’s what this looks like in Python:

import sssom_pydantic
from sssom_pydantic.contrib.jskos_export import to_jskos

url = "https://w3id.org/biopragmatics/biomappings/sssom/biomappings.sssom.tsv"
mappings, converter, metadata = sssom_pydantic.read(url)

jskos_concept = to_jskos(mappings, converter=converter, metadata=metadata)

Mapping from SSSOM to Wikidata

2026-01-08T15:47:00+00:00

At the 4th Ontologies4Chem Workshop in Limburg an der Lahn, I proposed an initial crosswalk between the Simple Standard for Sharing Ontological Mappings (SSSOM) and the Wikidata semantic mapping data model. This post describes the motivation for this proposal and the concrete implementation I’ve developed in sssom-pydantic.

This work is part of the NFDI’s Ontology Harmonization and Mapping Working Group, which is interested in enabling interoperability between SSSOM and related data standards that encode semantic mappings.

The TL;DR for this post is that I implemented a mapping from SSSOM to Wikidata in sssom-pydantic in cthoyt/sssom-pydantic#32. One high-level entrypoint is the following function, which reads an SSSOM file and prepares QuickStatements which can be reviewed in the web browser, then uploaded to Wikidata.

This script can be run from Gist with uv run https://gist.github.com/cthoyt/f38d37426a288989158a9804f74e731a#file-sssom-wikidata-demo-py

Semantic Mappings in SSSOM

The Simple Standard for Sharing Ontological Mappings (SSSOM) is a community-driven data standard for semantic mappings, which are necessary to support (semi-)automated data integration and knowledge integration, such as in the construction of knowledge graphs.

While SSSOM primary a tabular data format that is best serialized in TSV, it uses LinkML to formalize the semantics of each field such that SSSOM can be serialized to and read from OWL, RDF, and JSON-LD. Here’s a brief example:

subject_id	subject_label	predicate_id	object_id	object_label	mapping_justification
wikidata:Q128700	cell wall	skos:exactMatch	GO:0005618	cell wall	semapv:ManualMappingCuration
wikidata:Q47512	acetic acid	skos:exactMatch	CHEBI:15366	acetic acid	semapv:ManualMappingCuration

Semantic Mappings in Wikidata

Wikidata has two complementary formalisms for representing semantic mappings. The first uses the exact match (P2888) property with a URI as the object. For example, cell wall (Q128700) maps to the Gene Ontology (GO) term for cell wall by its URI http://purl.obolibrary.org/obo/GO_0005618.

The second formalism uses semantic space-specific properties (e.g. P683 for ChEBI) with local unique identifiers as the object. For example, acetic acid (Q47512) maps to the ChEBI term for acetic acid using the P683 property for ChEBI and local unique identifier for acetic acid (within ChEBI) 15366.

Wikidata has a data structure that enables annotating qualifiers onto triples. Therefore, other parts of semantic mappings modeled in SSSOM can be ported:

Authors and reviewers can be mapped from ORCiD identifiers to Wikidata identifiers, then encoded using the S50 and S4032 properties, respectively
A SKOS-flavored mapping predicate (i.e., exact, narrow, broad, close, related) can be encoded using the S4390 property
The publication date can be encoded using the S577 property
The license can be mapped from text to a Wikidata identifier, then encoded using the S275 property

Note that properties that normally start with a P when used in triples are changed to start with an S when used as qualifiers. Other fields in SSSOM could potentially be mapped to Wikidata later.

Finding Wikidata Properties using the Semantic Farm

The Semantic Farm (previously called the Bioregistry) maintains mappings between prefixes that appear in compact URIs (CURIEs) and their corresponding Wikidata properties. For example, the prefix CHEBI maps to the Wikidata property P683.

These mappings can be accessed in several ways:

via the Semantic Farm’s SSSOM export. Note: this requires subsetting to mappings where Wikidata properties are the object.
via the Semantic Farm’s live API,

via the Bioregistry Python package (this will get renamed to match Semantic Farm, eventually) using the following code:

import bioregistry

# get bulk
prefix_to_property = bioregistry.get_registry_map("wikidata")

# get for a single resource
resource = bioregistry.get_resource("chebi")
chebi_wikidata_property_id = resource.get_mapped_prefix("wikidata")

Notable Implementation Details

I’ve previously built two package which were key to making this work:

wikidata-client, which interacts with the Wikidata SPARQL endpoint and has high-level wrappers around lookup functionality. I’m also aware of WikidataIntegrator - I’ve contributed several improvements, but working with its codebase doesn’t spark joy and the last time I tried to use it, it was fully broken due to some of its dependencies not working on modern Python.
quickstatements-client, which implements an object model for QuickStatements v2 and an API client.

Along the way to this PR, I made improvements to the wikidata-client in cthoyt/wikidata-client#2 to add high-level functionality for looking up multiple Wikidata records based on values for a property (e.g., to support ORCID lookup in bulk).

All other changes were made in sssom-pydantic in cthoyt/sssom-pydantic#32.

The other key challenge was to avoid adding duplicate information to Wikidata - unlike a simple triple store, we could accidentally end up with duplicate statements. Therefore, the sssom-pydantic implementation looks up all existing semantic mappings in Wikidata for entities appearing in an SSSOM file, then filters appropriately to avoid uploading duplicate mappings to Wikidata.

Pulling it All Together

This new module in sssom-pydantic implements the following interactive workflows:

Read an SSSOM file, convert mappings to Wikidata schema, then open a QuickStatements tab in the web browser using read_and_open_quickstatements()
Convert in-memory semantic mappings to the Wikidata schema, then open a QuickStatements tab in the web browser using open_quickstatements()

Here’s what the QuickStatements web interface looks like after preparing some demo mappings:

It also implements the following non-interactive workflows, which should be used with caution since they write directly to Wikidata:

Read an SSSOM file, convert mappings to Wikidata schema, then post non-interactively to Wikidata via QuickStatements using read_and_post()
Convert in-memory semantic mappings to the Wikidata schema, then post non-interactively to Wikidata via QuickStatements using post()

I’m a bit hesitant to start uploading SSSOM content to Wikidata in bulk, because I don’t yet have a plan for how to maintain mappings that might change over time in their upstream single source of truth, e.g., mappings curated in Biomappings. Otherwise, I think this is a good proof of concept and would like to get feedback about additional qualifiers that could be added, and if the ones I chose so far were the best.

Validating Prefix Maps in LinkML Schemas

2026-01-06T09:36:00+00:00

LinkML enables defining data models and data schemas in YAML informed by semantic web best practices. As such, each definition includes a prefix map. Similarly to my previous posts on validating the prefix maps appearing in Turtle files and in unfamiliar SPARQL endpoints, this post showcases describes a new extension to the Bioregistry that validates prefix maps in LinkML definitions.

Here’s an abridged excerpt of a LinkML definition borrowed from CatCore, a data model under development by NFDI4Cat, the NFDI consortium interested in catalysis:

id: https://w3id.org/nfdi4cat/catcore
name: catcore-metadata
title: CatCore Metadata Reference Model

prefixes:
  catcore: https://w3id.org/nfdi4cat/catcore/
  voc4cat: https://w3id.org/nfdi4cat/voc4cat_
  CHMO: http://purl.obolibrary.org/obo/CHMO_
  OBI: http://purl.obolibrary.org/obo/OBI_
  AFR: http://purl.allotrope.org/ontologies/result#AFR_
  AFP: http://purl.allotrope.org/ontologies/process#AFP_
  AFQ: http://purl.allotrope.org/ontologies/quality#AFQ_
  NCIT: http://purl.obolibrary.org/obo/NCIT_
  nmrCV: "http://nmrML.org/nmrCV#NMR:"
  linkml: https://w3id.org/linkml/
  AFRL: http://purl.allotrope.org/ontologies/role#AFRL_
  APOLLO_SV: http://purl.obolibrary.org/obo/APOLLO_SV_
  SIO: http://semanticscience.org/resource/SIO_

default_prefix: catcore

In biopragmatics/bioregistry#1786, I implemented the bioregistry validate linkml command. It can be used to check the prefix map in this file and give feedback on non-standard CURIE prefix usage, unknown CURIE prefixes, etc. while giving suggestions for fixes, when possible.

Running the command on the file that contains the example prefixes from above gives the following output:

$ bioregistry validate linkml --tablefmt github --use-preferred https://github.com/HendrikBorgelt/CatCore/raw/refs/heads/main/src/catcore/schema/catcore.yaml

prefix	uri_prefix	issue	solution
catcore	https://w3id.org/nfdi4cat/catcore/	unknown CURIE prefix
AFR	http://purl.allotrope.org/ontologies/result#AFR_	unknown CURIE prefix
AFP	http://purl.allotrope.org/ontologies/process#AFP_	unknown CURIE prefix
AFQ	http://purl.allotrope.org/ontologies/quality#AFQ_	unknown CURIE prefix
nmrCV	http://nmrML.org/nmrCV#NMR:	non-standard CURIE prefix	Switch to preferred prefix: NMR
AFRL	http://purl.allotrope.org/ontologies/role#AFRL_	unknown CURIE prefix
SIO	http://semanticscience.org/resource/SIO_	non-standard CURIE prefix	Switch to preferred prefix: sio

Curation feedback is not absolute - it’s always possible that the Bioregistry is missing key content. Luckily, it conforms to the open data, open code, open infrastructure (O3) guidelines, so it’s easy for anyone to perform a drive-by curation to fix any minor issues. The Bioregistry has public, well-defined curation guidelines, code of conduct, and project governance to support making curation contributions. Alternatively, the issue tracker allows non-technical users to post requests that the Bioregistry team can follow up on.

Based on the output above, I made improvements to the Bioregistry in biopragmatics/bioregistry#1788 to add four new prefixes for the Allotrope semantic spaces and add SIO (stylized with capital letters) as the “preferred prefix” for the Semantic Science Integrated Ontology.

Note that LinkML is developed by members of the OBO Community, and therefore, its prefixes often skew towards OBO community preferences. Therefore, you might want to use the --use-preferred flag if a lot of your prefixes are stylized in uppercase or with mixed case.

Books I Read in 2025

2026-01-01T21:22:00+00:00

Here are the books I read in 2025. My goals for the year were to get some more variety, and I think I managed that.

Turning Darkness Into Light (The Memoirs of Lady Trent, #6) by Marie Brennan
Jade City (The Green Bone Saga #1) by Fonda Lee
Jade War (The Green Bone Saga #2) by Fonda Lee
Jade Legacy (The Green Bone Saga #3) by Fonda Lee
A Court of Mist and Fury (ACOTAR, #2) by Sarah J. Maas
Everyone You Hate is Going to Die by Daniel Sloss
Intimacies by Katie Kitamura
A Court of Wings and Ruin (ACOTAR, #3) by Sarah J. Maas
The Raven Tower by Ann Leckie
The Night Circus by Erin Morgenstern
The Fall by Albert Camus
The Lies of Locke Lamora (Gentleman Bastard, #1) by Scott Lynch
The Will of the Many (Hierarchy, #1) by James Islington
The Empress of Salt and Fortune by Nghi Vo
Red Seas Under Red Skies (Gentleman Bastard, #2) by Scott Lynch
Reckless by Cornelia Funke
The Republic of Thieves (Gentleman Bastard, #3) by Scott Lynch
Isles of the Emberdark by Brandon Sanderson
The Priory of the Orange Tree by Samantha Shannon
The River Has Roots by El-Mohtar Amal
Marytr! by Kaveh Akbar
A Man Called Ove by Fredrik Backman
Open Throat by Henry Hoke
The Midnight Library by Matt Haig
The Tainted Cup (Shadow of the Leviathan, #1) by Robert Jackson Bennett
The Catcher in the Rye by J.D. Salinger
The Strength of the Few (Hierarchy, #2) by James Islington
A Drop Of Corruption (Shadow of the Leviathan, #2) by Robert Jackson Bennett
Piranesi by Susanna Clarke
The Sun Also Rises by Ernest Hemingway

All comments below are spoiler-free, except a minor note about the ending of Martyr!.

Highlights:

The Night Circus was my favorite. It evoked a really special magical feeling
The Green Bone Saga was an excellent trilogy, with perfect pacing, progression, and knew the perfect spot to end.
Title drops are very important. The ending of Open Throat was very cathartic.
There was only one Brandon sandwich this year, but… SHARD GUNS

Comments:

I was in a reading slump in the summer, so I also re-read Stormlight Archive. Now knowing the end of Wind and Truth, it’s crazy to see the foreshadowing to the end.
I liked The Tainted Cup and A Drop of Corruption because Din and Ana are very interesting characters and the world building is great. But, I’m not sure if I’m sold on mystery, at least in books. I enjoy film and television adaptations, though.
A new bookstore opened in Fall in Beuel that is hosting silent reading nights once per month. I had a great time going there with friends, and we even instituted our own silent reading nights (with snacks).

Disappointments:

The Will of the Many perfected the genres and archetypes that it pulls from, but The Strength of the Few mostly missed the mark. Because of the twist from the end of the first book, it had to do a lot of world-building and create new plot and character arcs for the protagonist(s). I think it didn’t match the tone promise of the first novel and ultimately felt like the whole book was three side quests with new side characters in whom I wasn’t as invested in as in the first book. The Strength of the Few had severe middle child syndrome and felt bloated, but I think that it can be salvaged in the next (and last?) installment.
(minor spoilers) Martyr! told the story of a deeply troubled man who wanted to write a book on what makes death meaningful and decide himself if he wanted to continue living. While it had a positive ending, I think it ultimately failed to deliver any (well-organized) revelations about what makes a meaningful life, a meaningful death, or on martyrdom, either in-story to the protagonist or on a meta level to me as the reader. I would instead look to novels by Kurt Vonnegut for this kind of cleverness.
I’m still motivated to read classics, but was pretty disappointed with Catcher in the Rye and The Sun Also Rises. I didn’t feel like they had much going in terms of character, plot, or theme that secures their longevity. Obviously, it was an achievement to write a character as hatable as Holden Caulfield, but neither book had clear tone/plot/theme promises nor delivered on them.

Won’t Finish:

The Name of the Rose by Umberto Eco. The world building, characters, and writing annoyed me, and I was completely checked out by the time the plot started. We read this as part of book club in preparation for a trip to Kloster Eberbach, where the adaptation of the book was filmed. However, 4/6 of our bookclub DNF’d this one, and we instituted a 200-page maximum for future books. At least the Kloster was beautiful and had good wine!
Neuromancer by William Gibson. This book single-handedly created the cyberpunk aesthetic. However, the story was weak and the characters were uncomfortably outdated, so I put this one down. I imagine many writers had a similar experience and thought they could do better. This is probably why there are so many good stories with cyberpunk aesthetic!

Reading goals for 2026: more variety and more reading in German!

Annotating the Literature with Named Entity Recognition

2025-12-19T09:01:00+00:00

Annotating the literature with mentions of key concepts from a given domain is often the first step towards extracting more substantial structured knowledge. This can be challenging, as it typically encompasses acquiring and processing the relevant literature and ontologies then installing and applying difficult-to-use named entity recognition (NER) workflows. This post highlights software components I’ve implemented to simplify this workflow. I demonstrate it by annotating the biomedical literature available through PubMed with Medical Subject Headings (MeSH) terms, and also comment on how this can be generalized to other natural sciences, engineering, and humanities disciplines.

I’ve been building software for the last ten years that simplifies and democratizes access to these resources. Here, I’m going to highlight three components:

PubMed Downloader provides a wrapper around PubMed’s API and around bulk download and processing of the source data. While this resource only contains biomedical text, its place in the workflow can be replaced with any other text source.
SSSLM provides a wrapper around NER methods such as Gilda and spaCy. SSSLM uses a pared-down version of Gilda as its default NER tool because Gilda is fast, interpretable, and easy to install (after removing some parts). SSSLM and the methods it wraps are fully domain-agnostic.
PyOBO provides a wrapper around fetching and processing ontologies, controlled vocabularies, databases, and other resources that can be used as a dictionary. It also has a high-level workflow, pyobo.get_grounder() for getting content into ssslm. It’s built on the Semantic Farm (previously called the Bioregistry) to enable it to find and access ontologies across disciplines.

Demonstration

The following is a demonstration on how to get the abstracts of 5 articles from PubMed, perform named entity recognition (NER) using Medical Subject Headings (MeSH), output the results (below). Note that the following code can be run as a script using uv run, as it makes explicit its dependencies as PEP-723 inline metadata .

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "click>=8.3.1",
#     "pubmed-downloader>=0.0.12",
#     "pyobo[gilda-slim]>=0.12.13",
#     "tabulate>=0.9.0",
# ]
# ///

import click
import pubmed_downloader
import pyobo
from tabulate import tabulate

# get a grounder loaded up with a specific version of MeSH.
# if you don't specify a version, the latest will be used.
grounder: ssslm.Grounder = pyobo.get_grounder("mesh", versions="2018")

# get 5 PubMed identifiers about diabetes. note that the
# PubMed API has been horrifically slow lately, so please be patient
pubmed_ids: list[str] = pubmed_downloader.search("diabetes", backend="api", retmax=5)
click.echo(f"got {len(pubmed_ids)} pubmed IDs")

for article in pubmed_downloader.get_articles(pubmed_ids, error_strategy="skip", progress=True):
    abstract: str = article.get_abstract()

    # get a list of annotations, which contain the offsets of the entity
    # and the grounding to a Bioregistry-standardized CURIE.
    # more generally, this can be applied to any string from any source
    annotations: list["ssslm.Annotation"] = grounder.annotate(abstract)

    rows = [
        (
            annotation.start,
            annotation.end,
            f"[{annotation.curie}](https://semantic.farm/{annotation.curie})",
            annotation.name,
            round(annotation.score, 3),
        )
        for annotation in annotations
    ]
    headers = ["Start", "End", "CURIE", "Name", "Score"]
    table = tabulate(rows, headers=headers, tablefmt="github")

    click.echo(
        f"**{article.title.rstrip().rstrip('.')}** "
        f"([pubmed:{article.pubmed}](https://semantic.farm/pubmed:{article.pubmed}))"
        f"\n\n> {abstract}\n\n{table}\n\n"
    )

Parting Thoughts

Normally I post parting thoughts at the bottom of each post, but since the results take up a lot of space, I’ll put them here.

There are many directions to take these tools. The first might be to use a subset of MeSH that’s most appropriate for the annotation task. For example, if we just wanted to see diseases, then it only makes sense to use the MeSH Disease branch. Similarly, there are many other ontologies, controlled vocabularies, and databases in the diseases space such as MONDO, DOID, SNOMED-CT, and many more. These can be incorporated into the grounder with pyobo.get_grounder(["mesh", "mondo", "doid", "snomedct]), but will lead to redundancy issues. I’ve previously published SeMRA where I addressed mapping between equivalent entities, but am currently working on using these results to assemble coherent and comprehensive lexica that can be easily reused by SSSLM in the Biolexica project (which will also get renamed to be domain-agnostic).

Other domains can be directly used. For example, in the energy domain, the Open Energy Ontology can be used with pyobo.get_grounder("oeo"). In general, the Semantic Farm can be used to find ontologies from other domains. Within the NFDI, there are collections for each NFDI consortia that contain lists of relevant ontologies, controlled vocabularies, databases, and other resources that mint identifiers.

I hope this was a helpful introduction! If you’ve got questions about these workflows or want to see a demo on your favorite literature source/ontology/NER method, post an issue to the relevant package’s issue tracker.

Results

Investigation of intake pattern of SGLT2 inhibitors among shift workers with diabetes: a crossover study (pubmed:41413602)

Shift workers experience regular changes in their waking hours due to fluctuating work schedules. The timing of their medication intake differs depending on whether they are working a day or night shift. Sodium-glucose co-transporter 2 ( SGLT2) inhibitors are prescribed once a day and are often taken before or after breakfast. However, studies on the optimal dosing times for the effective treatment of shift workers are lacking. In this study, we investigated whether the effects were different by the pattern of SGLT2 inhibitor intake for shift workers with diabetes. Seven shift workers with diabetes who were taking an SGLT2 inhibitor were analyzed. All participants took the medication upon waking for 14 days, followed by administration at a fixed time for another 14 days. Glucose levels were measured over 14 days when the drug was administered either upon waking or at a fixed time of day. The time in range (TIR), which indicates the percentage of time during which the glucose level is within the range of 70-180 mg/dL, was used as the main evaluation index. The mean HbA1c of the participants was 7.1%. The TIR was 88.5% in the administration upon waking group and 84.9% in the administration at a fixed time group. No significant difference in TIR values was observed between the two administration groups. A TIR of 70% or higher is recommended to prevent the onset of diabetic complications. Consistent intake of SGLT2 inhibitors, regardless of whether it is during the day or night shift, may help stabilize blood glucose levels in shift workers throughout the day and night, thereby preventing the development of complications.

Start	End	CURIE	Name	Score
6	13	mesh:D009274	Occupational Groups	0.54
82	96	mesh:D010561	Personnel Staffing and Scheduling	0.54
219	233	mesh:D027981	Symporters	0.54
317	326	mesh:D062408	Breakfast	0.762
410	417	mesh:D009274	Occupational Groups	0.54
515	530	mesh:D000077203	Sodium-Glucose Transporter 2 Inhibitors	0.549
548	555	mesh:D009274	Occupational Groups	0.54
583	590	mesh:D009274	Occupational Groups	0.54
624	639	mesh:D000077203	Sodium-Glucose Transporter 2 Inhibitors	0.549
729	743	mesh:D009934	Organization and Administration	0.54
781	788	mesh:D005947	Glucose	0.778
832	836	mesh:D004364	Pharmaceutical Preparations	0.54
981	988	mesh:D005947	Glucose	0.762
1064	1069	mesh:D020481	Index	0.762
1141	1155	mesh:D009934	Organization and Administration	0.54
1191	1205	mesh:D009934	Organization and Administration	0.54
1298	1312	mesh:D009934	Organization and Administration	0.54
1383	1405	mesh:D048909	Diabetes Complications	0.54
1428	1444	mesh:D000077203	Sodium-Glucose Transporter 2 Inhibitors	0.549
1524	1537	mesh:D001786	Blood Glucose	0.762
1554	1561	mesh:D009274	Occupational Groups	0.54

Men’s health needs assessment in the Toledo District of Southern Belize (pubmed:41413521)

Belize is a small country in Central America with a growing burden of non-communicable disease (NCD), including hypertension and diabetes. Toledo District is the southernmost and poorest district in the country. Reliable national level health data for Belize is readily available, but the data is rarely disaggregated by sex or district. Reducing the burden of NCDs is a high priority for the Ministry of Health and Wellness. Belize’s progress to date on Sustainable Development Goal (SDG) 3 (Good Health and Wellbeing) has been modest with many indicators stagnating or progress increasing at less than 50% of the required rate. SDG 3 describes the need to reduce the risks of NCDs and to strengthen the capacities of the healthcare workforce. The objective was to perform a men’s health needs assessment to identify and prioritize men’s health needs in the Toledo District. This was a mixed methods study. Qualitative data were collected from semi-structured interviews. Interviews were recorded, transcribed, and analyzed using Thematic Analysis. Quantitative data included epidemiological data from national vital statistics or disease registries and other public sources. Data were collected between January and June 2017. Belizean men have among the highest risk for cardiac or diabetes related illness or death in the Americas. Diabetes and hypertension are responsible for 4.49% and 1.23% of Disability Adjusted Life Years in men respectively and are increasing by 2.51% annually. Fifty-seven interviews (55 individuals and two groups) from nine villages were carried out. Four themes emerged from the qualitative data. Men in Toledo: • have poor health literacy; • have reasonable access to health resources, but do not use them; • inability to clearly articulate health priorities; • do not process risk well. Men in Toledo suffer from a high prevalence of NCDs including hypertension and diabetes and understand health and risks poorly. This may contribute to Belize’s struggle to achieve the goals of SDG 3.4.1. Strengthening the healthcare workforce by improved training of community health workers (CHWs) and providing health education to men in Toledo is required to address these concerns.

Start	End	CURIE	Name	Score
0	6	mesh:D001531	Belize	0.778
29	44	mesh:D002489	Central America	0.778
70	94	mesh:D000073296	Noncommunicable Diseases	0.549
112	124	mesh:D006973	Hypertension	0.762
236	242	mesh:D006262	Health	0.762
252	258	mesh:D001531	Belize	0.778
321	324	mesh:D012723	Sex	0.762
405	411	mesh:D006262	Health	0.778
426	432	mesh:D001531	Belize	0.778
455	483	mesh:D000076502	Sustainable Development	0.556
498	504	mesh:D006262	Health	0.778
546	556	mesh:D007202	Indicators and Reagents	0.54
669	674	mesh:D012306	Risk	0.762
723	733	mesh:D003695	Delivery of Health Care	0.54
734	743	mesh:D000078329	Workforce	0.762
776	788	mesh:D054526	Men’s Health	0.725
789	805	mesh:D020380	Needs Assessment	0.762
833	845	mesh:D054526	Men’s Health	0.725
846	851	mesh:D006301	Health Services Needs and Demand	0.54
893	900	mesh:D008722	Methods	0.762
1112	1128	mesh:D014798	Vital Statistics	0.762
1132	1139	mesh:D004194	Disease	0.762
1140	1150	mesh:D012042	Registries	0.762
1237	1240	mesh:D008571	Men	0.762
1264	1268	mesh:D012306	Risk	0.762
1312	1317	mesh:D003643	Death	0.762
1325	1333	mesh:D000569	Americas	0.778
1348	1360	mesh:D006973	Hypertension	0.762
1400	1430	mesh:D000087509	Disability-Adjusted Life Years	0.556
1434	1437	mesh:D008571	Men	0.762
1628	1631	mesh:D008571	Men	0.778
1655	1670	mesh:D057220	Health Literacy	0.762
1700	1716	mesh:D006295	Health Resources	0.762
1773	1790	mesh:D006292	Health Priorities	0.762
1809	1813	mesh:D012306	Risk	0.762
1820	1823	mesh:D008571	Men	0.778
1853	1863	mesh:D015995	Prevalence	0.762
1882	1894	mesh:D006973	Hypertension	0.762
1923	1929	mesh:D006262	Health	0.762
1934	1939	mesh:D012306	Risk	0.762
1971	1977	mesh:D001531	Belize	0.778
2004	2009	mesh:D006040	Goals	0.762
2042	2052	mesh:D003695	Delivery of Health Care	0.54
2053	2062	mesh:D000078329	Workforce	0.762
2087	2111	mesh:D003150	Community Health Workers	0.762
2133	2149	mesh:D006266	Health Education	0.762
2153	2156	mesh:D008571	Men	0.762
2182	2189	mesh:D019484	Address	0.762

Risk factors of ventilator-associated pneumonia in patients with acute exacerbation of chronic obstructive pulmonary disease: a meta-analysis and systematic review (pubmed:41413500)

This meta-analysis aimed to identify risk factors for ventilator-associated pneumonia (VAP) in patients with Acute exacerbations of Chronic obstructive pulmonary disease (AECOPD). We systematically searched PubMed, Web of Science, CINAHL, Cochrane Library, Embase, CNKI, and other databases for studies investigating risk factors for VAP in patients experiencing AECOPD. The search encompassed records from database inception up to July 2, 2025. The quality of the studies was assessed using the Newcastle-Ottawa Scale. Meta-analysis was performed using Stata 18.0. A total of 16 articles were included, encompassing 3,664 subjects and 16 risk factors. Meta-analysis results showed that, Age (OR: 2.49, 95%CI : 1.49, 4.17; P<0.001), Smoking history (OR: 2.70, 95%CI : 1.65, 4.44; P<0.001), Acute physiology and chronic health evaluation composite score (APACHE Ⅱ) score (OR: 3.03, 95%CI : 1.98, 4.65; P<0.001), Sequential organ failure assessment (SOFA) score (OR: 2.75, 95%CI : 1.90, 3.99; P<0.001), Diabetes (OR: 2.11, 95%CI : 1.38, 3.24; P = 0.001), Underlying Diseases (OR: 3.42, 95%CI : 1.85, 6.32; P<0.001), Duration of mechanical ventilation (OR: 4.53, 95%CI : 2.68, 7.65; P<0.001), Tracheal intubation (OR: 4.21, 95%CI : 1.85, 9.57; P = 0.001), Indwelling gastric tube ( OR: 3.31, 95%CI : 1.38, 7.95; P = 0.008), Total parenteral nutrition (OR: 1.86, 95%CI : 1.29, 2.70; P = 0.001), Combined antibiotics (OR: 2.79, 95%CI : 1.32, 5.93; P = 0.007), Tracheotomy (OR: 2.92, 95%CI : 2.04, 4.17; P<0.001), History of mechanical ventilation within one year (OR: 2.92, 95%CI : 2.04, 4.17; P = 0.005), Use acid suppressants (OR: 2.10, 95%CI : 1.49, 2.97; P<0.001) were associated with the development of VAP in AECOPD patients. This study identified 14 risk factors associated with the risk of VAP in AECOPD patients. This finding is helpful for early identification of high-risk patients, which is of great value for reducing mortality and improving the clinical prognosis of patients with mechanical ventilation.

Start	End	CURIE	Name	Score
5	18	mesh:D017418	Meta-Analysis	0.762
37	49	mesh:D012307	Risk Factors	0.762
54	85	mesh:D053717	Pneumonia, Ventilator-Associated	0.54
132	169	mesh:D029424	Pulmonary Disease, Chronic Obstructive	0.549
207	213	mesh:D039781	PubMed	0.778
222	229	mesh:D012586	Science	0.778
248	255	mesh:D007990	Libraries	0.556
317	329	mesh:D012307	Risk Factors	0.762
394	401	mesh:D011996	Records	0.762
407	415	mesh:D019991	Database	0.762
520	533	mesh:D017418	Meta-Analysis	0.772
639	651	mesh:D012307	Risk Factors	0.762
653	666	mesh:D017418	Meta-Analysis	0.772
735	742	mesh:D012907	Smoking	0.778
743	750	mesh:D006664	History	0.762
794	840	mesh:D018806	APACHE	0.549
858	866	mesh:D018806	APACHE	0.53
1072	1080	mesh:D004194	Disease	0.778
1072	1080	obo:mesh#C	Diseases	0.778
1136	1158	mesh:D012121	Respiration, Artificial	0.54
1211	1221	mesh:D007440	Intubation	0.762
1332	1358	mesh:D010289	Parenteral Nutrition, Total	0.549
1411	1422	mesh:D000900	Anti-Bacterial Agents	0.54
1466	1477	mesh:D014140	Tracheotomy	0.778
1521	1528	mesh:D006664	History	0.778
1532	1554	mesh:D012121	Respiration, Artificial	0.54
1767	1779	mesh:D012307	Risk Factors	0.762
1800	1804	mesh:D012306	Risk	0.762
1941	1950	mesh:D009026	Mortality	0.762
1978	1987	mesh:D011379	Prognosis	0.762
2005	2027	mesh:D012121	Respiration, Artificial	0.54

Randomized trial assessing transverse supraumbilical incisions for cesarean sections in morbid obese women with pannus (pubmed:41413498)

BACKGROUND AND OBJECTIVE: The high prevalence of Morbidly obese Egyptian patients presents surgical problems for cesarean sections (CS), including a higher risk of wound infections. This study examines the impact of a transverse supraumbilical (TSU) incision in these patients. We conducted a randomized controlled trial on 72 morbidly obese patients (BMI >40 kg/m²) scheduled for cesarean section at Ain Shams University Hospital from March 2016 to August 2018. Participants were divided into Group A (36 patients) with a transverse supraumbilical (TSU) incision and Group B (36 patients) with a conventional Pfannenstiel incision. The primary outcome measured was the incidence of wound infection, while secondary outcomes included operative time, postoperative pain, hospital stay, blood loss, postoperative mobility, and intestinal motility. The results indicated no significant differences between the groups regarding age, BMI, parity, diabetes mellitus, and history of previous cesarean sections. The incidence of surgical site infection was significantly lower in the transverse supraumbilical group (11.1%, 4/36) compared to the Pfannenstiel group (58.3%, 21/36), with an absolute risk reduction of 47.2% (95% CI: 27.8% to 66.6%). Other parameters like operative time, hematocrit drop, pain score, hospital stay, and intestinal motility showed no significant differences between the groups (P>0.05). Supraumbilical transverse incisions are a safe, effective alternative to Pfannenstiel incisions in morbidly obese women, with better wound infection rates and easier access. Further research is needed to confirm the benefits and to assess patient satisfaction. This study was registered prospectively in clinicaltrials.gov (NCT02692729) on 1.3.2016.

Start	End	CURIE	Name	Score
35	45	mesh:D015995	Prevalence	0.762
113	130	mesh:D002585	Cesarean Section	0.762
156	160	mesh:D012306	Risk	0.762
164	180	mesh:D014946	Wound Infection	0.762
293	320	mesh:D016449	Randomized Controlled Trial	0.762
381	397	mesh:D002585	Cesarean Section	0.762
411	421	mesh:D014495	Universities	0.556
422	430	mesh:D006761	Hospitals	0.556
670	679	mesh:D015994	Incidence	0.762
683	698	mesh:D014946	Wound Infection	0.762
734	748	mesh:D061646	Operative Time	0.762
750	768	mesh:D010149	Pain, Postoperative	0.549
770	783	mesh:D007902	Length of Stay	0.54
785	810	mesh:D019106	Postoperative Hemorrhage	0.502
825	844	mesh:D005769	Gastrointestinal Motility	0.54
934	940	mesh:D010298	Parity	0.762
942	959	mesh:D003920	Diabetes Mellitus	0.762
965	972	mesh:D006664	History	0.762
985	1002	mesh:D002585	Cesarean Section	0.762
1008	1017	mesh:D015994	Incidence	0.762
1021	1044	mesh:D013530	Surgical Wound Infection	0.54
1181	1204	mesh:D061366	Numbers Needed To Treat	0.54
1262	1276	mesh:D061646	Operative Time	0.762
1278	1288	mesh:D006400	Hematocrit	0.762
1295	1299	mesh:D010146	Pain	0.762
1307	1320	mesh:D007902	Length of Stay	0.54
1326	1345	mesh:D005769	Gastrointestinal Motility	0.54
1523	1528	mesh:D014930	Women	0.762
1542	1557	mesh:D014946	Wound Infection	0.762
1591	1599	mesh:D012106	Research	0.762
1648	1668	mesh:D017060	Patient Satisfaction	0.762

Associations of Perfluoroalkyl and Polyfluoroalkyl Substances With Cardiovascular Disease Incidence in Adults With Prediabetes: Findings From the Diabetes Prevention Program (pubmed:41413398)

Perfluoroalkyl and polyfluoroalkyl substances (PFAS) are persistent, widespread environmental contaminants linked to cardiometabolic outcomes including obesity, hyperlipidemia, and diabetes. We examined whether baseline plasma PFAS concentrations are associated with incident cardiovascular disease (CVD) in adults with prediabetes, leveraging data from DPPOS (Diabetes Prevention Program Outcomes Study). Among 1382 participants, we quantified baseline plasma concentrations of 6 PFAS. We used Cox proportional hazards models to estimate the risks of developing CVD outcomes during a median of 21 years of follow-up for each PFAS and used quantile g-computation to evaluate the joint effect of all 6 PFAS. Effect modification by age, sex, menopausal status, diet, and physical activity was explored. The incidence of major adverse cardiovascular events was 9.6%; 3.9% had CVD-related death. Each increase in interquartile range (1.1 ng/mL) in 2-( In adults with prediabetes, higher plasma concentrations of select PFAS, but not their mixture, were prospectively associated with increased CVD risk. These findings underscore PFAS as a potential environmental risk factor for CVD in high-risk populations.

Start	End	CURIE	Name	Score
152	159	mesh:D009765	Obesity	0.762
161	175	mesh:D006949	Hyperlipidemias	0.54
220	226	mesh:D010949	Plasma	0.762
276	298	mesh:D002318	Cardiovascular Diseases	0.54
308	314	mesh:D000328	Adult	0.762
320	331	mesh:D011236	Prediabetic State	0.54
381	388	mesh:D019542	Program	0.778
454	460	mesh:D010949	Plasma	0.762
495	526	mesh:D016016	Proportional Hazards Models	0.549
543	548	mesh:D012306	Risk	0.762
679	684	mesh:D007596	Joints	0.54
735	738	mesh:D012723	Sex	0.762
759	763	mesh:D004032	Diet	0.762
769	786	mesh:D015444	Exercise	0.54
805	814	mesh:D015994	Incidence	0.762
885	890	mesh:D003643	Death	0.762
951	957	mesh:D000328	Adult	0.762
963	974	mesh:D011236	Prediabetic State	0.54
983	989	mesh:D010949	Plasma	0.762
1093	1097	mesh:D012306	Risk	0.762
1159	1170	mesh:D012307	Risk Factors	0.54
1192	1203	mesh:D011153	Population	0.762

Machine-Actionable Training Materials at BioHackathon Germany 2025

2025-12-09T11:08:00+00:00

I recently attended the 4^th BioHackathon Germany hosted by the German Network for Bioinformatics Infrastructure (de.NBI). I participated in the project On the Path to Machine-actionable Training Materials in order to improve the interoperability between DALIA, TeSS, mTeSS-X, and Schema.org. This post gives a summary of the activities leading up to the hackathon and the results of our happy hacking.

Team

Our project, On the Path to Machine-actionable Training Materials, had the following active participants throughout the week:

Nick Juty & Phil Reed (University of Manchester)
Leyla Jael Castro & Roman Baum (Deutsche Zentralbibliothek für Medizin; ZB Med)
Petra Steiner (University of Darmstadt)
Oliver Knodel & Martin Voigt (Helmholtz-Zentrum Dresden-Rossendorf; HZDR)
Dilfuza Djamalova (Forschungszentrum Jülich; FZJ)
Jacobo Miranda (European Molecular Biology Laboratory; EMBL)

Nick and Petra were our team leaders and Phil acted as the project’s de facto secretary. On the first day of the hackathon, we were briefly joined by Alban Gaignard (Nantes University), Dimitris Panouris (SciLifeLab), and Harshita Gupta (SciLifeLab) to present their current related work. Similarly, Dominik Brilhaus (Heinrich-Heine-Universität Düsseldorf) joined on the first day to share his perspective from DataPLANT (the NFDI consortium for plants) as a training materials creator. Finally, Helena Schnitzer (FZJ) participated in some Schema.org discussions through the week.

Goals

We categorized our work plan into three streams:

Training Material Interoperability - survey the landscape of relevant ontologies and schemas for annotating learning materials, curate mappings/crosswalks between existing data models, develop a programmatic toolbox, and begin federating between training material platforms
Training Material Analysis - analyze training materials at scale to group similar training materials, reduce redundancy, and semi-automatically construct learning paths
Modeling Learning Paths - collect use cases and develop a (meta)data model for learning paths

Training Material Interoperability

Interoperability is third pillar of the FAIR data principles. Metadata describing training materials may be captured and stored in one of several data models including the DALIA Interchange Format (DIF) v1.3, the format implicitly defined by the TeSS API, and the Schemas.org Learning Material profile. Further, metadata records conforming to these data models are filled with references to terms in other ontologies, controlled vocabularies, databases, and other resources that mint (persistent) identifiers. Our overarching goal at the hackathon was to improve interoperability on both levels.

Indexing Ontologies and Schemas

Our first concrete goal for training material interoperability at the hackathon was to survey ontologies, controlled vocabularies, databases, and other resources that mint (persistent) identifiers that might appear in the metadata describing a learning material. For example, TeSS uses the EDAM Ontology to annotate topics onto training materials. For the same purpose, DALIA uses the Hochschulcampus Ressourcentypen (I’ll say more on how we deal with the conflicting resources in the section below on mappings).

Our second concrete goal was to survey schemas that are used in modeling open educational resources and training materials, for example, Schema.org, OERSchema, and MoDALIA, which encodes the DALIA Interchange Format (DIF) v1.3.

The Semantic Farm (https://semantic.farm) is comprehensive database of metadata about resources that mint (persistent) identifiers (e.g., ontologies, controlled vocabularies, databases, schemas) such as their preferred CURIE prefix for usage in SPARQL queries and other semantic web applications. It imports and aligns with other databases like Identifiers.org (for the life sciences) and BARTOC (for the digital humanities) to support interoperability and sustainability. It follows the open data, open code, and open infrastructure (O3) guidelines and has well-defined governance to enable community maintenance and support longevity.

It’s the perfect place to index all the learning material and open educational resource-related ontologies, controlled vocabularies, databases, and schemas.

I gave a tutorial on how to search the Semantic Farm for ontologies, controlled vocabularies and other resources that mint (persistent) identifiers, and how to contribute any that are missing. In short, they can be contributed by filling out the new prefix request template on GitHub. If you’re interested to add a new entry, you can directly use the form, read the contribution guidelines, or watch a short YouTube tutorial.

While I had done some significant preparatory work before the hackathon by creating many new entries in the Semantic Farm, the team found and added several new and important entries to the Semantic Farm during the hackathon too. Here are two highlights:

Martin Voigt contributed the prefix amb for the Allgemeines Metadatenprofil für Bildungsressourcen (General Metadata Profile for Educational Resources) in biopragmatics/bioregistry#1781. This is a metadata schema for learning materials produced by the Kompetenzzentrum Interoperable Metadaten (KIM) within the Deutsche Initiative für Netzwerkinformation e.V. that was heavily inspired by Schema.org and the Dublin Core Learning Resource Metadata Initiative (LRMI)

Dilfuza Djamalova and Jacobo Miranda contributed the prefix gtn for Galaxy Training Network training materials in biopragmatics/bioregistry#1779. This resource contains multi- and cross-disciplinary training materials for using the Galaxy workflow management system. Below, I describe how we ingested transformed the training materials from GTN into a common format such they can be represented according to the DALIA Interchange Format (DIF) v1.3, the implicit data model expected by TeSS, and in Schema.org-compliant RDF.

Ultimately, we collated relevant ontologies, controlled vocabularies, schemas and other resources that mint (persistent) identifiers in a collection such that they can be easily found and shared.

Semantic Mappings and Crosswalks

I alluded to the different resources used by TeSS and DALIA to annotate disciplines. The issue of partially overlapping ontologies, controlled vocabularies, and database is quite widespread, and can manifest in a few different ways. The figure above shows that redundancy can arise because of different focus within a domain (i.e., the chemistry example), different hierarchical specificity (i.e., the disease example), and due to massive generic resources having overlap across many domains (e.g., like UMLS, MeSH, NCIT).

This is problematic when integrating learning materials from different sources, e.g., TeSS and DALIA, because two learning materials may be annotated with different terms describing the same discipline. Therefore, the solution is to create semantic mappings between these terms.

I’ve worked for several years on the Simple Standard for Sharing Ontological Mappings (SSSOM) standard for storing semantic mappings, so this was naturally the target for our work. Further, I have been working on a domain-agnostic workflow for predicting semantic mappings with lexical matching and deploying a curation interface called SSSOM Curator. I gave a tutorial for using SSSOM Curator to the team based on a previous tutorial I made (that can be found on YouTube here). We prepared predicted semantic mappings between several learning material-related ontologies in biopragmatics/biomappings#204, but we didn’t prioritize semantic mapping curation during the hackathon. Here’s what they look like in the SSSOM Curator interface for Biomappings:

Where curating correspondences between concepts in ontology, controlled vocabularies, and databases is often called semantic mapping, curating correspondences between schemas and properties therein is often called crosswalks. We put a bigger emphasis on producing crosswalks between Schema.org and MoDALIA. This is actually a more complex problem due to the fact that correspondences between elements in schemas can be more sophisticated (e.g., mapping between two fields for first and last names to a single name field), but there are at least a few places where properties can be mapped with SSSOM.

An interesting lesson learned is that some curators find using SKOS relationships challenging because the narrow and broader relations have the opposite direction than what they would expect. For example, X skos:narrowMatch Y means that X is narrower than Y, not X has a narrow match Y. Many vocabularies use a verb as part of the predicate to reduce this confusion - I’m sure if it were X skos:isNarrowMatchFor Y, then this would not have been a problem. Deep down, the real issue is that transparent identifiers (i.e., human-readable ones) are bad, since they can’t be changed over time. See the excellent article, Identifiers for the 21^st century, by McMurry et al. (2017) for a more detailed discussion on what makes a good identifier.

Operationalizing Crosswalks

The next step was to translate the abstract crosswalks between DALIA, TeSS, and Schema.org into a concrete implementation using a general purpose programming language (i.e., Python).

The Scaling Problem

Given that we only focused on these three data models, it’s not unrealistic to produce a DALIA-TeSS crosswalk, TeSS-Schema.org crosswalk, and DALIA-Schema.org crosswalk. However, this approach does not scale well - in general, it requires curating and implementing ${N}\choose{2}$ crosswalks with $N$ being the number of schemas.

An alternative is to use a hub-and-spoke model, in which one data model is targeted as the intermediary used for interchange and storage. This reduces the burden on curators of crosswalks, as they only have to curate a single crosswalk for any given data model into the intermediary. Similarly, it reduces the burden on code maintainers as only a single crosswalk has to be implemented.

The challenge with open educational resources and learning materials is that no existing data model is sufficient to cover the (most important) aspects of all other data models. This motivated us to implement a unified, generic data model for learning materials to serve as the interoperability hub between DALIA, TeSS, Schema.org, and other data models.

graph TD
    subgraph alltoall ["All-to-All (complex, burdensome)"]
        dalia[DALIA] <--> tess[TeSS]
        dalia <--> schema[Schema.org]
        dalia <--> oerschema[OERschema]
        dalia <--> amb["Allgemeines Metadatenprofil für Bildungsressourcen (AMB)"]
        dalia <--> lrmi["Learning Resource Metadata Initiative (LRMI)"]
        dalia <--> erudite[ERuDIte]
        tess <--> schema
        tess <--> oerschema
        tess <--> amb
        tess <--> lrmi
        tess <--> erudite
        schema <--> oerschema
        schema <--> amb
        schema <--> lrmi
        schema <--> erudite
        oerschema <--> amb
        oerschema <--> lrmi
        oerschema <--> erudite
        amb <--> lrmi
        amb <--> erudite
        lrmi <--> erudite
    end

    subgraph hub ["Hub-and-Spoke (maintainable, extensible)"]
        direction TB
        hubn[Unified OER Data Model] <--> daliaspoke[DALIA]
        hubn[Unified OER Data Model] <--> tessspoke[TeSS]
        hubn[Unified OER Data Model] <--> schemaspoke[Schema.org]
        hubn[Unified OER Data Model] <--> oerschemaspoke[OERschema]
        hubn[Unified OER Data Model] <--> ambspoke["Allgemeines Metadatenprofil für Bildungsressourcen (AMB)"]
        hubn[Unified OER Data Model] <--> lrmispoke["Learning Resource Metadata Initiative (LRMI)"]
        hubn[Unified OER Data Model] <--> eruditespoke[ERuDIte]
    end

    alltoall --> hub

The famous XKCD comic, Standards (https://xkcd.com/927), proselytizes that any proposal of a unified standard that covers everyone’s use cases is doomed to be an $N+1$ competing standard. While I’m doing my best to present the work done in preparation for the hackathon and at the hackathon in a linear way, the truth is that most steps also included discussion, hacking, trying, failing, and repeating. Therefore, I can confidently say that for practical reasons, implementing a new de facto standard was the only realistic choice.

The OERbservatory Data Model

During the hackathon, we implemented the open source OERbservatory Python package. I first want to talk about three major features that it includes:

a unified, generic object model for open educational resources that’s effectively the union of the best parts of DALIA, TeSS, Schema.org, and a few other data models we found
import and export to two open educational resource and learning materials data models - DALIA and TeSS. We didn’t have time during the hackathon to implement import and export to Schema.org.
import from three external learning material repositories - OERhub, OERSI, and the Galaxy Training Network (GTN)

Here’s an excerpt of the object model, implemented using Pydantic. Note that Pydantic uses a combination of Python’s type system and type annotations to express constraints and rules, similarly to how SHACL does. However, we get the benefit of Python type checking and the Python runtime to check that we’ve encoded this all correctly. Finally, all Pydantic models can be serialized and deserialized from JSON.

class EducationalResource(BaseModel):
    """Represents an educational resource."""

    model_config = ConfigDict(arbitrary_types_allowed=True)

    reference: Reference | None = Field(
        None,
        description="The primary reference for this learning material",
        examples=[Reference(prefix="dalia", identifier="")]
    )
    title: InternationalizedStr = Field(..., description="The title of the learning material")
    authors: list[Author | Organization] = Field(
        default_factory=list,
        description="An ordered list of authors (i.e., persons or organizations) of the learning material",
        examples=[
            Author(name="Charles Tapley Hoyt", orcid="0000-0003-4423-4370"),
            Organization(name="NFDI", ror="05qj6w324"),
        ],
        min_len=1,
    )
    ...

Technology Comparison (content warning: programming culture wars)

Extracting Semantic Mappings from BioPortal in SSSOM

2025-11-23T11:08:00+00:00

Earlier this week, a question was asked on OBO Foundry Slack on where to find semantic mappings to terms in the Systematized Nomenclature of Medicine - Clinical Terms (SNOMED-CT). While some are available in the SeMRA Disease Mappings Database, there are many more available within BioPortal, which has access to the entire SNOMED-CT source data and has produced semantic mapping predictions using LOOM. This post is about how I implemented an API wrapper for generic OntoPortal instances’ mapping endpoints and a post-processing pipeline that converts OntoPortal’s custom mapping format into SSSOM.

Interacting with BioPortal

BioPortal is an instance of a more generic backend called OntoPortal. I’ve previous developed ontoportal-client, a Python package that both has a generic wrapper for any OntoPortal’s API and pre-configured wrappers for BioPortal, AgroPortal, EcoPortal, and several others.

The OntoPortal API endpoint for retrieving mappings is /mappings, which takes a comma separated pair of two ontologies as a parameter like in https://data.bioontology.org/mappings?apikey=&ontologies=SNOMEDCT,AERO. I was able to relatively easily implement this in ontoportal-client in cthoyt/ontoportal-client#10, which enables automatically paging through results using the following code:

from ontoportal_client import BioPortalClient

# follow https://github.com/cthoyt/ontoportal-client?tab=readme-ov-file#%EF%B8%8F-configuration
# to configure BioPortalClient to be instantiated without need for explicit configuration
client = BioPortalClient()
for record in client.get_mappings("SNOMEDCT", "AERO"):
    pass

Each record is a dictionary object corresponding to the JSON returned by the API (after stripping pagination metadata):

{
  "id": null,
  "source": "LOOM",
  "classes": [
    {
      "@id": "http://purl.obolibrary.org/obo/ogms/OMRE_0000023",
      "@type": "http://www.w3.org/2002/07/owl#Class",
      "links": {
        "self": "https://data.bioontology.org/ontologies/AERO/classes/http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2Fogms%2FOMRE_0000023",
        "ontology": "https://data.bioontology.org/ontologies/AERO",
        "children": "https://data.bioontology.org/ontologies/AERO/classes/http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2Fogms%2FOMRE_0000023/children",
        "parents": "https://data.bioontology.org/ontologies/AERO/classes/http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2Fogms%2FOMRE_0000023/parents",
        "descendants": "https://data.bioontology.org/ontologies/AERO/classes/http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2Fogms%2FOMRE_0000023/descendants",
        "ancestors": "https://data.bioontology.org/ontologies/AERO/classes/http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2Fogms%2FOMRE_0000023/ancestors",
        "instances": "https://data.bioontology.org/ontologies/AERO/classes/http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2Fogms%2FOMRE_0000023/instances",
        "tree": "https://data.bioontology.org/ontologies/AERO/classes/http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2Fogms%2FOMRE_0000023/tree",
        "notes": "https://data.bioontology.org/ontologies/AERO/classes/http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2Fogms%2FOMRE_0000023/notes",
        "mappings": "https://data.bioontology.org/ontologies/AERO/classes/http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2Fogms%2FOMRE_0000023/mappings",
        "ui": "http://bioportal.bioontology.org/ontologies/AERO?p=classes&conceptid=http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2Fogms%2FOMRE_0000023",
        "@context": {
          "self": "http://www.w3.org/2002/07/owl#Class",
          "ontology": "http://data.bioontology.org/metadata/Ontology",
          "children": "http://www.w3.org/2002/07/owl#Class",
          "parents": "http://www.w3.org/2002/07/owl#Class",
          "descendants": "http://www.w3.org/2002/07/owl#Class",
          "ancestors": "http://www.w3.org/2002/07/owl#Class",
          "instances": "http://data.bioontology.org/metadata/Instance",
          "tree": "http://www.w3.org/2002/07/owl#Class",
          "notes": "http://data.bioontology.org/metadata/Note",
          "mappings": "http://data.bioontology.org/metadata/Mapping",
          "ui": "http://www.w3.org/2002/07/owl#Class"
        }
      },
      "@context": {
        "@vocab": "http://data.bioontology.org/metadata/",
        "@language": "en"
      }
    },
    {
      "@id": "http://purl.bioontology.org/ontology/SNOMEDCT/3415004",
      "@type": "http://www.w3.org/2002/07/owl#Class",
      "links": {
        "self": "https://data.bioontology.org/ontologies/SNOMEDCT/classes/http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FSNOMEDCT%2F3415004",
        "ontology": "https://data.bioontology.org/ontologies/SNOMEDCT",
        "children": "https://data.bioontology.org/ontologies/SNOMEDCT/classes/http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FSNOMEDCT%2F3415004/children",
        "parents": "https://data.bioontology.org/ontologies/SNOMEDCT/classes/http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FSNOMEDCT%2F3415004/parents",
        "descendants": "https://data.bioontology.org/ontologies/SNOMEDCT/classes/http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FSNOMEDCT%2F3415004/descendants",
        "ancestors": "https://data.bioontology.org/ontologies/SNOMEDCT/classes/http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FSNOMEDCT%2F3415004/ancestors",
        "instances": "https://data.bioontology.org/ontologies/SNOMEDCT/classes/http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FSNOMEDCT%2F3415004/instances",
        "tree": "https://data.bioontology.org/ontologies/SNOMEDCT/classes/http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FSNOMEDCT%2F3415004/tree",
        "notes": "https://data.bioontology.org/ontologies/SNOMEDCT/classes/http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FSNOMEDCT%2F3415004/notes",
        "mappings": "https://data.bioontology.org/ontologies/SNOMEDCT/classes/http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FSNOMEDCT%2F3415004/mappings",
        "ui": "http://bioportal.bioontology.org/ontologies/SNOMEDCT?p=classes&conceptid=http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FSNOMEDCT%2F3415004",
        "@context": {
          "self": "http://www.w3.org/2002/07/owl#Class",
          "ontology": "http://data.bioontology.org/metadata/Ontology",
          "children": "http://www.w3.org/2002/07/owl#Class",
          "parents": "http://www.w3.org/2002/07/owl#Class",
          "descendants": "http://www.w3.org/2002/07/owl#Class",
          "ancestors": "http://www.w3.org/2002/07/owl#Class",
          "instances": "http://data.bioontology.org/metadata/Instance",
          "tree": "http://www.w3.org/2002/07/owl#Class",
          "notes": "http://data.bioontology.org/metadata/Note",
          "mappings": "http://data.bioontology.org/metadata/Mapping",
          "ui": "http://www.w3.org/2002/07/owl#Class"
        }
      },
      "@context": {
        "@vocab": "http://data.bioontology.org/metadata/",
        "@language": "en"
      }
    }
  ],
  "process": null,
  "@id": "",
  "@type": "http://data.bioontology.org/metadata/Mapping"
}

There’s both a lot of noise in this output and several pieces of key information that need to be inferred. When designing ontoportal-client (and other similar wrappers), I’ve had to grapple with staying true to the source, versus injecting logic that processes and makes useful. For now, I’ve decided that ontoportal-client shouldn’t make any judgments on the data that comes out of the API. Also, since I wrote the package, the format has changed as well, and I am not super interested in taking on that maintenance burden (which makes the suggestion in cthoyt/ontoportal-client#3) difficult to address.

Converting to SSSOM

If not in ontoportal-client, then where should I put the code that processes OntoPortal mappings? I had two options. The first is in the Semantic Mapping Reasoner and Assembler ( SeMRA; code, paper), which is a generic place for assembling semantic mappings. At the time, I designed the internal data model in SeMRA to go beyond what’s possible in SSSOM because I was interested in keeping track of provenance of how semantic mappings were used to infer other ones. Slowly, I’m porting out the SSSOM-specific code from SeMRA into a stand-alone library, sssom-pydantic. This serves as an alternative to the sssom-py (which I also help maintain) that is more focused on creating a reusable and high-performance data structure based on Pydantic.

Therefore, I implemented processing around a generic OntoPortal client in cthoyt/sssom-pydantic#14. It can be used like this (warning: subject to change):

import bioregistry
from sssom_pydantic.contrib.ontoportal import from_bioportal
from sssom_pydantic import SemanticMapping

converter = bioregistry.get_converter()
mappings: list[SemanticMapping] = from_bioportal("SNOMEDCT", "AERO", converter=converter)

You have to bring your own curies.Converter because OntoPortal’s data model doesn’t return a meaningful prefix map for parsing IRIs. The Bioregistry is a good and quick way to get a comprehensive prefix map.

Warning: BioPortal doesn’t provide an option to only return mappings between entities defined in the two given ontologies. For example, if you ask for mappings between SNOMEDCT and AERO, you will also get mappings between OGMS and SNOMEDCT (because OGMS terms are imported in AERO). This means that you should probably apply post-hoc filtering to only retain relevant mappings.

One way to do this is to rely on the definition of the converter, since any mappings with subject or objects with URIs that can’t be parsed are discarded:

import curies
from sssom_pydantic.contrib.ontoportal import from_bioportal

converter = curies.Converter.from_prefix_map(
    {
        "AERO": "http://purl.obolibrary.org/obo/AERO_",
        "SNOMEDCT": "http://purl.bioontology.org/ontology/SNOMEDCT/",
    }
)
mappings = from_bioportal("SNOMEDCT", "AERO", converter=converter)

Bulk Download

Ideally, I could get all mappings from BioPortal in bulk, instead of needing to hit the mappings API many times for each pair of two ontologies. The motivation for this post originally came from a question on the OBO Foundry Slack about where one could get SNOMED-CT mappings, so I wrote the following script to go through all ontologies in the Bioregistry that have BioPortal alignment to check for semantic mappings from SNOMED-CT to that mapping.

import bioregistry
import click
import pystow
import requests.exceptions
import sssom_pydantic
from sssom_pydantic import MappingSet
from sssom_pydantic.contrib.ontoportal import from_bioportal
from tqdm.contrib.logging import logging_redirect_tqdm
from tqdm import tqdm

MODULE = pystow.module("semra", "bioportal")
internal_to_bioportal = bioregistry.get_registry_map("bioportal")
converter = bioregistry.get_converter()

for internal, bioportal in tqdm(sorted(internal_to_bioportal.items())):
    if bioportal == "SNOMEDCT":
        continue
    name = f"snomedct-{internal}.sssom.tsv"
    path = MODULE.join(name=name)
    if path.is_file():
        tqdm.write(click.style(f"{bioportal} already cached to {path}", fg="green"))
        continue
    tqdm.write(click.style(bioportal, fg="green"))
    metadata = MappingSet(id=f'https://w3id.org/biopragmatics/mappings/bioportal/{name}')
    with logging_redirect_tqdm():
        try:
            mappings = from_bioportal("SNOMEDCT", bioportal, converter=converter)
        except requests.exceptions.HTTPError:
            tqdm.write(click.style(f"failed on {bioportal}\n", fg="red"))
        else:
            tqdm.write(click.style(f"{bioportal} got {len(mappings):,} mappings", fg="green"))
            if mappings:
                sssom_pydantic.write(mappings, path, converter=converter, metadata=metadata)
                tqdm.write(click.style(f"{bioportal} wrote to {path}\n", fg="green"))

As of writing, I haven’t been able to get this script to run to completion. The BioPortal API is often slow and gives timeouts. I included caching so I could resume after failure. As I mentioned earlier, this script doesn’t yet post-process mappings to the correct subset.

Thanks to John Graybeal for the suggestion on where to begin. He’s also helped me get in touch with the BioPortal team, so hopefully we can collaborate to get the API working using SSSOM directly or at least to get a bulk export of mappings.

Databases as Ontologies Part 1 - Background and Software

2025-10-14T08:14:00+00:00

This is the first of a two-part post about encoding databases as ontologies. In this post, I give a background on the problems in biocuration that led me to start encoding databases as ontologies, the software I have written to do it, and the repository I have created to store the resulting artifacts in a FAIR, open, and sustainable way. See also the second part which describes how I applied these tools to encode the HGNC (HUGO Gene Nomenclature Committee) database as an ontology.

Motivation for PyOBO

During my graduate studies, I was working with the Biological Expression Language (BEL), which curated causal, correlative, and associative relationships between chemicals, proteins, diseases, and other biological entities. At the time, BEL used a syntax similar to CURIEs for referencing entities that had a (pseudo-)prefix and a string label for an entity, which pointed to a BEL namespace file. Selventa had produced several namespace files in 2015 (e.g., for GO, DO, HGNC) before going out of business, but, because the namespace files relied on names instead of identifiers, they quickly became out of date. Naturally, I wanted to renew the existing BEL namespace files and also create new ones for additional ontologies to support the curation I was doing.

With Selventa’s ad hoc scripts as inspiration, I set out on developing PyOBO to make unified access to both ontologies and ontology-like resources (e.g., databases), both for the purposes of making new BEL namespaces, but also for creating simple, unified access to their contents. Initially, PyOBO wrapped Daniel Himmelstein’s obonet to read ontologies in the OBO flat file format and constructed caches for efficient querying of fields like the names, synonyms, and descriptions.

import pyobo

>> > pyobo.get_name("credit:software")
"Software"

Since, PyOBO has grown to have more detailed and opinionated processing workflows to handle the messy content curated across many ontologies, to be able to handle additional formats (despite OBO being the most approachable ontology format for outsiders, it also has the most issues associated with its serialization and parsing), and to implement an internal domain-specific language (DSL) for representing ontologies such that ontology-like resources could be converted into ontology files.

Databases as Ontologies

The conversion of ontology-like resources into ontologies became increasingly more interesting to me, especially because of my parallel interests in automatic construction of knowledge graphs. I found that ontologies were a convenient target for standardizing relationships, e.g., using the Relation Ontology (RO). Others were working on similar problems in parallel, such as the OBO community’s converter for the NCBI Taxonomy Database. Notably, Chris Mungall gave a talk in 2021 entitled Limits of ontologies: How should databases be represented in OBO? which organized thoughts and ongoing challenges, most of which are still relevant!

PyOBO now contains a suite of 60+ sources for resources covering chemistry, biology, medicine, cultural heritage, the semantic web, and other disciplines. Each source implements automated, version-aware download and caching of data files from the resource and transformation of the downloaded data into an OWL ontology within a concrete instance of an abstract base class provided by PyOBO. Typically, resources make available tabular data in which each row corresponds to a record in the resource that can either be transformed into a class or named individual within an OWL ontology. The remaining columns can often be mapped into other annotation properties (i.e., metadata) or object properties (i.e., part of the logical definition). For example, the label can be mapped to an annotation property using rdfs:label and the species (if applicable) can be mapped to an object property using RO:0002162 (in taxon).

Below is a demonstration of a minimal implementation of a PyOBO source for the CRediT (Contributor Roles Taxonomy), an informally constructed controlled vocabulary for describing how authors contributed to creative works. Note that the following is a relatively simple PyOBO source that only uses a subset of PyOBO’s DSL for encoding ontology components. The full version of this script is available in the PyOBO repository here.

from more_itertools import chunked
from pyobo.struct import Obo, Term
from pyobo.utils.path import ensure_json, ensure_open

PREFIX = "credit"
URI_PREFIX = "https://credit.niso.org/contributor-roles/"
DATA_URL = "https://api.github.com/repos/CASRAI-CRedIT/Dictionary/contents/Picklists/Contributor%20Roles"
ROOT_TERM = Term.from_triple(prefix="cro", identifier="0000000")


class CreditGetter(Obo):
    ontology = PREFIX
    static_version = "2022"
    root_terms = [ROOT_TERM.reference]

    def iter_terms(self, force: bool = False):
        yield ROOT_TERM
        for records in ensure_json(PREFIX, url=DATA_URL, name="picklist-api.json", force=force):
            with ensure_open(PREFIX, "picklist", url=records["download_url"], backend="requests", force=force) as file:
                header, *rest = file.read().splitlines()
                data = {key.removeprefix("## "): value for key, value in chunked(rest, 2)}
                term = Term.from_triple(
                    prefix=PREFIX,
                    identifier=data["Canonical URL"].removeprefix(URI_PREFIX).rstrip("/"),
                    name=header.removeprefix("# Contributor Roles/"),
                    definition=data["Short definition"],
                )
                term.append_parent(ROOT_TERM)
                yield term

Each PyOBO source inherits a fully automated workflow for converting resources into ontologies in the OBO flat file format and OWL functional-style syntax (OFN). Through OFN, PyOBO wraps OWLAPI to enable conversion to OWL/XML, RDF/XML, OBO Graph JSON and any other supported format. As an aside: it’s a goal of mine to reimplement key parts of OWLAPI in Python (or Rust) since it’s written in Java, which due to the language’s waning popularity, poses a risk for OWLAPI’s further maintenance and development.

from pyobo.sources.credit import CreditGetter

ontology = CreditGetter()
ontology.write_ofn("credit.ofn")
ontology.write_obo("credit.obo")
ontology.write_owl("credit.owl")  # enabled by OWLAPI

Here’s what the (abridged to only show a single term) OBO export looks like. Full exports of CRediT in all formats (e.g., OFN, OBO, OWL/XML) can be found here.

data-version: 2022
idspace: credit https://credit.niso.org/contributor-roles/ "Contributor Roles Taxonomy"
idspace: dcterms http://purl.org/dc/terms/ "Dublin Core Metadata Initiative Terms"
idspace: doap http://usefulinc.com/ns/doap# "Description of a Project"
idspace: foaf http://xmlns.com/foaf/0.1/ "Friend of a Friend"
idspace: orcid https://orcid.org/ "Open Researcher and Contributor"
ontology: credit
property_value: dcterms:title "Contributor Roles Taxonomy" xsd:string
property_value: dcterms:license "CC-BY-4.0" xsd:string
property_value: foaf:homepage "https\://credit.niso.org/" xsd:anyURI
property_value: doap:maintainer orcid:0000-0002-9298-3168
property_value: IAO:0000700 CRO:0000000

[Term]
id: credit:conceptualization
name: Conceptualization
def: "Ideas; formulation or evolution of overarching research goals and aims."
is_a: CRO:0000000

PyOBO implements several use case-specific exporters, such as exporting semantic mappings in the Simple Standard for Sharing Ontological Mappings (SSSOM), exporting synonyms and literal mappings in the Simple Standard for Sharing Literal Mappings (SSSLM), and nodes/edges files for import into the Neo4j graph database. It also implements high-level workflows to support named entity recognition (NER) and named entity normalization (NEN), and embedding entities using (medium) language models or graph machine learning.

PyOBO has a deep integration with the Bioregistry in order to achieve consistent identifier standards. As such, it’s required that all sources in PyOBO have a corresponding record in the Bioregistry, which covers key metadata including the name, description, homepage, license, URI prefix, and contact person for the resource. These fields are automatically propagated from the Bioregistry into ontology metadata using appropriate annotation properties, making ontologies produced by PyOBO much more transparent and understandable.

Because PyOBO is free and open source on GitHub, anyone can contribute additional sources to the upstream repository. Alternatively, PyOBO has a plugin system based on Python’s entry points such that additional sources can be implemented without making an upstream contribution. This supports use cases such as when the resource is not publicly available or when the code should not become publicly available (e.g., in the context of a company). For example, some pharmaceutical companies use the plugin system to implement sources for their compound registration system and internal ontology system.

OBO Database Ingest

The OBO Database Ingest (obo-db-ingest) is a GitHub repository that runs the scripts for a subset of PyOBO sources whose data are permissively licensed, and stores the OBO, OWL, OFN, OBO Graph JSON, SSSOM, SSSLM, and Neo4j files (e.g, see the folder for MeSH). It has a single Python script containing PEP 723-compliant inline script metadata such that it is fully-self contained and can be run with uv run, assuming a Java runtime is available for ROBOT and OWLAPI.

In theory, this script can be run on a chronological basis using GitHub Actions to keep the outputs up-to-date. However, in practice, usually one or more resources fail due to a combination of unreliable services (on the database’s side) or issues where updates to the underlying data cause the scripts to fail. I’m still working on making this script and PyOBO’s sources themselves more resilient to such failures! One stop-gap could be to have smaller workflows running on the most important resources that are updated frequently (e.g., HGNC updates monthly) versus other resources which are updated infrequently (e.g., MeSH updates yearly). As an alternative, I often run this script locally, which usually takes less than an hour because PyOBO cleverly caches versions, and the script avoids duplicate work for data that hasn’t been updated.

The repository is structured such that there’s a consistent location for the latest output of each PyOBO source as well as version-specific outputs. This method indeed has its limits, since git is not really meant to be a file storage system, especially for big files.

Because of its simple structure, it’s possible to assign persistent URLs (PURLs) to each resource, configured by W3ID. PURLs abstract away the URL that points to a specific physical infrastructure required for storing and serving files, making it possible to be changed later, and therefore making resources more reliable to reference. As such, PURLs are an often-requested feature by ontologies that would like to import and incorporate PyOBO sources. They also enable the ontologies to be incorporated into tools like the EBI’s Ontology Lookup Service (OLS), for example, I’ve already done this for MeSH.

Here’s what a few PURLs look like for OBO files (replace the file extension with .ofn, .sssom.tsv, etc. to get other resources):

Resource	Latest OBO PURL	Version Type	Example Versioned OBO PURL
Reactome	https://w3id.org/biopragmatics/resources/reactome/reactome.obo	Sequential	https://w3id.org/biopragmatics/resources/reactome/83/reactome.obo
Interpro	https://w3id.org/biopragmatics/resources/interpro/interpro.obo	Major/Minor	https://w3id.org/biopragmatics/resources/interpro/92.0/interpro.obo
DrugBank Salt	https://w3id.org/biopragmatics/resources/drugbank.salt/drugbank.salt.obo	Semantic	https://w3id.org/biopragmatics/resources/drugbank.salt/5.1.9/drugbank.salt.obo
MeSH	https://w3id.org/biopragmatics/resources/mesh/mesh.obo.gz	Year	https://w3id.org/biopragmatics/resources/mesh/2023/mesh.obo.gz
UniProt	https://w3id.org/biopragmatics/resources/uniprot/uniprot.obo.gz	Year/Month	https://w3id.org/biopragmatics/resources/uniprot/2022_05/uniprot.obo.gz
HGNC	https://w3id.org/biopragmatics/resources/hgnc/hgnc.obo	Date	https://w3id.org/biopragmatics/resources/hgnc/2023-02-01/hgnc.obo
CGNC	https://w3id.org/biopragmatics/resources/cgnc/cgnc.obo	unversioned	N/A

The script also outputs a full manifest that can be consumed by downstream resources that want to consume all content in the repository, such as the KG Registry.

This post continues in a second part, where I describe how I implemented a more complex PyOBO source for the HGNC (HUGO Gene Nomenclature Committee) database.

Databases as Ontologies Part 2 - A Case Study with HGNC

2025-10-14T08:14:00+00:00

This is the second of a two-part post about encoding databases as ontologies. In the first part, I gave a background on how I started working on this problem and the software stack I developed along the way. In this post, I explain the philosophy and design about how I encoded the HGNC (HUGO Gene Nomenclature Committee) database as an ontology using PyOBO.

While the previous post used CRediT (Contributor Roles Taxonomy) to demonstrate encoding as an ontology a simple resource that only contains names and descriptions for its identifiers, the goal of this post is to describe the design decisions take to ontologize a more complex resource: the HGNC (HUGO Gene Nomenclature Committee) database.

The HGNC assigns names, symbols, and numeric identifiers to human genes. Gene symbols like AKT1 are the primary names referenced in the biomedical literature (which are sometimes misunderstood by Microsoft Excel). HGNC identifiers are used to unambiguously reference human genes in biocuration efforts like DECIPHER, MedlinePlus, GeneCards, and the Alliance of Genome Resources. They are also the targets of grounding human genes in manual literature curation workflows (like for BEL, BioPAX, SBML) and text mining workflows (like INDRA).

I see the following three major benefits in ontologizing HGNC:

To support the standardized reuse of HGNC terms within semantic web applications and ontologies. While databases can create fields with well-defined semantics where they place either numeric HGNC identifiers or references to HGNC gene symbols, semantic web applications often require the use of (consistent) URIs and ontologies further require consistent as classes/individuals with the appropriate axioms. For example, the MONDO Disease Ontology annotates genes’ relationships to disease (such as being a disease driver), but they are forced to use workarounds to reference HGNC records, since they are not encoded in an ontology.
To support the standardized distribution of HGNC. HGNC has its own ad hoc distribution formats (JSON, SQL, TSV). Ontologizing HGNC enables standard tooling to consume and reuse the database.
To support the standardized interpretation of HGNC. The content of HGNC does not have formally defined semantics - for example, if you download the JSON dump, how does one know what the mane_select key means, or what virus integration site means in the locus_type field? Ontologizing HGNC enables for a single person or small group to do the hard work of understanding the meaning of the fields and values used in the source data, then encode their hard-earned domain knowledge with formal semantics such that everyone can understand it. I’ll use the locus_type and locus_group fields as an example to illustrate this.

I don’t want to bury the lede, so here’s a link to the PyOBO source script for HGNC that implements everything I’m about to describe. Actionable feedback and pull requests are welcome if you have concrete ideas for improvement.

Lexicalization of a Gene

Each record contains up to five lexical components (i.e., name, description, synonyms), which are mapped to the ontology as follows:

Key	Cardinality	Predicate	Synonym Type
`symbol`	one	`rdfs:label`	N/A
`name`	one	`dcterms:description`	N/A
`alias symbol`	zero or more	`oboInOwl:hasExactSynonym`	`OMO:0003016` (gene symbol synonym)
`alias_name`	zero or more	`oboInOwl:hasExactSynonym`	`OMO:0003008` (previous name)
`previous_symbol`	zero or more	`oboInOwl:hasExactSynonym`	`OMO:0003015` (previous gene symbol)

The dichotomy of gene symbols (short form) and gene names (long form) requires a making the important design decision of which to use as the label. I chose to use the gene symbol because of its ubiquitous use, see discussion here. An alternative to this lexicalization could be to mark the name as the primary label with rdfs:label and to use the symbol as an exact synonym with type abbreviation OMO:0003000. However, using the gene symbol as the primary label is so ubiquitous that this seemed appropriate. Further, HGNC does not provide dedicated textual descriptions, and in their place, the name is often a reasonable alternative.

Here’s an example record in OBO flat file format to illustrate:

[Term]
id: hgnc:100
name: ASIC1
def: "acid sensing ion channel subunit 1"
synonym: "ACCN2" RELATED OMO:0003015 []
synonym: "BNaC2" RELATED OMO:0003016 []
synonym: "acid sensing (proton gated) ion channel 1" RELATED OMO:0003008 []
synonym: "acid-sensing (proton-gated) ion channel 1" RELATED OMO:0003008 []
synonym: "amiloride-sensitive cation channel 2, neuronal" RELATED OMO:0003008 []
synonym: "hBNaC2" RELATED OMO:0003016 []

As an aside: the classes and properties needed to curate an ontology, or ontologize a database, aren’t always available from the start. In many situations, this leads to making ad hoc classes or properties to get the job done - I am not above this. Initially, I had created ad hoc synonym types for gene symbol synonyms and previous gene symbols. Later, I requested two new synonym types in the OBO Metadata Ontology (OMO) to cover these use cases. This is actually a difficult step, because it requires justifying to the community why they are useful. In this case, I think it’s clear, since all model organism databases (MODs) make these kinds of synonyms, and I was able to give a good justification based on the fact that I also made similar ad hoc synonym types for the PyOBO source for the Rat Genome Database (RGD). After doing the design work and making the pull request, I updated both the HGNC and RGD sources in PyOBO to reuse these terms in biopragmatics/pyobo#447.

Classification of a Gene by Locus Type

Each gene is annotated with a locus group and locus type. These correspond to a classification, which translates into an ontology as parent-child relationships between classes, mediated by the rdfs:subClassOf relationship (often abbreviated by is a). Here’s a count summary of all locus groups at the time of writing:

Locus Type	Frequency
protein-coding gene	19,297
pseudogene	14,602
non-coding RNA	9,634
other	1,004

Here’s a count summary of all locus types at the time of writing. It’s clear that the locus type is more granular than locus group and completely subsumes it. Therefore, I’ll throw away the locus group and only discuss the locus type here. Looking ahead, I’ve included my manual mapping from each ad hoc values used in HGNC to formal terms in the Sequence Ontology (SO).

Locus Type	Frequency	Sequence Ontology
gene with protein product	19,297	SO:0001217
pseudogene	14,361	SO:0000336
RNA, long non-coding	6,296	SO:0002127
RNA, micro	1,912	SO:0001265
RNA, transfer	591	SO:0001272
RNA, small nucleolar	568	SO:0001267
immunoglobulin gene	230	SO:0002122
T cell receptor gene	205	SO:0002133
immunoglobulin pseudogene	203	SO:0002098
readthrough	151	SO:0000697
RNA, cluster	119	SO:0003001 (see PR)
fragile site	116	SO:0002349
endogenous retrovirus	110	SO:0000100
unknown	69	SO:0000704 (mapped to top-level gene)
complex locus constituent	69	SO:0000997
RNA, ribosomal	60	SO:0001637
RNA, small nuclear	51	SO:0001268
region	46	SO:0001411
T cell receptor pseudogene	38	SO:0002099
RNA, misc	29	SO:0001266
virus integration site	8	SO:0003002 (see PR)
RNA, Y	4	SO:0002359
RNA, vault	4	SO:0002358

I created this issue on the PyOBO tracker when I started preparing this mapping, since there were a few already available in the info box for the locus type on a given gene page on the HGNC website. However, several were incorrect and most were missing. Therefore, I had to manually map several to terms in the Sequence Ontology. Many mappings were easy, but several required discussion with the HGNC and Sequence Ontology teams (as you can see on the issue). HGNC was proactive and incorporated my mappings into their front-end.

There were several cases where there was no appropriate term in the Sequence Ontology. For some, the maintainers created new terms. Unfortunately, for some, the maintainers were unresponsive, so I had to make my own PRs to the repository which probably won’t get accepted in a timely fashion. However, I was able to use the placeholder identifiers in the PyOBO source module even though they haven’t yet been merged and released.

As an aside: annotating locus types is not just a human gene problem, but all model organism databases (MODs) need to work on. I already have a thread for taking a similar approach for FlyBase, but it would be great to do the same for MGI (mouse), RGD (rat), and other MODs for which PyOBO encodes a source. In general, it would be great to see the Alliance of Genome Resources (AGR) push their members towards adopting more shared semantics in the way they curate, especially for locus types.

Chromosomal Locations

The location field connects a gene to its chromosomal location by encoding the location as a string. Initially, I had created an ad hoc relation to encode this string field (obo:hgnc#has_location). In biopragmatics/pyobo#451, I adapted this to map the chromosomal location strings to classes in the Chromosome Ontology (CHR) and use a combination of well-established relations, based on the apparent values for the chromosomes. Note that this is a first attempt at ontologization, and the relations might need updating.

Single Point Annotations

RO:0001025 (located in) is used for single point annotations, such as in hgnc:10080.

[Term]
id: hgnc:10080
name: RNPS1
is_a: SO:0001217 ! protein_coding_gene
relationship: RO:0001025 CHR:9606-chr16p13.3 ! located in 16p13.3 (Human)

Pairs of Points

Multiple RO:0001025 (located in) is used for pairs of point annotations, e.g., when written like Xq28 and chrYq12, like in hgnc:38513:

id: hgnc:38513
name: WASIR1
is_a: SO:0002127 ! lncRNA_gene
relationship: RO:0001025 CHR:9606-chrXq28 ! located in Xq28 (Human)
relationship: RO:0001025 CHR:9606-chrYq12 ! located in Yq12 (Human)

There’s also a single example of a location containing an “or” in hgnc:3829 which looks like 10q23.3 or 10q24.2. There are more sophisticated ways of represent “or” logic in OWL, but not serializable directly in the OBO flat file format.

Ranges

RO:0002223 (starts) and RO:0002229 (ends) are used for ranges of chromosomes, e.g., when written like 8q11.23-q12.1, like in hgnc:10263:

[Term]
id: hgnc:10263
name: RP1
is_a: SO:0001217 ! protein_coding_gene
relationship: RO:0002223 CHR:9606-chr8q11.23 ! starts 8q11.23 (Human)
relationship: RO:0002229 CHR:9606-chr8q12.1 ! ends 8q12.1 (Human)

Special case: Mitochondria

Genes that are mapped to the mitochondrial chromosome get mapped to the Gene Ontology (GO) term GO:0000262 instead of a Chromosome Ontology term, like in hgnc:50279:

[Term]
id: hgnc:50279
name: MT-LIPCAR
is_a: SO:0002127 ! lncRNA_gene
relationship: RO:0001025 GO:0000262 ! located in mitochondrial chromosome

Qualified Annotations

Some annotations that end with a qualifier “not on reference assembly”, “unplaced”, or “alternate reference locus” get them annotated as comment axioms.

[Term]
id: hgnc:10082
name: RNR1
is_a: SO:0003001
relationship: RO:0001025 CHR:9606-chr13p12 {rdfs:comment="not on reference assembly -named gene is not annotated on the current version of the Genome Reference Consortium human reference assembly; may have been annotated on previous assembly versions or on a non-reference human assembly"} ! located in 13p12 (Human)

Unprocessable Locations

After processing HGNC, there were several locations that could not be mapped to CHR. I made an issue on the Chromosome Ontology’s issue tracker noting all the locations that were not mappable. However, several of these could be errors on the side of HGNC as well, and requires checking each manually.

unhandled location	count	appears in
10q23.3 or 10q24.2	1	hgnc:3829
Yp13.3	1	hgnc:6012
17qter	1	hgnc:8841
13cen, GRCh38 novel patch	1	hgnc:15732
Xp22.22	1	hgnc:10199
1p36.13q41	1	hgnc:36026
12q22.32	1	hgnc:58534
1q13.1	1	hgnc:32558
3q25.22	1	hgnc:32563
7p36.1	1	hgnc:34871
11p11.2	1	hgnc:58650
22pter	1	hgnc:1838
18p22.3	1	hgnc:58557
Xp11.32	1	hgnc:37114
17q12b	1	hgnc:49316

Here’s a few observations I had on this:

the ter seems to be an annotation related to trisomy
13cen, GRCh38 novel patch is a weird outlier
1p36.13q41 might be a typo
the “or” entry probably should be processed and not actually get a term, but keeping here for completeness
some of them might have typos between “p” and “q”

In general, these kinds of unmapped items are not blockers towards ontologizing a resource. It’s generally valuable to include logging in a PyOBO source when there is content that is unhandled, since this can be valuable feedback for the upstream resources themselves.

Membership in Gene Groups

HGNC has a secondary categorization of genes into gene groups (formerly called gene families). There’s a variety of purposes for gene groups which themselves have a hierarchical classification. However, based on the contents of gene groups, I don’t think that it’s appropriate to use rdfs:subClassOf for relations between genes and gene sets. Instead, I have opted to use RO:0002350 (member of), which is defined as a mereological relation (i.e., a part-of relation) between an item and a collection.

Genes and Enzymes

HGNC annotates genes with Enzyme Commission (EC) codes. There’s spirited discussion in the ontology world about how we should ontologize enzymes. For example, the Gene Ontology ( GO) models them as catalytic activities within GO’s molecular function branch.

We typically classify proteins based on their catalytic activities, which means that to model the relationship between a gene and an enzyme, we need to use a property chain connecting the gene to the protein it encodes (RO:0002205), and the catalytic activity that the protein enables (RO:0002327).

graph LR
    gene["AKT1 (HGNC:391)"] -- " has gene product (RO:0002205) " --> protein["RAC-alpha serine/threonine-protein kinase (uniprot:P31749)"] -- " enables (RO:0002327) " --> activity["non-specific serine/threonine protein kinase (EC:2.7.11.1)"]
    gene -- has gene product that enables --> activity

This property chain doesn’t yet exist in RO, so I made a new term request and associated pull request to proactively mint a new identifier for use in PyOBO. I have a pull request to PyOBO (biopragmatics/pyobo#455) waiting to reflect this change depending on feedback. Otherwise, the old ontologization uses a property chain of gene product of and member of, to consider an EC class as a more general classification class.

I’m still undecided on what’s the best modeling choice. I am keen to fill out the following chart in a more satisfying way, that captures a bit more nuance in the fact that enzymes are a classification that implies the ability to carry out an activity, but when going against the historical choices of a resource as large as go, I am punching outside my ontological weight class 🤷.

Remaining Logical Axioms and Semantic Mappings

As I come to a close, the only remaining content to ontologize are the many database cross-references. A first and simple approach is to use oboInOwl:hasDbXref, but this is a missed opportunity to encode domain knowledge about each of the resources. The following chart gives an overview of the remaining logical axioms and semantic mapping types:

graph LR
    genegroup[Gene Group
HGNC] -- " member of
(RO:0002350) " --- gene[Gene]
    geneclass[Gene Class
SO] -- is a --- gene
    gene -- " transcribed to (RO:0002511) " --> rna[RNA
RNA Central, miRBase, snoRNABase]
    gene -- " has gene product
(RO:0002205) " --> protein[Protein
UniProt]
    gene -- " has exact match
(skos:exactMatch) " --> external1[External
NCBIGene, Ensembl, Orphanet, OMIM, RefSeq,...]
    gene -- " has database cross-reference
(oboInOwl:hasDbXref) " --> externa2[External
CCDS,...]
    gene -- " is orthologous to
(RO:HOM0000017) " --> orthology[Orthologous Gene
MGI, RGD]
    gene -- " has gene product that enables
(RO:0002205 + RO:0002327) " --> enzyme[Enzyme
EC]
    gene -- " located in
(RO:0001025) " --> chr[Chromosome Region
CHR]

To briefly summarize this diagram:

References to other model organism databases are modeled as orthology relationships
References to UniProt (the protein database) are modeled as has gene product (i.e., a broader relationship than translation)
References to RNA databases are modeled as transcription
References to databases that are nomenclature resources for genes are modeled as exact matches
References to databases that can have potentially multiple experimental measurements for a given gene are modeled with database cross-references
I already mentioned in more detail above how enzymes, gene groups, and locations are annotated.

The implementation of this logic can be found in the PyOBO source module for HGNC.

What Was Skipped

There are many extra fields in HGNC that I throw away which effectively duplicate the HGNC identifier or gene symbol, such as agr (which reuses the HGNC identifier) and lncrnadb (which reuses the HGNC gene symbol).

Rather than representing the fact that the external database provides information about this term, a better solution is to add additional providers to the Bioregistry, such that a given HGNC identifier can be used to create a link to the database itself. This isn’t a perfect solution, because some databases only cover a subset of genes. There’s more discussion about this on this issue on OMO issue tracker, specifically in this comment.

While I don’t offer an isomorphic (i.e., covers everything that’s there) solution for ontologization of this part of the content, I do believe that the rest of my choices address the three big benefits I mentioned at the start.

Wow, this is my first ever double blog post. It took me a full day to write it, not to mention the years of work that went into the software ecosystem itself and the time put into improving the HGNC PyOBO source module in preparation for writing it. I am very happy to be reporting on this, and to see how it will positively impact the community.

If you made it this far and are interested in collaborating to make your own resource accessible through PyOBO, please get in touch using my contact information at the bottom of the page or by opening an issue on the PyOBO issue tracker.

I’m also open to collaboration through grant writing or contract/consulting work via my current employer (RWTH Aachen University) for extending and applying PyOBO and the wider Biopragmatics Stack in new domains. This has been previously successful in the DARPA ASKEM and DTRA RAPTER projects, and is now a key contribution to several of the DFG-funded German NFDI consortia.

Bridging NFDI’s culture and chemistry knowledge graphs

2025-10-07T21:18:00+00:00

At the sixth NFDI4Chem consortium meeting, Torsten Schrade from the NFDI4Culture consortium gave a lovely and whimsical talk entitled A Data Alchemist’s Journey through NFDI which explored ways that we might federate and jointly query both consortia’s knowledge via their respective SPARQL endpoints. He proposed a toy example in which he linked paintings depicting alchemists trying to make gold to compounds containing gold. This post is about the steps I took to automate his toy example and extend it to not only chemicals or compounds represented in Iconclass, but also equipment and devices.

Part 1: The Semantic Lay of the Land

The first step in effectively integrating data and writing federated SPARQL queries is to understand the landscape ontologies, controlled vocabularies (CVs), and databases whose entities will be referenced in SPARQL queries and will appear in the knowledge graph(s). It also requires an understanding of the syntaxes used to reference these entities, namely the uniform resource identifier (URI) syntax (e.g., http://purl.obolibrary.org/obo/CHMO_0000073) or compact uniform resource identifier (CURIE) syntax (e.g., CHMO:0000073). For more information on URIs and CURIEs, see my previous post.

For example, the following SPARQL query against the NFDI4Chem knowledge graph uses SPARQL’s PREFIX syntax to abbreviate long URIs and write a compact query.

PREFIX prov: 
PREFIX rdf: 
PREFIX CHMO: 
PREFIX nfdi4chem.doi: 

SELECT * WHERE {
  ?dataset prov:wasGeneratedBy/prov:used ?experiment .
  ?experiment prov:wasGeneratedBy/rdf:type CHMO:0000073 .
}
LIMIT 4

When executed, this query returns the experiments containing an artifact generated by scanning electron microscopy (CHMO:0000073) and the datasets that they are part of:

dataset	experiment
nfdi4chem.doi:YJAIWVDPLYJOFU-UHFFFAOYSA-B/CHMO0000073	nfdi4chem.doi:YJAIWVDPLYJOFU-UHFFFAOYSA-B/CHMO0000073/spectrum
nfdi4chem.doi:YJAIWVDPLYJOFU-UHFFFAOYSA-B/CHMO0000073	nfdi4chem.doi:YJAIWVDPLYJOFU-UHFFFAOYSA-B/CHMO0000073/spectrum
nfdi4chem.doi:ABCUGETYYULVMM-UHFFFAOYSA-N/CHMO0000073	nfdi4chem.doi:ABCUGETYYULVMM-UHFFFAOYSA-N/CHMO0000073/spectrum
nfdi4chem.doi:ABCUGETYYULVMM-UHFFFAOYSA-N/CHMO0000073	nfdi4chem.doi:ABCUGETYYULVMM-UHFFFAOYSA-N/CHMO0000073/spectrum

This immediately leads to a few questions:

What are prov, rdf, CHMO and nfdi4chem.doi that appear with in PREFIX lines of the SPARQL query and later in the results?
Where do they come from?
How do I know the correct URI prefix to use with the CURIE prefix (e.g., http://www.w3.org/ns/prov# goes with prov)?
How do I know which prefixes are relevant in my domain (e.g., chemistry)?
How do I know which prefixes to use when writing a SPARQL query? (I’m going to gloss over this question, since the best answer is usually for the knowledge graph maintainers to write more/better documentation)

Besides the last question, the answer is the Bioregistry: an open source, community-curated database of prefixes for ontologies, CVs, databases, and other resources that mint identifiers that might appear in SPARQL queries or in knowledge graphs. In the next few sections of this post, I’ll explain how the Bioregistry answers these questions.

Caveat: despite its name, the Bioregistry is cross-disciplinary, and we’re currently working on rebranding it to better reflect this.

Resolving prefixes, CURIEs, and URIs with the Bioregistry

The Bioregistry can be used interactively from its web interface to search for prefixes either quickly from the homepage or in more detail from the catalog page.

If you already know the prefix, you can construct a URL by adding it to the end of https://bioregistry.io like https://bioregistry.io/CHMO. These lead to pages that describe the resource, say what is the URI prefix that should appear in your SPARQL queries. Alternatively, entries in the Bioregistry have textual descriptions, keywords, and tags that can be searched to find ontologies, CVs, and databases relevant for your domain.

If you have a CURIE, you can use the Bioregistry to automatically link to a webpage by adding it to the end of https://bioregistry.io like https://bioregistry.io/CHMO:0000073.

Extending the Bioregistry

When a prefix is missing from the Bioregistry, anyone can make a suggestion to add a new one, even if it’s not for your resource (video tutorial). When maintainers suggest a prefix for their own resource, it’s also a valuable opportunity to get feedback on if the prefix and URI schema follow best practices.

rdfs, prov, and CHMO are already in the Bioregistry, but there’s nothing that reflects nfdi4chem.doi. This means there’s an opportunity here to add a new prefix! I already chatted with the Chemotion team about this at the NFDI4Chem meeting and am keen to help them adopt best practices in minting identifiers, then get their resources registered in the Bioregistry.

Part 2: Operationalization of Resources

After we have found the relevant prefixes, CURIEs, and URIs using the Bioregistry, we also want to make sure that we can download the associated ontologies, CVs, or databases. For ontologies, there are explicit fields in the Bioregistry to link to OWL, OBO, or SKOS artifacts (when available).

However, for databases, each typically needs its own custom adapter into an ontology-like format. This section focuses on doing this for Iconclass.

Iconclass

Iconclass is a controlled vocabulary used to annotate parts of images with what they depict. For example, iconclass:49E3911 is used to annotate a part of an image depicting an alchemist trying to make gold. Iconclass identifiers implicitly contain the hierarchy:

iconclass:49E391 is used to annotate an alchemist at work
iconclass:49E39 is used to annotate alchemy
iconclass:49E3 is used to annotate chemistry
iconclass:49E is used to annotate science and technology
iconclass:49 is used to annotate education, science, and learning
iconclass:4 is used to annotate Society, Civilization, Culture

Here’s how its web browser looks:

Iconclass doesn’t appear to be curated like an ontology, so the hierarchy is often confusing to follow. I found it easier to understand the logic behind the hierarchy if you prepend “depiction of” or “depiction of something related to” in front of each label. This also makes up for some of the illogical hierarchical relations. Here’s how I imagine the same hierarchy could be clarified with better labels:

iconclass:49E391 depiction of an alchemist at work
iconclass:49E39 depiction of alchemy
iconclass:49E3 depiction of something related to chemistry
iconclass:49E depiction of something related to science or technology
iconclass:49 depiction of something related to education, science, or learning
iconclass:4 depiction of something related to society, civilization, or culture

Because Iconclass isn’t curated as an ontology, there isn’t an OWL or OBO file that can be used with standard tooling. However, some of the source data is available on GitHub at iconclass/data, so it’s possible to write custom code that wrangles it into an ontology-like shape. I’ve actually done this for dozens of repositories already, and written PyOBO, a library of reusable tooling to support ingesting new resources across domains in an ontology-like shape.

Accordingly, I added a source to PyOBO to ingest Iconclass in biopragmatics/pyobo#433. This not only enables it to generate ontology-like artifacts in the OWL and OBO formats, but also gives access to the text mining and FAIR mapping tools built on top of PyOBO.

Along the way, I found that Iconclass has a lot more weird and irregular identifiers than I had earlier assumed. I was able to make an additional pull request to the Bioregistry in biopragmatics/bioregistry#1686 to update the underlying regular expression pattern and add extra examples to demonstrate the weirdness. This is important because PyOBO uses the Bioregistry for regular expression validation of identifiers internally, and without this update, the Iconclass source doesn’t work!

Part 3: Bridging the Semantic Gap

The next goal was to identify entries in Iconclass correspond to elements, compounds, laboratory equipment, or other terms relevant in the chemistry domain, and create semantic mappings that can serve as a “semantic bridge” between disciplines.

This makes use of Simple Standard for Sharing Ontological Mappings (SSSOM) as a community standard for storing semantic mappings and giving access to standardized tooling for accessing, querying, and applying them.

First attempt: lexical matching

The Biomappings project provides tools for predicting semantic mappings using lexical matching in SSSOM. Much like the Bioregistry, this has a bit of a nomenclature issue and is actually cross-disciplinary. Anyway, it can quickly be used to spin up a workflow for matching any two vocabularies available through PyOBO with a few lines. I gave it a try to match Iconclass to the Chemical Methods Ontology (CHMO):

from biomappings.lexical import lexical_prediction_cli

if __name__ == "__main__":
    lexical_prediction_cli(__file__, "iconclass", "chmo")

This usually works well for matching entities in resources curated as ontologies, but because Iconclass’s labels aren’t typical, it wasn’t able to generate more than a handful of matches.

Second attempt: language models and embedding similarity

This prompted me to take a different approach that relies on (medium) language models to generate embeddings, which are better able to capture the subtle differences in the way entities are labeled. This led to the following improvements:

I added functionality to PyOBO to get a dataframe of embeddings for all entities in a given ontology or controlled vocabulary in biopragmatics/pyobo#434
I extended the lexical prediction workflow in Biomappings to have a method that combines embedding generation in PyOBO with similarity calculation and finally the application of a similarity cutoff for calling mappings in biopragmatics/biomappings#206.

After this, I was able to update my workflow to look like this:

from biomappings.lexical import lexical_prediction_cli

if __name__ == "__main__":
    lexical_prediction_cli(
        __file__,
        "iconclass",
        "chmo",
        method="embedding",
        cutoff=0.9
    )

Third Attempt: NER

Embedding similarity worked well enough for mapping Iconclass records to CHMO and the Ontology for Biomedical investigations (OBI) (another ontology containing experimental equipment), but it didn’t work at all for ChEBI because Iconclass records that mention chemicals typically have a large amount of other text.

This led me to reformulate mapping as a named entity recognition (NER) task, for which I implemented yet another new workflow in Biomappings: biopragmatics/biomappings#209

from biomappings.lexical import lexical_prediction_cli

if __name__ == "__main__":
    lexical_prediction_cli(
        __file__,
        "iconclass",
        "chebi",
        method="ner",
    )

After all of this, I added a first set of curations to the Biomappings project in biopragmatics/biomappings#205 which are stored in SSSOM within the GitHub repository. Normally, I commit all predictions, but they are so noisy and numerous, that I only committed the curations since I focused on a small set that will support the bigger story here, and larger scale curation can be done for Iconclass to chemistry (and other domains) in a follow-up.

Interlude 1: Exploring New SPARQL Endpoints

Before diving in fully federated queries across multiple sources, I want to warm up by making self-contained queries over the respective NFDI4Culture and NFDI4Chem knowledge graphs. Unfortunately, neither of them are well documented for what I want to do, so put on your pith helmet and get ready for some digital archaeology.

Querying NFDI4Culture

The NFDI4Culture Consortium makes its knowledge graph queryable from SPARQL here: https://nfdi4culture.de/resources/knowledge-graph.html. Using the example from Thorsten’s talk as a starting point, and I wrote the following SPARQL query for Iconclasses, objects they are annotated to, and URLs for digital depictions of those objects.

PREFIX cto: 
PREFIX iconclass: 
PREFIX rdfs: 
PREFIX schema: 

SELECT ?resource ?resourceLabel ?imageURL ?iconclass
WHERE {
  ?resource cto:subjectConcept ?iconclass ;
            schema:image ?imageURL .
  FILTER STRSTARTS(STR(?iconclass), STR(iconclass:))
  OPTIONAL { ?resource rdfs:label ?resourceLabel }
}
LIMIT 50

One record that was returned was Bildnis Professor Hoffmann (bildindex:obj00003367) from the BildIndex der Kunst & Architektur. Notably, this image depicts the eponymous professor with his microscope (iconclass:49E2512). BildIndex didn’t give enough context for me to figure out who Professor Hoffman was (e.g., by connecting to Wikidata or other external resources that describe notable people), but it did say that the painting is in Bonn! Maybe I will go track it down in person to bring this blog post full circle.

Minor Criticisms

I immediately identified two issues with the NFDI4Culture knowledge graph: first, there are no English labels for resources. This is understandable given this is a German project and unlike in science, there isn’t a huge pressure for internationalization, but it does reduce the value of the resource for anyone outside the Germanophone world. Second, even worse, there are no labels for Iconclass in German, English, nor any other language. Because I produced ontology artifacts for Iconclass (as described earlier in this post), I can get these labels in RDF by federating over my own OWL files, but I think that this would be an important addition to the NFDI4Culture knowledge graph to improve its own usability.

Querying NFDI4Chem

Time to switch domains! The NFDI4Chem Consortium makes its knowledge graph queryable from SPARQL here: https://search.nfdi4chem.de/sparql. I started with the following SPARQL query to investigate which measurement processes from CHMO appear like NMR, mass spectrometry, X-ray diffraction, and microscopy.

PREFIX schema: 
PREFIX rdfs: 

SELECT ?o ?label (COUNT(?o) as ?count)
WHERE {
  ?s ?p ?o .
  OPTIONAL { ?o schema:name ?label }
  FILTER(STRSTARTS(STR(?o), "http://purl.obolibrary.org/obo/CHMO_"))
}
GROUP BY ?o ?label
ORDER BY DESC(?count)

While most of the techniques currently appearing in the NFDI4Chem knowledge graph are a bit too modern to be of (wide) interest in the cultural heritage world, the term for scanning electron microscopy (CHMO:0000073) is a descendant of microscopy (CHMO:0000067), and this is something that has a corresponding Iconclass (49E2512) for the depiction of a microscope which I curated earlier using Biomappings.

On the Impedance between Processes and Material Entities

You’d be correct in saying that I made a bit of a hop from the physical instrument of a microscope (CHMO:0000953) to the process of microscopy (CHMO:0000067). Please give me a bit of wiggle room, since it’s already hard enough to thread the needle between culture and chemistry. Luckily, CHMO uses a logical axiom on the definition of microscopy to denote that all microscopy has a microscope as a participant (BFO:0000057).

While we’ll need this axiom later to construct a more sophisticated SPARQL query that can resolve the impedance between the sense that I curated in the mappings and what appears in experimental data, it doesn’t appear that the NFDI4Chem knowledge graph imports a reasoned version of CHMO that materializes this axiom as a triple . This means that we’ll have to inject this triple ourselves via federation later.

It’s easy to get confused in the semantic web, ontologies, and RDF world. Ontologies are typically encoded using the OWL schema, which can be serialized to XML and RDF. However, this doesn’t mean that it is RDF that represents the graph you might expect an ontology induces. For example, the existential restriction that all microscopy has a microscope as a participant is encoded in OWL like this:

CHMO:0000067 rdf:type owl:Class ;
    rdfs:subClassOf OBI:0000185 ,
        [ rdf:type owl:Restriction ;
          owl:onProperty BFO:0000057 ;
          owl:someValuesFrom CHMO:0000953 .
        ] .

This means if you want to get the aforementioned triple , you’ll need the following SPARQL:

SELECT ?s ?p ?o WHERE {
    ?s rdfs:subClassOf [
        rdf:type owl:Restriction;
        owl:onProperty ?p ;
        owl:someValuesFrom ?o
    ] .
}

The NFDI4Chem knowledge graph doesn’t actually import CHMO (nor any other ontologies at the moment) so in order to incorporate this into a larger query, we’d have to federate once again!

Exploring Microscopy in NFDI4Chem

After a bit of exploring, I was able to construct a query (which you may remember from earlier in this post) that returns all experimental results (?experiment) that were generated by microscopy, and the dataset/record where they were recorded.

PREFIX prov: 
PREFIX rdf: 
PREFIX CHMO: 

SELECT * WHERE {
  ?dataset prov:wasGeneratedBy/prov:used ?experiment .
  ?experiment prov:wasGeneratedBy/rdf:type CHMO:0000073 .
}

One of the twelve records (as of September 18^th, 2025) returned by this query was a scanning electron microscopy (SEM) dataset performed on a zirconium-containing inorganic molecule. The URI for the experiment, https://doi.org/10.14272/YJAIWVDPLYJOFU-UHFFFAOYSA-B/CHMO0000073/spectrum, does not resolve, but the dataset at https://doi.org/10.14272/YJAIWVDPLYJOFU-UHFFFAOYSA-B/CHMO0000073 does. These URIs make me want to cry - they do not follow any best practices for how identifiers should look as prescribed by Julie McMurry et al. (2017) in Identifiers for the 21^st century. Despite my grumblings, they can still be used as URIs and function well enough for SPARQL queries.

Ultimately, here’s the image from the SEM in the experiment in question, though I wasn’t able to construct a SPARQL query that returned it:

Minor Criticisms

Given that the NFDI4Chem knowledge graph was only deployed in earnest a few weeks ago and one of its first demos was at the NFDI4Chem Consortium 6.0 Meeting, it’s still lacking in documentation, examples, schematic diagrams, and other training materials to support people like me who want to navigate the knowledge graph and construct SPARQL queries - I have high hopes that this will be forthcoming.

Querying Mappings

While SSSOM is usually stored in TSV, it has a specification for converting to RDF, which means that mappings encoded in SSSOM can be leveraged in SPARQL queries through federation.

However, there wasn’t already a CLI tool for spinning up a lightweight SPARQL endpoint based on SSSOM, so I contributed one to sssom-py, the first-party SSSOM Python package, in mapping-commons/sssom-py#619. It works by combining the built-in functionality to create an in-memory RDF cache using RDFlib with Vincent Emonet’s package rdflib-endpoint, which serves a web application based on FastAPI that has a SPARQL endpoint for an RDFlib graph and a user interface via YASGUI.

Here’s a one-liner to deploy the mappings from the Biomappings project, including the Iconclass mappings described earlier:

$ uv pip install sssom
$ sssom serve-rdf https://w3id.org/biopragmatics/biomappings/sssom/biomappings.sssom.tsv

Here’s an example query that finds mappings using Iconclasses:

PREFIX skos: 
PREFIX owl: 
PREFIX sssom: 

SELECT ?iconclass ?o ?justification
WHERE {
    ?iconclass skos:relatedMatch ?o .
    FILTER STARTSWITH(?iconclass, "https://iconclass.org/")
}

It can also be expanded to use reifed information, like the mapping justification:

PREFIX skos: 
PREFIX owl: 
PREFIX sssom: 

SELECT ?iconclass ?o ?justification
WHERE {
    [] a owl:Axiom ;
        owl:annotatedSource ?iconclass ;
        owl:annotatedProperty skos:relatedMatch ;
        owl:annotatedTarget ?o ;
        sssom:mapping_justification ?justification .

    FILTER STARTSWITH(?iconclass, "https://iconclass.org/")
}

These were some big interludes! Now would be a great time for making a joke about either Brandon Sanderson books or the official Wind Waker strategy guide that said there’s actually a lot of stuff to do before going to fight Ganondorf underwater. But, let’s push forward.

Part 4: Asking Multidisciplinary Questions

Finally, you’ve arrived at the big conclusion. What can do by bridging the chemistry knowledge graph and culture knowledge graph? We can connect datasets in Chemotion electronic laboratory notebooks (ELNs) that annotate the instrument used for generation with depictions of those instruments in cultural heritage objects like paintings.

Is this the most practical query? No. But it’s cool, and more importantly, it demonstrates a few things:

Even a mundane question might require federating many different resources
NFDI has successfully built many detailed resources that support these questions
We’re almost at the point where these questions can be operationalized
Where the NFDI’s semantic stack has gaps, I am ready to contribute my own technologies. I’m very excited to have recently joined NFDI during summer 2025, and I’m looking forward to helping shape its semantics roadmap

Here’s an architectural diagram of the resources that are required for making the query I described, which highlights the boundaries between resources and the relationships that connect across them.

graph LR
    subgraph nfdi4culture [NFDI4Culture]
        painting[Painting] -- annotated with --> iconclass[Iconclass]
    end

    subgraph biomappings [Biomappings]
        iconclass -. federation -.- iconclass2[Iconclass]
        iconclass2 -- depicts (Biomappings) --> instrumentClass2[Class of Instruments]
    end

    subgraph chmo [CHMO Ontology]
        instrumentClass[Class of Instruments] -. federation -.- instrumentClass2
        instrument[Instrument] -- participates in --> measurement[Measurement]
        instrument -- is a --> instrumentClass
    end

    subgraph nfdi4chem [NFDI4Chem]
        measurement-. federation -.- measurement2[Measurement]
        measurement2[Measurement] -- appears in experiment --> experiment[Experiment]
    end

Finally, to tie up the whole post, here’s what I imagine the SPARQL that would encode this query would look like. I say this in the conjunctive because this SPARQL won’t actually run - it relies on the existence of two services that don’t yet exist:

An endpoint that serves semantic mappings, like I described above
An endpoint that serves ontologies.

I got creative here, and imagined that we might have a Base4NFDI service for mappings in the future. We could also extend the Ubergraph to include CHMO to make this query work, or potentially incorporate an extension of this service into the Terminologies for NFDI (TS4NFDI).

Without further ado, here’s the query. There are notes inside it explaining what it’s doing.

PREFIX prov: 
PREFIX rdf: 
PREFIX owl: 
PREFIX BFO: 
PREFIX skos: 
PREFIX cto: 
PREFIX schema: 
PREFIX iconclass: 
PREFIX rdfs: 
PREFIX CHMO: 

SELECT ?dataset ?experiment ?instrument ?painting ?paintingLabel ?paintingURL
WHERE {
    # This subquery connects datasets, experiments, and the
    # measurement processes using the NFDI4Chem knowledge graph
    SERVICE  {
        ?dataset prov:wasGeneratedBy/prov:used ?experiment .
        ?experiment prov:wasGeneratedBy/rdf:type ?measurmentProcess .
    }

    # This subquery connects measurement processes to the instruments
    # that participate (BFO:0000057) in them. It fictionally uses
    # Ubergraph as a service, but this won't work because it doesn't
    # yet incorporate CHMO
    SERVICE  {
        # this could be extended further to do variable
        # hierchical traversal of measurement processes'
        # and instruments' subclassess, but skipped for
        # brevity
        ?measurmentProcess rdfs:subClassOf [
            rdf:type owl:Restriction;
            owl:onProperty BFO:0000057 ;
            owl:someValuesFrom ?instrument
        ] .
    }

    # This subquery uses SSSOM from Biomappings to look up relationships
    # between iconclasses and instruments. It fictionally imagines a
    # Base4NFDI service that serves semantic mappings via SPARQL.
    SERVICE  {
        ?iconclass skos:relatedMatch ?instrument .
    }

    # Missing: variable-length hierarchical traversal of the
    # iconclasses

    # The final subquery connects iconclasses to paintings
    # and their corresponding URLs for depiction via the
    # NFDI4Culture knowledge graph
    SERVICE  {
        ?painting cto:subjectConcept ?iconclass ;
                  schema:image ?paintingURL .
        FILTER STRSTARTS(STR(?iconclass), STR(iconclass:))
        OPTIONAL { ?painting rdfs:label ?paintingLabel }
    }
}

Like many of my favorite blog posts, this one was also something of a misadventure. After hearing Thorsten’s talk, I whispered to Philip, who was sitting next to me, something like, neat! I bet we could code that up tonight in an evening hacking session.

We didn’t quite get around to that because the consortium meeting was also a great opportunity to socialize with our excellent colleagues who are typically scattered throughout Germany. I did give it a shot after the nightly activities were over and made a screencast of the steps in preparing Iconclass for PyOBO. Maybe you’ll find it interesting watching me code - it’s a relatively quiet video with no narration except the occasional expletive (you’ve been warned). I got as far as the first attempt at generating semantic mappings from Iconclass to CHMO, OBI, and ChEBI.

As you might have guessed from reading, it actually ended up taking several weeks to make improvements to different pieces of software, to curate new data, and to write it all up. And, it’s not even working yet. So it goes!

Double post-script, wow! If you’re in the NFDI sphere and have ideas for other cross-disciplinary queries, please let me know. I am excited to extend this demonstration, especially to something more practical.

Representing Negative Knowledge

2025-10-07T21:18:00+00:00

Representing negative knowledge in the semantic web is an open problem. This post is going to be a living document where I keep notes on use cases, potential solutions, and awful hacks.

SSSOM

In the Simple Standard for Sharing Ontological Mappings (SSSOM), we decided to represent negative knowledge by adding a modifier onto a predicate. This means that a semantic mapping known to be false gets represented like this in TSV:

subject_id	subject-label	predicate_id	predicate_modifier	object_id	object_label	mapping_justification
CHEBI:8925	ryanodine	skos:exactMatch	Not	mesh:D031309	Ryania	semapv:ManualMappingCuration

Since SSSOM is heavily tied to the OBO community, this gets turned into RDF with the following axiom:

[] a owl:Axiom ;
    owl:annotatedSubject CHEBI:8925 ;
    owl:annotatedProperty skos:exactMatch ;
    owl:annotatedObject mesh:D031309 ;
    sssom:predicate_modifier sssom:NegatedPredicate .

It’s debated whether asserting an axiom also requires the assertion of the triple itself CHEBI:8925 skos:exactMatch mesh:D031309, however, in this case, it’s clear that triples represent true knowledge, so we would not want to concretize it.

RDF Surfaces

Damien Goutte-Gattat made me aware of RDF Surfaces in a discussion on a SSSOM PR. To summarize his comment, RDF surfaces create a part of an RDF graph that contains negated assertions, something like:

@prefix log:  .
@prefix FBbt:  .
@prefix UBERON:  .
@prefix skos: http://www.w3.org/2004/02/skos/core#> .

(_:x) log:onNegativeSurface {
  FBbt:00004508 skos:exactMatch UBERON:0000056 .
} .

This paper is still under review and also might be of limited use because it uses sneaky N3 syntax { } which are used to express formulae.

I haven’t actually gone out to do a deep survey on this. If you are aware of something relevant, please let me know.

Suggesting new relations in ROR from Wikidata

2025-09-25T15:35:00+00:00

I was looking at the different NFDI consortia in the Research Organization Registry (ROR), and found that the only two that have a parent relations to the NFDI (ror:05qj6w324) are NFDI4DS (ror:00bb4nn95) and MaRDI (ror:04ncnzm65). This felt strange to me, so I started looking around Wikidata to see if I could automatically make a curation sheet to send along to them. I found that Wikidata already has detailed pages for all NFDI consortia, and that they also include relationships to the parent. This blog post is about the steps I took to write a workflow to find relationships in Wikidata that are appropriate for submission to ROR.

Getting Wikidata

In Wikidata, an entity can be annotated with a ROR identifier via property P6782. I wanted to write a SPARQL query for the to retrieve all triples for which both the subject and object have and ROR identifier.

SELECT ?subject ?subjectROR ?subjectLabel ?predicate ?object ?objectROR ?objectLabel
{
  ?subject ?predicate ?object ;
           wdt:P6782 ?subjectROR .
  ?object wdt:P6782 ?objectROR .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],mul,en". }
}

In the first version of this post, I ran the SPARQL queries against first-party Wikidata Query Service (WQS) (permalink to this query at https://w.wiki/FUVA) and ran into timeout issues. This motivated the following section of this post on the exploration of the underlying triples and alternate query formulations below.

However, both Egon (here) and Tiago (here) suggested I also try the University of Freiburg’s QLever mirror of Wikidata available at https://qlever.cs.uni-freiburg.de/wikidata (permalink to this query at https://qlever.cs.uni-freiburg.de/wikidata/Vz75fY).

QLever is blazingly fast and returns about 67K rows in only a few seconds. That all being said, I retained my exploration below for posterity.

Working around timeouts on the WQS

The next step in any investigation with a blasphemous ?subject ?predicate ?object pattern is to look into the predicates and try to cut them down. I set to reformulating the query to count the frequency of appearance of each predicate.

SELECT DISTINCT ?p ?pLabel (COUNT(?p) as ?count)
{
  ?subject wdt:P6782 ?subjectROR;
           ?predicate ?object .
  ?object wdt:P6782 ?objectROR .
  ?p wikibase:directClaim ?predicate .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],mul,en". }
}
GROUP BY ?p ?pLabel
ORDER BY DESC(?count)

This query uses the sneaky wikibase:directClaim to map between the wd: entity namespace and wdt: direct property namespace so the query service could look up the label for the link. The problem was, this query was still too heavy and caused a timeout. Therefore, I had to simplify the query to just get the counts without the label, then use a second query and join the data externally (I also tried a nested query along the way, but it still timed out).

SELECT DISTINCT ?predicate (COUNT(?predicate) as ?count)
{
  ?subject wdt:P6782 ?subjectROR ;
           ?predicate ?object .
  ?object wdt:P6782 ?objectROR .
}
GROUP BY ?predicate
ORDER BY DESC(?count)

With that out of the way, I tried re-writing the original query by formatting in the 147 predicates I pulled out into the VALUES ?predicate { ... } (abbreviated), like:

SELECT ?subject ?subjectROR ?subjectLabel ?predicate ?object ?objectROR ?objectLabel
{
  VALUES ?predicate { ... }
  ?subject ?predicate ?object ;
           wdt:P6782 ?subjectROR .
  ?object wdt:P6782 ?objectROR .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],mul,en". }
}

This still caused timeouts, so I resorted to a loop in Python, which also let me simplify the query to skip the Wikidata IDs and just pull out RORs for the subject and object (where the {...} gets replaced with a different property on each):

SELECT ?subjectROR ?objectROR
WHERE {
    ?subjectROR ^wdt:P6782/wdt:{...}/wdt:P6782 ?objectROR .
}

I really like this because it uses paths to reduce the need to specify the middle entities which don’t get used. I don’t know if the SPARQL engine is able to optimize on it, but it’s cool. Maybe not so readable, but cool. The loop created a super-sized TSV with the predicate and labels added back.

The workflow I implemented for this lives in https://github.com/cthoyt/ror-wikidata-enrichment. The data from Wikidata is in this file, licensed under CC0.

Do you want this workflow to better reflect your organization? Check out my other blog post on how to curate data about your research organization

Getting ROR

I’ve previously implemented a source in PyOBO that wraps downloading and structuring ROR’s data dump into a readily usable format, so getting ROR’s triples was as easy as:

import pyobo

df = pyobo.get_relations_df("ror")

I also had to map the part of and has part relations from BFO to Wikidata properties. I did this by hand because it was faster than doing it the sustainable way, which would have been to pull the mappings from SSSOM-like annotations in the BFO ontology or from Wikidata itself (since I curated those into Wikidata years ago when we were preparing the (unpublished) relation ontology paper).

I made an intermediate output of all of thet triples here, licensed under CC0.

Putting it all together

While I’m glossing over a few steps that you can grok by reading my python script, it was possible to finish getting the data in the right shape to compare with tools in PyOBO and the Bioregistry

The final step was to take the difference between the Wikidata triples and the ROR triples, filter for triples that make sense within the ROR schema (which for now is just part of and has part relationships), and then dump the results out. There were around 67K records before filtering around 2.8K after filtering. Here are a few examples:

subjectROR	subjectLabel	predicate	predicateLabel	objectROR	objectLabel
00k4nrj32	Essex County Hospital	P361	part of	02wnqcb97	National Health Service
022efad20	University of Gabès	P527	has part(s)	01hwc7828	Institut des Régions Arides
04p4gjp18	Center of Excellence on Hazardous Substance Management	P361	part of	028wp3y58	Chulalongkorn University
04tnv7w23	École Supérieure Polytechnique d’Antsiranana	P361	part of	00pd4qq98	Université d’Antsiranana
02f4ya153	Barro Colorado Island	P361	part of	01pp8nd67	Smithsonian Institution

Coda

The point of all of this was to automate adding the missing NFDI consortia relationships to the parent NFDI organization in ROR, because I’m interested in creating queries over the organization landscape related to NFDI to support an upcoming section on Internationalization. And like most things in my work life, I ended up cleaning some data and making upstream contributions along the way. Let’s see how receptive ROR is to this! The triples are all here and I can easily make them a different format for submission.

Caveat: if you look into the data, you might notice that some of the entities don’t have labels. I realized this is happening because I haven’t updated my PyOBO importer to get the 2.0 data dump from ROR, and I’m stuck on old version 1.36. This can be fixed independently of this workflow. Here’s the rows related to the NFDI consortia that need new relations, which are all missing labels until I fix this.

subjectROR	predicate	predicateLabel	objectROR	objectLabel
00enhv193	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
02cxb1m07	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
03xrvbe74	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
020tty630	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
04ncnzm65	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
01f5dqg10	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
001jhv750	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
0310v3480	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
01d2qgg03	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
01k9z4a50	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
03a4sp974	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
05wwzbv21	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
0305k8y39	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
0238fds33	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
03f6sdf65	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
0033j3009	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
01vnkaz16	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
01v7r4v08	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
04dy2xw62	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
01xptp363	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
034pbpe12	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
05nfk7108	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
00r0qs524	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
00bb4nn95	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur
03fqpzb44	P361	part of	05qj6w324	Nationale Forschungsdateninfrastruktur

Switching from using Tox to Just

2025-09-21T09:21:00+00:00

I became aware of just while watching Hynek’s second video on uv a few months ago. I immediately fell in love with its elegance and simplicity, so I have begun replacing task running in my repositories that relied on tox with just. This post gives a bit of background, context, and walks through making the switch on one of my repositories that has some annoying dependencies.

What is Tox

tox is a tool tailored for developing Python packages. It takes care of creating a virtual environment for the configuration then installing the current package with the specified extras and dependency groups. Here’s what my testing configuration in the ssslm package looked like:

[testenv]
description = Run unit and integration tests.
commands =
    coverage run -p -m pytest --durations=20 {posargs:tests}
    coverage combine
    coverage xml
extras =
    gilda-slim
    web
    scispacy
    gliner
    rdflib
    ontology
    pandas
dependency_groups =
    tests
    en-core-sci-sm

This can be run with tox -e py after installing tox (e.g., uvx tox -e py or uvx --with tox-uv tox -e py to enable virtual environment creation with uv)

Tox can actually do a lot more things than this, including skipping installing the current repository as a Python package to instead install/run development tools like ruff. It also has the ability to refer from the parts of one configuration to another, which is useful for having different flavors of the same command (e.g., documentation build vs. documentation test).

Check the Tox documentation or see the tox.ini file in SSSLM v0.1.2, the last version before I switched from Tox to Just: https://github.com/cthoyt/ssslm/blob/v0.1.2/tox.ini.

What is Just

just is a more generic tool for writing tasks, which is a much better fit than Makefiles - read the Just homepage for excellent arguments which I won’t recapitulate here. Here’s what the previous Tox configuration looks using Just’s custom syntax in a file called justfile, this time using uv to do the heavy lifting:

[doc("run unit and integration tests")]
test:
    just coverage erase
    uv run --group tests --all-extras --no-extra scispacy --no-extra gilda -m coverage run -p -m pytest
    just coverage combine
    just coverage xml

[doc("run `coverage` with a given subcommand")]
@coverage command:
    uvx --from coverage[toml] coverage {{command}}

This can be run with just test after installing just (e.g., uvx --from rust-just just test).

Note that uv is now doing the heavy lifting for environment management and installation instead of Tox, and Just simply ties it all together.

Making the Switch

I finally got around to testing replacing running my tests with tox with running my tests with just in cthoyt/ssslm#32. I did this because:

tox is feeling very slow these days. The tox-uv extension was a nice respite, but it only uses uv for virtual environment construction (iirc) and doesn’t take advantage of the other fast parts of uv.
tox creates a wastefully large cache in every project
uvx is a more fit-for-purpose solution for many of the things I use tox for, i.e., installing and running a tool in an isolated virtual environment

I had to start with two other PRs to SSSLM (cthoyt/ssslm#31 and cthoyt/ssslm#33) that did some restructuring of tests and the way dependencies were declared such that it would even be possible to run tests without all extras/groups installed.

Then, I had to look into the issue caused by ScispaCy, which is running based on notoriously old dependencies (only works on Python <= 3.12 and with NumPy < 2.0). FYI, I’m not just complaining, I’ve been making upstream PRs to their repository to help get to broader compatibility.

The biggest problem with old NumPy dependencies is they’re often either 1) not available as a wheel or 2) difficult to compile in an automated setting.

One solution is to use the tool.uv.conflicts configuration to say that the pandas and scispacy extras can’t be installed at the same time because their (transitive) dependencies have conflicts. Don’t confuse these names with the packages - the extras include a list of related things that are defined in the project.optional-dependencies block of my project configuration.

Ideally, I wanted my test configuration in my justfile to have two back-to-back calls to uv run like this:

$ uv run --group tests --all-extras --no-extra scispacy --no-extra gilda -m coverage run -p -m pytest
$ uv run --group tests --group en-core-sci-sm --extra scispacy --extra ontology -m coverage run -p -m pytest

The first command runs most tests, and the second one just installs the scispacy extras and runs that. However, this didn’t work (it didn’t seem to manage to install the scispacy extra). I think there is a solution for this, but I am still learning about the nuances of uv run and uv’s notion of locking.

I tabled getting ScispaCy tests for SSSLM to work for now, since this is not a generic concern of most packages. The next steps are to test replacing all Tox environments with corresponding just commands in SSSLM, then upstream this to my cookiecutter template https://github.com/cthoyt/cookiecutter-snekpack so all of my repositories can benefit.

What We Really Want

It would be great if uv had a built-in notion of task definitions, since most of the things I had in my tox configuration (and now justfile) are calls to install Python environments or run Python things inside them (this is true for testing, linting, documentation building, publishing, etc.).

There’s a long-standing issue on their tracker astral-sh/uv#5903 that I’m sure will be addressed in the future, when they’ve made an excellent design for the developer experience. I’m looking forward to the future where I can write a follow-up post entitled Switching from just + uv to just uv (wordplay intended).

Exploring an unfamiliar SPARQL endpoint with the Bioregistry - a case study from NFDI4Culture

2025-09-11T10:26:00+00:00

Earlier this week at the sixth NFDI4Chem consortium meeting, Torsten Schrade from the NFDI4Culture consortium gave a lovely and whimsical talk entitled A Data Alchemist’s Journey through NFDI which explored ways that we might federate and jointly query both consortia’s knowledge via their respective SPARQL endpoints. This post is about the very first steps I took when looking into this new (to me) SPARQL endpoint, namely to identify what prefixes and semantic spaces are present, then added a new CLI tool to the Bioregistry to do this reproducibly.

The NFDI4Culture’s SPARQL endpoint https://nfdi4culture.de/sparql is wrapped by a nice user interface here for interactive querying in the browser.

It has two example queries to list all research data repositories and to list all research data portals, but otherwise I’m a bit stuck to better understand its schema.

Checking the Prefix Map

However, the NFDI4Culture SPARQL endpoint based on Virtuoso, so there is a way to look at what are the default CURIE prefixes and URI prefixes by navigating to here.

In my previous post, I demonstrated generalizing the notion of prefix map validation to incorporate prefix maps from either JSON-LD or in the beginning of turtle files.

I extended this even further to extract the prefix map from the Virtuoso SPARQL endpoint page in biopragmatics/bioregistry#1691. Note that this doesn’t work for all triple stores, it just works on Virtuoso because of the way that it provides a special page for showing the prefix map. I wasn’t able to find a way to get it directly, so the implementation does HTML scraping and parsing.

Here’s how you can use the validator I wrote:

$ bioregistry validate virtuoso https://nfdi4culture.de/sparql --tablefmt github

prefix	uri_prefix	issue	solution
as	https://www.w3.org/ns/activitystreams#	unknown CURIE prefix	Switch to CURIE prefix ac, inferred from URI prefix
bif	http://www.openlinksw.com/schemas/bif#	unknown CURIE prefix
dawgt	http://www.w3.org/2001/sw/DataAccess/tests/test-dawg#	unknown CURIE prefix
dbpprop	http://dbpedia.org/property/	unknown CURIE prefix	Switch to CURIE prefix dbpedia.property, inferred from URI prefix
fn	http://www.w3.org/2005/xpath-functions/#	unknown CURIE prefix
formats	http://www.w3.org/ns/formats/	unknown CURIE prefix
gqi	http://www.openlinksw.com/schemas/graphql/intro#	unknown CURIE prefix
gql	http://www.openlinksw.com/schemas/graphql#	unknown CURIE prefix
gr	http://purl.org/goodrelations/v1#	unknown CURIE prefix
ldp	http://www.w3.org/ns/ldp#	unknown CURIE prefix
math	http://www.w3.org/2000/10/swap/math#	unknown CURIE prefix
nci	http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#	non-standard CURIE prefix	Switch to standard prefix: ncit
ogc	http://www.opengis.net/	unknown CURIE prefix
ogcgml	http://www.opengis.net/ont/gml#	unknown CURIE prefix
ogcgs	http://www.opengis.net/ont/geosparql#	unknown CURIE prefix
ogcgsf	http://www.opengis.net/def/function/geosparql/	unknown CURIE prefix
ogcgsr	http://www.opengis.net/def/rule/geosparql/	unknown CURIE prefix
ogcsf	http://www.opengis.net/ont/sf#	unknown CURIE prefix
product	http://www.buy.com/rss/module/productV2/	unknown CURIE prefix
protseq	http://purl.org/science/protein/bysequence/	unknown CURIE prefix
rdfdf	http://www.openlinksw.com/virtrdf-data-formats#	unknown CURIE prefix
sc	http://purl.org/science/owl/sciencecommons/	unknown CURIE prefix
scovo	http://purl.org/NET/scovo#	unknown CURIE prefix
sd	http://www.w3.org/ns/sparql-service-description#	unknown CURIE prefix
sioc	http://rdfs.org/sioc/ns#	unknown CURIE prefix	Switch to CURIE prefix sioc.core, inferred from URI prefix
sql	http://www.openlinksw.com/schemas/sql#	unknown CURIE prefix
stat	http://www.w3.org/ns/posix/stat#	unknown CURIE prefix
vcard2006	http://www.w3.org/2006/vcard/ns#	unknown CURIE prefix	Switch to CURIE prefix vcard, inferred from URI prefix
virtcxml	http://www.openlinksw.com/schemas/virtcxml#	unknown CURIE prefix
virtrdf	http://www.openlinksw.com/schemas/virtrdf#	unknown CURIE prefix
xf	http://www.w3.org/2004/07/xpath-functions	unknown CURIE prefix
xsl10	http://www.w3.org/XSL/Transform/1.0	unknown CURIE prefix
xsl1999	http://www.w3.org/1999/XSL/Transform	unknown CURIE prefix
xslwd	http://www.w3.org/TR/WD-xsl	unknown CURIE prefix
yago	http://dbpedia.org/class/yago/	unknown CURIE prefix

Interpreting the results

Some of the key takeaways from this table are:

The feedback on as is a false positive - a look at https://www.w3.org/ns/activitystreams# shows that the W3 standard wants as to be the preferred prefix
There are several true positive suggestions, like fixing the ncit prefix.
There’s a whole group of URI spaces using opengis.net from the Open Geospatial Consortium, dealing with geospatial data that could be registered in the Bioregistry
There are a large number URI spaces from openlinksw.com, which correspond to Virtuoso itself. These might be good to put in the Bioregistry, but are all very short, which means that there is high potential for overlap. However, there aren’t any reported conflicts
There are several w3.org standard prefixes that should be registered in the Bioregistry.
There are several prefixes that I’m not familiar with that also use short acronyms, which makes me a bit hesitant to add in the Bioregistry. In these cases, I usually either don’t add them because of lack of wide-spread use or add them using a less generic prefix.

Something I still want to implement is a generic workflow for identifying putative URI spaces in a given remote SPARQL endpoint. If there are only a small number of triples that can be exhaustively queried, then this workflow can be used. Otherwise, I am still considering different options.

Validating the FAIRness of knowledge graphs and ontologies in RDF using the Bioregistry

2025-09-04T12:30:00+00:00

Using standard CURIE prefixes and URI prefixes in semantic web artifacts such as Resource Description Framework (RDF) promotes interoperability, enables reuse in downstream data integration, and makes data more FAIR. The Bioregistry defines a set of standard CURIE prefixes and URI prefixes against which RDF files can be validated/standardized. This blog post describes a new CLI tool bioregistry validate ttl in the Bioregistry Python package that can run validation on Turtle files (a common serialization of RDF).

RDF data stored in Turtle files typically begins with a stanza defining a prefix map. For example, one of the Turtle files in the Chemotion Knowledge Graph (Chemotion-KG) begins with the following six prefixes:

@prefix nfdicore:  .
@prefix ns1:  .
@prefix ns2:  .
@prefix obo:  .
@prefix rdfs:  .
@prefix xsd:  .

The following command can be used to validate it. Using --tablefmt github results in a nice table that can be formatted into a blog post, otherwise it outputs text in a more vertical format.

$ bioregistry validate ttl https://github.com/ISE-FIZKarlsruhe/chemotion-kg/raw/4cb5c24af/processing/output_bfo_compliant.ttl

prefix	uri_prefix	issue	solution
nfdicore	https://nfdi.fiz-karlsruhe.de/ontology/	non-standard CURIE prefix	Switch to standard prefix: nfdi.core
ns1	http://purls.helmholtz-metadaten.de/mwo/	unknown CURIE prefix	Consider switching to the more specific CURIE/URI prefix pair mwo: `http://purls.helmholtz-metadaten.de/mwo/mwo_`
ns2	http://purl.obolibrary.org/obo/chebi/	unknown CURIE prefix

I was able to directly open an issue on the GitHub repository to give feedback. In general, I think this is a very powerful use of the Bioregistry because it can support groups interested in making knowledge graphs and ontologies towards improving their data and ultimately making it more FAIR.

In case you’re interested in how I implemented this, check this PR. I was able to reuse some ideas from a previous JSON-LD validator, extend them, and abstract the code. Later, I will be able to implement similar validators for XML files, ontologies, and any other resource from which I can extract a prefix map.

I also left a TODO inside the code, since this can be extended with several other ways of validating URI prefixes. Ultimately, this may get upstreamed into the curies package to make it even more accessible to groups making their own prefix maps or using custom instances of the Bioregistry.

A historical analysis of ChEMBL

2025-08-26T15:44:00+00:00

I’ve recently submitted an article to the Journal of Open Source Software (JOSS) describing chembl-downloader, a Python package for automating downloading and using ChEMBL data in a reproducible way. In this post, I use chembl-downloader to show how the number of compounds, assays, activities, and other entities in ChEMBL have changed over time.

ChEMBL has made 37 releases so far. 35 of them have been major releases, and two have been minor releases (v22.1 and v24.1). While it only began bundling a SQLite dump of the database v19, Eloy Felix recently informed me that the team constructed SQLite dumps for all previous versions, too. This is great, because I use the SQLite dump as the primary mechanism for querying the database.

chembl-downloader automates downloading any version of ChEMBL’s SQLite database, unpacking it from the TAR archive, connecting to it, making a SQL query, and returning the results as a Pandas DataFrame object with chembl_downloader.query() or chembl_downloader.query_scalar() like in:

import chembl_downloader

sql = "SELECT * FROM activities LIMIT 5"
rows = chembl_downloader.query(sql, version=1)

sql = "SELECT COUNT(*) FROM activities"
count: int = chembl_downloader.query_scalar(sql, version=1)

These functions can be used to write a loop and run the same SQL query over every version of ChEMBL with a few two caveats:

to get all versions of ChEMBL, it needs to include v22.1 and v24.1. chembl.versions() provides convenient access to construct this iterator
SQLite does not allow for opening a compressed database (without paying for an extension), so each version needs to be uncompressed. Unfortunately, most personal computers (including mine) don’t enough hard disk space to have an uncompressed copy of each version of ChEMBL

Results of Temporal Analysis

I wrote a CLI utility chembl_downloader history which downloads, decompresses, analyzes, and then deletes each version of ChEMBL iteratively over the span of about three hours.

It summarizes the dates of release, number of compounds, number of named compounds, (i.e., with a pref_name), number of assays, and number of activities, and several other entity types to (almost) match what’s summarized on the ChEMBL homepage. The results can be downloaded as a TSV and are as follows:

Version	Date	Compounds	Named Compounds	Assays	Activities	Documents	Targets	Cells	Tissues	Drug Warnings	Drug Indications	Drug Mechanisms
35	2024-12-01	2,496,335	42,231	1,740,546	21,123,501	92,121	16,003	2,129	782	1,676	55,442	7,330
34	2024-03-28	2,431,025	42,387	1,644,390	20,772,701	89,892	15,598	2,023	782	1,676	55,442	7,330
33	2023-05-31	2,399,743	41,923	1,610,596	20,334,684	88,630	15,398	2,021	782	1,636	51,582	7,098
32	2023-01-26	2,354,965	41,923	1,536,903	20,038,828	86,361	15,139	2,015	759	1,636	51,582	7,098
31	2022-07-12	2,331,700	41,585	1,498,681	19,780,369	85,431	15,072	2,000	757	1,293	48,816	6,656
30	2022-02-22	2,157,379	41,549	1,458,215	19,286,751	84,092	14,855	1,991	752	1,293	48,816	6,656
29	2021-07-01	2,105,464	41,383	1,383,553	18,635,916	81,544	14,554	1,978	743	1,262	45,902	6,202
28	2021-01-15	2,086,898	41,049	1,358,549	17,276,334	80,480	14,347	1,950	739	1,256	42,988	5,347
27	2020-05-18	1,961,462	40,834	1,221,361	16,066,124	76,086	13,382	1,831	707	0	37,259	5,134
26	2020-02-14	1,950,765	40,822	1,221,311	15,996,368	76,076	13,377	1,830	707	0	37,259	5,070
25	2019-02-01	1,879,206	39,885	1,125,387	15,504,603	72,271	12,482	1,670	655	0	29,457	4,992
24.1	2018-05-01	1,828,820	39,877	1,060,283	15,207,914	69,861	12,091	1,667	655	0	29,163	4,992
24	2018-05-01	1,828,820	39,877	1,060,283	15,207,914	69,861	12,091	1,667	655	0	29,163	4,992
23	2017-05-18	1,735,442	39,584	1,302,147	14,675,320	67,722	11,538	1,624	125	0	13,504	4,305
22.1	2016-11-17	1,686,695	39,422	1,246,683	14,371,197	65,213	11,224	1,619	111	0	12,573	3,834
22	2016-09-28	1,686,695	39,422	1,246,132	14,371,219	65,213	11,224	1,619	111	0	12,573	3,834
21	2015-02-12	1,592,191	39,347	1,212,831	13,968,617	62,502	11,019	1,612	0	0	5,951	3,799
20	2015-02-03	1,463,270	39,016	1,148,942	13,520,737	59,610	10,774	1,647	0	0	0	2,266
19	2014-07-23	1,411,786	38,910	1,106,285	12,843,338	57,156	10,579	1,653	0	0	0	2,239
18	2014-04-02	1,359,508	35,817	1,042,374	12,419,715	53,298	9,414	1,655	0	0	0	2,233
17	2013-09-16	1,324,941	32,692	734,201	12,077,491	51,277	9,356	1,746	0	0	0	2,213
16	2013-05-15	1,295,510	23,532	712,836	11,420,351	50,095	9,844	1,432	0	0	0	0
15	2013-01-30	1,254,575	23,528	679,259	10,509,572	48,735	9,570	1,432	0	0	0	0
14	2012-07-18	1,213,242	16,573	644,734	10,129,256	46,133	9,003	0	0	0	0	0
13	2012-02-29	1,143,682	16,397	617,681	6,933,068	44,682	8,845	0	0	0	0	0
12	2011-11-30	1,077,189	16,658	596,122	5,654,847	43,418	8,703	0	0	0	0	0
11	2011-06-07	1,060,258	16,264	582,982	5,479,146	42,516	8,603	0	0	0	0	0
10	2011-06-07	1,000,468	16,159	534,391	4,668,202	40,624	8,372	0	0	0	0	0
9	2011-01-04	658,075	3,746	499,867	3,030,317	39,094	8,091	0	0	0	0	0
8	2010-11-05	636,269	0	488,898	2,973,034	38,462	8,088	0	0	0	0	0
7	2010-09-03	602,500	0	485,095	2,948,069	38,204	8,078	0	0	0	0	0
6	2010-09-03	600,625	0	481,752	2,925,588	38,029	8,054	0	0	0	0	0
5	2010-06-07	578,715	0	459,823	2,787,240	36,624	7,493	0	0	0	0	0
4	2010-05-26	565,245	0	446,645	2,705,136	35,821	7,330	0	0	0	0	0
3	2010-04-30	547,133	0	432,022	2,490,742	34,982	7,330	0	0	0	0	0
2	2009-12-07	517,261	0	416,284	2,404,622	33,956	7,192	0	0	0	0	0
1	2009-10-28	440,055	0	329,250	1,936,969	26,299	5,694	0	0	0	0	0

The same results can be viewed as charts:

These charts show when certain features were introduced, such as cells in v15, drug indications in v20, tissues in v22, and drug warnings in v28.

The number of named compounds seems to have plateaued in v19 in 2014. This is strange, considering that ChEMBL links to many external resources like ChEBI that have nice preferred names that be imported. However, much like I found in my recent post about the EFO identifier column in ChEMBL’s diseases table, the pref_name column in the compounds table might not actually mean what I guess it does.

Change over Time

In order to investigate the changes over time, I also took the discrete derivative of each:

There are a few interesting places where the numbers dropped, such as the number of targets in v17 and the number of assays in v24 (which might have been a mistake that triggered the v24.1 release). I’m sure there’s a bit of explanation in the READMEs for these releases - please comment at the end of the post if you happen to take a look and have more explanation.

Overall, this analysis shows that the amount of content added between ChEMBL versions is relatively consistent (though keep in mind it’s on a log axis). The time for each release is also only slightly increasing on average.

Future Ideas

I would love to extend the idea of a temporal analysis towards other target-centric metrics like:

Are there examples of targets where the chemical space gets a lot bigger?
Conversely, are there targets where new compounds just seem to be in the same old neighborhood?
Are there widely conflicting activities added over time?
How does the ability of a QSAR model trained on a given version of ChEMBL perform with respect to the data that’s added later?

I presented one such example in the chembl-downloader manuscript where I re-ran one of Pat Walter’s analyses on 5-lipoxygenase activating protein (CHEMBL4550) in this notebook. There, the number of activities increased by more than double since the original analysis, but the distribution was roughly the same.

If you’re interested in teaming up to do a retrospective analysis on your favorite target (or, maybe even using knowledge graphs for interesting aggregations of targets based on gene sets, disease associations, etc.), then let me know.

Measuring the impact of the Bioregistry

2025-08-22T14:54:00+00:00

The Bioregistry is a database and toolchain for standardization of prefixes, CURIEs, and URIs that appear in linked (open) data. While I created it in 2019 as a component of PyOBO in order to support parsing database cross-references appearing in biomedical ontologies, it has since become an independent project with a community-driven governance model and much broader applications. This post is a first attempt to quantify its usage and impact.

What are Usage and Impact?

For a foundational resource like the Bioregistry, there are two kinds of usage. First, direct usage encompasses when a workflow directly reuses the Bioregistry’s data, software, or web application. For example:

The SSSOM-py Python package for interacting with semantic mappings uses the Bioregistry for supplying a comprehensive default prefix map during parsing of SSSOM files.
The BridgeDb identifier mapping service uses the Bioregistry Python package to test its source metadata are properly standardized (see here)
The Protegé ontology editor uses the Bioregistry’s API to look up information about prefixes.
WikiPathways uses the Bioregistry’s resolution service to linkify compact URIs ( CURIEs).

Second, indirect usage encompasses any other data, software, web application, etc. that builds on direct usages. For example, any ontology that is edited using Protegé indirectly uses the Bioregistry, like the Disease Ontology (DO).

I consider usage to be a very good proxy for impact, especially when considering indirect usage. Allen Baron et al. recently published The DO-KB Knowledgebase: a 20-year journey developing the disease open science ecosystem, which made a quantification of DO’s cumulative impact over the last decades. One area of this study focused on literature citations and resulted in the development of a reusable, open-source software package for such analyses. I don’t think anyone would disagree that the DO has had high impact.

I think that because DO is built using tools that rely on the Bioregistry, it’s fair to claim that DO’s (recent) impact is partially due to the Bioregistry.

The Bioregistry is in the excellent position where there are a variety of direct and indirect usages, many of which are highly impactful, meaning that the Bioregistry can share (a bit) in their glory. Later in this post, I’ll give a more quantitative justification for that statement.

Why Quantify Usage and Impact?

I think some of the main reasons for quantifying a project’s usage and impact are to:

justify its continued maintenance and improvement
get credit for making something important
use previous usage as examples for increased adoption
get funding to continue maintaining and improving it

At the moment, I am preparing to apply for the Bioregistry to be recognized by the Global Biodata Coalition as a Global Core Biodata Resource (GCBR). I am also preparing an application on behalf of current affiliation, RWTH Aachen University, to join the German Network for Bioinformatics Infrastructure (deNBI), which will include offering the Bioregistry as key bioinformatics infrastructure. Both applications require quantitative evidence of the usage and impact of the proposed resource as part of their respective key performance indicators (KPIs).

The Bioregistry by the Numbers

There are two kinds of statistics I think are important to convey about the Bioregistry. The first is related to community involvement. I’ve carefully planned the governance structure of the project to be sustainable (based on the open data, open code, and open infrastructure (O3) guidelines). This was successful in no small part due the way that there is a very low barrier for entry for small, external contributions (i.e., a drive-by curation). Therefore, it makes sense to highlight the number of unique contributors there have been to the data/code of the project as well as the volume of contributions in the forms of issues, discussions, and pull requests:

Statistic	Count
Contributors	75
Open Issues	146
Closed Issues	471
Total Issues	617
Open pull requests	13
Closed pull requests	1,012
Total pull requests	1,025

The second kind of statistic that’s important describes the content of the resource itself.

Statistic	Count
Prefixes	2,024
Prefix Synonyms	547
External positive mappings	9,162
External negative mappings	157
Total external mappings	9,319
Collections	18
External Registries	33

Note that this table is from mid-August 2025. Most of these numbers increase over time. To give some further context to these statistics, here’s a chart that shows how the Bioregistry stacks up against other related resources. Keep in mind, the Bioregistry is also a meta-resource that incorporates their important parts, too. You can understand this chart by looking at the percentage in parentheses (like +146% over Identifiers.org) and think: wow, those numbers are much bigger than 0%, which means the Bioregistry is much more comprehensive!

Direct Usage

Who uses the Bioregistry Website?

The Bioregistry website exposes metadata about ontologies, databases, and other resources that mint identifiers. This enables biocurators, data stewards, librarians, and others interested in identifying appropriate ontologies for reuse in their resources, data management plans, etc. It also allows researchers who consume linked (open) data to find context about the kinds of prefixes, CURIEs, or URIs that appear.

The following chart breaks down the nearly 4,400 unique users of the Bioregistry’s website by country. It shows that the Bioregistry is having a global impact, with potential for further growth in Africa (which I hope to do by getting in touch with the Africa PID Alliance).

API and Resolver Usage

Many usages of the Bioregistry are in the form of the resolution of links like https://bioregistry.io/chembl:CHEMBL4303805. Under the hood, the Bioregistry can expand prefixes like chembl:CHEMBL4303805 to URLs. This makes it a perfect service to support other applications that reference entities and want to provide external links, without having to maintain the links themselves.

I’m just getting this working, so it should also be able to better keep track of unique users (note it’s just showing 1 so far) and their countries. Luckily, all of this is GDPR-compliant from the beginning.

Code Usage

The Bioregistry distributes a Python package that can be installed with pip install bioregistry. I used this query to search GitHub for places where the bioregistry Python package is imported and took detailed notes about their context (see details here).

I found a large variety of usages across software packages, databases, ontologies, knowledge graphs, frameworks, and data models. Because the LinkML modeling language’s runtime indirectly depends on the Bioregistry, there are also dozens to hundreds of projects that indirectly use it. I also noted some usages from large organizations in the bioinformatics space, such as SciBite, Synapse, and the NFDI.

Indirect Usage

I used Wikidata as a backend to assess the indirect usage of the Bioregistry. I did this in a few steps:

Make sure all direct usages have a Wikidata item that has a relationship to the Bioregistry Wikidata item via the uses P1547 (depends on software) or P2283 (uses) predicates.
Identify well-known usages of direct usages, make sure they have Wikidata items, and are connected to them
Automate querying Wikidata for all direct and indirect usages
Search PubMed and quantify mentions of all direct and indirect usages

Indirect Software Dependencies

I’ve been developing quickstatements-client for automating adding content to Wikidata. I added an extension to it that pulls metadata from the Python Package Index (PyPI) and adds Python software packages. It does this recursively for a given package and its dependencies while adding appropriate P1547 (depends on software) relations between them.

Unfortunately, this workflow is still limited because it doesn’t find depedent software. This could be solved by doing a bulk download of PyPI and a large-scale network analysis. It might also be possible to extract this information from GitHub. However, for now, this is a good first step.

Indirect Ontology Dependencies via ODK

The Ontology Development Kit (ODK) uses the Bioregistry in several ways. It is used by many ontologies both in and out of the OBO Foundry. I developed a workflow for identifying ODK usage by searching GitHub and iteratively filtering out false positives. This resulted in over 140 repositories, more than half of which could directly be mapped back to the Bioregistry, given it tracks the repository associated with each prefix.

I have previously used quickstatements-client for automatically adding records for OBO Foundry ontologies. In next steps, I will extend this for the other ontologies identified by this analysis and also map them back to the ODK using the P1547 (depends on software) relation.

As a side note, for repositories using the ODK that couldn’t be mapped, I automated making stub curations in the Bioregistry, which lead to the curation of dozens of new prefixes.

Querying Wikidata

There’s a long tail of different ways to curate indirect usages of the Bioregistry. However, I believe that the largest cumulative impact for now will be though the lens of the ontologies built using it. After working through the scenarios above, I wrote a SPARQL query that recovers all direct and indirect dependencies (see live table below).

Literature Analysis

The final step was to analyze the literature for mentions of each of these direct and indirect dependencies. In typical Charlie fashion, I wanted to develop high-quality, modular, reusable software for doing this analysis, which lead to pubmed-downloader for wrapping API-based and bulk queries to PubMed (in addition to bulk downloading and processing, but that’s a story for a different blog post).

This is pretty great, it shows there are potentially thousands of papers that mention software, ontologies, databases, etc. that either directly or indirectly use the Bioregistry. It also shows, unsurprisingly, that there’s a power-law distribution in which is mentioned most in the literature.

The code for the literature analysis is available in its own repository. Of course, there is still lots of room for improvement and optimization, such as:

Making the image above nicer!
Using synonyms for search
Removing false positives
Incorporating citation networks

I’m interested to also make this work flow semi-automated to help assess the impact of other software/data resources, especially other key software that’s supporting the OBO Foundry like SSSOM-py.

Parting thoughts: it does seem like the Bioregistry is having a meaningful impact as a relatively young project. However, it’s still an uphill climb to get more direct recognition and adoption, which requires investing a lot of time in community building. This isn’t easy for me as an early career researcher not only because of lack of time/funding but also because of my relative lack of authority in the community of scientists who would benefit most from the Bioregistry (which comes with time).

I consider Chris Mungall and his group as the gold standard of being able to make large impact fast - they are known and trusted in the community, they have dedicated support staff that can focus on community management, they have many developers, and many projects where they can push their technologies. I’ll get there eventually!

The Bioregistry and BiomarkerKB

2025-08-22T08:32:00+00:00

The Bioregistry is a community-driven registry of semantic spaces and their metadata. When I learned about BiomarkerKB at the International Society for Biocuration’s 18th Annual International Biocuration Conference, I was excited to curate new records (and prefixes) in the Bioregistry to cover BiomarkerKB’s semantic spaces on biomarkers. This post summarizes the discussions I’ve had with its maintainers, Jeet and Raja, throughout the Bioregistry curation process and also gives insight into how databases can benefit from being represented in the Bioregistry.

How to Contribute to the Bioregistry

The Bioregistry follows the open data, open code, open infrastructure (O3) guidelines, meaning that anyone can make suggestions or additions either by creating an issue or a pull request on its GitHub repository. Both maintainers of ontologies, controlled vocabularies, databases, etc. and proactive community members can request new prefixes (i.e., add new records that represent a semantic space) by following the project’s contribution guidelines (TL;DR, watch this tutorial).

Contributing New Prefixes for BiomarkerKB

Most importantly, I coordinated with its maintainers Raja and Jeet to make a pull request to the Bioregistry’s GitHub repository and add two new prefixes. I was also able to explain the relationship between the records in the Bioregistry and their resource. I recorded part of our discussion that is generally useful for anyone who’s making a resource, and want to think about some of the benefits:

As a follow-up, Jeet and Raja requested a summary of this discussion that they could incorporate into BiomarkerKB’s FAQ. Below is a draft of that text, which could be easily adapted for any resource.

Text for BiomarkerKB’s FAQ

The Bioregistry is a community-driven registry of semantic spaces and their metadata. It contains records for three semantic spaces created for BiomarkerKB:

obci for the Ontology for Biomarkers of Clinical Interest
biomarkerkb.biomarker for condition-specific biomarkers
biomarkerkb.canonical for condition-agnostic biomarkers

The Bioregistry enables BiomarkerKB to unambiguously communicate how its entities should be written as URIs and compact URIs (CURIEs) for use in semantic web and linked (open) data settings. This supports many kinds of scientists, including:

biocurators, to identify BiomarkerKB’s semantic spaces as high quality resources for use in annotating their own data
data providers, to make their data and knowledge more FAIR before and during publication
data scientists, to validate biomarker data annotated with BiomarkerKB semantic spaces against an actionable
data stewards, to reference the Bioregistry records as components of research data management plans (DMPs)
web developers, to resolve identifiers from BiomarkerKB’s semantic spaces to human-readable web pages

If you’re a database maintainer and would like some specific attention given to representing your resource in the Bioregistry, or explainaning how the Bioregistry could be used for your project, please reach out! My contact information is at the bottom of my website.

Text-based embeddings of ontology terms

2025-08-04T08:32:00+00:00

The Ontology Lookup Service (OLS) is now indexing dense embeddings for ontology terms constructed from term labels, synonyms, and descriptions using LLMs. I maintain a Python client library for the OLS (ols-client) and was recently asked to implement a wrapper to the OLS’s API endpoint that exposes these embeddings. This post is a demo of how to use that code, and how I replicated the same embedding functionality with PyOBO to arbitrarily extend it to ontologies and databases not in OLS.

I’ve been working on modeling clinical trials in collaboration with Sebastian Duesing at the Ontology for Biomedical Investigations (OBI), so I’m going to use the OBI term for clinical trial (OBI:0003699) that we recently minted together as an example.

Embeddings from OLS

Below is a screenshot of the OLS page for this term, as of August 4^th, 2025:

There’s a newly added section on similar terms. Here’s what it says about these:

Similarity results are derived from LLM embeddings and have not been manually curated. Model: text-embedding-3-small

From a first glance, the results here have very high precision. This shouldn’t be surprising, considering that labels, synonyms, and descriptions contain very high signal. Of course, there’s room for arguing about the nuance of ontological differences, but this is a largely unhelpful discussion in my experience when the goal is to do ontology merging and data integration. In my recent post where I described the landscape of clinical trial modeling in the OBO Foundry and related biomedical ontologies, I actually had already curated several of these semantic mappings by hand, which I’m hoping to add to OBI via SSSOM. Looking forward for the OLS, it would be great if there were a mini curation interface where these could be confirmed or rejected as exact mappings, and persist the resulting curations as SSSOM.

There was a request to expose the embeddings via my OLS client package, which I solved with only a few lines of code here. Now, you can get the embedding for clinical trial (or any other term, based on the ontology/IRI combination) as a list of floating point numbers:

from ols_client import EBIClient

client = EBIClient()
embedding: list[float] = client.get_embedding(
    "obi", "http://purl.obolibrary.org/obo/OBI_0003699"
)

The next step is to be able to calculate the (cosine) similarity between two terms, which can be done between the OBI term for clinical trial, and the National Cancer Institute Thesaurus (NCIT) term for clinical trial as a single floating point number:

from ols_client import EBIClient

client = EBIClient()
similarity: float = client.get_embedding_similarity(
    "obi",
    "http://purl.obolibrary.org/obo/OBI_0003699",
    "ncit",
    "http://purl.obolibrary.org/obo/NCIT_C71104",
)

Embeddings from PyOBO

I think using an OpenAI model is a bit overkill for two reasons. First, there are smaller, non-large language models like BERT that can get the same job done. Second, they’re free to download and can be run on commodity hardware, versus needing to pay OpenAI for access to their embeddings. Specifically, I’ve been looking at SBERT (Sentence-BERT), which is a variant of the BERT architecture that works better on sentences and can be easily used via the sentence-transformers Python package.

I’ve developed the pyobo package, which gives unified access to ontologies and databases that are ontology-like. It has functionality for getting the labels, synonyms, and descriptions for terms in both.

It wasn’t difficult to re-implement the same functionality as the OLS in PyOBO such that it can be run locally on a larger variety of resources. Here’s the same lookup for text embedding and similarity:

import pyobo

>>> pyobo.get_text_embedding("OBI:0003699")
[-5.68335280e-02  7.96175096e-03 -3.36112119e-02  2.34440481e-03 ... ]

>>> pyobo.get_text_embedding_similarity("OBI:0003699", "NCIT:C71104")
0.24702128767967224

This could be improved with the ability to do batch lookup, which is probably the way people would want to use this functionality. Even better, because of how ML is implemented on GPUs and related hardware, batching effectively comes for free, only limited by memory contraints.

Text embeddings aren’t the end of the story - I’ve been working for several years on applications fo graph machine learning in biomedical applications. A lot of the good methodological and software engineering ideas I’ve had/encountered have gone into the PyKEEN Python software package. One of those ideas that jointly leverages text and knowledge graph embeddings is NodePiece, the work of PyKEEN core developer Michael Galkin. A guide on using it with PyKEEN is here, but there’s a lot more research to present on this than fits in the postscript of a blog post or a software tutorial. Along with the last few years of my work on data integration, I haven’t yet published my magnum opus on biomedical knowledge graph construction and applications with graph machine learning.

Inference over Semantic Mappings with SeMRA

2025-04-28T13:17:00+00:00

Assembling and inferring missing semantic mappings is a timely problem in biomedical data and knowledge integration. I’ve been developing the Semantic Mapping Assembler and Reasoner (SeMRA) as a generic toolkit for this. In this blog post, I highlight its inference capabilities.

SeMRA implements the chaining and inference rules described in the SSSOM specification. The first rule is inversions:

from semra import Mapping, EXACT_MATCH, Reference
from semra.inference import infer_reversible

r1 = Reference(prefix="chebi", identifier="107635", name="2,3-diacetyloxybenzoic")
r2 = Reference(prefix="mesh", identifier="C011748", name="tosiben")

mapping = Mapping(s=r1, p=EXACT_MATCH, o=r2)

mappings = infer_reversible([mapping])

graph LR
    A[2,3-diacetyloxybenzoic
chebi:107635] -- skos:exactMatch --> B[tosiben
mesh:C011748]
    B -. "skos:exactMatch
(inferred)" .-> A

The second rule is about transitivity. This means some predicates apply over chains. SeMRA further implements configuration for two-length chains and could be extended to arbitrary chains.

from semra import Reference, Mapping, EXACT_MATCH
from semra.inference import infer_chains

r1 = Reference.from_curie("mesh:C406527", name="R 115866")
r2 = Reference.from_curie("chebi:101854", name="talarozole")
r3 = Reference.from_curie("chembl.compound:CHEMBL459505", name="TALAROZOLE")

m1 = Mapping(s=r1, p=EXACT_MATCH, o=r2)
m2 = Mapping(s=r2, p=EXACT_MATCH, o=r3)

mappings = infer_chains([m1, m2])

graph LR
    A[R 115866
mesh:C406527] -- skos:exactMatch --> B[talarozole
chebi:101854]
    B -- skos:exactMatch --> C[TALAROZOLE
chembl.compound:CHEMBL459505]
    A -. "skos:exactMatch
(inferred)" .-> C

The third rule is generalization, which means that a more strict predicate can be relaxed to a less specific predicate, like owl:equivalentTo to skos:exactMatch.

from semra import Reference, Mapping, EXACT_MATCH, EQUIVALENT_TO
from semra.inference import infer_generalizations

r1 = Reference.from_curie("chebi:101854", name="talarozole")
r2 = Reference.from_curie("chembl.compound:CHEMBL459505", name="TALAROZOLE")

m1 = Mapping(s=r1, p=EXACT_MATCH, o=r2)

mappings = infer_generalizations([m1])

graph LR
    A[talarozole
chebi:101854] -- owl:equivalentTo --> B[TALAROZOLE
chembl.compound:CHEMBL459505]
    A -. "skos:exactMatch
(inferred)" .-> B

The third rule can actually be generalized to any kinds of mutation of one predicate to another, given some domain knowledge. For example, some resources curate oboInOwl:hasDbXref predicates when it’s implied that they mean skos:exactMatch because the resource is curated in the OBO flat file format.

from semra import Reference, Mapping, DB_XREF
from semra.inference import infer_dbxref_mutations

r1 = Reference.from_curie("doid:0050577", name="cranioectodermal dysplasia")
r2 = Reference.from_curie("mesh:C562966", name="Cranioectodermal Dysplasia")
m1 = Mapping(s=r1, p=DB_XREF, o=r2)

# we're 99% confident doid-mesh dbxrefs actually are exact matches
mappings = infer_dbxref_mutations([m1], {("doid", "mesh"): 0.99})

graph LR
    A[cranioectodermal dysplasia
doid:0050577] -- oboInOwl:hasDbXref --> B[Cranioectodermal Dysplasia
mesh:C562966]
    A -. "skos:exactMatch
(inferred)" .-> B

There’s a lot more to say about semantic mappings - a good first place to look before getting into the guts of the code is the accompanying manuscript:

Assembly and reasoning over semantic mappings at scale for biomedical data integration
Hoyt, C. T., Karis K., and Gyori, B. M.
bioRxiv, 2025.04.16.649126

I wish I could unpack Callables in Python type annotations

2025-04-23T13:23:00+00:00

Following the theme of my previous two posts, I’ve run into another typing conundrum where I want to unpack a pre-existing Callable into a class with Generic[P, T] where P is a parameter specification type (i.e. ParamsSpec)

After figuring out the right way to declare a generic featuring a ParamSpec, I updated the class-resolver package to use the shiny new (and more accurate) annotations. Unfortunately, reality set in, and within hours, someone reported this caused errors in PyKEEN, my graph machine learning software package that heavily uses class-resolver to make its knowledge graph embedding models modular and configurable.

I was able to fix PyKEEN, but the fact that I was previously using a named type alias for Normalizer: TypeHint = Callable[[torch.Tensor], torch.Tensor] and then had to re-write it yet again the variable declaration’s type hint as FunctionResolver[[torch.Tensor], torch.Tensor] was not great. What I would love is some way to unpack Normalizer into the arguments of FunctionResolver[...].

Let me demonstrate in a more self-contained way:

from typing import Callable, Generic, Unpack, ParamSpec, TypeVar

P = ParamSpec("P")
T = TypeVar("T")

class Box(Generic[P, T]):
    def __init__(self, func: Callable[P, T]) -> None:
        self.func = func

def f(x: int) -> str:
    return str(x)

# This works!
box_1: Box[[int], str] = Box(f)

FType = Callable[[int], str]

# I wish I could do this, in case I already
# had a type variable referring to Callable[[int], str]
box_2: Box[Unpack[FType]] = Box(f)

While I’m abusing the usage of typing.Unpack, some way of extracting the arguments of one type and splatting them into another seems like it might have a use case. Maybe.

Using ParamSpec with Python Generics

2025-04-22T08:31:00+00:00

I’ve been working on applying strict static typing to my Python package class-resolver and ran into an interesting way of using generics in combination with parameter specification variables (i.e., ParamSpecs).

Normally, if you want to type annotate a function, you use the Callable, which works like the following:

from collections.abc import Callable

#: the [int] represents a function that takes in a single integer,
#:  and returns a single floating point number
IntToFloat = Callable[[int], float]

# this function fits the type annotation above, impl omitted
def square_root(x: int) -> float:
    ...

# simple example to show how to write functions that consume functions
def applies_int_to_float(func: Callable[[int], float], x: int) -> float:
    return func(x)


>>> applies_int_to_float(square_root, 9)
3.0

However, if you want to get generic, you need to use a combination of ParamSpec for the input variable signature and TypeVar for the return value.

from collections.abc import Callable
from typing import TypeVar

X = TypeVar("X")
T = TypeVar("T")

# simple example to show how to write functions that consume functions
def applies_unary_function(func: Callable[[X], T], x: X) -> T:
    return func(x)


>>> applies_unary_function(square_root, 9)
3.0

If you want to make the input fully generic, you can use ParamSpec and reference P.args (or P.kwargs, not shown here):

from collections.abc import Callable
from typing import ParamSpec, TypeVar

P = ParamSpec("P")
T = TypeVar("T")

def applies_unary_function_generic(func: Callable[P, T], *args: P.args, **kwargs: P.kwargs) -> T:
    return func(*args, **kwargs)


>>> applies_unary_function_generic(square_root, 9)
3.0

This gets even a bit more complicated if you want to type annotate a class that can take in a generic set of functions. Here’s an example on how this works:

from collections.abc import Callable
from typing import Generic, ParamSpec, TypeVar

P = ParamSpec("P")
T = TypeVar("T")


class ListOfFunctions(Generic[P, T]):
    def __init__(self, functions: list[Callable[P, T]]) -> None:
        self.functions = functions


def identity(x: int) -> int:
    return x


def plus_two(x: int) -> int:
    return x + 2

my_list: ListOfFunctions[[int], int] = ListOfFunctions([identity, plus_two])

The most important part of this discovery for me was actually how to type-annotate the resulting object, which magically is able to accept a list in the place where the ParamSpec should be.

Here’s the new way to write the same class using PEP-0695 type parameter syntax, introduced in Python 3.12:

from collections.abc import Callable


class ListOfFunctions[**P, T]:
    def __init__(self, functions: list[Callable[P, T]]) -> None:
        self.functions = functions

My two big wishes for typing in the future:

make the builtin any be a valid substitution for typing.Any
make some cute syntax, so I don’t need to import from collections.abc import Callable

A dilemma with PEP-696 default generics when using optional static typing in Python

2025-04-19T15:45:00+00:00

This post describes an issue I’ve had with writing correct types when using PEP-696 defaults in typing.TypeVar. I posted the exploration in a companion repository on GitHub.

The motivation behind this comes from my work in biomedical data integration and the semantic web. I wrote the curies Python package to provide a fully generic and reusable data model for representing pairs of prefixes and local unique identifiers via the curies.Reference class.

I extended this class in the bioregistry in order to validate and standardize prefixes and local unique identifiers using its detailed set of metadata rules. This is implemented in the bioregistry.NormalizedReference.

In other places, like ssslm, I’ve built on the curies.Reference data structure for maximum compatibility. However, sometimes I want to be able to inject the bioregistry.Reference class to get the guarantees of standardization. However, I don’t want the ssslm package to have to know about the Bioregistry, since I want to keep ssslm package generic.

The solution might be with PEP-0696, which extends the ability to specify generic types with defaults. This means places where I used to hard-code a curie.Reference, I can now use a generic on the entire class that has curie.Reference as the default… in theory.

In practice, it isn’t so easy. I boiled it down to a fully self-contained example (which I also dumped in https://github.com/cthoyt/python-typing-dilemma).

from typing import Any, TypeVar

type Record = dict[str, Any]

class Element:
    def __init__(self, record: Record) -> None:
        self.record = record

class DerivedElement(Element):
    pass

# note, we're using PEP-696 default keyword, which is available from Python 3.13 onwards
T = TypeVar("T", bound=Element, default=Element)

def from_record_1(record: Record, element_cls: type[T] = Element) -> T:
    return element_cls(record)

This should work. In fact, MyPy is able to infer the types correctly for the return value. The big question is: what’s the correct way to type-annotate element_cls? If you run MyPy on this, you get (abridged output):

$ git clone https://github.com/cthoyt/python-typing-dilemma
$ cd python-typing-dilemma
$ uvx --python 3.13 mypy --strict main.py
main.py:23: error: Incompatible default for argument "element_cls" (default has type "type[Element]", argument has type "type[T]")  [assignment]

I tried a few other things:

Defining a second type variable TType = TypeVar("TType", bound=type[Element]) or TType = TypeVar("TType", bound=type[Element], default=type[Element]) (not included in the repo)
Defining a second type based on the first TType = TypeVar("TType", bound=type[T]) (not included in the repo)
Using None as a sentinel value (try #2)
Using overloads (try #3, suggested by Guido)

None of this worked, so I asked for help. Turns out, other people ran into this issue already and brought it up with MyPy.

https://github.com/python/mypy/issues/3737
https://github.com/python/mypy/issues/12962
https://github.com/python/mypy/issues/18812

Right now, I don’t think my use case can be solved, so I’ll have to sit tight!

The EFO_ID column in ChEMBL’s drug indications table isn’t what you think it is

2025-04-17T16:42:00+00:00

ChEMBL periodically curates clinical trial information into its DRUG_INDICATION table. However, there’s some weird inconsistencies in the way it references disease concepts in external vocabularies. This blog post is an exploration of that table.

As of ChEMBL v35, the DRUG_INDICATION table contains the following columns:

DRUGIND_ID - a unique identifier for the chemical-indication pair
MOLREGNO - a foreign key to the molecules table
MAX_PHASE_FOR_IND - the maximum phase achieved by clinical trials of the chemical-indication pair
MESH_ID - the local unique identifier from Medical Subject Headings (MeSH) for the indication
MESH_HEADING - the label in MeSH for the given MeSH ID
EFO_ID - A compact URI (CURIE) for a term in the Experimental Factor Ontology (EFO) ( in theory)
EFO_TERM - a label for the CURIE in the EFO_ID column

This is already strange, considering that for cell lines, tissues, and targets, ChEMBL has created its own table which contains the cross-references to external vocabularies. Here, they’re baked into the pivot table.

The funny business is about the EFO_ID column:

It’s strange that the MESH_ID column uses local unique identifiers but the EFO_ID column uses compact URIs (CURIEs). CURIEs are a syntax for referencing an entity in an ontology or database that takes the form of :. The prefix is usually the acronym for the resource and the local unique identifier is the ID inside the resource (usually a number). More on this in my previous post.
The CURIEs in the EFO_ID column aren’t all using EFO as the prefix!

Let’s have a look at what’s actually in the EFO_ID column by using the chembl_downloader Python package to automatically download the latest version of ChEMBL and run SQL queries over it.

import chembl_downloader

chembl_downloader.query("""\
   SELECT DISTINCT efo_id, efo_term
   FROM DRUG_INDICATION
   WHERE efo_id NOT LIKE 'EFO:%'
""")

efo_id	efo_term
HP:0001945	Fever
Orphanet:309005	Disorder of lipid metabolism
HP:0003124	Hypercholesterolemia
Orphanet:79211	Combined hyperlipidemia
HP:0000023	Inguinal hernia
…	…

Using a bit of SQL string processing to identify the prefixes and the Bioregistry to retrieve the name and homepage gives a bit more context about what the prefixes in CURIEs in the EFO_ID column represent.

import bioregistry
import chembl_downloader

sql = """\
SELECT prefix, count(prefix) as count
FROM (
    SELECT substr(efo_id, 0, instr(efo_id, ":")) as prefix
    FROM DRUG_INDICATION
)
GROUP BY prefix
HAVING count(prefix) > 0
ORDER BY count(prefix) DESC
"""

df = chembl_downloader.query(sql)
df["name"] = df["prefix"].map(bioregistry.get_name)
df["homepage"] = df["prefix"].map(bioregistry.get_homepage)

prefix	count	name	homepage
EFO	37,603	Experimental Factor Ontology	http://www.ebi.ac.uk/efo
MONDO	13,532	Mondo Disease Ontology	https://monarch-initiative.github.io/mondo
HP	3,381	Human Phenotype Ontology	http://www.human-phenotype-ontology.org/
Orphanet	359	Orphanet	http://www.orpha.net/consor/
MP	281	Mammalian Phenotype Ontology	https://www.informatics.jax.org/vocab/mp_ontology/
GO	50	Gene Ontology	http://geneontology.org/
DOID	45	Human Disease Ontology	http://www.disease-ontology.org
CHEBI	19	Chemical Entities of Biological Interest	http://www.ebi.ac.uk/chebi
UBERON	1	Uber Anatomy Ontology	http://uberon.org

The ones that stand out to me are CHEBI, UBERON, and GO, since these resources are respectively for chemicals, anatomical entities, and biological processes/cellular components/molecular functions.

I wrote the following function to do a bit of exploring, based on the prefix.

import chembl_downloader

def print_indications_with_prefix(prefix: str) -> None:
    sql = f"""\
        SELECT DISTINCT
            MOLECULE_DICTIONARY.chembl_id,
            MOLECULE_DICTIONARY.pref_name,
            DRUG_INDICATION.efo_id,
            DRUG_INDICATION.efo_term
        FROM MOLECULE_DICTIONARY
            JOIN DRUG_INDICATION ON MOLECULE_DICTIONARY.molregno == DRUG_INDICATION.molregno
        WHERE DRUG_INDICATION.efo_id LIKE '{prefix}:%'
        ORDER BY MOLECULE_DICTIONARY.pref_name
        """
    df = chembl_downloader.query(sql)
    df["chembl_id"] = df["chembl_id"].map(lambda s: f"[{s}](https://bioregistry.io/chembl.compound:{s})")
    df["efo_id"] = df["efo_id"].map(lambda s: f"[{s}](https://bioregistry.io/{s})")
    print(df.to_markdown(tablefmt="github", index=False))

Using UBERON returns a single result, which appears to be a mistake / an abuse of the database schema.

chembl_id	pref_name	efo_id	efo_term
CHEMBL4650497	PEGSITACIANINE	UBERON:0000029	lymph node

Using CHEBI returns a large number of diagnostic agents. This is part of the “role” hierarchy within ChEBI, and also what I would consider an abuse of the database schema.

chembl_id	pref_name	efo_id	efo_term
CHEMBL1234270	ARFOLITIXORIN	CHEBI:44185	methotrexate
CHEMBL5314823	DIGADOGLUCITOL	CHEBI:33295	diagnostic agent
CHEMBL4650354	FLORBENGUANE F18	CHEBI:33295	diagnostic agent
CHEMBL5095045	FLORZOLOTAU (18F)	CHEBI:33295	diagnostic agent
CHEMBL5314559	FLOTUFOLASTAT F 18 GALLIUM	CHEBI:33295	diagnostic agent
CHEMBL4298157	FLUBROBENGUANE F18	CHEBI:33295	diagnostic agent
CHEMBL5314633	IODINE I124 EVUZAMITIDE	CHEBI:33295	diagnostic agent
CHEMBL4298185	LONAPEGSOMATROPIN	CHEBI:37845	growth hormone
CHEMBL5314761	PEGFOSIMER MANGANESE	CHEBI:33295	diagnostic agent
CHEMBL4650497	PEGSITACIANINE	CHEBI:33295	diagnostic agent
CHEMBL4297334	PIFLUFOLASTAT F18	CHEBI:33295	diagnostic agent
CHEMBL5314483	RIZEDISBEN	CHEBI:33295	diagnostic agent
CHEMBL5314650	TECHNETIUM TC-99M LABELED CARBON	CHEBI:33295	diagnostic agent
CHEMBL4298067	TOMARALIMAB	CHEBI:35610	antineoplastic agent
CHEMBL5314445	VIDOFLUFOLASTAT(18F)	CHEBI:33295	diagnostic agent
CHEMBL4594280	VIPIVOTIDE TETRAXETAN	CHEBI:33295	diagnostic agent
CHEMBL4594411	XENON XE-129, HYPERPOLARIZED	CHEBI:33295	diagnostic agent
CHEMBL5314610	ZOPOCIANINE	CHEBI:33295	diagnostic agent
CHEMBL5314611	ZOPOCIANINE SODIUM	CHEBI:33295	diagnostic agent

Using GO returns aging, regulation of ovulation (both positive and negative), and wound healing as the four unique biological processes. This is a little less controversial than UBERON and CHEBI, but it still has a bit of a mismatch for the idea of an “indication”.

chembl_id	pref_name	efo_id	efo_term
CHEMBL1566	ACARBOSE	GO:0007568	aging
CHEMBL600	ACETYLCYSTEINE	GO:0007568	aging
CHEMBL1399	ANASTROZOLE	GO:0007568	aging
CHEMBL25	ASPIRIN	GO:0007568	aging
CHEMBL1201556	BECAPLERMIN	GO:0042060	wound healing
CHEMBL5315086	BETULA PUBESCENS BARK	GO:0042060	wound healing
CHEMBL1200800	CALCIUM ACETATE	GO:0007568	aging
CHEMBL1200539	CALCIUM CARBONATE	GO:0007568	aging
CHEMBL113313	CAPROMORELIN	GO:0007568	aging
CHEMBL1042	CHOLECALCIFEROL	GO:0007568	aging
CHEMBL2108185	CORIFOLLITROPIN ALFA	GO:0060279	positive regulation of ovulation
CHEMBL429910	DAPAGLIFLOZIN	GO:0007568	aging
CHEMBL1421	DASATINIB	GO:0007568	aging
CHEMBL139	DICLOFENAC	GO:0007568	aging
CHEMBL367149	DOCONEXENT	GO:0007568	aging
CHEMBL1200969	DUTASTERIDE	GO:0007568	aging
CHEMBL135	ESTRADIOL	GO:0007568	aging
CHEMBL2108390	FIBRIN	GO:0007568	aging
CHEMBL500468	GHRELIN	GO:0007568	aging
CHEMBL389621	HYDROCORTISONE	GO:0007568	aging
CHEMBL13817	IBUTAMOREN	GO:0007568	aging
CHEMBL460026	ICOSAPENT	GO:0007568	aging
CHEMBL2109042	INFLUENZA VIRUS VACCINE	GO:0007568	aging
CHEMBL471737	IVABRADINE	GO:0007568	aging
CHEMBL444814	L-CITRULLINE	GO:0007568	aging
CHEMBL191	LOSARTAN	GO:0007568	aging
CHEMBL1201419	LUTROPIN ALFA	GO:0060279	positive regulation of ovulation
CHEMBL2107951	MALTODEXTRIN	GO:0007568	aging
CHEMBL267936	MECAMYLAMINE	GO:0007568	aging
CHEMBL1201716	MECASERMIN	GO:0007568	aging
CHEMBL1431	METFORMIN	GO:0007568	aging
CHEMBL650	METHYLPREDNISOLONE	GO:0007568	aging
CHEMBL4074884	MITOQUINONE MESYLATE	GO:0007568	aging
CHEMBL438497	NICOTINAMIDE RIBOSIDE	GO:0007568	aging
CHEMBL3	NICOTINE	GO:0007568	aging
CHEMBL1201574	ONABOTULINUMTOXINA	GO:0007568	aging
CHEMBL1234886	OXYGEN	GO:0007568	aging
CHEMBL395429	OXYTOCIN	GO:0007568	aging
CHEMBL90593	PRASTERONE	GO:0007568	aging
CHEMBL50	QUERCETIN	GO:0007568	aging
CHEMBL165	RESVERATROL	GO:0007568	aging
CHEMBL413	SIROLIMUS	GO:0007568	aging
CHEMBL1200574	SODIUM CHLORIDE	GO:0007568	aging
CHEMBL93268	SODIUM NITRITE	GO:0007568	aging
CHEMBL136478	SODIUM NITROPRUSSIDE	GO:0007568	aging
CHEMBL1201621	SOMATROPIN	GO:0007568	aging
CHEMBL48802	SULFORAPHANE	GO:0007568	aging
CHEMBL386630	TESTOSTERONE	GO:0007568	aging
CHEMBL2107067	TESTOSTERONE UNDECANOATE	GO:0007568	aging
CHEMBL3545347	TXA127	GO:0007568	aging

There wasn’t really a point to this post other than to show off a quirk I found in ChEMBL. This is useful to be aware of when automatically processing the database in bulk, e.g., for building a knowledge graph.

There are two other follow-up questions I would have about this table:

Are there any EFO terms that are outside the disease hierarchy (i.e., not a child of EFO:0000408)?
Why are there DOID terms? The combination of EFO and MONDO should cover everything. Answering this question actually isn’t so difficult given my recent work on assembling mappings with SeMRA, specifically for the disease landscape. I’ll try to come back to this in a future post.

If you made it this far: what did you think about my clickbait title?

Data Modeling and Integration with Clinical Trials

2025-01-23T18:23:00+00:00

I’ve recently worked with clinical studies from ClinicalTrials.gov and other international registries. This post is a review on how to access data, a proposal for how it can be modeled using the Ontology for Biomedical Investigations (OBI), a proof-of-concept ontologization of ClinicalTrials.gov, and some insights into how this data can be integrated with other resources to address classical problems in drug discovery from a knowledge graph perspective.

This is a long read, so here’s a table of contents:

Automated download of ClinicalTrails.gov with clinicaltrials-downloader
Summarizing ClincialTrials.gov study types, allocations, and phases
Example clinical studies
Proposing an ontology meta-model
Proof-of-concept ontology export of ClinicalTrials.gov using pyobo
Reflections and what’s missing
What’s this all useful for, anyway?

Automated Download

Similar to ChEMBL, DrugBank, and UMLS, I authored a Python package, that automates download and caching clinical studies from ClinicalTrials.gov. Its source code is available under the MIT license at https://github.com/cthoyt/clinicaltrials-downloader, and it can be installed with pip install clinicaltrials-downloader. The package exposes two main functions for getting the raw, unprocessed data:

from clinicaltrials_downloader import get_studies, get_studies_slim

# contains all fields, (~2GB, gzipped)
studies = get_studies()

# contains a useful subset of the fields, much smaller (~70MB, gzipped)
studies_slim = get_studies_slim()

Keep in mind that the data is updated daily, so the caches become out of date quickly. Pass force=True to either of the downloader functions to update the local cache of the database.

Summarization

I generated a few summary tables over the slim subset of ClinicalTrials.gov using this script. The tables can be regenerated with the following command, adding --force if you want to refresh the cache:

$ uv run --script https://gist.githubusercontent.com/cthoyt/12a3cb3c63ad68d73fe5a2f0d506526f/raw/98bd3795ec02ebd2c8bb8746e3a3a5d23aeffd75/clinicaltrials-summary.py

As an aside, if you’re not yet familiar with PEP-723, you can now put the requirements and other metadata inside a script. This also allows me to tell uv to use the development version of PyOBO and my fork of Gilda that doesn’t include all its heavy requirements.

Study Type and Allocation

ClinicalTrials.gov contains three main study types:

interventional study (i.e., clinical trial) - a study in which participants are assigned zero or more diagnostic, therapeutic, or other types of interventions depending on the arm into which they are allocated
observational study - a study in which participants are assessed for biomedical or health outcomes. They may receive interventions, but they are not assigned like in interventional studies
expanded access (i.e., compassionate use) - a mechanism through which patients who are not participants in a clinical trial to receive access to non-approved/experimental medicine.

Interventional studies can be divided into two categories based on their allocation - the used to assign participants to an arm of a clinical study. They are randomized and non-randomized.

The table below adjusts the internal labels for legibility, aggregates missing values and NA entries, and sorts by most common.

Study Type	Allocation	Count
Interventional	Randomized	261,643
Observational		120,775
Interventional		95,249
Interventional	Non-Randomized	42,759
Expanded Access		966
		902

Clinical Trial Phases

The phase primarily communicates the objective of a clinical trial (i.e., interventional study). Observational trials and expanded access studies therefore do not have phases.

Image above from the Hepatitis B Foundation.

There are six common phases appearing on ClinicalTrials.gov:

Early Phase 1 (formerly, Phase 0) - Assess oral bioavailability, pharmacokinetics (very small group; almost always left out of diagrams like the one above). This is not the same thing as pre-clinical trials, which are often done with biochemical assays, cellular assays, and work with model organisms.
Phase 1 - Assess safety in healthy volunteers (small group)
Phase 2 - Assess efficacy and side effects (medium group)
Phase 3 - Assess efficacy, effectiveness, and safety ( large group)
Phase 4 - Post-approval surveillance
Phase Not Applicable (N/A) - Applied to trials without phases, such as trials with devices or behavioral interventions

The table below adjusts the internal clinical trial phases’ labels for legibility, aggregates missing values and NA entries, and sorts by progression.

Phase	Count
0	5,434
1	44,195
1, 2	15,219
2	59,412
2, 3	6,982
3	39,160
4	33,129
N/A or missing	318,763

Some trials are annotated with multiple phases, either 1/2 or 2/3. These could also be combined in a different way for “maximum clinical phase” aggregation operations.

Unsurprisingly, there is an attrition through the progression of phases, but it is not as stark as I would have expected. It might also be interesting to stratify this by year to see if trials are more likely to succeed as time goes on.

Example Clinical Studies

Having examples and doing spot-checks is always helpful when exploring new data, so I generated a table containing example clinical trials for each study type, allocation, and phase. While there are many studies with more than one intervention and/or condition, this table only shows trials with a single one of each to reduce complexity.

Study/Phase(s)	NCT ID	Title	Condition	Intervention
Expanded Access	NCT01317953	Oral Green Tea Extract for Small Cell Lung Cancer	Small Cell Lung Carcinoma	(-)-epigallocatechin 3-gallate
Observational	NCT03418987	The Vertebral Vector in a Horizontal Plane. A Simple Way to See in 3D.	Scoliosis
Non-Randomized (Phase 0)	NCT01209533	Inhaled Iloprost in Mild Asthma	Asthma	iloprost
Non-Randomized (Phase 1)	NCT01682187	A Dose-Escalation Study in Participants With Recurrent Malignant Glioma	Glioma	lomustine
Non-Randomized (Phase 1/2)	NCT00379587	Rituximab for Prevention of Chronic GVHD	Hematologic Neoplasms	rituximab
Non-Randomized (Phase 2)	NCT00176787	Radiation Therapy With Capecitabine in Rectal Cancer	Rectal Neoplasms	capecitabine
Non-Randomized (Phase 2/3)	NCT04431453	Study of Remdesivir in Participants Below 18 Years Old With COVID-19	COVID-19	remdesivir
Non-Randomized (Phase 3)	NCT03184987	A Long-term Safety Study of Fixed Dose Combination Therapy Fluticasone […]	Asthma	albuterol
Non-Randomized (Phase 4)	NCT03282487	Optimising Steroid Replacement in Patients With Adrenal Insufficiency	Adrenal Insufficiency	cortisol
Randomized (Phase 0)	NCT04293887	Efficacy and Safety of IFN-α2β in the Treatment of Novel Coronavirus Patients	Coronavirus Infections	interferon
Randomized (Phase 1)	NCT01166087	Bioequivalence Study of Fluoxetine Hydrochloride Delayed-Release Capsules […]	Malnutrition	fluoxetine
Randomized (Phase 1/2)	NCT00106587	Treatment of In-Stent Restenosis by Paclitaxel Coated PTCA Balloons […]	Coronary Restenosis	paclitaxel
Randomized (Phase 2)	NCT00094887	Nitric Oxide Inhalation to Treat Sickle Cell Pain Crises	Anemia, Sickle Cell	nitric oxide
Randomized (Phase 2/3)	NCT00136487	Celecoxib (Celebrex) Versus Placebo in Men With Recurrent Prostate Cancer	Prostatic Neoplasms	celecoxib
Randomized (Phase 3)	NCT00843687	A Comparison of the Pharmacokinetics and Safety of Long-acting Injectable […]	Schizophrenia	risperidone
Randomized (Phase 4)	NCT03586687	Osteoarthritis Shoulder Injection Study	Osteoarthritis	triamcinolone

Proposing an Ontology Meta-model

Given my goal to create an ontology export of ClinicalTrials.gov, I had to start by making some modeling decisions. The first was that each clinical study in the resource is an instance. This meant that I had to start by finding the right class for each, corresponding to the study types and allocations that I explored above.

Searching for Existing Ontology Classes

The Ontology for Biomedical Investigations (OBI) is a high quality ontology that contains terms for assays, devices, objectives, and other aspects of biomedical investigations. Therefore, I would ideally be able to find terms corresponding to the study types and allocations that I explored above already inside it.

I used the Ontology Lookup Service (OLS) to search for classes corresponding to the study types I explored above, but didn’t find anything specific enough in OBI. However, I did note that if there would be classes for clinical studies, then they would appear under its high-level class for investigation (OBI:0000066)

While searching in the OLS, I did find the following relevant classes, each with their own issues:

The Semantic Science Integrated Ontology (SIO) has a term for clinical trial (SIO:001000), but SIO doesn’t follow best practices from Open Biological and Biomedical Ontology (OBO) Foundry, meaning that it is difficult to reuse and less trustworthy.
The Ontology of Precision Medicine and Investigation (OPMI) has a term for clinical trial (OPMI:0004507). I’m not comfortable reusing terms from this ontology for two main reasons:
1. It’s use-case specific, and curated based on project-based needs, which means that it’s not a reliable resource.
2. It’s not curated using modern ontology infrastructure, so I’m not sure that I can trust it will be maintained.
The Informed Consent Ontology (ICO) also has a term for clinical trial (ICO:0000065) but I have the same reservations as for OPMI. There’s an overlap of the same authors with OPMI, so I have reservations to invest in reusing terms they haven’t been able to deduplicate themselves.
Eagle-I Resource Ontology (ERO) is an OBO Foundry ontology that has a term clinical trial (ERO:0000016) which nicely subclasses OBI’s investigation class, but ERO has been abandoned and marked as deprecated in the OBO Foundry.
The Ontology for MicroRNA Target (OMIT) haphazardly imported all of MeSH at some point and has a term Clinical Trial (OMIT:0016936).
The Clinical Trials Ontology (CTO) is an OBO Foundry ontology that nominally has the correct scope and has a term clinical trial (CTO:0000220), but there are potential issues with its design choices, and it was produced by a group that has historically had difficulty maintaining its resources and actively participating in the OBO community. Despite these issues making it less suitable for reuse, the associated workshop proceeding CTO: a Community-Based Clinical Trial Ontology and its Applications in PubChemRDF and SCAIView contains some interesting ideas.

Honorable mentions from non-ontology resources, which are a not ideal from a modeling perspective to use as a parent class in an ontology:

BioLink model has a term clinical trial (biolink:ClinicalTrial).
SNOMED has a term Clinical trial (procedure) (SNOMED:110465008), but is not an ontology and is notorious for being a closed resource, hampering reuse.
NCIT has a term for Clinical Trial (NCIT:C71104), but it is not curated an ontology (despite OBO Foundry having an OWL conversion of it).
MeSH has a term Clinical Trials as Topic (mesh:D002986).

Despite all of what I could find, none of these terms were part of an ontology that I can trust. Further, most of them conflated interventional clinical studies, i.e., clinical trials, with all other clinical studies.

Therefore, the next step was to get in touch with OBI and ask them to mint an authoritative term, that also can capture the nuance in clinical studies that is lost in the other resources. I did that in their issue tracker obi-ontology/obi#1831. They were very receptive, we had a nice conversation that brought up several points, and they challenged me to go even further than just proposing the parent terms and begin to develop a standardized model.

A draft proposal

The following diagram represents a draft proposal that includes several terms that OBI could mint as well as the kinds of relations between them. This is not a perfect proposal - its goal is to be a discussion piece for an upcoming OBI community call. There are still several parts missing and open questions.

Here are some missing parts to this model that could be added incrementally:

A more detailed categorization of expanded access studies based on the expanded access types and expanded access status
A more detailed categorization of observational studies based on the 1) assembly of groups and cohorts and 2) observational study models such as case-control, case-only, case-cross-over, ecologic or community studies, and family-based.
A model for eligibility criteria and enrollment
A model for outcomes, linked to OBI’s assay terms
A model for investigators, funder types, and sponsors to support bibliometric investigation
A model for geolocation data associated with clinical study sites
A model for capturing adverse events and reasons for trial cancellation/end. In downstream applications, this could be used in tandem with the FDA’s Adverse Event Reporting System (FAERS) and the Vaccine Adverse Event Reporting System (VAERS).

There is high potential for applying natural language processing methods to extract more detailed information from the unstructured parts of clinical study records. I’ve been focusing on ClincialTrials.gov as an example in this post, but other clinical trial registries comprise almost exclusively unstructured text.

Proof-of-concept Ontology Export of ClinicalTrials.gov

PyOBO is Python software package that implements an in-memory data structure for OBO/OWL ontologies as well as I/O operations. On top of this, it implements workflows for converting databases like HGNC, MeSH, and ChEMBL into OBO/OWL ontolgies. These workflows are careful to make good design decisions, reusing classes and relations from other OBO ontologies when possible. This is crucial for them to be readily integratable with other resources.

The obo-db-ingest repository is responsible for automatically downloading new versions of the resources covered by PyOBO, converting them to OBO/OWL, archiving them to Zenodo, and assigning persistent URLs (PURLs) so the files can be accessed in a sustainable way. It’s also careful to include licensing information such that anyone can download these resources in a ready-to-use format, whereas the underlying resources are often less easy to use directly.

As a proof-of-concept, I implemented a converter for ClinicalTrials.gov in PyOBO at https://github.com/biopragmatics/pyobo/blob/main/src/pyobo/sources/clinicaltrials.py. The draft converter uses temporary classes to represent the study types and allocations I’m proposing to OBI. It also mints some of its own relationships, which would ideally be encoded in either OBI or the Relation Ontology (RO) for maximum reusability.

The initial export contains more than 500K clinical studies; nearly one million literature references, and near two million relationships between trials, interventions, and conditions (there are still several places for expansion and improvement discussed below). The ClinicalTrials.gov data is licensed under an equivalent to a public domain dedication, so there are few restrictions on remixing and redistributing the data this way.

A summary page can be found in the obo-db-ingest repository here and the exported artifacts are listed here:

Artifact	Download PURL
OBO	https://w3id.org/biopragmatics/resources/clinicaltrials/clinicaltrials.obo.gz
OFN	https://w3id.org/biopragmatics/resources/clinicaltrials/clinicaltrials.ofn.gz
OWL	https://w3id.org/biopragmatics/resources/clinicaltrials/clinicaltrials.owl.gz
OBO Graph JSON	https://w3id.org/biopragmatics/resources/clinicaltrials/clinicaltrials.json.gz
Nodes	https://w3id.org/biopragmatics/resources/clinicaltrials/clinicaltrials.tsv

Here’s what some OBO instances for clinical studies look like for each clinical study type:

[Instance]
id: clinicaltrials:NCT00000102
name: Congenital Adrenal Hyperplasia\: Calcium Channels as Therapeutic Targets
property_value: clinicaltrials:has_intervention mesh:D009543 ! has intervention Nifedipine
property_value: clinicaltrials:investigates_condition mesh:D000308 ! investigates condition Adrenocortical Hyperfunction
property_value: clinicaltrials:investigates_condition mesh:D000312 ! investigates condition Adrenal Hyperplasia, Congenital
property_value: clinicaltrials:investigates_condition mesh:D006965 ! investigates condition Hyperplasia
property_value: clinicaltrials:investigates_condition mesh:D047808 ! investigates condition Adrenogenital Syndrome
instance_of: interventional-clinical-trial

[Instance]
id: clinicaltrials:NCT00000104
name: Does Lead Burden Alter Neuropsychological Development?
property_value: clinicaltrials:investigates_condition mesh:D007855 ! investigates condition Lead Poisoning
property_value: clinicaltrials:investigates_condition mesh:D011041 ! investigates condition Poisoning
instance_of: observational-clinical-trial

[Instance]
id: clinicaltrials:NCT00000106
name: 41.8 Degree Centigrade Whole Body Hyperthermia for the Treatment of Rheumatoid Diseases
property_value: clinicaltrials:investigates_condition mesh:D003095 ! investigates condition Collagen Diseases
property_value: clinicaltrials:investigates_condition mesh:D012216 ! investigates condition Rheumatic Diseases
instance_of: randomized-interventional-clinical-trial

[Instance]
id: clinicaltrials:NCT00000250
name: Cold Water Immersion Modulates Reinforcing Effects of Nitrous Oxide - 2
property_value: clinicaltrials:has_intervention mesh:D009609 ! has intervention Nitrous Oxide
property_value: clinicaltrials:investigates_condition mesh:D009293 ! investigates condition Opioid-Related Disorders
property_value: clinicaltrials:investigates_condition mesh:D019966 ! investigates condition Substance-Related Disorders
instance_of: non-randomized-interventional-clinical-trial

[Instance]
id: clinicaltrials:NCT00040625
name: ALIMTA \(Pemetrexed\) Alone or in Combination With Cisplatin for Patients With Malignant Mesothelioma.
property_value: clinicaltrials:has_intervention mesh:D000068437 ! has intervention Pemetrexed
property_value: clinicaltrials:investigates_condition mesh:D000086002 ! investigates condition Mesothelioma, Malignant
property_value: clinicaltrials:investigates_condition mesh:D008654 ! investigates condition Mesothelioma
instance_of: expanded-access-study

Reflections, and, what’s missing?

I first became familiar with ClinicalTrials.gov and other clinical study registries while working on the RAPTER project, funded by the America Defense Threat Reduction Agency (DTRA) with the goal to integrate vaccine information and build computational tools to quicken the development of vaccines in response to future pandemics.

One of the key issues to overcome was the accuracy of the data within these resources. For example, ClinicalTrials.gov contains both a free-text and processed field for its conditions and interventions. In many cases, re-processing was required to ensure complete and accurate information. In my draft export of ClincalTrials.gov, I exclusively used the processed data fields, but a more careful conversion would require additional data science techniques.

Similarly, for data from the World Health Organization and other clinical trial registries, full NER and relation extraction is required to identify interventions, conditions, outcomes, and other fields.

Even when processed data is available, it’s using MeSH identifiers, which are not readily integrable with other resources. That’s why ontologies often curate MeSH cross-references themselves. I created the Biomappings project as a way to quickly predict and curate MeSH mappings to other chemical and disease vocabularies, such as Chemical Entities of Biological Interest (ChEBI) and the Disease Ontology (DOID). I also built SeMRA, a workflow for assembling and inferring mappings to best reuse existing mappings available from a wide variety of sources. These two approaches are crucial for making clinical study and trial information actionable in a data integration scenario.

I didn’t even mention the PICO (patient/population, intervention, comparison and outcomes) data model - this is a whole other can of worms for a different discussion. I will try to come back to that if I get the chance to write up some of my work with Jeremy Zucker’s team at PNNL on the y0 Causal Reasoning Engine and Judea Pearl-style causal inference applications in clinical study statistical analysis.

There’s a lot of work to do in this space, but this is a nice first step. For me, exporting lots of resources in a standard ontology format makes it easy to load up a complete set of nodes when building knowledge graphs. As a coda, I will say a little bit about what I like to do with this kind of data once I have got it structured and integrated with other sources.

What’s this all useful for, anyway?

My work in the past decade has focused on constructing and applying knowledge graphs to tasks in drug discovery. While a large part of this has incorporated machine learning, artificial intelligence, and causal inference, many problems can be formulated as queries over a graph-like data structure.

In this last section, I’m going to give a few examples of graph queries that can enable experts to explore and generate interesting, explainable, testable hypotheses. In practice, I have been using Neo4j and RDF as graph models with their respective Cypher and SPARQL query languages, but the case studies here will stay at a high level.

Maximum Phase for a Drug

   graph LR
     drug[Drug] -- intervention in --> trial[Clinical Trial] -- has phase --> phase[Phase]
     drug -. has maximum phase (aggregated) .-> phase

Because the same intervention may appear in multiple clinical trials, it’s useful to know the maximum phase reached over all trials. If the maximum phase is four, then it can be concluded that the drug was approved for use. If the maximum phase is three (and they are all completed), then it can be concluded that the drug ultimately failed to be effective. Such drugs are can be good candidates for repositioning since the expensive safety studies have already been run.

Maximum Phase for a Scaffold or Substructure

   graph LR
     substructure[Substructure] -- substructure of --> drug[Drug] -- intervention in --> trial[Clinical Trial] -- has phase --> phase
     substructure -. has maximum phase (aggregated) .-> phase

For small molecule drugs, the principle of structure-activity relationship (SAR) states that there is often a high correlation between the chemical structure and its functional activity. Many similar drugs work because they share a privileged substructure. For example, the beta-hydroxy lactone appearing in statins enable their HMC-CoA reducatase inhibitor activity, which confers their ability to reduce LDL and risk of cardiovascular disease. Similarly, sulfonamides were a classic substructure used in first-generation antibacterials due to their competitive inhibition of dihydropteroate synthase, a key part of folate biosynthesis in most non-human, non-eukaryote organisms.

Ontologies like ChEBI contain relationships between these substructures (and scaffolds) and chemicals. By combining ChEBI and ClinicalTrials.gov in a single graph, the previous query that aggregated maximum phase over a specific chemical can be generalized to substructures. Further, the “substructure of” relationship is transitive, meaning that multiple substructure relations can be chained together to ask the same question at many levels of granularity.

Ultimately, you can get results like beta-hydroxy lactone have appeared in clinical trials of a maximum phase of 4 and ask even more granular questions, like what is the maximum clinical phase for drugs with an arsenic atom as part of their structure (surprisingly, the answer is more than 0!). On the flip side, this can either identify opportunities for structures that haven’t been in clinical trials, or beg the question of why they haven’t so far.

Note that the aggregation over maximum phase is just an example - being able to query over a combination of one resource (here, ClinicalTrials.gov) and the hierarchy of another (here, ChEBI) is a more general and powerful concept.

Maximum Phase for Pairs of Substructures and Diseases

   graph LR
     substructure[Substructure] -- substructure of --> drug[Drug] -- intervention in --> trial[Clinical Trial] -- studies --> disease[Disease]
     trial -- has phase --> phase[Phase]
     substructure -. has maximum phase (aggregated) .-> phase

The previous query becomes even more powerful when extending the path from the clinical trial to a disease. Then, the aggregation operation can be done over the pair of the substructure and the disease to answer questions like what’s the maximum phase that arsenic-containing drugs have been used in for each disease?, or, more specifically, what’s the maximum phase that arsenic-containing drugs have been used in for malaria?

Maximum Phase for Pairs of Substructures and Disease Classes

   graph LR
     substructure[Substructure] -- substructure of --> drug[Drug] -- intervention in --> trial[Clinical Trial] -- studies --> disease[Disease] -- subclass of --> diseaseclass[Disease Class]
     trial -- has maximum phase --> phase[Phase]
     substructure -. has maximum phase .-> phase

The second case study extended the way we look at drugs by adding the hieararchical substructure relationships from ChEBI. If we want to aggregate diseases at a different granularity, we can do so by incorporating the subclass relationships from a resource like the Disease Ontology. Then, we can ask the previous question at a chosen level of granularity, like what’s the maximum phase that arsenic-containing drugs have been used in for each parasitic infectious disease?

It’s a bit tricky to ask the open-ended question what’s the maximum phase that arsenic-containing drugs have been used in for each disease class? because it depends on the construction of the disease hierarchy. Other common aggregations here are for rare diseases, cancers, neurodegenerative diseases, etc. In practice, I usualy have an additional way of tagging the terms in the hierarchy that I want to aggregate on, either by labeling the node in a property graph, or using yet another relationship to a node representing a grouping of my desired query terms.

Maximum Phase for Pairs of Vaccine Platforms and Disease Classes

   graph LR
     vaccineplatform[Vaccine Platform] -- platform for --> vaccine[Vaccine] -- intervention in --> trial[Clinical Trial] -- studies --> disease[Disease] -- subclass of --> diseaseclass[Disease Class]
     trial -- has phase --> phase[Phase]
     vaccineplatform -. has maximum phase .-> phase

The original use case for a query like the one before was for the RAPTER project, where we were asked to summarize for each vaccine platform (e.g., RNA vaccines, DNA vaccines, viral vector vaccines) what was the maximum clinical trial phase for trials over COVID-19 and a few other classes of parasitic and bacterial infections. The diagram above has the same shape and flavor as the previous one with substructures, chemicals, and diseases except the type of relation and direction between a vaccine and a vaccine platform is different. Despite this, the query is effectively shaped the same.

Phenotypic Drug Discovery Scenarios

When there exists a good cellular model of disease or good model organisms, it’s possible to eschew the typical target focus of a drug discovery campaign. However, during or after a phenotypic drug discovery, it’s useful to uncover a deeper mechanistic context through a combination of target identification and mechanism of action deconvolution.

Target Identification

graph LR
  disease[Disease] -- studied in --> trial[Clinical Trial] -- uses intervention --> drug[Drug] -- regulates --> protein[Protein]
  disease -. has putative target .-> protein

Target identification is typically the process of identifying a protein whose modulation in a given disease context can result in a therapeutic effect.

If we have a successful clinical trial performed following a phenotypic drug discovery campaign, it might not be known what targets it modulates. Therefore, we can combine the clinical trial data with a chemical activity database like ChEMBL, so we can get all the targets for the chemical, then hypothesize that one or more of them are responsible for the drug’s therapeutic effect.

This is even more powerful when combining other databases like OpenTargets, which aggregates a wide variety of orthogonal evidence and provide workflows and resources for further triage and DisGeNet, which aggregates text mining co-occurrence evidence as a proxy for association, which can be useful when done at scale with the appropriate statistical adjustments.

Finally, target identification hypotheses are testable, especially following a phenotypic drug discovery campaign with a good cellular model or model organism. Tool compounds that are specific modulators for the target or genomic experiments like knockdowns, knockouts, or over-expression can provide more confident confirmation of the target’s viability.

Mechanism of Action Deconvolution

graph LR
  drug[Drug] -- intervention in --> trial[Clinical Trial] -- studies --> disease[Disease] -- has target --> protein[Protein]
  drug -. has putative mechanism of action .-> protein

The dual problem to target identification is mechanism of action (MoA) deconvolution. After a successful clinical trial following a phenotypic drug discovery campaign, but not know the target that your drug modulates. You can get the target information by integrating parts of a database like OpenTargets or DisGeNet, then you will be able to use the path query to propose putative targets.

MoA hypotheses are often easier to test than target identification hypotheses because they can be done in biochemical assays. However, there are lots of tricky targets (including ones that aren’t proteins) where this isn’t so straightforward.

For both target identification and MoA, you might start imagining ways to take into account chemical similarity, protein similarity, and other hierarchical structures for more sophisticated queries. Thinking about this is why I’ve loved transitioning from a medicinal chemist in my early career to a chem/bioinformatician in my mid-career. I’ll leave the last part as an exercise to the reader, or for someone who wants to pay for consultation ;)

Books I Read in 2024

2025-01-18T08:33:00+00:00

Here’s the books I read in 2024. If I were Dudley Dursley, I’d be very upset that I read one fewer new book than in 2023. But then, I’d remember that I re-read a lot of Cosmere in 2024 to prepare for Wind and Truth, which was great.

Iron Gold (Red Rising #4) by Pierce Brown
Dark Age (Red Rising #5) by Pierce Brown
Light Bringer (Red Rising #6) by Pierce Brown
The Fifth Season (Broken Earth #1) by N.K. Jemisin
Emma by Jane Austen
The Obelisk Gate (Broken Earth #2) by N.K. Jemisin
The Slow Regard of Silent Things (Kingkiller Chronicles) by Patrick Rothfuss
The Narrow Road Between Desires (Kingkiller Chronicles) by Patrick Rothfuss
The Three Body Problem (Remembrance of Earth’s Past #1) by Cixin Liu
The Dark Forest (Remembrance of Earth’s Past #2) by Cixin Liu
Death’s End (Remembrance of Earth’s Past #3) by Cixin Liu
Artificial Condition (Murderbot Diaries #2) by Martha Wells
Rogue Protocol (Murderbot Diaries #3) by Martha Wells
Exit Strategy (Murderbot Diaries #4) by Martha Wells
A Deadly Education (Scholomance #1) by Naomi Novik
Network Effect (Murderbot Diaries #5) by Martha Wells
Fugitive Telemetry (Murderbot Diaries #6) by Martha Wells
System Collapse (Murderbot Diaries #7) by Martha Wells
The Last Graduate (Scholomance #2) by Naomi Novik
The Golden Enclaves (Scholomance #3) by Naomi Novik
The Machine Stops by E.M. Forster
Grief is the Thing with Feathers by Max Porter
A Natural History of Dragons (Lady Trent #1) by Marie Brennan
The Tropic of Serpents (Lady Trent #2) by Marie Brennan
A Journey to the Center of the Earth by Jules Verne
Simple Passion by Annie Ernaux
Exhalation by Ted Chiang
Before the Coffee Gets Cold by Toshikazu Kawaguchi
The Sirens of Titan by Kurt Vonnegut Jr.
What You Can See from Here by Mariana Leky
The Rithmatist by Brandon Sanderson
The Voyage of the Basilisk (Lady Trent #3) by Marie Brennan
A Court of Thorns and Roses (ACOTAR #1) by Sarah J. Maas
The Stone Sky (Broken Earth #3) by N.K. Jemisin
The Alchemist by Paulo Coelho
In the Labyrinth of Drakes (Lady Trent #4) by Marie Brennan
Within the Sanctuary of Wings (Lady Trent #5) by Marie Brennan
Indigo Ridge by Perry Devney
City of Stairs (Divine Cities #1) by Robert Jackson Bennett
Bartleby the Scrivner by Herman Melville
Beowulf
Wind and Truth (Stormlight Archives #5) by Brandon Sanderson

Rereads

Cosmere: Stormlight Archive 1-4 and Warbreaker
Kingkiller Chronicles: Name of the Wind and A Wise Man’s Fear

What I Loved (minor spoilers):

Every single short story in Exhalation. You can check out my favorite, The Merchant and the Alchemist’s Gate, which is about how to find meaning in time travel, even if you can’t change anything.
The Alchemist. It beats you over the head with its moral philosophy about seizing your destiny. It’s great.
The sci-fi in the whole Three Body Problem trilogy. Lots of great ideas, even if the characters were weak at moments. I initially didn’t like Luo Ji, but he eventually had a great arc which I’ve come to understand follows a more typical Chinese archetype.
Lady Trent has such a great tone and voice as a narrator. The epistolary format worked so well for these books.
Fifth ideals!

Non-superlative Comments

Last year, I wrote that I wasn’t a big fan of Murderbot #1. I stand by my criticism of the first book, but I’m glad I tried the rest, because it addressed my qualms and was a lot of fun.
ACOTAR would have definitely benefited from having a major revision to tighten up the story and characters, but I am sad when people trash on it. I enjoyed it.

Disappointments

I started Persuasion by Jane Austen, but I didn’t care for any of the characters, so I haven’t finished it yet.
I heard so many times that The Fifth Season and its sequels were great. The first book had a cool premise and cool reveal at the end, but the second and third book just didn’t have any good characters or plot to hold it together. I should have put it down after the second book, but I just assumed that since each installment won a Hugo award, that there was a massive payoff at the end. There was not.
There weren’t enough Brandon Sandwiches, so I read his non-Cosmere novel, The Rithmatist, that will never get a sequel. When will I learn?
I’m very disappointed by the Netflix adaptation of Three Body Problem for not doing a better job of rewriting the characters. How can anyone in Hollywood trust Benioff and Weiss after Game of Thrones?

Funny Story

When I was in Gaspé over the summer, we were hanging out at a cafe that suggested Simple Passion by Annie Ernaux as the book of the week. My sister and I figured, we could read it and make some friends that week if they also had. We didn’t find anyone else who even noticed the sign about it being book of the week.

Exploring Event Venues in Wikidata

2025-01-17T17:08:00+00:00

I was working on making data about scholarly conferences more FAIR and a big question crossed my mind: what are all the conference venues? This post is about some queries I wrote for Wikidata, data issues I found, and a few drive-by curations that I did while looking for an answer, and my ideas for the future.

Querying Wikidata

Wikidata is always a good place to start looking for structured data because it contains a detailed and multidisciplinary ontology whose classes, subclass relationships, and instances are all queryable via SPARQL.

It has a top-level class for event venues (Q18674739), that even comes with a high-level schematic diagram on how this class relates to performance arts spaces:

I used the following SPARQL query to retrieve all the subclasses and investigate which might be relevant for conferences.

SELECT DISTINCT ?venueType ?venueTypeLabel ?venueTypeDescription
WHERE
{
  ?venueType wdt:P279* wd:Q18674739 .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],mul,en". }
}

The results from this query are embedded in the table below:

The results contain a lot of irrelevant venue types such as cinemas, theaters, and clubs. It’s interesting for me to make a list of these, and refine the query to exclude them like so:

SELECT DISTINCT ?venueType ?venueTypeLabel ?venueTypeDescription
WHERE
{
  ?venueType wdt:P279* wd:Q18674739 .
  FILTER(?venueType NOT IN (wd:Q41253, wd:Q622425))  # this list is shortened for demo purposes
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],mul,en". }
}

Data Exploration

I encountered two noteworthy data issues the first time I ran this query:

Some results had Wikidata identifiers in the labels column
Some results appeared to be for specific event venues, and not classes

The first issue can be attributed to the fact that not all Wikidata records have labels in english. This doesn’t mean that they aren’t valuable - it’s also an opportunity for doing manual or semi-automated translation to improve the data in Wikidata.

The second issue is related to the ontology structure of Wikidata. Because this is a query over subclasses of event venues, specific venues shouldn’t show up. This is because Wikidata records representing specific venues should use the instance of (P31) relationship to annotate what kind/type of event venue they are.

I identified a couple groups of event venues that seemed to have this issue:

dive sites, like Q14213916
cinemas in Brazil, like Q123856958
museums in Sweden, like Q60628067

I was able to go through and update the relationship types to be correct, so if you’re reading this later, you won’t see the issue. That’s the power of drive-by curation!

Follow-up

Next steps are to make a spreadsheet where I curate for each event venue subclass if they’re relevant or not.

After that, then I can construct a SPARQL query that uses the VALUES syntax to pick the list of superclasses, then the wdt:P31/wdt:P279* relation path to get all instances of any subclass of the selected event venue classes.

SELECT DISTINCT ?venue ?venueLabel
WHERE
{
  VALUES ?venueType { wd:Q1329623 }
  ?venue wdt:P31/wdt:P279* ?venueType .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],mul,en". }
}

At the time of writing, this query resulted in 8,728 distinct records in about 27 seconds.

While Wikidata is a great place to start, its nature as a fully open and community-driven resource has the drawback of being less trustworthy than resources that have an expert curation component (whether manual or semi-automated).

Persistent Identifier (PID) resources like the Research Organization Registry (ROR) demonstrated that there’s added value to having an expert-curated layer that both draws from and contributes back to Wikidata.

I’m working towards seeding a registry for event venues with a similar philosophy at https://github.com/event-venue-registry/evr. Importantly, this resource will follow the Open Data, Open Code, Open Infrastructure (O3) principles, meaning that the data and code are all in one GitHub repository that anyone can contribute to. Please reach out if you’d like to help.

Notes on Open Source Funding

2024-12-03T17:08:00+00:00

This stub post contains my notes about funding for open source software. It doesn’t follow a story like a lot of my posts, and is more like an ever-evolving notes sheet.

Here’s a list of resources for getting funding for open source, which also covers fiscal sponsorship:

Interesting blog posts about open source funding:

https://kvinogradov.com/algo-sponsors/
https://opensourcepledge.com/
https://heidiseibold.kit.com/posts/how-to-get-funding-by-caring-about-research-software-i (this one is really good, basically a superset of my list)

Please feel free to suggest things in the comments.

Downloading Audio from Soundcloud

2024-12-03T09:11:00+00:00

Brandon Sanderson has been releasing a few chapters a week of his upcoming novel, Wind and Truth, on his publisher’s website leading up to its December 6^th release. This includes the audiobook chapters, but they’re posted to Soundcloud and there’s no good way to listen at 1.6x speed. This post is a note sheet on how to download audio from Soundcloud and prepare it for my audiobook reader.

Get a package manager

First step is to get uv. I’ve been very excited to watch this fast tool replace many iffy Python packaging tools!

$ curl -LsSf https://astral.sh/uv/install.sh | sh

See also the full installation documentation.

Install the downloader tool

yt-dlp is a feature-rich command-line audio/video downloader with support for thousands of sites.

Second step is to install yt-dlp using uv tool install. uv installs the tool and its CLI in an isolated virtual environment but makes it available globally on your system.

$ uv tool install yt-dlp
Resolved 1 package in 106ms
Installed 1 package in 28ms
 + yt-dlp==2024.11.18
Installed 1 executable: yt-dlp

Look how fast it installs with a warm cache!

Download the audio

The chapters for Wind and Truth are indexed here. For example, you can navigate to interludes 3 and 4, scroll to the bottom, then open the Soundcloud embed in a new browser tab and copy the URL.

Third step is to invoke yt-dlp from the console with the URL as the only argument (in quotes!). The best thing about this tool is that it has a plugin-based architecture, so it can deal with basically anything you give it.

$ yt-dlp "https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/1931715512%3Fsecret_token%3Ds-WvtAtAMqqkc&color=%23ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&show_teaser=true&visual=true"
[SoundcloudEmbed] Extracting URL: https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/1931715512%3Fsecret_toke...ser=true&visual=true
[soundcloud] Extracting URL: https://api.soundcloud.com/tracks/1931715512?secret_token=s-WvtAtAMqqkc
[soundcloud] 1931715512: Downloading info JSON
[soundcloud] 1931715512: Downloading hls_mp3 format info JSON
[soundcloud] 1931715512: Downloading http_mp3 format info JSON
[soundcloud] 1931715512: Downloading hls_opus format info JSON
[info] 1931715512: Downloading 1 format(s): hls_opus_64
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 136
[download] Destination: Interludes - WIND AND TRUTH by Brandon Sanderson, narrated by Kate Reading and Michael Kramer [1931715512].opus
[download] 100% of   10.10MiB in 00:00:04 at 2.14MiB/s

Convert the audio’s format

The only issue is that it gives output in the Opus file format, which isn’t so easy to use. The fourth step is to convert this to m4b for easy use with an audiobook reader. After installing ffmpeg (e.g., with brew install ffmpeg), run

$ ffmpeg \
    -i "Interludes - WIND AND TRUTH by Brandon Sanderson, narrated by Kate Reading and Michael Kramer [1931715512].opus" \
    -c:a aac -b:a 128k -vn -f ipod interludes-3-and-4.m4b
ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
  built with Apple clang version 16.0.0 (clang-1600.0.26.4)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/7.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.100 / 61. 19.100
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
Input #0, ogg, from 'Interludes - WIND AND TRUTH by Brandon Sanderson, narrated by Kate Reading and Michael Kramer [1931715512].opus':
  Duration: 00:22:12.36, start: 0.000000, bitrate: 63 kb/s
  Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp
      Metadata:
        encoder         : Lavc58.91.100 libopus
Stream mapping:
  Stream #0:0 -> #0:0 (opus (native) -> aac (native))
Press [q] to stop, [?] for help
Output #0, ipod, to 'interludes-3-and-4.m4b':
  Metadata:
    encoder         : Lavf61.7.100
  Stream #0:0: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s
      Metadata:
        encoder         : Lavc61.19.100 aac
[out#0/ipod @ 0x6000016e4000] video:0KiB audio:20840KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 1.175908%
size=   21086KiB time=00:22:12.34 bitrate= 129.6kbits/s speed=43.8x
[aac @ 0x151004f50] Qavg: 668.452

Easy listening

I’m using BookPlayer, an open source project to listen to books. I usually sync them by using AirDrop to send the m4b files to my phone, then using its import menu.

Happy listening!

P.S., I wrote a script that automates crawling the Wind and Truth index, downloading, and converting these files.

Dependency Groups and ReadTheDocs

2024-11-19T11:39:00+00:00

PEP 735 introduced dependency groups in packaging metadata, which are complementary to optional dependencies in that they might not correspond to features in the package, but rather be something like development or release dependencies. I am slowly working towards updating my cookiecutter template cookiecutter-snekpack to use PEP 735. So far, uv and tox have released support - all that’s left is ReadTheDocs. This post summarizes the issue I added to their issue tracker and the following discussion.

Summary of optional dependencies and dependency groups.

Before PEP 735, optional dependencies were often used both for extra features (e.g., installing lxml for faster XML parsing) as well as development dependencies (e.g., for testing and documentation build). Here’s what the syntax for this looks like:

[project.optional-dependencies]
faster-xml = [
    "lxml",
]
tests = [
    "pytest",
    "coverage",
]
docs = [
    "sphinx>=8",
    "sphinx-rtd-theme>=3.0",
    "sphinx-click",
    "sphinx_automodapi",
]

When you’re using pip (or uv) to install, you can use square bracket notation to say which ones go in.

$ pip install .[tests,docs,faster-xml]

This is a bit problematic since it conflates what the purpose of optional dependencies are. Some build tools tried to address with their on custom configuration for development dependencies in pyproject.toml or elsewhere. However, dependency groups gives us a more principled approach towards categorizing dependencies that are not necessarily relevant for the code itself. Now, we can split up the example from before like this:

[project.optional-dependencies]
faster-xml = [
    "lxml",
]

[dependency-groups]
tests = [
    "pytest",
    "coverage",
]
docs = [
    "sphinx>=8",
    "sphinx-rtd-theme>=3.0",
    "sphinx_automodapi",
]

When you’re using pip (or uv) to install, you can use --dependency-groups to say which ones go in.

$ pip install --dependency-groups=tests,typing .[faster-xml]

Preparing an environment on ReadTheDocs

ReadTheDocs currently supports specifying optional dependencies (see https://docs.readthedocs.io/en/stable/config-file/v2.html#packages) with configuration like the following:

version: 2

python:
  install:
    - method: pip
      path: .
      extra_requirements:
        - docs

In my issue #11766, I suggested defining an alternate key dependency_groups that could work the same way.

version: 2

python:
  install:
    - method: pip
      path: .
      dependency_groups:
        - docs

ReadTheDocs code deep dive and implementation suggestion

I found a few places that would be relevant for a theoretical implementation. First, there’s a data structure that represents the slots available in the configuration for python > install. Here’s the code (permalink):

class PythonInstall(Base):
    __slots__ = (
        "path",
        "method",
        "extra_requirements",
    )

I’d simply add a new slot dependency_groups and update the related processing code (permalink):

...
if install.extra_requirements:
    extra_req_param = "[{}]".format(",".join(install.extra_requirements))
...

Here’s what I’d do:

# Added these next lines
dependency_group_args = []
if install.dependency_groups:
    # not clear if the equals is necessary or if
    # this can be broken into two parts
    dependency_group_args.append("--dependency-groups={}".format(",".join(install.dependency_groups))

extra_req_param = ""
if install.extra_requirements:
    extra_req_param = "[{}]".format(",".join(install.extra_requirements))
self.build_env.run(
    self.venv_bin(filename="python"),
    "-m",
    "pip",
    "install",
    "--upgrade",
    "--upgrade-strategy",
    "only-if-needed",
    "--no-cache-dir",
    "{path}{extra_requirements}".format(
        path=local_path,
        extra_requirements=extra_req_param,
    ),
    *dependency_group_args,
    cwd=self.checkout_path,
    bin_path=self.venv_bin(),
)

As a minor note, I would also do a bit of refactoring to store all the args into the list and then splat all of them into run()

There Has To Be A Better Way (TM)

As Raymond H. always says, there has to be a better way. Turns out, the RTD team is already working on a more generic way to override the installation command via the configuration in #11710.

version: 2

build:
  jobs:
    install:
      - pip install --dependency-groups=tests,typing

This reduces the need to make potentially lots of update to the rigid code I worked on above. This makes me very happy!

I’ve received a lot of poorly written issues and requests in my open source work (see the PyKEEN issue tracker) so I thought it was incredibly important to write this issue very well. It occurred to me that the thought process here is also maybe generally interesting, so that’s why I copied it to my blog.

Building Graphviz when installing PyGraphviz

2024-11-05T06:48:00+00:00

Graphviz is software for graph visualization written in C. PyGraphviz provides a nice Python wrapper for it. The issue is that getting Python to know about the C headers changes every few months. I’ll try and keep this blog post updated every time there are some changes.

November 2024

These days, pip install graphviz seemed to work but uv pip install graphviz gave the following:

$ uv pip install pygraphviz
Using Python 3.12.7 environment at /Users/cthoyt/.virtualenvs/orcid
Resolved 1 package in 597ms
error: Failed to prepare distributions
  Caused by: Failed to download and build `pygraphviz==1.14`
  Caused by: Build backend failed to build wheel through `build_wheel` (exit status: 1)

[stdout]
running bdist_wheel
running build
running build_py
creating build/lib.macosx-14.0-arm64-cpython-312/pygraphviz
copying pygraphviz/scraper.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz
copying pygraphviz/graphviz.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz
copying pygraphviz/__init__.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz
copying pygraphviz/agraph.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz
copying pygraphviz/testing.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz
creating build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_unicode.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_scraper.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_readwrite.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_string.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/__init__.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_html.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_node_attributes.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_drawing.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_repr_mimebundle.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_subgraph.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_close.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_edge_attributes.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_clear.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_layout.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_attribute_defaults.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
copying pygraphviz/tests/test_graph.py -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz/tests
running egg_info
writing pygraphviz.egg-info/PKG-INFO
writing dependency_links to pygraphviz.egg-info/dependency_links.txt
writing top-level names to pygraphviz.egg-info/top_level.txt
reading manifest file 'pygraphviz.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'pygraphviz.egg-info/SOURCES.txt'
copying pygraphviz/graphviz.i -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz
copying pygraphviz/graphviz_wrap.c -> build/lib.macosx-14.0-arm64-cpython-312/pygraphviz
running build_ext
building 'pygraphviz._graphviz' extension
creating build/temp.macosx-14.0-arm64-cpython-312/pygraphviz
clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -DSWIG_PYTHON_STRICT_BYTE_CHAR -I/Users/cthoyt/Library/Caches/uv/builds-v0/.tmpKw7zNP/include -I/opt/homebrew/opt/python@3.12/Frameworks/Python.framework/Versions/3.12/include/python3.12 -c pygraphviz/graphviz_wrap.c -o build/temp.macosx-14.0-arm64-cpython-312/pygraphviz/graphviz_wrap.o

[stderr]
warning: no files found matching '*.swg'
warning: no files found matching '*.png' under directory 'doc'
warning: no files found matching '*.html' under directory 'doc'
warning: no files found matching '*.txt' under directory 'doc'
warning: no files found matching '*.css' under directory 'doc'
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '.svn' found anywhere in distribution
no previously-included directories found matching 'doc/build'
pygraphviz/graphviz_wrap.c:9:9: warning: 'SWIG_PYTHON_STRICT_BYTE_CHAR' macro redefined [-Wmacro-redefined]
#define SWIG_PYTHON_STRICT_BYTE_CHAR
        ^
:2:9: note: previous definition is here
#define SWIG_PYTHON_STRICT_BYTE_CHAR 1
        ^
pygraphviz/graphviz_wrap.c:3023:10: fatal error: 'graphviz/cgraph.h' file not found
#include "graphviz/cgraph.h"
         ^~~~~~~~~~~~~~~~~~~
1 warning and 1 error generated.
error: command '/usr/bin/clang' failed with exit code 1
  Caused by: This error likely indicates that you need to install a library that provides "graphviz/cgraph.h" for pygraphviz@1.14

Here’s one solution that work (of the many possible ones):

$ export CFLAGS="-I$(brew --prefix graphviz)/include"
$ export LDFLAGS="-L$(brew --prefix graphviz)/lib"
$ uv pip install pygraphviz
Using Python 3.12.7 environment at /Users/cthoyt/.virtualenvs/orcid
Resolved 1 package in 2ms
   Built pygraphviz==1.14
Prepared 1 package in 908ms
Installed 1 package in 1ms
 + pygraphviz==1.14

There’s also talk of using --global-option to pass -I/opt/homebrew/opt/graphviz/include and -L/opt/homebrew/opt/graphviz/lib, but I couldn’t figure this out for uv pip install.

2023 Era

I was on Python 3.10 on a M2 Mac with macOS 13. pygraphviz didn’t have a pre-built wheel for my systen so python -m pip install pygraphviz gave the following error:

$ python -m pip install pygraphviz
...
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk -DSWIG_PYTHON_STRICT_BYTE_CHAR -I/Users/cthoyt/.virtualenvs/indra/include -I/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c pygraphviz/graphviz_wrap.c -o build/temp.macosx-13-x86_64-cpython-310/pygraphviz/graphviz_wrap.o
      pygraphviz/graphviz_wrap.c:3020:10: fatal error: 'graphviz/cgraph.h' file not found
      #include "graphviz/cgraph.h"
               ^~~~~~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

I figured out based on the Homebrew troubleshooting in the docs (https://pygraphviz.github.io/documentation/stable/install.html#homebrew) that the following works:

python -m pip install --use-pep517 \
    --config-setting="--global-option=build_ext" \
    --config-setting="--build-option=-I$(brew --prefix graphviz)/include/" \
    --config-setting="--build-option=-L$(brew --prefix graphviz)/lib/" \
    pygraphviz

Some Haskell I Tried to Write

2024-09-26T10:49:00+00:00

I’m working through making a contribution to pandoc that adds first-class support for author role annotations using the Contribution Role Taxonomy (CRediT) and also outputs compliant Journal Publishing Tag Set (JATS) XML. This has lead me down a (losing) journey with learning the Haskell programming language, so I thought I would post a short note on a function I tried to understand.

For some context, the first stream of changes I sent were in jgm/pandoc #10153. I have done some in-place squashes on the git history, so apologies to future readers if this isn’t a helpful thread. I appreciated the help from Pandoc’s maintainer John MacFarlane, but he suggested I add the following code and I just don’t know enough Haskell to make sense of it:

addCreditName :: M.Map Text Text -> M.Map Text Text
addCreditName rolemap =
  case M.lookup "credit-name" rolemap of
    Just _ -> rolemap
    Nothing -> maybe id (M.insert "credit-name")
      (M.lookup "credit-id" rolemap >>= flip M.lookup creditNames)

The goal was actually pretty simple. I have a dictionary that maybe has a credit-name key. If it does, then we’re done. If not, and it has a credit-id key, get the value out of that and look up the credit-name using an external creditNames dictionary. The problem is, I can’t understand this without going on a massive deep-dive on Haskell to actually understand the way Haskell programs treat function calls, monads (which is a yo-dawg in the category of jokes), and function polymorphisms.

One thing to keep in mind is that Haskell is a functional programming language and everything is supposed to be immutable. This means that mapping insertion operations (M.insert) are returning a new dictionary. That made it possible to at least try a more verbose way of doing what I needed to.

I like Haskell’s case statement and first-class notion of optionals, so I thought I’d try re-writing this code from above in a bit more straight-forward way:

addCreditName :: M.Map Text Text -> M.Map Text Text
addCreditName role =
  -- Try looking if there's a "credit-name" key in the role dictionary
  case M.lookup "credit-name" role of
    -- If there's already an explicitly specified "credit-name"
    -- key in the role dictionary, then we don't have to do anything
    Just _ -> role
    Nothing ->
      case M.lookup "credit-id" role of
        -- If there isn't already a "credit-id" key in the role
        -- dictionary, then we aren't able to do anything
        Nothing -> role
        Just creditIdentifier ->
          -- Try looking up the value from the "credit-id" key, which
          -- we stored in the `creditIdentifier` variable, is in the
          -- creditNames dictionary, which is defined as a constant above
          case M.lookup creditIdentifier creditNames of
            -- If the credit-id value from the role dictionary is not
            -- in the creditNames lookup dictionary, then we can't do anything
            Nothing -> role
            -- If the credit-id value from the role dictionary is in
            -- the creditNames lookup dictionary, insert it back into the
            -- role dictionary under the "credit-name" key and return
            Just creditName -> M.insert "credit-name" creditName role

The funny thing is, I have been called out before on increasing confusion by getting tricky when writing Python code. I guess I had it coming!

Programmatic Access to a Wordpress User List

2024-09-20T10:31:00+00:00

The International Society of Biocuration (ISB) partners with the journal Database to get discounts for its members when they publish there. This means the ISB’s executive committee needs to send a member list to the journal’s editor. Historically, this has been done manually by exporting the list from the membership management plugin in the ISB Wordpress blog once per month and emailing it to th This post is about my journey trying to automate it

1. There must be an API for this

Wordpress has a programmatic API, and specifically an endpoint to list users.

After logging into the ISB’s Wordpress site, I was able to list users by navigating to the endpoint in browser https://biocuration.org/wp-json/wp/v2/users. Note: this page won’t work for you unless you’re on the EC and have admin powers. I wanted to replicate accessing this page through a Python script, so I was suggested by the official Wordpress documentation to create an application password and then use simple HTTP authentication. The docs said try this, after replacing the username with the Wordpress account (for me, that’s cthoyt) and the application password, which is a string with the form of XXXX XXXX XXXX XXXX XXXX XXXX. The documentation suggested doing the following curl command, which works:

curl --user "USERNAME:PASSWORD" https://biocuration.org/wp-json/wp/v2/users

If you want to translate this to Python, it looks like this.

import pystow, base64, requests

username = pystow.get_config("isb", "wordpress_username", raise_on_missing=True)
password = pystow.get_config("isb", "wordpress_password", raise_on_missing=True)

token = base64.b64encode(f"{username}:{password}".encode()).decode("utf-8")

res = requests.get(
    'https://www.biocuration.org/wp-json/wp/v2/users',
    headers={"Authorization": f"Basic {token}", "user-agent": "curl"},
)

Luckily, encoding the username/password string with basic authorization is such a common pattern that the request (and any other modern library) has a nice par for this.

import pystow, requests

username = pystow.get_config("isb", "wordpress_username", raise_on_missing=True)
password = pystow.get_config("isb", "wordpress_password", raise_on_missing=True)

res = requests.get(
    'https://www.biocuration.org/wp-json/wp/v2/users',
    auth=(username, password),
    headers={"user-agent": "curl"}
)

I burned a ton of time with this because it turns out that Wordpress blocks both the user agents for the request and httpx libraries. Therefore, you have to explicitly set the user-agent header to something else, or you get a HTTP 403 forbidden error.

2. The API doesn’t do what I wanted

It turns out that by default, all the users that never made a post get filtered out. This is bad since the only users who are making posts on the ISB are the handful who have or are currently on the executive committee.

So, obviously, the next step is to start injecting in PHP code to change how the API works. I found a comment from Tim Jensen, a Wordpress developer, that suggests adding the following hooks into the Wordpress theme’s functions.php file by navigating to https://www.biocuration.org/wp-admin/theme-editor.php?file=functions.php&theme=executive:

function remove_has_published_posts_from_api_user_query($prepared_args, $request)
{
    unset($prepared_args['has_published_posts']);
    return $prepared_args;
}
add_filter('rest_user_query', 'remove_has_published_posts_from_api_user_query', 10, 2);

This actually worked! But, it wasn’t the end of the story.

3. Full names are part of the plugin’s metadata

It turns out that the user data model isn’t all that comprehensive in Wordpress. What we really needed was the full name and email address for each person, and that was stored in an auxillary SQL table created and managed by the memberships plugin. Note that the plugin lives on top of Wordpress’s first-party users list, and doesn’t modify the API access to the user list.

So obviously, the next step was to add new API endpoints. I learned PHP at the ripe age of 14 and when I learned some other languages, I never looked back, so this wasn’t so easy. Plus, Wordpress has a huge set of functions and idioms on top. With a little help from ChatGPT, I was able to write a new API endpoint to add to functions.php that got users whose membership expiration date is in the future and joined in the metadata for their first and last names:

function export_pmp_members_to_csv() {
    global $wpdb;

    // Set headers to make the response downloadable as a CSV file
    header('Content-Type: text/csv');
    header('Content-Disposition: attachment; filename="members_export.csv";');
    header('Pragma: no-cache');
    header('Expires: 0');

    // Open output stream (in-memory file)
    $output = fopen('php://output', 'w');

    if (!$output) {
        return new WP_Error('file_error', 'Unable to generate CSV file.');
    }

    // Add CSV header row
    fputcsv($output, ['id', 'first', 'last', 'email', 'level', 'start', 'ends']);

    // Query to get PMP members and their membership details
    $results = $wpdb->get_results("
        SELECT
            u.ID,
            um_first_name.meta_value AS first_name,
            um_last_name.meta_value AS last_name,
            u.user_email, ml.name as membership_name,
            mu.startdate,
            mu.enddate
        FROM {$wpdb->users} u
        LEFT JOIN {$wpdb->prefix}pmpro_memberships_users mu ON u.ID = mu.user_id
        LEFT JOIN {$wpdb->prefix}pmpro_membership_levels ml ON mu.membership_id = ml.id
        LEFT JOIN {$wpdb->prefix}usermeta um_first_name ON u.ID = um_first_name.user_id
            AND um_first_name.meta_key = 'first_name'
        LEFT JOIN {$wpdb->prefix}usermeta um_last_name ON u.ID = um_last_name.user_id
            AND um_last_name.meta_key = 'last_name'
        WHERE mu.membership_id IS NOT NULL
            AND (mu.enddate > CURDATE() OR mu.enddate IS NULL)
    ");

    // Loop through results and write each row to the CSV
    foreach ($results as $row) {
        fputcsv($output, [
            $row->ID,
            $row->first_name,
            $row->last_name,
            $row->user_email,
            $row->membership_name,
            $row->startdate,
            $row->enddate
        ]);
    }

    // Close the output stream
    fclose($output);

    // Terminate script to ensure no extra output is sent
    exit;
}

function register_pmp_export_route() {
    // the actual path to the endpoint isn't important here
    register_rest_route('path/to/endpoint/', '/export-members', [
        'methods' => 'GET',
        'callback' => 'export_pmp_members_to_csv'
        'permission_callback' => function() {
            // Optionally, restrict access to logged-in users with specific capabilities
            return current_user_can('manage_options'); // Restrict to admin users
        }
    ]);
}
add_action('rest_api_init', 'register_pmp_export_route');

4. Automate it

Why stop at just being able to export the sheet? I wanted to go another mile and make sure that Wordpress sends an email to the right person at the journal on a monthly basis.

/ Function to generate and send the CSV via email
function send_pmp_members_csv_via_email() {
    global $wpdb;

    // Create a temporary file to store the CSV
    $tmp_file = tempnam(sys_get_temp_dir(), 'pmp_csv_') . '.csv';
    $output = fopen($tmp_file, 'w');

    if (!$output) {
        return; // Handle error appropriately
    }

    // Add CSV header row
    fputcsv($output, ['ID', 'Username', 'First Name', 'Last Name', 'Email', 'Membership Level Name', 'Joined Date', 'Expires Date']);

    // Query to get PMP members and their membership details
    $results = $wpdb->get_results("
        SELECT u.ID,
            u.user_login,
            um_first_name.meta_value AS first_name,
            um_last_name.meta_value AS last_name,
            u.user_email,
            ml.name as membership_name
            mu.startdate,
            mu.enddate,
        FROM {$wpdb->users} u
        LEFT JOIN {$wpdb->prefix}pmpro_memberships_users mu ON u.ID = mu.user_id
        LEFT JOIN {$wpdb->prefix}pmpro_membership_levels ml ON mu.membership_id = ml.id
        LEFT JOIN {$wpdb->prefix}usermeta um_first_name ON u.ID = um_first_name.user_id
            AND um_first_name.meta_key = 'first_name'
        LEFT JOIN {$wpdb->prefix}usermeta um_last_name ON u.ID = um_last_name.user_id
            AND um_last_name.meta_key = 'last_name'
        WHERE mu.membership_id IS NOT NULL
        AND (mu.enddate > CURDATE() OR mu.enddate IS NULL)
    ");

    // Write data to CSV
    foreach ($results as $row) {
        fputcsv($output, [
            $row->ID,
            $row->user_login,
            $row->first_name,
            $row->last_name,
            $row->user_email,
            $row->membership_name,
            $row->startdate,
            $row->enddate
        ]);
    }

    // Close the file
    fclose($output);

    // Prepare email
    $to = ''; // Replace with recipient email
    $subject = 'ISB Monthly Members Report';
    $message = 'Please find the attached CSV file containing the list of active members of the International Society of Biocuration as of today.';
    $headers = array('Content-Type: text/csv; charset=UTF-8');

    // Send email with attachment
    wp_mail($to, $subject, $message, $headers, $tmp_file);

    // Clean up temporary file
    unlink($tmp_file);
}

// Schedule the cron job to run monthly
function schedule_monthly_csv_email() {
    if (!wp_next_scheduled('monthly_pmp_csv_email')) {
        wp_schedule_event(time(), 'monthly', 'monthly_pmp_csv_email');
    }
}
add_action('wp', 'schedule_monthly_csv_email');

// Hook the function to the cron event
add_action('monthly_pmp_csv_email', 'send_pmp_members_csv_via_email');

Hopefully I don’t have to write anymore PHP for a long time :)

Easier ORCID

2024-06-08T11:33:00+00:00

The Open Researcher and Contributor Identifier (ORCID) database is an invaluable resource that supports the unambiguous identification of researchers. However, its first party data dump is too complex, verbose, and unstandardized for many use cases. This post describes open source software I wrote that automates downloading, processing, and exporting ORCID into a more usable form. I put the results on Zenodo under the CC0 license.

Challenges to Overcome

ORCID currently contains on the scale of tens of million records, meaning that there isn’t a reasonable way to access the data in bulk via its public API. As an alternative, ORCID dumps its public content once per year on FigShare. The most recent (2023) dump is available at doi:10.23640/07243.24204912. Previous versions are deposited with under different DOIs:

2022 ORCID Public Data File (https://doi.org/10.23640/07243.21220892.v4)
2021 ORCID Public Data File (https://doi.org/10.23640/07243.16750535.v1)
2020 ORCID Public Data File (https://doi.org/10.23640/07243.13066970)
…

Unfortunately, this arrangement makes it difficult to automatically discover new versions without writing software that searches FigShare programmatically and has a heuristic for guessing which might be a newer record. I don’t have a solution for this yet, but I can imagine one.

Only making a yearly dump means that the derived artifacts can become out of date quickly. Other large databases like PubChem make monthly and nightly dumps on their FTP servers which are deleted when no longer relevant. For example, monthly dumps from more than a year ago can be safely deleted and nightly dumps only need to exist until they are replaced by the next one. Since ORCID is using FigShare as an archival system, this would be a disk space-intensive operation. Using the ORCID API or secondary data distribution via Wikidata could be good stopgaps for consumers who want the most recent data.

ORCID distributes its data as XML. They also provide Java software to convert it to JSON, but since 2018 are pretty adamant about not running this software and distributing the JSON artifact themselves. This is unfortunate, since XML is hugely verbose both in terms of the way that data gets structure and the markup itself. Further, the data structure contains a huge amount of provenance information that isn’t useful for many downstream consumers (both in terms of when things were updated, by whom, and which API endpoint could be used to update it further). An example from the JSON converter library also illustrates that converting from XML to JSON accomplish some obvious simplifications that most users would want.

Another tricky thing about consuming the ORCID data is that the summary file that contains all the records is 32 gigabytes compressed and has a very strange internal file structure. This means that you either have to uncompress it, which takes a long time with its tens of millions of files, or iterate through the file handles inside it. I also haven’t figured out a good way to open a specific file inside the compressed archive beyond iterating through all the handles. The file names themselves are also difficult to guess because of the directory structure used.

Solutions

I wrote a Python package, orcid_downloader that can automatically download the right file from FigShare, iterate through the individual compressed XML files for each record, and process them. The package can be used to iterate over records and process them in your own way like:

import orcid_downloader

for record in orcid_downloader.iter_records():
    ...

The main goal of this was also to create a simplified version of the ORCID data dump that is more straightforwards, accessible, and smaller. I would imagine most people would be interested in just downloading the results instead of rebuilding them, so the results of this process are posted on Zenodo at doi:10.5281/zenodo.10137939. It uses Zenodo’s versioning system to make sure that all different versions (both from updates to the yearly dump or improvements to the processing pipeline) are all in the same Zenodo record.

So far, this includes the name, aliases, external identifiers, employers, education, and publications linked to PubMed. Along the way, I realized that ORCID did not consistently ground educational institutions to the Research Organization Registry (ROR) like it did for employers. I also had it double-check all groundings for employers, since these were incomplete. I also did some minor string processing, such as standardization of education types (e.g., Bachelor of Science, Master of Science), standardization of PubMed references, and standardization of aliases (e.g., pruning off titles like Dr.)

The records.jsonl.gz file is a JSON Lines file where each row represents a single ORCID record in a simple, well-defined schema (see schema.json). Here are a few rows ( expanded for viewing comfort) as example:

{
  "orcid": "0000-0001-5045-1000",
  "name": "Patricio Sánchez Quinchuela",
  "employments": [
    {
      "name": "Universidad de las Artes",
      "start": 2021,
      "role": "Especialista de Proyectos y docente",
      "xrefs": {
        "ror": "016drwn73"
      }
    },
    {
      "name": "Universidad Regional Amazónica IKIAM",
      "start": 2019,
      "end": 2021,
      "role": "Director",
      "xrefs": {
        "ror": "05xedqd83"
      }
    }
  ],
  "educations": [
    {
      "name": "Universidad Nacional de Educación a Distancia Facultad de Ciencias Políticas y Sociología",
      "start": 2020,
      "role": "Doctorando del Programa de Sociología",
      "xrefs": {
        "ringgold": "223339"
      }
    }
  ]
}
{
  "orcid": "0000-0001-5101-6000",
  "name": "Céline LEPERE"
}
{
  "orcid": "0000-0001-5001-3000",
  "name": "Vincent Nguyen",
  "employments": [
    {
      "name": "Troy High School",
      "start": 2020,
      "role": "Student",
      "xrefs": {
        "ringgold": "289570"
      }
    }
  ]
}
{
  "orcid": "0000-0001-5002-1000",
  "name": "Sameer Abbood",
  "employments": [
    {
      "name": "University of Al-Ameed",
      "role": "Doctor of Philosophy"
    }
  ]
}

Many records in ORCID are relatively unhelpful, i.e., ones that only have a name and no other (public) information. Therefore, I created a high-quality subset that only contains records with at least one ROR-grounded employer, at least one ROR-grounded education, or at least one publication indexed in PubMed. The point of this subset is to remove ORCID records that are generally not possible to match up to any external information. It is listed in the same Zenodo record as records_hq.jsonl.gz.

External Identifiers

An ORCID record has two places that make cross-references to external nomenclature authorities:

The “Websites & Social Links” section which allows a researcher to give a link with a name. This is a trove of links to LinkedIn, Google Scholar, GitHub, and other external identifiers. ORCID itself doesn’t standardize them, but using a combination of the Bioregistry and custom parsing, many can be standardized.
The “Other IDs” section is generated based on applications that connect to ORCID and send structured links. This includes Scopus, Web of Science (formerly ResearcherID), Loop, and some others. This also needs quite a bit of standardization, probably due to a combination of bugs in ORCID, bugs in external services, legacy data, and other things.

Reflections

I found several interesting things while parsing these sections:

I discovered several new nomenclature authorities that weren’t already registered in the Bioregistry. This includes Loop, Digital Author ID (Dutch), Authenticus (Portuguese), Dialnet (Spanish), SciProfiles, Ciência (Portuguese), and KAKEN (Japanese). I still have to send new prefix requests for these.
I found several fields that were totally junk or didn’t make sense. For example, there is sometimes a reference to the ORCID record itself. There are also references to external IDs that aren’t really IDs, or at least don’t follow enough of the guidelines from Identifiers in the 21^st Century to be useful.
Both the “Websites & Social Links” and “Other IDs” needed standardization. In some places, this was as easy as using the Bioregistry prefix standardization, but in other places required more custom URL parsing. This is especially true for Google Scholar, which can appear with a number of domain names (e.g., https://scholar.google.com or https://scholar.google.es). The local unique identifier appears here inside the URL parameters, which can be in any order along with the language tag, so this needs URL parsing instead of more simple URI prefix handling a la the curies Python Package.

Summary

Here’s a breakdown of the top external cross-references, standardized with the Bioregistry (where possible). Note that this was prioritized by the most common cross-references, and is not complete. To capture all would be a lot of work and require many more corner cases to less common services. I also threw away links to non-professional social networks like Facebook/Instagram. I also made the value judgement to throw away links to Twitter since it doesn’t reflect open and inclusive scientific community values anymore.

Resource	Count
`scopus`	1,400,735
`wos.researcher`	608,543
`sciprofiles`	259,654
`loop`	229,224
`linkedin`	191,321
`researchgate`	125,242
`google.scholar`	52,131
`github`	13,397
`gnd`	7,621
`isni`	4,105
`dai`	1,982
`authenticus`	1,422
`dialnet`	1,210
`wikidata`	147

The results of this process are first available as part of the records file(s). Second, they are available through a dedicated file (sssom.tsv.gz) in the Simple Standard for Sharing Ontological Mappings (SSSOM) that solely focuses on the cross-references. Here’s what the first few lines of that file look like:

subject_id	subject_label	predicate_id	object_id	mapping_justification
orcid:0000-0001-5099-6000	Debashis Bhowmick	skos:exactMatch	scopus:57214299968	semapv:ManualMappingCuration
orcid:0000-0001-5009-9000	Ali Gargouri	skos:exactMatch	loop:470724	semapv:ManualMappingCuration
orcid:0000-0001-5084-9000	Luana Licata	skos:exactMatch	loop:1172627	semapv:ManualMappingCuration
orcid:0000-0001-5084-9000	Luana Licata	skos:exactMatch	scopus:6603618518	semapv:ManualMappingCuration
orcid:0000-0001-5124-3000	Wojciech Nawrocki	skos:exactMatch	loop:661557	semapv:ManualMappingCuration
orcid:0000-0001-5075-0000	Xueyong Pang	skos:exactMatch	wos.researcher:K-6721-2018	semapv:ManualMappingCuration
orcid:0000-0001-5103-2000	Bartlomiej Dec	skos:exactMatch	scopus:57194469902	semapv:ManualMappingCuration
orcid:0000-0001-5020-8000	Sapna Gambhir	skos:exactMatch	scopus:35811915000	semapv:ManualMappingCuration
orcid:0000-0001-5074-2000	Martin Perez-Santos	skos:exactMatch	scopus:56082352000	semapv:ManualMappingCuration

What are Cross-References Useful For?

There are many different nomenclature authorities because each have different goals and data models associated with records. Different communities also value different nomenclature authorities differently. For example, life scientists are more often using ORCID when publishing, but including persistent identifiers in publications is not yet common in computer science papers. Further, computer scientists more often link to their DBLP, arXiv, OpenReview, or other computer-science focused pages.

When assembling data and knowledge from more than a single resource, it’s important to resolve the identifiers used for researchers to a single identifier - it’s not good if different knowledge is connected to an ORCID and a DBLP for a single researcher. This can be resolved using a combination of semantic mappings (i.e., cross-references) and software for the large-scale automated assembly of mappings such as the Semantic Mapping Reasoning Assembler (SeMRA). It allows for specifying a priority list of nomenclature authorities to assemble coherent knowledge from ORCID and other sources simultaneously.

This is also a much more valuable process when combining other mapping resources. Wikidata has put a lot of effort into capturing bibliographic metadata, especially to support the Scholia project. The following SPARQL query against Wikidata shows that there are 1,811,573 (about 10% of all ORCID records) links from Wikidata entries to ORCID as of June 2024. Run it yourself.

SELECT (count(DISTINCT ?orcid) as ?total)
WHERE { ?item wdt:P496 ?orcid }

OpenCheck tried to create mappings between Twitter, GitHub, and ORCID using their APIs, but became defunct when Twitter shut off their public APIs. OpenAlex, Microsoft Academic Graph, Open Academic Graph, and other bibliographic knowledge graphs all have to address this problem internally, as well. Other mapping resources probably exist, please let me know if you’re aware of other ones!

Affiliations

ORCID breaks up affiliations into several blocks: educations, employments, invited positions, etc. I’m focusing on educations and employments here. The processing code could be extended for the others later.

Employments contained ROR references but educations didn’t, and even when they’re available, they’re incomplete. PyOBO has implemented a loader for ROR. Any resource loaded through PyOBO can also be used for named entity normalization through an interface to Gilda. This allowed for grounding of a large number of missing education and employer entries. Here’s the total number of cross-references made from educations and employments to ROR and other nomenclature authorities.

Resource	Count
`ror`	8,725,215
`ringgold`	6,614,519
`grid`	1,404,183
`funderregistry`	698,384
`lei`	1,206

Education Roles

Both educations and employments also have associated roles. These do not use a controlled vocabulary, but there are a number of patterns that could be standardized. This task had to be split between education and employment entries. I just focused on education entries for now. The “role” field in each education entry corresponds to the degree.

I started by looking for existing resources (both structured and unstructured) that have lists of degrees. Here are a few things I found:

https://degree.studentnews.eu lists degrees conferred in the EU/Europe
https://github.com/vivo-ontologies/academic-degree-ontology is an incomplete/abandoned effort from 2020 to ontologize degree names
Wikidata has a class for academic degree https://www.wikidata.org/wiki/Q189533. Its SPARQL query service _ can be queried with the following, though note that the Wikidata class hierarchy is broken in several places.
```
   SELECT ?item ?itemLabel WHERE {
      ?item wdt:P279* wd:Q189533 .
      SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
    }
```

In the end, these were either incomplete or not organized well enough to directly use. It also turns out that people often conflate the degree (e.g., Master of Science) with the field that it’s in (e.g., Chemistry) or include some combination (e.g., Master of Science in Chemistry). This meant that a lot of string processing and heuristics would be required on top of lexical approaches. Instead, I took the tried-and-true method of listing the strings by frequency and just curating from the top. The results are in https://github.com/cthoyt/orcid_downloader/blob/851af81d8aacebf2768bfc591080cbceab2047cc/src/orcid_downloader/standardize.py. Of course, this is incomplete in many ways, and could be extended to capture further cases. I also found that there are a huge number of Spanish and Portuguese entries that I needed help from my international friends to get the best translations (since the meaning is pretty subtle for many). Further, the results would be more useful as a proper ontology that could extend and replace VIVO’s Academic Degree Ontology. I’ll leave this for future work. Here’s a summary of the most frequent roles that have been standardized so far:

Education Role	Count
Doctor of Philosophy	1,231,845
Master	426,625
Bachelor	402,297
Master of Science	360,052
Bachelor of Science	322,853
Doctor of Medicine	202,884
Bachelor of Arts	108,778
Master of Arts	78,601
Postdoctoral Researcher	37,548
Bachelor of Medicine, Bachelor of Surgery	27,722
Master of Technology	22,473
Bachelor of Technology	21,465
Bachelor of Engineering	21,345
Diploma	20,695
Master of Business Administration	19,660
Master of Philosophy	18,069
Master of Education	18,055
Master of Public Health	17,760
Bachelor of Education	17,376
Master of Engineering	14,260

Authorship

Authorships are extracted and standardized in the pubmeds.tsv.gz file, which contains an ORCID column and PubMed column that has been pre-sanitized to only contain local unique identifiers. This information is also available through the main records file.

While the field inside the XML data is supposed to contain local unique identifiers, there was a huge variety of what actually showed up there. This included local unique identifiers (i.e., 36402838), local unique identifiers with junk attached (e.g., 36402838/), valid Compact URIs (CURIEs), invalid CURIEs, URIs, free text that’s totally irrelevant. Overall, there were 3,175,196 (99.85%) that were valid local unique identifiers, 2,832 (0.09%) that were able to be cleaned up, and 2,080 (0.07%) that were junk and couldn’t be salvaged. Inside the junk were a few things:

DOIs
PMC identifiers,
a few stray strings that contain a combination of pubmed, PMC, and DOIs
a lot with random text (keywords)
some with full text citations

Later, other identifier types could be added in here too.

Lexical Indexing

One of the original goals of processing ORCID in bulk was to ground and disambiguate author lists in publications. Therefore, I made two pre-built Gilda indexes for named entity recognition (NER) and named entity normalization (NEN). One contains all records, and the second is filtered to high-quality records. The following Python code snippet can be used for grounding:

from gilda import Grounder

url = "..."
grounder = Grounder(url)
results = grounder.ground("Charles Tapley Hoyt")

The ORCID downloader also has its own extension that does a smarter job with caching and some clever name preprocessing

import orcid_downloader

results = orcid_downloader.ground_researcher("Charles Hoyt")

Ontology Artifact

The file orcid.ttl.gz is an OWL-ready RDF file that can be opened in Protégé or used with the Ontology Development Kit. It can also be converted into OWL XML, OWL Functional Notation, or other OWL formats using ROBOT. This artifact can serve as a replacement for the ones generated by https://github.com/cthoyt/orcidio, which was a smaller-scale way of turning ORCID records for contributors to OBO Foundry ontologies into a small OWL file. Now, the export here contains all ORCID records with names.

@prefix orcid:  .
@prefix rdfs:  .
@prefix oboInOwl:  .
@prefix owl:  .
@prefix skos:  .
@prefix human:  .

human: a owl:Class ;
    rdfs:label "Homo sapiens"^^xsd:string .

orcid:0000-0001-5000-5000 a human: ;
   rdfs:label: "Joel Adam Gordon"^^xsd:string ;
   oboInOwl:hasExactSynonym "Joel Gordon"^^xsd:string .

orcid:0000-0001-5099-6000 a human: ;
   rdfs:label: "Debashis Bhowmick"^^xsd:string ;
   oboInOwl:hasExactSynonym "Bhowmick D. S."^^xsd:string ;
   oboInOwl:hasExactSynonym "D. S. Bhowmick"^^xsd:string ;
   oboInOwl:hasExactSynonym "Debashis S Bhowmick"^^xsd:string ;
   skos:exactMatch "scopus:57214299968" .

orcid:0000-0001-5084-9000 a human: ;
   rdfs:label: "Luana Licata"^^xsd:string ;
   skos:exactMatch "loop:1172627"^^xsd:string ;
   skos:exactMatch "scopus:6603618518"^^xsd:string .

It’s still TBD on the best way to encode the cross-references.

Code

The artifacts described here were all automatically generated with code in https://github.com/cthoyt/orcid_downloader.

Like I mentioned a few times throughout, this is a work in progress. Doing practical data science is hard work, and there is a lot of room for improvement. I’m still recovering from burnout, so working at a slow pace only when I felt inspired also was fine for me. I know there are lots of things I would like to improve given more motivation, but that will have to wait for now.

Discussions and Follow-ups from Biocuration 2024

2024-03-11T13:52:00+00:00

I’ve just returned from the 17^th Annual International Biocuration Conference at the Indian Biological Data Centre (IBDC) in Faridabad, India. I wanted to highlight some of the interesting conversations I had while I was there, and ideas for follow-up. Most were centered around the Bioregistry and the Semantic Mapping Assembler and Reasoner (SeMRA), which I gave an oral presentation on.

I talked to Guy Cochrane and Chuck Cook from the Global Biodata Coalition (GBC). They chaired a session on sustainability of biocurated resources, with specific focus on the Global Biodata Coalition’s Global Biodata Core Resources (GBCR) initiative. I felt like my talk from last year’s biocuration conference on the Open Code, Open Data, Open Infrastructure (O3) roadmap (preprint) would have fit right in here. I am very keen to have their perspectives, as GBC has first worked on evaluation of resources and is second working towards funding resources. Since they have not worked on practical recommendations for supporting sustainability, I eagerly volunteered to join their work in some capacity to help advise on this.

GBC also published a workflow for evaluating the landscape of biological databases (press release / publication / code). When possible, this workflow aligned on FAIRsharing, but given that it is a limited resource and only has partial mappings to relevant related resources like re3code, BARTOC, etc. I suggested using the Bioregistry as a mapping hub to enrich the output of this workflow, which will definitely be run again on a periodic basis.

Lynn Schriml presented recent updates on the Disease Ontology, which prompted a relevant question from Harpreet Singh - Chief Data Officer at the Indian Council of Medical Research (ICMR) who himself works with clinical data and has wondered how to best annotate - using MeSH, SNOMED, ICD, or other disease resources. I had an interesting discussion with him following the talk which gave big motivation to the talk I was about to give on the large scale assembly and reasoning over semantic mappings. I was very excited, since I love to add (last minute) shout-outs into my conference talks that motivate parts of the work based on questions or discussions from earlier parts of the conference.

There were a series of talks that motivated further discussions about mappings. One of the most interesting was the talk from Shivani Sharma, a curator at the Indian Biological Data Centre (IBDC) and one of the local organizers. She works on the Indian Metabolome Data Archive. Many of the lines of work at the IBDC have practical applications towards agriculture and integrate medium- and large-scale experimental work, biocuration, and downstream analysis. Often, these applications are oriented towards improving crop yields and avoiding disease. Shivani showed a slide where they considered a large number of metabolomics nomenclature resources to use for annotating their data. However, they were not familiar with methods for incorporating multiple nomenclature resources, meaning that their curators were running into issues where their chosen metabolomics database did not cover chemicals they needed to annotate. This often lead to them having to create their own ad hoc annotations, which also create issues for data integration. I am looking forwards to catching up with them again, incorporating new metabolomics resource into PyOBO, ingesting mappings into SeMRA, and filling in the gaps using Biomappings to support their curators.

Scott V. Nguyen from the American Type Culture Collection (ATCC) also approached me about this work, since he’s currently trying to curate mappings between cell lines in their resource and other public resources. It was lucky that one of the examples from my talk was specifically about the cell lince scenario, which I hope he can ingest and reduce his curation workload. Rachel Lyne also presented on COSMIC, a cancer cell line resource that also creates its own accession numbers and could benefit from this work, but I didn’t get a chance to talk about it with her yet.

I also met Yasunori Yamamoto, who works on TogoID, a secondary database of semantic mappings that covers select domains within biomedicine. We discussed how they could make use of the Simple Standard for Sharing Ontology Mappings (SSSOM) to ingest more mappings from different resources, especially from Biomappings or potentially from the outputs of SeMRA (which I presented on).

Matt Jeffreys presented on the annotations database in European PubMed Central which allows for tagging articles, sentences, or tokens in articles with annotations. They already showed how this applies to named entity recognition (NER) and MeSH term annotations, but we discussed how SeMRA and comprehensive semantic mapping databases could help unify other annotations of overlapping vocabularies, e.g., if someone put Disease Ontology (DO) NER annotations, which overlap with MeSH terms in the disease (C) and psychiatric disorders (F) branches.

I discussed with Raja Mazumdar and Jeet Vora from George Washington University who both work on GlyGen and are plugged into the NIH’s Common Fund Data Ecosystem (CFDE) about how they can continue to use the Bioregistry to standardize the annotations in their resources. Jeet has got in touch earlier this year and helped update the records in the Bioregistry related to GlyGen. Raja’s talk also motivated two new prefix additions to the Bioregistry for Biocompute Objects and for OncoMX data objects. Further, Raja is very interested in improving his data using the Bioregistry, since it already uses a Python script to validate its JSON and TSV components, it will be easy to incorporate the Bioregistry Python package’s validation functions.

Earlier this winter, I presented to the American National Institutes of Health (NIH) BISTI group about different avenues through which they could use the Bioregistry to create more value for the NIH and its grantees. One of those discussions was about improving GenBank’s internal database catalog. By chance, I talked with Ilene Karsch Mizrachi, a program head at the NIH about this. She was attending the conference and made big contributions to the discussions about the Indian relation to the International Nucleotide Sequence Database Collaboration (INSDC). However, it turns out she was the one who made/contributed to this GenBank table, many years ago. We will try and follow up by enriching this table with information from the Bioregistry.

At last year’s biocuration conference, Chris Hunter presented on GigaDB, and we had some initial discussions about using the Bioregistry (or other related parts of the Biopragmatics Stack) to make standardized annotations on data sets deposited in their database, such as cell line annotations. We picked back up that conversation, and it seems that the GigaDB developers are working with PHP - since we got CZI funding to make the Bioregistry available in other languages, making a wrapper from Rust to PHP (within the curies.rs framework).

There was an entire session on the final day of the conference on structural bioinformatics, which included several presentations from the American and European loci of the Protein Databank (PDB). The first discussion was with Marcus Bage, who is currently trying to annotate protein modifications. We discussed the implications of the vast number of resources that partially cover this domain in different senses, including GO, MOD, SBO, MOP / PSI-MI, and UniProt’s internal vocabulary. A long time ago, I mapped these together in PyBEL, but this was only a partial solution, too!

The second discussion was with Brinda Vallat about the upcoming change for PDB accession numbers. It turns out that the 4 character code is estimated to fill up in 2029, so it’s time for PDB to make a change. Unfortunately, their solution is to switch to local unique identifiers that look like pdb_000002GC4, which is problematic for two main reasons. First, it’s not backwards compatible with existing IDs. Second, it introduces a banana (i.e., a redundant copy of the name/acronym of the database in the local unique identifier). The reasoning behind adding in the banana was to make it easier to find references in papers. I can understand this, since we don’t yet have general solutions for referencing concepts across different publishers (though, we solved this in Manubot by integrating the Bioregistry). However, this increases confusion for consumers. I suggested they simply extend the existing IDs to be able to have more than 4 characters, and suggest people reference their entities with CURIEs like PDB:2GC4 within papers, which solves both issues simultaneously. Similarly, I talked to Ibrahim Roshan Kunnakkattu about creating more careful identifier recommendations for the PDB’s Chemical Component Dictionary as well as using some of the automated mapping tools I presented for filling out references to ChEBI, ChEMBL, PubChem, and more.

I also had the unique pleasure to spend time in person with Tiago Lubiana, who is highly aligned on many of my interests in data standardization, semantic web, and open science. He has been a helpful contributor in the Bioregistry, Wikidata, and the OBO Foundry. Writing up some of the things we discussed would take a whole blog post, so instead, here’s a nice picture we got together.

Overall, like every Biocuration conference, I was very happy to find people interested in my work, and more importantly, interested in the idea of improving their own data standardization! I also had lots of other interesting discussions that don’t require any follow-up. I am also planning on writing a post that gives a more high-level summary of the different parts of the conference itself, not just focusing on my work.

Semantic Pydantic

2024-01-10T13:38:00+00:00

Using Pydantic for encoding data models and FastAPI for implementing APIs on top of them has become a staple for many Python programmers. When this intersects with the semantic web, linked open data, and the natural sciences, we are still lacking a bridge to annotate our data models and APIs to make them more FAIR (findable, accessible, interoperable, and reusable). In this post, we build an extension to Pydantic and FastAPI to annotate data models’ fields and API endpoints’ query, path, and other parameters using the Bioregistry, a comprehensive catalog of metadata about semantic spaces from the semantic web and the natural sciences.

As a demonstration, we will build a data model and API that serves information about scholars.

First Steps with Pydantic

We’ll use Open Researcher and Contributor (ORCID) identifiers as primary keys, include the researcher’s name, and start with a single cross-reference, e.g., to the author’s DBLP identifier. We’ll encode this data model using Pydantic in the Python programming language as follows:

from pydantic import BaseModel, Field

class ScholarV1(BaseModel):
    """A model representing a researcher, who might have several IDs on different services."""

    orcid: str = Field(...)
    name: str = Field(...)
    dblp: str | None = Field(None)

print(ScholarV1.schema_json(indent=2))

JSON Schema - Version 1

Books I Read in 2023

2024-01-01T21:22:00+00:00

I finally got back into reading! Over winter break 2022, I started the Stormlight Archive then followed up in 2023 by reading the entirety of Brandon Sanderson’s Cosmere, as well as a some other fantasy, science fiction, and literary fiction. Here’s the list.

Oathbringer (Stormlight Archive #3) by Brandon Sanderson
Rhythm of War (Stormlight Archive #4) by Brandon Sanderson
The Final Empire (Mistborn #1) by Brandon Sanderson
The Well of Ascension (Mistborn #2) by Brandon Sanderson
The Hero of Ages (Mistborn #3) by Brandon Sanderson
The Alloy of Law (Mistborn #4) by Brandon Sanderson
Shadows of Self (Mistborn #5) by Brandon Sanderson
The Bands of Mourning (Mistborn #6) by Brandon Sanderson
Warbreaker by Brandon Sanderson
Elantris by Brandon Sanderson
The Shadow of the Wind (The Cemetery of Forgotten Books #1) by Carlos Ruiz Zafón
The Angel’s Game (The Cemetery of Forgotten Books #2) by Carlos Ruiz Zafón
The Emperor’s Soul by Brandon Sanderson
The Hope of Elantris by Brandon Sanderson
The Eleventh Metal by Brandon Sanderson
Mistborn: Secret History by Brandon Sanderson
Shadows for Silence in the Forests of Hell by Brandon Sanderson
Sixth of Dusk by Brandon Sanderson
White Sands by Brandon Sanderson
The Lost Metal (Mistborn #7) by Brandon Sanderson
Edgedancer (Stormlight Archive #2.5) by Brandon Sanderson
What It Means When a Man Falls from the Sky by Lesley Nneka Arimah
Tress of the Emerald Sea by Brandon Sanderson
The Prisoner of Heaven (The Cemetery of Forgotten Books #3) by Carlos Ruiz Zafón
The Name of the Wind (The Kingkiller Chronicle #1) by Patrick Rothfuss
The Wise Man’s Fear (The Kingkiller Chronicle #2) by Patrick Rothfuss
The Labyrinth of Spirits (The Cemetery of Forgotten Books #4) by Carlos Ruiz Zafón
Yumi and the Nightmare Painter by Brandon Sanderson
Hyperion (The Hyperion Cantos #1) by Dan Simmons
The Fall of Hyperion (The Hyperion Cantos #2) by Dan Simmons
Dawnshard (Stormlight Archive #3.5) by Brandon Sanderson
The Sunlit Man by Brandon Sanderson
All Systems Red (Murderbot Dairies #1) by Martha Wells
Endymion (The Hyperion Cantos #3) by Dan Simmons
The Rise of Endymion (The Hyperion Cantos #4) by Dan Simmons
A Memory Called Empire (Teixcalaan #1) by Arkady Martine
A Desolation Called Peace (Teixcalaan #2) by Arkady Martine
Crime and Punishment by Fyodor Dostoevsky
The Strange Case of Dr. Jekyll and Mr. Hyde by Robert Louis Stevenson
Anna Karenina by Leo Tolstoy
Red Rising (Red Rising Saga #1) by Pierce Brown
Golden Son (Red Rising Saga #2) by Pierce Brown
This is How You Lose the Time War by Amal El-Mohtar and Max Gladstone

Cosmere moments I really enjoyed (spoilers):

Every single oath (finished or not), particularly Elhokar and Eshonai
I guessed the twist in Tress of the Emerald Sea
Kelsier’s not dead!
Dalinar Kholin’s entire character arc
Wayne, Pattern, and Nightblood being hilarious
Michael Kramer and Kate Reading’s voices
All the tidbits in Sunlit Man… I can’t wait for Stormlight 5 to fill in some of the gaps.

Other non-Cosmere highlights (spoilers):

Narrator reveal at the end of The Labyrinth of Spirits
Anna Karenina’s inner monologue as she commits suicide was devastating
Rothfuss’s prose is like honey
The messiah arc in the second Hyperion duology is really nice. It’s a much better version of the awful “love connects the universe” cliché at the end of Interstellar that actually creates plot- and character-driven relevance.

Disappointments:

Knowing the Doors of Stone might never come out but reading Kingkiller Chronicle #1 and #2 anyway
I didn’t like that All Systems Red had the entire plot get resolved off-screen. My sisters were really hoping I’d like this book, and I was only neutral. I’ll still give the next one a try.
I missed the Year of Sanderson Kickstarter (and the hilarity of the announcement video). I hadn’t really started until after.
The Russian classics were pretty slow, but I guess that’s how it goes.

My goal in 2024 is to read more books from different genres, especially ones I’ve never touched before.

Unlocking UMLS

2023-09-01T08:00:00+00:00

The Unified Medical Language System (UMLS) is a widely used biomedical and clinical vocabulary maintained by the United States National Library of Medicine. However, it is notoriously difficult to access and work with due to licensing restrictions and its complex download system. In the same vein as my previous posts about DrugBank and ChEMBL, this post describes open source software I’ve developed for downloading and working with this data. It also works for RxNorm, SemMedDB, SNOMED-CT, and any other data accessible through the UMLS Terminology Services (UTS) ticket granting system.

The first big issue with the UMLS is its licensing. Here’s an excerpt from the How to License and Access the Unified Medical Language System® (UMLS®) Data page accessed on August 28^th, 2023:

Please sign up for a new UMLS Terminology Services (UTS) account with your preferred identity provider at the UTS

 homepage.

Complete and submit the license request form. NLM will send the license approval e-mail within 5 business days

 after reviewing your authenticated license request.

You will sign in using identity provider credentials to download files or access web interfaces that require UTS

 authentication such as the UTS, VSAC, SNOMED CT, or RxNorm.

These are a few big hurdles:

We typically expect scientific data to be available for download without login. Specifically, most data can be downloaded by following a link that points directly to a file. For example, ChEMBL v33 can be downloaded as a gzipped SQLite file from https://ftp.ebi.ac.uk/pub/databases/chembl/ChEMBLdb/latest/chembl_33_sqlite.tar.gz. Rather than providing a data download link, UMLS, has a complicated API called the UMLS Terminology Services (UTS) ticket granting system that needs to be asked for a specific file, polled for a unique access key, then downloaded via an ephemeral (i.e., disappearing) URL that only works once.
We typically expect scientific data to be licensed under a standard, widely used license such as those from Creative Commons. Using well-understood licenses reduces the cognitive and legal burden of consumers when deciding if and how they can reuse, modify, or redistribute data. UMLS uses a non-standard terms of service that makes it more difficult to understand how the data can be stored, modified, or redistributed.
Further, we hope that data is permissively licensed such that it can be re-used, modified, and re-distributed. The Creative Commons CC BY 4.0 and CC0 licenses are golden standards of permissibility. UMLS does not have a permissive license, meaning (from my best interpretation) that you can not redistribute UMLS and you can (probably) not redistribute data derived from UMLS. As an aside, Creative Commons also has license containing clauses to be explicit about restrictions such as the share-alike (SA), non-commercial (NC), and non-distribution (ND). While these clauses aren’t ideal for scientific data, it would at least be nice for UMLS to use a Creative Commons license with the appropriate combination of these clauses (I guess all three) to make it more explicit about its restrictions.
The most bizarre facet of UMLS is that they require you to fill out a user survey each year to keep access.

I want to 1) convert UMLS into an OWL ontology and 2) extract and encode its semantic mappings to external vocabularies like the Medical Subject Headings (MeSH) with Simple Standard for Sharing Ontology Mappings (SSSOM). Given all of these hurdles, it’s probably the case that I am not allowed to redistribute these artifacts.

All together, I consider this a big bummer. The United States National Library of Medicine (NLM) maintains several highly influential resources, but I have found in many instances that they lack a community perspective. Regardless, even as an expat, I pay American taxes, and it makes me upset that the government funds the development and maintenance of resources that I can’t easily use.

How To Break Free

Despite all of this rigamarole, there’s a process to subvert these issues by automating the interaction with the UMLS Terminology Services (UTS) and therefore enabling automated download of UMLS and the following (non-exhaustive) list of resources:

RxNorm
SemMedDB
SNOMED-CT
potentially more in the future

This has been implemented in the open source umls_downloader Python package. It can be installed with the following one-liner in your shell:

$ pip install umls_downloader

Below, I’ll walk you through using it. Throughout, keep in mind that full documentation for the umls_downloader is available at umls-downloader.readthedocs.io, which describes the other functionality and other data that can be downloaded.

Usage

UMLS has three different distributions that are described here. The following Python code downloads the most simple and straightforward file, MRCONSO.RRF as a zip archive:

from umls_downloader import download_umls

path = download_umls(version="2023AA", api_key="")

This code is smart and does not need to download the file more than once. It uses pystow to choose a stable path ~/.data/bio/umls relative to the current user’s home directory. Inside this directory, it also uses the version of the data to create a subdirectory. Finally, this function returns the path to the data, such that no file paths ever need to be hard-coded.

Warning This still requires an API key, which requires creating an account, agreeing to UMLS’s terms and conditions, etc. This can be done here: https://uts.nlm.nih.gov/uts/edit-profile.

Automating Configuration of UTS Credentials

There are two ways to automatically set the API key, so you don’t have to worry about getting it and passing it around in your python code:

Set UMLS_API_KEY in the environment. This can be done in your interactive session or in the configuration for your shell such as in a .bashrc file for the Bourne Again Shell (bash).
Create ~/.config/umls.ini and set in the [umls] section a api_key key. Mine looks like:
```
[umls]
api_key=1234567890abcdefghijklmno
```

Now you can omit the api_key keyword like in the following:

from umls_downloader import download_umls

# Same path as before
path = download_umls(version="2023AA")

Download the Latest Version

First, you’ll have to install bioversions with pip install bioversions, whose job it is to look up the latest version of many databases. Then, you can modify the previous code slightly by omitting the version keyword argument:

from umls_downloader import download_umls

# Same path as before (when run on September 1st, 2023)
path = download_umls()

Download and open the file

The UMLS file is zipped, so it’s usually accompanied by the following boilerplate code:

import zipfile
from umls_downloader import download_umls

path = download_umls()
with zipfile.ZipFile(path) as zip_file:
    with zip_file.open("MRCONSO.RRF", mode="r") as file:
        for line in file:
            ...

This exact code is wrapped with the umls_downloader.open_umls() using Python’s context manager, so it can more simply be written as:

from umls_downloader import open_umls

with open_umls() as file:
    for line in file:
        ...

Note The version and api_key arguments work the same for umls_downloader.open_umls() as in umls_downloader.download_umls()

At this point, it’s up to you to decide how you want to consume the MRCONSO.RRF file. Below, I give a demo on how parsed this file in PyOBO in order to convert UMLS to an OWL ontology.

Why not an API?

The UMLS provides an API for access to tiny bits of data at a time. There are even two recent (last 5 years) packages umls-api connect-umls that provide a wrapper around them. However, API access is generally rate limited, difficult to use in bulk, and slow. For working with UMLS (or any other database, for that matter) in bulk, it’s necessary to download full database dumps.

UMLS Conversions

Building on top of the automated download of UMLS, I implemented a fit-for-purpose processor with the PyOBO framework that converts UMLS into an ontology (encoded either as OWL, OBO, or OBO Graph JSON) which can therefore be used to generate semantic mappings in the SSSOM format. The code that implements this can be found here. After installing PyOBO with pip install pyobo, you can automatically download and convert UMLS first into an ontology encoded in the OBO flat file format, then convert to OWL with the following code. Note: you’ll need robot for the second step:

import pyobo

umls = pyobo.get_ontology("umls")

# Write simple OBO Format
umls.write_obo("umls.obo")

# Convert to OWL
from pyobo.utils.misc import obo_to_owl

obo_to_owl("umls.obo", "umls.owl")

In an ideal world, the results of such a conversion could be included as a part of the OBO Database Ingestion, which converts database resources available through PyOBO into ontology artifacts, archives them on GitHub and Zenodo, and gives them PURLs all on a weekly basis to make sure the most up-to-date version is available as well as all previous named versions. Instead, we live in a world with pineapple pizza and restrictive licenses.

One of the nice qualities of UMLS is that it is a semantic mapping hub. It provides mostly complete mappings between many vocabularies including MeSH, NCIT, SNOMED-CT, HPO, LOINC, and more. However, there are a few caveats to consider:

UMLS mappings aren’t all 1-to-1. For example, MeSH mappings typically include many UMLS terms (narrower) pointing to the same MeSH term (broader). For other vocabularies, such as NCBITaxon, UMLS mappings are more reliably 1-to-1. Thanks to Tiago Lubiana for pointing this out.
Mapping provenance is not available, so the mapping_justification field in SSSOM is uniformly filled with sempav:UnspecifiedMatching.
Similarly, UMLS does not apply precise semantic predicates for each mapping. This means that they are output in PyOBO and as SSSOM with oboInOwl:hasDbXref instead of more detailed types such as skos:exactMatch, skos:narrowMatch, and skos:broaderMatch. Tools like Boomer can be used to address this (in part). The Semantic Mapping Reasoning Assembler (SeMRA) can also be configured with prior knowledge about UMLS mapping assumptions when aggregating and reasoning over semantic mappings at scale.

With that in mind, anything that can be loaded as an ontology in PyOBO can also be exported with SSSOM, which I show below. For UMLS, this looks like:

import pyobo

df = pyobo.get_sssom_df("umls", names=False)
df.to_csv("umls.sssom.tsv", sep="\t", index=False)

Note You can set names=True to have PyOBO look up the names for all entities, but this is a bit of a rabbit hole since it requires getting and processing many external resources.

There’s much more to say about UMLS and SSSOM, but this is a good place to pause and publish this post, since getting UMLS as SSSOM is a task a lot of people have asked me for help with lately. I might also come back and explain more about how I use the other resources from UMLS’s UTS.

Reproducibility Pilot in the Journal of Cheminformatics

2023-08-27T08:00:00+00:00

I’ve been working on improving reproducibility in the field of cheminformatics for some time now. For example, I’ve written posts about making data from DrugBank and ChEMBL more actionable. Over the last year, I’ve been preparing a concept with the editors of the Journal of Cheminformatics on how to include an assessment of reproducibility to reviews of manuscripts submitted to the journal. This has resulted in an editorial Improving reproducibility and reusability in the Journal of Cheminformatics as well as a call for papers. In this post, I want to summarize the first generation review criteria we developed, give an example of it applied in practice

The Seven (First Generation) Criteria

There are many potential directions for reproducibility. Given the fact that typical computational scientists are not trained as software engineers, we decided on seven very simple criteria that can be easily reviewed and easily addressed:

Does the repository contain a LICENSE file in its root?
Does the repository contain a README file in its root?
Does the repository contain an associated public issue tracker?
Has the repository been externally archived on Zenodo, FigShare, or equivalent that is referenced in the README?
Does the README contain installation documentation?
Is the code from the repository installable in a straight-forward manner?
Does the code conform to an external linter (e.g., black for Python)?

These correspond to important details that are complementary to other considerations of reproducibility, but often overlooked. Throughout the pilot, the editors and reviewers will try to support authors in addressing each of these points during revision. I imagine that there will be future iterations of these criteria as the community begins to expect these as standard practice. For example, we can narrow criteria 1 to specifically say that the software should be licensed with an OSI-approved license and not accept science made with non-open licenses. We could further narrow point 7 to have additional community style requirements (e.g., passes parts of flake8, as you know I love from my post on flake8 hell). We could also include additional guidelines that e.g. say that the results presented in the paper should be reproducible with a single command from the command line, e.g., a shell script. The rabbit hole could go very deep, so again, it’s worth saying that these are very non-controversial criteria for the first generation.

That being said, many repositories don’t follow these! Since these criteria are so simple, I’m interested in automating their assessment and further applying it to the entire Journal of Cheminformatics backlog. I’ll describe this more in a future post.

Without further ado, the text below is what I sent verbatim in the review for Drug-Protein Interaction Prediction via Multi-View Variational Autoencoder and Cascade Deep Forests, which is pre-printed on Research Square and has associated code here. I have tried my best to include actionable links and information with each piece. I would like to also automate sending separate GitHub issues for each of these points as a more concrete to-do list for authors, then also send an “epic” issue that lists all of them together. With the magic of the GitHub API, this is possible.

My First Reproducibility Review for the Reproducibility Pilot

Below, I apply the seven point reproducibility review prescribed by Improving reproducibility and reusability in the Journal of Cheminformatics to the default branch of repository https://github.com/Macau-LYXia/MVAE-DFDTnet (commit c0858c8), accessed on August 27^th, 2023.

Does the repository contain a LICENSE file in its root?
No. The GitHub license picker can be used to facilitate adding one by following this link: https://github.com/Macau-LYXia/MVAE-DFDTnet/community/license/new?branch=main. Ideal software licenses for open source software include the MIT License, BSD family of licenses, and other licenses approved by the Open Source Initiative. A simple, informative guide for picking a license can be found at https://choosealicense.com.
Does the repository contain a README file in its root?
No. A minimal viable README file contains:
- A short, one line description of the project
- Information on how to download, install, and run the code locally
- Brief documentation describing the single most important use case for the repository. For scientific code, this is ideally a one-liner in Python code, a shell script, or a command line interface (CLI) that can be used to reproduce the results of the analysis presented in a corresponding manuscript, use the tool presented in the manuscript, etc.
- Link to an archive on an external system like Zenodo, FigShare, or an equivalent.
- Citation information, e.g., for a pre-print then later for a peer reviewed manuscript
GitHub can be used to create a README file with https://github.com/Macau-LYXia/MVAE-DFDTnet/new/main?filename=README.md. Repositories typically use the Markdown format, which is explained here.
Does the repository contain an associated public issue tracker?
Yes. It is available at https://github.com/Macau-LYXia/MVAE-DFDTnet/issues.
Has the repository been externally archived on Zenodo, FigShare, or an equivalent that is referenced in the README?
No, there is no README. This is also not mentioned in the manuscript. See https://docs.github.com/en/repositories/archiving-a-github-repository/referencing-and-citing-content for a tutorial on how to do this.
Does the README contain installation documentation?
No, there is no README. This is also not mentioned in the manuscript.
Is the code in the repository installable in a straight-forward manner?
No, the code is not laid out in a typical structure, e.g., as described in https://blog.ionelmc.ro/2014/05/25/python-packaging. Further, there is no setup configuration that encodes the dependencies or facilitates reuse of the code.
Does the code in the repository conform to an external linter (e.g., black for Python)?
No. The Python code has not been linted, e.g., using black. Similarly, the Matlab code has not been linted, e.g. using checkcode.

Scientific integrity depends on enabling others to understand the methodology (written as computer code) and reproduce the results generated from it. This reproducibility review reflects steps towards this goal that may be new for some researchers, but will ultimately raise standards across our community and lead to better science. Because the work presented in this article only yet address one of the seven points of the reproducibility review, I recommend rejecting the article and inviting later resubmission following addressing the points.

For posterity, this review has also been included on https://github.com/Macau-LYXia/MVAE-DFDTnet/issues/1.

The Future is Looking Good

The example above isn’t so great - it’s possible that these authors have never considered most of these points about reproducibility before. The reality is that many computational scientists are not trained in this since their mentors were not primarily trained as computational scientists themselves. Combine with the perverse incentive structure in academia, it’s understandable how this can be left out from some publications. I experienced something similar in my doctoral studies, and had to bootstrap my own philosophy on reproducibility as well as the practical skills to achieve it. I also understand not everyone is in the position where they have the flexibility/freedom/initiative to do this.

That all being said, we are now entering an era where progressive and newly minted PIs actually have training as computational scientists. The next paper in my queue for a reproducibility review is for https://github.com/Steinbeck-Lab/cheminformatics-python-microservice, which will pass the 7 criteria with flying colors. I’m looking forward to the future when we expect more excellent science on the regular. See you there!

I’m not sure how people will view the way I talk about reviews - I am quite open with posting reviews on GitHub and also openly discussing the fact that I’ve reviewed something. Ideally, I don’t accept reviews for papers that don’t have pre-prints, since I personally think the review process should be open. I hope it’s the case that I haven’t been rude or unfair. If that’s the case, someone can help me change the way I write about these topics.

Querying Journals and Publishers in Wikidata

2023-06-22T17:00:23+00:00

Today’s short post is about three SPARQL queries I wrote to get bibliometric information about journals and publishers out of Wikidata.

Each of the following queries can be readily copy-pasted into the Wikidata Query Service and run in the browser.

Journals

The following SPARQL query gets information about journals:

SELECT ?journal ?journalLabel (GROUP_CONCAT(?issn) as ?issns)
WHERE
{
  ?journal wdt:P31 wd:Q5633421 ;
           wdt:P236 ?issn .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". } # Helps get the label in your language, if not, then en language
}
GROUP BY ?journal ?journalLabel

Follow this link to populate the Wikidata Query Service with this query. Note that this query takes a while to run and may time out since there are on the scale of 100K journals.

Journals might have multiple International Standard Serial Numbers (ISSNs) because a different one is assigned to the print and electronic versions of the journal, among other things.

Get the ISSN-L (the normalized/preferred) ISSN for each:

SELECT ?journal ?journalLabel ?issn
WHERE
{
  ?journal wdt:P31 wd:Q5633421 .
  OPTIONAL { ?journal wdt:P7363 ?issnl }
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". } # Helps get the label in your language, if not, then en language
}

Get a forward mapping from all ISSNs to ISSN-L. Note that these have been filtered to scientific journals (wd:Q5633421)

SELECT ?issn ?issnl
WHERE
{
  ?journal wdt:P31 wd:Q5633421 ;
           wdt:P7363 ?issnl ;
           wdt:P236 ?issn .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". } # Helps get the label in your language, if not, then en language
}

Publishers

The following SPARQL query gets information about publishers:

SELECT DISTINCT ?publisher ?publisherLabel ?ror ?grid ?isni
WHERE
{
  ?publisher wdt:P31/wdt:P279+ wd:Q2085381 ;
             rdfs:label ?publisherLabel .
  FILTER ( LANG(?publisherLabel) = "en" )
  OPTIONAL { ?publisher wdt:P6782 ?ror }
  OPTIONAL { ?publisher wdt:P2427 ?grid }
  OPTIONAL { ?publisher wdt:P213 ?isni }
}

Follow this link to populate the Wikidata Query Service with this query. This query returns the Research Organization Registry (ROR) identifier when available. This registry effectively subsumes the Global Research Identifier Database (GRID), which has since been shut down, but this might be helpful for integrating data that hasn’t been updated. The International Standard Name Identifier (ISNI) is also included when available. Wikidata has several other nomenclature authorities such as GND, VIAF, RingGold, and others that are omitted for brevity (each has their own corresponding Wikidata property.).

Later, I could consider adding a clause to make sure there’s a “scientific journal” in the publisher to remove some irrelevant records.

Connections between Journals and Publishers

Finally, the publisher (P123) relation can be used to identify the relationships between journals and their respective publishers.

SELECT DISTINCT ?journal ?journalLabel ?publisher ?publisherLabel
WHERE
{
  ?journal wdt:P31 wd:Q5633421 ;
           rdfs:label ?journalLabel ;
           wdt:P123 ?publisher .
  FILTER ( LANG(?journalLabel) = "en" )
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". } # Helps get the label in your language, if not, then en language
}
ORDER BY ?journalLabel

Follow this link to populate the Wikidata Query Service with this query.

Rather than using the Wikidata label service for the journal label, I more explicitly wrote it out to ensure that there is an english label, and to remove anything without an english label.

Modeling and Querying Awards in Wikidata

2023-06-08T10:09:23+00:00

I was recently nominated for the International Society for Biocuration’s Excellence in Biocuration Early Career Award (results will be announced on June 14^th!). This made me curious about how to model nominations and awards on Wikidata. In this post, I’ll describe how to curate awards, nominations, recipients, and how to make SPARQL queries to get them.

Summarizing an Individual

I’m going to use SPARQL with the Wikidata Query Service to see what’s already in Wikidata. First, I want to find all awards that I’ve personally received using the P166 (award received) property. Note that the following query also takes advantage of Wikidata’s reification so I can reach into the qualifiers of each statement to figure out when the award was given.

SELECT ?award ?awardLabel ?year ?conferer ?confererLabel
WHERE {
  VALUES ?person { wd:Q47475003 }
  ?person p:P166 ?award_statement .
  ?award_statement ps:P166 ?award .
  OPTIONAL { ?award wdt:P1027 ?conferer . }
  OPTIONAL {
    ?award_statement pq:P585 ?date .
    BIND(year(?date) AS ?year)
  }
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}

See this query in action at https://w.wiki/6odU.

As of time of writing, the only award that is listed here is the Bernie Lemire Award. This was given to me by the Northeastern University Department of Chemistry at the end of my bachelor’s degree for service to the department and academic excellence. I am very proud of this award! You can switch out wd:Q47475003 for your Wikidata identifier.

A similar SPARQL query can be written to identify all of the awards for which I was nominated by swapping the predicate to P1411 (nominated for). This isn’t necessarily a superset of the awards received since some awards are decided without a nomination. It might also be the case depending on how curation is done that these are out of sync.

SELECT ?award ?awardLabel ?year ?conferer ?confererLabel
WHERE {
  VALUES ?person { wd:Q47475003 }
  ?person p:P1411 ?award_statement .
  ?award_statement ps:P1411 ?award .
  OPTIONAL { ?award wdt:P1027 ?conferer . }
  OPTIONAL {
    ?award_statement pq:P585 ?date .
    BIND(year(?date) AS ?year)
  }
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}

See this query in action at https://w.wiki/6odV.

Summarizing an Award

Many awards are given on a periodic basis (e.g., yearly, bi-yearly). Scholia is an excellent frontend to Wikidata that already has a way of summarizing awards. Some examples:

Summarizing a Conferrer

Finally, I want to summarize all awards nominated or given by an organization. In this example, I’m going to look at the International Society for Biocuration (ISB; Q23809291).

Winners

The following query shows all of the recipients for all of the various awards conferred by the ISB:

SELECT ?award ?awardLabel ?recipient ?recipientLabel ?year
WHERE {
  ?recipient p:P166 ?award_statement .
  ?award_statement ps:P166 ?award .
  OPTIONAL {
    ?award_statement pq:P585 ?date .
    BIND(year(?date) AS ?year)
  }
  ?award wdt:P1027 wd:Q23809291 .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}
ORDER BY DESC(?year) ?awardLabel

See this query in action at https://w.wiki/6odW or the results embedded below.

At the time of writing, this only returned a paltry 9 rows, meaning more curation is necessary! Considering this award is about biocurators, we better get our act together 🙃. Update June 4^th, 2023: I went back and curated the full catalog.

Nominations

Similarly, the following query can be used to identify all nominations:

SELECT ?award ?awardLabel ?nominee ?nomineeLabel ?year
WHERE {
  ?nominee p:P1411 ?award_statement .
  ?award_statement ps:P1411 ?award .
  OPTIONAL {
    ?award_statement pq:P585 ?date .
    BIND(year(?date) AS ?year)
  }
  ?award wdt:P1027 wd:Q23809291 .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}
ORDER BY DESC(?year) ?awardLabel

See this query in action at https://w.wiki/6odX or the results embedded below.

There are only 5 results at the time of writing, and these are for my fellow nominees for the Excellence in Biocuration Early Career Award that I recently curated! There’s a lot of work to do here for keeping a history of the ISB’s awards. Update June 4^th, 2023: I went back and curated the full catalog. It turns out that the ISB did not publish the list of nominees for any awards until 2022, so this list will remain short.

Wikidata-wide Summary

More generally, it turns out that there are only a bit more than 55K nomination relations in total for all of Wikidata. You can check this with:

SELECT (count(*) AS ?count)
WHERE { ?nominee wdt:P1411 ?award . }

Curating an Award

Award objects don’t have to be complicated - the most important information is to include a useful instance annotation (e.g., to science award (Q11448906)) and the following:

Field of Work (P101)
Conferred By (P1027)
Website (P856)
Inception (P571)

See https://www.wikidata.org/wiki/Q118947746 as an example.

Curating an Individual

On a given Wikidata page, you can add a statement for either nominated for or award received using Wikidata’s amazing curation interface that has search built in. It’s recommended to add a point in time (P585) annotation to make a distinction between different periods. Further, it’s recommended to add a refernce using the reference url (P854) property that pints to a webpage with an announcement about the nomination or award.

Overall, I think modeling awards is hard, since these are less concrete than other academic information such as employment or education. Still, this is the next step in making my resume 100% auto-generated by SPARQL and Wikidata!

See also an analysis by Chris Mungall of the gender distribution of awards in Wikidata.

Update June 4^th, 2023: I won the Excellence in Biocuration Early Career Award! Nico Matentzoglu won the Excellence in Biocuration Advanced Career Award and we are both excited to see that the community was interested to recognize people who work on fundamental underlying technologies.

Re-implementing the N2T ARK Resolver

2023-04-11T18:44:23+00:00

Archival Resource Keys (ARKs) are flavor of persistent identifiers like DOIs, URNs, and Handles that have the benefit of being free, flexible with what metadata gets attached, and natively able to resolve to web pages. Name-to-Thing (N2T) implements a resolver for a variety of ARKs, so this blog post is about how that resolver can be re-implemented with the curies Python package.

In a lot of ways, ARKs look and act like CURIEs. For example, ark:/53355/cl010277627 could be interpreted as having the prefix ark and the local unique identifier /53355/cl010277627. The first part of each ARK between the first two slashes corresponds to the provider. In this example, 53355 corresponds to the Louvre museum in Paris, France and cl010277627 is the local unique identifier corresponding to the Vénus de Milo statue.

However, I might have just committed ARK blasphemy. In N2T, it appears that the ARK prefix and provider code stay grouped together in the front half like ark:/53355/ and then the back half cl010277627 represents the local unique identifier. This is very similar to the two-layer identifiers in DOI and the arbitrary number of layer identifiers in OID.

The point is, if we can interpret this enough like CURIEs, we can use the curies package to implement a resolver. The first step we can take is to download the N2T data from https://n2t.net/e/n2t_full_prefixes.yaml. Then we can parse out the ARKs (there are other things in N2T we’ll disregard) with the following code:

import pystow
import yaml

URL = "https://n2t.net/e/n2t_full_prefixes.yaml"
PROTOCOLS = {"https://", "http://", "ftp://"}


def get_prefix_map():
    """Get the prefix map from N2T, not including redundant ``ark:/`` in prefixes."""
    with pystow.ensure_open("n2t", url=URL) as file:
        records = yaml.safe_load(file)
    prefix_map = {}
    for key, record in records.items():
        uri_prefix = record.get("redirect")
        if (
            not uri_prefix
            or all(not uri_prefix.startswith(protocol) for protocol in PROTOCOLS)
            or uri_prefix.count("$id") != 1
            or not uri_prefix.endswith("$id")
            or not key.startswith("ark:/")
        ):
            continue
        key = key.removeprefix("ark:/")
        prefix_map[key] = uri_prefix.removesuffix("$id") + "/" + key + "/"
    return prefix_map

This prefix map removes ark:/ from the beginning of the prefixes in N2T and also adds the provider code into the URI prefix to make the URIs more focused on the local unique identifiers within each provider, rather than the entire ARK space.

Once we have a prefix map, we can make a curies.Converter and a Flask web application for resolving in a few lines:

from curies import Converter, get_flask_app


def get_app():
    """Get an ARK resolver app, noting that it uses a non-standard delimiter and URL prefix."""
    prefix_map = get_prefix_map()
    print(prefix_map)
    converter = Converter.from_prefix_map(prefix_map, delimiter="/")
    app = get_flask_app(converter, blueprint_kwargs=dict(url_prefix="/ark:"))
    return app

The two tricks here are:

We want to remove the redundant ark:/ then interpret the ARK provider code as the prefix and the rest as the local unique identifier. However, we still want to be able to write URLs in our resolver that have the ark:/ prefix. Luckily, Flask has the facility to define a default url_prefix before a given blueprint that we invoke directly.
Unlike CURIEs that use a colon : as the delimiter between the prefix and local unique identifier, ARKs use a slash /. We can also set this in the Converter’s settings.

Now, all we need to do is instantiate the app and serve it with any WSGI tool like Gunicorn, Uvicorn, or Flask’s built-in development server (from Werkzeug). Navigating to http://localhost:5000/ark:/53355/cl010277627 redirects to https://collections.louvre.fr/ark:/53355/cl010277627 and gets some nice art from the Louvre. In general, you can stick any ARK after http://localhost:5000/ark: that is resolvable via N2T when running this server.

All of this code is on GitHub and can be run with the following:

git clone https://github.com/cthoyt/n2t-ark-resolver
cd n2t-ark-resolver
python -m pip install -r requirements.txt
python wsgi.py

Update: since posting this, I have heard from John Kunze that the ARK format is currently being updated to look more like URNs and therefore not have the slash after ark:/ anymore. If/when that happens, there are only a few bits of string pre-processing in this script that need to be updated to keep everything running.

The Representatives of Monkey Jack - German Battle of the Bands Finale

2023-03-29T19:33:23+00:00

This blog is normally about very serious science, but I’m taking a break from that for the evening to advertize my band’s upcoming show on April 8^th in the SPH Music Masters Finale (aka, the German Battle of the Bands). We need your support! There are streaming tickets available, and this post has a guide on how to navigate the German website to get tickets (or just text me, I’ll hook you up).

Here’s a video of us playing. If you love it, let me know. If you hate it, slap like and subscribe.

When is this?

The show is on April 8^th at the Live Music Hall in Cologne, Germany. Because this is Easter weekend, it’s the perfect thing to do while you’re relaxing with your family. The show starts at 15.30 CEST / 9:30AM EST. The order of the 10 bands playing will be determined on the morning of, so I’ll send out a mass text about what time that will be for everyone who’s streaming.

How does the battle work?

The battle of the bands is judged in two ways: half of the score is by the judges and half by the audience. Each audience member gets two votes - this usually means they vote for the band they support and a second band that they liked after seeing them for the first time.

There will be 10 bands playing, which means this is going to be a loooooong day. The best spots in the show will be in the middle or towards the end, after the afternoon settles into evening. The order will be determined just before the show starts, based on which band sells the most pre-sale tickets. This means that getting a streaming ticket will support us to get a better spot, which is highly correlated with winning.

Who is Monkey Jack, and why do you represent him?

As you noticed, we’re the representatives of Monkey Jack. We will tell The Tale of Monkey Jack at our show. You should prepare by bringing a banana with you to the show, which you will need at the end of the Tale when we begin The Ritual.

Getting a Ticket

First, navigate here for tickets.

Second, click Streamingtickets. If you want to be cool while speaking German, you should throw in some English words (or internationalisms).

Third, click Auswählen. This verb means that you are pledging your allegiance to Monkey Jack and promise to follow Him.

Fourth, click Bitte Wählen (please choose). This is a drop down menu to show your support for Monkey Jack and his representatives. Note that you only need one streaming ticket per stream, obviously you should throw a party/ritual to represent Monkey Jack yourself.

Fifth, click Monkey Jack.

Sixth, You can fill in the form with your information. The image below annotates what each of the fields means. Plz is short for “Postleitzahl”, which means Zip code. Don’t worry about the part of the form with the country picker. You’re a German now.

After you click it, it will bring you to a PayPal page. They’ll email you a confirmation within 5-10 minutes and send the streaming link the day of the show.

Resources masquerading as OBO Foundry ontologies

2023-03-11T23:11:23+00:00

Several controlled vocabularies and ontologies that aren’t themselves OBO Foundry ontologies use unsanctioned OBO PURLs. This post is about how to use the Bioregistry to identify which resources are doing this and to give some insight into how we arrived in this situation.

Background on the OBO Foundry

The OBO Foundry is a set of independent, interoperable biomedical ontologies that aspire to using shared development principles. One such principle is to use a principled approach for creating persistent uniform resource locators (PURLs) for local unique identifiers in each ontology. These PURLs follow the form http://purl.obolibrary.org/obo/_. For example, a prefix might be GO (for the Gene Ontology) and local unique identifier might be 0032571 (for response to vitamin K in GO) resulting in the PURL http://purl.obolibrary.org/obo/GO_0032571.

While most semantic web resources allow the use of any IRIs (internationalized resource identifiers), the OBO Foundry enforces that its PURLs resolve to something useful for readers (e.g., to the Ontology Lookup Service). The resolver behind http://purl.obolibrary.org is implemented and maintained in a GitHub repository with corresponding .htaccess files for each OBO Foundry ontology. Correct and useful configuration for each ontology are a requirement for acceptance to the OBO Foundry.

At the core of the OBO Foundry are several high quality, well-known, generally useful ontologies such as the Gene Ontology and the Cell Ontology. Inclusion in the OBO Foundry has therefore become a de facto stamp of approval for ontologies that (until now) 254 ontologies have (for better or worse) successfully sought out.

The Masquerade

Unfortunately, some ontologies and controlled vocabularies have adopted OBO PURLs even though they are not OBO Foundry ontologies. This is a problem for a few reasons:

The PURLs are not registered in https://github.com/OBOFoundry/purl.obolibrary.org/ and therefore do not resolve
The quality of the ontology has not been vetted, meaning that low-quality ontologies using OBO PURLs could damage community trust in the OBO Foundry
These PURLs might conflict with other URIs prefix synonyms, increasing complexity for data integration
Most importantly: it’s confusing.

One of the jobs of the Bioregistry is to catalog the URI format strings for identifier resources useful for the life and natural sciences. This allows us to assess how big the problem of non-OBO Foundry ontologies are using OBO PURLs, and why. Without further ado, here’s the list of offending resources that appear in the Bioregistry:

prefix	name	evidence	uri_prefix
aeon	Academic Event Ontology	curated	`http://purl.obolibrary.org/obo/AEON_`
cemo	COVID-19 epidemiology and monitoring ontology	extra	`http://purl.obolibrary.org/obo/cemo.owl#`
covoc	CoVoc Coronavirus Vocabulary	curated	`http://purl.obolibrary.org/obo/COVOC_`
decipher	DECIPHER CNV Syndromes	biocontext	`http://purl.obolibrary.org/obo/DECIPHER_`
dermo	Human Dermatological Disease Ontology	curated	`http://purl.obolibrary.org/obo/DERMO_`
efo	Experimental Factor Ontology	biocontext	`http://purl.obolibrary.org/obo/EFO_`
gorel	GO Relations	biolink	`http://purl.obolibrary.org/obo/GOREL_`
hpath	Histopathology Ontology	curated	`http://purl.obolibrary.org/obo/MC_`
idocovid19	COVID-19 Infectious Disease Ontology	curated	`http://purl.obolibrary.org/obo/COVIDO_`
lbo	Livestock Breed Ontology	curated	`http://purl.obolibrary.org/obo/LBO_`
lpt	Livestock Product Trait Ontology	curated	`http://purl.obolibrary.org/obo/LPT_`
mesh	Medical Subject Headings	biocontext	`http://purl.obolibrary.org/obo/MESH_`
msio	Metabolomics Standards Initiative Ontology	curated	`http://purl.obolibrary.org/obo/MSIO_`
omia	Online Mendelian Inheritance in Animals	biocontext	`http://purl.obolibrary.org/obo/OMIA_`
omim	Online Mendelian Inheritance in Man	biocontext	`http://purl.obolibrary.org/obo/OMIM_`
pride	PRIDE Controlled Vocabulary	curated	`http://purl.obolibrary.org/obo/PRIDE_`
reo	Reagent Ontology	curated	`http://purl.obolibrary.org/obo/REO_`
roleo	Role Ontology	curated	`http://purl.obolibrary.org/obo/RoleO_`
soybase	SoyBase	prefixcommons	`http://purl.obolibrary.org/obo/`
uniprot.isoform	UniProt Isoform	extra	`http://purl.obolibrary.org/obo/UniProtKB_`
vido	Virus Infectious Disease Ontology	curated	`http://purl.obolibrary.org/obo/VIDO_`
vsmo	Ontology for vector surveillance and management	curated	`http://purl.obolibrary.org/obo/VSMO_`
xl	Cross-linker reagents ontology	curated	`http://purl.obolibrary.org/obo/XL_`

In the evidence column, there are a few possible entries:

curated - this is the URI prefix manually curated in the Bioregistry. This happens when the primary ontology artifact uses OBO PURLs.
extra - this is when there’s a manually curated extra URI prefix in the Bioregistry (in addition to the primary one) that uses an OBO PURL. This usually is done to enable the Bioregistry’s IRI parser to handle cases that appear in third-party data that incorrectly constructs IRIs.
biocontext, biolink, and prefixcommons - this is when other registries have assigned OBO PURLs as their URI expansions

It’s worth noting that there are probably lots more resources doing this, e.g., that are listed in BioPortal, but have not been included in the Bioregistry because of their lack of notability, utility, or reuse.

Here's the code that generated the table (before minor modifications)

Compliance of Bioregistry Prefixes to the W3C Standard

2023-01-11T23:17:00+00:00

This post gives a brief background on the formal definition of the syntax and semantics of compact uniform resource identifiers (CURIEs) from the Worldwide Web Consortium (W3C) and investigates how many prefixes in the Bioregistry are compliant with the standard.

Syntax

The W3C’s CURIE 1.0 Syntax is unfortunately obfuscated. Understanding it requires navigating through several pages and reading cryptic definitions in a BNF-like notation. Below is a short explanation of the two important parts and a nice simplification:

safe_curie  :=  '[' curie ']'
curie       :=  [ [ prefix ] ':' ] reference
prefix      :=  NCName
reference   :=  irelative-ref

where NCName is defined on this page as

NCName     ::= (Letter | '_') (NCNameChar)*
NCNameChar ::= Letter | Digit  | '.' | '-' | '_' | CombiningChar | Extender

and irelative-ref is defined here by referencing external RFC 3987. Understanding this part is not strictly necessary for checking Bioregistry prefixes.

After unpacking all of these nested references and making the reasonable assumption that the strange characters referenced by CombiningChar and Extender are unlikely to appear in any real prefixes, we arrive a the following regular expression for validating prefixes: ^[a-zA-Z_][a-zA-Z0-9.-_]*

Bioregistry Compliance

It’s relatively easy to write a script that checks Bioregistry prefixes against this regular expression.

import re
import bioregistry
from tabulate import tabulate

W3C_PREFIX = re.compile("^[a-zA-Z_][a-zA-Z0-9.-_]*")
failed = [
    (
        f"[{resource.prefix}](https://bioregistry.io/{resource.prefix})",
        resource.get_name(),
    )
    for resource in bioregistry.resources()
    if not W3C_PREFIX.match(resource.prefix)
]
print(tabulate(failed, headers=["prefix", "name"], tablefmt="github"))

This script produces the following table as an output:

prefix	name
3dmet	3D Metabolites
4dn.biosource	4D Nucleome Data Portal Biosource
4dn.replicate	4D Nucleome Data Portal Experiment Replicate

Note that only three prefixes (at the time of writing) are non-compliant, each because it starts with a number instead of a letter or underscore. Overall, the Bioregistry is doing pretty good! Note that this does not check preferred prefixes nor synonyms. This might be good for a future update to this post or a follow-up post.

In the future, it might be nice to enforce some kind of prefix compliance at the unit test level to automate checking prefixes are appropriate. This might also include a blacklist of certain generic prefixes (e.g., gene) or other rules discussed in the project’s contribution guidelines.

Idiomatic conversion between URIs and compact URIs

2023-01-10T12:28:00+00:00

The semantic web and ontology communities needed a reusable Python package for converting between uniform resource identifiers (URIs) and compact URIs (CURIEs) that is reliable, idiomatic, generic, and performant. This post describes the curies Python package that fills this need.

After installing with pip install curies or checking out the code on GitHub and installing a local copy, you can directly jump in to using the curies package. Its main data structure is curies.Converter. It can be instantiated with various class methods corresponding to data in one of several formats.

The most common format is a prefix map, a dictionary containing a one-to-many mapping from CURIE prefixes to URI prefixes. It can be used in combination with the Converter.from_prefix_map class method. The following example includes some (but not all) of the CURIE and URI prefixes used by ontologies in the Open Biological and Biomedical Ontology (OBO) Foundry.

from curies import Converter

prefix_map = {
    "CHEBI": "http://purl.obolibrary.org/obo/CHEBI_",
    "MONDO": "http://purl.obolibrary.org/obo/MONDO_",
    "GO": "http://purl.obolibrary.org/obo/GO_",
    # ... and so on
    "OBO": "http://purl.obolibrary.org/obo/",
}
converter = Converter.from_prefix_map(prefix_map)

The Converter class indexes the prefix map using a trie data structure, which makes search of the beginning of sequences (such as strings) efficient. The curies implementation builds on the implementation of this data structure in the PyTrie package.

Conversion

A uniform resource identifier (URI) that corresponds to one of the URI prefixes registered in the converter can be compressed into a compact URI (CURIE) using the Converter.compress method. In the following example, we use the canonical URI (within the scope of the OBO Foundry) for the Gene Ontology term for response to vitamin K (GO:0032571).

>>> converter.compress("http://purl.obolibrary.org/obo/GO_0032571")
'GO:0032571'

When some URI prefixes are partially overlapping (e.g., http://purl.obolibrary.org/obo/CHEBI_ for GO and http://purl.obolibrary.org/obo/ for OBO), the longest URI prefix will always be matched. For example, compressing http://purl.obolibrary.org/obo/GO_0032571 returns GO:0032571 instead of OBO:GO_0032571.

If there’s no matching URI prefix, then compress() will return None.

>>> converter.compress("http://example.com/missing:0000000") is None
True

Similarly, a CURIE can be expanded into a URI using the Converter.expand method.

>>> converter.expand("GO:0032571")
'http://purl.obolibrary.org/obo/GO_0032571'

If there’s no matching CURIE prefix, then expand() will return None.

>>> converter.expand("missing:0000000") is None
True

Getting Prefix Maps

The curies package includes functions for loading several prefix maps from external resources. These are not cached in order to take advantage of the most recent versions. This is particularly important for resources like the Bioregistry that are updated frequently.

Name	Function	Description
Bioregistry	`curies.get_bioregistry_converter`	A high-coverage, general purpose registry for the life and natural sciences.
OBO Foundry	`curies.get_obo_converter`	A set of orthogonal ontologies for the life sciences constructed for mutual interoperability
Prefix Commons	`curies.get_prefixcommons_converter`	A medium-coverage, general purpose registry for the life and natural sciences
Gene Ontology	`curies.get_go_converter`	A project-specific prefix map for the Gene Ontology, includes several duplicate and non-standard definitions
Monarch	`curies.get_monarch_converter`	A project-specific prefix map for the Monarch Initiative, includes several duplicate and non-standard definitions

Loading from the Bioregistry

The bioregistry Python package has first-class support for the curies package through the generic function bioregistry.get_converter. This can be used as an alternative to curies.get_bioregistry_converter in cases when the Bioregistry is installed and it’s desired to use local data.

import bioregistry
from curies import Converter

converter: Converter = bioregistry.get_converter()

Loading from `prefixmaps`

The prefixmaps Python package keeps various prefix maps under version control that also has partial support for the curies package using the extended prefix map data structure (as opposed to a prefix map, this includes synonyms). See Converter.from_extended_prefix_map for more information on how to use this data structure.

from prefixmaps import load_context
from curies import Converter

extended_prefix_map = load_context("obo").as_extended_prefix_map()
converter = Converter.from_extended_prefix_map(extended_prefix_map)

Here’s a short (probably incomplete) list of other packages I’ve found that have related functionalities:

https://github.com/prefixcommons/prefixcommons-py (Python)
https://github.com/prefixcommons/curie-util (Java)
https://github.com/geneontology/curie-util-py (Python)
https://github.com/geneontology/curie-util-es5 (Node.js)
https://github.com/endoli/curie.rs (Rust)

This post didn’t touch the more advanced features of the Converter class such as its support for CURIE prefix synonyms and URI prefix synonyms. It also didn’t touch the curies.chain() function which enables several pre-instantiated converters to be used in succession, similarly to the Python built-in collections.ChainMap class. These are described in the documentation at curies.readthedocs.io

Long-term Funding for Small Biomedical Databases

2023-01-04T15:32:00+00:00

Way back in 2021, during the annual general assembly of the International Society for Biocuration (ISB) at the 14th Annual International Biocuration Conference (Biocuration 2021) , there was a discussion about the notably underutilized budget of the society that resulted in an informal open call for ideas for new small funding schemes. Concurrently, discussions with external stakeholders for the relatively new (at the time) Bioregistry project often included questions about the sustainability and longevity of the resource. We had conservatively estimated it would cost about 100 USD/year to run the Bioregistry site, so this seemed like the perfect opportunity to ask for a small amount funding distributed over a relatively long period of time. This post is about the more general reality of funding for small resources in the life sciences, how we petitioned the ISB for funding, and what happened next.

Bioregistry Cost Estimate

The Bioregistry is an integrative registry of biomedical ontologies, databases, and other identifier resources that is useful for standardizing the identification of biomedical concepts, making data more FAIR, and enabling data- and knowledge integration.

It runs a fully automated nightly workflow that updates its data, packages its code and data, and pushes it to the Python Package Index (PyPI). We carefully constructed this workflow to fully run on free, open infrastructure (e.g., GitHub Actions) and require little maintenance. After running this workflow for nearly three years, we’ve empirically found that the maintenance effort averages between 10-15 minutes per month for bug fixes and/or additional data curation. We won’t factor this into our initial cost estimate as this effort could be discretionary (i.e., not tied to a grant, donated by maintainers, users, etc.).

After, the workflow builds a Docker image and pushes it to the biopragmatics/bioregistry repository on DockerHub. This image is built with the the latest Python alpine base image, which significantly reduces non-essential components. It also has the benefit of being compatible with Docker environments running on both Intel and ARM architectures. The final compressed image weights less than 40 MB of disk space and runs inside Docker with about 65 MB of memory at baseline. This easily fits on a dedicated t4g.nano instance on Amazon Web Services (AWS) which costs about 37 USD/year on-demand or around 20 USD/year reserved (i.e., if you pay up front). Two or more of these instances could be set up behind a load balancer to ensure uptime in case one goes down, but we won’t factor this into our initial cost estimate.

The bioregistry.io domain is registered with Namecheap and costs about 33 USD/year. The SSL/TLS certificate for bioregistry.io (so it can be served with HTTPS) is managed through the AWS Certificate Manager and does not cost anything additional.

Finally, we’re going to exclude the costs of proactive curation of the resource. The combination of import from 23+ external registries, the ability for community members to suggest new prefixes in a streamlined way, and the decentralization of moderation effort across many institutions results in a negligible cost.

In a best case scenario, using a reserved instance and paying for a domain name brings this bill to 53 USD/year. Duplicating the deployment behind a load balancer would bring it up to 73 USD/year. For good measures in case costs go up, let’s call it 100 USD/year. That’s pretty darn cheap for a resource that’s already proven useful in several applications and is being pretty widely adopted. Most resources don’t last more than two or three years (disclaimer: I’m spit-balling here). If we got 1,000 USD, we could already promise people that this service will be running consistently for ten years. If we got 1,500 USD, we could extend that promise to fifteen years.

In order to gauge how small this amount is, consider that the International Society of Biocuration is currently offering six travel grants worth about the same amount each to attend the 16th Annual International Biocuration Conference (Biocuration 2023). Sure, I’m biased since the Bioregistry is my baby, but it would be difficult to argue that this is not pennies on the dollar compared to what some other resources cost to deploy and maintain.

What’s the Problem with Funding the Bioregistry

The Bioregistry was a spin-off project of a spin-off project. A long time ago, I was working on parsing and validating biomedical knowledge encoded in the Biological Expression Language. In order to support the validation of names and identifiers from various ontologies and databases, I started developing PyOBO as a tool for unified access to ontologies and databases. I curated extra rules for standardizing prefixes, CURIEs, and URIs that eventually got spun out into the Bioregistry and took on a life of its own. Each of these steps was part of a bigger project with bigger goals that weren’t necessarily focused on the development of high-quality, reusable underlying technology. But, developing high-quality, reusable underlying technology is just the way that I think, so that’s how I did it. And I took those nice tools with me from project to project, improving them as I went. This means that the Bioregistry has now played a role in several of my recent projects, but never sat up front and center when it came to writing grants.

So here’s the more general issue: grants are usually about building new, big things with big budgets that are done within the short timeframe of a project, then they’re usually lost or forgotten. The Bioregistry is a small, important resource that does not need massive development but it does need a long commitment to maintenance in order to provide the stability to the community that other resources have yet been able to provide. Further, the fact that it already exists makes it much less novel and therefore effectively disqualified from being the focus of most potential grants.

With potential costs so low, it would actually feasible for my current department, the Laboratory of Systems Pharmacology to commit discretionary funding external to a grant. However, this doesn’t engender trust the same as having external funding or being part of an institution that is well-known for the longevity of its infrastructure platforms (e.g., the NIH). Further, discretionary funding also raises questions about what happens when the main people on my team working on the Bioregistry leave, both from a responsibility and technical perspective. This seemed to be the sentiment from potential stakeholders of the Bioregistry that were shared at the Fall 2021 Workshop on Prefixes, CURIEs, and IRIs.

Communication with the ISB

After thinking about all of this, I finally write up a proposal (somewhere between one and two pages) and shared it with the executive board of the ISB. Unfortunately, I shared this on Slack and have been unable to find the original message since you can’t search Slack messages older than 90 days now. As a paraphrase, I focused on some of the following ideas:

As opposed to monolithic infrastructures and projects maintained by large organizations like the United States National Library of Medicines (NLM), the European Bioinformatics Institute (EMBL-EBI), and the Swiss Institute of Bioinformatics, most biocuration projects are small and limited in scope. There are several public systems that could support many of them:

archival systems (e.g., Zenodo, FigShare)
hosting of static sites (e.g., institutional websites, GitHub Pages)
hosting of dynamic sites (e.g., Amazon Web Services, Microsoft Azure, Heroku, Shiny, Google Could Platform)
domain name registration (e.g., institutional domain, NameCheap)
continuous integration and deployment (e.g., GitHub Actions, GitLab Pipelines)

These are relatively inexpensive and can be used to build resources that have high longevity and sustainability. Most biocuration projects end up being one-off publications anyway and are never updated, so I suggested that the ISB give small grants to groups that are using best practices and the above tools to create things that could have longevity, if given appropriate small funding across a long period of time. The biggest problem facing small curation projects and websites is that they can not acquire small amounts of long-term funding to support their technical infrastructure (e.g., domain name hosting, cloud compute resources). It’s becoming easier to build websites for new small biocuration projects, but again, there are small, recurring costs associated with them like paying for a domain name and cloud compute resources. I would suggest introducing a small grant whose award would be spread across a potentially long time (e.g., 10-15 years) to support those costs (but not explicitly not for major development nor maintenance).

Is this really a big deal?

My group has recently been thinking about how bad the longevity situation really is. We build the Bioregistry Health Report that checks on a weekly basis how many prefixes’ homepages still resolve and how many URI format strings still resolve. The results were bleak - only about 27% (as of today) do. This isn’t an indictment of the badness of the Bioregistry or the registries that it imports and aligns either - it’s true that sometimes the URLs for sites change, but this number reflects the more general problem that most just go away.

The Results

Unfortunately, the ISB EC didn’t accept the proposal. I didn’t get very direct feedback as to why, but I think the general concept of funding research was problematic for one reason or another for their organizational structure. I’m not discouraged, I still think this is important and will continue to write and re-write this proposal until it finally convinces the right funding body. I’ll also be touching on this point at my (~hopefully~ accepted abstract) for Biocuration 2023 that I posted about yesterday. Please let me know if you’ve got any other arguments supporting doing this that I can include!

Promoting the longevity of curated scientific resources through open code, open data, and public infrastructure

2023-01-03T16:28:00+00:00

The 16th Annual International Biocuration Conference (Biocuration 2023) is taking place in Padua, Italy from April 24-26^th, 2023. While I’m serving as a co-chair of the conference, I also think this is a great venue to communicate some of my thoughts on longevity and sustainability that have been gestating during the development of the Bioregistry and other Biopragmatics projects. This blog post contains the abstract I’ve submitted for oral presentation.

Note: if you want to submit your own abstract, you have until the end of today (January 3^rd, 2023, anywhere on earth) to submit via EasyChair.

Abstract

Many model organism databases, pathway databases, ontologies, and other curated resources that support research in the life and natural sciences combine expert-curated data with surrounding software code and services. However, such resources are often maintained internally by members of a single institution and are therefore susceptible to fluctuations in funding, personnel, and institutional priorities. Too often, resources go out of date, are abandoned, or become inaccessible, for example, when a grant runs out or a key person moves on. Therefore, we need better solutions for creating resources that are less susceptible to such external factors and can continue to be used and maintained by the community that they serve.

We propose a new model for the creation and maintenance of curated resources that promotes longevity through a combination of technical and social workflows, and a progressive governance model that supports and encourages community-driven curation. 1) The technical aspect of our model necessitates open data, open code, and open infrastructure. Both code and data are permissively licensed and kept together under public version control. This enables anyone to directly suggest improvements and updates. Further, automation is used for continuous integration (e.g., semi-automated curation, quality assurance) and continuous delivery (e.g., static website generation, export in multiple formats). 2) The social aspect of our model first prescribes the composition of training material, curation guidelines, contribution guidelines, and a community code of conduct that encourage and support potential community curators. Second, it requires the use of public tools for suggestions, questions, discussion as well as social workflows like pull requests for the submission and review of changes. 3) The governance aspect of our model necessitates the division of responsibilities and authority (e.g., for reviewing/merging changes to the code/data) across multiple institutions such that it is more robust to fluctuation in funding and personnel that can also be updated over time. It prescribes liberal attribution and acknowledgement of the individuals and institutions (both internal and external to the project) who contribute on a variety of levels (e.g., code, data, discussion, funding). More generally, our model requires that a minimal governance model is codified and instituted as early as possible in a project’s lifetime.

This talk will provide a perspective on how existing resources relate to our model, describe each of our model’s aspects in more detail (illustrated through the Bioregistry resource), and provide a practical path towards both creating new sustainable resources as well as revitalizing existing ones.

Recording

My talk was accepted! You can check the slides or watch the video recording:

Connecting Preprints to Peer-reviewed Articles on Wikidata

2023-01-02T18:27:00+00:00

After the BioCypher preprint went up on the arXiv, I checked in on the missing co-author items list on the Scholia page that reflects my Wikidata entry. In addition to the several co-authors of the BioCypher manuscript that I don’t know personally, I was curious to see which other papers of mine did not have fully complete co-author annotations. This post has a few SPARQL queries that I used to look into this as well as a few ongoing questions I have about the relationship between distinct entries for preprints and published articles.

First, I wrote two SPARQL queries for the Wikidata Query Service:

There were around 200 co-authors that I had included through a painstaking combination of manual curation and usage of the Author Disambiguator tool. However, when I looked at the ambiguous authors, i.e., authors that only stored by name via author name string (P2093) instead of by reference to a Wikidata entry via author (P50). At the time, this included co-authors from only four unique manuscripts including:

The appearance of the BioCypher manuscript on this list was no surprising because we just preprinted it on arXiv. However, I very clearly remember carefully curating all the co-authors of the other papers. After more careful inspection, it turns out that I had indeed done this curation for the preprints of each of these articles, but the ones appearing on the list corresponded to duplicate Wikidata entries not for the preprints, but for the published papers. This led me to a couple questions, which I don’t have answers for yet:

Should there two different entries for a preprint and a publication?
What’s even the right word for the dichotomy between a preprint and a publication? I don’t think it’s post-print.
It appears that there are multiple different entries between preprints and publications, so question 1) is a “perfect world” question. Since, in reality, there are duplicates, how should we handle them?
- Should we connect them via some kind of relationship? I recently noticed Tiago Lubiana had been using followed by (P156) for papers he had curated and have started using that myself in some cases (see notes below).
- Should we merge these two entries into one? There are various properties for the identifiers within preprint servers that can help point to pre-prints, though preprints are given different DOIs than the publication so maybe this would create confusion.
- How will this work with the advent of “overlay journals”, like what eLife is doing by more heavily relying on peer review attached to existing preprints?
To what extent does this confusion affect Wikidata content related to me (e.g., my papers)?

In order to assess my own Wikidata cleanliness, I wrote the following SPARQL query:

SELECT ?preprint ?preprintDate ?followedBy ?article ?articleDate ?label
WHERE
{
  VALUES ?author { wd:Q47475003 }
  ?preprint wdt:P31 wd:Q580922 ;
    wdt:P50 ?author ;
    rdfs:label ?preprintLabel .
  ?article wdt:P31 wd:Q13442814 ;
    wdt:P50 ?author ;
    rdfs:label ?label .
  OPTIONAL { ?preprint wdt:P577 ?preprintDate }
  OPTIONAL { ?article wdt:P577 ?articleDate }
  OPTIONAL { ?preprint wdt:P156 ?followedBy }
  FILTER (LCASE(?preprintLabel) = LCASE(?label))
  FILTER (?preprint != ?article)
  FILTER (LANG(?preprintLabel) = "en")
  FILTER (LANG(?label) = "en")
}
ORDER BY DESC(?articleDate)

Here are the live results from running that SPARQL query, embedded via the Wikidata Query Service:

At the time of writing, there are 15 duplicates (based on case insensitive string matching). I’ve begun curating the followed-by relationships, but am holding out since I might be able to come up with a script to automatically generate appropriate quickstatements.

Interestingly, the fact that these are different entries allows an alternate view that gives insight in turnover from preprint date to publication date. Considering that I typically preprint the paper and send for peer review simultaneously, this is an interesting statistic.

Usually I try and write blog posts about something I made or some insight that I got out of working on something, but I’m not really sure where to go from here. Ideally, I’d like to see the entirety of PubMed, PMC, other major scholarly article indexes, arXiv, bioRxiv, other preprint servers, and other bibliographic content automatically aligned in full on Wikidata. I’ve heard that there are concerns about the technical limitations about the service so this might not be feasible in the near future. In the mean time, if you’ve got some answers for my questions, please let me know.

Global Core Biodata Resources in the Bioregistry

2022-12-19T12:14:00+00:00

The Global Biodata Coalition released a list of Global Core Biodata Resources (GCBRs) in December 2022, comprising 37 life science databases that they considered as having significant importance (selected following this procedure). While the Bioregistry does not generally cover databases, many notable databases have one or more associated semantic spaces that are relevant for inclusion. Accordingly, 33 of 37 of the GCBRs (that’s 89%) have one or more directly-related prefixes in the Bioregistry. This post gives some insight into this landscape.

Background on the Bioregistry

The Bioregistry is a catalog of identifier schema for concepts in the life and natural sciences. These identifier schemata often arise from databases that create stable, locally unique identifiers for a given entity type. For example, the Universal Protein Resource (UniProt) creates stable, locally unique identifiers for proteins such as P0DP23 for Calmodulin-1. Similarly, the Chemical Entities of Biological Interest (ChEBI) creates stable, locally unique identifiers for chemicals such as 138488 for alsterpaullone. The Bioregistry contains records about these identifier schema including the regular expression pattern that can be used to validate locally unique identifiers (e.g., the UniProt one is quite complicated but the ChEBI one is simply a string that looks like a number ^\\d+$), the prefix that should be used when constructing compact URIs (CURIEs) (e.g., uniprot for UniProt), a URI format string that can be used to convert the local unique identifier into a URI (e.g., for usage in semantic web applications), and other useful metadata for the standardization of the identification of life and natural sciences concepts.

Some databases induce more than one identifier schema. For example, in addition to the identifier schema for proteins, UniProt also has disjoint identifier schemata for subcellular locations, diseases, and several others. In the case of UniProt, the main identifier schema is for proteins, and is therefore given the same prefix as the name of the database (i.e., uniprot). The prefixes for additional identifier schemata are constructed as subspaces using a dot-delimiter. Alternatively, some databases that mint multiple identifiers schemata, such as the Clinical Interpretation of Variants in Cancer (CIViC) database, do not have a “main” entity type and therefore use subspaces for all of its prefixes (e.g., civic.gid for genes, civic.vid for variants, etc.)

Back to GCBRs

It’s important to note that the Bioregistry maintains records for the identifier schemata, and not the databases themselves. Other catalogs like Wikidata and FAIRsharing already do an excellent job of maintaining records on databases and other larger efforts. With this background out of the way, we can return to the main question of this post: how do the Global Core Biodata Resources (GCBRs) relate to the Bioregistry? I stratified the list of 37 into four categories:

19 databases that have a single identifier schema (i.e., correspond 1-to-1 with Bioregistry records)
12 databases that have multiple identifier schemata (i.e., correspond 1-to-many with Bioregistry records)
2 databases that have a more complicated relationship to Bioregistry records
4 databases that don’t have identifier schemata (this turns out to be a very short list!)

GCBRs with a Single Identifier Schema

The following 19 databases in the GCBR list have a one-to-one correspondence with a Bioregistry prefix. In some cases, this categorization is partly subjective as many of these databases’ curators are heavily involved in other related efforts that their databases heavily reuse. For example, the Zebrafish Information Network is a model organism database that is heavily involved in the curation of ontologies for zebrafish anatomy and development (see zfa), developmental stages (see zfs), and phenotypes (see zfa).

Database	Bioregisry Prefix
Alliance of Genome Resources Knowledge Base	`agrkb`
Bacterial Diversity Metadatabase	`bacdive`
Chemical Entities of Biological Interest	`chebi`
EcoCyc	`ecocyc`
VEuPathDB ontology	`eupath`
Global Biodiversity Information Facility	`gbif`
Genome Aggregation Database	`gnomad`
Gene Ontology	`go`
InterPro	`interpro`
Mouse Genome Informatics	`mgi`
PDB Structure	`pdb`
European PubMed Central	`pmc`
PomBase	`pombase`
ProteomeXchange	`px`
Reactome	`reactome`
Rhea, the Annotated Reactions Database	`rhea`
Saccharomyces Genome Database	`sgd`
UCSC Genome Browser	`ucsc`
Zebrafish Information Network Gene	`zfin`

GCBRs with Multiple Identifier Schemata

The following 12 databases in the GCBR list have identifier schemata and therefore correspond to multiple Bioregistry records. Note, this list might be incomplete in cases where there are other relevant identifier schemata that haven’t been added to the Bioregistry. If you’re aware of one, please let me know or send a new prefix request!

Database	BioregistryPrefix	Name
CIViC	`civic.aid`	CIViC Assertion
	`civic.did`	CIViC Disease
	`civic.eid`	CIViC Evidence
	`civic.gid`	CIViC Gene
	`civic.sid`	CIViC Source
	`civic.tid`	CIViC Therapy
	`civic.vid`	CIViC Variant
BRENDA	`brenda`	BRENDA Enzmye (duplicate of `eccode`)
	`brenda.ligand`	BRENDA Ligand
	`brenda.ligandgroup`	BRENDA Ligand Group
	`bto`	BRENDA Tissue Ontology
ChEMBL	`chembl`	ChEMBL
	`chembl.compound`	ChEMBL Compound (subspace)
	`chembl.target`	ChEMBL Target (subspace)
Ensembl	`ensembl`	Ensembl Gene, Transcript, etc.
	`ensemblglossary`	Ensembl Glossary
FlyBase	`flybase`	FlyBase Gene
	`fbbt`	Drosophila Gross Anatomy
	`fbcv`	FlyBase Controlled Vocabulary
	`fbrf`	FlyBase Reference Report
	`fbsp`	Fly Taxonomy
	`fbtc`	FlyBase Cell Line
HGNC	`hgnc`	HGNC Gene
	`hgnc`	HGNC Gene Group
PANTHER	`panther.family`	PANTHER Family
	`panther.node`	PANTHER Node
	`panther.pathway`	PANTHER Pathway
	`panther.pthcmp`	PANTHER Pathway Comparison
PharmGKB	`pharmgkb.disease`	PharmGKB Disease
	`pharmgkb.drug`	PharmGKB Drug
	`pharmgkb.gene`	PharmGKB Gene
	`pharmgkb.pathways`	PharmGKB Pathway
Orphanet	`orphanet`	Orphanet
	`orphanet.ordo`	Orphanet Rare Disease Ontology
Rat Genome Database	`rgd`	Rat Gene
	`rgd.qtl`	Rat Quantitative Trait Loci
	`rgd.strain`	Rat Strain
UniProt	`uniprot`	UniProt Protein
	`uniprot.arba`	UniProt Association-Rule-Based Annotator
	`uniprot.chain`	UniProt Chain
	`uniprot.disease`	UniProt Disease
	`uniprot.isoform`	UniProt Isoform
	`uniprot.keyword`	UniProt Keyword
	`uniprot.location`	UniProt Location
	`uniprot.proteome`	UniProt Proteome
	`uniprot.ptm`	UniProt Post-translational Modification
	`uniprot.resource`	UniProt Resource
	`uniprot.tissue`	UniProt Tissue
	`uniprot.var`	UniProt Variant
	`uniparc`	UniProt Archive
	`uniref`	UniProt Reference Clusters
Wormbase	`wormbase`	Wormbase Gene
	`wbrnai`	Wormbase RNAi

GCBRs with a Confusing Relationship to Identifier Schemata

This list only gets two databases:

The reason that the DDBJ and ENA (and also NCBI GenBank, but it was not included as a GCBR) have a complicated relationship is because of their involvement in the International Nucleotide Sequence Database Collaboration (INSDC). This is a coordination effort between the DDBJ, EMBL-EBI and NCBI to promote interoperability between nucleotide sequence and related databases. Here’s a screenshot from their website that explains how certain services interact across these platforms:

Modeling this in the Bioregistry has proven very tricky. There are several ongoing discussions related to clarifying the relationships between these databases and various identifier schemata in the Bioregistry’s issue tracker (see #108, #118, and #131). Here are a few of the relevant prefixes:

ena.embl (ENA-specific)
bioproject (DDBJ-specific)
genbank (NCBI-specific)
insdc.run (non-specific)

If you’ve got some insight into these resources, please join for discussion on the Bioregistry issue tracker.

GCBRs with No Identifier Schemata

The four GCBRs that contain no identifier schemata have a common attribute: they all reuse other identifier schemata.

Clinical Genome Resource (uses hgnc, mondo, pharmgkb.pathways, etc.)
GENCODE (uses genbank)
GWAS Catalog (uses dbsnp, hgnc.symbol, and efo)
STRING (uses uniprot)

Interestingly, since these resources transitively use GenBank, dbSNP, and the Experimental Factor Ontology but these databases were not themselves included in the list of GCBRs. There are several other examples of databases transitively used by other GCBRs appearing in other parts of this categorization where this is also true. Without exhaustively going through all four resources, I thought I’d use the GWAS Catalog to illustrate how it looks when a database re-uses other identifier schemata-providing databases.

First, a genome-wide association study (GWAS) identifies statistical correlation between genomic markers such as single nucleotide polymorphisms (SNPs) and disease, phenotypes, or other traits across a large and diverse population, usually on the scale of thousands to tens of thousands of individuals. The GWAS Catalog is an EMBL-EBI database of published GWASs, their metadata (e.g., experimental design), and the statistically significant associations they identified. It maintains information about the following kinds of things:

Type	Example	Vocabulary
Variant	rs7329174	dbSNP (`dbsnp`)
Gene	ELF1	HGNC Gene Symbols (`hgnc.symbol`)
Region	2q37.1	-
Trait	breast carcinoma	Experimental Factor Ontology (`efo`)

A given association comprises a p-value for the association between a variant and a trait. Typically, there is a gene annotated to the SNP to make interpretation more simple, though this is no easy task. I’d suggest following Eric Faumann on Twitter (@Eric_Fauman) (or Mastodon if/when he moves there, because I’m not a big Twitter fan anymore) for interesting examples of this.

Interestingly, through the process of writing this post, I realized GWAS Catalog assigns such as GCST000858 which have their own unique semantic space and provider worthy of an entry in the Bioregistry, but that will probably have to wait until after winter break.

Afterthoughts

I’ve got a few parting comments on the construction of the list of GCBRs:

There appears to be an over-representation of gene/genome and model organism databases. This probably reflects the fact that these are the oldest kinds of bioinformatics resources. Overall, the list does include a nice variety of different kinds of resources.
There appears to be an over-representation of databases from the EBI, SIB, and NCBI. Since some of the criteria for inclusion were to demonstrate existence for more than five years, this makes it difficult for most databases to even be considered, as most don’t have appropriate funding or governance models for longevity. It’s also likely that there’s significant overlap between the professional networks of members of these large, prominent institutions and the Global Biodata Coalition, which could have lead to increased interest and familiarity with certain resources that ultimately were accepted.
There appears to be an over-representation of American and European resources. This might be partially due to the criteria that resources were required to have an English language version, but I think more generally it’s due to the unfortunate reality that there aren’t many reliable resources being developed and maintained in other locale for topics in scope for the GCBR list. I don’t think this is a reflection on a lack of interest for diversity and inclusion on the part of the Global Biodata Coalition. Further, efforts like INSDC seem like a good idea to coordinate efforts outside of the typical western research bubble, and the inclusion of the DDBJ in the list seems to pay deference to this sentiment.

While these observations might be thought-provoking, I’ve only made them at a subjective level. I think a more interesting follow-up would to be to use linked data approaches (e.g., through Wikidata) to aggregate relevant information together about each of these resources and make more objective summaries. To support that, I’ve created a collection on the Bioregistry bioregistry.collection:0000010 that contains the list of prefixes mentioned here.

Serendipitously, the scientific program committee of the 16th Annual International Biocuration Conference (Biocuration 2023) has invited the Executive Director of the Global Biodata Coalition, Guy Cochrane, to give a keynote presentation at the conference. This will be very timely, and I’m looking forwards to meeting him in person and chatting about this. If you’re a reader of my blog, you’re likely a creator, user, or both of resources like the one in the GCBR list. I’d suggest you consider submitting an abstract for the conference and meeting me in Padua in April!

A First Look at OpenCheck

2022-11-15T18:39:00+00:00

There has been legitimate concern about the future of Twitter over the last week due to its new ownership and management. This is pretty upsetting considering how great it’s been to use to connect to and to follow other researchers. OpenCheck is currently working to map Twitter handles to ORCID identifiers and capture the directed follow graph of researchers on Twitter in case the service becomes unusable in the near future. This post is about my initial exploration of the resource. Update in November 2024 - OpenCheck has been shut down.

How It Works

The OpenCheck website is pretty sleek. There are two big buttons to connect it to your Twitter account and ORCID account. After authorization, it uses the Twitter API to look your followers and follows. Then, it looks up if any of those Twitter accounts have also registered and adds it to this CSV which they make available under the permissive CC0 license.

Wikidata to the Rescue

My first thought was that there are tons of ORCID-Twitter connections available under the CC0 license through Wikidata that they could use to complement this content. The following SPARQL query retrieves all of these links:

SELECT ?item ?itemLabel ?orcid ?twitter
WHERE
{
  ?item wdt:P31 wd:Q5;
        wdt:P496 ?orcid;
        wdt:P2002 ?twitter .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}

This query can be run using the Wikidata Query Service by following this link and pressing the play button. As of November 15th, 2022, it returns 13,784 results.

Graph Machine Learning

The OpenCheck ORCID graph is a relatively small directed graph, so my second thought was that would be interesting to apply some graph machine learning methodologies to it. I wrote a simple script using GRAPE to learn low-dimensional vector representations (i.e., embeddings) for nodes in the graph (i.e., ORCID identifiers) that could be useful for clustering, classification, or other downstream machine learning tasks. These resulting embeddings for each ORCID identifier can be downloaded as a TSV file here and are summarized below:

I put this script in cthoyt/opencheck-embed and additionally configured GitHub Actions to automatically re-run the script every night to generate new embeddings for the newest dataset.

I had first tested the idea of continuous analysis built on top of GitHub’s continuous integration service by periodically generating differential reaction fingerprints on the Rhea reaction database in cthoyt/rhea-fingerprints. The OpenCheck ORCID graph is a different scenario because the data are updated in real time and aren’t versioned. Overall, I think these are both good examples of what is possible using modern, public infrastructure, and I would like to see them used in more and more scientific workflows.

It appears that OpenCheck is an effort from a company called Metarational, LLC run by John Beatty that has has a more long-term goal of supporting verification online (i.e., better than an $8 blue star). I’m excited to see what they do next!

If you want to make sure that your ORCID, Twitter, and other academic profiles on the internet are linked, check out my previous post on curating your own Wikidata profile.

Curating Publications on Wikidata

2022-02-12T18:10:00+00:00

This blog post is a tutorial on how to curate the links between a researcher and scholarly works (e.g., pre-prints, publications, presentations) on Wikidata using Scholia and the Author Disambiguator tool.

Ensure content is available

While Wikidata is full of useful scholarly information, it’s up to the community to include new content. Therefore, before we go about curating links between your Wikidata entry and your publications, we first have to ensure there are Wikidata entries for your publications. There are several ways to do this, but this tutorial will focus on getting the list of your publications from your citation manager.

This tutorial will assume you’re using Mendeley, a citation manager that can export EndNote XML documents. This tutorial might be later extended to include an example with Zotero, which can also export EndNote XML documents. After opening the application, follow these instructions:

Select the publications you want to put into Wikidata
Right click on one of them
Select “export”
Choose “EndNote XML” as the filetype

Uploading to Wikidata

Create an account on Wikidata. Keep in mind this is not the same as the Wikidata entry about you.
Store your Wikidata credentials in a configuration file at ~/.config/wikidata.ini (where ~ means your home directory). It should look like:
```
[wikidata]
username = 
password = 
```
Install Python.

Run the following in your terminal:

$ pip install citation-url[endnote]
$ python -m citation_url.endnote 

This will now parse your EndNote XML file and do its best to populate Wikidata with all the DOIs, PubMed identifiers, PMC identifiers, arXiv identifiers, PDF links, and other references.

Update your Wikidata entry

This tutorial already assumes you’ve created a Wikidata entry. If you haven’t done this already, go back and read this tutorial. Before continuing, make sure that you’ve edited the “synonyms” on your Wikidata entry so it has a wide variety of how your name might have been written in publication. This means different ordering of first/last name, different usages of single letters versus writing out in full, sometimes omitting the middle name, using dots after letters or not, etc.

This step is crucial because the curation interface in the next step relies on direct string matching between this list of synonyms and what the Wikidata Integrator was able to pull from CrossRef, EuropePMC, and other metadata resources.

Curate Wikidata using the Author Disambiguator

Follow this link to sign in to the Author Disambiguator tool using your Wikidata username/password. We’ll need this for later steps.
Scholia is a frontend for navigating scholarly data within Wikidata. Navigating to your Scholia curation by replacing in https://scholia.toolforge.org/author//curation#missing-author-resolving with your Wikidata identifier. It starts with a Q followed by some numbers coming directly after the title on your page. For example, mine is Q47475003. You can follow this link to see my curation page on Scholia as an example to make sure you’re in the right place for yours.
Click the link next to the specific text string you want to curate:
This will bring you to the Author Disambiguator page. Click the checkboxes on the publications that are yours (1), or use the boxes to check entire groups (2).
Scroll down past all the check boxes to the section labeled Potential Author Items. Select the radio box corresponding to your Wikidata entry (1), then click Link Selected Works to Author (2).
A new window will appear so you can track the status of the job, but you can close it and it will finish in the background. Give a few minutes, then your profile on Scholia will be updated.

You Should Use a Private Email on Publications

2022-02-06T21:55:00+00:00

While we were recently preparing to submit a manuscript, the lead author said they looked at my last few papers and noticed I always used a private email address instead of an institutional email address. They asked, perplexed, if they should also use my private email address with our submission. The answer was a resounding yes; always use a private email address. Here’s why.

I actually started thinking about this way back one thousand years ago in 2020 and started an interesting discussion on Twitter:

Lesson for young researchers - don’t use your institutional email address on papers.

You *will* leave, you won’t get to keep it, and you’ll miss out on lots of people who want to talk to you because of the interesting work you did.

Try @ORCID_Org instead :) #AcademicChatter
— Charles Tapley Hoyt (@cthoyt) June 23, 2020

I came back to this again when submitting the Gilda manuscript at the end of 2021, then got a distracted by an ankle injury and sort of lost track of all the blog posts I had been writing. Now I’m finishing it up in early February 2022. The rest of this post is an elaboration on my ideas and follow-up discussion on Twitter.

You can’t take it with you

If you’re a researcher who sometimes writes and submits publications, the chances are pretty high that you currently work at some kind of institution and will not always work at that institution. Here’s what might happen to your institutional email address when you leave:

Your old institution doesn’t care about you after you leave, and deletes your account the moment you walk out the door.
Your old institution doesn’t care about you after you leave, and says that they will continue your access for a limited amount of time after you leave to save face. Then it deletes your account.
Your old institution doesn’t care about you after you leave, and says that they will continue to provide technical support to you and all previous employees indefinitely with their infinite money and benevolence. I’m being sarcastic; this is really, really unlikely.

Whether 1, 2, or 3, if you used your institutional email address on a paper you published, it is now a dead link into the abyss. Anyone who might want to get in touch with you to chat about your research (or god forbid, ask you for code or data that you didn’t deposit in an appropriate place before publishing).

If you had used your personal email address, which won’t go away, then you wouldn’t have this problem.

Alternatively, some publishers allow you to annotate your ORCID identifier on to the manuscript, then you could potentially maintain your current working email address through ORCID, but again, not a lot of publishers support this (yet).

Who does this most affect?

As you become more senior, the chances of you moving institutions decreases. So this is an issue that disproportionately affects young researchers twice: first because you are harder to reach and second because your ability to network is more crucial as a young researcher than to an veteran one.

Do your best to disregard institutional policy

Institutions obviously want as much attribution as possible when you publish while working there, and using a prominent email address is one way to get that goal. Therefore, many institutions have a policy that you have to use your institutional email when publishing. A few courses of actions you could take:

Ask the editor to include multiple email addresses
Disregard institutional policy, it doesn’t support you as a researcher or a person.

I’m worried about getting a bunch of spam

I can’t speak for all situations, but I’ve been using my personal gmail address all over the internet and still haven’t got a ton of spam. It’s sitting at the bottom of this blog post if you want to prove me wrong. If you’re worried about this, make a new private email account you just use for publishing.

Difficulties for editors

One follow-up conversation I had on Twitter was with John R. Yates III, the editor-in-chief of the Journal of Proteome Research. He gave interesting insight that editors highly prefer institutional email addresses because they are perceived to be more trustworthy. There are several reasons why this is not true (e.g., you could use an outdated address, spoof it, etc.) but ultimately this serves to distract from adopting more sustainable ways of identifying authors and reviewers like with ORCID.

Relevant Twitter Threads

https://twitter.com/travisdrake/status/1577970169951010817

Abstracting the parameters of a Machine Learning Model

2022-02-06T18:00:00+00:00

As a follow-up to my previous post on refactoring and improving a machine learning model implemented with PyTorch, this post will be a tutorial on how to generalize the implementation of a multilayer perceptron (MLP) to use one of several potential non-linear activation functions in an elegant way.

We’ll pick up with the seventh (final) model version of the MLP from my previous post:

from itertools import chain

from more_itertools import pairwise
from torch import nn


class MLP7(nn.Sequential):
    def __init__(self, dims: list[int]):
        super().__init__(*chain.from_iterable(
            (
                nn.Linear(in_features, out_features),
                nn.ReLU(),
            )
            for in_features, out_features in pairwise(dims)
        ))

Incremental Improvements

This MLP uses a hard-coded rectified linear unit as the non-linear activation function between layers. We can initially generalize MLP7 to use a variety of non-linear activation functions by adding an argument to its __init__() function:

from itertools import chain

from more_itertools import pairwise
from torch import nn


class MLP8(nn.Sequential):
    def __init__(self, dims: list[int], activation: str = "relu"):
        if activation == "relu":
            activation = nn.ReLU()
        elif activation == "tanh":
            activation = nn.Tanh()
        elif activation == "hardtanh":
            activation = nn.Hardtanh()
        else:
            raise KeyError(f"Unsupported activation: {activation}")
        super().__init__(chain.from_iterable(
            (
                nn.Linear(in_features, out_features),
                activation,
            )
            for in_features, out_features in pairwise(dims)
        ))

The first issue with this MLP8 is it relies on a hard-coded set of conditional statements and is therefore hard to extend. It can be improved by using a dictionary lookup:

from itertools import chain

from more_itertools import pairwise
from torch import nn

activation_lookup: dict[str, nn.Module] = {
    "relu": nn.ReLU(),
    "tanh": nn.Tanh(),
    "hardtanh": nn.Hardtanh(),
}


class MLP9(nn.Sequential):
    def __init__(self, dims: list[int], activation: str = "relu"):
        activation = activation_lookup[activation]
        super().__init__(chain.from_iterable(
            (
                nn.Linear(in_features, out_features),
                activation,
            )
            for in_features, out_features in pairwise(dims)
        ))

Unfortunately, the approach in MLP9 rigid because it requires pre-instantiation of the activations. If we needed to vary the arguments to the nn.HardTanh class (i.e., the minimum and maximum values), the previous approach wouldn’t work. We can change the implementation to lookup on the class before instantiation then optionally pass some arguments:

from itertools import chain

from more_itertools import pairwise
from torch import nn

activation_lookup: dict[str, type[nn.Module]] = {
    "relu": nn.ReLU,
    "tanh": nn.Tanh,
    "hardtanh": nn.Hardtanh,
}


class MLP10(nn.Sequential):
    def __init__(
        self,
        dims: list[int],
        activation: str = "relu",
        activation_kwargs: None | dict[str, any] = None,
    ):
        activation_cls = activation_lookup[activation]
        activation = activation_cls(**(activation_kwargs or {}))
        super().__init__(chain.from_iterable(
            (
                nn.Linear(in_features, out_features),
                activation,
            )
            for in_features, out_features in pairwise(dims)
        ))

MLP10 is a big improvement in terms of flexibility, but it has a few remaining practical issues:

you have to manually maintain the activation_lookup dictionary,
you can’t pass a pre-instantiated instance of an activation class via the activation argument
you have to get the casing of the string keys just right
the default is hard-coded as a string, which means this has to get copied (error-prone) in any place that creates an MLP
you have to re-write this logic for all of your classes

The `class-resolver`

Before showing MLP11, the final solution, I want to first describe the class-resolver package. Its job is to make it easy to generate a dictionary-like object that you can use to look up classes (like we prepared for MLP10). It’s smart and takes care of several things for you:

Automatically assigns keys in the dictionary based on the class name. If all the classes in the resolver share a suffix, it automatically strips it.
It uses some simple string normalization during lookup, so it’s insensitive to capitalization, varied usage of underscores, or other punctuation.
It keeps track of a default value to grab when you pass None
It allows for classes and instances to be passed through

After making a ClassResolver instance, you can use the ClassResolver.lookup() function to get the class you need:

from class_resolver import ClassResolver
from torch import nn

activation_resolver = ClassResolver(
    [nn.ReLU, nn.Tanh, nn.Hardtanh],
    base=nn.Module,
    default=nn.ReLU,
)

# Default lookup
assert nn.ReLU == activation_resolver.lookup(None)

# Name-based lookup
assert nn.ReLU == activation_resolver.lookup("relu")
assert nn.ReLU == activation_resolver.lookup("ReLU")

# Class-based lookup
assert nn.ReLU == activation_resolver.lookup(nn.ReLU)

Built on top of the ClassResolver.lookup() function is the ClassResolver.make() function, which first looks up the class, then gives you an instance of it (optionally using keyword arguments you pass).

# Default instantiation
assert nn.ReLU() == activation_resolver.make(None)

# Name-based instantiation
assert nn.ReLU() == activation_resolver.make("relu")
assert nn.ReLU() == activation_resolver.make("ReLU")

# Class-based instantiation
assert nn.ReLU() == activation_resolver.make(nn.ReLU)

# Class-based instantiation w/ keyword arguments
assert nn.Hardtanh(0.0, 6.0) == activation_resolver.make("hardtanh", {
    "min_val": 0.0, "max_value": 6.0
})

Bringing it All Together

Let’s apply that to MLP10 and make our final MLP11:

from itertools import chain

from class_resolver import ClassResolver
from more_itertools import pairwise
from torch import nn

activation_resolver = ClassResolver(
    [nn.ReLU, nn.Tanh, nn.Hardtanh],
    base=nn.Module,
    default=nn.ReLU,
)


class MLP11(nn.Sequential):
    def __init__(
        self,
        dims: list[int],
        activation: None | str | nn.Module | type[nn.Module] = None,
        activation_kwargs: None | dict[str, any] = None,
    ):
        super().__init__(chain.from_iterable(
            (
                nn.Linear(in_features, out_features),
                activation_resolver.make(activation, activation_kwargs),
            )
            for in_features, out_features in pairwise(dims)
        ))

Now, you can instantiate the MLP with any of the following:

MLP11(dims=[10, 200, 40])  # uses default, which is ReLU
MLP11(dims=[10, 200, 40], activation="relu")  # uses lowercase
MLP11(dims=[10, 200, 40], activation="ReLU")  # uses stylized
MLP11(dims=[10, 200, 40], activation=nn.ReLU)  # uses class
MLP11(dims=[10, 200, 40], activation=nn.ReLU())  # uses instance

MLP11(dims=[10, 200, 40], activation="hardtanh",
      activation_kwargs={"min_val": 0.0, "max_value": 6.0})  # uses kwargs
MLP11(dims=[10, 200, 40], activation=nn.HardTanh,
      activation_kwargs={"min_val": 0.0, "max_value": 6.0})  # uses kwargs
MLP11(dims=[10, 200, 40], activation=nn.HardTanh(0.0, 6.0))  # uses instance

In practice, it makes sense to stick to using the strings in combination with hyper-parameter optimization libraries like Optuna.

Because the usage of class-resolver for resolving activation functions from PyTorch is so common, we’ve made it available through contrib module in class_resolver.contrib.torch. In fact, the activation_resolver comes with some extra logic to automatically grab all activation modules from torch.nn.modules.activation. Therefore, we can rewrite the example for MLP11 to simply import it.

from itertools import chain

from class_resolver.contrib.torch import activation_resolver
from more_itertools import pairwise
from torch import nn


class MLP(nn.Sequential):
    def __init__(
        self,
        dims: list[int],
        activation: None | str | nn.Module | type[nn.Module] = None,
        activation_kwargs: None | dict[str, any] = None,
    ):
        super().__init__(chain.from_iterable(
            (
                nn.Linear(in_features, out_features),
                activation_resolver.make(activation, activation_kwargs),
            )
            for in_features, out_features in pairwise(dims)
        ))

Refactoring a Machine Learning Model

2022-02-06T15:45:00+00:00

This blog post is a tutorial that will take you from a naive implementation of a multilayer perceptron (MLP) in PyTorch to an enlightened implementation that simultaneously leverages the power of PyTorch, Python’s built-ins, and some powerful third party Python packages.

This tutorial is going to assume the following imports for all code blocks:

import itertools as itt

import more_itertools
from torch import nn
from torch.nn import functional as F

itertools is a builtin library for helping deal with lists, sets, and other iterables.
more_itertools is a third-party extension to itertools, highly regarded in the Python community.
You should already be familiar with PyTorch and writing your own subclasses of torch.nn.Module by implementing your own __init__() and forward() functions.

This tutorial isn’t really about the theory nor application of machine learning models - it’s just about the best ways to implement them. I’m also going to commit the sin of omitting docstrings and a lot of type annotations, since most of the MLP should be pretty obvious.

Let’s start with a naive implementation, that reflects some old habits from C or Java programming:

import torch
from torch import nn
from torch.nn import functional as F


class MLP1(nn.Module):
    def __init__(self, dims: list[int]):
        super().__init__()
        layers = []
        for i in range(len(dims) - 1):
            in_features, out_features = dims[i], dims[i + 1]
            layers.append(nn.Linear(in_features, out_features))
        self.layers = nn.ModuleList(layers)

    def forward(self, x: torch.FloatTensor) -> torch.FloatTensor:
        rv = x
        for layer in self.layers:
            rv = layer(rv)
            rv = F.relu(rv)
        return rv

Incremental Improvements

MLP1 uses the dreaded range(len(...)) pattern, which can almost always be replaced with direct iteration. However, in this case, it uses the index to get the next element with it. Luckily, more_itertools has a function pairwise() that does exactly this. MLP1 can then be refactored into:

import torch
from more_itertools import pairwise
from torch import nn
from torch.nn import functional as F


class MLP2(nn.Module):
    def __init__(self, dims: list[int]):
        super().__init__()
        layers = []
        for in_features, out_features in pairwise(dims):  # this line changed
            layers.append(nn.Linear(in_features, out_features))
        self.layers = nn.ModuleList(layers)

    def forward(self, x: torch.FloatTensor) -> torch.FloatTensor:
        rv = x
        for layer in self.layers:
            rv = layer(rv)
            rv = F.relu(rv)
        return rv

The application of F.relu in forward() is suspect for a few reasons:

Because it lives as a hard-coded call in forward(), there’s no way to make it into a hyper-parameter that can be chosen by a user
Because it’s the functional form F.relu and not nn.ReLU, it can’t be stacked with other layers

MLP2 can be refactored to address both of those by using the modular form nn.ReLU in the layers after creating each nn.Linear.

import torch
from more_itertools import pairwise
from torch import nn


class MLP3(nn.Module):
    def __init__(self, dims: list[int]):
        super().__init__()
        layers = []
        for in_features, out_features in pairwise(dims):
            layers.append(nn.Linear(in_features, out_features))
            layers.append(nn.ReLU())  # this line changed
        self.layers = nn.ModuleList(layers)

    def forward(self, x: torch.FloatTensor) -> torch.FloatTensor:
        rv = x
        for layer in self.layers:
            rv = layer(rv)
        return rv

Now that the forward() function is just a successive application of layers, it can be exchanged with a nn.Sequential. MLP3 can be refactored to look like:

import torch
from more_itertools import pairwise
from torch import nn


class MLP4(nn.Module):
    def __init__(self, dims: list[int]):
        super().__init__()
        layers = []
        for in_features, out_features in pairwise(dims):
            layers.append(nn.Linear(in_features, out_features))
            layers.append(nn.ReLU())
        self.layers = nn.Sequential(*layers)  # this line changed

    def forward(self, x: torch.FloatTensor) -> torch.FloatTensor:
        return self.layers(x)

The following two improvements will make the construction of the layers list that goes in nn.Sequential much more elegant. First, we’ll refactor MLP4 to use the extend function of a list rather than append:

import torch
from more_itertools import pairwise
from torch import nn


class MLP5(nn.Module):
    def __init__(self, dims: list[int]):
        super().__init__()
        layers = []
        for in_features, out_features in pairwise(dims):
            layers.extend((
                nn.Linear(in_features, out_features),
                nn.ReLU(),
            ))
        self.layers = nn.Sequential(*layers)

    def forward(self, x: torch.FloatTensor) -> torch.FloatTensor:
        return self.layers(x)

An Aside on List Comprehensions

As we prepare to refactor MLP5, we’ll take a short aside to discuss list comprehensions in Python. Here are a few resources to get you started:

The minimum amount of information you need to know for this tutorial is that anytime we see code that looks like

old_list = ...
new_list = []
for x in old_list:
    new_list.append(transform(x))

we know that we can transform it using a list comprehension like

old_list = ...
new_list = [
    transform(x)
    for x in old_list
]

There’s an analogous pattern for when we’re successively extending a list, like what we did when writing MLP5. If we see code that looks like

old_list = ...
new_list = []
for x in old_list:
    new_list.extend(transform(x))

we can transform it into something more elegant using itertools.chain.from_iterable() like

from itertools import chain

old_list = ...
new_list = list(chain.from_iterable(
    transform(x)
    for x in old_list
))

While this may be a few extra lines (because it’s broken up for readability), it has the advantage that it’s only one logical line and can be used in more clever ways.

Bringing it All Together

We’ll apply this template to our code to get a one-liner for instantiating our nn.Sequential (though notice it’s again broken up onto multiple lines for readability):

from itertools import chain

import torch
from more_itertools import pairwise
from torch import nn


class MLP6(nn.Module):
    def __init__(self, dims: list[int]):
        super().__init__()
        self.layers = nn.Sequential(*chain.from_iterable(
            (
                nn.Linear(in_features, out_features),
                nn.ReLU(),
            )
            for in_features, out_features in pairwise(dims)
        ))

    def forward(self, x: torch.FloatTensor) -> torch.FloatTensor:
        return self.layers(x)

Finally, since we’re now just creating a module that wraps the exact functionality of nn.Sequential, it’s possible to directly subclass nn.Sequential. We’ll refactor on MLP6 to get our final result:

from itertools import chain

from more_itertools import pairwise
from torch import nn


class MLP7(nn.Sequential):
    def __init__(self, dims: list[int]):
        super().__init__(*chain.from_iterable(
            (
                nn.Linear(in_features, out_features),
                nn.ReLU(),
            )
            for in_features, out_features in pairwise(dims)
        ))

MLP7 is now a much more simple implementation that uses a few neat tricks to reduce error-prone logic. I hope you enjoy applying these patterns to your own models, and if you have any other ideas you’d like me to include here, please leave comment or get in touch!

While we were originally aiming at reducing complexity, this model still has the issue that it contains a hard-coded reference to the ReLU non-linear activation function, which could be easily generalized to support alternate non-linear activation functions. In my next post, I’ll demonstrate the thought process behind this and the ultimate solution using the class-resolver.

The Official Rules of Python Packaging Speedrunning

2022-01-24T00:30:00+00:00

I figured over the holiday break or early days of the new year, I’d catch up on some serious blogging. Instead, here’s my first post of 2022: a silly take on a topic I actually care a lot about. Here are the rules for Python Packaging Speedruns.

What’s a speedrun?

Speedrunning is the sport of finishing a video game as fast as possible. Some games are so broken (e.g., The Legend of Zelda: Ocarina of Time) that it’s not really even that interesting to just get to the end, so many games introduce additional variants to Any% such as 100%. For each game, the speedrunning community decides when the timer should start (e.g., most games have a start screen and this is usually pretty obvious) and when the timer should end (e.g., when the credits screen rolls or when the last user input is given on a final boss).

What’s a packaging speedrun?

The video game speedrunning community has the benefit that the same speedrun can be done over and over again, with pretty reliable conditions on how it starts and end. However, each Python packaging speedrun will be done on a different repository. It doesn’t really make sense for more than one person to do a packaging speedrun on the same repository, since the goal besides being fast is to make practical improvements to unpackaged code and submit a pull request to original authors.

Python Packaging Speedrun Any%

I’m going to propose a set of minimal rules for a Python packaging speedrun, which I’ll designate as Any% because it’s the simplest version that has the least restrictions. Other categories can take these rules and build on top of them.

Rule 1: Timing

Python packaging speedruns are timed in the following way:

Start the clock when you click the fork button on GitHub
End the clock after all commits have been made to your fork AND a pull request has been made.

Rule 2: Package Layout

To ensure imports aren’t sneakily being done via directory structure, the src/ layout is mandatory for all Python packaging speedruns. Read Hynek Schlawack’s excellent explanation why this restriction is necessary here.

Rule 3: Minimal Unit Testing

To make sure that packaging was done properly, automated unit tests should check that the package can be imported using the same directory structure where tests/test_trivial.py contains the following:

# test_trivial.py

import importlib
import unittest

class TestTrivial(unittest.TestCase):
    def test_import(self):
        name = ""
        module = importlib.import_module(name)
        self.assertIsNotNone(module)

Rule 4: Automated Application of Testing

This file should be run with tox on a tox.ini that minimally contains the following [testenv]:

# tox.ini

[tox]
envlist =
    py

[testenv]
commands =
    pytest tests/
deps =
    pytest
description = Run unit tests.

Note the [tox] section was included for ergonomic usage of the tox command from the command line.

Rule 5: Provenance

Finally, like video game speedruns, there needs to be video proof, preferably uploaded to YouTube or witnessed on Twitch.

Potential Variants

Since the Any% is pretty simple, I think there is room for all sorts of variants including:

Passes flake8 with a pre-determined set of plugins (e.g., flake8-black, flake8-isort, flake8-docstrings, pydocstyle)
Passes flake8 with previous plugins and harder ones (e.g., flake8-print, darglint)
Passes mypy

Parting thoughts

Each speedrun will require a different amount of effort to achieve these things based on the size of the package and what state the original code was in. Obviously, Python packaging speedruns can’t be so rigorously compared as video game speedruns. That’s okay.

I posted my first Python packaging speedrunning on YouTube this evening. It doesn’t actually follow the rules I proposed here because I started thinking about this after I was done tweeting about it. Please let me know if you have any ideas on how to improve these rules, have an idea for a new category, or if you want me to link to your Python packaging speedrun video.

How to Pick a Unique Prefix

2021-12-17T13:17:00+00:00

After the recent incident on the OBO Foundry where an inexperienced group submitted a new ontology request using a prefix that already existed in the BioPortal, there has been a renewed interest in implementing an automated solution to protect against this.

The Big Picture

A more general issue is that there can be prefix conflicts between different registries. I found a few examples of this happening while building the Bioregistry in the Spring 2021. Notably, this included the conflict between the Geographical Entity Ontology and the Gene Expression Omnibus which both used the prefix geo in the OBO Foundry and Identifiers.org, respectively. These conflicts needed thoughtful mediation. After discussing on GitHub with Bill Hogan, the responsible author of the Geographical Entity Ontology, we decided that to use the prefix geogeo in the Bioregistry. The Gene Expression Omnibus maintained its usage of geo due to its much wider usage and longer history.

As a follow-up, I began curating a list of conflicts and implemented a technical solution in the Bioregistry’s nightly automated alignment workflow to prevent automated alignment between known conflicting prefixes from different registries, but this only a partial solution to a problem that is ultimately dependent on having confidence in the external resources (which is indeed its own issue).

What the Bioregistry Aligns

A small, select set of registries are fully automatically ingested in the Bioregistry which includes (at the time of writing) Identifiers.org, Name-to-Thing, the OBO Foundry, and the Ontology Lookup Service. The remaining registries are excluded for a variety of reasons including redundancy with other resources (e.g., AberOWL and OntoBee), a lack of modernization or alignment (e.g., NCBI’s registry), general inclusion of non-nomenclature resources (e.g., UniProt’s registry, FAIRsharing), and a lack of minimum quality standards (e.g., BioPortal). A summary and slightly more detailed explanation about these sources can be found here.

In many ways, the fact that the Bioregistry fully imports some resources and automatically aligns with others

The Bioregistry imports Identifiers.org, OBO Foundry, and N2T as well as many other resources (see for a full list), so it can be a one-stop shop for most resources. However, it does not import all of BioPortal, so users should check there too.

How to Check Your Prefix is Unique

Ultimately the point of this post is to present a workflow for any potential who want to check their new ontology request has a unique prefix (which will soon be a technical requirement in the OBO Foundry). Because the Bioregistry imports many resources, it’s sufficient to just check the Bioregistry and BioPortal (assuming you’re interested in respecting the BioPortal content).

Manual Check

The first way to check if your prefix is unique is to manually read through some of the sites.

Resource	Home Page	Prefix List
Bioregistry	https://bioregistry.io	https://bioregistry.io/registry
Bioportal	https://bioportal.bioontology.org	https://bioportal.bioontology.org/ontologies

While the BioPortal API is locked behind API key access, the Bioregistry additionally has a search endpoint at https://bioregistry.io/api/search?q=...

Data Dumps

The second way to check if your prefix is unique is by comparing it to full dumps of the Bioregistry and BioPortal. The Bioregistry can be downloaded in several formats that are updated on a nightly basis:

BioPortal doesn’t offer any first-party data dumps, but the Bioregistry generates one nightly here

Programmatic Access

The third way to check if your prefix is unique is by comparing it to the Bioregistry and BioPortal using code from the bioregistry python package (which is updated nightly).

Programmatic way to check if something is in the Bioregistry:

import bioregistry

query = "EPSO"
available_in_bioregistry = bioregistry.normalize_prefix(query) is None

Programmatic way to check if something is in BioPortal:

from bioregistry.external.bioportal import get_bioportal

query = "EPSO"
bioportal_dict = get_bioportal()
available_in_bioportal = query not in bioportal_dict

Being high quality and enabling external contribution and improvements are core to the philosophy of the Bioregistry. While no solution is perfect for listing all possible prefixes and it might be necessary to do a bit of extra googling before picking a prefix, this is a great place to start. If during the process of choosing a prefix you find you might create a conflict, please consider also suggesting a new entry in the Bioregistry.

A Glossary for the Bioregistry and Biopragmatics Stack

2021-10-07T13:26:00+00:00

There are a lot of terms that I’ve been throwing around when talking about the Bioregistry, so this blog post is a first draft of a gloassary of all of them.

Later, I will revise this further and put it either on the Bioregistry website, or make a totally new repo on the Biopragmatics GitHub organization.

Semantic spaces

While a controlled vocabulary enumerates a set of named entities, a semantic space enumerates a set of stable local identifies for entities. Most high-quality controlled vocabularies also assign local identifiers for their named entities and are also semantic spaces. For example, the Chemical Entities of Biological Interest (ChEBI) is a well-known ontology in the biomedical domain that is both a controlled vocabulary and a semantic space.

The term local identifier is synonymous with identifier and accession, but has the added qualifier local as a reminder that two semantic spaces may use the same one. For example, the Chemical Entities of Biological Interest (ChEBI) entry for 6-methoxy-2-octaprenyl-1,4-benzoquinone and the Human Disease Ontology (DOID) entry for gender identity disorder share the local identifier of 1234.

Formalizing local identifiers

It’s often useful to have a regular expression that describes local identifiers of a given semantic space. For example, both ChEBI and DOID use local identifiers that look like numbers, which match the regular expression ^\d+$. The ^ and $ denote the beginning and end of the regular expression and appear exactly the same in all regular expressions for local identifiers. The \d will match a number and the + means that the preceding token (\d) can be matched one or more times in a row.

It’s important to remember that identifiers might look like numbers, but they should never be treated as such. For example, the Gene Ontology (GO) uses identifiers that are left-padded with zeros like in 0032571 for response to vitamin K. The regular expression pattern for GO entries is ^\d{7}$, since there are always exactly seven numbers. Regular expressions don’t have a straightforward way to describe numbers that are left padded with zero, so keep in mind that this is approximation is a good balance between precision and simplicity.

There are a variety of patterns used for identifiers, including integers (^\d+$; e.g., PubMed), zero padded integers (^\d{7}$; e.g., GO and other OBO Ontologies), universally unique identifiers (UUIDs; e.g., NCI Pathway Interaction Database, NDEx), and many other variations.

Origins

Semantic spaces arise from several kinds of resources such as:

Ontologies like the Gene Ontology (GO), Chemical Entities of Biological Interest (ChEBI), and Experimental Factor Ontology (EFO)
Controlled Vocabularies like Entrez Gene, InterPro, and FamPlex
Databases like Protein Data Bank and Gene Expression Omnibus

Completeness

Semantic spaces typically fall into one of several “completeness” categories:

Complete by Definition like Enzyme Classification
Complete, but Subject to Change like HGNC
Always Incomplete like Chemical Entities of Biological Interest (ChEBI) and the Protein Data Bank (PDB)

Scope

Semantic spaces have a variety of scopes:

Single entity type like HGNC
A few entity types like the Gene Ontology (GO)
Many entity types like Medical Subject Headings (MeSH), Unified Medical Language System (UMLS), National Cancer Institute Thesaurus (NCIT)

Relationship to Projects and Organizations

Semantic spaces do not always correspond one-to-one with projects, such as how the ChEMBL database contains both the ChEMBL Compound and ChEMBL Target semantic spaces or how the Uber Anatomy Ontology (UBERON) contains both UBERON and UBPROP semantic spaces for terms and properties, respectively.

Providers

A provider returns information about entities from a given semantic space. A provider is characterized by a URI format string, or URI formatter, into which a local identifier from its semantic space can be substituted for a special token (e.g., $1). For example, the following formatter can be used to get a web page about HRAS by replacing $1 in the URI format string http://www.genenames.org/cgi-bin/gene_symbol_report?hgnc_id=$1 by its HGNC identifier, 5173.

Well-behaved URI format strings only have one instance of the special token that occurs at the end. Poorly-behaved URI format strings may have additional characters following the special token as in http://rebase.neb.com/rebase/enz/$1.html for REBASE or as in http://eawag-bbd.ethz.ch/$1/$1_map.html for the UM-BBD Pathway database.

Content Type

While providers typically return human-readable HTML, they can also return many other data types, including:

Images (e.g., https://www.ebi.ac.uk/chebi/displayImage.do?defaultImage=true&chebiId=132964 for the ChEBI entry on fluazifop-P-butyl)
XML (e.g., https://www.uniprot.org/uniprot/P10636.xml for UniProt entry on human Microtubule-associated protein tau)
JSON (e.g., https://gen3.biodatacatalyst.nhlbi.nih.gov/ga4gh/drs/v1/objects/0000ffeb-36e0-4a29-b21d-84423bda979d for NCBI’s BioData Catalyst)
RDF

Providers can return any other information that can be transferred via HTTP, FTP, or related data transfer protocols. Alternatively, content negotiation could be used to return multiple kinds of data from the same provider URI.

Responsibility

Most semantic spaces have an associated first-party provider that returns information via a web page. Some semantic spaces, like ChEBI, have several first-party providers for different content types (e.g., HTML, image). Some semantic spaces, like Entrez Gene, have additional external providers, including databases that use its identifiers like the Comparative Toxicogenomics Database. Some semantic spaces, such as many OBO ontologies, do not have an associated first party provider and rely solely on third party browsers like AberOWL, OntoBee, and the Ontology Lookup Service.

Naming things on the semantic web

There are two (mostly) interchangeable formalisms for naming things in the semantic web: uniform resource identifiers (URIs) and compact uniform resource identifiers (CURIEs).

Uniform Resource Identifiers (URIs)

The semantic web community has adopted the internationalized resource identifier (IRI) as the de facto standard for naming entities. In practice, usage is often restricted to IRIs that are also uniform resource identifiers (URIs) (i.e., they only use ASCII characters) and that are also valid uniform resource locators (URLs) (i.e., they point to a web page). In applied semantic web contexts like biomedicine, the subtleties between URLs, URIs, and IRIs are disregarded and the term URI is preferred such as in the seminal paper Identifiers for the 21st Century. A more detailed explanation on the difference between URLs, URIs, and IRIs can be found here.

For a given semantic space like ChEBI, URIs can usually be constructed given two parts:

A URI prefix (in red)
A local identifier (in orange)

All URIs from the same semantic space have the same URI prefix (in red), but a different local identifier (in orange). Here’s an example, using the ChEBI local identifier for alsterpaullone:

https://www.ebi.ac.uk/chebi/searchId.do?chebiId=138488

There may be potentially many URI prefixes corresponding to the same semantic space and therefore many URIs describing the same entity. For example, ChEBI also serves images with:

https://www.ebi.ac.uk/chebi/displayImage.do?defaultImage=true&imageIndex=0&chebiId=138488

Compact Uniform Resource Identifiers (CURIEs)

A compact uniform resource identifier (CURIE) allows for the replacement of a URI prefix in a URI with a short prefix. As a short recapitulation of the W3C specification, a CURIE has three parts:

A prefix (in red)
A delimiter (in black)
A local identifier from the given semantic space (in orange)

Since everyone agrees on what ChEBI is within the biomedical domain, it makes sense to use chebi as the prefix for ChEBI local identifiers. However, there is no globally unique set of prefixes used across the semantic web (nor should there be). Therefore, when using CURIEs, you need at minimum a prefix map (described below) and ideally a registry that stores additional metadata about each prefix.

Here’s the same example as in the URI section above for alsterpaullone, but now condensed into a CURIE:

chebi:138488

Converting between URIs and CURIEs

A prefix map associates each prefix to exactly one URI prefix. It can be used to expand CURIEs into URIs. Disregarding (for now) how to choose the best URI prefix, one potential prefix map that could be used to expand the example CURIE for alsterpaullone could be:

{
  "chebi": "https://www.ebi.ac.uk/chebi/searchId.do?chebiId="
}

A simple algorithm for expanding a CURIE to a URI is as follows:

Split the CURIE on the first instance of the delimiter, usually a colon :
Look up the left-hand side of the split (i.e., the prefix) in the prefix map
String concatenate the resulting URI prefix with the right-hand side of the split (i.e., the local identifier)

A reverse prefix map associates one or more URI prefixes to each prefix. It can be used to contract URIs into CURIEs. Disregarding (for now) how to chose the best prefix for each URI prefix, one potential reverse prefix map that could be used to contract the two example URIs for alterpaullone could be:

{
  "https://www.ebi.ac.uk/chebi/searchId.do?chebiId=": "chebi",
  "https://www.ebi.ac.uk/chebi/displayImage.do?defaultImage=true&imageIndex=0&chebiId=": "chebi"
}

Because it’s possible some URI prefixes might overlap, it’s a good heuristic to check a given URI against a reverse prefix map in decreasing order by URI prefix length.

Poorly Behaved URIs

Unfortunately, not all URLs that provide information about entities in semantic spaces can be trivially split into a URI prefix and a local identifier. For example, the REBASE entry for Asp14HI has the URI http://rebase.neb.com/rebase/enz/101.html. Note the pesky .html at the end, which if removed, causes an HTTP 404 error due to the implementation of the REBASE website.

While this creates a big problem for parsing URIs into CURIEs, it’s still possible to generate a URI from a CURIE given a slight variation on a prefix map, which relies on the previously described notion of URI formatters (see the section above on Providers)

A URI prefix corresponds to a special case of a URI formatter where there is exactly one instance of $1 that appears at the end of the string. Therefore, it is more valuable to curate URI formatters and programmatically generate prefix maps when possible. The fact that some URIs are hard to construct easily is also one of the motivations for resolver services, described in a later section.

Open Biomedical Ontologies CURIEs

The Open Biomedical Ontologies (OBO) Foundry provides a persistent URL service (PURL) to create stable URIs for biomedical entities curated in their ontologies (e.g., Human Disease Ontology, Phenotype And Trait Ontology). They have four parts:

A URI prefix (in red; always the same)
An ontology prefix (in orange)
A delimiter (in black; always the same)
An ontology local identifier (in blue)

http://purl.obolibrary.org/obo/DRON_0000005

Confusingly, the entire combination of the ontology’s prefix, the delimiter, and the ontology’s local identifier (e.g., DRON_0000005) are considered in some contexts as a local identifier in a theoretical semantic space for OBO, whose URI prefix is http://purl.obolibrary.org/obo/. This confusion lead to services like Identifiers.org to denote these ontologies as having the “namespace embedded in the local unique identifier” and therefore include the prefix again in the regular expression pattern describing the local identifiers, e.g. ^DOID:\d+$ for the Human Disease Ontology.

This notation of the regular expression makes no sense for several reasons:

The regular expression should correspond to the local identifiers of a semantic space like DOID, not a registry like the OBO PURL system.
If you follow the simple algorithm for constructing a CURIE from a prefix and identifier, you end up with identifiers that look like CURIEs like DOID:11337 or redundant CURIEs that look like DOID:DOID:11337.
Identifiers.org doesn’t even handle CURIEs constructed following the rules for embedding the prefix in the local identifier.
It creates ambiguities in spreadsheets where columns are supposed to contain local identifiers or CURIEs.

The solution is simply to drop the entire notion of namespaces embedded in local unique identifiers. Since this would require updating a lot of data in a lot of places, the interim solution is to programmatically normalize identifiers and CURIEs in the meantime to remove instances of this redundancy.

Registry

A registry is a special kind of semantic space that enumerates other semantic spaces and assigns them local identifiers. Due to the connection with prefix maps and CURIEs, the local identifiers in registries are also colloquially called prefixes.

A registry also collects additional metadata about each semantic space, including its name, its canonical prefix, its stylized prefix, additional prefix synonyms, its homepage, an example local identifier, a regular expression pattern for local identifiers, and one or more URI format strings from both first-party and third-party sources. However, there are a wide variety of metadata standards across various biomedical and semantic web registries, and not all fields are included.

Like with semantic spaces, a high-quality registry should have an associated first-party provider that comprises a website for exploring its entries and their associated metadata.

Metaregistry

A metaregistry is a special kind of registry that assigns local identifiers to a collection of registries; it could even contain an entry about itself. It collects additional metadata about each registry, such as a description of its metadata standards and capabilities. Most importantly, a metaregistry contains mappings between equivalent entries in its constituent registries. Before the publication of this article, to the best of our knowledge, there were no dedicated metaregistries. Some registries such as FAIRSharing and the MIRIAM/Identifiers.org registry contain limited numbers of entries referring to other registries (e.g., BioPortal), but they neither delineate these records as representing registries, provide additional metadata, nor provide mappings.

The only metaregistry in the biomedical domain is the Bioregistry.

Resolver

A resolver uses a registry to generate a URI for a given CURIE based on the registry’s default provider for the semantic space with the given prefix, then redirects the requester to the constructed URI. Resolvers are different from providers in that they are general for many semantic spaces and do not host content themselves. Two well-known resolvers are Identifiers.org and Name-To-Thing.

Lookup Service A lookup service is like a provider but generalized to provide for many semantic spaces. They typically have a URI format string into which a compact identifier can be placed like OntoBee, but many require more complicated programmatic logic to construct. Some well-known lookup services are the OLS, AberOWL, OntoBee, and BioPortal.

How to Curate the INDRA Database

2021-09-16T12:30:00+00:00

With the recent paper on Gilda and approaching INDRA 2 and INDRA database papers coming up, I’ve put together a visual guide on how to curate statements extracted by INDRA through the web interface at https://db.indra.bio.

Navigate to the INDRA database portal using this link.

Enter search text. This can be a gene symbol, chemical, or any other biomedical entity. In this tutorial, we’ll search for “AKT1.”

Now, click the “Ground with Gilda” button. Gilda will automatically look up the most likely database identifier that goes with your search string. It’s quite smart and can even disambiguate different senses of the same word. If it’s not sure, it will ask you to pick from a list.

In this example, I chose AKT1, which was pretty easy for Gilda to ground and didn’t need me to check. It also reports the confidence in the grounding and the namespace to which it grounded in the box. Click the search button to get going to the next step!

Now you will see the search results. On the left, it has many kinds of statements that INDRA models. They’re hierarchical, meaning you can click on one to expand to more specific statement types.

On the right are badges for the different sources that give evidence for each statement. The ones with the black text correspond to databases, like BioGRID, and the ones with white text correspond to reading systems, like REACH.

The next image shows the expansion of AKT affects BAD to AKT1 phosphorylates BAD. Note that the third level is the same as the second - this is because there are actually some more specific phosphorylation events contained in here as well!

Open up the curation interface by pressing the pencil button next to the evidence you want to curate. Note that some sources, like BioPAX (from Pathway Commons) don’t give evidence text, so these aren’t appropriate for curation via the INDRA Database.

Now that the curation menu has come up, you can select one of several error types. Use your best judgement if a statement is really correct. Note that INDRA does synonym disambiguation, so the label for the statement may not match to the highlighted text. You can click the entity names in the statement header to open pages with more information about the entities, including their synonyms.

If you want to leave a note that explains why you made the curation you did, that would be very helpful! Finally, smash that submit button.

You’ve now contributed a curation! Thank you very much. You and the rest of the scientific community will now disproportionately benefit from this small amount of effort because of the large-scale extraction efforts of the INDRA Database and combination with other curations.

Stay tuned for the upcoming INDRA 2 paper that outlines how curations can be used to assess the quality of each statement (at each level of hierarchical abstraction, too) as well as the INDRA Database paper describing this resource. We’re currently working on updating the API for bulk downloading curations from the INDRA Database and for use with the INDRA assemble corpus utility indra.tools.assemble_corpus.filter_by_curation().

Update Here’s the INDRA 2 preprint:doi:10.1101/2022.08.30.505688

What’s a CURIE, and Why You Should be Using Them

2021-09-14T08:47:00+00:00

Compact uniform resource identifiers, or CURIEs, are an important formalism for referencing biomedical entities. This post explains what they are, how to write them yourself, and a brief outline of how they fit in to the semantic web, linked open data, and open biomedical ontology worlds.

In the semantic web, linked open data, and ontology communities, uniform resource identifiers (URIs) are used to reference named entities. For a given nomenclature, like the Chemical Entities of Biological Interest (ChEBI), URIs usually have two parts:

A URI prefix (in red)
A unique local identifier from the given nomenclature (in orange)

All the URIs from the same nomenclature will have the same URI prefix, but a different unique local identifier. Here’s an example, using the ChEBI unique local identifier for alsterpaullone:

https://www.ebi.ac.uk/chebi/searchId.do?chebiId=138488

The Trouble with URIs

URIs are inconvenient because each named entity could be referenced by potentially many URIs. For example, a URI could start with either http or https. Even worse, there are several competing services that each try to mint the one true URI for each. XKCD sums the ensuing chaos up pretty well:

For the example molecule, alsterpaullone, here are some (but not all) of the possible URIs that could be used to reference it:

Provider	URI
First-party	https://www.ebi.ac.uk/chebi/searchId.do?chebiId=138488
Identifiers.org	https://identifiers.org/CHEBI:138488 https://identifiers.org/CHEBI/138488 http://identifiers.org/CHEBI:138488 http://identifiers.org/CHEBI/138488
OBO Library PURL	http://purl.obolibrary.org/obo/CHEBI_138488
Name-to-Thing	https://n2t.net/chebi:138488

The real issue with URIs is that the URI prefix (the beginning part) doesn’t really tell you anything. In fact, given a URI, you usually have to do some detective work to figure out which nomenclature authority it goes with.

One solution was to use resolvers that create “persistent URLs”, but in the end, there are many competing resolvers that don’t cover everything. For example, the OBO PURL system doesn’t cover HGNC and UniProt. The Identifiers.org system doesn’t cover many ontologies.

For practical purposes, it makes sense to keep track of the commonly used names of each resource, then just generate the kinds of URIs that people might want depending on what software or data systems they’re working with rather than prescribing one URI to be the canonical one.

Come to the Dark Side, We have CURIEs

The solution is to use compact uniform resource identifiers (CURIEs), which replace the URI prefix with a more approachable prefix. A CURIE has three parts:

A prefix (in red)
A delimiter (in black)
A unique local identifier from the given nomenclature (in orange)

Since everyone agrees on what ChEBI is, it makes sense to use chebi as the prefix for ChEBI unique local identifiers. Here’s the same example for alsterpaullone, condensed as a CURIE:

chebi:138488

The best part of a CURIE is that you can associate your favorite URI prefix with its corresponding prefix depending on your use case. You can even have a database that stores all of the possible ones for you. Replacing URIs with prefixes is so common, that it’s a core part of the SPARQL query language, which is used both in the semantic web and ontologies to traverse data stored in the resource description framework (RDF) schema. Here’s an example SPARQL query that has these prefixes prominently at the top:

prefix obo: 
prefix owl: 
prefix rdfs: 
prefix rdf: 

SELECT ?x ?p ?y
WHERE {
  {?x rdfs:subClassOf [
    a owl:Restriction ;
    owl:onProperty ?p ;
    owl:someValuesFrom ?y ]
  }
  UNION {
   ?x rdfs:subClassOf ?y .
   BIND(rdfs:subClassOf AS ?p)
  }
  ?x a owl:Class .
  ?y a owl:Class .
}

This example was borrowed from the SPARQL queries in the repository for the Core Ontology for Biology and Biomedicine (COB). You don’t have to understand the SPARQL itself, just check the first 4 lines that start with prefix ....

How to Build a CURIE

Most common vocabularies can be written as CURIEs. Here are a few examples:

Name	Prefix	Example Unique Local Identifier	Example CURIE
Gene Ontology	go	0032571	go:0032571
HGNC	hgnc	16793	hgnc:16793
UniProt	uniprot	P0DP23	uniprot:P0DP23
Disease Ontology	doid	0110974	doid:0110974
Medical Subject Headings	mesh	C063233	mesh:C063233

As you might guess, most prefixes are either the acronym for a nomenclature authority or the name itself. There are a few cases where this isn’t true, like for Disease Ontology. Their acronym is DO, but then that add ID which is usually shorthand for “identifier”, and therefore get doid as a prefix.

Just to reiterate, it’s really easy to make a CURIE. You take the prefix, a colon, then the unique local identifier and smash them together! Note: some communities, like the Open Biomedical Ontologies Foundry, like to stylize prefixes with uppercase or mixed-case. If you live in URI world, this is a big deal, but for most practical purposes, it’s nice to be able to just keep it all lowercase.

How do you know what’s the right prefix for each resource? And who even keeps track of this stuff? The Bioregistry keeps an up-to-date list that you can browse or search here. My team at Harvard Medical School has been building this with help from the community to serve the community needs that previous registries didn’t - most importantly, to make the data open and transparent and to enable community suggestions in an open and fair way.

How to Code with Me - Beyond Linters

2021-09-13T23:18:00+00:00

This post is about my personal code style guide that are beyond the enforcement of my flake8 plugins or black. I’ll try and update it over time.

Exit code blocks as early as possible

Sometimes you have an if statement that does some stuff when the conditional is true, or raises an error if it isn’t. Here’s the wrong way to write this:

if cond:
    ...  # big code block
else:
    raise ValueError

It’s better to raise the value error as quickly as possible, because then you can dedent the big code block. This is particularly good if you have lots of nested code blocks in conditionals, loops, etc.

if not cond:
    raise ValueError

...  # big code block

The same thing is true for return statements. The following code where a value is set then returned is bad:

if cond:
    ... # big code block
    value = ... # final line that assigns value
else:
    value = ...  # one-liner assigning value
return value

It’s better to flip the conditional and return the value quickly, since it only takes one line to assign it in the else block

if not cond:
    return ...  # one-liner assigning value

... # big code block
return ... # final line that assigns value

Pre-loading a PostgreSQL Docker Container

2021-08-28T14:44:00+00:00

PostgreSQL is a powerful relational database management system that can be easily downloaded and installed from its official image on DockerHub using Docker. However, it’s not so straightforward to preload your own data. This blog post is about preparing a derivative of the base PostgreSQL Docker image that’s preloaded with your own database and pushing it back to DockerHub for redistribution.

Prerequisites I’m going to assume you have a modern version of docker running. I’m using Docker Desktop for Mac. I’ve also installed PostgreSQL using brew install postgresql, which puts a suite of command line utilities, including createdb, which I use in the middle of this tutorial. I’m usually using fish, but the following instructions are given with Bourne-again shell (bash) syntax.

Notes Throughout this post, I’ll shorten PostgreSQL to postgres. All the commands in this tutorial are run from the shell of the host system, i.e. I did not ssh or exec into the Docker image itself to run them.

Run the base image

The first step is to run a base image. This could be as simple as docker run postgres, but there are a few options to add to make the rest of this process more simple. I’ve written this post while preparing the docker image for the Biolookup Service, so biolookup appears many times as names/passwords throughout. I’ll definitely be writing another post soon about what that service is and what it does (think Ontology Lookup Service, but not just restricted to ontologies).

$ docker pull postgres
$ docker run \
    -p 5434:5432 \
    --name postgres-biolookup \
    --detach \
    -e POSTGRES_PASSWORD=biolookup \
    -e PGDATA=/var/lib/postgresql/pgdata \
    --shm-size 1gb \
    postgres

-p/--publish This takes an argument looking like :. The corresponds to the port inside the docker container, and the corresponds to what’s visible outside. I’m mapping from the default postgres port inside the container (i.e., 5432) to a non-default one outside (i.e., 5434) to avoid conflict with my local installation of postgres.
--name This gives a nice name to the container for lookup later. This doesn’t have to be the same as the name you give when you push to dockerhub, but it’s probably better to stay consistent. If you don’t give one, docker assigns a silly name for you.
-d/--detach Rather than running in my current shell, this backgrounds it. Since I used --name, I can look up my image directly using $(docker ps --filter "name=postgres-biolookup" -q).
-e/--env This allows you to specify environment variables.
1. Setting POSTGRES_PASSWORD explicitly sets the password for the default postgres user (named postgres).
2. Setting PGDATA ensures that a docker commit will actually persist the database’s content. The cryptic path that came after is just the standard path postgres uses. Move along.

--shm-size By default, the shared memory is 64mb. When loading up this big database, this caused the following crash:

  ...
  File "/Users/cthoyt/dev/pyobo/src/pyobo/database/sql/cli.py", line 52, in load
    _load(
  File "/Users/cthoyt/dev/pyobo/src/pyobo/database/sql/loader.py", line 65, in load
    _load_definition(engine=engine, table=defs_table, path=defs_path, test=test)
  File "/Users/cthoyt/dev/pyobo/src/pyobo/database/sql/loader.py", line 97, in _load_definition
    _load_table(
  File "/Users/cthoyt/dev/pyobo/src/pyobo/database/sql/loader.py", line 311, in _load_table
    cursor.execute(sql)
psycopg2.errors.DiskFull: could not resize shared memory segment "/PostgreSQL.1699521131" to 67128576 bytes: No space left on device

Luckily, StackOverflow had me covered and suggested increasing the shared memory to 1gb using --shm-size.

Create the database

Creating the database on the already running postgres docker image is a bit more straightforwards:

$ PGPASSWORD=biolookup createdb -h localhost -p 5434 -U postgres biolookup

PGPASSWORD=biolookup sets the password in the environment when this command gets run so there’s no need to manually interact with it. -h is for host, -p is for password, and -U is for username. -e can be added optionally to show the commands that are run for debugging. The final part biolookup is the name of the database that gets created.

Load the database

The motivation for this post was to prepare a rather large database for the Biolookup Service, which contains hundreds of millions of identifiers, names, definitions, and alternative identifiers for biomedical entities. I’ve written previously about building this database, which after an incredible effort boils down to running pyobo database build from the shell. It also automatically uploads its contents to Zenodo:

Data	DOI
Names
Definitions
Alternative Identifiers
Species

Loading the database requires the biolookup Python package. It automatically downloads the data from the latest releases on Zenodo if not available locally, then puts it in the database.

$ python -m pip install biolookup
$ biolookup load --uri postgresql+psycopg2://postgres:biolookup@localhost:5434/biolookup --test

The --test makes the database only load 100K records instead of hundreds of millions of records. For building a real database, remove this.

Commit and push to DockerHub

The docker commit checks what the difference between the base image and the current state of the image is. Because of the -e PGDATA=..., it also tracks the new data added. For the Biolookup Service, the image has gone from about 300mb to almost 40gb, so be patient. I went and made breakfast while this was happening and it was done by the time I came back. For reference, it was a saturday morning American breakfast.

$ docker commit \
    -a "Charles Tapley Hoyt " \
    -m "Added biolookup schema and data" \
    $(docker ps --filter "name=postgres-biolookup" -q) \
    biopragmatics/postgres-biolookup:latest

After committing, it’s time to push to DockerHub. You might need to do docker login before this. The name of the image takes the form /[:]. Make sure you push to an organization that you have rights to, and the tag (the part after the colon) is optional.

$ docker push biopragmatics/postgres-biolookup:latest

You can check to see it was uploaded properly here.

Run locally

Since the biolookup web application is automatically installed with PyOBO and the database is now built locally, you can test it locally with:

$ biolookup web --sql --uri postgresql+psycopg2://postgres:biolookup@localhost:5434/biolookup

Run with Docker Compose

You can use the following configuration as a docker-compose.yml file to orchestrate the pre-loaded database with the front-end web application (more information on that here):

version: "3"

services:
  app:
    image: biopragmatics/biolookup:latest
    environment:
      BIOLOOKUP_SQLALCHEMY_URI: postgresql+psycopg2://postgres:biolookup@database/biolookup
    restart: always
    ports:
      - "8765:8765"
    depends_on:
      - database
  database:
    image: biopragmatics/postgres-biolookup:latest
    ports:
      - "5432:5432"

You can run this with docker-compose up --detach. When you do this, you need to be patient (1-5 minutes) for the database to start up before making requests from the web application. If you didn’t detach when running docker-compose, postgres will actually log when it’s ready. If you try making a request before it’s done starting, you’ll probably get an error message that looks like this:

Traceback (most recent call last):
  ...
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 605, in __connect
    connection = pool._invoke_creator(self)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 578, in connect
    return dialect.connect(*cargs, **cparams)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 584, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/usr/local/lib/python3.9/site-packages/psycopg2/__init__.py", line 122, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL:  the database system is starting up

If you want to more correct about this, you can use the pg_isready to check the connection status of a PostgreSQL server. But I think it’s practical enough just to wait a few minutes

Next Steps

My next steps are to figure out the best way to automate the first three steps (running the base image, creating the database, and loading the database) then hopefully do it in an automated setting in GitHub Actions.

Special thanks to Ben Gyori for outlining how to do this and getting me going in the right direction. Thanks to Scott Colby for feedback and pro docker tips.

Machine Learning Needs More Generators

2021-08-18T11:21:00+00:00

I’ve spent the last two days cleaning up some research machine learning code that blew up when I tried applying it to my own data due to memory constraints. This post is about the antipattern that caused this, how I fixed it, and how you can avoid it too.

The INDRA Lab has been collaborating with my old PhD cellmate Daniel Domingo-Fernández and his master’s student Helena Balabin on cross-modal transformers. We’ve jointly pre-trained a transformer (STonKGs) on knowledge graph embeddings on the INDRA database and the associated evidence text for each triple.

We then fine-tuned models for various downstream tasks, including one for rating the correctness of a given statement (relevant since most are from large-scale text mining). We’re very interested in comparing this to INDRA’s belief system that will be described in an upcoming publication from our group.

I wanted to apply this fine-tuned model to the self-updating models in EMMAA, but ran into some memory errors since the code was building up big lists, converting the lists into DataFrames using pandas, then exporting to disk as a TSV. The solution is to use generator functions (that create iterable for use with for loops) and write the results to a file directly. The basic anti-pattern looks like this:

import pandas as pd


def f(df: pd.DataFrame) -> pd.DataFrame:
    new_rows = []
    for _, row in df.iterrows():
        new_row = {
            ...  # somehow build up a new row, as a dictionary
        }
        new_rows.append(new_row)
    return pd.DataFrame(new_rows)

There are two problems with this:

The input must be completely built up before calling f()
The output must be completely built up before returning

This compounds for every function f(), g(), h(), and so on that takes in the result from the last dataframe transformation, since they all have to keep everything in memory. In STonKGs, the first transformation is to look up the embeddings for a given source/target/evidence triple from both the knowledge graph embedding and the pre-trained BERT language model. The second transformation is to pre-process the embeddings. The third is to actually apply the STonKGs model to jointly embed them. The fourth is to apply the fine-tuned model. Each of these happens at varying speeds, but are unfortunately decoupled.

Python has really powerful tool for using ideas from functional programming in an approachable way that allows us to solve this issue. Rather than building up a huge list, we can simply yield each piece so another function can consume them with a for loop. When I refactor code that looks like this, I make a second helper function that does the hard work, and try and maintain the original function’s interface by calling the helper function:

import pandas as pd


def f(df: pd.DataFrame) -> pd.DataFrame:
    return pd.DataFrame(f_helper(df))


def f_helper(df: pd.DataFrame) -> Iterable[Mapping[Any, Any]]:
    # new_rows = []  # we don't need this anymore!
    for _, row in df.iterrows():
        new_row = {
            ...  # somehow build up a new row, as a dictionary
        }
        yield new_row

If you’re not familiar with yield, here are a two videos to get you thinking aboout how to use loops like a Pythonista:

Now that we have f_helper, we solved issue #2. The solution to issue #1 is to have functions that take in only the parts that are needed to build up each new row. This means you should accept an iterable, and have the high level function slice up the dataframe or do whatever pre-procesing is necessary first:

import pandas as pd


def f(df: pd.DataFrame) -> pd.DataFrame:
    it = (row for _, row in df.iterrows())
    return pd.DataFrame(f_helper(it))


def f_helper(rows: Iterable[Mapping[Any, Any]]) -> Iterable[Mapping[Any, Any]]:
    for row in rows:
        new_row = {
            ...  # somehow build up a new row, as a dictionary
        }
        yield new_row

With this refactoring, the code still does what it used to, but now you can think about how you might string together f_helper(), g_helper(), h_helper(), and so on directly, since they don’t need to get completely materialized as a list. If you do the composition of several functions that take in iterables and yield stuff (i.e., they return an iterable of the stuff that gets yielded), then you don’t have to worry about running out of memory since it only needs to have the results of one set of transformations from the compositions of all functions at a time. In my case, I just printed the results to a file and then it no longer needed to be in memory.

This was a quick blog post I wrote while the code I updated was running. The full PR I did can be found here for reference. Feel free to chime in on that PR if you have some questions about how I did this or get in touch with any of the contact info on the bottom of my blog.

Organizing the Public Data about a Researcher

2021-08-17T11:21:00+00:00

In a previous post, I described how to formalize the information about a research organization using Wikidata. This post follows the same theme, but about this time about a given researcher. Not only can you follow this post to make your own scientific profile easier to find and navigate, but you can also use Wikidata to improve the profiles of your co-workers and collaborators.

There are several other places besides Wikidata for putting your scholarly profile. The first (and worst) is by making a resume in an awful Word document or PDF. Much like what ends up in english prose in scientific texts, this is where knowledge goes to die. The next less bad is to put it in a closed repository like LinkedIn, ResearchGate, Scopus, Loop, or one of many others. These might be nice, but don’t forget they’re companies and do not want people to be able to use their troves without paying up. The runner-up is ORCID, which is absolutely amazing for providing persistent identifiers to individuals, linking published work in high-profile repositories like PubMed, and in some cases linking grants. While it will be the cornerstone of this tutorial, it ultimately falls short as a tool for integrating the wide variety of information available about any given researcher. So who’s the winner? Wikidata, of course! It has an incredible flexibility for creating new relationships and a first-class notion of scholarly works, awards, affiliations, employers, and the relationships between them that make it a perfect place for storing information.

Anyone can edit Wikidata, meaning you can create an entry for your curmudgeonly PI who wouldn’t be caught dead making a profile on this new thing called “the internet”. There are three notability criteria for what’s allowed to be added as an item in Wikidata:

References another Wikimedia commmons page
References a clearly defined entity/concept that has meaningful external provenance
Fulfills a structural need to make statements in other items more useful

Adding researchers usually fits one or both of criteria 2 or 3. Further, there are several people in the bibliometrics community who have created bots for importing and aligning information from external sources that make the entries that you supply so much more rich overnight (e.g., some bots automatically translate the descriptions of pages into new languages, some import publication metadata, some align identifiers with external resources).

Let’s get to it.

Step 1: ORCID

The ORCiD identifier is the single unambiguous identifier for each researcher.

As unique as you are, the truth is your name probably isn’t all that unique. ORCID was founded specifically to help solve the problem of name ambiguity in research. Want to learn more? Start here! #openscience #openresearch #namedisambiguity https://t.co/Q2J1bVrolc
— ORCID Organization (@ORCID_Org) August 17, 2021

If you don’t have one, it only takes 2 minutes to make one here. ORCID profiles look like a bunch of sets of 4 numbers/letters with dashes between them. Mine is 0000-0003-4423-4370. You can use a resolver like the Bioregistry to look up web pages on ORCID IDs with links that look like https://bioregistry.io/orcid:0000-0003-4423-4370 or more specifically use ORCID’s web address directly like https://orcid.org/0000-0003-4423-4370.

It’s super intuitive how to put information in here. There are a few minimal things to include to make it possible for people to find you:

Include any variations on your name that might appear in publications. In my professional life, I write out my name in full, including my middle name, but there are all sorts of variations that pop up including:
- Charles Hoyt
- C.T. Hoyt
- C. Hoyt
- Hoyt, C.T.
- Hoyt, C.
- Hoyt, Charles Tapley
Include your education. Since most people reference their highest academic degree in prose, this again helps people find you and disambiguate you from other people. Even ORCID knows about this
Make sure you set each field to “public” so everyone can see it! These are marked as private by default, so make sure you click the following box (thanks to Ann Reynolds for pointing this out)

I happen to enjoy spending time to maintain my ORCID profile, and here’s what it looks like:

Step 2: Wikidata

I’ll reiterate the short summary I gave on Wikidata from my previous post:

Wikidata is an open, community-curated platform of knowledge. It stores entities, their relations to other entities, their relations to scalar values, and added context for each relationship. Typically, relationships have a subject, relation, and object and can be read like a simple sentence in the english language.

There are lots of working groups that maintain its ontology (i.e., the rules for how curation should be done) around certain domains, such representing organization structures. This means there are lots of tools already built in to Wikidata for potential curators like you and me to create rich pages for their organizations.

One of the curation rules shared across all domains in Wikidata is that each entity should have a “type”. This means on the page for Albert Einstein, there is a relationship stating he is an instance of a human. The “instance of” item on Wikidata is a special kind called a “property” and is one of the places where the ontology lives - there are specific rules for each property on how it should be used in a relationship, like what’s allowed to be the subject and what’s allowed to be the object. For”instance of”, there are no rules about the subject. However, the object of the relationship where “instance of” is the property should be a “class” of thing. It wouldn’t make sense for the type of another entity to be an instance of “Albert Einstein”.

One thing to note: use the Wikidata search bar to see if there’s already a page for the person you’re looking for! It’s a big problem if there are duplicate entries on Wikidata. Scholia also has some tools for looking people up by ORCID or other identifiers that might be helpful.

Ontology for Researchers (and other Humans)

Unlike an organizations, all researchers will have the type of human. Typically, humans have certain pieces of information associated with them using the following properties:

sex or gender - for humans, this can include: male, female, non-binary, intersex, transgender female, transgender male, and agender.
country of citizenship
native language - there are Wikidata entries for most (if not all) common languages
languages spoken, written or signed - additional non-native languages. It doesn’t have any information about levels, so I’m a bit hesitant about advertizing my B1 level of German!
nickname - a slightly more specific place to put a nickname than the synonyms list at the top of the page. For example, I go by Charles Tapley Hoyt professionally, but use Charlie in my everyday life
official website - one or more personal websites for the researcher
official blog - one or more blog written for the researcher. It happens that my blog and personal site are on the same page.

I’d highly suggest doing all of these before moving on to the next few sections, which have quite a few bells and whistles.

Affiliations:

educated at (P69) - links a person to the institutions at which they earned their bachelor’s, master’s, or doctoral degrees, as well as any other education.
employer (P108) - the top-level organization that you’re associated with. For me at the moment, that’s Harvard Medical School. For specific departments or teams within the organization, you can use the “affiliation” property.
affiliation (P1416) - useful for subdivisions inside organizations. For example, while I am employed by Harvard Medical School, I am appointed as a postdoc in the Laboratory of Systems Pharmacology, which itself has information about its membership within the Harvard Program in Therapeutic Science.
member of (P463) - any professional societies, working groups, or other organizations. For example, I belong to the American Chemical Society.
doctoral advisor (P184) - useful for building up academic trees. I asked my doctoral advisor about his doctoral advisor and actually ended up emailing him to find out some more information about my academic lineage. It turns out that there’s a lot of information in the Academic Tree website that gets propagated through to Wikidata already by bots. After I added a few missing links, I found out that I am in the lineage of Emil Erlenmeyer after whom the eponymous Erlenmeyer flask is named. Depending on if you left your doctoral work happy or not, you might not want to revisit this. Totally understandable.

Accomplishments:

academic degree (P512) - the highest academic degree obtained (e.g., Doctor of Philosophy). Using the “qualifiers”, it’s also possible to add information about the doctoral advisor, when the degree was received, the opponents during the disputation (i.e., the examination committee)
academic thesis (P1026) - make an entry for your master’s or PhD thesis, and link them to you this way!
award received (P166) - make an entry for the awards you’ve received, then connect them here.

Some of the affiliations and accomplishments will require going even deeper into your curation and making additional Wikidata entries. For example, many awards probably don’t already have a Wikidata entry. Same is likely true about your academic thesis.

General External Account Links:

Academic External Account Links:

The ORCID identifier is definitely the most important, but Google and DBLP often pre-index researchers even if they have not got an ORCID, so this might be the best you can do for some researchers.

There are tons of other IDs you can link as well as other properties. When I was working on my Wikidata entry, I used Egon Willighagen’s as an example. I’d suggest browsing through ours to go to the next level.

Tutorial

I’ll refer back to my previous post on curating information about an organization, since it works basically the same way.

One of the important things I mentioned before is to add lots of synonyms for your name, especially in ways they might appear in journal articles.

Scholia

If you do a really good job curating information about yourself, your co-workers, and your affiliations, Scholia becomes a very powerful tool for showing information about you. For example, here’s my Wikidata page rendered by Scholia: https://scholia.toolforge.org/author/Q47475003.

One part that’s really cool is the co-author graph:

When you start off, there won’t be much here. Luckily, it has a curation link that takes advantage of the synonyms on the Wikidata and helps connect papers that are already on Wikidata to the given researcher’s page. Here’s the link for mine (though I keep it pretty up-to-date, and it’s likely empty): https://scholia.toolforge.org/author/Q47475003/curation

There’s quite a bit more with respect to getting papers and associations between researchers and events like conferences, but this is already a great place to start. See the next post in the series on curating your papers in Wikidata.

Reproducibly Loading the ChEMBL Relational Database

2021-08-05T14:36:00+00:00

In his blog post, Some Thoughts on Comparing Classification Models, Pat Walters illustrated enlightened ways to convey the results of training and evaluating machine learning models on hERG activity data from ChEMBL (spoiler: it includes box plots). It started by querying the ChEMBL relational database, but featured a common issue that hampers reproducibility: hard-coded configuration to a local database based on a specific database (MySQL). This blog post is about how to address this using chembl_downloader and make code using ChEMBL’s SQL dump more reusable and reproducible.

While the original blog post pointed to code PatWalters/comparing_classifiers, there’s an updated version at PatWalters/jcamd_model_comparison that includes the code that queries ChEMBL (among other things). The original notebook began like this in cells 2 and 4 (edited for clarity):

import mysql.connector as sql
import pandas as pd

sql = """
select
    canonical_smiles,
    cs.molregno,
    md.chembl_id as mol_chembl_id,
    standard_relation, standard_value, standard_type, standard_units, description,
    td.organism, assay_type, confidence_score,
    td.pref_name, td.chembl_id as tgt_chembl_id
from activities act
    join assays ass on act.assay_id = ass.assay_id
    join target_dictionary td on td.tid = ass.tid
    join compound_structures cs on cs.molregno = act.molregno
    join molecule_dictionary md on md.molregno = cs.molregno
where ass.tid = 165
    and assay_type in ('B','F')
    and standard_value is not null
    and standard_units = 'nM'
    and act.standard_relation is not null
    and standard_type = 'IC50'
    and standard_relation = '='
"""

with sql.connect(
    host='localhost',
    database='chembl_26',
    user='pwalters',
    password='itsasecret',
) as con:
    df = pd.read_sql(sql, con=con)

There are two main issues with this code:

It uses a specific database (MySQL) and user-specific connection configuration which means:
- Nobody else can run this code without editing it.
- There’s no information about how to get, preprocess, or load the database before running the script.
It relies on a specific version of ChEMBL, which means that we can’t benefit from new compounds and assays in new releases without editing it.

To be fair, this is from a Jupyter notebook that’s not necessarily supposed to be reused. But what if were so easy to fix this anti-pattern that there’s no excuse not to? Here’s how using the chembl_downloader Python package:

import chembl_downloader
import pandas as pd

sql = ...  # omitted for brevity

version = "26"
with chembl_downloader.connect(version=version) as con:
    df = pd.read_sql(sql, con=con)

With only a single (logical) line changed, this code now knows how to download the ChEMBL 26 SQLite dump from the source, store it in a deterministic location, extract it, and load with SQLite.

This means that anyone can run it without knowing how to download ChEMBL themselves, which version to get, how to name the file, or where to put it on their machine. It also relies on SQLite, which is effectively available on all devices that run Python and has exactly the same programmatic API, but without the need to run or connect to extra software. While a RDBMS like MySQL might be more powerful for some kinds of queries, the difference is negligible when querying single assays. It also implicitly solves the problem that the user doesn’t know if there was any pre-processing done to the file.

Under the hood, it’s using the pystow package to deterministically pick a folder (~/.data/chembl/26/) into which the file ftp://ftp.ebi.ac.uk/pub/databases/chembl/ChEMBLdb/releases/chembl_26/chembl_26_sqlite.tar.gz is download (~/.data/chembl/26/chembl_26_sqlite.tar.gz).

Since the pattern of connecting to the database then running a SQL query with pandas is so common, the chembl_downloader has a query() function that wraps the two lines from the previous example:

import chembl_downloader

sql = ...  # omitted for brevity

version = "26"
df = chembl_downloader.query(sql, version=version)

Getting the Newest Version

What about making this code automatically updating to the newest version of ChEMBL? Just use the chembl_downloader.latest() to the latest version up for you. Under the hood, it’s using the bioversions package to do this.

import chembl_downloader

sql = ...  # omitted for brevity

version = chembl_downloader.latest()
df = chembl_downloader.query(sql, version=version)

Doing this is so common, that you can actually just omit the version argument and it will look up the latest for you.

Meaningful Improvement

I made a pull request to update the notebook based on these suggestions. The new notebook is here and features the most recent version of ChEMBL at the time of writing (ChEMBL 29) instead of ChEMBL 26. The table below shows how much improvement updating ChEMBL gives:

Flag	ChEMBL 26	ChEMBL 29	Increase	Percent Increase
Active	4191	4601	410	9%
Inactive	2048	2274	226	11%

I’d say getting 9% more actives and 11% more inactives basically for free by writing better code is a pretty big success. In this notebook, the AUC-ROC of the prominently presented LGBM classifier improved by about 1%. This could have just as easily have gone down, but I think it was worth checking.

One time, I received negative feedback from authors I asked why they hadn’t updated the analysis they presented in their manuscript using the newest version of ChEMBL (this was a few months ago, so the jump was from ChEMBL 25 to ChEMBL 28). One excuse they gave was that new data would (probably) not change their results. Depending on what kind of stuff you do, 1% might be a big deal. Or not.

When I got that feedback, I checked in on the code that had been released along with the manuscript to see if I could do it myself. It wasn’t pretty. I’d guess the authors really just didn’t want to ever touch their code again because it was very complicated, relied on tons of finnicky dependencies, and was overall written poorly. I don’t think shaming scientists for writing bad code is a very constructive nor a good way to motivate them to write better code. I’ve found on many occassions that authors usually just don’t have the right training or mindset to do reproducible/reusable science. A better solution is to offer pull requests to their code that demonstrates how to fix the issues and explain in detail how it works. Then, the best you can do is hope that they learn something and use it in their next publication.

So for the case of these authors, I looked into their downstream dependencies and began getting in touch with their code owners (maintainer would be a strong word) then sending pull requests to make them more reusable. I also ended up writing a bit of my own code to see if I could ultimately re-write the analysis to be a little more automatic. This chembl_downloader package is one of the tools I built along the way! I might come back and write a blog post about the original paper that caused the negative feedback too, because it was indeed a very cool paper! But first, I want to show that it can be reproduced.

Reproducibly Loading the ChEMBL SDF

2021-08-04T12:46:00+00:00

ChEMBL is easily the most useful database in a cheminformatician’s toolbox, containing structural and activity information for millions of diverse compounds. In his recent blog post, Generalized Substructure Search, Greg Landrum highlighted some new RDKit features that enable more advanced substructure queries. It started by loading molecules from the ChEMBL 29 SDF dump, but it featured a common issue that hampers reproducibility: a hard-coded local file path to the ChEMBL data. This blog post is how to address this using chembl_downloader and make code using ChEMBL’s SDF dump more reusable and reproducible.

Getting Data Reproducibly

The code in the blog post began by loading up ChEMBL 29 like this (edited for clarity and imports omitted for brevity):

in_path = "/home/glandrum/Downloads/chembl_29.sdf.gz"

with gzip.open(in_path) as file:
    data = []
    for i, mol in enumerate(rdkit.Chem.ForwardSDMolSupplier(file)):
        ...
        data.append(...)

out_path = "../data/chembl29_sssdata.pkl"
with open(out_path, 'wb') as file:
    pickle.dump(data, file)

There are three main issues with this code:

It relies on a local file path to the ChEMBL data, which means:
- Nobody else can run this code without editing it.
- There’s no information on how to get or preprocess this file before running the script.
It relies on a specific version of ChEMBL, which means that we can’t benefit from new compounds in new releases without editing it.
It outputs data to a relative file path, which might not work based on the way the script is run or the directory structure on your drive

To be fair, this is a blog post that’s not necessarily supposed to be reused. But what if were so easy to fix this anti-pattern that there’s no excuse not to? Here’s how using the chembl_downloader Python package:

import chembl_downloader

version = "29"                                             # <-- This line changed for this example
in_path = chembl_downloader.download_sdf(version=version)  # <-- This line changed for this example

with gzip.open(in_path) as file:
    data = []
    for i, mol in enumerate(rdkit.Chem.ForwardSDMolSupplier(file)):
        ...
        data.append(...)

out_path = "../data/chembl29_sssdata.pkl"
with open(out_path, 'wb') as file:
    pickle.dump(data, file)

With only a single line changed, this code now knows how to download the ChEMBL 29 from the source and store it in a deterministic location on your hard drive. This means that anyone can run it without knowing how to download ChEMBL themselves, which version to get, how to name the file, or where to put it on their machine. It also implicitly solves the problem that the user doesn’t know if there was any pre-processing done to the file at "/home/glandrum/Downloads/chembl_29.sdf.gz". Under the hood, it’s using the pystow package to determinisically pick a folder (~/.data/chembl/29/) into which the file ftp://ftp.ebi.ac.uk/pub/databases/chembl/ChEMBLdb/releases/chembl_29/chembl_29.sdf.gz is download (~/.data/chembl/29/chembl_29.sdf.gz).

Getting the Newest Version

import chembl_downloader

version = chembl_downloader.latest()                       # <-- This line changed for this example
in_path = chembl_downloader.download_sdf(version=version)

with gzip.open(in_path) as file:
    data = []
    for i, mol in enumerate(rdkit.Chem.ForwardSDMolSupplier(file)):
        ...
        data.append(...)

out_path = "../data/chembl29_sssdata.pkl"
with open(out_path, 'wb') as file:
    pickle.dump(data, file)

Note, if you omit the version argument completely, it automatically looks up the version as well. However, there’s one more thing to update before we’ve addressed our third point: where the file is output. There are two goals in fixing the output:

Make the path deterministic
Make the path based on the version of ChEMBL that’s being used, so if a newer version gets used, it doesn’t delete the old file

The solution comes by using pystow to pick a deterministic path, which the download_sdf() function is actually using under the hood, too:

import chembl_downloader

version = chembl_downloader.latest()
in_path = chembl_downloader.download_sdf(version=version)

with gzip.open(in_path) as file:
    data = []
    for i, mol in enumerate(rdkit.Chem.ForwardSDMolSupplier(file)):
        ...
        data.append(...)

import pystow                                                  # <-- This line changed for this example
out_path = pystow.join("chembl", version, name="sssdata.pkl")  # <-- This line changed for this example
with open(out_path, 'wb') as file:
    pickle.dump(data, file)

The pystow.join method creates a path to ~/.data/chembl//sssdata.pkl. Now this code is ready to stand the test of time and a variety of different uses!

Coda

Because the pattern of getting the SDF from ChEMBL then opening it with a ForwardSDMolSupplier is so common, it’s actually included in its own function supplier(). The code could be compressed one more time like:

import chembl_downloader

version = chembl_downloader.latest()

with chembl_downloader.supplier(version=version) as suppl:
    data = []
    for i, mol in enumerate(suppl):
        ...
        data.append(...)

import pystow

out_path = pystow.join("chembl", version, name="sssdata.pkl")
with open(out_path, 'wb') as file:
    pickle.dump(data, file)

Tales from the Bonner Ausländeramt

2021-07-26T12:46:00+00:00

This is a more personal blog post about my experience as an american expat in Germany - specifically about my experiences at the Bonner Ausländeramt (Bonn’s Foreigner’s Office of the City of Bonn).

I moved to Germany in August 2015 and immediately began my first incarnation of the Ausländer Shuffle (which I affectionately name after the NU Shuffle). Luckily, as a USA citizen, I entered the country with a 90-day tourist visa which required no prior paperwork or pre-registration; just show up. In that 90 days, I had to get all of my paperwork in order, which included a local cell phone number, proof of local housing, proof of German insurance, proof of minimum bank account funds, and ultimately proof of enrollment at the university. This was about as non-linear as a Kurt Vonnegut novel. I had the benefit that my housing had been pre-registered by my study program. For everyone else, the first thing they’d have to do is find housing, and even worse, do it without the benefit of all the other paperwork that they couldn’t get without already having housing.

After that was all done, I was able to sign up for my first appointment with the Bonner Ausländeramt (Terminvereinbarung im Ausländeramt). Side bar: an adjectival is when you turn a noun into an adjective. In the German language, most places can be converted into an adjectival by adding the -er suffix. That’s why it’s called Wiener Schnitzel - it’s Schnitzel from Wien (Vienna). Sign-ups for appointments are online (https://www.bonn.de/antrag_auslaenderamt), then they send you information about your appointment by snail mail. Unfortunately, this office has not yet digitalized. You’re assigned a case worker based on your last name, and you typically stay with them the entire time you live in the city. They will keep a paper file in their office on you, next to all the other paper files for all the other foreigners in Bonn. People from the USA are typically given a two year visa when studying. In theory, this is more than enough time for most study programs. Studying in Germany is cheap (just a 200-300 Euro administration per semester) so a lot of people decide to take their time. I didn’t (oops).

My second encounter with the Ausländeramt was two years later after I had finished my master’s degree in November 2017, before starting my Ph.D.. The Ausländeramt snail mailed me unprompted a few months ahead of time with a date and checklist of all the things I would need to renew my visa. They either have a very boilerplate document for renewals ( most likely) or just assumed that my studies hadn’t yet concluded, and therefore most of the document wasn’t applicable. I had my Ph.D. work contract with Fraunhofer, but not yet proof of my Ph.D. student status (I learned later that this is its own nightmare) but they effectively let it slide and issued a new student/work visa. Side bar again: it’s really great for companies to have students working for them. You have so few rights when you’re also a student, and companies don’t have to follow many of the worker laws. This is partially why Fraunhofer’s business model to recruit as many students as possible to do most of the work is so great for Fraunhofer.

My third encounter with the Ausländeramt was two years later after I had finished my Ph.D. in December 2019, but they did not issue me an appointment unprompted (I’m not sure why). I ended up finding the sign up form again, that I had forgotten about from four years prior and signing up for an appointment a few months in advance. After finishing graduate studies in Germany, you can get an 18-month “work search visa” which is mostly the same as a work visa but is issued even if you don’t already have a job. I was exited by this because I wanted to take some time off. The only issue was they issued my appointment after when my visa expired. This was particularly bad becuase it was in December, and I wanted to travel for the holidays. If I did this and my visa expired while I was out of the country, I would have a big problem - you can’t re-enter for 6 months after your visa expires. I tried emailing (unsuccessfully) and phoning the office every day for two whole weeks. They don’t answer the phone. It turns out that sometimes it forwards to the City Hall, and they are very annoyed to get questions about Ausländer affairs. With the end of December approaching and still no plan how to get home for Christmas, I ended up going to the office at 6:45AM (it opens at 8AM) one day try and get a walk-in appointment. I was the 5th person in line, and by the time office opened at 8AM, the line was going down the street. I knew I would be quick, but there were many people who were unprepared, had extenuating circumstances, or were just a more difficult case to deal with, so only a few of them would be seen between 8AM and noon when the office closed (and it’s only open 4 days a week). Being 5th, I got to see the walk-in administrator around 9:30AM, and they easily issued me a Fiktionsbescheinigung after only a few minutes of describing my situation. That document states that my case has basically been approved, but the paperwork is still pending. Effectively, I could travel with my expired visa + this document freely, and I then proceeded to book a very expensive last minute flight back home. Unfortunately, my appointment was just after New Year’s, so I didn’t get much vacation at home.

All the people I’ve talked to at the Ausländeramt were good people. They’re just really understaffed, overworked, and without the benefit of competent programmers and a progressive digitalization agenda to make their lives and the lives of their clients better. Then the Coronavirus hit. Everything was shut down for months. They issued emergency visa extensions since they couldn’t work in office, and none of the work could be done from home. The facade collapsed and it left many people confused, unsure, and anxious about their statuses.

My fourth encounter with the Ausländeramt should have been sometime around today, since my visa is no longer valid tomorrow. Unfortunately, I signed up two months ago for an appointment and never received any post. I emailed the office to ask for a confirmation, and got nothing back. The office doesn’t answer their phone line (this is the same now as it was before the pandemic). The difference between this time and last time is when I went to get a walk-in appointment, I was refused at the door by security because of pandemic precautions - only appointment holders can enter. This is a bit of a Catch-22: Can’t go in the office without an appointment. Can’t get an appointment without going in the office.

It turns out they’re still rotating which staff are in on any given day and many have home office (where they can’t really work, since all of it is done on paper that lives in the office itself). I have been very anxious about this day coming and officially tomorrow I will be in Germany illegally.

Last time I ended up on the phone line with the City Hall instead of the Ausländeramt, they weren’t very sympathetic to my cause, and also realized that they could effectively speak in their fastest and most ornamented German such that I couldn’t understand them anymore. Luckily, my German friend offered to help me this time with the shuffle and we got on the phone with the City Hall. They were surprisingly helpful this time, but what happened next was a total farce:

We talked to the nice lady at the city hall. She figured out what the office number for my case worker was (this is very secret information) and shared it with us so we could call directly.
We called directly. It didn’t answer (just went on ringing). After trying this at 15 minute intervals, we decided to call back the city and ask if we could have another case worker’s number. She gave us the name and number of another case worker in the office. We called them. They said that they would share the number of the person who shared the office with my case worker (these guys are cramped in there with all the files.)
We called the office mate of my case worker. No answer (but this time, it cut out after a few rings). We supposed that she was in office, but obviously busy. We tried a few times, no luck.
We tried the first case worker again that the nice lady at the city hall had given us. She was busy too, so we had to wait a while before she picked up. We asked her about the other lady, and she was nice enough to walk in the other room and ask the second case worker when she would be available for us to call.
We called back when she was available. She looked up my file, saw my visa expires tomorrow, was sympathetic to my cause, and said that she would snail mail a Fiktionsbescheinigung to me. Luckily, I live in walking distance from the Ausländeramt and this should arrive tomorrow.

Let’s hope this means I won’t be here illegally come tomorrow! By the end of this experience, I think we had spent way too much time of several people at the Bonn City Hall and also the Ausländeramt. My friend had spent a few hours with me, too, so what was the real cost of all of this? I actually still haven’t solved my problem since I still don’t have an appointment to actually extend my visa, either. My friend hadn’t experienced this kind of problem when he lived in Belgium and France because EU citizens all share the right of free movement between countries, and was just as jarring for him. He emailed our mayor, Katja Dörner, a nice letter about the experience too. She’s part of the “Green Party” which is pretty progressive, so maybe this will be a wake up call.

I’ve been lucky enough to be an easy case for the Ausländeramt - but it can’t be overstated the advantage I have enjoyed based on my country of origin and my educational status (among other things). However, most of the things I’ve done at the Ausländeramt could have been automated. The system for creating appointments should be 100% automated. It should be more granular as to what you need from the appointment, and better inform clients what they need and better allot time. The system should know when your visa expires, and automatically issue a Fiktionsbescheinigung (perhaps with an apology note) if they can’t get you an appointment before the expiration. Even most of the stuff that you need to do at the appointment (present documents, etc.) can be automated through the internet (even if only applicable to a subset of easy cases, like mine)! I’m sure this would make this a much more pleasent experience for the case workers and clients.

Update 2021-07-28: I’ve received a Fiktionsbescheinigung in the post, it was signed and sent on the 26th. It’s good through the end of January 2022 - that’s how behind the office is. Ooof.

Pythagorean Mean Rank Metrics

2021-04-19T12:46:00+00:00

The mean rank (MR) and mean reciprocal rank (MRR) are among the most popular metrics reported for the evaluation of knowledge graph embedding models in the link prediction task. While they are reported on very different intervals ($\text{MR} \in [1,\infty)$ and $\text{MRR} \in (0,1]$, their deep theoretical connection can be elegantly described through the lens of Pythagorean means. This blog post describes ideas Max Berrendorf shared with me that I recently implemented in PyKEEN and later wrote up as a full manuscript.

If you want to cite some of the ideas in this post, you can use:

@article{hoyt2022metrics,
    archivePrefix = {arXiv},
    arxivId = {2203.07544},
    author = {Hoyt, Charles Tapley and Berrendorf, Max and Gaklin, Mikhail and Tresp, Volker and Gyori, Benjamin M.},
    eprint = {2203.07544},
    month = {mar},
    title = {A Unified Framework for Rank-based Evaluation Metrics for Link Prediction in Knowledge Graphs},
    url = {https://arxiv.org/abs/2203.07544},
    year = {2022}
}

The Formulation of the Link Prediction Task

The link prediction task in knowledge graphs is effectively a binary classification task for each potential triple $(h, r, t)$ on whether it is true or not. While the accuracy, precision, recall, $F_1$, Matthews correlation coefficient (MCC), the area under the precision-recall curve, and area under the receiver operating characteristic curve (AUC-ROC or AUROC) are typically used as metrics for binary classifications, the link prediction task has the added twist that there are only positive examples in a knowledge graph. This means that during training, potential negative triples need to be sampled from the finite set of all possible triples. This is called negative sampling within the knowledge graph embedding model literature but more generally called positive unlabeled learning in the wider machine learning community. This works because there’s a much higher likelihood that a triple not already in the knowledge graph is negative than positive. Even if a triple is sampled that should be positive, the model and training algorithm should remain robust.

This also means that alternative metrics need to be introduced that don’t rely on the existence or non-existence of negative examples - enter hits@k , mean rank, and mean reciprocal rank. Each of these relies on the concept of a ranking (not to be confused with the rank of a matrix) that is calculated based on combining the true and sampled triples into a list, then sorting based on their scores (calculated by the trained knowledge graph embedding model). The ranks are the positions of the positive triples in this list, where lower ranks correspond to better performance. For the molecular biologists reading, this is very similar to the rank-based visual inspection of GSEA results on differential gene expression experiments. The hits@k, mean rank, and mean reciprocal rank are all summary statistics over the set of ranks (again, the positions in the list of all true triples + sampled triples, sorted by score).

Definitions of Rank-based Metrics

In the following definitions, I’m going to use $\mathcal{I}$ to denote the set of all ranks for true triples.

Hits @ K

While this post isn’t about hits@k, it’s worth summarizing the alternative approach that it takes to summarizing the rank list because it’s a much more application-driven metric. Effectively, the hits@k describes the fraction of true entities that appear in the first $k$ entities of the sorted rank list. It is given as:

\[\text{score}_k = \frac{1}{|\mathcal{I}|} \sum \limits_{r \in \mathcal{I}} \mathbb{I}[r \leq k]\]

For example, if Google shows 20 results on the first page, then the percentage of results that are relevant is the hits @ 20. The hits@k, regardless of $k$, lies between $(0, 1]$ where closer to 1 is better.

This metric does not differentiate between cases when the rank is larger than $k$. This means that a miss with rank $k+1$ and $k+d$ where $d \gg 1$ have the same effect on the final score. Therefore, it is less suitable for the comparison of different models.

Mean Rank

The mean rank (MR) computes the arithmetic mean over all individual ranks. It is given as:

\[\text{MR} =\frac{1}{|\mathcal{I}|} \sum \limits_{r \in \mathcal{I}} r\]

It has the advantage over hits@k that it is sensitive to any model performance changes, not only what occurs under a certain cutoff and therefore reflects average performance. With PyKEEN’s standard 1-based indexing, the mean rank lies on the interval $\text{MR} \in [1,\infty)$ where lower is better.

While it remains interpretable, the mean rank is dependent on the number of candidates. A mean rank of 10 might indicate strong performance for a large candidate set size of 1,000,000, but incredibly poor performance for a small candidate set size of 20.

Mean Reciprocal Rank

The mean reciprocal rank (MRR) is the arithmetic mean of reciprocal ranks, which can alternatively be formulated as the inverse of the harmonic mean of ranks. It is defined as:

\[\text{MRR} =\frac{1}{|\mathcal{I}|} \sum_{r \in \mathcal{I}} r^{-1} = \bigg(\frac{|\mathcal{I}|}{ \sum_{r \in \mathcal{I}} r^{-1}}\bigg)^{-1}\]

It has been argued that the mean reciprocal rank has theoretical flaws by Fuhr (2018). However, this opinion is not undisputed, cf. Sakai (2021). Despite its potential flaws, MRR is still often used during early stopping due to its behavior related to low rank values. While the hits@k ignores changes among high rank values completely, and the mean rank changes uniformly across the full value range, the mean reciprocal rank is more affected by changes of low rank values than high ones without disregarding them completely like hits@k does for low rank values. Therefore, it can be considered as soft a version of hits@k that is less sensitive to outliers. It is bound on $\text{MRR} \in (0, 1]$ where closer to 1 is better.

Inverse Arithmetic Mean Rank and Harmonic Mean Rank

It’s odd that the mean rank and mean reciprocal rank are formulated in a way such that $\text{MR} \in [1,\infty)$ and $\text{MRR} \in (0,1]$. Since the mean reciprocal rank is just the inverse of the harmonic mean rank, then it would make sense to report the harmonic mean rank (HMR) as well, defined by:

\[\text{HMR} = \frac{|\mathcal{I}|}{ \sum_{r \in \mathcal{I}} r^{-1}} = \frac{1}{\text{MRR}}\]

It has the benefit that it’s more easily comparable to the mean rank because $\text{HMR} \in [1,\infty)$. Alternatively, the inverse arithmetic mean rank (IAMR) could be defined as the inverse of the mean rank (which, remember, is really the arithmetic mean rank) by:

\[\text{IAMR} = \bigg(\frac{1}{|\mathcal{I}|} \sum \limits_{r \in \mathcal{I}} r \bigg)^{-1} = \frac{1}{\text{MR}}\]

This has the benefit that it’s more comparable to the mean reciprocal rank because $\text{IAMR} \in (0,1]$.

Demistying the Metrics

Since we’re thinking about arithmetic means and harmonic means over the ranks, it would make sense to investigate the third Pythagorean mean: the geometric mean.

This figure (from Wikimedia) shows the relationship between the arithmetic mean (A), geometric mean (G), and harmonic mean (H) on a triangle circumscribed by a circle of diameter $a+b$.

Getting Geometrified

For sets of positive numbers (which ranks always are), there is a nice property that the arithmetic mean is the biggest, geometric mean is next, and the harmonic mean is the smallest. Since the mean rank (based on the arithmetic mean) tends to be biased towards low ranks, and the mean reciprocal rank (based on the harmonic mean) tends to be biased towards high ranks, it could be the case that geometric mean of ranks might balance these two biases. We can define the geometric mean rank (GMR) as:

\[\text{GMR} = \sqrt[\|\mathcal{I}\|]{\prod \limits_{r \in \mathcal{I}} r}\]

Like the MR and HMR, $\text{GMR} \in [1,\infty)$. Like with the other metrics, its inverse, the inverse geometric mean rank (IGMR) can be defined as:

\[\text{IGMR} = \frac{1}{\text{GMR}} = \bigg(\sqrt[\|\mathcal{I}\|]{\prod \limits_{r \in \mathcal{I}} r}\bigg)^{-1}\]

where $\text{IGMR} \in (0,1]$.

Additional Statistics

While we proposed the geometric mean rank and its inverse after examining the relationships between the mean rank and mean reciprocal rank, the idea of reporting aggregations over the set of ranks could be extended much further. For example, it makes sense to report other aggregate statistics over the ranks, such as the median rank. This could prove to be more robust to outliers, but it is not clear whether outlier ranks are a major issue in the utility of the existing evaluation metrics or for the newly proposed one. Reporting the standard deviation of ranks, the variance of ranks, and the median absolute deviation of ranks could even enable statistical testing between the results from two different models trained and evaluated on the same dataset, which has been sorely lacking in previous benchmarking studies on using knowledge graph embedding models for link prediction.

Usage in PyKEEN

PyKEEN has always been focused on easy usage. If you want to get these metrics, do the following:

from pykeen.pipeline import pipeline

results = pipeline(model='PairRE', dataset='FB15k-237', ...)
igmr = results.get_metric('inverse_geometric_mean_rank')

The rules for get_metric() are outlined here. Usually, you should write out the metric name in full, with underscores in the middle. The common ones like mrr, mr, and igmr can be accessed by synonym.

We’ve implemented all of these statistics in PyKEEN pull request #381, reported for the left sided, right sided, and two-sided evaluation as well as for the optimistic, pessimistic, and realistic rankings. This blog post was adapted and extended from the PyKEEN documentation.

Current Perspectives on KGEMs in and out of Biomedicine

2021-04-05T12:13:00+00:00

After many discussions scientists from AstraZeneca’s knowledge graph and target prioritization platform (BIKG) about the PyKEEN knowledge graph embedding model package, I joined them in writing a review on biomedical knowledge graphs. I’m giving a talk in their group tomorrow - this blog post is a longer form of some ideas I’ll be presenting there. Here are the slides.

Who is the Arbiter of Benchmarks?

The publication of each new knowledge graph embedding model is usually accompanied by an evaluation of the link prediction task on common benchmark datasets and a comparison to previous models’ performances on those datasets. There’s a variety of ways authors report this, either by copying the original results from the original papers describing the competitor models, by re-running the author’s original code, by reimplementing the code themselves, or by running an implementation from a larger knowledge graph embedding model package.

Who can be trusted? The original authors of each competitor model were/are motivated to show that theirs was better than others, so their paper might have inflated metrics. The authors of the current paper are motivated to show that previous models are worse, so they may deflate metrics for old models (and inflate theirs). Knowledge graph embedding model packages are motivated to inflate metrics to show that using their package is better – I’ve indeed been asked questions about why other packages are “performing better” than PyKEEN and had to answer that they’re just evaluating differently that is less conservative, and sometimes misleading. Even in a world without these competing interests, it’s very hard to understand each other’s code, especially since it’s not just about the model itself but also the training and evaluation pipeline.

This motivates large benchmarking studies that use unified frameworks and re-implementations of models to investigate the relative performances of models (and can for a brief moment, let you throw away your metric fixation). The PyKEEN benchmark is the largest and most comprehensive to date, covering many models, datasets, and configurations. I was proudly a part of this work under the leadership of Mehdi Ali @mali-git and with an amazing team. There’s also the smaller LibKGE benchmark as well as several others that we’re maintaining in a curated list in this meta-review repository.

The Open Graph Benchmark (OGB) provides a potential solution to these issues by externalizing and standardizing the code that evaluates the results. There are still sneaky ways to game this that I won’t describe in detail, but I like the concept. However, the framework does not cover some of the more popular datasets and the ones that it does introduce in the link prediction task, it does not do so with any provenance on how they’re created. During my talk, Benedek Rozemberczki (@benedekrozemberczki) brought up another interesting point - many of the benchmarks in OGB have surpassed the size that can be addressed by the physical infrastructure or cloud compute budgets of many small academic institutions. As machine learning continues to rely on larger, more exotic models, we will soon have to reckon with this divide.

Testing/Validation Leakage

Leakage is when triples in the testing/validation sets can be trivially inferred from triples in the training set. This leads to an over-estimation in the performance of the model.

The most common form of leakage occurs when a knowledge graph contains inverse triples. An example:

A and B are entities in a knowledge graph
part of and has part are inverse relations in the knowledge graph
A part of B is in the training set
B has part A is in the testing set

This scenario occurs in two datasets widely used in knowledge graph embedding model benchmarking: the FreeBase 15K (FB15k) dataset proposed by Bordes et al. (2013) alongside the TransE model and the WordNet-18 ( WN18) dataset proposed by Bordes et al. (2014) alongside the Unstructured Model. Toutanova and Chen (2015) outlined an algorithm for removing leakage due to inverse triples and applied it to respectively derive two new datasets: the Freebase 15K-237 (FB15k-237) dataset and the WordNet-18 (RR; WN-18RR) dataset.

Dataset	Entities	Relations	Triples
WN-18	40,943	18	151,442
WN-18RR	40,559	11	92,583
FB15k	14,951	1,345	592,213
FB15k-237	14,505	237	310,079

These datasets are of roughly the same size and are widely available. Therefore, if I read a paper that presents results on only FB15k and WN-18 but not FB15k-237 and WN-18RR, I am concerned that the authors either do not know about this problem, or are intentionally misleading the reader.

An alternative kind of testing leakage can occur in chemoinformatic applications. The following three chemical structures are from a structural class called statins (CHEBI:87631) that inhibit the HMG-CoA reductase enzyme and are often used to lower blood pressure to reduce the risk of heart disease.

mevastatin (CHEBI:34848)	lovastatin (CHEBI:40303)	simvastatin (CHEBI:9150)

All three share a stereospecific cyclic lactone motif (top right part of the molecule) that result in their functional activity. If you’re building a classifier for activity against HMG-CoA reductase, or more generally, a link prediction model that includes chemical-protein or chemical-enzyme class predictions, including some of these molecules in the training set and others in the test set would enable trivial inference and ultimately result in an over-estimation of the performance of your model (barring discussion about how this could be considered trivial is a huge triumph of modern pharmacology and cheminformatics). The solution often referenced in proteochemometrics papers is to use scaffold-based splitting like Bemis-Murcko scaffold splitting or to pre-cluster molecules based on structural similarity and to split based on clusters. More information on these methods is reviewed in Bongers et al. (2019) and Parks et al. (2020).

Redundant Information in Graphs

Transcriptomics experiments are often analyzed with statistical over-representation analysis or gene set enrichment analysis (GSEA) with respect to gene sets derived from pathway databases like KEGG, Reactome, WikiPathways, the Pathway Interaction Database, and BioCyc, functional annotations such as those from the Gene Ontology, and biological signatures such as those listed in MSigDB. In addition to p-values, the high number of statistical tests also motivates the reporting of adjusted p-values to address multiple hypothesis testing. However, these adjustments do not consider the high overlap in information due to the redundancy of pathway databases or potential correlations between expert-defined pathways and certain functional or biological signatures. Therefore, several results from the same theme or biological phenomena often co-occur as enriched which often leads to an inflated sense of concordance of results and thus higher confirmation bias. Further, Mubeen et al. (2019) showed that the variability of definitions in each pathway database for the “same” pathway (e.g., apoptosis) caused significantly different results in many downstream tasks.

Many benchmark biological knowledge graphs such as Hetionet and OpenBioLink include multiple pathway databases. Therefore, link prediction tasks between genes and pathways (such as the case scenario presented in Ali et al., 2019) could be skewed both during the training and evaluation of link prediction models, and ultimately in the interpretation of predicted results. While methodological improvements like SetRank (Simillion et al., 2017) have been proposed to mitigate the redundancies in the over-representation analysis setting, there have not yet been methodological improvements for link prediction models that consider these kinds redundancies. Further, these kinds of redundancies do not fit as neatly into the “leakage” issue.

Tools like ComPath can be used to enrich KGs with links between functionally equivalent pathways, but this kind of information is not readily used by typical knowledge graph embedding models. Alternative rule-based and symbolic reasoning systems might present solutions in this area.

The Effect of Non-Determinism

There are several meaningful non-deterministic factors that affect the performance of a model:

The effect of random initialization. The commonly used pytorch.nn.Embedding initializes its weightings by default with the vanilla pytorch.nn.init.normal_ function. Using alternative parameters for the distribution (e.g., Xavier/Glorot) have been shown to improve performance in a variety of settings.
The effect of non-determinism during training that occurs during batch shuffling. This has shown to quicken convergence and mitigate overfitting based on the ordering of training data.
The effect of random negative sampling when training under the stochastic local closed world (sLCWA) assumption.

To compensate for the non-determinism, several trials using the same settings should be run and the distribution of each metric (or at minimum, summary statistics over their means and standard deviations) should be presented. Below, the distribution of mean ranks resulting from 10 trials of training of several models on the Hetionet dataset are presented (credit to Stephen Bonner @sbonner for running this trial and generating the chart).

From this chart, it is obvious that the DistMult model has the potential to perform both very well and very poorly based on the random initialization. There are two reasons that this might be the case for this model/dataset in particular:

The interaction for DistMult function uses the Hadamard operators, which commute, so f(h,r,t) = f(t,r,h). With the Hetionet dataset, where there is meaningful directionality and also the existence of inverse edges, this could be a liability to both meaningful training and accurate evaluation
The L_1 norm formulation of DistMult makes it much more susceptible to random initialization than if the L_2 norm were specified. Because the original formulation of the DistMult model uses the idiom of matrix factorization, the norm’s p is not configurable within PyKEEN (the tool used to train these models and generate these charts).

While there are a few insights into the formulations of models and the properties of datasets that could inform users to avoid using models like DistMult with datasets like Hetionet, the more general case is likely more bleak - even small changes in hyper-parameters could potentially affect the robustness of a model to non-determinism. However, there does not yet exist a comprehensive benchmarking study (even over small KGs) to confirm this nor elucidate general insight.

As an end note - this is not meant to say that non-determinism should be mitigated nor removed. It is a vital part to creating meaningful results and fixing the random seed is certainly not a viable solution.

Splits Aren’t Gospel

Benchmark datasets for knowledge graph embedding models typically consist of a pre-stratified training, testing, and validation set. The FB15k, FB15k-237, WN18, WN18-RR, and YAGO3-10 are the most commonly used to assess the performance of new models or compare the performance of old models due to their relatively small sizes. More specifically, the same exact pre-stratification is used in all papers.

While this potentially could make results more comparable across different experiments, it presents a huge liability in that it is another “random” effect that is completely discarded. Rather than present results on a single split, results should be presented as a distribution over several splits to show if there are certain properties of the split that cause some models to perform better than others. Perhaps this kind of evaluation would have informed Bordes et al. about the issues with the FB15k and WN18 datasets before they were later identified by other authors. This also presents a practical problem since there are potentially many ways to generate dataset splits, and typical knowledge graph embedding model researchers likely aren’t interested in adding another level of cross-validation to their already complex pipelines.

This issue is more apparent when applying knowledge graph embedding models to datasets that are not pre-stratified such as the Common Sense Knowledge Graph, NELL, or any biomedical knowledge graph. It can again be demonstrated with another image from Stephen Bonner. Below, the distribution of mean ranks resulting from 10 random splits of the Hetionet dataset and using the same training configuration for several models.

The confidence intervals support an even more startling conclusion - the effect of random splitting of the dataset could cause a model like DistMult to outperform models like ComplEx due to its huge variance. Further, datasets like Hetionet definitely are susceptible to the leakage problem. Note: there’s a prototype implementation of the Toutanova and Chen algorithm for removing leakages in PyKEEN, but it’s neither finished nor integrated in the PyKEEN pipeline. Combine with DistMult’s penchant for modeling inverse relations the same as the canonical ones, this could wreak havoc on its performance depending on how bad the leakage was by chance. Like with random initialization, there has not yet been a comprehensive benchmarking that could give more general insight into the liabilities of not reporting distributions of results over several random splits. Even further, combine with the previous section’s prescription to report distribution over multiple random initializations, doing a “proper” investigating of robustness seems to be getting more high dimensional.

Do KGEMs Actually Learn?

The formulation of the link prediction task on knowledge graphs as a binary classification task over a dataset of true positives and randomly sampled negatives (under the local closed world assumption) is already a concession to many more obvious machine learning tasks (like classifying Mendel’s irises). Even further, many machine learning techniques have the concept of a null model against which they can compare if their results are meaningful, or just good by chance. In classification tasks, this is often done with y-scrambling which compares the evaluation metric obtained from the real dataset to the evaluation metrics obtained by repetitively shuffling the response variable and generating a null distribution of evaluation metrics against which the real one can be compared. Ultimately, the p-value of a one-tailed test against this distribution tells you if you actually learned something, or if your results are the same as what you’d get with a random dataset.

While this is a pretty typical test that reviewers ask for in other machine learning tasks, it has not (yet) penetrated the knowledge graph embedding model domain. I think this likely limited by the difficulty in writing performant research code. While tools like PyTorch BigGraph and Amazon’s DGL are able to scale across many machines or GPUs, they are not accessible to anyone without significant dev-ops abilities nor are they frameworks that could support researchers. PyKEEN itself has the opposite problem (for now, get hyped for new improvements in 2021). A solution might lie in Marius, a new knowledge graph embedding model library that could be the best of both worlds. Assuming these weren’t considerations, there are two themes through which I could see the question “did I learn anything?”: knowledge graph reorganization and knowledge graph deterioration.

The idea of reorganization is a bit more classic, coming from previous approaches. There are a few concepts that could be used to generate derived knowledge graphs based on a given one that could be use to generate null networks:

Degree-preserving network shuffle (pmc:PMC3307026; pmc:PMC5168876)
Xswap (paper; code)
Relabeling of nodes, similarly to the y-scrambling technique

Himmelstein et al (2017) provides a gold standard on how these techniques could be used to report on how meaningful results are from a link prediction model on a knowledge graph.

The idea of deterioration encompasses randomly removing edges from the knowledge graph and training. It could determine how much of knowledge graph is actually needed to train the model to make meaningful prediction. This would be done for multiple increments (e.g., remove 10% of the training set, remove 20% of the training set) with multiple trials on each.

Are KGEMs King Now?

The COVID-19 pandemic has not only been a battle for public health, but it has also resulted in the largest wave of low quality scientific work in recent memory. Lots of this low-quality work included half-baked usage of network representation learning, matrix completion, and for the first time, knowledge graph embedding models in the mainstream to predict drug-disease links. It was lacking on many fronts: the formulations of knowledge graphs, the evaluation of training and link prediction, and the communication of results.

As a critique, I will share with you my own drug repositioning algorithm for COVID-19: take the list of all drugs in DrugBank, and randomly select 100 of them. You’d be surprised how competitive this algorithm is with others.

The more upsetting examples was from a very prominent network science group that used the RotatE (and other standard) knowledge graph embedding models for the link prediction task for drug repositioning for COVID-19. They evaluated the task on its ability to recover drugs that had already gone into clinical trials for COVID-19, and claimed victory because they were able to recapitulate enough of the list. This was a bad evaluation because the drugs in the clinic were there based on inference made on the same data used to train and evaluate the model. More importantly, it was bad because it was a famous group that misrepresented the way science is done - link prediction in biomedicine is a way to generate hypotheses that can be tested in the lab. Focusing communication around its ability to recapitulate what’s in the training set (i.e., overfitting) is one of the first lessons that students are taught to avoid in their introduction to machine learning classes.

Does this mean that knowledge graph embedding models are king now? No - there’s still a long way to go in biomedicine before they are accepted as canonical tools, that must include some more high-profile papers in that include experimental evaluation of the results. This will most likely happen in the chemistry or target prioritization domains, where the predictions of the model can motivate experiments of actual value.

As a few of the vignettes in this post mentioned, there are still problems with model formulation that can cause huge performance deficits on biomedical knowledge graphs. A different class of models based on random walks, while less complex than knowledge graph embedding models, continue to perform well on tasks in biomedical knowledge graphs because their formulation is oriented towards extracting local community structure - a feature that is very important in biology. Several attempts have been made to extend these to better take into account directionality and edge type. Some of those generalizations even fit into the philosophy and architecture of graph neural networks (GNNs). Further, there are lots of quantitative data such as chemical-enzyme IC₅₀ values, chemical-receptor EC₅₀ values, GWAS p-values, etc. that all have widely different scales and dynamic ranges that are difficult to simultaneously incorporate in any of these methods in a meaningful way.

Willkommen in der Zukunft (The Future)

Random parting thoughts:

Biomedical knowledge graph benchmarks are consistently topping out around 5 million edges. If I had to guess, that’s because that’s the number where if you go higher, you have to start using some packages that are not meant for users like Amazon’s DGL or PyTorch’s BigGraph. We’ll need more accessible, scalable systems if we’re ever going to move on from drug repositioning, where there are only on the scale of thousands of chemicals against hundreds of proven targets, to bona fide drug discovery, where there are on the scale of billions of chemicals (or more if you want to get fancy with virtual screening) against potentially the whole genome of ~20K protein coding genes, the products of other non-protein-coding genes, and other stuff floating around in and around the cell.

Negative knowledge should be emphasized. Formats like the Biological Expression Language (BEL) should support its curation, natural language processing systems and tools like INDRA should start extracting in bulk, and the concept of the open world assumption, local closed world assumption, and closed world assumption need to be amended to better reflect the reality that we know some negative knowledge, but it’s distributed across different relations in different densities.

Genes, RNAs, and proteins are not the same! Even worse, they can have several variants due to differential splicing, and they can have all sorts of pre- and post-translational modifications. BEL and related formats like BioPAX can represent these, but the knowledge graph approach to biology tends to throw this away completely. On the other hand, high granular information on this level of specificity is likely too low to make dense knowledge graphs over which typical tasks like link prediction can be successful. This motivates a greater focus on modeling the hierarchies of information (a task at which INDRA excels) to generate knowledge graphs that balance these competing qualities.

The entire semantic infrastructure for building knowledge graphs is improving (I’d like to say in some small part due to the tools I’ve built) that enables mappings between entities, such as Inspector Javert’s Xref database and the new SSSOM format to make mapping data more standardized.

Explaining MCI Conversion with Path Queries to NeuroMMSig

2021-02-23T22:53:00+00:00

In late 2017, I visited the Critical Path Institute in Tucson, Arizona with my colleague Daniel Domingo-Fernández to use our Alzheimer’s disease map encoded in the Biological Expression Language (BEL) and the tools we built with PyBEL to help contextualize their mild cognitive impairment (MCI) conversion models. We got very interesting results, but they had a major overlap with unpublished work of one of our colleagues on the role of KANSL1 in Alzheimer’s disease, so we never reported them. Last week, his paper finally made it publication (congratulations, Sepehr!) so I thought it would be fun to rehash the old results and look at how the results might have changed over time with improvements to the underlying knowledge graph.

After a long flight from Germany and layover in Phoenix (remember before the pandemic when that was a thing?), we were received at the Tucson Airport by Klaus Romero, the Director of Quantitative Medicine at the Critical Path Institute (C-Path). He helped us acclimatized to the quiet, flat plains of the Sonoran Desert with a ride down its long stretches of highways in his BMW V8 on our way to our accommodations in the city.

We were lucky to have arrived at C-Path when we did - Klaus’s team was mostly remote, but met in person once or twice a year. They were the mavericks of the institute - the team of computational biologists, pharmacologists, and toxicologists who took advantage of the deep ties of the institute to regulatory bodies like the American Food and Drug Administration (FDA) to pilot some of the first computational tools to gain regulatory approval for use in a clinical setting.

As our working lay on the computational side of neurodegeneration, we were introduced to Daniela Conrado and her work on a clinical data-driven conversion model from mild cognitive impairment (MCI) to full Alzheimer’s disease. Ultimately, it was a linear mixed-effect model that implicated several clinical exams and measurements as co-variates:

Clinical Dementia Rating (CDR)
Clinical Dementia Rating Scale (sum of boxes) (CDR-SOB)
Mini Mental State Exam (MMSE)
APOE ε4 status
beta amyloid 40
beta amyloid 42
hippocampal volume
hippocampal atrophy

It became our goal to use NeuroMMSig (the Alzheimer’s disease map) to identify a potentially explanatory sub-graph (i.e., a mechanism) that connected all the co-variates. We were well-positioned by using a disease map encoded in BEL that allows for the inclusion of multi-scale knowledge across the molecular, pathway, and phenotypic levels in addition to correlative and associative relationships that aren’t so easy (or possible) with other modeling formalisms like BioPAX, SBML, SBGN, Cell Designer, GO-CAM, etc. After spending months re-curating the NeuroMMSig subgraph following the end of the AETIONOMY project, we were excited to apply it and all of its shiny new goodness. What comes below is a slightly edited version of the report we wrote at the end of our visit, hence its more scientific voice (as opposed to my light-hearted blogging voice).

Methodology

When we first mapped the co-variates in the MCI conversion model to nodes in our knowledge graph, we found some had high information density, like the amyloid and APOE nodes, but the clinical measurements were relatively low. While this would generally lead to poor robustness, an advantage of curated knowledge graph-based approaches is that we were able to curate additional relevant knowledge for these nodes by making use of automated and semi-automated relation extraction pipelines as well as manual curation for enrichment. Further, the clinical relationships were curated during the work presented in NIFT project, so we were able to adapt the curation guidelines and apply them once again. We did some curation with the help of master’s student of molecular biology Lukas Beniusis who was working in our group back in Germany as a student research assistant at the time. Ultimately, the resulting curated content found its way to GitHub in the CONIB repository under the CC BY 4.0 license. After substantially broadening the connections with other nodes, more hypotheses could be generated in algorithmic steps.

Graph traversal algorithms were used to find paths connecting the co-variates in the Alzheimer’s disease map that had the appropriate polarity (a series of increases, decreases, positive correlations, and/or negative correlations that matched the observations of the MCI model). These paths were combined to sub-graphs that could be interpreted and so individual paths could be prioritized by their explanatory potential. Finally, the sub-graphs were filtered to best rationalize the MCI model with the relevant molecular biology. All of these steps were done in a local instance of BEL Commons that was loaded with the NeuroMMSig BEL file.

We didn’t spend the entire time in Arizona working. We also got to run in a dried up river bed, play Frogger trying to cross the street, and eat Klaus and his wife’s amazing cooking.

Results

The most interesting hypotheses are presented in the graph below. The first hypothesis (yellow) involves three entities classically associated with AD: the amyloid precursor protein (APP); one of its secretases (BACE1); and the amyloid beta 42 (Aβ-42) proteolytic cleavage fragment from the cleavage of APP by BACE1. They participate in a positive feedback loop where an increase in the abundance of the secretase leads to more amyloid beta aggregation. Since this process is widely described in the literature, there are multiple connections, or crosstalks, from this part of the graph to the more novel paths we describe in the following.

rs2732703

The second hypothesis (red) focuses on the possible effects of the rs2732703 variant in the KANSL1 gene, which has been statistically linked to the hippocampus, and indirectly to hippocampal volume. The gene product of KANSL1 forms an acetylation complex with KAT8 that regulates the expression of genes involved in the oxidative phosphorylation localized in the mitochondria. Therefore, dysregulation of the transcription of the KANSL1 gene due to this variant might lead to oxidative stress and ultimately to inflammation — one of the main processes associated with AD.

There may be a crosstalk between APP and KAT8 since the APP intracellular domain (AICD) has been observed to form a ternary complex with FE65 and KAT5. Because KAT5 and KAT8 both share the domain that mediates the formation of this complex (histone acetyltransferase domain, MYST-type ( IPR002717)), we suggest that KAT8 could act through the same mechanism as KAT5 to regulate the expression of multiple genes (Müller et al., 2013).

DYRK1A

The third hypothesis (green) suggested by the subgraph involves DYRK1A (dual specificity tyrosine-phosphorylation-regulated kinase 1A) which is negatively correlated with the Clinical Dementia Rating Scale - Sum of Boxes (CDR-SoB), a cognitive score included in the clinical MCI model. This kinase phosphorylates two proteins deposited in the brain that are hallmarks of AD: tau protein (MAPT) and APP (Wegiel et al., 2011). While hyperphosphorylation of the Tau protein causes the aggregation of neurofibrillary tangles, phosphorylation of APP increases its cleavage leading to higher levels of beta-amyloid. Furthermore, DYRK1A acts as a regulator of apoptotic processes and neuronal death, which might result in hippocampus atrophy (Choi & Chung, 2011).

SERPINE1

The final hypothesis (blue) is predicated by the positive correlations between SERPINE1 abundance and two co-variates from the MCI model: the CDR-SoB and the mini mental state exam (MMSE) (Oh et al., 2014). These correlations motivate further curation that could provide more causal insight as well as investigation of measurements of this protein as a biomarker for these clinical endpoints.

Afterthoughts

Our findings highlighted crosstalk between several well-known mechanisms (e.g., beta-amyloid, oxidative stress, and inflammation) and the co-variates observed by the MCI model. Furthermore, the analysis identified various controllers that might regulate these processes: rs2732703, DYRK1A, and SERPINE1 that may be appropriate to investigate in a clinical setting.

However, knowledge-based approaches are inherently incomplete, and are not a replacement for hypothesis-driven experiments or clinical trials. Even in our analysis, we were not able to account for some co-variants, such as sex, because of the nature of the data structure for storing knowledge and the granularity of available knowledge itself. Interpreting hypotheses generated by a purely knowledge-driven environment also requires great care. In this exercise, we postponed exploration of another classic co-variate, the APOE mutations due to this constraint.

Additional knowledge enrichment around the clinical endpoints used in MCI models could be used to provide further support for these hypotheses as well as facilitate the generation of new hypotheses. Back in 2017, we wrote that future work could include expansion of the graph around the genes of interest with dedicated literature mining approaches as well as enriching the graph with further knowledge around the co-variates. I mentioned Sepehr’s paper A Systems Biology Approach for Hypothesizing the Effect of Genetic Variants on Neuroimaging Features in Alzheimer’s Disease at the onset in which he prioritized curation around the SNPs associated with Alzheimer’s disease and related neurodegenerative diseases like Parkinson’s disease through genome-wide association studies (GWASs). He also converged on the KANSL1 gene through an orthogonal analysis after which he (luckily) did orthogonal curation to ours to add additional content to NeuroMMSig than what we did. In the years since this analysis, we also made huge improvements to PyBEL and BEL Commons, the interactive visualization platform in which we did the analysis presented here that makes it much more simple and flexible.

Despite several publications using the NeuroMMSig BEL content, none were ever accompanied by a release of the underlying BEL content. In 2018, I began to lead work on the Human Brain Pharmacome Project which included additional curation to enrich NeuroMMSig. I made sure that all of this found its way directly into the open source. If/when the NeuroMMSig BEL content is made publicly available, I will follow up this post with a very short script showing how to do the following to reproduce the graph presented above:

Load a BEL graph
Create a shortest paths query using the pybel.struct.get_subgraph_by_all_shortest_paths() function
Visualize the resulting subgraph in a Jupyter notebook.

Until then, good luck with the next wave of (endless) beta amyloid antibodies!

References

Choi, H. K., & Chung, K. C. (2011). Dyrk1A positively stimulates ASK1-JNK signaling pathway during apoptotic cell death. Experimental neurobiology, 20(1), 35-44.
Domingo-Fernández, D., et al. (2017). Multimodal Mechanistic Signatures for Neurodegenerative Diseases (NeuroMMSig): a web server for mechanism enrichment. Bioinformatics, 33(22), 3679-3681.
Müller, T., et al. (2013). A ternary complex consisting of AICD, FE65, and TIP60 down-regulates Stathmin1. Biochimica et Biophysica Acta (BBA)-Proteins and Proteomics, 1834(1), 387-394.
Oh, J., Lee, H. J., Song, J. H., Park, S. I., & Kim, H. (2014). Plasminogen activator inhibitor-1 as an early potential diagnostic marker for Alzheimer’s disease. Experimental gerontology, 60, 87-91.
Wegiel, J., Gong, C. X., & Hwang, Y. W. (2011). The role of DYRK1A in neurodegenerative diseases. The FEBS journal, 278(2), 236-245.

Subgraph Reference PMIDs

rs2732703: 25778476, 27768893, 21034527, 22544363, 27768893, 23092889
DYRK1A: 21156028, 22110360, 25116835
SERPINE1: 25304332

Adding Structured Data to Docstrings

2021-02-20T20:12:00+00:00

Writing excellent documentation is crucial for open source software projects. It’s also a lot of hard work. While I consider tools like Sphinx combine with services like ReadTheDocs completely invaluable, I’ve recently hit a bit of a roadblock when it comes to making the README of a GitHub repository a bit more dynamic. This blog post is about the dark magic I invented as a solution (i.e., the docdata package).

How Python Documentation Works

Before beginning, I want to give a quick refresher on what documentation looks like in Python. For any class or object, you can write a docstring using triple-double quotes on the first line after the definition

def sin(x):
    """Compute the sin of x."""
    ...


class MyClass:
    """This is my class."""
    ...

It turns out that this is a bit more extensible than I thought. While the triple-double quoted string is the community standard, you can also get away with using triple-single quote, single quote, or double quote as well. Even crazier, you can introduce a blank line before the docstring. While these are possible, please don’t do this. There’s another interesting feature about the docstring that makes it different from any old string sitting in Python code - it’s not evaluated. This means that if you use an expression that isn’t a string literal, it won’t be set to the docstring. The following code illustrates this:

def get_docstr():
    return """this is my docstr"""


def my_func():
    get_docstr()
    return 5


assert my_func.__doc__ is None
assert my_func() == 5

Not only are docstrings useful for readers of your code, but Python considers them with great respect. PEP 257 outlines in full detail, but the important thing to keep in mind is that Python code can introspect on the docstring for any function, class, etc. with the special __doc__ attribute (e.g., MyClass.__doc__ stores the docstring itself). This feature is what enables tools like Sphinx to exist without having to write an entirely new parser for Python files.

A Tale of Two READMEs

The rest of this journey will be told through the perspective of my work on the documentation of PyKEEN, a machine learning library for learning low-dimensional embeddings for nodes and edges in knowledge graphs. In this blog post, you don’t need to understand anything about the package itself other than it has several types of interchangeable components that can be combined to create a model that gets trained on a dataset. Its documentation uses the sphinx-automodapi extension to generate pretty lists of all the datasets, models, loss functions, regularizers, etc. (example).

The problem is that most people start to use a given package by either looking at the README file in the GitHub repository, or the splash text on the PyPI project page (which, for PyKEEN and most packages, is created with the README on upload). I wanted to generate beautiful tables describing the components on the README file the same as in the Sphinx documentation, so I started by writing a template markdown file using jinja as a templating language. For each type of component, I programatically built a table, formatted it as markdown with tabulate, and formatted it into the template.

The tricky part was making these tables better than just lists of the names of the classes. Sphinx has a deep integration with the restructured text (RST) format and provides custom “directives” like :class: that allow for automatic linking between documentation for modules, classes, functions, variables, or anything else. Luckily, the sphinx-automodapi uses a standard format for its documentation. For example, the pykeen.models.ComplEx class gets built with a URL like https://pykeen.readthedocs.io/en/stable/api/pykeen.models.ComplEx.html . The general form for is https://pykeen.readthedocs.io/en/stable/api/.html. I was able to take advantage of this and generate a column with the name of the class in PyKEEN with a link to the documentation for the class on ReadTheDocs. This also gives insight to users who might want to import these classes themselves.

The next tricky part was providing some context besides just the name and class name. For new users looking at the models in PyKEEN, it’s also useful to show a citation. This typically includes the first author’s last name and the year as in “Ali et al., 2019”. Additionally, the citation should link to the paper itself for further reading past what the PyKEEN documentation for the model provides. As an aside, it’s one of my personal goals for PyKEEN’s documentation to be an educational resource that in many cases will be more useful than reading manuscripts written by computer scientists, whose goals are to make themselves smart more than to motivate and educate the reader. In RST, there’s a syntax for linking citations that the PyKEEN documentation organizes in a documentation-wide bibliography. Unfortunately, my templating system is not as powerful as Sphinx, and does not parse all of these files. The solution I had was to standardize the citation keys and the format of the first line of each model’s docstring such that the name and year could be extracted with some simple text processing because I enforced the standard that all model docstrings ended with the RST citation.

from pykeen.models import ComplEx

doc = ComplEx.__doc__
doc_lines = doc.splitlines()
line = doc_lines[0]  # get the first line of the docstring
l, r = line.find('['), line.find(']')
author, year = line[1 + l: r - 4], line[r - 4: r]

Then the author and year could be formatted into a new column in the previous format. However, programatically getting the citation link was a completely different problem. One solution I considered was to start adding class variables with this information, but that would quickly become a distraction to users.

My Solution

Then, I remembered a cool feature of flasgger, which automatically generates a Swagger interface for Flask applications by embedding the Swagger definition as YAML inside each route’s docstring. I didn’t look into their code for an implementation and tried it my own with some pretty fast success! The simple (but robust) code that I wrote for PyKEEN seemed generally useful, so I moved it into its own package docdata - so others could easily use it. It went very fast because I recently put a lot of effort into creating a Cookiecutter package with all of my favorite settings that I’ve covered in previous posts on this blog.

It does the same thing as flasgger - it allows the final few lines following a delimiter string (i.e., ---) of a docstring to be parsed as YAML and stored in the object. It can be applied as a decorator to functions and classes, or simply to any data or Python object that has a docstring. A demonstration shows it all:

from docdata import parse_docdata, get_docdata


@parse_docdata
class MyClass:
    """This is my class.

    ---
    author: Charlie
    motto:
    - docs
    - are
    - cool
    """


assert get_docdata(MyClass) == {
    'author': 'Charlie',
    'motto': ['docs', 'are', 'cool'],
}

The data can also be accessed directly via MyClass.__docdata__ but dunders are scary, and I thought it would be nice to have a getter as well.

This was immediately useful for PyKEEN models because I was able to store all the citation information in a structured way at the bottom of the docstring. Then, I was able to improve my table generator to make a much more rich column for citations that included the link to each. I also did something for datasets in PyKEEN, but additionally included statistics about each dataset’s entities, relations, and triples to make the PyKEEN README even more useful. The full pull request on PyKEEN can be found at pykeen/pykeen!303.

If you’re interested in the philosophy of documentation, a good place to start is here, or any other talk given by Daniele Procida.

Adding New Literature Sources to the Wikidata Integrator

2021-01-23T20:12:00+00:00

Scholia is a powerful frontend for summarizing authors, publications, institutions, topics, etc. that draws content from Wikidata. However, the content that’s available in Wikidata depends on what has been manually curated by community members and what has been (semi-) automatically imported by scripts and bots. The Wikidata Integrator from the Su Lab at Scripps automates the import of bibliometric information from Crossref and Europe PMC. This blog post is about how I added functionality to it to import from three prominent preprint servers in the natural sciences (arXiv, bioRxiv, and ChemRxiv) that can serve as a guide to others who want to have content about their field included with this tool.

A while back, I wanted to put an arXiv paper about the role of metadata in reproducible computational research that I co-authored with Jeremy Leipzig (@jermdemo) on Wikidata so it would appear on my Scholia page, but I didn’t want to do it by hand. I had already had good experiences using Magnus Manske’s SourceMD, Author Disambiguator, and related tools for queueing import of peer-reviewed papers with PubMed identifiers or digital object identifiers (DOIs), ORCID identifiers of co-authors, and ultimately assisted disambiguation of author names.

The only problem was that the SourceMD tool doesn’t deal with preprints. As an aside, both bioRxiv and ChemRxiv assign DOIs to their preprints that can be queried through Crossref, but this doesn’t exactly solve the problem of handling preprints in a special way, and it doesn’t solve the problem for arXiv. I asked around on Twitter and was turned towards the Wikidata Integrator project, which already supported building both general pipelines for automating content import in Wikidata and a specific one for publications. All I had to do was figure out how it works, and get hacking! I ended up sending the following three pull requests:

Server	Pull Request
arXiv	SuLab/WikidataIntegrator#140
bioRxiv	SuLab/WikidataIntegrator#169
ChemRxiv	SuLab/WikidataIntegrator#140

At the time of writing, the arXiv and bioRxiv ones have been accepted, and the ChemRxiv one is waiting for review - I think the maintainer was busy at a conference this week, and now I’m writing on a Saturday. In another unrelated pull request to the project, I added a vanity CLI, so when you install the code with pip install wikidataintegrator, a program wikidataintegrator-publication is installed for direct usage from the shell. It can be used like this:

$ wikidataintegrator-publication --idtype arxiv 2101.05136
$ wikidataintegrator-publication --idtype biorxiv 2020.08.20.259226
$ wikidataintegrator-publication --idtype chemrxiv 13607438

This resulted in the following three Wikidata pages:

arXiv: Q104846171
bioRxiv: Q104920313
ChemRxiv: Q104931192

If you’re using a DOI or PubMed identifier as the --idtype, you also have to specify a --source, but since each of arXiv, bioRxiv, and ChemRxiv have their own custom sources, this isn’t necessary. The program will print the Wikidata identifier (starting with a Q followed by some numbers) of the newly created item, or an error message if there was a problem. It’s quite smart and avoids creating duplicate pages by checking if the ID type has already been used with a pre-defined Wikidata property that goes with it. More on that below in the tutorial, since we have to make that definition when adding new sources.

Implementing a New Importer

Luckily, all the work in implementing a new importer happens in one python module: wikidataintegrator.wdi_helpers.publication . You can begin by clicking the edit button, which will automatically fork the repository and create a new branch. I’m terrible using git and managing multiple remotes, so this is my preferred way to start a PR in any repository that I might want to PR more than once.

Adding the right metadata

The first step is to identify the Wikidata property corresponding to entries in your new source. This was already the case for arXiv (P818) and bioRxiv (P3951), so I added an entry to the ID_TYPES dictionary in the Publication class where the key is the name of the source (not necessarily the name on the property, keep it short and simple).

The second step is to identify the Wikidata item corresponding to the source itself and add it to the SOURCES dictionary in the Publication class (just below the ID_TYPES dictionary).

Implementing the getter

The third step is where the domain logic about your source comes in. You need to implement a function that takes in an identifier that spits back an instance of the Publication class. It also has to have an id_type argument where the default value matches to the key you used in the sources. This actually isn’t used anywhere, but must be there because of the interface that consumes it. This function can live towards the bottom of the Python file and isn’t inside a class. It’s best to put it next to the bioRxiv, arXiv, and Crossref ones.

Most of the way you get data from your source is up to you. Most sources have some kind of endpoint that can be queried and returns JSON - note that Wikidata Integrator has a consistent USER AGENT that tells services what kind of code is querying it. This is important if you’re hitting an API with many queries so sysadmins can see what’s going on.

The Publication class is pretty self-explanatory except for a few parts.

The authors keyword arguments takes a list of dictionaries whose keys are full_name and orcid. You can omit orcid or pass None. bioRxiv isn’t currently providing author information like ORCID identifiers in its API, so it did not show up in this example, but you can see in the other parts of the code for PMC and PubMed how this works.
Make sure that the keys in the ids keyword argument correspond to keys in ID_TYPES. If your document has more than one ID, you can put them her (though there is some debate whether this is a good idea).
You have to set the instance_of property not with the __init__() because it gets handled with a Python descriptor. This is a design choice in the Publication class that is out of our hands.
Don’t forget to set the published_in_qid which will correspond to the venue on your wikidata item.

Tying it all together

In the fourth step, you need to register your newly implemented function in the SOURCE_FUNCT dictionary inside the PublicationHelper class. Make sure the key you use is consistent with before, and use the function as the value. Don’t use parentheses, since you want to actually have the function be the value and not the result of calling the function. This is part of a programming paradigm called “functional programming”.

In the fifth and final step, you need to scroll up to the get_or_create() function in the Publication class to tell it which ID is the primary key for your item. Add a new conditional elif self.source == 'your key' to check for your source key (same as all the other places you added it) then add the key corresponding to your source from the ids (this was the part from step 3) and the correct property. The previous examples use PROPS which is redundant. You can use self.ID_TYPES['your key'].

Testing

Try running the following code with a valid ID and hope that everything works! If you are having issues here, then you can always send a draft pull request to solicit help from the maintainers of the project.

$ wikidataintegrator-publication --idtype "your key" "your id"

Be careful here, since this will hit the live Wikidata instance. If you make a new item that has a problem, please try to fix it since deleting entries from Wikidata isn’t so common, and we don’t want to add to the mess!

My Source Doesn’t Have a Wikidata Property

In the case of ChemRxiv, DOIs are available for each article, so I did not need to add a new entry to ID_TYPES dictionary. However, scholarly articles on Wikidata typically use the DOI to point to the peer-reviewed article, and a preprint-specific property to point to the preprint describing the same paper (I know, confusing…).

I created a property proposal for “ChemRxiv ID” on Wikidata to help rectify this. You can propose a new property from this page but beware: Wikidata property maintainers are quite cautious to add new things and aren’t necessarily giving the most prompt feedback.

I had a lot of fun working on this new codebase and this blog post, and it was a reminder of the nice discussion I had with Andrew Su last year. I hope this post enables others to add support for medRxiv, Preprints.org, and other places where people are leaving their pre-prints!

Organizing the Public Data about your Research Organization

2021-01-17T18:23:00+00:00

If you’ve ever read a scientific paper, you know that the information that makes it into the author affiliations is a mess. I’m a big fan of Manubot and fully support its mission to upend the modern scientific publishing model. Like how they use structured ORCID identifiers for identifying authors in manuscript metadata, they are also working towards using ROR identifiers for organizations. There are still a few growing pains for ROR, so I chimed in on a discussion on GitHub about how Wikidata might be a potential solution for organizing and retrieving information about reserach organizations. I said I’d describe my idea more in detail, so here I go!

Step 1: Wikidata

Wikidata is an open, community-curated platform of knowledge. It stores entities, their relations to other entities, their relations to scalar values, and added context for each relationship. Typically, relationships have a subject, relation, and object and can be read like a simple sentence in the english language.

There are lots of working groups that maintain its ontology (i.e., the rules for how curation should be done) around certain domains, such representing organization structures. This means there are lots of tools already built in to Wikidata for potential curators like you and me to create rich pages for their organizations.

One of the curation rules shared across all domains in Wikidata is that each entity should have a “type”. This means on the page for Albert Einstein, there is a relationship stating he is an instance of a human. The “instance of” item on Wikidata is a special kind called a “property” and is one of the places where the ontology lives - there are specific rules for each property on how it should be used in a relationship, like what’s allowed to be the subject and what’s allowed to be the object. For”instance of”, there are no rules about the subject. However, the object of the relationship where “instance of” is the property should be a “class” of thing. It wouldn’t make sense for the type of another entity to be an instance of “Albert Einstein”.

Ontology for Organizations

When making a Wikidata page for an organization, it should have a type of either organization or one of its subclasses representing various kinds of organizations, parts of organizations, or combinations of subclasses and parts. Some examples are:

university research group like the Laboratory of Systems Pharmacology
research institute like the Fraunhofer SCAI
department like Fraunhofer SCAI Department of Bioinformatics
university like Northeastern University
faculty like the Maastricht Faculty of Health, Medicine and Life Sciences
academic department like Maastricht University Department of Bioinformatics (BiGCaT)
business like Pfizer

Typically, organizations have certain pieces of information associated with them using the following properties:

short name - is there a short version of your organization’s name? BiGCaT is again a good example.
country - in which country is your organization based? Wikidata already has items for countries, so the autocomplete will help you fill out this very easily.
official website - what is the URL of the homepage for your organization?
part of and has part - what are the parent organization(s) and child organizations? If your item is a department, it is likely “part of” a school, faculty, college, university, or research institution. It’s best to be as specific as possible. If you are describing a department that is part of a faculty in a university (see the BiGCaT example above), it doesn’t make sense to be redundant and also write that it is a part of the University.
field of work - what topics do your organization work on / research? This can be things like cancer, machine learning, etc.

Leadership and organizer links:

chairperson - who is the leader of the organization? This person should themselves have a Wikidata page. For a department, this is equivalent to the department chair/head.
chief executive officer - who is the CEO of a business?
founded by - who founded a business?
director / manager - who is the PI of the lab?

External Account Links:

Tutorial

Create a new item. You don’t need to have a Wikidata account or be logged in, but there are lots of benefits, so that’s highly suggested through this portal.
Input the name for the item. If you’re making a page for the Northeastern Department of Chemistry, it makes most sense to include the name of the parent organization inside the label for the item. There are some instances where this isn’t the case, such as the example above of the Maastricht Faculty of Health, Medicine and Life Sciences, but this could cause confusion. The rest of the form is pretty obvious, but if you aren’t sure what to entry for the description, writing what kind of thing it is might be best. For the Northeastern Department of Chemistry, one might write “academic department”
You’ll see a mostly blank page. Start by clicking the “+ add statement” link in the middle to bring up the following box. On the left where “Property” is greyed out, you can type the name of the relationship and click the right suggestion from the dropdown
In the middle, you can type the object of the relationship. If the property accepts another Wikidata item, it will show an autocomplete field. If the property accepts a scalar (like an identifier), then no dropdown will pop up.
After checking what you’ve typed is correct, click publish! Don’t worry, all items can be edited/updated later.
Repeat for all information you know about the organization. It’s a bit high-minded, but you can also follow the links for the properties mentioned above to see the rules they define for how they should be used. The cool thing about ontologies is that properties can actually be the subject of relationships that describe how they work using “meta-properties” 🤯.

If you’ve got any suggestions for improving this tutorial, feel free to get in touch with me (contact information at the bottom of this post) or make a PR to this page directly on GitHub!

Step 2: Research Organization Registry

The Research Organization Registry (ROR) is an organization dedicated to assigning unique identifiers to all research organizations. Definitely check out their homepage, I couldn’t write it any better than they did. They can accomplish the same goal that you can on Wikidata, but there’s a clear advantage towards having nomenclature authorities that manually curate and maintain data.

Their curation page has a link to their request form where you can send information about your organization to them. Even better, they accept Wikidata identifiers, so you don’t have to type everything all over again! I personally filled out the form and had some very nice email exchanges later with their team, so I hope you have a nice experience too. Eventually, they will assign your organization a ROR identifier as well as curate relationships between your organization and others.

Finally, the whole loop that started with Wikidata can be closed by copying your newly minted ROR identifier back to your organization’s Wikidata item. Wikidata has a property ROR ID whose object is the scalar identifier from ROR!

Step 3: Optional but Cool Things

Wikidata, ROR, and GRID (below) are by far the best solutions for storing scholarly metadata because of their licensing and API accessibility. Ringgold and ISNI are not good options for building a reusable infrastructure since they are both closed and paid systems, but they are still relevant (until the revolution) for some aspects of modern scientific publication.

Global Research Identifier Database

The Global Research Identifier Database (GRID) is a parallel effort that imports data from ROR on a quarterly (every three months) basis. It assigns an additional GRID identifier to an organization that is already in ROR. GRID is released about every three months via FigShare under the Creative Commons Public Domain 1.0 International license, which means anyone can use it any way they want.

Like with ROR, GRID has a Wikidata property GRID ID that links a Wikidata item to the scalar GRID identifier.

August 10, 2021 update: GRID is shutting down and getting consumed by ROR (ref).

Ringgold and ISNI

Many publishers use the Ringgold system for organization resolution (such as Manuscript Central). For a group interested in scholarly publishing (which could either be an academic or commercial organization), you can simultaneously apply for a Ringgold and ISNI by emailing isni@ringgold.com (copied from their FAQ).

These can be linked to a Wikidata item with the Ringgold ID and ISNI properties.

Scholia

Once your Wikidata page is full of information, you can use Scholia to visualize all the information attached to your organization through Wikidata. For example, see the University of Cambridge. To get the most of this, you should annotate for each member of your organization that they are affiliated, have been educated at, or have been employed at your organization, whether it’s current or past - Wikidata has additional metadata for a relationship for representing the time period during which it was true.

Scholia shows off how easy it is to build a system on top of the data in Wikidata and the (free!) SPARQL infrastructure they provide for accessing the data however you want. This is the original reason for me bringing up Wikidata in the Manubot issue tracker. It would be totally reasonable to hit their SPARQL endpoint with one query per Manubot build and continue being good open source citizens.

This blog post might have gone off on a couple tangents besides what was relevant for Manubot, but because I did all of these things to set up the internet presence of my previous employer, it could serve as a good guide for any new organizations to get up to speed.

See also the newer post on organizing all of the public data about a researcher, like you!

How to Code with Me - Wrapping a Flask App in a CLI

2021-01-11T00:14:00+00:00

Previous posts in my “How to Code with Me” series have addressed packaging python code and setting up a command line interface (CLI) using click. This post is about how to do this when your Python code is running a web application made with Flask and how to set it up to run through your CLI.

The name of the package I’ll be referring to in this tutorial is granola_explosion (not a real package!) that follows the src/ layout. If you’re not familiar with this, check my previous post on organizing a Python package.

Example Flask Application

Let’s assume that the Flask application is in a module called granola_explosion.wsgi located at src/granola_explosion/wsgi.py. This tutorial isn’t about building a Flask application, so below I’ll give a minimum working example. Your Flask application may be much larger, even spanning multiple files. The important thing is that the flask.Flask instance is living in this file.

# wsgi.py

from flask import Flask

app = Flask(__name__)

@app.route()
def home():
    return "There's no place like home."

if __name__ == '__main__':
    app.run()

Note that the app.run() is enclosed in if __name__ == '__main__'. This means that the app only gets run if the granola_explosion.wsgi is run as a script. Later, we will be importing this module inside our CLI, and we don’t want it to run until we tell it to (and with our very own options).

Run a Flask Web Application with Click

Let’s also assume your command line interface is in a module called granola_explosion.cli located at src/granola_explosion/cli.py using a click.Group to organize several subcommands. The following example shows how you can import the app object and run it from inside the command line.

# cli.py
import click


@click.group()
def main():
    """Run the Granola Explosion CLI."""


@click.command()
def web():
    from .wsgi import app
    app.run()


if __name__ == '__main__':
    main()

Now, you can run your web application with python -m granola_explosion.cli web!

You can actually call your module and flask.Flask instance whatever you want, but these two are pretty standard and recognized by external tools (more on that later), and will make it easier for other people to understand what your code does.

Configure Your Application

Normally, you can pass options like host and port into the flask.Flask.run() function. Below, we use click options to pass these through.

# cli.py
import click


@click.group()
def main():
    """Run the Granola Explosion CLI."""


@click.command()
@click.option('--host', default='0.0.0.0')
@click.option('--port', default=5000, type=int)
def web(host: str, port: int):
    from .wsgi import app
    app.run(host=host, port=port)


if __name__ == '__main__':
    main()

I’ve written these options so many times, that I made a package called more_click that holds them for easy importing like in the following:

# cli.py
import click
from more_click import host_option, port_option


@click.group()
def main():
    """Run the Granola Explosion CLI."""


@main.command()
@host_option
@port_option
def web(host: str, port: str):
    from .wsgi import app
    app.run(host=host, port=port)


if __name__ == '__main__':
    main()

Using GUnicorn

The flask.Flask.run() function is convenient, but it’s only meant to be a lightweight development server - even your own server yells at you every time you start it!

The official documentation and many excellent tutorials point to using more powerful servers like Gunicorn, but they start throwing around the (scary) acronym WSGI and tend to have very dense documentation that looks like it’s written only for sysadmins.

If you’re like me, you’re a big fan of keeping as much code in Python as possible, rather than floating around in various shell scripts and dockerfiles. You would also probably like to be able to run a Flask app with Gunicorn with the ease of app.run().

It turns out that gunicorn is actually written in Python, and this is possible if you’re willing to read through the codebase and understand the complicated design patterns they use. Or, you could use the more_click.run_app, which takes care of all of it for you. The implementation of this function lives here, for the adventurous among you. It’s basically a drop-in replacement for app.run() except it’s called as run_app(app). Then, you can use the with_gunicorn keyword argument to turn on using Gunicorn.

# cli.py
import click
from more_click import host_option, port_option, run_app


@click.group()
def main():
    """Run the Granola Explosion CLI."""


@main.command()
@host_option
@port_option
def web(host: str, port: str):
    from .wsgi import app
    run_app(app=app, with_gunicorn=True, host=host, port=port)


if __name__ == '__main__':
    main()

Now, your app runs with Gunicorn! If you want to be able to quickly switch back and forth between Flask and Gunicorn as a server, you can use the handy more_click.with_gunicorn_option. Further, you can specify the number of workers for your Gunicorn server based on the following complete example:

# cli.py
import click
from more_click import host_option, port_option, with_gunicorn_option, workers_option, run_app


@click.group()
def main():
    """Run the Granola Explosion CLI."""


@main.command()
@host_option
@port_option
@with_gunicorn_option
@workers_option
def web(host: str, port: str, with_gunicorn: bool, workers: int):
    from .wsgi import app
    run_app(app=app, with_gunicorn=with_gunicorn, host=host, port=port, workers=workers)


if __name__ == '__main__':
    main()

Ultimate Lazy Mode

For ultimate lazy mode, I’ve written a wrapper around the complete example in more_click.make_web_command. This uses a standard wsgi-style string to locate the app. While this is a little less explicit than normal Python code that relies on the import machinery, it has the benefit that it can lazily import the module in which your Flask application lives. This could help avoid importing big requirements, as well as allow your package to specify Flask requirements as optional. You might want to do this if your package can be used to perform a service locally, but also contains a Flask application that wraps it with a RESTful service as well that not all users might need.

# cli.py
import click
from more_click import make_web_command


@click.group()
def main():
    """My awesome CLI."""


make_web_command('my_package_name.wsgi:app', group=main)

if __name__ == '__main__':
    main()

The make_web_command() function actually returns the command itself, so you can save it and add it to the group manually instead of passing the group argument.

# cli.py
import click
from more_click import make_web_command


@click.group()
def main():
    """My awesome CLI."""


web = make_web_command('my_package_name.wsgi:app')

main.add_command(web)

if __name__ == '__main__':
    main()

Since any click command can be run by itself directly, the following minimal CLI also works well for apps that don’t need the click Group.

# cli.py
from more_click import make_web_command

web = make_web_command('granola_explosion.wsgi:app')

if __name__ == '__main__':
    web()

I intentionally did not cover the built-in Flask Script because it doesn’t fit in with my paradigm of “everything must be packaged.”

This is my first post of 2021! Happy new year!

Pathway Relationships

2020-12-30T08:00:00+00:00

Domingo-Fernandez et al. published ComPath: An ecosystem for exploring, analyzing, and curating mappings across pathway databases. in 2018 describing the overlap between human pathways in KEGG, Reactome, and WikiPathways. A lot of the underlying machinery I developed to support this project has been improved since, and it’s time to spread the search to other organisms besides humans and other databases. This blog post is about some additional relation types needed to capture the relations between pathways appearing in these databases.

Like many of my blog posts, this one was inspired by a tweet. After the following discussion, I thought it would be good to better organize the ideas and elaborate.

#KEGG provides a namespace of non-species-specific pathway terms that all of its species-specific pathways can map to. @reactome could you provide the same?
— Charles Tapley Hoyt (@cthoyt) December 30, 2020

This blog post will follow apoptosis, one of the most ubiquitous pathways in biology that covers all manners of programed cell death. This blog post isn’t about the nitty-gritty difference between pathways, biological processes, and mechanisms - so we will consider all variants of apoptosis and apoptotic process effectively the same.

Resource	Prefix	Identifier
Gene Ontology (GO)	go	GO:0006915
Medical Subject Headings (MeSH)	mesh	D017209
Kyoto Encyclopedia of Genes and Genomes (KEGG)	kegg.pathway	map04210
NCI Thesaurus (NCIT)	ncit	C17557

KEGG, Reactome, and WikiPathways all provide human-specific variants of these pathways (below) as well as many other species, including both model organisms and not.

Resource	Prefix	Identifier
KEGG	kegg.pathway	hsa04210
Reactome	reactome	R-HSA-109581
WikiPathways	wikipathways	WP254

Pathways are Equivalent

Two pathways are equivalent and can be represented with skos:exactMatch if they both have the same species specificity. The following relationships are between the non-species specific pathways for apoptosis:

Subject	Predicate	Object
GO:0006915	skos:exactMatch	mesh:D017209
GO:0006915	skos:exactMatch	kegg.pathway:map04210
mesh:D017209	skos:exactMatch	kegg.pathway:map04210
…	…	…

The following relationships are between the human-specific pathways for apoptosis in KEGG, Reactome, and WikiPathways:

Subject	Predicate	Object
kegg.pathway:hsa04210	skos:exactMatch	reactome:R-HSA-109581
kegg.pathway:hsa04210	skos:exactMatch	wikipathways:WP254
wikipathways:WP254	skos:exactMatch	reactome:R-HSA-109581

Similarly, the relationships between cow-specific (Bos Taurus; BTA) pathways for apoptosis in KEGG, Reactome, and WikiPathways:

Subject	Predicate	Object
kegg.pathway:bta04210	skos:exactMatch	reactome:R-BTA-109581
kegg.pathway:bta04210	skos:exactMatch	wikipathways:WP1018
wikipathways:WP1018	skos:exactMatch	reactome:R-BTA-109581

While equivalences begins to tame the ontology of pathways, it is missing links between the GO, MeSH, and NCIT terms to Reactome and WikiPathways.

Species-Specific Variant of a Pathway

GO, MeSH, NCIT, and many other nomenclatures do not contain species-specific variants of their pathways. However, KEGG contains both a parent pathway, prefixed with map and species-specific pathway, prefixed with their internal 3 or 4-letter species code.

Subject	Predicate	Object
kegg.pathway:hsa04210	speciesSpecific	kegg.pathway:map04210
kegg.pathway:bta04210	speciesSpecific	kegg.pathway:map04210
…	…	…

It should generally hold that when X speciesSpecific Y and Y skos:exactMatch Z are true, X speciesSpecific Z. This allows KEGG to serve as a bridge between the species-specific and non-species-specific pathway worlds. However, Domingo-Fernandez et al. showed that there are huge discrepancies between KEGG, Reactome, and WikiPathways, so there is still need to curate/infer the same kinds relationships in Reactome and WikiPathways.

Unfortunately, Reactome and WikiPathways do not (yet) have parent terms for non-species-specific pathways. Asking about this was the point of the tweet that inspired this blog post. Because Reactome uses a standardized nomenclature where all variants of each pathway across species have the same numerical part to their identifier (e.g., R-HSA-109581 and R-BTA-109581), they could institute a similar parent nomenclature like KEGG’s. WikiPathways identifiers do not have this sort of regularity, but they have the benefit of being highly receptive to external input and improvements.

Side bar: I’ve seen an elegant solution for this in OBO that defines child terms with an intersection of the Relation Ontology relation RO:0002160 (only in taxon) to a given species and the parent term, but this is an unnecessarily complicated alternative for the goal of representing the relation between two entities.

Pathways are Orthologs

Two genes with similar evolutionary history and function appearing in two organisms are called orthologs. Orthology is incredibly important for studying biology because it allows us to make inferences about how human biology works by studying model organisms like mice and rats. There are several databases collecting orthology relationships, such as HomoloGene.

It follows that orthology could be applied to pathways as well. In fact, Reactome’s web interface already has a box below each pathway linking to the orthologous pathways as seen on https://reactome.org/content/detail/R-HSA-109581:

However, this information is not programatically available (AFAIK), and it is not available for other databases like WikiPathways and KEGG. Therefore, we can introduce a relationship orthology to start curating triples like:

Subject	Predicate	Object
kegg.pathway:hsa04210	orthology	kegg.pathway:bta04210

Orthology relationships effectively convey the same information as speciesSpecific with the advantage that they do not require the addition of a parent term. However, between N orthologous pathways, there will be a complete subggraph of (1/2) _ N _ (N-1) edges (also called a clique in graph theory). Depending on the downstream use case, these kinds of subgraphs can be problematic.

Because kegg.pathway:hsa04210 skos:exactMatch reactome:R-HSA-109581, we can infer reactome:R-HSA-109581 orthology kegg.pathway:bta04210. However, I think it would be best to only curate orthology relationships within a given database because it will increase the size (N) of the clique.

Pathway is About a Concept

KEGG, Reactome, and WikiPathways not only include pathways, but also other “maps” about specific topics such as diseases, families of proteins, and other biological entities.

For example, KEGG has an entry kegg.pathway:hsa05010 entitled Alzheimer disease - Homo sapiens (human). When using Gilda to generate lexical matchings, the MeSH entry mesh:D000544 (Alzheimer Disease) appeared highly ranked. However, KEGG’s notion of pathway and MeSH’s notion of a disease are not the same, and these two terms should not be considered equivalent. For this case, not only KEGG but also Reactome and WikiPathways, we can introduce a new relationship pathwayAbout. It turns out that WikiPathways also has an Alzheimer’s disease “pathway” as well.

Subject	Predicate	Object
kegg.pathway:hsa05010	pathwayAbout	mesh:D000544
wikipathways:WP2059	pathwayAbout	mesh:D000544

Note that KEGG and WikiPathways both have specificity in their pathways for organisms, but diseases in MeSH and other nomenclatures aren’t typically stratified by their target organisms. Therefore, the mouse-specific Alzheimer’s disease pathway in WikiPathways (wikipathways:WP2075) could also have the same relationship.

Another example is opsins - a family of light-sensitive proteins. Reactome has a pathway reactome:R-HSA-419771 (Opsins) that is not the same as the MeSH entry mesh:D055355 (Opsins) describing the protein family.

There is specific interest in connecting disease maps appearing in pathway databases to the diseases themselves. WikiPathways has already begun doing this as can be seen on https://www.wikipathways.org/index.php/Pathway:WP2059.

It might be justified to propose an alternate relationship with more specific semantics. More information on various disease-specific curation projects outside major pathway databases can be found at https://disease-maps.org.

Disease-specific Variant of a Pathway

This is a bit of an afterthought, but it might be mentioning that there are places, like NeuroMMSig, that curate disease-specific variants of pathways. These would need their own dedicated relationships to connect to the “canonical” pathway and to the disease that they describe.

I have to give a huge shout-out to Daniel Domingo-Fernández, Josep Marín-Llaó, Carlos Bobis-Álvarez, and Yojana Gadiya who have done the curation in the ComPath project as well as Ben Gyori who laid the groundwork for improving the lexical mappings with the Gilda software as well as contributed tons of curations for MeSH-GO mappings.

There are still many disjoint resources that need normalization, including the Pathway Ontology, which looks to have lots of information. I’ll be working on it via this GitHub issue.

There’s also PathBank, (curated by Yojana but not appearing in the original ComPath publication), BioCyc, MetaCyc, and many others. Each must first be included in PyOBO as I described in a previous post before getting into curation, so I can reuse all the code.

I’m not super happy with any of the names I’ve given to relationships in this post, either. I’m open to suggestion for improvement. We alternatively discussed using skos:broader and skos:narrower, as well. Further, I’d love to see these kinds of relationships appear in the Relation Ontology itself, but unfortunately I have not been super successful in petitioning them for improvements in the past, so I may start another open ontology project focused on relationships themselves.

This is all part of a greater effort, Biomappings, which Ben and I have been working on to make it much easier to curate equivalences and related mappings. I’ll have more to say about that in a future post.

Making DrugBank Reproducible

2020-12-14T08:00:00+00:00

If you’re reading my blog, there’s a pretty high chance you’ve used DrugBank, a database of drug-target interactions, drug-drug interactions, and other high-granularity information about clinically-studied chemicals. DrugBank has two major problems, though: its data are password-protected, and its license does not allow redistribution. Time to solve these problems once and for all.

I’d guess that most of the thousands of people who use DrugBank have run into the same issue as me to get the data: you have to navigate through the DrugBank site, create an account, log in, then download the data manually. There are some hints on the site on how this can be done through the shell, but I think it’s the unfortunate case that many people using bioinformatics resources just aren’t comfortable using the shell. Further, the file is zipped, which means that unzipping it requires further knowledge of the arcane arts of which shell programs and which flags to use with them.

Even after getting past this, users need to write their own programs that read the XML content of the file, extract the relevant parts, and put it somewhere for later. If you’re like me, you probably want to have as little to do with data in the XML format as possible, and you probably deleted the original zip archive along with the XML file after you got the data out of it that you actually wanted.

The trouble with the scenario described in the last two paragraphs is that any analysis done on the resulting files required lots of manual steps in the middle. These steps can’t be automated, and therefore the downstream analysis can’t be automated either. It’s highly likely for a research to have the time or motivation to do these steps exactly the same way as described in a paper, if it’s even described at all.

Because of DrugBank’s licensing rules, you can’t redistribute the data in the format that is necessary for downstream use either.

This is no small problem - hundreds of papers cite DrugBank per year. The hype of drug repositioning for COVID-19 no doubt helped in 2020, with 147 citations as of the time of writing this post on December 14th, 2020.

Search conducted on December 14th, 2020 with https://pubmed.ncbi.nlm.nih.gov/?term=drugbank.

It’s difficult to tell which, if any, of these efforts are meaningful. Without the ability to reproduce the steps taken to do analyses based on DrugBank, we can not even begin to evaluate these papers. Even further - imagine the intense pressure on the few famous scientists in network-based drug repositioning who receive the most requests to review more papers. How could we possibly expect these scientists to provide thorough reviews if it’s not easy (or possible, for that matter) for them to re-produce the analyses?

The Solution

The first time I put to writing the need to automate the acquisition of data was in my (still not accepted through peer review) article on the reproducible acquisition and conversion a wide range of biological databases into Biological Expression Language. In this article, we presented a suite of 50+ scripts which each downloaded, converted, and exported a biological database like DrugBank, GWAS Catalog, HIPPIE, and others into a common schema for networks biology downstream applications like signal transduction pathway analysis, knowledge graph embedding, and other methods. Of all of them, DrugBank remained the most vexing.

It’s time for a solution - for code that can help get us out of this rut. This code needs to be able to download the data from DrugBank, open it up, and help transform it all without asking the user to intervene too much.

Enter drugbank_downloader. This is a Python library with three functions exactly for this. First, you have to install it with:

$ pip install drugbank-downloader

Download A Specific Version

If you want your analysis to be reproducible, you should specify what version of the database you want to use. Luckily, DrugBank does keep an archive of old versions. The following code describes how to use the drugbank_downloader.download_drugbank() function, which asks for a version, a DrugBank username, and the corresponding DrugBank password. You’re free to get the username and password into the script however you want - usually loading from the environment using os.getenv() is a good choice. Hard-coding them is not so much, because it could potentially violate the terms and conditions of use of DrugBank, and just isn’t good practice in general, especially if you reuse passwords. Get LastPass.

import os
from drugbank_downloader import download_drugbank

username = ...  # suggestion: load from environment with os.getenv('DRUGBANK_USERNAME')
password = ...

path = download_drugbank(version='5.1.7', username=username, password=password)

# This is where it gets downloaded: ~/.data/drugbank/5.1.7/full database.xml.zip
expected_path = os.path.join(os.path.expanduser('~'), '.data', 'drugbank', '5.1.7', 'full database.xml.zip')
assert expected_path == path.as_posix()

This script takes care of downloading the data and storing it at ~/.data/drugbank/5.1.7/full database.xml.zip. After it’s been downloaded once, the function is smart and doesn’t need to download again. This is all taken care of by the pystow behind the scenes. If you specify an alternate version, it’s pretty obvious where in the filepath it changes.

Download the Latest Version

If you want to automate running your code on the most recent version of DrugBank, you can use the same function, but you’ll first have to install bioversions with pip install bioversions. This package’s job is to look up the most recent version of varions biological databases. I’ll have a whole post on that later. Then, you can modify the previous code slightly by simply removing the version argument.

import os
from drugbank_downloader import download_drugbank

username = ...  # suggestion: load from environment with os.getenv('DRUGBANK_USERNAME')
password = ...

path = download_drugbank(username=username, password=password)

# This is where it gets downloaded: ~/.data/drugbank/5.1.7/full database.xml.zip based on the latest
# version, as of December 14th, 2020.
expected_path = os.path.join(os.path.expanduser('~'), '.data', 'drugbank', '5.1.7', 'full database.xml.zip')
assert expected_path == path.as_posix()

Don’t Bother Unpacking - read `full database.xml.zip` Directly

Like I said before, it’s not enough just to get the file. It needs to be unzipped and opened. If you’re not familiar with doing this in python, you can use the following code:

import zipfile
from drugbank_downloader import download_drugbank

path = download_drugbank(username=..., password=...)

with zipfile.ZipFile(path) as zip_file:
    with zip_file.open('full database.xml') as file:
        pass  # do something with the file

You don’t have time to remember this. Just use drugbank_downloader.open_drugbank() instead:

from drugbank_downloader import open_drugbank

with open_drugbank(username=..., password=...) as file:
    pass  # do something with the file, same as above

There’s some magic using contextlib to make this work with the with statement. Just keep reading, though…

Reading Drugbank’s XML

After you’ve opened the file, you probably want to read it with an XML parser like:

from xml.etree import ElementTree
from drugbank_downloader import open_drugbank

with open_drugbank(username=..., password=...) as file:
    tree = ElementTree.parse(file)
root = tree.getroot()

You don’t have time to remember this either. Just use drugbank_downloader.parse_drugbank() instead:

from drugbank_downloader import parse_drugbank

root = parse_drugbank(username=..., password=...)

DrugBank: Endgame

Once you’ve got the XML python object, you can write all of the code to extract the parts you want, load them in a pandas.DataFrame, or do whatever you’d like. If these steps are slow, you could even write code that caches it in the middle. Below is my template for an expensive processing step.

from drugbank_downloader import parse_drugbank

def process_data():
    ...

def save_my_parsing_results(processed_data, file):
    ...

def load_my_parsing_results(path):
    ...

def get_processed_data():
    cache_path = ...
    if os.path.exists(cache_path):
        processed_data = load_my_parsing_results(cache_path)
    else:
        root = parse_drugbank(username=..., password=...)
        processed_data = process_data(root)
        with open(cache_path, 'w') as file:
            save_my_parsing_results(root, file)
    return processed_data

I understand that most scientists aren’t trained as software engineers, nor are they incentivized to write code that’s reproducible, but I at least hope this code helps cut a few non-reproducible steps out of your science. Happy drug hunting!

Scoring Inverse Triples

2020-12-11T08:00:00+00:00

When training a knowledge graph embedding model with inverse triples, two scores are learned for every triple (h, r, t) - one for the original and one for the inverse triple (t, r', h). This blog post is about investigating when/why there might be meaningful differences between those scores depending on the dataset, model, and training assumption.

In PyKEEN, each interaction model (e.g., TransE, ConvE, Tucker) inherits from the base model pykeen.models.Model and implements the scoring methods Model.score_h(), Model.score_r(), Model.score_t(), and Model.score_hrt(). The last, score_hrt() takes in a triple or sequence of triples and calculates a real-valued score for each. This is quite efficient in batch because of the way most models are implemented with matrix multiplication in PyTorch. The remaining three take pairs:

score_h() takes relation/tail pairs and provides scores across all possible tails
score_r() takes head/tail pairs and provides scores across all possible relations
score_t() takes head/relation pair and provides scores across all possible heads

Each of these functions are used slightly differently during the training and inference workflows. One of the reviewers for the PyKEEN software paper pointed out that the while Model.predict_scores_all_tails() and Model.predict_scores_all_heads() inference workflows allow the user to choose if the forward triples or inverse triples are used for predictions, that this functionality is not exposed to the user for use in further study.

Mehdi (@mali-git; he’s a really good guy) began pykeen/pykeen#208 refactoring the implementation of the head prediction and tail prediction workflows in the pykeen.model.Model base model class that better abstracted the operations for generating inverse triples, and how they are applied in each inference workflow.

I often serve the role of project manager for PyKEEN, so I was initially a bit skeptical of adding new functionality without some good examples of what a user might do with it. After a bit of discussion, Mehdi suggested we could showcase a comparison the distributions of forward and inverse triples scores and provided a minimal example. While he was busy actually working on the implementation, I took his example to the extreme and created an entire experimental setup for this investigation to accompany the new implementation.

Comparison of Distributions

Medhi provided a new implementation of Model.score_hrt_inverse for exactly this kind of investigation. The following code can be used to get the scores for the testing triples from the Nations dataset.

from pykeen.datasets import Nations
from pykeen.pipeline import pipeline

nations = Nations()
testing_triples = nations.testing.mapped_triples

res = pipeline(
    dataset=nations,
    model='RotatE',
    training_loop='LCWA',
    training_kwargs=dict(num_epochs=60),
)

# Score with original triples
scores_forward = res.model.score_hrt(testing_triples)

# Score with inverse triples
scores_inverse = res.model.score_hrt_inverse(testing_triples)

These results are both PyTorch tensors, so don’t forget to use scores_*.detach().numpy() for use in your own comparisons and visualizations. The first thing I did was plotted the distributions of both with seaborn.histplot.

The following two plots show that sometimes the distributions match up pretty well, and sometimes they don’t. The distributions are shown normalized to make it easier to compare - the shapes are more important than the actual counts because the number of testing triples changes from dataset to dataset. Sorry about the inconsistency in bin widths - I’ve tried a lot and haven’t figured it out. If you know the solution, please let me know (or make a PR to PyKEEN for ultimate PR cred).

Nations	Kinships

This was just one comparison for the same model/training assumption. In order to make it possible to look at many facets at the same time, the data need to be simplified. Because the scores have a pairwise correspondence (the same triple was scored two ways), it makes sense to plot distribution of residuals (the forward score minus the inverse score). Here’s the same plots simplified with residuals:

Nations	Kinships

The PyKEEN core team is a very special place - Max, Mehdi, and Laurent are all excellent machine learning researchers and scientists. The part where I excel is in organization and software development, so I’ve taken care of aspects like the pipeline(), visualization, and data science aspects of our work. Naturally, I gravitate towards any tasks that can be solved with super deep loops. So without further ado, below are presented the results from the script that runs this experiment for all datasets, models, and training assumptions. It assigns row by dataset, column by model, and color by training assumption. Note: the score axes will not be share because each model produces different ranges of scores with different density functions. This itself is also an interesting and upcoming avenue of research!

While the script that generated this chart is extensible to all datasets and models, I haven’t got the computational power available today to run them all. All that needs to be done is to add a few more strings to the lists of models, or even better, replace them with the python entrypoint-loaded classes. However, with many experiments, I think the small datasets like Nations and Kinships are sufficient to demonstrate a given phenomena.

There are a few major insights to be taken from this chart. First, the RotatE, TransE, and DistMult model seem to operate pretty similarly for forwards and inverse triples. This is demonstrated by two aspects of the distributions - their normality and tightness (low standard deviation). When considering that each of these are translational distance models, this should come as no surprise.

ComplEx shows neither normality in its distributions nor tightness. This is because ComplEx’s interaction function is explicitly supposed to model the directionality of triples through its use of complex-valued entity embeddings and the use of the non-commutative multiplication operation on complex numbers.

SimplE is an interesting model to consider when studying the difference between forward and inverse triples because it explicitly models the forward and inverse of a given triple then takes the average for its final score. Its strange behavior based on the difference between the stochastic local closed-world assumption (sLCWA) and local closed-word assumption (LCWA) during training could be due to the fact that it is a complete graph (e.g, all pairs of entities have a triple in both directions).

There are definitely more insights and more satisfying explanations to be gained by looking at a larger-scale analysis, but this shows that very quickly, with the artifacts of training and very little time, insight can be gleamed into the possible drawbacks for inference for a given dataset-model-training assumption combination.

Generating Testing Knowledge Graphs with Literals

2020-12-07T08:00:00+00:00

PyKEEN has a wide variety of functionality related to knowledge graph embedding models and handling various sources of knowledge graphs. This post describes the journey towards properly testing the functionality of an exotic set of knowledge graph embedding models that incorporate feature vectors for entities via triples with numeric literals.

PyKEEN implements the LiteralE class of knowledge graph embedding models that allow for the inclusion of numeric features for each entity. It then combines these features with the embedding for each entity using the g function before calculating a score using an arbitrary interaction function. In the original paper, Kristiadi et al. used the interaction functions from the DistMult and ComplEx.

Unfortunately, the original paper used three datasets for benchmarking: FB15k-237, WN18RR, and YAGO3-10. While these aren’t the largest datasets PyKEEN can handle, they are too big to be used during unit testing, which should run very quickly and be as minimal as possible. Therefore, we punted writing proper tests for LiteralE models because of the lack of availability of a small, well-studied dataset like Nations or Kinships.

Today, that came to bite us in the butt because we decided to improve the implementation of the underlying interface for loading triples from files (see pykeen/pykeen#193). That meant that it was time to set off on creating a dataset that was small and had literals in it and implementing proper tests for the LiteralE models.

My first idea was to take an arbitrary dataset, like Nations, and generate random features for it. I implemented an algorithm that created (slightly meaningful) random representations, but then @mberr gave the excellent suggestion that the CIA’s World FactBook contains all sorts of information for each country in tabular form, and that I could use that to extend the Nations dataset to have literals. I did the dirty work of grabbing some features (area, population, etc.) for the ten entities in the Nations dataset then a bit of improvement on the underlying PyKEEN code for dataset loading. It all come to fruition in pykeen/pykeen#199 where @mberr helped update the LiteralE implementations, but he rightfully pointed out that my random dataset generation code was no longer necessary and didn’t belong in the pull request.

Luckily, I have a blog, which seems like a great place to share my code and my thought process!

Generating Random Literals

The algorithm was the following:

Pick a pre-existing dataset
Train a knowledge graph embedding model on the dataset
Generate a random transformation matrix that can be multiplied by each entity’s embedding to generate a feature
Apply a bit of noise
Assign each feature a random name and write to a file whose rows are in the form of

While it would have been possible to directly generate random values for each dummy feature, using a transformation on the entity embeddings means that each feature will have some correlation to the embeddings itself, which could be useful for testing later.

I’ve posted the code to Gist in case we ever need it again. Please feel free to reuse, this code is under the MIT License.

Referring to SARS-CoV-2 Proteins in BEL

2020-09-17T00:05:00+00:00

Many of the proteins in the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are cleavage products of the replicase polyprotein 1ab (uniprot:P0DTD1). Unfortunately, the bioinformatics community is not so comfortable with proteins like this and nomenclature remains tricky. Luckily, the Biological Expression Language (BEL) has exactly the right tool to encode information about these proteins using the fragment() function.

This image was modified from the C&EN article What do we know about the novel coronavirus’s 29 proteins?

UniProt lists each of the 16 non-structural proteins (often written as symbols nsp1-nsp16) as protein chains of the main protein entry, uniprot:P0DTD1. These chains are assigned identifiers following the regular expression pattern of PRO_\d{10}. The Identifiers.org registered this pattern under the prefix uniprot.chain. While it resolves to URLs following the pattern of https://www.uniprot.org/uniprot/#, it appears that the parent protein’s UniProt identifier is looked up automatically . This is really good news and means that we can start using stable CURIEs to identify these proteins, even if like me, you’ve never used this prefix before.

Alternatively, BEL allows you to write out the relationship between the parent protein and the fragment using the fragment() / frag() function (docs). For example, the nsp1 fragment from position 1-180 can be written in BEL either as p(uniprot.chain:PRO_0000449619) or as a fragment p(uniprot:P0DTD1 ! R1AB_SARS2, frag(1_180)). The entire table of non-structural proteins is written out below for your copy/paste convenience in BEL coding.

Symbol	Chain	Positions	Name	BEL
nsp1	PRO_0000449619	1 – 180	Host translation inhibitor nsp1	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(1_180))`
nsp2	PRO_0000449620	181 – 818	Non-structural protein 2	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(181_818))`
nsp3	PRO_0000449621	819 – 2763	Non-structural protein 3	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(819_2763))`
nsp4	PRO_0000449622	2764 – 3263	Non-structural protein 4	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(2764_3263))`
nsp5	PRO_0000449623	3264 – 3569	3C-like proteinase	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(3264_3569))`
nsp6	PRO_0000449624	3570 – 3859	Non-structural protein 6	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(3570_3859))`
nsp7	PRO_0000449625	3860 – 3942	Non-structural protein 7	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(3860_3942))`
nsp8	PRO_0000449626	3943 – 4140	Non-structural protein 8	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(3943_4140))`
nsp9	PRO_0000449627	4141 – 4253	Non-structural protein 9	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(4141_4253))`
nsp10	PRO_0000449628	4254 – 4392	Non-structural protein 10	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(4254_4392))`
nsp12	PRO_0000449629	4393 – 5324	RNA-directed RNA polymerase	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(4393_5324))`
nsp13	PRO_0000449630	5325 – 5925	Helicase	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(5325_5925))`
nsp14	PRO_0000449631	5926 – 6452	Proofreading exoribonuclease	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(5926_6452))`
nsp15	PRO_0000449632	6453 – 6798	Uridylate-specific endoribonuclease	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(6453_6798))`
nsp16	PRO_0000449633	6799 – 7096	2’-O-methyltransferase	`p(uniprot:P0DTD1 ! R1AB_SARS2, frag(6799_7096))`

I’m not sure what happened to #11. UniProt isn’t listing it here. There’s also the Replicase polyprotein 1a, which lists nsp1-nsp11, but I’m not sure what the difference is yet.

When I first started writing this, I wasn’t actually aware of the existence of the uniprot.chain entry in Identifiers.org. This makes things a lot better! However, this leaves two tasks for me:

Integrate the uniprot.chain nomenclature into PyOBO such that identifiers can be validated and easily resolved to names

Generate equivalence relationships in BEL linking the CURIE-named and ontologically-defined versions of each as in:

p(uniprot.chain:PRO_0000449619) equivalentTo p(uniprot:P0DTD1 ! R1AB_SARS2, frag(1_180))
...
p(uniprot.chain:PRO_0000449633) equivalentTo p(uniprot:P0DTD1 ! R1AB_SARS2, frag(6799_7096))

Happy BEL coding!

How to Code with Me - Making a CLI

2020-06-11T15:48:00+00:00

One of the cardinal sins in computational science is to hard code a file path in your analysis. This post is a guide to reorganizing your code to avoid this and then to generate a command line interface (CLI) using click.

The best way around this is to make all of your code live inside a function that takes a file path as an argument. Here’s an example of some sinful code:

# sinful_analysis.py
import pandas as pd

df = pd.read_csv('/Users/cthoyt/data/example.tsv')
analysis = do_analysis(df)
save_analysis(analysis, '/Users/cthoyt/data/analysis.tsv')

Here’s the same code, but enlightened:

# enlightened_analysis.py
import pandas

def do_enlightened_analysis(input_path, output_path):
    df = pd.read_csv(input_path)
    analysis = do_analysis(df)
    save_analysis(analysis, output_path)

The enlightened code doesn’t contain any references to the file paths on which you’re doing analysis. In fact, the enlightened code can’t even be run directly without passing the file paths as variables. This pattern gets you in the mindset of separating the code from the configuration for running the code. Again, this is important because the file path will change depending on who’s running it, if you decide to do spring cleaning on your hard drive, or if you get new files.

There are lots of ways you might pass the input and output paths into this function. The most obvious, since you’ve probably read my previous blog post and you’re now a packaging master, is to import enlightened_analysis and run it from the Python REPL. Another way would be to make a one-off Python script whose job is to actually run the analysis (as opposed to this example, which is creating the workflow to be run). Though this are both better than the sinful analysis, it’s a problem since you have to manually interact with Python to run your code.

You’re likely familiar with using the CLI for pip. Wouldn’t it be terrible if you had to write a Python script that calls pip (like R makes you do with install.packages(), ughhh!!). This is the same visceral reaction you should have to having to make specific python code for an analysis.

Making your first CLI

As you might have guessed, the solution is to make a CLI. After making your function that does the hard work, the job of the CLI should just take care of getting the configuration (e.g., file paths) from the user and passing them to your functions that do the hard work.

CLIs should be very very short! If you’re putting lots of logic inside your CLI, then you should probably reconsider refactoring that logic into more generally reusable functions. Here’s an example of the hello world CLI:

# cli_simple.py

if __name__ == '__main__':
    print('hello')

Then you run this python script with python cli_simple.py. It’s boring. You didn’t need to read this guide to do this. However, you might not know what if __name__ == '__main__' is. It turns out that all python files know their name and store it in the __name__ variable when they’re imported. However, if you are running a python file as a script then __name__ gets set to the string '__main__'. This allows you to make sure that the print('hello') is only ever run if the user is actually running the script as a CLI.

However, don’t be tempted to put lots of code in if __name__ == '__main__'. You should always make a function main() where all of the code that runs the CLI, and just call it.

# cli_simple_2.py

def main():
    print('hello')

if __name__ == '__main__':
    main()

You still run this script with python cli_simple_2.py. Another reason we introduced main() is to take care of getting information from the user. That’s where click comes in. It makes functions do all sorts of magical things to get information from command line arguments. All you have to do is use function decorator (something starting with the @ symbol) to annotate that the function is a click.command(). If you’re not already familiar with decorators, check this short video or this long video.

Run pip install click in your shell then update your code to look like this:

# cli_simple_3.py
import click

@click.command()
def main():
    print('hello')

if __name__ == '__main__':
    main()

You still run this script with python cli_simple_3.py and it does exactly the same as the last one. However, once your main function is a click.command(), you can do all sorts of wonderful things. The first is to pass arguments from the command line into the function. Update your code to look like the following:

# cli_simple_4.py
import click

@click.command()
@click.argument('text')
def main(text):
    print(text)

if __name__ == '__main__':
    main()

You probably notice that the call to main() at the bottom does not include an argument for text. That’s because click is decorating the original function in the meantime, which means that the actual thing called main is a function that takes no arguments in the end. This is good, because click uses the extra decorators (e.g. click.argument(...), click.option(...)) to figure out what to put in the arguments from the function that we actually wrote. Notice that the @click.argument('text') matches up to the variable name. That’s no coincidence.

You can now run this script from the command line with python cli_simple_4.py. You’ll see that it yells at you for forgetting the text argument. Better not! Try again with python cli_simple_4.py "Hello World!" and you’ll be happy to see you’re now at Hello World for CLIs. From here you can do all sorts of stuff which is all outlined in the excellent click documentation.

click also automatically generates documentation for you, so it’s always possible to run the command without arguments and with the --help flag as in python cli_simple_4.py --help. It will give you information about all of the arguments, their types, and more.

CLIs in Package World

It’s my strong opinion that almost all code should be packaged, and the CLI is no exception. To finish our original problem, we’ll create a python file cli.py in the package where enlightened_analysis.py is and import our function from there. Then we’ll add the right arguments to click, pass them to the right place, and profit!

# cli.py
import click
from .enlightened_analysis import do_enlightened_analysis

@click.command()
@click.argument('input_path')
@click.argument('output_path')
def main(input_path, output_path):
    do_enlightened_analysis(input_path, output_path)

if __name__ == '__main__':
    main()

If you’ve done it right, a the body of the main() function for your CLI should very boring function. Of course, there are other ways to organize your code, but this is a good way to do it until you’re more comfortable.

However, now we’re living in package world. In this tutorial, I’ve skipped the explanation of turning the enlightened_analysis.py and cli.py into a package. I’ll assume from here that you’ve done this and named the package superanalysis. If you’re not familiar with doing that, check my previous blog post or my tutorial on YouTube.

We don’t want to interact with this code by running it as a script with python cli.py. Instead, we want to interact with the code via the package. Further, if you cd into the place where the code is and run python cli.py, you’ll get an import warning because relative imports don’t work when you’re not in a Python package context. This error is a good thing - it’s a reminder that you should always live in the packaged world.

The solution to the problem is to use the -m flag in the python CLI. Remember that enlightened_analysis.py and cli.py modules are in a package called superanalysis (that you should have also already installed). You can now run the CLI using python -m superanalysis.cli . This is also going to set __name__ to '__main__' the same way as running it as a script, but you’re in the python package context!

Vanity is a Virtue

The -m can almost be used to run any python file inside your package as command line interface, which means you should always wrap up code for the CLI in if __name__ == '__main__' so it doesn’t accidentally get run if the module is imported.

The exception is the __init__.py files can’t be run as a module. If you were to write python -m superanalysis, it wouldn’t run the __init__.py file as a script and instead would throw an error. If you want to associate a CLI with the package , you need to make an additional file called __main__.py sitting next to cli.py in the superanalysis package. As an aside, this also works in subpackages.

# __main__.py

"""Entrypoint module, in case you use `python -m superanalysis`.

Why does this file exist, and why `__main__`? For more info, read:

 - https://www.python.org/dev/peps/pep-0338/
 - https://docs.python.org/3/using/cmdline.html#cmdoption-m
"""

from .cli import main

if __name__ == '__main__':
    main()

This python module simply reuses the main function we already wrote before. It can basically be copied verbatim from package to package, but don’t forget to change the first line to match yours! I like to copy it because it also has the information from the python docs on why it works.

Now, you can run python -m superanalysis instead of python -m superanalysis.cli. We can also do one better. Wouldn’t it be nice to make a CLI function so we could just run superanalysis ? You’re in luck, because since we grouped all of our code in a main() function, we can make a small addition to the setup.cfg’s entry points to tell pip to automatically create a superanalysis CLI in your shell by doing the following:

[options.entry_points]
console_scripts =
    superanalysis = superanalysis.cli:main

The left part is the name of the CLI that will be in your shell, then the right part has the path to a module followed by a colon : then the name of the function to be run.

That’s pretty much it! Now you can make beautiful command line interfaces. There’s one more topic that I think is worth noting at the end of this tutorial, and that’s to use command line groups. This allows you to organize subcommands in your CLI and import other CLIs from other modules. It would look like this:

# cli.py

import click

@click.group()
def main():
    pass  # becuase this is a group, you don't actually need to do anything in it

@main.command()  # note that the main function now can assign commands
def subcommand1():
    print('hello world')

@main.command()
def subcommand2():
    print('other greeting')

@main.group()
def subgroup():
    pass

@subgroup.command()
def turtle():
    print('you can go as deep as you want with subcommands')

# You can include other CLIs from other modules in your package
# to make everything much more unified
from .my_other_module.cli import main as other_module_command
main.add_command(other_module_command)

You can see that rather than using the click.command() decorator, the main() got the click.group() decorator. This means that it can be used to issue subcommands or even subgroups! At the end, it was also used to combine CLIs from another part of the same package. This is good if you have a package that does lots of things, but you want a single unified CLI to access all of it. Just be careful with how the functions are named (it doesn’t always have to be main) because if two are called the same thing then there will be a name clash and one sub-command or sub-group won’t get shown.

CLIs are really powerful! In the end, you can write in your README how you used your CLI on your data to run your experiment. This gives others the best shot at reproducing your work. Happy hunting, and see you the next installment of “How to Code with Me”!

The Curation of Neurodegeneration Supporting Ontology

2020-06-09T19:31:00+00:00

While I led the curation program in the Human Brain Pharmacome project during my Ph.D. from 2018-2019 at Fraunhofer, we built the Curation of Neurodegeneration Supporting Ontology (CONSO). This post outlines the project’s needs for quality control and re-curation that lead to its generation, the curation process, and how CONSO constitutes an example of how to follow the guidelines I proposed in a previous blog post on building ontologies.

Before I joined Fraunhofer for my master’s and Ph.D., previous group members had made first attempts at generating several narrow ontologies, terminologies, and controlled vocabularies within the field of neurodegenerative disease research which were then used as controlled vocabularies (CVs) in knowledge graphs curated for the AETIONOMY project. However, the project did not outline a maintenance plan and around late 2015, AETIONOMY’s curation program effectively ended.

Three years later (at the start of 2018), I was tasked with revitalizing the curation program as part of the Human Brain Pharmacome project. This meant assessing the quality and currentness of knowledge graphs and their underlying CVs before leading a new curation campaign to enrich them (to be described in a future blog post). The remainder of this blog post is about the issues with these underlying ontologies, how we addressed them, and a description of the resulting ontology that we generated.

Assessing Controlled Vocabularies used by NeuroMMSig

The first step was to categorize the ontologies, terminologies, and CVs that were used in the knowledge graphs (the set of which comprises NeuroMMSig (paper, website) from now on) into high and low confidence. The first criterion for high confidence was that they were either in a repository of high quality nomenclatures (either OLS or OBO Foundry) or they were registered in the MIRIAM database at Identifiers.org. The second criterion for high confidence was that they were being actively maintained in a version controlled repository like the Human Phenotype Ontology or FamPlex. Because there are a few examples of standard ontologies that are not maintained, the confidence labels help convey this distinction. As an aside, the difference between ontology, terminology, and CV is not important for their application in the curation of knowledge graphs (see my previous blog post). For the remainder of this post, I’ll be calling them CVs.

The list of CVs used in NeuroMMSig that did not meet the high confidence criteria included three previously published ontologies (i.e., ADO, PDON, MSO) that could be found in the Protégé OWL format through the NCBO BioPortal. The underlying issue with these three resources is that they are actually text mining dictionaries masquerading as ontologies. As outlined in their respective publications, they were created to train a dictionary-based named entity recognition system. From there, careless usage of Protégé led to confusing organization, messy and cryptic entity labels, lack of meaningful entity hierarchies, lack of definitions or provenance for each entity, lack of curator provenance for each entity, and non-compliance with the MIRIAM standards for entity identifiers. Further, the lack of maintenance meant that when entities originally novel to these resources were curated again for inclusion in high confidence, mainstream ontologies, that there would exist no mappings (e.g., equivalences) to these high confidence resources. This could create redundancies that undermine the value of the downstream resources, like NeuroMMSig, that use these low confidence resources. Like I outlined in my previous post, the solution to this problem is to not compete with groups maintaining high confidence resources, and instead to contribute to them directly.

A fourth previously published CV (i.e., NIFT) in NeuroMMSig that did not meet the high confidence criteria only existed as a Biological Expression Language (BEL) namespace file served by Fraunhofer’s web-based BEL curation interface. Because it could be shut down at any time, I started by copying it to a more persistent server but later began using GitHub directly for versioning and distribution. While not masquerading as an ontology, this CV also had issues with messy names (inclusion of strange punctuation, inconsistent usage of capitalization, etc.). By virtue of its lack of structure or organization that benefits ontologies, the assessment of each term for its novelty, correctness, and whether it could be replaced with an already existing entry in a high confidence ontology became its own time-consuming research project.

There were additional low confidence CVs generated as ad-hoc extensions to high confidence CVs. For example, the Brain Region and Cell-type Ontology (BRCO; unpublished) was an extension to Uberon. Unfortunately, the value of this extension was questionable as it seemed most (if not all) of the extensions made actually duplicated efforts of the original ontology itself. As several years had passed between the generation of this extension and my assessment, it’s possible that some of the terms included in BRCO were later curated by Uberon by virtue of its excellent maintenance. This phenomenon persisted across other examples of ad-hoc ontology extensions I found in NeuroMMSig.

Finally, there were several other novel low confidence CVs only distributed as BEL namespaces that suffered from a mixture of the previously described problems.

Making a Home for Lost and Forgotten Names

It should also be noted that my assessment of the low confidence CVs focused on their respective terms that appeared in the NeuroMMSig knowledge graphs. There were also hundreds of terms appearing in NeuroMMSig that had not been assigned based on any CV at all (what I called “naked names”). Some could be assigned via text mining tools, but others needed a home. It was finally time to start curating an ontology that could unify all of the existing abandoned CVs used by NeuroMMsig and solve their respective problems. And so, along with my team of excellent students, work was begun on the Curation of Neurodegeneration Supporting Ontology (CONSO; data, website).

We began by re-curating all of the terms from low confidence CVs that appeared in NeuroMMSig. Each was given its own entry in CONSO along with a persistent identifier, a typographically sane preferred label, a definition in english prose, references to papers where it appeared, and contact information of the curator in case there were questions. We found that the curator attribution was incredibly important as questions arose about previous terms. We maintained synonyms and xrefs for each term as well as implementing code that leverages several services (e.g., OLS, PubChem, etc.) for new mappings to propose for curation. After, we did the same for all of the names appearing in NeuroMMSig that were not qualified with a namespace/prefix. Later, CONSO was improved with completely novel curation to support additional curation of neurodegeneration disease phenomena that became the Curation of Neurodegeneration in BEL (CONIB; data, website).

CONSO maintains entities of several types, including chemicals, chemical roles, proteins, protein families, protein variants, protein isoforms, and protein isoform families (see figure below). It also maintains its own vocabulary of relationships and instances of those relationships. For example, we were unable to find a relation that describes the epitope corresponding to a given antibody in a high quality resource (e.g., the Relation Ontology). Therefore, we created our own relationship and started curating instances of it. Because it’s going as far as defining its own semantics, CONSO is a proper ontology.

While CONSO’s hierarchy is currently neither aligned to the Basic Formal Ontology nor the Systems Biology Ontology for the classification of entity types, the current organizational scheme is best suited to support downstream curation of biomedical relations. While I’ve written quite a bit about the applications of terminologies and ontologies by this point, I am definitely not an expert in its pedagogy and will have to bring in others for discussion on how to do this best. One of the other reasons why I haven’t done that yet is that we are slowly making pull requests, issues, and contributions to other ontologies to move all of the best stuff out of CONSO into other, better maintained ontologies. Like the resources that it supersedes, CONSO is not meant to be a high confidence resource, but rather an intermediate stage for curation as we find it a more appropriate home.

CONSO uses continuous integration on Travis-CI and an extensible verification script to check the integrity of the ontology on each push. The example above shows that isoforms are checked to have a relationship to their reference protein and that chemicals should be annotated with InChI, SMILES, and also InChI-key (not shown). There are also checks that all chemicals have been assigned chemical roles from the ChEBI ontology or CONSO (we’ve curated a few ourselves that get put in ChEBI later), that antibodies have been annotated with their targets.

It also performs simple checks that new terms are added with the next consecutive identifier, that minimum appropriate metadata has been added for new terms (we set the bar very high), and that all sheets use valid CONSO identifiers. Overall, these kinds of checks make it much easier to work in a team. As CONSO was curated by eight authors, they lessened the burden on explaining the rules for curation to new contributors - all changes must be made as pull requests and they must pass the tests on Travis-CI. If they don’t, new contributors just need to follow the instructions given to make updates until they do.

As the original goal of CONSO was to support the re-curation and enrichment of NeuroMMSig, it’s nice to see in the above figure how its development allowed us to reduce the number of BEL warnings in NeuroMMSig from almost 30,000 to zero (in addition to some other BEL-specific fixes). This figure comes from and old blog post on the Fraunhofer Human Brain Pharmacome site (which appears to have been taken down?) and my Ph.D. defense - and will hopefully make it to primetime in the next release of NeuroMMSig itself. Further, we were able to improve the overall quality of NeuroMMSig by normalizing many terms that were redundant across namespaces as previously described. CONSO was created using the guide in my previous blog post. More specifically, the experiences with CONSO inspired that guide to highlight the things that worked well (and not the things that didn’t). In the end, CONSO is automatically converted into an OWL file, an OBO file, a BEL namespace for BEL curation, and a website for search and exploration that is automatically deployed with GitHub Pages. It’s archived in Zenodo as well at .

CONSO could not have been done without all of the hard work of the curators at Fraunhofer during the AETIONOMY project that provided the basis for this work. However, I would like to make a special thanks to my curation team - Rana Aldisi, Lingling Xu, Sandra Spalek, Esther Wollert, Kristian Kolpeja, and Yojana Gadiya for their immense contributions, Daniel Domingo-Fernández for helping when I needed it, and Stephan Gebel for learning so much so quickly in order to take over the curation program after I finished my Ph.D. and left Fraunhofer.

How to Code with Me - Organizing a Package

2020-06-03T08:00:00+00:00

This blog post is the next installment in the series about all of the very particular ways I do software development in Python. This round is about where to put your code, your tests, your CLI, and the right metadata for each.

Package Structure

After following the debate on pypa/packaging.python.org#320, I’ve opted to use the src/ layout as aptly described by Ionel Cristian Mărieș, and Hynek Schlawack.

This means that there’s a top-level tests/ directory (will come back to that later) and no possibility of mixing up your working directory when making imports. I also enforce the sole usage of relative imports to make sure there are no accidental circular imports. While I know this is allowed and reasonable sometimes, I assume others will misuse it. Additionally, relative imports force users to access their scripts like modules using the command line like python -m my_module.my_submodule. This is good because I believe there should be no such thing as Python scripts. You should always think about packaging and how someone else will use your code later.

Licensing

Your package should have a file called LICENSE (no extension) that tells people how they’re allowed to use your code. Even if you’re working in a company and won’t be sharing code, it’s still good practice.

An excellent resource to help you choose a license is https://choosealicense.com/. I normally pick MIT License because it’s easy for other people to use and modify.

Ignore the Junk

This repository uses a .gitignore file to make sure no junk gets committed. GitHub will ask you if you want a pre-populated .gitignore added to your repo on creation. You can also go to gitignore.io to get more options.

Things that are especially bad to commit to repos:

compiled python files (*.pyc)
Jupyter notebook checkpoint folders (.ipynb_checkpoints/)
documentation builds (let ReadTheDocs take care of this!)
tox and other automation/build tool caches (.tox/, .pytest_cache/, .mypy_cache/, build/, dist/, etc.)
basically any file you didn’t make on purpose

I usually use this pre-configured .gitignore with has ignores appropriate for Python, Jupyter Notebooks, IntelliJ/PyCharm, Mac, Windows, and Linux.

Packaging

I use a declarative setup in all of my packages. It’s not easy to figure out everything in this documentation, so I either copy-paste from a previous project (usually pybel/pybel or use my cookiecutter template.

First, you need to create a setup.py file in the root of your repository when using a declarative setup. It should always look exactly like this:

# -*- coding: utf-8 -*-

"""The setup module."""

import setuptools

if __name__ == '__main__':
    setuptools.setup()

The first section in the setup.cfg is [metadata]. The top of the setup.cfg for PyBEL looks like this:

[metadata]
# The name of the package (should be same as what's in `src/{your project name}`)
name = pybel
# The version of the package (you should start with 0.0.1-dev for new projects)
version = 0.14.6-dev
# A one line description of your package, should be the same as the module-level docstring
# in src/{your project}/__init__.py
description = Parsing, validation, compilation, and data exchange of Biological Expression Language (BEL)
# The `file:` magical prefix tells it to load what's in your README.rst. You did write a nice readme, right?
long_description = file: README.rst

The next few lines describe the places where project resources live on the internet:

# [metadata]
# Where is the project
url = https://github.com/pybel/pybel
# Where can people get your code
download_url = https://github.com/pybel/pybel/releases
# You can put whatever key-values here you want, but these three are special
project_urls =
    Bug Tracker = https://github.com/pybel/pybel/issues
    Source Code = https://github.com/pybel/pybel
    Documentation = https://pybel.readthedocs.io

Next is author and licensing information.

# [metadata]
# Author information
author = Charles Tapley Hoyt
author_email = cthoyt@gmail.com
# Who is actually taking care of the code? Might not be the same as the author
maintainer = Charles Tapley Hoyt
maintainer_email = cthoyt@gmail.com

# What kind of license are you using? This uses a SPDX identifier (https://spdx.org/licenses/)
license = MIT
# The thing you put here is the name of the file in the same directory as the setup.cfg
license_file = LICENSE

The license_file entry obviously points to the file. The license entry is what gets shown on PyPI using the Software Package Data Exchange controlled vocabulary.

Next comes the PyBEL classifiers. This is a list of trove classifiers that are a controlled vocabulary for describing your project’s development status, who should use it, its topics, etc.

# [metadata]
# Search tags
classifiers =
    Development Status :: 5 - Production/Stable
    Environment :: Console
    Intended Audience :: Developers
    Intended Audience :: Science/Research
    License :: OSI Approved :: MIT License
    Operating System :: OS Independent
    Programming Language :: Python
    Programming Language :: Python :: 3.8
    Programming Language :: Python :: 3.7
    Programming Language :: Python :: 3.6
    Programming Language :: Python :: 3.5
    Programming Language :: Python :: 3 :: Only
    Topic :: Scientific/Engineering :: Bio-Informatics
    Topic :: Scientific/Engineering :: Chemistry

Again, the license is very important! pyroma (see below) won’t pass if you don’t have this. Also, the other things are important too, because this will tell users that you’re cool and only allow the newest Python versions. Unfortunately, at the time of writing this post, I still had to support Python 3.5 in PyBEL for downstream dependencies :/

Next are the keywords, which can be whatever you want. Here’s what I’ve got for PyBEL:

# [metadata]
keywords =
    Biological Expression Language
    BEL
    Domain Specific Language
    DSL
    Systems Biology
    Networks Biology

Next is the [options] section. First we’ll tell it what the requirements for the package are:

[options]
install_requires =
    networkx>=2.1
    sqlalchemy
    click
    click-plugins
    bel_resources>=0.0.3
    more_itertools
    requests
    requests_file
    pyparsing
    tqdm

This is part that’s hard to explain. The packages and package_dir option are tricky… you just have to do it this way and everything magically works. Then, you basically say the same thing one more time in the [options.packages.find].

# [options]
# You're always supposed to set zip_safe = false
zip_safe = false
# If you have some non-python files inside your `src/{your package/` directory you
# want to come for the ride when other people use your code, do this
include_package_data = True
# Always tell people what python you support! Is redundant of classifiers, but that's how it is.
python_requires = >=3.5

# Where is my code?
packages = find:
package_dir =
    = src

[options.packages.find]
where = src

Testing (The Prequel Series)

To make a tiny little test that shows everything works, make the following file in tests/test_import.py:

# -*- coding: utf-8 -*-

"""Test the module can be imported."""

import unittest

class TestImport(unittest.TestCase):
    """A test case for import tests."""

    def test_import(self):
        """Test that PyBEL can be imported."""
        import pybel

Now you’re ready to run some tests. Do the following in your shell:

pip install pytest
pytest tests/

Because pytest isn’t actually a requirement to use pybel, but it’s useful to have installed, you can specify it in an optional requirement in the [options.extras_require] section of your setup.cfg like in:

[options.extras_require]
testing =
    pytest
docs =
    sphinx
    sphinx-rtd-theme
    sphinx-click
    sphinx-autodoc-typehints

You can install PyBEL with the testing and docs extras like:

pip install -e .[testing,docs]

Then you wouldn’t have to worry about the availability of pytest. However, there’s a better way to make sure that pytest is available, and more generally for any testing or build task, we have tox.

Building with Tox

There are three parts to automated builds with tox. First, you have to pip install tox. Second, you have to make a file in the root of the repository called tox.ini like below:

# The name of the default tox environment is [testenv]
[testenv]
# This is a list of commands to run as if you were in the shell yourself
commands =
    pytest tests/
# This is a list of extra dependencies to install with pip just for this testing environment
deps =
    pytest
description = Run the tests using pytest.

Third, you just have to run tox when the working directory in your shell is in the root of the repository. Then everything is taken care of for you! tox makes a new virtual environment, installs the repository using the setup.cfg/setup.py, installs the tox environment-specific dependencies, then runs the commands in order. There are tons of other options available for customizing tox listed on their documentation. There will be several more examples here that show some of them being used.

Since you’re using the extras in the setup.cfg, you can actually rewrite this configuration to use them:

[testenv]
commands =
    pytest tests/
extras =
    testing

Packaging Metadata

I use pyroma to make sure that I remembered to put everything in the packaging metadata. It can be run with

python -m pip install pyroma
pyroma --min=10 .

I welcome and encourage you to copy my configuration, but don’t forget to carefully change everything to your metadata. It’s pretty embarrassing if you accidentally attribute your work to me. I’ve done it before by accident. I’ve seen others do it too…

[testenv:pyroma]
deps =
    pygments
    pyroma
skip_install = true
commands = pyroma --min=10 .
description = Run the pyroma tool to check the package friendliness of the project.

This environment adds the skip_install key, which just says not to bother pip installing the whole package for the tests. This makes sense here because checking the metadata contained in setup.cfg doesn’t require actually installing the code.

Where to Put Persistent Data

If your application needs to download data that is not related to user input or configuration, it’s best that it has a default location for storing stuff that isn’t in a place a normal user will delete or corrupt. This is a situation where it might make sense to make a folder in the user’s home directory.

# -*- coding: utf-8 -*-

"""Constants for PyBEL."""

import os

__all__ = [
    'PYBEL_HOME',
]

# Have a reasonable default location
_DEFAULT_HOME = os.path.join(os.path.expanduser('~'), '.pybel')
# Allow the user to modify the location with an environment variable
PYBEL_HOME = os.path.abspath(os.getenv('PYBEL_HOME', _DEFAULT_HOME))
os.makedirs(PYBEL_HOME, exist_ok=True)

Now you can import PYBEL_HOME and do all sorts of nice os.path.joins to build the directory structure to help organize the data you might need. For example, PyBEL will download a copy of Daniel Himmelstein’s hetionet for conversion to BEL and put it in its cache folder.

Becuase this kind of configuration is so ubiquitous, I’ve written an package that supports doing this called pystow that simplifies the previous code to:

import pystow

pybel_module = pystow.module("pybel")
PYBEL_HOME = pybel_module.base

Where to Put Configuration

Only use the following section if your package actually needs configuration. Keep in mind that there should always be reasonable defaults for anything that can be configured, so users don’t actually have to engage with configuration.

Most packages put configuration inside the ~/.config/ folder, so you should do the same. PyBEL uses something a bit different, but here’s a mock of how loading configuration might look.

import os
from configparser import ConfigParser

CONFIG_PATH = os.path.join(os.path.expanduser('~'), '.config', 'pybel.ini')

cfp = ConfigParser()
cfp.read(CONFIG_PATH)

try:
    config = cfp['pybel']
except KeyError:
    config = {}


def get_config(key):
    """Get PyBEL-specific configuration."""
    return config.get(key)

You could also use JSON, but ini/cfg files are ubiquitous for configuration so it’s best to stick to what’s expected. Because I’ve written the previous code so often, I encapsulated it in a function in pystow:

import pystow

def get_config(key):
    pystow.get_config("pybel", key)

Note, this function has a few more bells and whistles than the boilerplate code for fallbacks, passthroughs, error handling, and type coercion.

Code Style

Mercilessly use flake8 to check your code has good style. If your code doesn’t have good style, nobody else will be able to read it. I already wrote a whole blog post on this one called Flake8 Hell.

Random Code Style Necessities

Every python file must start with the file encoding, a newline, then the module docstring like:

# -*- coding: utf-8 -*-

"""The module-level docstring."""

This docstring has to follow flake8 rules, meaning there’s a short description that fits on the first line then there’s a period. After that, there can be a blank line before any other restructured text-formatted documentation you’d like.

Bonus Round: How to Code with Ben Gyori

This isn’t something I do, but to maintain a clean git history, Ben Gyori frequently reminds me to rebase on master. This keeps a more linear history of what happened and when. Here are his instructions:

git fetch --all
# This is your master
git checkout master
# This is their master
git merge --ff-only upstream/master
git rebase master 
# Optionally
git push -f origin 

This is by no means everything I have to say on this topic. I’ll be back with more on documentation, ReadTheDocs, using CI, checking unit test coverage, and more.

A Reading List of Academic Articles using the Biological Expression Language (BEL)

2020-05-22T13:50:00+00:00

This post is evolving from a reading list to a review of the academic papers published that are either about or use the Biological Expression Language (BEL). It’s divided into the categories of software/visualization tools, algorithms/analytical frameworks, data integration, natural language processing, curation workflows, and downstream applications.

In my history of BEL, I imposed some quality and impact standards. Without getting into a debate about how peer review is broken because it’s missing these standards, this list is more inclusive to papers whether they are likely to be reproducible or useful to the community, or not.

Some papers in this list were tricky to find - many of the PMI papers do not mention BEL in the abstract and the journals in which their papers were published don’t seem to index properly in MEDLINE. You’re welcome to use my search on PubMed and my search on Europe PubMed Central to see for yourself. I also looked through the publication lists of some key authors from Selventa and PMI over the last 10 years to find several other application papers. As always, this list is incomplete due to the lack of findability/accessibility of papers, my lack of knowledge of everything, and lack of time to deeply read through all authors’ histories. If you know of a search tool that might be helpful for improving this list, please let me know. I’ve been meaning to check out CoCites:

Am thrilled this paper is finally out. The most exciting project I've ever worked on, a totally new method for searching scientific literature with stunning results.

We work to launch the method as a webtool in the coming months. Stay tuned. @cocites
1/https://t.co/Ok7RW6yXIn pic.twitter.com/tkpRqOT6bj
— Cecile Janssens (@cecilejanssens) February 7, 2020

As always, improvements are welcome. Check my contact info at the bottom of this post. I’ll be making a major update to this post later to be more like a review as I’m putting together the new BEL reference paper.

Software and Visualization

PyBEL: a computational framework for Biological Expression Language.
Hoyt, C. T., Konotopez, A. & Ebeling, C. (2017).
Bioinformatics, 34(4), 703–704.

BEL Commons: an environment for exploration and analysis of networks encoded in Biological Expression Language.
Hoyt, C. T., Domingo-Fernández, D., & Hofmann-Apitius, M. (2018).
Database, 2018(3), 1–11

INDRA-IPM: Interactive pathway modeling using natural language with automated assembly.
Todorov, P. V., Gyori, B. M., Bachman, J. A., & Sorger, P. K. (2019).
Bioinformatics, 35(21), 4501–4503.

Algorithms and Analytical Frameworks

A network-based approach to quantifying the impact of biologically active substances.
Hoeng, J., Deehan, R., Pratt, D., Martin, F., Sewer, A., Thomson, T. M., Drubin, D. A., Waters, C. A., de Graaf, D., & Peitsch, M. C. (2012).
Drug Discovery Today, 17(9), 413–418.

Assessment of network perturbation amplitudes by applying high-throughput data to causal biological networks.
Martin, F., Thomson, T. M., Sewer, A., Drubin, D. a, Mathis, C., Weisensee, D., Pratt, D., Hoeng, J., & Peitsch, M. C. (2012).
BMC Systems Biology, 6, 54.

Quantitative assessment of biological impact using transcriptomic data and mechanistic network models.
Thomson, T. M., Sewer, A., Martin, F., Belcastro, V., Frushour, B. P., Gebel, S., Park, J., Schlage, W. K., Talikka, M., Vasilyev, D. M., Westra, J. W., Hoeng, J., & Peitsch, M. C. (2013).
Toxicology and Applied Pharmacology, 272(3), 863–878.

Reverse causal reasoning: applying qualitative causal knowledge to the interpretation of high-throughput data.
Catlett, N. L., Bargnesi, A. J., Ungerer, S., Seagaran, T., Ladd, W., Elliston, K. O., & Pratt, D. (2013).
BMC Bioinformatics, 14(1), 340.

An algorithm for score aggregation over causal biological networks based on random walk sampling.
Vasilyev, D. M., Thomson, T. M., Frushour, B. P., Martin, F., & Sewer, A. (2014).
BMC Research Notes, 7, 516.

Quantification of biological network perturbations for mechanistic insight and diagnostics using two-layer causal models.
Martin, F., Sewer, A., Talikka, M., Xiang, Y., Hoeng, J., & Peitsch, M. C. (2014).
BMC Bioinformatics, 15(1), 238.

Multimodal mechanistic signatures for neurodegenerative diseases (NeuroMMSig): a web server for mechanism enrichment.
Domingo-Fernández, D., Kodamullil, A. T., Iyappan, A., Naz, M., Emon, M. A., Raschka, T., Karki, R., Springstubbe, S., Ebeling, C., & Hofmann-Apitius, M. (2017).
Bioinformatics, 33(22), 3679–3681.

BEL2ABM: Agent-based simulation of static models in Biological Expression Language.
Gündel, M., Hoyt, C. T., & Hofmann-Apitius, M. (2018).
Bioinformatics, 34(13), 2316–2318.

BioKEEN: a library for learning and evaluating biological knowledge graph embeddings.
Ali, M., Hoyt, C. T., Domingo-Fernández, D., Lehmann, J., & Jabeen, H. (2019).
Bioinformatics, 35(18), 3538–3540.

Quantifying mechanisms in neurodegenerative diseases using candidate mechanism perturbation amplitude algorithm.
Karki, R., Kodamullil, A. T., Hoyt, C. T., & Hofmann-Apitius, M. (2019).
BMC Bioinformatics, 20(1), 494.

NPA: an R package for computing network perturbation amplitudes using gene expression data and two-layer networks..
Martin, F., Gubian, S., Talikka, M., Hoeng, J., & Peitsch, M. C. (2019).
BMC Bioinformatics, 20(1), 451.

Leveraging Structured Biological Knowledge for Counterfactual Inference: a Case Study of Viral Pathogenesis
Zucker, J., Paneri, K., Mohammad-Taheri, S., Bhargava, S., Kolambkar, P., Bakker, C., Teuton, J., Hoyt, C. T., Oxford, K., Ness, R., & Vitek, O. (2021).
IEEE Transactions on Big Data, 2021.3050680

Data Integration

Using Drugs as Molecular Probes: A Computational Chemical Biology Approach in Neurodegenerative Diseases.
Emon, M. A. E. K., Kodamullil, A. T., Karki, R., Younesi, E., & Hofmann-Apitius, M. (2017).
Journal of Alzheimer’s Disease, 56(2), 677–686.

ComPath: an ecosystem for exploring, analyzing, and curating mappings across pathway databases.
Domingo-Fernandez, D., Hoyt, C. T., Alvarez, C. B., Marin-Llao, J., & Hofmann-Apitius, M. (2018).
npj Systems Biology and Applications, 5(1), 3.

Integration of Structured Biological Data Sources using Biological Expression Language.
Hoyt, C. T., Domingo-Fernández, D., Mubeen, S., Llaó, J. M., Konotopez, A., Ebeling, C., Birkenbihl, C., Muslu, Ö., English, B., Müller, S., de Lacerda, M. P., Ali, M., Colby, S., Türei, D., Palacio-Escat, N., & Hofmann-Apitius, M. (2019).
bioRxiv, 631812.

PathMe: merging and exploring mechanistic pathway knowledge.
Domingo-Fernández, D., Mubeen, S., Marín-Llaó, J., Hoyt, C. T., & Hofmann-Apitius, M. (2019).
BMC Bioinformatics, 20(1), 243.

The Impact of Pathway Database Choice on Statistical Enrichment Analysis and Predictive Modeling.
Mubeen, S., Hoyt, C. T., Gemünd, A., Hofmann-Apitius, M., Fröhlich, H., & Domingo-Fernández, D. (2019).
Frontiers in Genetics, 10, 654442.

Natural Language Processing

Construction of biological networks from unstructured information based on a semi-automated curation workflow.
Szostak, J., Ansari, S., Madan, S., Fluck, J., Talikka, M., Iskandar, A., De Leon, H., Hofmann-Apitius, M., Peitsch, M. C., & Hoeng, J. (2015).
Database, 2015, bav057.

Training and evaluation corpora for the extraction of causal relationships encoded in biological expression language (BEL).
Fluck, J., Madan, S., Ansari, S., Kodamullil, A. T., Karki, R., Rastegar-Mojarad, M., Catlett, N. L., Hayes, W., Szostak, J., Hoeng, J., & Peitsch, M. (2016).
Database, 2016, 1–20.

BioCreative V track 4: a shared task for the extraction of causal network information using the Biological Expression Language.
Rinaldi, F., Ellendorff, T. R., Madan, S., Clematide, S., van der Lek, A., Mevissen, T., & Fluck, J. (2016).
Database, 2016, baw067.

Coreference resolution improves extraction of Biological Expression Language statements from texts.
Choi, M., Liu, H., Baumgartner, W., Zobel, J., & Verspoor, K. (2016).
Database, 2016, 1–14.

The BEL information extraction workflow (BELIEF): evaluation in the BioCreative V BEL and IAT track.
Madan, S., Hodapp, S., Senger, P., Ansari, S., Szostak, J., Hoeng, J., Peitsch, M., & Fluck, J. (2016).
Database, 2016(September), 1–17.

BelSmile: a biomedical semantic role labeling approach for extracting biological expression language from text.
Lai, P. T., Lo, Y. Y., Huang, M. S., Hsiao, Y. C., & Tsai, R. T. H. (2016).
Database, 2016(June), 1–9.

BELTracker: evidence sentence retrieval for BEL statements.
Rastegar-Mojarad, M., Komandur Elayavilli, R., & Liu, H. (2016).
Database, 2016(June), 1–11.

BELMiner: Adapting a rule-based relation extraction system to extract biological expression language statements from bio-medical literature evidence sentences.
Ravikumar, K. E., Rastegar-Mojarad, M., & Liu, H. (2017).
Database, 2017(1), 1–12.

Automatic Extraction of BEL-Statements based on Neural Networks.
Ali, M., Madan, S., Fischer, A., Petzka, H., & Fluck, J. (2017).
Proceedings of BioCreative VI Challenge and Workshop, October.

From word models to executable models of signaling networks using automated assembly.
Gyori, B. M., Bachman, J. A., Subramanian, K., Muhlich, J. L., Galescu, L., & Sorger, P. K. (2017).
Molecular Systems Biology, 13(11), 954.

The extraction of complex relationships and their conversion to biological expression language (BEL) overview of the BioCreative VI (2017) BEL track.
Madan, S., Szostak, J., Komandur Elayavilli, R., Tsai, R. T. H., Ali, M., Qian, L., Rastegar-Mojarad, M., Hoeng, J., & Fluck, J. (2019).
Database, 2019(1), 1–17.

Combining relation extraction with function detection for BEL statement extraction.
Liu, S., Cheng, W., Qian, L., & Zhou, G. (2019).
Database, 2019(4), 1–12.

Hierarchical sequence labeling for extracting BEL statements from biomedical literature.
Liu, S., Shao, Y., Qian, L., & Zhou, G. (2019).
BMC Medical Informatics and Decision Making, 19(Suppl 2).

Extraction of causal relations based on SBEL and BERT model
Shao, Y., Li, H., Gu, J., Qian, L., & Zhou, G. (2021).
Database, 2021.

Curation Workflows

On crowd-verification of biological networks.
Ansari, S., Binder, J., Boue, S., Di Fabio, A., Hayes, W., Hoeng, J., Iskandar, A., Kleiman, R., Norel, R., O’Neel, B., Peitsch, M. C., Poussin, C., Pratt, D., Rhrissorrakrai, K., Schlage, W. K., Stolovitzky, G., & Talikka, M. (2013).
Bioinformatics and Biology Insights, 7, 307–325.

sbv IMPROVER: Modern Approach to Systems Biology BT - Biological Networks and Pathway Analysis.
Guryanova, S., & Guryanova, A. (2017).
T. V Tatarinova & Y. Nikolsky (eds.), pp. 21–29. Springer New York.

Re-curation and rational enrichment of knowledge graphs in Biological Expression Language.
Hoyt, C. T., Domingo-Fernández, D., Aldisi, R., Xu, L., Kolpeja, K., Spalek, S., Wollert, E., Bachman, J., Gyori, B. M., Greene, P., & Hofmann-Apitius, M. (2019).
Database, 2019(1).

Content and Applications

A computable cellular stress network model for non-diseased pulmonary and cardiovascular tissue.
Schlage, W. K., Westra, J. W., Gebel, S., Catlett, N. L., Mathis, C., Frushour, B. P., Hengstermann, A., Van Hooser, A., Poussin, C., & Wong, B. (2011).
BMC Syst Biol, 5.

Construction of a Computable Cell Proliferation Network Focused on Non-Diseased Lung Cells.
Westra, J. W., Schlage, W. K., Frushour, B. P., Gebel, S., Catlett, N. L., Han, W., Eddy, S. F., Hengstermann, A., Matthews, A. L., & Mathis, C. (2011).
BMC Syst Biol, 5.

Construction of a computable network model for DNA damage, autophagy, cell death, and senescence.
Gebel, S., Lichtner, R. B., Frushour, B., Schlage, W. K., Hoang, V., Talikka, M., Hengstermann, A., Mathis, C., Veljkovic, E., Peck, M., Peitsch, M. C., Deehan, R., Hoeng, J., & Westra, J. W. (2013).
Bioinformatics and Biology Insights, 7, 97–117.

A modular cell-type focused inflammatory process network model for non-diseased pulmonary tissue.
Westra, J. W., Schlage, W. K., Hengstermann, A., Gebel, S., Mathis, C., Thomson, T., Wong, B., Hoang, V., Veljkovic, E., Peck, M., Lichtner, R. B., Weisensee, D., Talikka, M., Deehan, R., Hoeng, J., & Peitsch, M. C. (2013).
Bioinformatics and Biology Insights, 7, 167–192.

Systematic verification of upstream regulators of a computable cellular proliferation network model on non-diseased lung cells using a dedicated dataset.
Belcastro, V., Poussin, C., Gebel, S., Mathis, C., Schlage, W. K., Lichtner, R. B., Quadt-Humme, S., Wagner, S., Hoeng, J., & Peitsch, M. C. (2013).
Bioinformatics and Biology Insights, 7, 217–230.

Case study: the role of mechanistic network models in systems toxicology.
Hoeng, J., Talikka, M., Martin, F., Sewer, A., Yang, X., Iskandar, A., Schlage, W. K., & Peitsch, M. C. (2014).
Drug Discovery Today, 19(2), 183–192.

In vitro systems toxicology approach to investigate the effects of repeated cigarette smoke exposure on human buccal and gingival organotypic epithelial tissue cultures..
Schlage, W. K., Iskandar, A. R., Kostadinova, R., Xiang, Y., Sewer, A., Majeed, S., Kuehn, D., Frentzel, S., Talikka, M., Geertz, M., Mathis, C., Ivanov, N., Hoeng, J., & Peitsch, M. C. (2014).
Toxicology Mechanisms and Methods, 24(7), 470–487.

A vascular biology network model focused on inflammatory processes to investigate atherogenesis and plaque instability.
De León, H., Boué, S., Schlage, W. K., Boukharov, N., Westra, J. W., Gebel, S., VanHooser, A., Talikka, M., Fields, R. B., Veljkovic, E., Peck, M. J., Mathis, C., Hoang, V., Poussin, C., Deehan, R., Stolle, K., Hoeng, J., & Peitsch, M. C. (2014).
Journal of Translational Medicine, 12(1).

Micropublications: A semantic model for claims, evidence, arguments and annotations in biomedical communications.
Clark, T., Ciccarese, P. N., & Goble, C. A. (2014).
Journal of Biomedical Semantics, 5(1), 1–33.

Computational Modelling Approaches on Epigenetic Factors in Neurodegenerative and Autoimmune Diseases and Their Mechanistic Analysis.
Khanam Irin, A., Tom Kodamullil, A., Gündel, M., & Hofmann-Apitius, M. (2015).
Journal of Immunology Research, 2015, 1–10.

Semi-automated curation allows causal network model building for the quantification of age-dependent plaque progression in ApoE−/− mouse.
Szostak, J., Martin, F., Talikka, M., Peitsch, M. C., & Hoeng, J. (2016).
Gene Regulation and Systems Biology, 10, 95–103.

Community-reviewed biological network models for toxicology and drug discovery applications.
Namasivayam, A. A., Morales, A. F., Lacave, Á. M. F., Tallam, A., Simovic, B., Alfaro, D. G., Bobbili, D. R., Martin, F., Androsova, G., Shvydchenko, I., Park, J., Val Calvo, J., Hoeng, J., Peitsch, M. C., Racero, M. G. V., Biryukov, M., Talikka, M., Pérez, M. B., Rohatgi, N., … Xiang, Y. (2016).
Gene Regulation and Systems Biology, 10, 51–66.

Reasoning over genetic variance information in cause-and-effect models of neurodegenerative diseases.
Naz, M., Kodamullil, A. T., & Hofmann-Apitius, M. (2016).
Briefings in Bioinformatics, 17(3), 505–516.

Of Mice and Men: Comparative Analysis of Neuro-Inflammatory Mechanisms in Human and Mouse Using Cause-and-Effect Models.
Kodamullil, A. T., Iyappan, A., Karki, R., Madan, S., Younesi, E., & Hofmann-Apitius, M. (2017).
Journal of Alzheimer’s Disease, 59(3), 1045–1055.

Comorbidity Analysis between Alzheimer’s Disease and Type 2 Diabetes Mellitus (T2DM) Based on Shared Pathways and the Role of T2DM Drugs.
Karki, R., Kodamullil, A. T., & Hofmann-Apitius, M. (2017).
Journal of Alzheimer’s Disease, 60(2), 721–731.

Novel approaches to develop community-built biological network models for potential drug discovery.
Talikka, M., Bukharov, N., Hayes, W. S., Hofmann-Apitius, M., Alexopoulos, L., Peitsch, M. C., & Hoeng, J. (2017).
Expert Opinion on Drug Discovery, 12(8), 849–857.

A systematic approach for identifying shared mechanisms in epilepsy and its comorbidities.
Hoyt, C. T., Domingo-Fernández, D., Balzer, N., Güldenpfennig, A., & Hofmann-Apitius, M. (2018).
Database, 2018(1).

Construction of a suite of computable biological network models focused on mucociliary clearance in the repiratory tract.
Yepiskoposyan, H., Talikka, M., Vavassori, S., Martin, F., Sewer, A., Gubian, S., Luettich, K., Peitsch, M. C., & Hoeng, J. (2019).
Frontiers in Genetics, 10(FEB), 1–12.

34th Annual Meeting & Pre-Conference Programs of the Society for Immunotherapy of Cancer (SITC 2019): part 2 : National Harbor, MD, USA. 10 November 2019.
(2019).
Journal for Immunotherapy of Cancer, 7(Suppl 1), 283.

See entry entitled “Reverse causal inferencing on lung adenocarcinoma patients reveals a stem cell-like molecular subtype associated with pack year history”

The reduction of DSS-induced colitis severity in mice exposed to cigarette smoke is linked to immune modulation and microbial shifts.
Lo Sasso, G., Phillips, B. W., Sewer, A., Battey, J. N. D., Kondylis, A., Talikka, M., Titz, B., Guedj, E., Peric, D., Bornand, D., Dulize, R., Merg, C., Corciulo, M., Ouadi, S., Yanuar, R., Tung, C. K., Ivanov, N. V, Peitsch, M. C., & Hoeng, J. (2020).
Scientific Reports, 10(1), 3829.

A Computational Approach for Mapping Heme Biology in the Context of Hemolytic Disorders.
Humayun, F., Domingo-Fernández, D., Paul George, A. A., Hopp, M. T., Syllwasschy, B. F., Detzel, M. S., Hoyt, C. T., Hofmann-Apitius, M., & Imhof, D. (2020).
Frontiers in Bioengineering and Biotechnology, 8(March), 1–10.

COVID-19 Knowledge Graph: a computable, multi-modal, cause-and-effect knowledge model of COVID-19 pathophysiology.
Domingo-Fernández, D., Baksi, S., Schultz, B., Gadiya, Y., Karki, R., Raschka, T., Ebeling, C., Hofmann-Apitius, M., & Kodamullil, A. T. (2020).
bioRxiv, 2020.04.14.040667.

Unravelling the debate on heme effects in COVID-19 infections.
Hopp, M., Domingo-Fernández, D., Gadiya, Y., Detzel, M. S., Schmalohr, B. F., Steinbock, F., Imhof, D., & Martin Hofmann-Apitius, M. (2020).
bioRxiv, 2020.06.09.142125

Data-Driven Modeling of Knowledge Assemblies in Understanding Comorbidity Between Type 2 Diabetes Mellitus and Alzheimer’s Disease.
Karki, R., Madan, S., Gadiya, Y., Domingo-Fernández, D., Kodamullil, A. T., & Hofmann-Apitius, M. (2020).
Journal of Alzheimer’s Disease, 78(1), 87–95.

A Systems Biology Approach for Hypothesizing the Effect of Genetic Variants on Neuroimaging Features in Alzheimer’s Disease.
Khatami, S. G., Domingo-Fernández, D., Mubeen, S., Hoyt, C. T., Robinson, C., Karki, R., Iyappan, A., Kodamullil, A. T., & Hofmann-Apitius, M. (2021).
Journal of Alzheimer’s Disease, 201397

The Trouble with Ontologies, or, How to Build an Ontology

2020-05-12T08:00:00+00:00

Everyone’s talking about biomedical ontologies! Let’s look at where most people go wrong and how to do it right.

The Trouble with Biomedical Ontologies

There’s a lot of confusion within the biomedical community as to what constitutes an ontology. It’s been aggravated by the impenetrable discourse used by ontologists similar to the way that mathematicians and computer scientists have a habit of using obscure mathematical notation to obfuscate their work. Even worse, this has led to confusion within the biomedical community as to what an ontology is for.

For example, biocurators find ontologies useful as a research tool for looking up the definitions of entities using tools like the Ontology Lookup Service (OLS). Database maintainers and biocurators often look to ontologies to decide what their data should look like and enable other people to understand it readily. Similarly, natural language processing (NLP) researchers find ontologies incredibly useful in named entity recognition because ontologies often contain preferred labels in many languages and synonyms for its terms. Software developers and end-users find ontologies useful for organizing and conveying information in a hierarchical manner.

While it might be true that ontologies have the ability to support all of those uses, none accurately portray ontologies. The following table presents more appropriate vocabulary to describe each of these use cases:

Phrase	Definition
Semantic Space	An enumerated set of entities
Controlled Vocabulary	An enumerated set of entities and their names
Dictionary	An enumerated set of entities and their definitions
Thesaurus	An enumerated set of entities and their synonyms
Taxonomy / Hierarchy	An enumerated set of entities with one parent for each
Multi-Hierarchy	An enumerated set of entities with one or more parents for each

Database maintainers need controlled vocabularies to improve the utility of their databases, researchers can leverage dictionaries to learn the definitions, NLP developers need thesauri containing synonyms to train their models, and software developers need hierarchies or multi-hierarchies to support organization.

The actual purpose of an ontology is to first define the rules for how information is organized and second to apply those rules and actually store information. This means that ontologies can define that entities need an identifier, they need a preferred label, they can have synonyms, they can have equivalences to other entities in other ontologies, they can have relationships in the form of triples (like parent-child relationships), and they can even have relationships in the form of quadruples or higher-dimensional tuples. There are “upper-level” ontologies that take care of some of the common definitions that can be shared throughout a domain - you’ve probably noticed that most biomedical ontologies have this core of previously mentioned relationship types. That enables “lower-level” ontologies to focus on storing the information. Since most researchers only interact with the “lower-level” ontologies, it can be understood where the confusion came from.

In practice, this confusion is mostly harmless. We have numerous high-quality ontologies covering a huge amount of biology and related fields, as well as the OBO Foundry to vet them for quality and host them. Note: BioPortal is another place for hosting, but it seems like it has a much lower (or non-existent?) threshold for quality.

The danger lies in the edge cases. When there are no high quality biomedical ontologies in a given area, researchers are often inclined to generate their own. The rest of this blog post is about what happens next, where it all goes wrong, and how you can avoid it when you’re in the situation that you just became an ontologist, too.

How to Build Your Own Biomedical Ontology

The curation I trust most is by people who know what they’re doing, and more importantly by people who love what they’re doing.

Given the choice between curating information about a new protein (e.g. the proteins in the novel coronavirus) myself or having one of the excellent curators at UniProt do it, I would choose UniProt every time. Given the choice between the world’s leading coronavirus researcher and a UniProt curator with no previous knowledge about the novel coronavirus, I would still choose the UniProt curator. The UniProt curators love what they do, they know how to do it well, and they do it right. Same goes for lots of other groups that I’ve praised elsewhere in this blog. So keep in mind while you’re reading this guide that you might be causing more harm than good by making yet another ontology.

Preparation

Before you start curating you need to do a bit of planning.

Pick a memorable name and prefix

There are a lot of ontologies, so pick a name that’s both unique and descriptive. Then, you need to pick a relatively short “prefix” which will be the first part of compact URIs (CURIEs) that point to entries in your ontology. Many ontologies use an acronym as their prefix, but make sure you don’t cause a conflict or confusion with a previously existing one. You can search through registries like Identifiers.org, the OLS, or the OBO Foundry (which, on a side note, don’t exactly contain all the same stuff) to check out the existing landscape.

Pick a scheme for identifiers

Even if you’re just here to build and maintain your controlled vocabulary, it’s still necessary to give identifiers to each of the entries in your ontology. In practice, each entry’s identifier should conform to the Minimal Information Requested in the Annotation of Biochemical Models (MIRIAM) standard. It states that identifiers should have the following five properties:

Uniqueness
Perenniality
Standards-Compliant
Resolvability
Free Usability

There’s a really important thing to keep in mind - identifiers are not numbers. Even if they look like them, they’re strings. If that doesn’t make sense to you, then just think about what it would mean to “add” two identifiers to each other. It would be nonsense to think of identifiers as numbers because they don’t do what numbers do. With that in mind, there are a few common identifier schemes:

Identifiers that look like numbers, like PubMed identifiers. An example from PubMed is 29048466.
Identifiers that look like numbers, but are a fixed width with left-padded zeros. An example from the Experimental Factor Ontology is 0004859.
Identifiers that look like numbers, but are a fixed width with left-padded zeros and are prefixed with the prefix itself separated by a colon, so the identifier itself looks like a CURIE. An example from the Gene Ontology is GO:0006915. This is sort of confusing, and has been dubbed the GOGO problem. Or the Bananananana problem.
Identifiers that look like numbers, but are prefixed with part of all of the prefix. An example from ChEMBL is CHEMBL941 where the prefix is chembl.compound.
Identifiers that have a short letter prefix then a fixed width number with left-padded zeros. An example from MeSH is D013313.

My favorite is the MeSH style, because it allows for the most information to be conveyed succinctly. You should use numbers of width 6 or 7, even if you only plan on curating a few dozen or a hundred terms.

Please don’t use the GO style identifiers, because this creates a ton of confusion.

You should also write down what the regular expression that goes with your identifiers for later validation. In the MeSH example, the regular expression is ^(C|D)\d{6,9}$, which means it either starts with C or D, and is followed by between 6 and 9 numbers. The ^ means beginning of the string and $ means end of the string, so it’s clear that nothing can precede or follow.

Pick your scope

The last, and most important, part of planning is to pick the scope of your ontology. You have to choose what kinds of entities you want to include (genes, proteins, side effects, etc.). Keep in mind that if you’re picking one of these examples, there’s probably already a good nomenclature source for it, so it’s best you don’t curate it again.

Curation

It’s time to start curating entries in your ontology. Most people go right to Protégé.

Don’t.

Protégé is a perfect way to get bogged down in the ivory tower that is ontology. Instead, it’s better to focus on the aspects of the ontology that I think are practically the most important. So in this guide, we’re going to use a set of interconnected tab-separated values (TSV) documents. Why TSV? Because comma-separated values (CSV) documents look awful and Excel sheets can’t be diff’d / viewed in GitHub. However, I would accept the following alternative:

I propose we change CSV to Clown Separated Values since that tends to be about how well CSV parsers work. pic.twitter.com/C8Uys3H9Kp
— Kevin Jones 🏳️‍🌈 🧼👏 (@vcsjones) September 8, 2019

In a later post, I’ll come back to how to programatically generate OWL, OBO, BEL, and other formats that are commonly used for ontologies from your curation sheets.

Curate entities

The most important thing in an ontology is the entities. Make a file called entities.tsv. It needs a few columns to hold the most important information for each entity:

Identifier - the identifier of the entity
Name - the preferred name of the entity in the main language of the ontology
Type - the entity type. For example, the Gene Ontology has three entity types - biological process, cellular component, and molecular function (they call them “namespaces”). This isn’t the same as the parent of the entity
References - keep a comma-separated list of CURIEs pointing to resources that have more information about this entity from PubMed, PubMed Central, etc.
Description - a short description of the entity written as prose. Shamelessly borrow from Wikipedia if appropriate, but remember to cite your source!
Curator ORCID identifier - it’s really important to keep track of who added entries to the ontology so you can get in touch when there is confusion or errors are found. The ORCID identifier is the best unambiguous way to do this.

You should write and enforce a style guide (e.g., only proper nouns are capitalized in labels for entities) for names and definitions while you’re here.

If there are other pieces of information that all entities must have in your ontology, then you can also include it in this sheet. Later, the properties and relationships sheets can be used for other information and other relationships such as parent/child relationships, physical properties, etc..

Curate synonyms

Make a file called synonyms.tsv. It needs a few columns to describe synonyms for each term and the provenance of where they came from:

Identifier - the identifier of the entity that matches to the entities.tsv sheet
Synonym - the actual text you found
Provenance - a CURIE describing the source that had the synonym. This might be a PubMed, PubMed Central, URL, or related.
Synonym Semantics - is this an exact synonym, a broad synonym, a narrow synonym, or a related synonym? Each entry should only be one of EXACT, NARROW, BROAD, or RELATED as defined in the OBO 1.4 standard. If you’re not sure, just put EXACT.

Curate xrefs

Make a file called xrefs.tsv. It needs three columns:

Identifier - the identifier of the entity that matches to the entities.tsv sheet
Xref Prefix - the prefix for the data source that describes the same entity
Xref Identifier - the local identifier of the entity in the xref’s data source

The prefix and identifier for the prefix are split to avoid the headache of parsing CURIEs later.

It’s best to consider xrefs as equivalences. All other relationships should be in the relationships page (later).

Curate typedefs

An xref is a very specific type of relationship, so it has first-class status. The parent-child relationship is also first-class and it goes without saying. The rest of the relationships (that can be expressed realized as triples) need to be either pulled in from a previously existing ontology like the Relation Ontology or PSI-MI, or defined in a structured way. Make a file called typedefs.tsv, to borrow from the OBO nomenclature for defining relationships. It should have the following columns:

Prefix - could be the same as the current ontology, or an external one
Identifier - if it’s from the current ontology, you might consider using a different identifier scheme than for entities. For example, WikiData uses ^Q\d+$ for entities and ^P\d+$ for relationships
Name - The preferred name of the relationship
(Optional) Inverse Of - If the relationship can be defined as the inverse of another, put its CURIE here
Parent(s) - a comma-separated list of the relation’s parents’ CURIEs. This is a special case that doesn’t appear in the relationships sheets because isA relationships are so important.

Curate relationships

You’re ready to use the relationships defined in typedefs.tsv to write out relationships. Make out_relations.tsv with the following columns:

Identifier - the identifier of the entity that matches to the entities.tsv sheet
Relation Prefix
Relation Identifier
Target Prefix
Target Identifier
Target Name (Optional, but useful for readers)

Similarly, make another sheet called in_relations.tsv with the following columns:

Source Prefix
Source Identifier
Source Name (Optional, but useful for readers)
Relation Prefix
Relation Identifier
Identifier - the identifier of the entity that matches to the entities.tsv sheet

Between these two sheets, you can encode relationships between entities in the ontology that are both incoming and outgoing, removing the need to define ad-hoc inverses of common relationships, like isA.

Curate properties

Properties are like relationships that point to scalar values instead of other entities. For a counterexample, synonyms are a first-class property that contains lots of extra metadata and therefore get their own sheet.

The rest of the properties will appear here. A good example of a property is the chemical formula, SMILES string, and mass of a given small molecule in the ChEBI ontology. However, not all entries in the ChEBI ontology are small molecules, so if they were following this guide, it might not have made sense to put that property in the entities.tsv sheet. Make a sheet called properties.tsv with the following columns:

Identifier - the identifier of the entity that matches to the entities.tsv sheet
Property - it’s up to you how to decide what the properties in your ontology are. It’s not as common to define it as precisely as with relationships
Value
(Optional) Data Type - the XSD data type of the value for the property. If this isn’t important to you, your life will probably be better by leaving it out

Now that you’ve made all the sheets, you can make sure that your curators do their best job to fill out entries in each of them every time a new entity is added. It’s also necessary to keep track of the uniqueness of entity identifiers as new ones are added. It’s best if they’re consecutive and increasing, too.

Maintenance

One of the other real dangers of starting your own ontology is the entire concept of maintenance and quality assurance. If you’re working in an academic group, it’s highly unlikely that you will have the resources, motivation, or willpower to maintain the ontology that you are building. This can be proven by reading through the ontologies listed in BioPortal. While this might be unavoidable, there are a few things that you can do before your time as a PhD student, Postdoc, or whatever comes to an end to make sure that your ontology is actually useful for somebody else.

I’ve already given my explanation of why to use TSV - it makes sure there’s no conflicts with spaces or commas, tabs never show up in real text, and GitHub will make nice renders of TSVs and show you the diffs as versions change, versus Excel documents, which are saved as binary.

Version Control

As I’ve just alluded, use version control. Keep track of how your ontology changes over time by making a repository on GitHub. I’ve heard rumors that git was created by Linus Torvalds to slow people down, so you should use an interactive GUI for git like GitHub Desktop (you’re not a martyr!). While you’re working on GitHub, you should use the GitHub Flow workflow, which involves forking (or branching), making pull requests, then reviewing and discussing before merging into master. This is more relevant for people who are working on teams. If you’re not working on a team, try pulling in a collaborator to review your work as a pull request. Or email/tweet me! I’d be happy to help if you’re working in open source with a publicly usable license.

Tooling

The next few suggestions rely on a bit of technical expertise. The first is that you should write scripts that validate the content’s integrity, formatting, correctness, or whatever rules you can come up with. Then you should use continuous integration (e.g., Travis-CI or GitHub Actions) to run those scripts on every commit to give feedback. If you’re working using the GitHub Flow fork/pull request workflow, then you can always ask your curators to make sure that their content doesn’t make the validation scripts fail before merging them into master.

Next, you should write scripts that export all of your content into common formats so others can consume it like OWL, OBO, BEL Namespace, etc. Additionally, it’s nice to automatically build a website that displays all the curated content and allows people to explore it. GitHub will even host the site for free.

These suggestions probably sound a bit abstract or scary if you’re not a seasoned programmer, so in a later post, I’ll provide you with a cookiecutter template repository with all the files, scripts, and configuration that you need to do this without any programming at all. An example of most of it in practice is the Curation of Neurodegeneration Supporting Ontology (CONSO) (source code; web site). It has a few differences from the recommendations I’ve made in this post - some of them inspired by choices I made during the curation of CONSO that I think could have been done better.

Choose a License

The license tells other people how they’re allowed to use your ontology. If you don’t use an appropriate open license, other people will not be legally allowed to use your ontology. And if that’s the case, there really wasn’t a point to making it (yes, I’m being pedantic here). Check out https://choosealicense.com/non-software/ for some pointers. I suggest the CC0 license, which is the most usable one out there. Don’t fear - people will cite your work and thank you for it, even if the license doesn’t legally obligate them to.

Making Releases and Long Term Maintenance

If you’re using GitHub, you can easily integrate the repository with Zenodo, which archives the repository when you make a tag and assigns a digital object identifier (DOI) to each release. You might also want to make releases to the OBO Foundry or BioPortal. You might also want to register your prefix at Identifiers.org to give your CURIEs maximum legitimacy.

Even with the best intentions, you will inevitably have to change some names over time. This is okay because your identifiers are persistent! However, you might have to retire entries. This might mean adding a column to entities.tsv with the date that a term is made obsolete.

Upgrade to more powerful tooling

While this guide was focused on how to get started with building ontologies if you’re new, there are obviously a lot of good reasons why people use more powerful formalisms like OWL to curate their ontologies. Chris Mungall, a prominent ontologist, gave me some feedback via Twitter that’s definitely worth a ready from anyone who made it this far in the following thread:

I really like Charlie's take here, even though some of it will annoy many ontologists and my @OBOFoundry colleagues! Lots of sensible advice, would make a good PLOS 10 rules article. I do have a few suggestions and comments... 1/ https://t.co/LXxa13LJ4a
— Chris Mungall (@chrismungall) May 13, 2020

To summarize, he linked to the following article to show how OWL enables the Gene Ontology to do many more powerful things, such as axiomization, that just don’t fit into a TSV-only world for curation.

Use of OWL within the Gene Ontology Christopher J Mungall, Heiko Dietze, David Osumi-Sutherland. bioRxiv, 010090; DOI: 10.1101/010090

He also gave two links to some previously developed tools that can help you get started with generating ontologies from tables:

https://github.com/INCATools/ontology-development-kit/
http://robot.obolibrary.org/template

These accomplish a similar goal to what I’ll present in the next blog post, and I’ll have to do my due diligence to figure out how they work to provide a more in-depth comparison.

You can’t compete with UniProt, the Disease Ontology, the Gene Ontology, or other groups that exist to maintain high quality resources.

So don’t.

Join them.

Make sure that your ontology is written well so the relevant parts can be incorporated into these and other high quality, maintained ontologies. Then, get in touch with their maintainers. Tweet at them, send GitHub issues, etc. They’ll be happy to get input on what they should do next, because, like I said before, these people love what they do. And there’s nothing better than seeing that something you are proud of is useful for other people.

Stay tuned for my next post where I’ll give you the code I wrote to do all the things I recommended before. I’ll put my money where my mouth is and present my ontology that led to building this curation environment and ultimately writing this post - the Curation of Neurodegeneration Supporting Ontology.

A Listing of Publicly Available Content in the Biological Expression Language (BEL)

2020-04-30T16:50:00+00:00

While many researchers have a pathway or pathology of interest, their first time curating content in the Biological Expression Language (BEL) may seem intimidating. This post lists several disease maps and BEL content sources that are directly available for re-use.

Manually Curated BEL

The bel-repository Python package has the job of taking BEL content that lives in a GitHub repository and making it easy to share with other people. It leverages pybel and tox to pre-compile BEL content and provide summaries that are auto-hosted by GitHub (when configured properly). The following is a short but growing list of some resources that are available through GitHub.

Selventa Large + Small Corpora

Along with their release of the OpenBEL Framework in 2012, Selventa published two files to GitHub that showcased the range of biological phenomena that could be expressed in BEL with a slight focus on atherosclerosis. They were excitingly named the “Selventa Small Corpus” and “Selventa Large Corpus”.

Because Selventa dissolved in 2016 and the belframework.org domain expired in mid-2019, the files have been updated and redistributed on GitHub at cthoyt/selventa-knowledge under the Creative Commons Attribution-Non-Commercial-ShareAlike 3.0 Unported License, same as the original. It can be installed in Python directly with:

pip install git+https://github.com/cthoyt/selventa-knowledge.git
selventa-knowledge summarize

It can be used in Python as a PyBEL BEL graph with:

import selventa_knowledge
selventa_graph = selventa_knowledge.get_graph()
selventa_graph.summarize()

EpiCom

This knowledge graph describes putative mechanisms involved in the pathogenesis of Epilepsy. It was published by Hoyt and Domingo-Fernández et al. (2018) in A systematic approach for identifying shared mechanisms in epilepsy and its comorbidities.

It is available on GitHub under neurommsig-epilepsy/neurommsig-epilepsy or can be installed directly with Python using:

pip install neurommsig-epilepsy
neurommsig-epilepsy summarize

After installation, it can be directly used with PyBEL like:

import neurommsig_epilepsy
epilepsy_graph = neurommsig_epilepsy.get_graph()
epilepsy_graph.summarize()

CONIB

The Curation of Neurodegeneration in BEL (CONIB) encodes biological phenomena related to tauopathies in the context of neurodegeneration.

It is available on GitHub under pharmacome/conib or can be installed directly with Python using:

pip install git+https://github.com/pharmacome/conib.git
conib summarize

After installation, it can be directly used with PyBEL like:

import conib
conib_graph = conib.get_graph()
conib_graph.summarize()

HemeKG

The Heme Knowledge Graph (HemeKG) encodes heme pathogenicity and pathway dysregulation in the context of hemolytic disorders. It was published by Humayun et al. in A computational approach for mapping heme biology in the context of hemolytic disorders.

It is available on GitHub under hemekg/hemekg or can be installed directly with Python using:

pip install hemekg
hemekg summarize

After installation, it can be directly used with PyBEL like:

import hemekg
heme_graph = hemekg.get_graph()
heme_graph.summarize()

COVID-19 Knowledge Graph

The COVID-19 Knowledge graph encodes biology surrounding the infection of the novel coronavirus. It was published by Domingo-Fernández et al. in COVID-19 Knowledge Graph: a computable, multi-modal, cause-and-effect knowledge model of COVID-19 pathophysiology.

It is available on GitHub under covid19kg/covid19kg or can be installed directly with Python using:

pip install git+https://github.com/covid19kg/covid19kg.git
covid19kg summarize

After installation, it can be directly used with PyBEL like:

import covid19kg
covid19_graph = covid19kg.get_graph()
covid19_graph.summarize()

Bio2BEL

The Bio2BEL ecosystem is built on top of PyBEL to convert as many reasonable structured data sources as possible into BEL, automatically. A list of most of the available repositories is available in the documentation. A few notable examples are included here. Notably, metadatabases like Pathway Commons will not be supported by Bio2BEL, because it’s almost always better to get the source data which is much higher granular. A pre-print describing the Bio2BEL philosophy and summarizing the content in it is available on bioRxiv.

Causal Biological Networks Database

The Causal Biological Networks database provides a swath of ground-truth biology. It has human, rat, and mouse networks that can be downloaded from the main page in a bespoke (and undocumented) JGF variant. Luckily, an importer can be found in the PyBEL documentation with instructions for unzipping the downloads and loading them into PyBEL with pybel.from_cbn_jgif_file().

The content has also been pre-processed and is available at https://github.com/pybel/cbn-bel

Hetionet

Hetionet is a large biomedical knowledge graphs created for drug repositioning.

Systematic integration of biomedical knowledge prioritizes drugs for repurposing Daniel S Himmelstein, Antoine Lizee, Christine Hessler, Leo Brueggeman, Sabrina L Chen, Dexter Hadley, Ari Green, Pouya Khankhanian, Sergio E Baranzini eLife (2017-09-22) DOI: 10.7554/eLife.26726

An export of Hetionet as BEL can be found at https://github.com/pybel/hetionet-bel.

OpenBioLink

OpenBioLink is a framework that produces biomedical knowledge graphs for link prediction.

OpenBioLink: A benchmarking framework for large-scale biomedical link prediction Breit, A., Ott, S., Agibetov, A., & Samwald, M. Bioinformatics (2020) DOI: 10.1093/bioinformatics/btaa274

KEGG, WikiPathways, and Reactome

The PathMe package converts KEGG, Reactome, and WikiPathways to BEL. Results may vary, because the variety of XML, BioPAX, and GPML/RDF used by the source databases changes all of the time, but this is a trove of content from other manual curation efforts. It was published by Domingo-Fernández et al in BMC Bioinformatics.

Leaving the PyBEL Ecosystem

While each repository distributes the source BEL scripts along with pre-compiled BEL JSON for easy use with PyBEL, some users may want to convert the content for other purposes. For example, you may want to upload it to one of many services that PyBEL interacts with.

Upload to BEL Commons

BEL Commons is a free, open-source platform for hosting BEL content. It includes the ability to interactively explore nodes, edges, and networks along with their associated metadata as well as enrich them with content from Bio2BEL.

Warning BEL Commons was originally developed and published in an academic capacity at Fraunhofer. While they hosted a public instance to support its original publication, it was permanently shut down in 2020.

import covid19kg
graph = covid19kg.get_graph()

import pybel
pybel.to_bel_commons(graph, host='...', user='...', password='...')

If you would like to host your own instance of BEL Commons, it can be downloaded from GitHub (bel-commons/bel-commons) and hosted easily with Docker. If you are hosting your own instance, all you have to do is use the host keyword. See the pybel.to_bel_commons() documentation.

Upload to BioDati Studio

Warning After a relatively short stint, BioDati was shut down some time in 2021, likely coinciding with the departure of William Hayes to go to Sage Therapeutics. Therefore, this section corresponds to infrastructure that no longer exists, but it will remain in this post for posterity.

BioDati is a paid, closed-source platform for hosting BEL content. However, they do have a demo instance running at https://studio.demo.biodati.com with which the examples in this module will be described.

import covid19kg
graph = covid19kg.get_graph()

import pybel
pybel.to_biodati(graph, host='https://nanopubstore.demo.biodati.com', user='...', password='...')

More information on uploading to BioDati can be found in the documentation of pybel.to_biodati().

Conversion to INDRA

INDRA is an excellent suite for automated model assembly developed in the Laboratory of Systems Pharmacology at Harvard Medical School. PyBEL can convert content to INDRA with the following code:

import covid19kg
graph = covid19kg.get_graph()

import pybel
statements = pybel.to_indra_statements(graph)

PyBEL has tons of other I/O formats, especially for analytical tools like HiPathia, SPIA, PyKEEN, and others that aren’t mentioned here, but can be found in its documentation here.

I hope you’re able to find some of this content useful! If you know about any other publicly available BEL content, please let me know, so I can update this post. All of my contact info is below. Or make a pull request against this page directly.

An Incomplete History of Selventa and the Biological Expression Language (BEL)

2020-04-28T08:00:00+00:00

The company and community that surround the Biological Expression Language (BEL) are enigmatic, to say the least. This post represents the best I could do to tell the history of Selventa and BEL.

If you’ve read my last few posts, you know that I’m making the best of quarantine time by being quite silly with the way that I’m talking about science. I have a habit of injecting opinion, but to tell the story of Selventa and the Biological Expression Language, I will try to refrain whenever possible. In the following, I chronicle the history of Selventa, the people who worked there, and the community that emerged from their work. It is not a complete history.

There are obvious things that I can not know about the inner workings of Selventa, despite the fact that I used to walk by their building on the way to my internship at Pfizer on Cambridge Park Drive between 2013-14. There are both things that I’ve learned through by word of mouth that I think are worth sharing and that I think are best considered gossip. There are things that I know because of my time and work at Fraunhofer that I’m not able to share due to non-disclosure agreements, too, though a secondary purpose of this post is to show off just how many people have been involved either directly or tangentially in this community, so I will try my best to share what I can. There were even a few cases where I found references to things I thought were under NDA on the internet, so I feel perfectly fine with sharing. I’ve put references to everything that can be qualified throughout this post.

There are also things that I’m aware of that I’ve chosen to exclude because of their lack of relevance, quality, impact, or contribution the community. For example, I have chosen to exclude some papers that have claimed to use BEL for modeling purposes, but have not shared their underlying knowledge graphs. I have also chosen to exclude papers that present new algorithms for BEL graphs that do not share code or examples.

Finally, I am not perfect and do not know everything. I’m certain I’ve missed something important, because it turns out that a lot of people have been working on BEL in the last twenty years. I would be happy to accept suggestions to add things. If a part of this post is about you and you think that I have portrayed you incorrectly, please get in touch. Finally, I plan on maintaining this post as time goes on and more cool things are published in this community. And so, we being on a dark and stormy night in 2001….

2001

2001 Selventa is founded by Noubar Afeyan as Genstruct, Inc. The concept was to create a computational complement to the company Beyond Genomics, Inc. - which was a systems biology company also founded by AGTC (a predecessor fund to Flagship).

2002

Summer 2002 Keith Elliston is hired by Flagship Ventures (later renamed Flagship Pioneering) to work with the founding team of Genstruct (Navin Chandra, Justin Sun, Ted Slater, Dexter Pratt) to find a technology and business model for the company. Keith was brought in by Jim Serum, who co-founded Viaken Systems with Elliston, and was on the board of Genstruct.

October 2002 Keith submits a funding proposal and business plan for Genstruct, based upon a new approach to using artificial intelligence for biological networks proposed by Navin Chandra and developed by the team. The board approves the plan, approves investment in Genstruct, and hires Keith as the CEO.

2003

January 2003 Genstruct does its first commercial project, a pilot project with Pfizer, where they successfully identified the mechanism of action of an unknown cancer drug. This leads to a more than 10 year collaboration between the companies.

Spring 2003. Biological Expression Language is created by Dexter Pratt, Navin Chandra, Keith Elliston, and Ted Slater. This development is based on work done by Navin at Perot Systems and Nets, work done by Dexter on CycL with Cycorp, and on biological ontologies by Keith and Ted.

August 20th, 2003 Genstruct files a patent that outlines many of the core ideas of modern systems biology years before they became mainstream. While it doesn’t mention BEL explicitly, it’s obvious that many of the ideas in this application became part of the BEL standard when it was later released to the public. The patent was accepted 8 years later in 2011. Thanks to Ted Slater for bringing attention to this!

September 8th, 2003 Genstruct raises $6.5M in a Series A with lead investors Flagship Pioneering and A.M. Pappas & Associates, during which they acquire the Pappas portfolio company Incellico, Inc. (led by John Wilbanks and Toby Segaran).

2004

2004 Toby Segaren, working with Justin Sun and other developers at the company, develop the first practical version of the Genstruct Inference Engine. The inference engine implements a reverse causal reasoning using a graph-based approach, and identifies upstream causes (mechanisms) for downstream observations (state changes). Bill Ladd develops the statistical methods that power the inference engine and that are used to evaluate the results of simulations.

February 24, 2004 Genstruct appoints Doug Lauffenberg, Director of the Biological Engineering Division and Uncas & Helen Whitaker Professor of Biological Engineering, Chemical Engineering and Biology at the Massachusetts Institute of Technology (MIT), to the board of directors.

November 2004 Genstruct and Pfizer extend and expand their partnership, applying the Genstruct platform to various R&D and Toxicology programs throughout Pfizer. (https://www.genomeweb.com/archive/genstruct-pfizer-expand-research-partnership#.X3jczJNKh24)

2006

August 2006 Genstruct and GSK extend their partnership to apply the Genstruct platform to define compound mechanisms of action in Oncology (https://www.fdanews.com/articles/61568-genstruct-extends-collaboration-in-cancer-with-glaxosmithkline)

Sometime in 2006 Gensruct reaches cashflow break even (ref). Keith Elliston’s LinkedIn profile claims that it was cash flow positive in 2008 and 2009.

2008

April 30, 2008 Genstruct and Sirtris win the BioIT World Best Practices Award for their work using the Genstruct Platform to identify the mechanisms of action for the Sirtris Sirt1 activators. Sirtris was acquired by GSK to further develop its Sirt1 activators.

July 2008 Genstruct begins a collaboration with Manuel Pietsch and his group on the use of the Genstruct technology to assess the stength and extent of toxicity using network analysis

2009

2009 Genstruct develops the “Network Perturbation Amplitude” algorithm, based on early work done by Jim Watters on pathway expression activation.

September 2009 Board appoints Chris Varma from Flagship Ventures as Executive Chairman.

2010

Jan 2010 Keith Elliston resigns from the company, Chris Varma named CEO (though this is missing from his LinkedIn profile).

May 17th, 2010 Genstruct raises $500K in a Venture Round. This is the second of three rounds of funding, the last of which will occur in late 2011.

May 2010 Genstruct hires David de Graaf as CSO.

July 2010 Chris Varma leaves Genstruct to join Third Rock Ventures.

November 10th, 2010 Genstruct is rebranded as Selventa (ref)

December 2010 David de Graaf named CEO.

2011

June 23rd, 2011 Selventa makes its first tweet from @selventa.

November 29th, 2011 Selventa raises $5M in its final Venture Round.

2012

April 27th, 2012 The OpenBEL Consortium begins and establishes http://openbel.org as a community resource (ref, written by Jordan Hourani on July 9th, 2012). There has been (and remains) great conflation between the name of the Biological Expression Language, the OpenBEL Framework (see next line), and the OpenBEL Consortium. Skipping ahead a few years: with the later deprecation and abandonment of the OpenBEL Framework, whose organization on GitHub also hosted the OpenBEL Consortium’s website, it became unclear how maintenance should proceed.

May 18th, 2012 Selventa makes their first public commit to their open source Java ecosystem, the OpenBEL Framework.

May 23rd, 2012 Kevin Davies writes Ring My BEL: Selventa Releases Biological Expression Language to be published on the Bio-IT World Website

May 23rd, 2012 OpenBEL joins Twitter @openbel and posts its first tweet, a re-tweet of a Selventa link to the previously mentioned Kevin Davies article. It’s not clear who was the author or who currently holds the credentials. It’s also not clear at this time whether the Twitter account was for the OpenBEL framework, or the OpenBEL Consortium, which Selventa would organize later that year.

May 31st, 2012 The first and second papers on the Network Perturbation Amplitude were published simultaneously in different journals, respectively authored by Florian Martin (Philip Morris International) and Julia Hoeng (Philip Morris international). Each paper contained several authors from both Philip Morris International (PMI) and Selventa.

June 18th, 2012 Selventa discloses that there is interest/collaboration with several pharmaceutical and software companies as well as academic, governmental and non-profit groups interested and plans to organize an external non-profit organization (The OpenBEL Consortium) to facilitate the community around the Biological Expression Language (ref 1, ref 2). This list includes Pfizer, Merck, Thomson Reuters (the department that was involved was later spun off into Clarivate Analytics to support the Metabase/Metacore), Fraunhofer, Harvard Medical School, IDBS (listed, but I wasn’t able to figure out who they were), Linguamatics, Entagen (since dissolved).

July 25th, 2012 The OpenBEL Google group is created. Through 2020, it remains a semi-active place for discussion in the BEL Community.

August 9th, 2012 David Fryburg, Selventa’s Chief Medical Officer from 2011-2015, authored a company profile in Future Medicine (paywall).

Fall 2012 Ted Slater, Selventa V.P. of Knowledge Engineering from 2002-2004 and later returning as CTO between 2012-2013, along with Dr. Diane H. Song, marketing, publish Biological Expression Language (BEL): Ringing In A Common Language For The Life Sciences in the Fall 2012 issue of Drug Discovery World as well as a companion piece Saved by the BEL - ringing in a common language for the life sciences

2013

August 26th, 2013 OpenBEL becomes a Linux Foundation Collaborative Project (ref 1, ref 2) as a first attempt at identifying external funding. I was unable to find evidence of when this ended, but I have heard from members at the time that it was ultimately unsuccessful and dissolved.

October 13th, 2013 Following the establishment of the sbv IMPROVER initiative by Philip Morris International and subsequent publications in 2011, 2012, and 2013, their first Network Verification Challenge was held between October 2013 and March 2014. It was published by Sam Ansari (PMI; with shared first authorship for all members sbv IMPROVER team) and marked the first use of BEL in the sbv IMPROVER (ref).

November 2nd, 2013 William Hayes, CTO of Selventa from 2012-2016, released the first version of the bel.rb Ruby package on rubygems. This likely marked the end of Selventa’s support for the OpenBEL Framework, as both Java was going out of style and their codebase had not aged well. However, it’s generally hard to tell when software projects are dead. The maintainers, Anthony Bargnesi and Nick Bargnesi, continued to make intermittent maintenance commits to the OpenBEL Framework’s codebase through June 24th, 2015.

November 23rd, 2013 Natalie Catlett (Selventa) and colleagues publish the Reverse Causal Reasoning algorithm in BMC Bioinformatics, this time with no co-authors from PMI.

2014

January 23rd, 2014 With the death of the OpenBEL Framework in sight, the @openbel Twitter account began to publicize the bel.rb Ruby package.

February 2014 Ted Slater publishes a review of BEL, Recent advances in modeling languages for pathway maps and computable biological networks which continues to serve as the appropriate paper to reference for the Biological Expression Language itself. When you skip ahead it might seem obvious that I’m collating information to put together a new reference paper describing the updates from the following six years.

July 11, 2014 Florian Martin (Philip Morris International) and colleagues published their third paper (I think; they have been quite prolific in the 2010’s) paper describing the Network Perturbation Amplitude analysis, this time with no co-authors from Selventa.

2015

Sometime between 2015-2017 With the withdrawn of support from Christoph Brockel (sometime between 2015 and 2017, when he left Pfizer), Pfizer divests from BEL. Its internal BEL-based analytical platform, the Causal Reasoning Engine and its underlying knowledgebase are publicized, but never released.

January 23rd, 2015 In concert with the sbv IMPROVER’s adoption of BEL from PMI, thefifth iteration of the BioCreative Challenge hosts its first BEL-specific text mining challenge. It was organized by OntoGene (Fabio Rinaldi), the sbv IMPROVER/PMI (Sam Ansari, Julia Hoeng), and Fraunhofer (Juliane Fluck, Martin Hofmann Apitius) following the footsteps of the sbvIMPROVER network verification challenge.

March 2015 David de Graaf steps out of CEO role at Genstruct, joins Flagship Venturelabs.

Sometime before April 2017 The second iteration of the sbv IMPROVER’s Network Verification Challenge was hosted with a focus on COPD. It’s not clear when this happened, so I’ll say before April 17th, 2015 because the CausalBioNet paper (see next bullet point) used the results. On May 15, 2015, Stéphanie Boué (PMI, with shared first authorship with the sbv IMPROVER team) published a summary of the challenge in F1000 Research.

April 17th, 2015 Stéphanie Boué (PMI) publishes the Causal Biological Networks Database (CausalBioNet) in Oxford Database as a summary of the results of the curation done in the second iteration of the sbv IMPROVER’s Network Verification Challenge. This is the first evidence I found of the participation of Anselmo Di Fabio’s company, Applied Dynamic Solutions (ADS), LLC, in the BEL Community, though the metadata listed on the paper’s page is wrong so it’s not obvious which co-authors had affiliations to that organization at the time, besides Anselmo. Later, William Hayes will join ADS after the dissolution of Selventa.

June 16th, 2015 Justyna Szostak (PMI) and Sumit Madan (Fraunhofer) publish the BELIEF text mining workflow following the fifth BioCreative challenge in Oxford Database. Here is another case where I omitted several other papers following the BioCreative challenge, as none of the other solutions were accessible. This is very, very sad in my opinon.

November 9th, 2015 Afroza Khanam Irin (Fraunhofer) publishes Computational Modelling Approaches on Epigenetic Factors in Neurodegenerative and Autoimmune Diseases and Their Mechanistic Analysis, which outlined a possible addition to the BEL specification to allow the codification of epigenetic modifications in BEL. Unfortunately, this proposal was not considered until the 2018 OpenBEL Consortium meeting, and it is still under a very slow debate.

Some time between 2015 and 2018 Luc Canard (Sanofi) became involved in Fraunhofer’s BEL activities throughout the AETIONOMY which cumulated in this publication.

2016

Sometime in 2016 Selventa dissolves (ref). I think this where this story gets interesting - because it’s also the part that we will be able to understand the least from an outside perspective. If you serch the internet for Selventa, you will indeed find lots of well-written press releases describing the contracts they had made over the years with several notable biotech and pharmaceutical industries. I’ve heard gossip that the reason it fell apart was because of mismanagement, but I can’t weigh in on that.

Later than sometime in 2016 The Selventa team disperses. Part of the technology team that supported the OpenBEL Framework moved to Applied Dynamic Solutions (ADS), LLC. Some of the computational biologists moved to PatientsLikeMe (PLM) (in waves), and I believe some of the computational team moved directly to Philip Morris International. Many continued working together, and with the industrial support for the BEL infrastructure in which they had invested, PMI patronized ADS to fill the void. Before its dissolution, Dexter Pratt had already moved to UCSD and begun work on the NDEx project. Ex-CEO Keith Ellison and current CEO David de Graaf continued their careers in VC and entrepreneurship. Luckily, we have LinkedIn to figure this kind of stuff out.

During its 15 years of operation, Selventa performed an enormous amount of curation to generate BEL content. I’m not sure what the actual number but I’ve heard that it has millions of edges in it. After the closure of Selventa, the intellectual property of the company was sold to Alexion. Dexter Pratt asked through the OpenBEL Google Group where it was, and Nimisha Schneider claimed that Alexion might own it now. I’m quite interested to know about the fate of this trove of curated content, as many of the Selventa papers alluded to its existence (but the reviewers didn’t seem to mind that they were publishing academic material while claiming industrial secrecy. I’m sorry to inject opinion here but I would rather we not have industrial publications than ones that can’t be reproduced.)

October 1st, 2016 Sumit Madan publishes the second and final publication on the BELIEF text mining pipeline. The lack of updates to this service and lack of further publications might lead the reader to believe the project is abandoned.

October 9th, 2016 Following a long hiatus in the development of open source software to support BEL, Charles Tapley Hoyt (Fraunhofer; that’s me!), Andrej Konotopez (Fraunhofer), and Christian Ebeling (Fraunhofer) release the first version of the PyBEL python package. It was later published in Oxford Bioinformatics. I may be biased, but I think this marked the beginning of the rejuvination of the BEL community. Many more developments from me and colleagues at Fraunhofer follow for the next 3 years through my master’s and doctoral work.

October 25, 2016 Christian Ebeling (Fraunhofer) presents PyBEL at the tranSMART Foundation 2016 Annual Meeting.

November 1st, 2016 Daniel Domingo Fernández (Fraunhofer) publishes the NeuroMMSig Web server in Oxford Bioinformatics, containing one of the first publicly available BEL knowledge graphs as well as one of the first publicly usable algorithms for BEL graphs.

Sometime in 2016 Cohen Veteran’s BioSciences contracts Fraunhofer to curate a knowledge graph for PTSD and TBI, supported by PyBEL and BEL Commons (ref). In addition, Exaptive developed additional software for visualization.

Sometime in 2016-2017 Boehringer Ingelheim contracts Fraunhofer to curate a knowledge graph for psychiatric conditions in the BEL4IMOCEDE project (ref 1, ref 2), supported by PyBEL and BEL Commons.

2017

January 24th, 2017 Asif Emon (Fraunhofer) publishes Using Drugs as Molecular Probes: A Computational Chemical Biology Approach in Neurodegenerative Diseases which jump started both the cheminformatics side of BEL and inspired the later Bio2BEL project.

February 22th, 2017 John Bachman (Harvard Medical School; HMS) and Ben Gyori (HMS) begin to integrate PyBEL into the INDRA project, divesting from a previous RDF dump of the Selventa Large Corpus whose provenance was untraceable (not sure about if this is true or not). INDRA was published in Molecular Systems Biology later that year.

February 26th, 2017 The sixth BioCreative Challenge hosts a text mining challenge for BEL. It was lead by Fraunhofer (Juliane Fluck, Sumit Madan, Martin Hofmann-Apitius) and Philip Morris International (Justyna Szostak). Again, almost all of the softwares published for this track did not include a demo.

May 22nd, 2017 The final version of bel.rb (v1.1.2) is released. The code remains unfunctional, putting an unofficial end to the bel.rb project. With the abandonment of the OpenBEL Framework and bel.rb, PyBEL remains the only open-source/user-facing BEL software (for a short time, see 2018).

June 11th, 2017 Charles Tapley Hoyt (Fraunhofer) deploys BEL Commons as the first interactive exploration tool for BEL following the abandonment of the OpenBEL Framework and Cytoscape tool. It is later published in Oxford Database and open source’d.

August 9th, 2017 Fraunhofer starts the Bio2BEL project open source on GitHub. This is a data and knowledge integration effort similar to Pathway Commons for BioPAX and Bio2RDF for RDF, but with a wider range of knowledge included and much greater focus on reproducibility and automation. It was later pre-printed but in late April 2020, has not yet been accepted for publication.

2018

January 30, 2018 BioDati, Inc. officially forms as a spin-off of ADS and announces that it is developing a product called BioDati Studio for BEL curation and visualization.

January 31, 2018 William Hayes (now of ADS/BioDati, Inc.) announces the launch of the BEL.bio website as a replacement for the OpenBEL website. It also announced the first release of their bel python package that would serve as a backend for their upcoming product.

While the announcement caused some confusion throughout the community as to whether OpenBEL was a site for the BEL community and whether it should be deprecated in favor of a new website advertizing another company’s product, some were happy to see leadership coming from an organization (be it academic or industrial) that would be able to commit to long term maintenance. Author note: as a recently started PhD student, I wasn’t in a position to support industrial usage of PyBEL. If you’ve ever worked with the industry, especially as a software developer, you know how needy they are. There were users from a certain company asking for help on a weekly basis until I offered the ultimatum that they should pay for this kind of consultancy. Ultimately, I appreciated William and Anselmo’s leadership from BioDati and saw the advantage in having several complementary software ecosystems. 2018 and 2019 years would be big ones for me and PyBEL, and our focus would diverge from the nominal curation interface and network visualization in BioDati.

February 23rd, 2018 Michaela Gündel (Fraunhofer) publishes the BEL2ABM workflow in Oxford Database, demonstrating that the use cases of BEL were evolving much further than Selventa and PMI’s published use cases.

May 14th, 2018 The 2018 OpenBEL Community Meeting occurs coincident to Bio-IT world in Boston, MA with stakeholders from PMI, Fraunhofer, BioDati, ADS/BioDati, Harvard Medical School, and several others. Together we nominated William Hayes (BioDati), Natalie Catlett (now at PatientsLikeMe), John Bachmann (Harvard Medical School), and Charles Tapley Hoyt (still me, at the time Fraunhofer) due to our mixed statuses in the industry and academy as well as our mixed roles as tool developers and tool users to serve as the BEL Language Committee going forwards. We agreed on guidelines for BEL Enhancement Proposals and published them at http://bep.bel.bio. Videos from this event are available at https://www.youtube.com/playlist?list=PLwXD2R4UjER0IfAQpqxOBkSe08gTPws41.

Also May 14th, 2018 Christian Ebeling (Fraunhofer) publishes an Atom) plugin for BEL syntax highlighting at https://atom.io/packages/language-bel.

June 4th, 2018 Dexter Pratt (on behalf of the Cytoscape Consortium) contracts Fraunhofer to improve interoperability between BEL and NDEx through the PyBEL framework. The results were posted to GitHub in their own repository but the utility of the CX format and NDEx interchange were eventually incorporated into the core of PyBEL.

June 27th, 2018 The third sbv IMPROVER Network Verification Challenge was held at PMI in Neuchatel, Switzerland. Further improvements were made to the CausalBioNet to investigate Xenobiotic metabolism and causal biological networks. The three winners were from Charité, University of Bonn, and the Swiss Institute of Bioinformatics - demonstrating further the reach of the BEL community and PMI’s excellent stewardship and engagement.

November 19th, 2018 Charles Tapley Hoyt (Fraunhofer) publishes the first version of a git-based workflow that uses Continuous Integration for writing BEL code in a team environment on GitHub. It is later used in Oxford Database with the rational enrichment workflow (see below).

December 10th, 2018 Following proposals and reviews submitted after the 2018 OpenBEL Community Meeting, William Hayes publishes the BEL v2.1 standard on behalf of the BEL Language Committee.

December 11th, 2018 Fraunhofer and Harvard Medical School jointly publish a workflow using INDRA and PyBEL for rationally enriching BEL graphs in Oxford Database available on GitHub (ref).

December 13th, 2018 Daniel Domingo-Fernández (Fraunhofer) publishes the ComPath pathway equivalence database and the ComPath web application in Nature as a first step towards unifying major public pathway databases in BEL. The source code and underlying data were published on GitHub.

2019

February 15th, 2019 Mehdi Ali (University of Bonn) publishes the BioKEEN machine learning package in Oxford Bioinformatics, introducing the BEL community to an entirely new type of qualitative analysis and hypothesis generation using BEL.

February 27th, 2019 Dénes Türei (EMBL, University of Heidelberg) makes the first commit towards integrating BEL with OmniPath.

September 3rd, 2019 After nearly a decade of non-reproducible papers, PMI publishes an open-source (albeit, R) implementation of their Network Perturbation Amplitude analysis in BMC Bioinformatics.

May 15th, 2019 Daniel Domingo-Fernández (Fraunhofer) publishes PathMe, the first software integrating KEGG, Reactome, and WikiPathways (in their variety of formats including XML, BioPAX, and GPML/RDF) as well as the accompanying PathMe web application in BMC Bioinformatics.

July 29th, 2019 At some point in 2019, the website (belframework.org) hosting the BEL resources necessary to all BEL files was allowed to expire (I think the transMart Foundation was paying for it and William was responsible, but I’m not sure). With this abandonment, previously written BEL files could no longer be compiled without a new resources server being deployed and the BEL files updated. Luckily, the website was being built from a repository on the OpenBEL GitHub organization, so only the files needed to be updated. The responsibility of maitenance of the Selventa Large Corpus and Selventa Small Corpus (previous released by Selventa under the CC-BY-3.0 license) was taken by Charles Tapley Hoyt (Fraunhofer) and moved to a new GitHub repository.

August 5th, 2019 Farah Humayun (University of Bonn) and Daniel Domingo-Fernández (Fraunhofer) make the first commit to the Heme Knowledge Graph (HemeKG). It is later published in Frontiers in Bioengineering and Biotechnology.

September 24, 2019 Charles Tapley Hoyt (Fraunhofer) announces the release of the BEL v2.2 specification on behalf of the BEL Language Committee.

December 9th, 2019 After a large curation project around neurodegenerative diseases and tauopathies lead by Stephan Gebel and Charles Tapley Hoyt, Fraunhofer makes its last public commit to the Curation of Neurodegeneration in BEL (CONIB) project before the the departure of Charles Tapley Hoyt following his PhD and interest in public curation in this project dwindled.

2020

March 4th, 2020 The Hetionet project adopts BEL as a distribution format for its integrative network suited for drug repositioning and target prioritization.

March 4th, 2020 The OpenBioLink project adopts BEL as a distribution format for benchmarking link prediction tasks on biological networks.

April 11th, 2020 Daniel Domingo-Fernández (Fraunhofer) releases the COVID19 disease map along with a pre-print on bioRxiv. Additionally, the paper serves as a reference for Fraunhofer’s new OrientDB instance that holds BEL and their Biomedical Knowledge Miner web application

April 3rd, 2020 Jeremy Zucker (Pacific Northwest National Labs) and students of Robert Ness/Olga Vitek (Northeastern; my alma mater!) begin developing pipelines for generating causal models (SCMs) from BEL graphs for modeling of COVID-19, resulting in further interest in BEL in the CoronaWhy working group.

May 9th, 2020 The OpenBEL community website, https://openbel.org is taken down.

There are a few things that I would like to mention as afterthoughts that I don’t know where to place on the timeline.

One of the most egregious ommisions I have made is the date of the BEL 2.0 release and the events that lead up to it. Even crazier, I don’t know much of the pre-2016 history of how my group at Fraunhofer got involved with Selventa - perhaps it was their long history of text mining (since the dictionary and CRF days) that got these groups together. If someone has that information, I would be really glad to include it here.

During my time at Fraunhofer, there were a lot of people working with BEL. Many of them made tools and algorithms that never got published, so unfortunately, they are not included in this history.

The Chemotoxicogenomics Database had been converted to BEL by Thomas Weigers a long time ago, when the XML BEL format existed (another thing that I think wasn’t worth bringing). I corresponded with him about it when I was at Fraunhofer, but unfortunately don’t have access my the emails anymore to double check exactly what we talked about. He did send me the database as XBEL which he said he made with a script he wrote but didn’t have anymore. Ultimately, I decided to re-write the converter to play nicer in the Bio2BEL ecosystem, which worked for a while and then broke because its downstream dependency for parsing the database wasn’t updated. I never got around to re-writing this again.

I’m not sure what happened at PatientsLikeMe, but from LinkedIn I can tell that there was a mass migration of ex-Selventa to Quartz Bio, who also appear to be hiring BEL people to join their team. After burning out from finishing my PhD in late 2019, I haven’t been proactive about keeping up with many people (quarantine life isn’t making me feel that kind of motivation either, which is somehow categorically different from the entire day I spent researching and preparing this blog post), though I do personally know some of them and could ask…

One of the other peculiarities about the history of BEL is the adjacent history of BioPAX. This happened way before my time, so I wonder why they diverged so much. I think that they’re ultimately trying to accomplish the same thing, which is to be a place for putting structured information (and modeling, to an extent). It’s most definitely the case that BioPAX has achieved greater popularity and penetration, due to (in my opinion) the excellent labs that are backing the standard and the fire hydrant of high impact papers coming out of them. However, I think this success might be holding BioPAX back, because many of these papers are simply pulling content from Pathway Commons to do gene set enrichment analysis. Now, if you’re working adjacent to an oncology unit in a hospital, then this is all you need to do to get results good results, write high impact papers, and ultimately help patients. Don’t get me wrong - I’m much happier to see papers that achieve their scientific goals using simple methods. I just think there’s much more potential for BioPAX. I think we’re seeing that movement happen for BEL already.

Then there’s that idea of converting between BioPAX and BEL. It’s sort of a non-starter, since all BioPAX is encoded differently - just because data is able to be stored in a given format (e.g., ontology deriving from BioPAX, RDF, etc.), it doesn’t necessarily mean that the content can be put together with other content in the same format in a meaningful way.

As I close, I will again acknowledge my biases. I’m quite proud of the work I did during my master’s and doctoral work at Fraunhofer. I’m thankful for all the people who were interested in my projects, contributed to them, and then joined me as co-authors on my publications. When it came to writing this history, I was in a situation where I had lots of high granular information to share on the things that I worked on and also the the desire to share as much of it as possible. I hope I did a good enough job at laying out the landscape of the other things going on outside my perspective.

If you’ve got something to add, all of my contact information is available on the footer of my blog. Or make a pull request against this page directly. Special thanks to Keith Ellison for suggesting these changes.

How to Code with Me - Flake8 Hell

2020-04-25T08:00:00+00:00

As scientists, we place huge importance on the communication of our results. We spend lots of time on editing, revising, and formatting so people can understand what we did. We also write a lot of code, so why aren’t we investing the same amount of love? Enter, flake8.

It’s incredibly important that we write following community standards so when other people read our work, they don’t have to think about how it’s organized. For scientific prose, this usually means the IMRD (introduction-methods-results-discussion) format. In Python, my current favorite programming language for science, this means using a standardized number of spaces for indents (4), using triple-double quotes for docstrings in the beginning of each module, class, and function, and lots more.

It’s pretty intimidating to figure out style. For english prose, Strunk and White wrote The Elements of Style. For Python, Guido van Rossum wrote PEP-8 and Raymond Hettinger presented Beyond PEP-8. Even with these resources, it’s still hard to learn which are rules and which are more like guidelines.

This post is a short explanation of how I use flake8 to keep a consistent style in the code in my Python projects. There’s a similar command line tool for fixing the style in R projects that’s already built into most operating systems - rm -rf *, but I won’t get more into that here.

It’s pretty easy to get up and running with flake8 - just run pip install flake8 then use it from the shell on a python file like flake8 my_file.py or flake8 my_directory/. Then, it outputs a list of problems that need to be fixed on a line-by-line basis in your code.

You can also install plugins with pip like that extend the kinds of things it checks. A few that I install are:

flake8-builtins - make sure you don’t accidentally name a variable the same thing as a builtin. This happens a lot with id.
flake8-bugbear - “find likely bugs and design problems in your program”, like when you have an unused variable in a loop
flake8-colors - add color to the flake8 output (explanation how to set up is below)
flake8-commas - add trailing commas where appropriate
flake8-comprehensions reminders to use list comprehensions where appropriate
flake8-docstrings - make sure your docstrings are present and written in the right format
flake8-import-order - make sure your imports are organized properly
flake8-print - make sure you never ever ever use print(). The literal only exception is when using print to get text into a file with print(..., file=...)
flake8-use-fstring - make sure you’re using f-strings instead of % or .format() formatting. Exception being for logging.
pep8-naming - make sure names of variables, classes, and modules look right.
pydocstyle - docstring style checker

In each of my repositories, I put all of the information on how to install flake8 and its plugins then run them in a tox configuration under the [testenv:flake8] header so they can easily reproducibly run with tox -e flake8. An example of part of one of my tox.ini files (which always lives in the root of the repository) is below:

[testenv:flake8]
skip_install = true
deps =
    flake8
    flake8-bandit
    flake8-builtins
    flake8-bugbear
    flake8-colors
    flake8-commas
    flake8-comprehensions
    flake8-docstrings
    flake8-import-order
    flake8-print
    flake8-use-fstring
    pep8-naming
    pydocstyle
commands =
    flake8 src/pybel/ tests/ setup.py
description = Run the flake8 tool with several plugins (bandit, docstrings, import order, pep8 naming).

Another configuration file you can set up in the root of the repository is .flake8. Unfortunately, the Python configuration file reader doesn’t allow some of the crazy characters that I want to use for the colors so this can’t be included in setup.cfg or tox.ini like most of your other configuration.

[flake8]
ignore =
    # line break before binary operator
    W503
exclude =
    .tox,
    .git,
    __pycache__,
    docs/source/conf.py,
    build,
    dist,
    tests/fixtures/*,
    *.pyc,
    *.egg-info,
    .cache,
    .eggs
max-line-length = 120
import-order-style = pycharm
application-import-names =
    pybel
    bel_resources
    tests
format = ${cyan}%(path)s${reset}:${yellow_bold}%(row)d${reset}:${green_bold}%(col)d${reset}: ${red_bold}%(code)s${reset} %(text)s

First thing you’ll notice is the ignore list. This isn’t here to turn flake8 off because you’re feeling lazy. If somebody includes a change in this list in their PR, you have to explain to them that compliance is not optional, then help them work through the problem that they obviously gave up on solving. It’s actually there for you, as the project maintainer, to enumerate the flake8 rules that you don’t agree with. For example, I totally disagree with the W503 line break before operator rule. I want to write long conditionals with and statements on the first line, like this:

if (
   condition_1
   and condition_2
   and condition_3
):
    print('all true')

One of the benefits of this style is you can add more lines with only single line diffs. The other is that the reader always sees the operation that goes with each line. Same could be done with arithmatic that could incorporate not only + but also -.

Next is the exclude block. Just copy/paste this each time, since it has lots of garbage you don’t want flake8 to bother with. One of the checkers in flake8 is for function “cyclomatic” complexity. You can make the maximum number higher with max-complexity. Normally, you want this to be enforced, but sometimes there’s no way around a complex function. For this, you can add a code comment noqa followed by the error code like # noqa:W123. Again, adding tags to ignore bad style just to pass flake8 is against the point.

The max-line-length is a very contentious setting. I think 120 is fine. Some people think 78, 79, or 80 is best because of the standard sizes of old computer screens or punch cards… When I get older and I can’t read my computer screen, I’ll probably make the text bigger and change my mind about this. If you find yourself breaking up lines in a totally non-sensical, unstyled way, then you’re conforming too tightly to the rules. Sorry about the mixed messages!

import-order-style = pycharm
application-import-names =
    pybel
    bel_resources
    tests

I copied this again because this part is really important. You have to tell flake8 what rules you use for import order. I use the pycharm rules, which group python builtin packages, then 3rd party packages, then my packages. The application-import-names is a place to list what are your packages.

Last is the format entry, which gives the nice colorful output. Copy paste this! I borrowed mine from Scott Colby.

After all of that, I set up Travis CI to run tox every time code is pushed to the repository. If you’re working in a team, you probably do something like the fork/pull request or branch/pull request workflow on GitHub to support doing code review before merging new code. The best part is that there’s a big box on each pull request that checks if flake8 passed (among other tests), which means that there were no errors detected.

I encourage my teammates to make pull requests as soon as they start working on code. GitHub even has a “draft pull request” mode now. However, before asking anyone to review your code, it has to pass flake8. And obviously, no code that isn’t passing flake8 can be merged.

This is a very painful process to get people used to. I’ve done it with many groups of people and always got pushback. However, everyone who has gone through the process with me has come out the other side happy that they did it. It’s important that when you start enforcing coding rules on other people that you are a resource for them - when somebody is frustrated by a flake8 error code they have never seen, they will likely forget how to use Google. They will probably ask you for help. You have to resist the urge to send lmgtfy links to them and be patient. Because eventually, they will do it on their own, and spread the gospel of flake8.

While a good arsenal of flake8 plugins provides a solid foundation, it’s not all that needs to be done to make your code readable and look good. Just like with reading and speaking, the best way to develop a sense of style is by reading lots of code (with the caveat that reading poorly written code probably won’t teach you much). Within the rules imposed by flake8, there is lots of space for style. If you watch lectures from David Beazley, you’ll notice a very different style from Raymond Hettinger, and also from me.

Now that you’ve made it to the end of this short guide, I wish you the best of luck on developing your own style!

Are you working with people who are particularly unsusceptible to Travis CI emails or checking the big red box on pull requests? You could try getting them set up with pre-commit hooks, which run the style checks locally any time they try and push (even if it’s to a branch) and it will give them the message in the console.

Is style not your thing at all / you’re not ready to let go of your identity as a Java/Perl developer? Maybe consider Black, which actually re-writes your code in a deterministic style. I don’t live by it, but it’s a great tool to run on a code base that’s never been loved before going back and stylizing it.

Inspector Javert’s Xref Database

2020-04-19T08:00:00+00:00

On top the issue of resolving identifiers to their names, the bioinformatics community has a hard time figuring out when two identifiers from different databases are equivalent. You know who else has the same problem? Inspector Javert. Get ready for a Les Miserables-themed post on how to address this long-standing problem.

I have to start my tale of woes by disclosing my source material. I loved both the 1985 and 1987 recordings from the respective original London and Broadway casts. But, for the purposes of this post, I will assume that you’ve seen the excellent 2012 film adaptation of Alain Boublil, Jean-Marc Natel, and Herbert Kretzmer’s musical adaptation of Victor Hugo’s novel Les Miserables and tell the story through that perspective. I also want to you to know that I enjoyed Russell Crowe’s Inspector Javert very much.

Les Miserables begins with the Work Song, in which the protagonist, prisoner:24601 is confronted by Inspector Javert while doing some… work. He insists he has a name, Jean Valjean and his identifier in his home village’s fictional database (that I just retconned) was faverolles:2468. Javert isn’t interested in his name. It’s enough that he has a cross-reference between faverolles:2468 is equivalent to prisoner:24601. He was only there to inform Jean Valjean that his parole has begun and issues him a passeport jaune (yellow ticket) for the commune of Pontarlier.

I’m sure this passport also had an identifier on it. I’m going to take a bit of creative freedom and say it was pontarlier:25791. It probably also had Jean Valjean’s prisoner number on it so everybody knew he was in the 1800’s fictional French convict database. The fictional 1800’s French took maintaining cross-references very seriously.

Jean Valjean never made it to Pontarlier. Instead, he broke his parole, forged some new documents, and went to Montreuil-sur-Mer under the new name of Monsieur Madeleine. It’s probably the case that his identifier for the Montreuil-sur-Mer city database was montreuil-sur-mer:1357, or something like this (more retcons!). It must have been a good fake, because even the king of France recognized him (note: this plot point did not appear in the film).

Javert figured out Jean Valjean broke his parole basically immediately and set out on his quest to find and capture prisoner:24601 once again. Until this point, Javert has access to the prisoner registry and yellow tickets. He knows prisoner:24601 is the same as both faverolles:2468 and pontarlier:25791.

The part that will hit close to home for many bioinformaticians is that when Javert goes to Montreuil-sur-Mer, he meets Monsieur Madeleine. He is unaware that it is Jean Valjean. There is no cross-reference between faverolles:2468 and pontarlier:25791 or montreuil-sur-mer:1357. If there were a cross-reference in the fictional French 1800’s inspector database, Javert could have arrested Jean Valjean on sight. Instead, Javert had to the hard work of curating cross-references himself and finding out who was the same in the montreuil-sur-mer database as prisoner:24601. Admittedly, he probably would have called this inspecting.

The next part that will hit even closer to home for many bioinformaticians is that after his inspecting, Javert actually identified the wrong guy! This lead to one of the my favorite songs in musical theater ever (Who Am I?), where Monsieur Madeleine (montreuil-sur-mer:1357, also actually Jean Valjean, but Javert didn’t yet realize this) admits that he is actually prisoner:24601. In this extended metaphor of a blog post, Jean Valjean’s confession in “Who Am I” is effectively the same as a database providing its own cross-references to other database. Would be nice if everyone did this, and did it well, huh?

You should know that Javert is a powerful cross-reference reasoning machine. He already knew faverolles:2468 was the same as prisoner:24601. Now he knew that montreuil-sur-mer:1357 was the same as prisoner:24601. This way, he could infer that montreuil-sur-mer:1357 (Monsieur Madeleine) is actually faverolles:2468 (Jean Valjean). One of the nice properties of cross-references is that they’re transitive through any number of connections. We’ll take advantage of this fact later. You’ll also have to excuse the fact that throughout this post, I’m operating under the assumption that “cross-references” and “equivalences” are the same thing. That’s not always true, and sometimes it can even get you in trouble. For example, provenance can be a cross-reference, disease-gene associations are considererd as cross-references in MONDO (I think), and OBO even gives specific semantics for when you should consider this assumption valid. We’ll just have to live with it for now.

Javert might have got lucky that Jean Valjean revealed himself once, but the show must go on! Jean Valjean had many more songs to sing and thus had to escape from Montreuil-sur-Mer to Paris. This meant that Javert has to find another mapping to Jean Valjean’s new paris identifier. And we already know that the French 1800’s inspector database of cross-references was not being maintained. Exhausting!

In the bioinformatics community, we have a very similar problem to Inspector Javert. There are lots of databases that are talking about the same things, but only a few of them provide mappings between each other. This means that we either have to curate our own cross-references, do our best to infer new cross-references based on ones we already have, or throw our hands in the air.

Luckily, we have a few standardized resources to fall back on. In addition to standardizing the storage of identifier/name pairs, the OBO format standardizes the way cross-references are stored and the OBO Foundry already contains quite a few cross-references imported from the ontologies that it covers.

One of the most difficult entity types to map from database to database are phenotypes because of the variety of language used to describe each, the differences in semantics of how each is defined, and the sheer number of databases. Unfortunately, some of the most popular like MeSH and to an extent, UMLS, NCIT, SNOMED-CT, and ICD (seemingly the culprits are mostly American!?) provide very little accessible information. Some are even paid, so the ony cross-references that exist are externally curated ones from other laudable sources like HP, DOID, and EFO. In fact, dealing with phenotypes is such a pain, that there is a project called the Monarch Initiative that has a huge staff trying to solve exactly this problem and publish the results through the Monarch Disease Ontology (MONDO). Normally, I would reference this XKCD comic about making new standards when hearing about something like this. But these are dire times, and one of my opinions is that you should always trust curators who love what they do.

There are also lots of cross-references available from databases that don’t maintain their nomenclature as an ontology. One example is BioGRID, which assigns proteins internal accession numbers, but almost all of them cross-reference out to Entrez Gene (I counted less than 15 that didn’t, and 3 of them are COVID-related, so cut them some slack). As an aside, I don’t really understand why BioGRID would go through the effort of maintaining their own accession numbers. In the literal handful of cases where they can’t reference Entrez Gene, I think it would be better to email the maintainers and work with them to make improvements.

It’s also worth noting that excellent resources like HGNC, MGI, RGD, SGD, Ensembl, UniProt, and others in the genome (and gene product) nomenclature do a stellar job at maintaining cross-references. So to all of the curators and maintainers who work there, I would like to sincerely thank you.

There are also community-curated cross-references sources. One of the notable ones is from Harvard Medical School, that’s mapping MeSH identifiers to gene identifiers in the Gilda GitHub repository. I think this is really a good time to point out that MeSH contains a bit of everything, is ubiquitous throughout the bioinformatics community, and in my opnion is is doing a huge disservice by not providing these kinds of mappings itself. Or, alternatively, it is, and both the Harvard guys and I have never found it. It’s not impossible, but we’re all very motivated, so I think we would have found if it did. If any MeSH maintainers are reading this and want help making this happen, I would be elated to donate my time to you to help solve this problem.

With all these data source in mind, I built an extensible pipeline in PyOBO for extracting cross-references from entries in OBO Foundry and other cross-reference sources. Throughout the process, I realized that these sources have an incredible variety in how they name prefixes and how the OBO format itself has been (ab)used. I made lots of improvements, wrote extensible code that allowed the specification of new rules through external files (and thus less code writing in the future), and did lots more curation. I won’t get into the technical part of that here, since you can read the code (if you dare).

After all that this coding, I wrote a script (just run obo javerts-xrefs) that takes all available cross-references, normalizes their namespaces, normalizes their identifiers, and dumps them in a big ‘ol TSV file. 5 columns - source namespace, source identifier, target namespace, target identifier, and provenance (ontology name or URL). No nonsense. Get it at . I’ll make updates periodically as I add new sources.

Once you have a database of cross-references, you have actually built an undirected graph. Equivalences go both ways, and they are transitive. This means that every connected component in an equivalence graph represents a set of entities that are mutually equivalent. In other words, if a path exists between two nodes in an equivalent graph, then they are equivalent.

Even better, you don’t have to materialize all of the possible inferred equivalences when you have an equivalence graph because identifying all of the nodes in a connected component can be done in linear time with respect to the size of the connected component, which is usually pretty small, by using a breadth-first or depth-first search.

Based off of that, one application of an equivalence graph is to identify all of the nodes that are equivalent to a given node. You can also get a little tricker and identify the paths through which the traversal must go if you want to establish an equivalency. You could even go further and weight edges based on how much you trust the source from which they came to identify how much you should believe in a mapping. For example, if you have a percent confidence in each mapping being right, then the confidence in the whole pathway would be the product of the confidences.

The actual problem I set out to solve was given a set of entities, remap all of them based on a prioritized list. For example, I might have a set of entities that contains HGNC genes, Entrez Genes, and OMIM genes. If my favorite nomenclature consortium is Entrez, my second favorite is HGNC, and my third favorite is OMIM and I have an equivalence database, I might want to remap all of my identifiers. This is very important during the curation of mechanistic biology (such as with BEL), since curators will likely use all sorts of identifiers with no clear guidelines or rules. This means that the same entity might appear twice with different identifiers in the same curated data!

Given a priority list, you can even transform an equivalence graph into a directed graph where each identifier has a single out edge pointing towards the identifier that is the best mapping. Then, each connected component would become a star graph. There’s actually a better data structure for this, since each entity points to exactly one thing - a mapping. This is a more efficient data structure for storage, and if your graph is implemented as an adjacency dictionary (becuase you’re using networkx, right?), then you basically already have this.

I’ve provided an implementation for all of these in PyOBO. They can be run as a web API with python -m pyobo.apps.mapper. There’s a keyword argument to allow you to load the TSV from Inspector Javert’s Xref Database directly, or if you’re feeling lucky, to regenerate it yourself. Below I will give a few examples of how to use it. Later, I would also like to host this service for anyone to use.

Install PyOBO with pip install git+https://github.com/pyobo/pyobo.git
Download Inspector Javert’s Xref Database from Zenodo, unpack it, and find the xrefs file.
Run the web service with python -m pyobo.apps.mapper -x inspector_javerts_xrefs.tsv.gz
Use the following code to figure stuff out!

import requests

# Get all entities mapped to MAPT, including through chains of xrefs
successful_request = requests.get('http://localhost:5000/mappings/hgnc:6893').json()
"""
{
    "orphanet:123144": [
        {
            "provenance": "hgnc",
            "source": "hgnc:6893",
            "target": "orphanet:123144"
        }
    ],
    "pr:P10636": [
        {
            "provenance": "hgnc",
            "source": "hgnc:6893",
            "target": "uniprot:P10636"
        },
        {
            "provenance": "pr",
            "source": "uniprot:P10636",
            "target": "pr:P10636"
        }
    ],
    ...
}
"""

# Keep in mind this isn't a validation service
unsuccessful_request = requests.get('http://localhost:5000/mappings/hgnc:0000').json()
# {"message": "could not find curie", "query": {"curie": "hgnc:0000"}, "success": False}

# Get all paths mapping MAPT in HGNC to Ensembl. Returns a list of paths (which are lists of xrefs)
path_request = requests.get('http://localhost:5000/mappings/hgnc:6893/ensembl:ENSG00000186868').json()
"""
[
    [
        {
            "provenance": "hgnc",
            "source": "hgnc:6893",
            "target": "ensembl:ENSG00000186868"
        }
    ]
]
"""

# Get the priority identifier for MAPT identified by Ensembl
prioritize_request = requests.get('http://localhost:5000/prioritize/cosmic:MAPT').json()
# {"found": True, "query": "cosmic:MAPT", "result": "hgnc:6893"}

# What happens when a CURIE can't be found for prioritization
unsuccessful_prioritize_request = requests.get('http://localhost:5000/prioritize/cosmic:NOPE').json()
# {"found": False, "query": "cosmic:NOPE"}

I’d like to give a big thanks to my high school music teacher, Ken Tedeschi, for helping me (and basically everyone else) fall in love with Les Mis in high school. Writing about my work was so much more fun in extended metaphor. I would also like to thank Hugh Jackman. You know, for being Hugh Jackman.

I have some random afterthoughts that I think might be worth including, that I’m adding after originally posting this.

You might be wondering why I didn’t get into a discussion about the Ontology Mapping Service (OXO) from the EBI. It looks to me like this project has been abandoned. Even if not, it’s API has most of the same issues that I described in a previous post.

I’m also aware of BridgeDB, from which I think I will be able to take inspiration to include more xrefs later. However, I think they’re limited in scope, and PyOBO is more about standardizing data so nobody has to figure out databases… again and again and again.

One glaring omission from this work is WikiData mappings. I have a plan to include curated information in the PyOBO metaregistry that links databases to their WikiData properties. That will allow me to build an automated framework for downloading these mappings, given the curation of the properties.

Ooh Na Na, What’s My Name?

2020-04-18T08:00:00+00:00

We have a big problem in the bioinformatics community with namespaces, identifiers, and names. And nobody’s posed the question better than Rihanna herself.

During my Ph.D. at Fraunhofer, one of the old text miners reminisced to me about the late 90’s and early naughties when they had to curate their own dictionaries of synonyms for entities. I was lucky enough to have joined the bioinformatics community after excellent nomenclature resources like ChEBI and the HGNC were established and accepted by the community as gospel.

I consider these sources excellent because it’s quite easy to get a list of the identifiers and corresponding names that they maintain (TSV, etc.). There are other nomenclatures, like the ExPASy Enzyme Classes, that are stored as text files in non-standard formats.

The Open Biomedical Ontology (OBO) format and OBO Foundry were first published in 2007 as a solution for standardizing a growing set of biomedical ontologies that few shared semantics. Many ontology maintainers adopted their format, or at least used the OWL to OBO converter tools to include their ontologies in a reusable format. However, there remain some notable holdouts like the Cell Line Ontology that have not begun to distribute their content as OBO.

In parallel, the Ontology Lookup Service (OLS) was published as one of many front-ends for exploring this growing list of resources. In comparison, it may have been one of the first tools to provide a nice user experience that included a search engine (powered by solr, because they’re living in the Java world).

Both are lacking - there does not exist a solid OBO ecosystem (though Martin Larralde’s pronto may well soon change that) and even worse, the content in OBO loosely follows the standard, at best. On the other hand, the OLS has both an over-engineered interface that isn’t quite user friendly. For example, if you want to look up programmed cell death (GO:0012501), you have to know the internal OLS key for the namespace and the PURL for the identifier, which is not so obvious. Then you can finally hit the API.

And still, both of them lack some of my favorite, and arguably most important namespaces, like HGNC, RGD, MGI, UniProt, Entrez Gene, and PubChem. As an aside, dealing with PubChem is for people operating on a whole different level, so I’m not blaming anyone for dropping the ball on that one. Later, I will confess to doing the same.

Even worse, the OBO Foundry and OLS can’t even agree on what to call some namespaces. A great example is the NCBI taxonomy database. On the NCBI site, they say that the namespace is called NCBI and compact uniform identifiers (CURIEs) should look like NCBI:txid175694, OBO Foundry says the namespace is NCBITaxon (one of the few notable mixed-case namespace names) and CURIEs should look like NCBITaxon:175694.

Identifiers.org came along to solve some of these ambiguities with a curated database, but it’s missing lots of the things in OBO Foundry and OLS, and it even disagrees on others. They call the NCBI taxonomy namespace taxonomy and say that identifiers should look like taxonomy:175694. Exhausting!

One more issue is the GOGO problem. Many OBO ontologies use local identifiers that also include the prefix because a given ontology might contain terms imported from other ones. However, this means that ontologies that originated from the OBO world have redundant identifiers, like from GO (e.g., GO:GO:0012501). I know what you’re wondering: is Dr. Claw in charge? Maybe.

The reason I went down this rabbit hole is because I want to support people to do better curation. This means I want them to use identifiers instead of ever changing names. For example, it turns out the half life of an HGNC gene symbol is very short - thousands of them change every year. However, if I want people to use identifiers instead of names in their databases, their papers, and other writing, there need to be really good tools for looking up the names that go with each identifier and the cross-references (equivalences) to other databases that are talking about the same thing.

So I built PyOBO. It includes tools for reading the OBO Foundry and getting all of the OBO resources that are available (as well as many manual fixes for incorrect metadata), it uses Daniel Himmelstein’s Obonet for parsing and storing pre-parsed files for fast loading, and it applies a swath of rule-based normalization that I’ve manually curated by personally reading all of the OBO files, their identifiers, their cross-references, relationships, properties, and everything else. When it comes to data, there really is no way around getting your hands dirty.

I also went ahead and wrote parsers and converters for lots of other databases like Entrez, ComplexPortal, InterPro, and others so they could play nice with the rest of the ecosystem. Of course, this is an ongoing process. There are always more databases to include, and when it comes to super-sized ones like PubChem, the paradigms I used might not hold up anymore (though I did write parser/converter for it and you’re welcome to use it).

After this long journey of a blog post, I think we’re ready to address Rihanna’s perrenial question: what’s my name? Until now, there really didn’t exist a service that let you look up the name for an entity by its CURIE. The link I gave for the OLS is the closest I have found, and that just doesn’t cut it.

After all of this coding, I wrote a script (just run obo ooh-na-na) that takes all of the available sources, normalizes their namespaces, normalizes their identifiers, and dumps them as a big ‘ol TSV file. 3 columns - namespace, identifier, and name. No nonsense. Probably legal! Get it at . I’ll make updates periodically as I add more sources, such as if/when I feel comfortable with including the PubChem dump - the CID-Title.gz file is about 1.3 gigabytes, which means this will significantly increase the size, but not so much that it’s unreasonable.

I can imagine that most people probably won’t want to download this file, or load it in memory (un-gzipped) every time they want to use it. I wrote a simple web service that wraps this dataset included in PyOBO. It should be as easy as running with the shell with python -m pyobo.apps.resolver then running the following python code:

import requests

# This is an exact match
successful_request = requests.get('http://localhost:5000/resolve/DOID:14330').json()
# {"identifier": "14330", "name": "Parkinson's disease", "prefix": "doid", "query": "DOID:14330", "success": True}

# This one remaps the prefix if you get it slightly wrong
successful_remapped_request = requests.get('http://localhost:5000/resolve/DO:14330').json()
# {"identifier": "14330", "name": "Parkinson's disease", "prefix": "doid", "query": "DO:14330", "success": True}

# This one can't find the identifier.
unsuccessful_request = requests.get('http://localhost:5000/resolve/DO:00000').json()
# {"identifier": "00000", "message": "Could not look up identifier", "prefix": "doid", "query": "DO:00000", "success": False}

# Keep in mind, the point of this service isn't to validate identifiers.
unsuccessful_crazy_request = requests.get('http://localhost:5000/resolve/DO:thisIsNotRightAtAll').json()
# {"identifier": "thisIsNotRightAtAll", "message": "Could not look up identifier", "prefix": "doid", "query": "DO:thisIsNotRightAtAll", "success": False}

# No mercy for bad prefixes
unsuccessful_prefix_lookup = requests.get('http://localhost:5000/resolve/notanamespace:0000').json()
# {"message": "Could not identify prefix", "query": "notanamespace:0000", "success": False}

It’s especially important that the service normalizes curies first, so both DOID:14330, doid:14330, and DO:14330 can all be resolved to their name, Parkinson’s disease. Because I did extensive manual curation of namespaces and their synonyms, NCBITaxon and taxonomy are both acceptable as well. However, this service doesn’t load from the aforementioned TSV, but rather takes advantage of PyOBO’s internal code for looking up mappings. I can imagine lots of ways I might re-write this service to directly take advantage of this dump (I also invite you to do the same, however best suits you) such as loading it into EdgeDB and auto-generating a GraphQL endpoint.

The last thing that I’m looking into getting this service hosted so everyone can benefit from it without doing dev-ops in their own organizations. Then I will continue to obfuscate all usage and documentation with references to pop culture. Enjoy!

Summarizing ChemRxiv

2020-04-15T08:00:00+00:00

A few months ago, the question was posed on science Twitter: “How many people have published on ChemRxiv?”

makes me wonder about the stats at @ChemRxiv https://t.co/Ml5X8F4ckJ
— Egon Willighⓐgen (@egonwillighagen) January 20, 2020

It was a good day for me, which meant I was in the mood to take up the first challenged posed on Twitter. I found that François-Xavier Coudert (@fxcoudert) has previously written a python client for ChemRxiv. I made a pair of pull requests (fxcoudert/tools#9 and fxcoudert/tools#10) to fix some bugs and make it importable from other python modules.

Unlike BioRxiv, the pre-print server for biology, ChemRxiv is implemented with FigShare. It turns out that all FigShare “institutions” like ChemRxiv are actually accessible through the main FigShare API. I think this is pretty cool, and made sure that the ChemRxiv client that I had updated was actually able to be run for any institution. Fun fact: the institution code for ChemRxiv is 259.

I got to work writing my own repository to wrap the client, take care of downloading all of the bibliographic information available, and generating some pretty pictures. I originally ran the scripts and generated pictures on January 20th, 2020 (the day Egon posed the question). Since the pandemic has got the whole science community introspecting, I came back to this today and thought it might be worth writing up as a blog post.

Without further ado, here are the most recent charts I’ve generated to answer three main questions. I’ve linked the images in such a way that the charts will be automatically updated with my GitHub repository. This also implicitly means that there’s a history of each image, but because two of them are plotting time course information, the history is already conveyed within the chart.

How many articles were contributed each month to ChemRxiv?

How many papers were submitted each month to ChemRxiv? Keep in mind that the current month is likely not complete.

How many unique authors contribute each month to ChemRxiv?

This only counts using the ORCID iDs of the first authors; it’s pretty inconsistent what other identifying information is included in the metadata for each article.

How many author submit multiple times each month?

How many authors submitted more than once per month? This chart shows spikes in August, which I will guess is when most people are submitting before their summer breaks :)

How many authors submitted for their first time each month?

How many unique first authors are there on ChemRxiv?

How many first authors have historically contributed to ChemRxiv at each month? We can take the first date of authorship for each author then count at each month how many unique first time authors there are. Then, we can use a cumulative sum to show how many authors have contributed to ChemRxiv at any point in time.

How many authors are prolific on ChemRxiv?

If we aggregate the data, we can ask how many authors have submitted lots of articles:

What licenses are popular on ChemRxiv?

The following chart shows the popularity of different licenses over time. The CC BY-NC-ND 4.0 license is a resounding victor. You can learn about Creative Commons (CC) licenses here.

If you’re interested to regenerate these charts yourself, you’re welcome to do so with the following code:

git clone https://github.com/cthoyt/chemrxiv-summarize
cd chemrxiv-summarize
python 01_download.py
python 02_process.py
python 03_visualize.py

Downloading takes a bit of time (about 40 minutes) but there’s a tqdm bar to keep you entertained in the mean time. Normally I package all of my code, but the one off scripts here didn’t seem to warrant it.

As a final note, I’d like to shout out to Marshall Brennan (@Organometallica) for being an excellent spokesperson and public face of ChemRxiv. Also, throughout this process I realized he also was a chemistry major in his bachelor’s at Northeastern University like me. Go huskies!

May 2020 Update: François-Xavier Coudert created the ChemRxiv-Dashboard, which makes some similar summaries to this. Check it out!

I made a dashboard for @ChemRxiv, fed by the @figshare
metadata API.https://t.co/rKyAOGkrVO pic.twitter.com/fLfjEabraz
— FX Coudert (@fxcoudert) May 19, 2020

November 2020 Update: I added a license chart and made some changes to enable this repo to be much more easily used for other FigShare institutions. If you’ve found this post from @figshare’s tweet and want help making these charts for your FigShare institution, please feel free to @ me on Twitter or send me an email.

How to Fix Your Monolithic Pull Request

2020-03-20T08:00:00+00:00

We’ve all been there. You started a new branch from master. You had a very specific goal in mind, The Original Goal. You made a pull request (PR) to go with it, too, The Original Pull Request. But then, you had an idea! And also, someone on your team asked you to solve another problem! Now the original code you wrote to address The Original Goal relies on that code … and now you’ve got dozens of files changed, hundreds of lines of diff, and nobody (including you) can understand what you’ve done. Like I said, we’ve all been there. Here’s what you can do to fix it:

1. Stop and Relax

Don’t do anything rash. Git is a pain to use, and you’re going to have to rely on it to keep a history for you of what you’ve done.

2. Summarize

First, you’re going to have to take a big step back. Write a summary of all the things you’ve done in The Original Pull Request. This should be about what the PR does and why it does it. Of course it could vary depending on the situation, but this summary shouldn’t be about exactly how the PR does it, because the implementation details are likely what lead to this situation in the first place.

Keep in mind that every PR has a box at the top that’s used to describe what’s in it. This is where you will put your summary.

3. Assessing Dependencies

Of all the things that The Original Pull Request, some of them are self-contained, and some of them rely on each other. It was probably the case that to accomplish The Original Goal, you had to address lots of smaller goals. You probably also had to change lots of code and write new code too.

Wouldn’t it have been nice if all of these implementations were already done, because then you could have just solved The Original Goal directly by using/applying previous code. That’s what we’re going to aim for.

But first, you need to figure out which things you did relied on which other ones, because you’re going to break The Original Pull Request up until it exactly matches up to addressing The Original Goal. don’t have any

4. The Break Up

After you understand which parts of The Original Pull Request depend on each other, pick one independent part of the code that accomplishes one sub-goal. Since you’re not doing this to be a martyr, and we all know git is too complicated to Do It Right, you’re going to copy/paste the files that are related to this change to your desktop*.

5. Escape the Madness

Before continuing, you’re going to make sure all of the code in your big messy branch for The Original Pull Request is committed and pushed. Even though we want to supersede what’s there, it never hurts to keep track of your descent into madness.

After there’s nothing lying around, switch back to master. If your team has taken good care of your repository, the master branch should be undisturbed by the chaos you’ve created in The Original Pull Request. Make a new branch from master, and name it appropriately for fixing the one sub-goal, from here out known as The Sub-Goal that you identified in Step 4. Now you can start updating the relevant files in your repository based on the files you copied to your desktop. I suggest you don’t copy/paste the contents of the whole files, because you might have forgotten about something else you changed in them. After all, you’re reading my guide because this was a mess.

6. The New Pull Request

Once you’ve finished making the new branch for your independent part of code that solves The Sub-Goal, you can make The New Pull Request.

You will now go through the entire process of writing a good summary of this branch for your co-developers, you will get their feedback, you will make updates, pass flake8, and so on. They will thank you for having code that accomplishes one thing, and can be described simply. They will thank you for not having too big of a diff, and for the things in the diff all being relevant and important. Then you can merge this branch into master.

7. Newfound Wisdom

Throughout transferring the code for The New Pull Request you have probably realized there are some things you did back in The Original Pull Request that you could do better, and made some updates in the code in The New Pull Request to reflect the wisdom you’ve gained along the way. That’s great! Congratulations!

After your team has approved The New Pull Request, you can merge it into master and both delete the branch locally and on the remote. Then you should switch back to the master branch. You can pull from master, and see your code that solved The Sub-Goal reflected here.

8. The Hard Part

This is the hard part. Now you have to switch back to the branch for The Original Pull Request. Now you have to update this branch from master. It’s going to be hard because now you’ve probably made different changes in The New Pull Request than in The Original Pull Request so there will likely be conflicts.

This is not a tutorial on how to solve merge conflicts. Use google to figure that out

I can’t understate: do this part really well. If you don’t, then the history in the original branch will be even more incomprehensible, and you won’t be able to tell if you lost any of your original work. Please, please, please do this well.

P.S. Like I said before, don’t be a martyr. Use tools like GitHub Desktop and PyCharm to help you merge. I heard that the git CLI was allegedly created by Linus Torvalds to slow other developers down.

Why are we going through all of this pain, rather than just pushing your team to let you merge The Original Pull Request? The reason you have to do this is because now all of the changes that addressed The Sub-Goal are part of master, and are no longer part of the diff of The Original Pull Request.

Now you’re one step closer to your team being able to understand, review, and eventually merge The Original Pull Request.

9. The Frustrating Part

This is the frustrating part. After you’ve gone through all of that work to split a tiny part of The Original Pull Request into a smaller, independent pull request, you’re not done. You will probably have to repeat steps 4-8 a few times. You’ll be tempted to throw away the branch for The Original Pull Request and maybe start over.

Don’t do that.

If you do, the same disorganization that lead to the mess of The Original Pull Request might just slip back into whatever you do next. Even worse, nobody else will be able to follow what you’ve done until now.

So relax. This is going to take a few days. You’re going to have to wait in between several iterations for feedback. That’s good. You need feedback. I need feedback. We all need to practice getting it and giving it. Embrace the opportunity to have your team help you improve your code, gain wisdom, and make your contributions sustainable.

Finishing Up

Eventually after several iterations of 4-9, you will have excised all of the code that was important for The Original Pull Request, but not directly accomplishing The Original Goal. As you removed independent parts, new parts became independent themselves. Eventually, The Original Pull Request will indeed match up exactly to The Original Goal, then you will be able to come back to it for review and merging.

I understand this is a frustrating process. The purpose of these steps were to help you think through a large piece of work you’ve done. You should be proud that you’ve solved a complex problem with many intricate parts. It was a lot of extra work to break it into many pull requests, and it might have taken more of your time the first time working through this process, but in the future, this might help you to start with small tasks rather than addressing The Original Goal all at once. GitHub, for example, has an issue tracker that is very helpful for this. I imagine that each issue should correspond to a Sub-Goal, and that each should have exactly one PR that addresses it. The Original Goal also deserves its own issue that points to all of the issues for its sub-goals. Eventually you will address this with a beautiful PR as well. Happy coding!

*If you’re thinking, why don’t I use cherry picking? If you know what cherry picking is in the context of git (and also how to use it) then you probably won’t have the issue that prompted this blog post. But also, you should go outside and pick some apples instead. Thanksgiving is never more than a few hundred days away. It pays to be ready.

Afterword

It might be illustrative to see where an example of where this was done in practice, so I’ll share some work I did with a text mining tool from Harvard Medical School, Gilda. It’s a simple yet powerful system for grounding of named entities based on dictionary lookup. Unfortunately, it didn’t include some dictionaries I wanted, and it didn’t have a UI to go with its web API.

So I set out on figuring out how it generated dictionaries, where it stored them, and how it loaded them to make the web app. I ended up making several modifications to accomplish this goal, but it was a huge PR. I’ve definitely annoyed the author, @bgyori, with PRs that are too big before, which he was ultimately not able to understand or merge.

Keep in mind, in your team, your teammates might be obligated to help you because you’re working towards a common goal, getting paid, etc. When you’re in the open source world, nobody really owes you anything, so you it’s in your best interest to make things as easy as possible on the package’s maintainer(s).

So I made a few different pull requests that were all totally independent:

Add constants for resource file paths #12
Make API more reusable #13
Make instantiation of Grounder more flexible #15

Maybe you’re seeing a theme here. I was improving lots of different bits of Gilda so I could reuse the package in new code later. The next incremental increase was:

Refactor functionality from the GrounderInstance class into the Grounder class #16

And finally with these in place, I realized that adding a web interface was parallel to my original goal, but not the core. What was really important was that throughout all of the Gilda functionality, I could load my own synonym list (which I’d generate using the HPO, EFO, and DOID). I was able to address the UI with:

Add minimimal UI to web interface #19

At the time of writing, we’re still working through this PR. But all of it is leading up to the point where I can load my own files into this web interface. It will seem so obvious to Ben when I send this PR next (but after giving him some space… I did just bombard him with 5 PRs in a few days) what I am trying to accomplish and why.

Want to see what happens when you try and do all of this in one PR? You will correctly guess that the PR is a total mess, impossible to understand, and riddled with questions that are really too big to answer when your head is already so far in the sand. Behold, in all its infamy, my failed PR from last summer (#4). At this point, you can’t even see what a mess it was from the linked web page but if you go back through the version history before I broke it into 5 smaller PRs (using the workflow described above) it was a monolith.

Host a Graduate Seminar Before Writing Your Thesis

2020-02-09T08:00:00+00:00

The other day I saw a tweet lamenting the drag that is literature review during preparation for writing your thesis.

I just love writing 15 page literature reviews for graduate school courses on literally any topic except my thesis topic.
— PhD Diaries (@thoughtsofaphd) February 7, 2020

I agree. I felt the same pain last fall when I wrote my doctoral thesis. Luckily, I had a strategy that made it a bit easier.

I learned it from one of my professors when I was doing my master’s degree in Life Science Informatics. Each semester, we had a seminar course in which each student was assigned research articles to read and present to the class with a short slide deck. Later, I joined his research group and realized that this course served as a literature review for him just as much as us.

So later when I was a Ph.D. student, I volunteered to run the seminar. I co-opted the concept, and planned the course to cover many of the topics I found interesting for my thesis. I already knew some of the papers very well, and a few were ones I had always been meaning to read. I tried to pick the most recent papers for topics when possible, but also threw in a few classics as well.

On the first day of the seminar, I shared the following course information. I thought it was important to make clear what my expectations were for students in terms of their prior knowledge. Since they all came from the same master’s program, I thought it was enough that they had passed one of the first semester lectures called “Biological Databases” which was about many of the resources and databases used in the systems and networks biology community. I also outlined what was the content for the course, what was expected, etc. then shared this all as a Google Doc so they could read it over and add comments.

I also made a list of possible papers and a tentative schedule that students could look over and decide which papers they found most interesting. The topics were arranged in a logical order to tell the story of my thesis, and for each section there were a few papers that I thought were very important, and a few extras just in case there was a lot of interest. During the first day of the seminar, I also went through the list of all papers and explained the topics to the students. I gave them this list via Google Docs as well, and they were able to claim papers for their presentations. Below, I’ve listed the final list of papers and the order in which they were presented. We were able to come to agreements for all students to present the papers I found most important. Maybe 40% of the class found a paper interesting and picked one the first day and the rest took the next week to decide, ask questions, or propose new papers.

Another consideration I had when picking this paper list was to choose work done by my colleagues that I found interesting and helpful. After, I invited them to come listen to the seminar and mediate discussion after. We were able to invite one of my collaborators Mehdi Ali (he’s a really good guy!) to discuss his work on using deep learning for relation extraction in natural language processing. I think that might have been the most engaging day of the whole series.

I added one aspect to this course compared to the previous seminar that I had attended: each student was not only responsible for presenting the paper that had been assigned from my list, but they were also responsible for finding a relevant pre-print (in the same or similar topic) and submitting a peer review through the pre-print system. When I was a student, I noticed many students did not read the references of the paper they were assigned in our seminars, and also had not considered other similar research to their paper. Asking them to find their own papers was a way to make this a more creative and fun process, and would directly prepare them to answer questions at the end of the presentation like “what will the authors do next?” or “how will this research be used by others?”

One of the funny things that happened during the pre-print presentations is the students found several of mine and presented those. I suppose this was inevitable given the contemporary nature of my work in the context of the topics chosen. I would actually explicitly encourage students to check out my pre-prints the next time I host a seminar, because I know the work very well and could mediate a nice discussion.

I learned a lot through the process of preparing this seminar. Its outline became the outline for my thesis, and a lot of the discussions became points that I addressed explicitly in my writing. I wouldn’t say that I was taking advantage of the students in this process - we all benefited from the experience. I hope you get some ideas about how you might be able to do this yourself, whether you’re a doctoral student, a postdoc, or something else!

Course Information

Title: Knowledge Assembly, Data Integration, and Modeling in Systems and Networks Biology
Period: Winter Semester 2018/2019
Location: Endenicher Allee 19A, Room U.105 on Wednesdays 13.00-14.30

Qualifications

Students should be comfortable with the material presented in the Biological Databases lecture during the first semester of the LSI curriculum.

Goal

Students will have the opportunity to practice reading, presenting, and discussing recent biomedical literature on the topics of knowledge assembly, data integration, and modeling in systems and networks biology.

Content

Students will be assigned papers and present on the holistic process of knowledge discovery in systems and networks biology that focus on the topics of knowledge assembly (e.g., natural language processing, modeling formalisms and formats, reasoning techniques), data integration (e.g., practical scenarios focusing on techniques on the data level, knowledge level, and analytical levels), and modeling strategies (e.g., rule-based modeling, agent-based modeling, mathematical modeling, hypothesis generation with knowledge-based approaches).

Assignment

Students will be assigned an article to read and present during a thirty (30) minute lecture. One goal of this lecture is to show an understanding of not only the material presented in the article, but also the relevant background information - this may entail following the references and reading other articles. Another goal is to not only educate, but entertain the audience. Students will also be expected to find a relevant pre-print article on arXiv, bioRxiv, or other pre-print server and post a peer-review for the author on the corresponding service. Following the presentation of their assigned article, students should include slides (1-3) briefly explaining the relevance of the pre-print that they found.

Method of Performance Review

Students will be assessed on the understanding of their assigned topic, the quality of their presentation, and their participation. Students missing more than 2 seminars will not pass the course without a doctor’s note.

Schedule

Week 0 - October 10th, 2018 - Syllabus Week

This week there will a short discussion of the syllabus and no presentation. For those in Bonn that aren’t aware of this wonderful tradition, welcome to Syllabus Week.

Week 1 - October 31st, 2018 - Named Entity Recognition

Mubassher Leser, U., & Hakenberg, J. (2005). What makes a gene name? Named entity recognition in the biomedical literature. Briefings in Bioinformatics, 6(4), 357–369.

Pre-print: https://www.biorxiv.org/content/early/2017/03/08/115022

Bachman, J. A., Gyori, B. M., & Sorger, P. K. (2018). FamPlex: A resource for entity recognition and relationship resolution of human protein families and complexes in biomedical text mining. BMC Bioinformatics, 19(1), 1–14.

Pre-print: https://www.biorxiv.org/content/early/2018/07/29/379446

Week 2 - November 7th, 2018 - Identifiers

Laibe, C., & Le Novère, N. (2007). MIRIAM Resources: tools to generate and resolve robust cross-references in Systems’ Biology. BMC Systems Biology, 1, 58.

Pre-print: https://www.biorxiv.org/content/early/2017/07/24/167619

Juty, N., Le Nover̀e, N., & Laibe, C. (2012). Identifiers.org and MIRIAM Registry: Community resources to provide persistent identification. Nucleic Acids Research, 40(D1), 580–586.

Pre-print: https://www.biorxiv.org/content/early/2018/02/14/101279

Week 3 - November 14th, 2018 - Information Extraction

Novichkova, S., et al. (2003). MedScan, a natural language processing engine for MEDLINE abstracts. Bioinformatics, 19(13), 1699–1706.

Pre-print: https://www.biorxiv.org/content/early/2018/08/29/403667

Ali, M., et al. (2017). Automatic Extraction of BEL-Statements based on Neural Networks. Proceedings of BioCreative VI Challenge and Workshop, (October).

Pre-print : https://osf.io/j76y3/

Week 4 - November 21nd, 2018 - Knowledge Representations

Demir, E., et al. (2010). The BioPAX community standard for pathway data sharing. Nature Biotechnology, 28(12), 1308–1308.

Pre-print: https://www.biorxiv.org/content/10.1101/192856v1

Hucka, M., et al. (2003). The systems biology markup language (SBML): a medium for representation and exchange of biochemical network models. Bioinformatics (Oxford, England), 19(4), 524–31.

Pre-print: https://www.biorxiv.org/content/early/2018/01/23/246470

Week 5 - November 28th - Knowledge Representations (cont…)

Le Novère, et al. (2009). The Systems Biology Graphical Notation. Nature Biotechnology, 27(8), 735–41.

Pre-print: https://www.biorxiv.org/content/early/2018/01/30/256750

Carbon, S., et al. (2017). Expansion of the gene ontology knowledgebase and resources: The gene ontology consortium. Nucleic Acids Research, 45(D1), D331–D338.

Pre-print: https://www.biorxiv.org/content/early/2018/10/07/437020

Week 6 - December 12th, 2018 - Pathway Databases and Semantic Data Integration

Croft, D., et al. (2014). The Reactome pathway knowledgebase. Nucleic Acids Research, 42(D1), D472–D477. AND Fabregat, A., et al. (2018). The Reactome Pathway Knowledgebase. Nucleic Acids Research, 46(D1), D649–D655.

Pre-print: https://www.biorxiv.org/content/early/2018/10/09/375097

Cerami, E. G., et al. (2011). Pathway Commons, a web resource for biological pathway data. Nucleic Acids Research, 39(SUPPL. 1), 685–690.

Pre-print: https://www.biorxiv.org/content/early/2018/09/03/353235

Khatri, P., Sirota, M., & Butte, A. J. (2012). Ten years of pathway analysis: Current approaches and outstanding challenges. PLoS Computational Biology, 8(2).

Pre-print: https://www.biorxiv.org/content/early/2018/09/13/416131

Gligorijević, V., & Pržulj, N. (2015). Methods for biological data integration: perspectives and challenges. Journal of The Royal Society Interface, 12(112), 20150571.

Pre-print: https://www.biorxiv.org/content/early/2018/12/13/358390

Week 8 - January 16th, 2019 - Applications

Saqi, M., et al. (2018). Navigating the disease landscape: knowledge representations for contextualizing molecular signatures. Briefings In Bioinformatics, (May), 1–15.

Pre-print: https://www.biorxiv.org/content/early/2018/11/23/475202

Himmelstein, D. S., et al. (2017). Systematic integration of biomedical knowledge prioritizes drugs for repurposing. ELife, 6.

Pre-print: https://www.biorxiv.org/content/early/2018/10/13/442640

Week 9 - January 23rd, 2019 - Applications

Lopez, C. F., et al. (2013). Programming biological models in Python using PySB. Molecular Systems Biology, 9(646), 646.

Pre-print: https://www.biorxiv.org/content/early/2018/12/23/503359

Gyori, B. M., et al. (2017). From word models to executable models of signaling networks using automated assembly. Molecular Systems Biology, 13(11), 954.

Pre-print: https://www.biorxiv.org/content/early/2018/05/15/322156

Encoding Biology in Knowledge Graphs

2020-01-23T08:00:00+00:00

How many molecular biology papers have you read today? This week? This month? If you’re like me, its not so many, and we’re falling behind very quickly. Here’s a chart made by the new PubMed that summarizes how many papers were published mentioning RAS in the last 50 years.

There were 4,483 publications listed in 2019. We can’t read that much, and even if we did, we couldn’t remember it all. That’s why we need to take the knowledge out of the unstructured text and store it in a structured form that can be read and stored in computers. This way, we can easily share it, query it, and write algorithms that can help us reason about the incredible amount of biological knowledge out there.

There are several formats in which this kind of information can be stored on a continuum between directly representing mechanistic biology to representing the knowledge itself. In the popular middle ground are BioPAX and BEL, which I’ll come back to in future posts.

It’s important to keep in mind that knowledge needs to be curated - this can either be manual, through natural language processing, or a mixture of both. I’ve written a paper on such a process, but for now this post should motivate a few following ones describing what it takes to deal with nomenclature, build ontologies, and then start extracting mechanistic biology from the literature.

Biosemantics vs. Biopragmatics

2020-01-22T08:00:00+00:00

In language, semantics describe the names and meanings of words. The bioinformatics community has aptly adopted biosemantics as a concept that encompasses the issues with the names and meanings of biological entities, usually in natural language processing and data integration. However, semantics does not capture the context of words, and biosemantics fails to describe the biological context and complex relationships between biological entities.

Pragmatics goes beyond semantics and describes the context of words. Because of this parallelism, I’ve begun to use the term biopragmatics to describe the family of computational approaches aimed at identifying and contextualizing the context of biological entities.