Organism#

lamindb provides access to the following public protein ontologies through bionty:

  1. Ensembl Species

  2. NCBI Taxonomy

Here we show how to access and search Organism ontologies to standardize new data.

import bionty as bt
import pandas as pd
πŸ’‘ loaded instance: testuser1/test-public-ontologies

PublicOntology objects#

Let us create a public ontology accessor with .public method, which chooses a default public ontology source from PublicSource. It’s a PublicOntology object, which you can think about as a public registry:

organisms = bt.Organism.public(organism="vertebrates")
organisms
PublicOntology
Entity: Organism
Organism: vertebrates
Source: ensembl, release-110
#terms: 317

πŸ“– .df(): ontology reference table
πŸ”Ž .lookup(): autocompletion of terms
🎯 .search(): free text search of terms
βœ… .validate(): strictly validate values
🧐 .inspect(): full inspection of values
πŸ‘½ .standardize(): convert to standardized names
πŸͺœ .diff(): difference between two versions
πŸ”— .to_pronto(): Pronto.Ontology object

As for registries, you can export the ontology as a DataFrame:

df = organisms.df()
df.head()
scientific_name division ontology_id assembly assembly_accession genebuild variation microarray pan_compara peptide_compara genome_alignments other_alignments core_db species_id
name
spiny chromis acanthochromis_polyacanthus EnsemblVertebrates NCBITaxon:80966 ASM210954v1 GCA_002109545.1 2018-05-Ensembl/2020-03 N N N Y Y Y acanthochromis_polyacanthus_core_110_1 1
eurasian sparrowhawk accipiter_nisus EnsemblVertebrates NCBITaxon:211598 Accipiter_nisus_ver1.0 GCA_004320145.1 2019-07-Ensembl/2019-09 N N N N N Y accipiter_nisus_core_110_1 1
giant panda ailuropoda_melanoleuca EnsemblVertebrates NCBITaxon:9646 ASM200744v2 GCA_002007445.2 2020-05-Ensembl/2020-06 N N N Y Y Y ailuropoda_melanoleuca_core_110_2 1
yellow-billed parrot amazona_collaria EnsemblVertebrates NCBITaxon:241587 ASM394721v1 GCA_003947215.1 2019-07-Ensembl/2019-09 N N N N N Y amazona_collaria_core_110_1 1
midas cichlid amphilophus_citrinellus EnsemblVertebrates NCBITaxon:61819 Midas_v5 GCA_000751415.1 2018-05-Ensembl/2018-07 N N N Y Y Y amphilophus_citrinellus_core_110_5 1

Unlike registries, you can also export it as a Pronto object via public.ontology.

Look up terms#

As for registries, terms can be looked up with auto-complete:

lookup = organisms.lookup()

The . accessor provides normalized terms (lower case, only contains alphanumeric characters and underscores):

lookup.giant_panda
Organism(name='giant panda', scientific_name='ailuropoda_melanoleuca', division='EnsemblVertebrates', ontology_id='NCBITaxon:9646', assembly='ASM200744v2', assembly_accession='GCA_002007445.2', genebuild='2020-05-Ensembl/2020-06', variation='N', microarray='N', pan_compara='N', peptide_compara='Y', genome_alignments='Y', other_alignments='Y', core_db='ailuropoda_melanoleuca_core_110_2', species_id=1)

To look up the exact original strings, convert the lookup object to dict and use the [] accessor:

lookup_dict = lookup.dict()
lookup_dict["giant panda"]
Organism(name='giant panda', scientific_name='ailuropoda_melanoleuca', division='EnsemblVertebrates', ontology_id='NCBITaxon:9646', assembly='ASM200744v2', assembly_accession='GCA_002007445.2', genebuild='2020-05-Ensembl/2020-06', variation='N', microarray='N', pan_compara='N', peptide_compara='Y', genome_alignments='Y', other_alignments='Y', core_db='ailuropoda_melanoleuca_core_110_2', species_id=1)

By default, the name field is used to generate lookup keys. You can specify another field to look up:

lookup = organisms.lookup(organisms.scientific_name)
lookup.ailuropoda_melanoleuca
Organism(name='giant panda', scientific_name='ailuropoda_melanoleuca', division='EnsemblVertebrates', ontology_id='NCBITaxon:9646', assembly='ASM200744v2', assembly_accession='GCA_002007445.2', genebuild='2020-05-Ensembl/2020-06', variation='N', microarray='N', pan_compara='N', peptide_compara='Y', genome_alignments='Y', other_alignments='Y', core_db='ailuropoda_melanoleuca_core_110_2', species_id=1)

Search terms#

Search behaves in the same way as it does for registries:

organisms.search("rabbit").head(3)
scientific_name division ontology_id assembly assembly_accession genebuild variation microarray pan_compara peptide_compara genome_alignments other_alignments core_db species_id __ratio__
name
rabbit oryctolagus_cuniculus EnsemblVertebrates NCBITaxon:9986 OryCun2.0 GCA_000003625.1 2009-11-Ensembl/2019-05 N Y N Y Y Y oryctolagus_cuniculus_core_110_2 1 100.0
rat rattus_norvegicus EnsemblVertebrates NCBITaxon:10116 mRatBN7.2 GCA_015227675.2 2020-11-Ensembl/2022-03 Y Y N Y Y Y rattus_norvegicus_core_110_72 1 72.0
blue-ringed sea krait laticauda_laticaudata EnsemblVertebrates NCBITaxon:8630 latLat_1.0 GCA_004320025.1 2019-11-Ensembl/2019-12 N N N Y Y Y laticauda_laticaudata_core_110_1 1 72.0

By default, search also covers synonyms:

organisms.search("sapiens").head(3)
scientific_name division ontology_id assembly assembly_accession genebuild variation microarray pan_compara peptide_compara genome_alignments other_alignments core_db species_id __ratio__
name
swainson's thrush catharus_ustulatus EnsemblVertebrates NCBITaxon:91951 bCatUst1.pri GCA_009819885.1 2020-01-Ensembl/2020-03 N N N N N Y catharus_ustulatus_core_110_1 1 69.230769
spiny chromis acanthochromis_polyacanthus EnsemblVertebrates NCBITaxon:80966 ASM210954v1 GCA_002109545.1 2018-05-Ensembl/2020-03 N N N Y Y Y acanthochromis_polyacanthus_core_110_1 1 65.454545
common snapping turtle chelydra_serpentina EnsemblVertebrates NCBITaxon:8475 Chelydra_serpentina-1.0 GCA_007922165.1 2019-11-Ensembl/2019-12 N N N N N Y chelydra_serpentina_core_110_1 1 64.285714

You can turn this off synonym by passing synonyms_field=None:

organisms.search("sapiens", synonyms_field=None).head(3)
scientific_name division ontology_id assembly assembly_accession genebuild variation microarray pan_compara peptide_compara genome_alignments other_alignments core_db species_id __ratio__
name
swainson's thrush catharus_ustulatus EnsemblVertebrates NCBITaxon:91951 bCatUst1.pri GCA_009819885.1 2020-01-Ensembl/2020-03 N N N N N Y catharus_ustulatus_core_110_1 1 69.230769
spiny chromis acanthochromis_polyacanthus EnsemblVertebrates NCBITaxon:80966 ASM210954v1 GCA_002109545.1 2018-05-Ensembl/2020-03 N N N Y Y Y acanthochromis_polyacanthus_core_110_1 1 65.454545
common snapping turtle chelydra_serpentina EnsemblVertebrates NCBITaxon:8475 Chelydra_serpentina-1.0 GCA_007922165.1 2019-11-Ensembl/2019-12 N N N N N Y chelydra_serpentina_core_110_1 1 64.285714

Search another field (default is .name):

organisms.search(
    "oryctolagus_cuniculus",
    field=organisms.scientific_name,
).head()
name division ontology_id assembly assembly_accession genebuild variation microarray pan_compara peptide_compara genome_alignments other_alignments core_db species_id __ratio__
scientific_name
oryctolagus_cuniculus rabbit EnsemblVertebrates NCBITaxon:9986 OryCun2.0 GCA_000003625.1 2009-11-Ensembl/2019-05 N Y N Y Y Y oryctolagus_cuniculus_core_110_2 1 100.000000
mus_musculus mouse EnsemblVertebrates NCBITaxon:10090 GRCm39 GCA_000001635.9 2020-08-Ensembl/2023-04 Y Y Y Y Y Y mus_musculus_core_110_39 1 67.500000
mola_mola ocean sunfish EnsemblVertebrates NCBITaxon:94237 ASM169857v1 GCA_001698575.1 2018-05-Ensembl/2018-07 N N N N N Y mola_mola_core_110_1 1 64.125000
oryzias_javanicus javanese ricefish EnsemblVertebrates NCBITaxon:123683 OJAV_1.1 GCA_003999625.1 2019-09-Ensembl/2019-11 N N N Y Y Y oryzias_javanicus_core_110_11 1 57.894737
meriones_unguiculatus mongolian gerbil EnsemblVertebrates NCBITaxon:10047 MunDraft-v1.0 GCA_002204375.1 2018-12-Ensembl/2020-03 N N N N N Y meriones_unguiculatus_core_110_1 1 57.142857

Standardize Organism identifiers#

Let us generate a DataFrame that stores a number of Organism identifiers, some of which corrupted:

df_orig = pd.DataFrame(
    index=[
        "spiny chromis"
        "silver-eye"
        "platyfish"
        "california sea lion"
        "This organism does not exist",
    ]
)
df_orig
spiny chromissilver-eyeplatyfishcalifornia sea lionThis organism does not exist

We can check whether any of our values are validated against the ontology reference:

validated = organisms.validate(df_orig.index, organisms.name)
df_orig.index[~validated]
❗ 1 term (100.00%) is not validated: spiny chromissilver-eyeplatyfishcalifornia sea lionThis organism does not exist
Index(['spiny chromissilver-eyeplatyfishcalifornia sea lionThis organism does not exist'], dtype='object')

Ontology source versions#

For any given entity, we can choose from a number of versions:

bt.PublicSource.filter(entity="Organism").df()
uid entity organism currently_used source source_name version url md5 source_website created_at updated_at created_by_id
id
1 6IUo Organism vertebrates True ensembl Ensembl release-110 https://ftp.ensembl.org/pub/release-110/specie... f3faf95648d3a2b50fd3625456739706 https://www.ensembl.org 2024-03-04 17:05:27.597876+00:00 2024-03-04 17:05:27.597912+00:00 1
2 JTGk Organism vertebrates False ensembl Ensembl release-109 https://ftp.ensembl.org/pub/release-109/specie... 7595bb989f5fec07eaca5e2138f67bd4 https://www.ensembl.org 2024-03-04 17:05:27.598091+00:00 2024-03-04 17:05:27.598110+00:00 1
3 3LoR Organism vertebrates False ensembl Ensembl release-108 https://ftp.ensembl.org/pub/release-108/specie... d97c1ee302e4072f5f5c7850eff0b642 https://www.ensembl.org 2024-03-04 17:05:27.598270+00:00 2024-03-04 17:05:27.598290+00:00 1
4 2Jzh Organism bacteria True ensembl Ensembl release-57 https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte... ee28510ed5586ea7ab4495717c96efc8 https://www.ensembl.org 2024-03-04 17:05:27.598473+00:00 2024-03-04 17:05:27.598495+00:00 1
5 1kdI Organism fungi True ensembl Ensembl release-57 http://ftp.ensemblgenomes.org/pub/fungi/releas... dbcde58f4396ab8b2480f7fe9f83df8a https://www.ensembl.org 2024-03-04 17:05:27.598665+00:00 2024-03-04 17:05:27.598685+00:00 1
6 2mIM Organism metazoa True ensembl Ensembl release-57 http://ftp.ensemblgenomes.org/pub/metazoa/rele... 424636a574fec078a61cbdddb05f9132 https://www.ensembl.org 2024-03-04 17:05:27.598830+00:00 2024-03-04 17:05:27.598845+00:00 1
7 2XQ6 Organism plants True ensembl Ensembl release-57 https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant... eadaa1f3e527e4c3940c90c7fa5c8bf4 https://www.ensembl.org 2024-03-04 17:05:27.598941+00:00 2024-03-04 17:05:27.598952+00:00 1
8 1Vzs Organism all True ncbitaxon NCBItaxon Ontology 2023-06-20 s3://bionty-assets/df_all__ncbitaxon__2023-06-... 00d97ba65627f1cd65636d2df22ea76c https://github.com/obophenotype/ncbitaxon 2024-03-04 17:05:27.599044+00:00 2024-03-04 17:05:27.599054+00:00 1

When instantiating a Bionty object, we can choose a source or version:

public_source = bt.PublicSource.filter(
    source="ensembl", version="release-110", organism="vertebrates"
).one()
organisms= bt.Organism.public(public_source=public_source)
organisms
PublicOntology
Entity: Organism
Organism: vertebrates
Source: ensembl, release-110
#terms: 317

πŸ“– .df(): ontology reference table
πŸ”Ž .lookup(): autocompletion of terms
🎯 .search(): free text search of terms
βœ… .validate(): strictly validate values
🧐 .inspect(): full inspection of values
πŸ‘½ .standardize(): convert to standardized names
πŸͺœ .diff(): difference between two versions
πŸ”— .to_pronto(): Pronto.Ontology object

The currently used ontologies can be displayed using:

bt.PublicSource.filter(currently_used=True).df()
Hide code cell output
uid entity organism currently_used source source_name version url md5 source_website created_at updated_at created_by_id
id
1 6IUo Organism vertebrates True ensembl Ensembl release-110 https://ftp.ensembl.org/pub/release-110/specie... f3faf95648d3a2b50fd3625456739706 https://www.ensembl.org 2024-03-04 17:05:27.597876+00:00 2024-03-04 17:05:27.597912+00:00 1
4 2Jzh Organism bacteria True ensembl Ensembl release-57 https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte... ee28510ed5586ea7ab4495717c96efc8 https://www.ensembl.org 2024-03-04 17:05:27.598473+00:00 2024-03-04 17:05:27.598495+00:00 1
5 1kdI Organism fungi True ensembl Ensembl release-57 http://ftp.ensemblgenomes.org/pub/fungi/releas... dbcde58f4396ab8b2480f7fe9f83df8a https://www.ensembl.org 2024-03-04 17:05:27.598665+00:00 2024-03-04 17:05:27.598685+00:00 1
6 2mIM Organism metazoa True ensembl Ensembl release-57 http://ftp.ensemblgenomes.org/pub/metazoa/rele... 424636a574fec078a61cbdddb05f9132 https://www.ensembl.org 2024-03-04 17:05:27.598830+00:00 2024-03-04 17:05:27.598845+00:00 1
7 2XQ6 Organism plants True ensembl Ensembl release-57 https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant... eadaa1f3e527e4c3940c90c7fa5c8bf4 https://www.ensembl.org 2024-03-04 17:05:27.598941+00:00 2024-03-04 17:05:27.598952+00:00 1
8 1Vzs Organism all True ncbitaxon NCBItaxon Ontology 2023-06-20 s3://bionty-assets/df_all__ncbitaxon__2023-06-... 00d97ba65627f1cd65636d2df22ea76c https://github.com/obophenotype/ncbitaxon 2024-03-04 17:05:27.599044+00:00 2024-03-04 17:05:27.599054+00:00 1
9 4yVc Gene human True ensembl Ensembl release-110 s3://bionty-assets/df_human__ensembl__release-... 832f3947e83664588d419608a469b528 https://www.ensembl.org 2024-03-04 17:05:27.599144+00:00 2024-03-04 17:05:27.599153+00:00 1
11 2akp Gene mouse True ensembl Ensembl release-110 s3://bionty-assets/df_mouse__ensembl__release-... fa4ce130f2929aefd7ac3bc8eaf0c4de https://www.ensembl.org 2024-03-04 17:05:27.599342+00:00 2024-03-04 17:05:27.599351+00:00 1
13 2UvD Gene saccharomyces cerevisiae True ensembl Ensembl release-110 s3://bionty-assets/df_saccharomyces cerevisiae... 2e59495a3e87ea6575e408697dd73459 https://www.ensembl.org 2024-03-04 17:05:27.599536+00:00 2024-03-04 17:05:27.599545+00:00 1
14 7llW Protein human True uniprot Uniprot 2023-03 s3://bionty-assets/df_human__uniprot__2023-03_... 1c46e85c6faf5eff3de5b4e1e4edc4d3 https://www.uniprot.org 2024-03-04 17:05:27.599634+00:00 2024-03-04 17:05:27.599643+00:00 1
16 5U7J Protein mouse True uniprot Uniprot 2023-03 s3://bionty-assets/df_mouse__uniprot__2023-03_... 9d5e9a8225011d3218e10f9bbb96a46c https://www.uniprot.org 2024-03-04 17:05:27.599828+00:00 2024-03-04 17:05:27.599837+00:00 1
18 5nkB CellMarker human True cellmarker CellMarker 2.0 s3://bionty-assets/human_cellmarker_2.0_CellMa... d565d4a542a5c7e7a06255975358e4f4 http://bio-bigdata.hrbmu.edu.cn/CellMarker 2024-03-04 17:05:27.600021+00:00 2024-03-04 17:05:27.600030+00:00 1
19 6AFz CellMarker mouse True cellmarker CellMarker 2.0 s3://bionty-assets/mouse_cellmarker_2.0_CellMa... 189586732c63be949e40dfa6a3636105 http://bio-bigdata.hrbmu.edu.cn/CellMarker 2024-03-04 17:05:27.600148+00:00 2024-03-04 17:05:27.600162+00:00 1
20 6cbC CellLine all True clo Cell Line Ontology 2022-03-21 https://data.bioontology.org/ontologies/CLO/su... ea58a1010b7e745702a8397a526b3a33 https://bioportal.bioontology.org/ontologies/CLO 2024-03-04 17:05:27.600258+00:00 2024-03-04 17:05:27.600268+00:00 1
21 6tvq CellType all True cl Cell Ontology 2023-08-24 http://purl.obolibrary.org/obo/cl/releases/202... 46e7dd89421f1255cf0191eca1548f73 https://obophenotype.github.io/cell-ontology 2024-03-04 17:05:27.600359+00:00 2024-03-04 17:05:27.600368+00:00 1
25 1PY3 Tissue all True uberon Uberon multi-species anatomy ontology 2023-09-05 http://purl.obolibrary.org/obo/uberon/releases... abcee3ede566d1311d758b853ccdf5aa http://obophenotype.github.io/uberon 2024-03-04 17:05:27.600757+00:00 2024-03-04 17:05:27.600766+00:00 1
29 6EOm Disease all True mondo Mondo Disease Ontology 2023-08-02 http://purl.obolibrary.org/obo/mondo/releases/... 7f33767422042eec29f08b501fc851db https://mondo.monarchinitiative.org 2024-03-04 17:05:27.601143+00:00 2024-03-04 17:05:27.601152+00:00 1
33 3V9D Disease human True doid Human Disease Ontology 2023-03-31 http://purl.obolibrary.org/obo/doid/releases/2... 64f083a1e47867c307c8eae308afc3bb https://disease-ontology.org 2024-03-04 17:05:27.601527+00:00 2024-03-04 17:05:27.601536+00:00 1
38 6fKX ExperimentalFactor all True efo The Experimental Factor Ontology 3.57.0 http://www.ebi.ac.uk/efo/releases/v3.57.0/efo.owl 2ecafc69b3aba7bdb31ad99438505c05 https://bioportal.bioontology.org/ontologies/EFO 2024-03-04 17:05:27.602006+00:00 2024-03-04 17:05:27.602014+00:00 1
40 6jHz Phenotype human True hp Human Phenotype Ontology 2023-06-17 https://github.com/obophenotype/human-phenotyp... 65e8d96bc81deb893163927063b10c06 https://hpo.jax.org 2024-03-04 17:05:27.602200+00:00 2024-03-04 17:05:27.602208+00:00 1
43 4q5A Phenotype mammalian True mp Mammalian Phenotype Ontology 2023-05-31 https://github.com/mgijax/mammalian-phenotype-... be89052cf6d9c0b6197038fe347ef293 https://github.com/mgijax/mammalian-phenotype-... 2024-03-04 17:05:27.602489+00:00 2024-03-04 17:05:27.602498+00:00 1
44 6Czy Phenotype zebrafish True zp Zebrafish Phenotype Ontology 2022-12-17 https://github.com/obophenotype/zebrafish-phen... 03430b567bf153216c0fa4c3440b3b24 https://github.com/obophenotype/zebrafish-phen... 2024-03-04 17:05:27.602585+00:00 2024-03-04 17:05:27.602594+00:00 1
46 55lY Phenotype all True pato Phenotype And Trait Ontology 2023-05-18 http://purl.obolibrary.org/obo/pato/releases/2... bd472f4971492109493d4ad8a779a8dd https://github.com/pato-ontology/pato 2024-03-04 17:05:27.602777+00:00 2024-03-04 17:05:27.602785+00:00 1
47 48aa Pathway all True go Gene Ontology 2023-05-10 https://data.bioontology.org/ontologies/GO/sub... e9845499eadaef2418f464cd7e9ac92e http://geneontology.org 2024-03-04 17:05:27.602872+00:00 2024-03-04 17:05:27.602881+00:00 1
49 3rm9 BFXPipeline all True lamin Bioinformatics Pipeline 1.0.0 s3://bionty-assets/bfxpipelines.json a7eff57a256994692fba46e0199ffc94 https://lamin.ai 2024-03-04 17:05:27.603062+00:00 2024-03-04 17:05:27.603071+00:00 1
50 3TI0 Drug all True dron Drug Ontology 2023-03-10 https://data.bioontology.org/ontologies/DRON/s... 75e86011158fae76bb46d96662a33ba3 https://bioportal.bioontology.org/ontologies/DRON 2024-03-04 17:05:27.603158+00:00 2024-03-04 17:05:27.603166+00:00 1
51 7CRn DevelopmentalStage human True hsapdv Human Developmental Stages 2020-03-10 http://aber-owl.net/media/ontologies/HSAPDV/11... 52181d59df84578ed69214a5cb614036 https://github.com/obophenotype/developmental-... 2024-03-04 17:05:27.603253+00:00 2024-03-04 17:05:27.603262+00:00 1
52 16tR DevelopmentalStage mouse True mmusdv Mouse Developmental Stages 2020-03-10 http://aber-owl.net/media/ontologies/MMUSDV/9/... 5bef72395d853c7f65450e6c2a1fc653 https://github.com/obophenotype/developmental-... 2024-03-04 17:05:27.603349+00:00 2024-03-04 17:05:27.603358+00:00 1
53 3Tlc Ethnicity human True hancestro Human Ancestry Ontology 3.0 https://github.com/EBISPOT/hancestro/raw/3.0/h... 76dd9efda9c2abd4bc32fc57c0b755dd https://github.com/EBISPOT/hancestro 2024-03-04 17:05:27.603444+00:00 2024-03-04 17:05:27.603453+00:00 1
54 5JnV BioSample all True ncbi NCBI BioSample attributes 2023-09 s3://bionty-assets/df_all__ncbi__2023-09__BioS... 918db9bd1734b97c596c67d9654a4126 https://www.ncbi.nlm.nih.gov/biosample/docs/at... 2024-03-04 17:05:27.603542+00:00 2024-03-04 17:05:27.603551+00:00 1