Example studies with fossil taxa not mapping

[chrsowen]

Hi Jonathan,
Two study ids are below for studies that include fossil taxa.
Thanks again for your help,
Chris

Study IDs: 1429 and 2550

[mtholder]

Hi Chris, Some of the unmapped taxa are extant. Can you list a couple of the fossil taxa for each of the studies? thanks,
Mark

[chrsowen]

Hi Mark,
I this is all of them from the two studies.
Thanks,
Chris

Study ID 2550:
Agriochoerus
Arctocyon
Basilosaurus
Cainotherium
Cebochoerus
Desmostylia
Diacodexis pakist
Diacodexis wasatch
Dissacus navajovious
Dissacus praenuntius
Elomeryx
Entelodontidae
Georgiacetus
Gobiohyus
Hapalodectes hetang
Hapalodectes leptogna
Harpagolestes
Heteromeryx
Homacodon
Hyopsodus
Hyracotherium
Leptictidae
Leptoreodon
Meniscotherium
Merycoidodon
Mesonyx
Mixtotherium
Mongolian Dissacus
Pachyaena gigantea
Pachyaena ossifraga
Palaeoryctoidea
Phenacodus
Plesiadapidae
Poebrotherium
Protoceras
Protocetus
Remingtonocetus
Sinonyx
Synoplotherium
Vulpavus

Study ID: 1429
Pachyrhachis problematicus

[jar398]

Spot checks:

Hmm.... genus Hyracotherium is in GBIF (via IRMNG), but not in OTT. Must
get filtered out for some reason. We will need to delve into the details of
Stephen's script ( taxomachine/data/process_gbif_taxonomy.py ).

3239467 5479 Hyracotherium Owen, 1840 Hyracotherium genus accepted
Hyracotherium Proc. geol. Soc. London, 3 (66), 163 (as Hyotherium p.
240). Interim
Register of Marine and Nonmarine Genera Animalia Chordata Mammalia
Linnaeus, 1758 Perissodactyla Owen, 1848 Equidae Gray, 1821

Homacodon is similar. It wouldn't surprise me if they were all of this
pattern, since these two were a random sample.

Jonathan

[mtholder]

It looks like commit 2aee0ad10cc2c969cba3cc330fcc484e5cf2d04c of taxomachine removed the scripts.
Checkout commit 010c215e6d88aa79e67291b6e65185fbf98c25dc if you need to see them...

[jar398]

Ouch! I guess Stephen didn't realize I was counting on the continued
presence of that directory...

an opportunity (?) to try out the method given here:

http://st-on-it.blogspot.com/2010/01/how-to-move-folders-between-git.html

That is, I could just copy the files over to opentree/smasher/, but it's
better if their histories can be preserved too.

Jonathan

On Wed, Jun 12, 2013 at 11:58 AM, Mark Holder notifications@github.comwrote:

It looks like commit 2aee0adhttps://github.com/OpenTreeOfLife/opentree/commit/2aee0ad10cc2c969cba3cc330fcc484e5cf2d04cof taxomachine removed the scripts.
Checkout commit 010c215010c215e6d88aa79e67291b6e65185fbf98c25dcif you need to see them...

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/32#issuecomment-19335653
.

[TonyRees]

Hi all,

Tony Rees here - compiler/maintainer of IRMNG - contains mostly genera & families, some species too...

I just passed the list of no-match names through IRMNG and all the first words matched as genera with the exception of:

Desmostylia
Entelodontidae
Leptictidae
Mongolian Dissacus
Palaeoryctoidea
Plesiadapidae

Desmostylia is an Order, Palaeoryctoidea is a superfamily as suggested by its termination, the others are families (also detectable by their termination) apart from "Mongolian Dissacus" which is clearly an unorthodox name form.

Hyopsodus appears to be a homonym (2 entries in IRMNG with different authorities) but on further investigation these may actually turn out to be the same. Protoceras is almost a real homonym (IRMNG entries in fossil Mammalia and extant Hymenoptera) but the latter is listed as a misspelling so does not really count. There is a third Protoceras record in Cephalopoda (Molluscs) which occurs in only one source and looks dubious.

Anyway so far as I am aware you have a copy of IRMNG (or two over time) which has been utilized for OTOL although as you say the same names have also been supplied to GBIF so you should be picking them up from there I would think. Let me know if this helps at all,

Regards - Tony Rees

[chrsowen]

Hi Everyone,
Here is another study where fossils are not mapping.
Study ID: 1804
Aistopoda
Amphibamus
Apateon
Captorhinidae
Doleserpeton
Eryops
Lysorophia
Microbrachis
Nectridea
Osteolepiformes
Procolophonidae
Seymouria
Synapsida
Tersomius
Triadobatrachus

[TonyRees]

Hi all,

I ran these names through IRMNG – you can do the same by pasting the list into the IRMNG online search page at http://www.cmar.csiro.au/datacentre/irmng/ . IRMNG had all of the genera bar one – Terpsomius was stored under a misspelling (Terpsomus), correct version added as of just now. You will also discover that Captorhinidae and Procolophonidae are family names; Aistopoda, Lysorophia, Nectridea and Osteolepiformes are orders; and only Synapsida is currently unresolved there (it is in fact an obsolete subclass name, IRMNG does not store subclasses at present).

All these names with the exception of Terpsomius and Synapsida would have been supplied with previous versions of IRMNG, I think, don’t know where those have ended up…

Cheers - Tony

Tony Rees
Manager, Divisional Data Centre,
CSIRO Marine and Atmospheric Research,
GPO Box 1538,
Hobart, Tasmania 7001, Australia
Ph: 0362 325318 (Int: +61 362 325318)
Fax: 0362 325000 (Int: +61 362 325000)
e-mail: Tony.Rees@csiro.aumailto:Tony.Rees@csiro.au
Manager, OBIS Australia regional node, http://www.obis.org.au/
Biodiversity informatics research activities: http://www.cmar.csiro.au/datacentre/biodiversity.htm
Personal info: http://www.fishbase.org/collaborators/collaboratorsummary.cfm?id=1566
LinkedIn profile: http://www.linkedin.com/pub/tony-rees/18/770/36

From: chrsowen [mailto:notifications@github.com]
Sent: Wednesday, 19 June 2013 1:15 AM
To: OpenTreeOfLife/opentree
Cc: Rees, Tony (CMAR, Hobart)
Subject: Re: [opentree] Example studies with fossil taxa not mapping (#32)

Hi Everyone,
Here is another study where fossils are not mapping.
Study ID: 1804
Aistopoda
Amphibamus
Apateon
Captorhinidae
Doleserpeton
Eryops
Lysorophia
Microbrachis
Nectridea
Osteolepiformes
Procolophonidae
Seymouria
Synapsida
Tersomius
Triadobatrachus

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/32#issuecomment-19618069.

[jar398]

I checked on the status of these in OTT

• "not in GBIF" means it's not in the GBIF dump that's available (from last
  July)
• currently we suppress IPNI and the IRMNG homonym list
• In the "why suppressed?" cases the answer could be that an ancestor
  belongs to a suppressed category such as the IRMNG homonym list; or
  possibly that the GBIF taxonomy was ill-formed (not a tree) at those
  points. The way the code is written it's hard to diagnose.

Aistopoda - not in GBIF
Amphibamus - not in GBIF
Apateon - suppressed by preprocessing script - don't know why
Captorhinidae - suppressed because IRMNG homonym list
Doleserpeton - why suppressed?
Eryops - suppressed because IRMNG homonym list
Lysorophia - not in GBIF
Microbrachis - why suppressed?
Nectridea - not in GBIF
Osteolepiformes - not in GBIF
Procolophonidae - IRMNG homonym list
Seymouria - suppressed because IPNI
Synapsida - not in GBIF
Tersomius - why suppressed? (PDB)
Triadobatrachus - why suppressed? (IRMNG)

Stephen, you must have had a good reason to exclude IPNI and the IRMNG
homonym list. What bad things happen if these are added to the taxonomy?

Jonathan

On Tue, Jun 18, 2013 at 11:15 AM, chrsowen notifications@github.com wrote:

Hi Everyone,
Here is another study where fossils are not mapping.
Study ID: 1804
Aistopoda
Amphibamus
Apateon
Captorhinidae
Doleserpeton
Eryops
Lysorophia
Microbrachis
Nectridea
Osteolepiformes
Procolophonidae
Seymouria
Synapsida
Tersomius
Triadobatrachus

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/32#issuecomment-19618069
.

[jar398]

Tony, thanks for your attention.

Here's the deal with IRMNG. OTT 2.0 gets all of its IRMNG content from GBIF. GBIF unfortunately is using a version of IRMNG from 2011 that shows a large number of invalid taxa as being accepted. These were corrected in later versions of IRMNG (including the one on the GNACLR site, from 2012), but we don't have a script yet to use any such later version.

To compensate for this problem Stephen added code to the GBIF preprocessing script to suppress all names attributed to the IRMNG homonym list, and all descendents of such names/taxa. For OTT 2.1 I have changed the script to only suppress such names when the taxon has no children, and that helps in many cases, e.g. Amphibamus and many others in the above lists.
(The IPNI issue is a red herring; Seymouria the amphibian appears just fine in OTT 2.1.)

This change for OTT 2.1 is only a halfway measure since it doesn't help us get newer IRMNG taxa such as Osteolepiformes.

If there is a Darwin Core archive for a post-2011 version of IRMNG, then GBIF should be notified so that they can pick it up and incorporate it into their taxonomy. They just released a new taxonomy last week and it still had the 2011 version of IRMNG.

I haven't decided whether to process IRMNG specially or just continue to rely on GBIF. If GBIF gets a newer version then any work we do to special-case IRMNG in our system will be redundant. Obviously that's my preference. If this IRMNG/GBIF coordination won't be happening any time soon, we will want to get a recent version of IRMNG and write a little script to prepare it for entry into our system. Laura may have a recent version, but I don't, other than the the GNACLR one.

[TonyRees]

Thanks for the heads-up, Jonathan. I guess I should therefore supply a new IRMNG dump to GBIF in time for their next NUB rebuild, perhaps within e.g. the next 3 months maybe.

Also I just discovered a bug in my code which was potentially showing some names (maybe 8,000 out of 70,000) as homonyms which should not be so on account of being misspellings (which technically are not homonyms and I therefore exclude when generating the homonyms list), now fixed. So if you like to re-crawl my list you should get something more accurate to use for OTT purposes.

Tony Rees
Manager, Divisional Data Centre,
CSIRO Marine and Atmospheric Research,
GPO Box 1538,
Hobart, Tasmania 7001, Australia
Ph: 0362 325318 (Int: +61 362 325318)
Fax: 0362 325000 (Int: +61 362 325000)
e-mail: Tony.Rees@csiro.aumailto:Tony.Rees@csiro.au
Manager, OBIS Australia regional node, http://www.obis.org.au/
Biodiversity informatics research activities: http://www.cmar.csiro.au/datacentre/biodiversity.htm
Personal info: http://www.fishbase.org/collaborators/collaboratorsummary.cfm?id=1566
LinkedIn profile: http://www.linkedin.com/pub/tony-rees/18/770/36

From: Jonathan A Rees [mailto:notifications@github.com]
Sent: Monday, 8 July 2013 6:22 AM
To: OpenTreeOfLife/opentree
Cc: Rees, Tony (CMAR, Hobart)
Subject: Re: [opentree] Example studies with fossil taxa not mapping (#32)

Tony, thanks for your attention.

Here's the deal with IRMNG. OTT 2.0 gets all of its IRMNG content from GBIF. GBIF unfortunately is using a version of IRMNG from 2011 that shows a large number of invalid taxa as being accepted. These were corrected in later versions of IRMNG (including the one on the GNACLR site, from 2012), but we don't have a script yet to use any such later version.

To compensate for this problem Stephen added code to the GBIF preprocessing script to suppress all names attributed to the IRMNG homonym list, and all descendents of such names/taxa. For OTT 2.1 I have changed the script to only suppress such names when the taxon has no children, and that helps in many cases, e.g. Amphibamus and many others in the above lists.
(The IPNI issue is a red herring; Seymouria the amphibian appears just fine in OTT 2.1.)

This change for OTT 2.1 is only a halfway measure since it doesn't help us get newer IRMNG taxa such as Osteolepiformes.

If there is a Darwin Core archive for a post-2011 version of IRMNG, then GBIF should be notified so that they can pick it up and incorporate it into their taxonomy. They just released a new taxonomy last week and it still had the 2011 version of IRMNG.

I haven't decided whether to process IRMNG specially or just continue to rely on GBIF. If GBIF gets a newer version then any work we do to special-case IRMNG in our system will be redundant. Obviously that's my preference. If this IRMNG/GBIF coordination won't be happening any time soon, we will want to get a recent version of IRMNG and write a little script to prepare it for entry into our system. Laura may have a recent version, but I don't, other than the the GNACLR one.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/32#issuecomment-20576822.

[TonyRees]

Hi Tony, Jonathan,
if there would be an updated IRMNG homonyms file that'll be greatly appreciated. Tony, just let us know once its available and we will update our backbone shortly after!
Thanks,
Markus

PS: meanwhile I will try to apply the same fix for genera from IRMNG without child species - thanks for the input!

On 08.07.2013, at 06:58, Tony.Rees@csiro.au wrote:

Thanks for the heads-up, Jonathan. I guess I should therefore supply a new IRMNG dump to GBIF in time for their next NUB rebuild, perhaps within e.g. the next 3 months maybe.

Also I just discovered a bug in my code which was potentially showing some names (maybe 8,000 out of 70,000) as homonyms which should not be so on account of being misspellings (which technically are not homonyms and I therefore exclude when generating the homonyms list), now fixed. So if you like to re-crawl my list you should get something more accurate to use for OTT purposes.

Tony Rees
Manager, Divisional Data Centre,
CSIRO Marine and Atmospheric Research,
GPO Box 1538,
Hobart, Tasmania 7001, Australia
Ph: 0362 325318 (Int: +61 362 325318)
Fax: 0362 325000 (Int: +61 362 325000)
e-mail: Tony.Rees@csiro.au
Manager, OBIS Australia regional node, http://www.obis.org.au/
Biodiversity informatics research activities: http://www.cmar.csiro.au/datacentre/biodiversity.htm
Personal info: http://www.fishbase.org/collaborators/collaboratorsummary.cfm?id=1566
LinkedIn profile: http://www.linkedin.com/pub/tony-rees/18/770/36

From: Jonathan A Rees [mailto:notifications@github.com]
Sent: Monday, 8 July 2013 6:22 AM
To: OpenTreeOfLife/opentree
Cc: Rees, Tony (CMAR, Hobart)
Subject: Re: [opentree] Example studies with fossil taxa not mapping (#32)

Tony, thanks for your attention.

Here's the deal with IRMNG. OTT 2.0 gets all of its IRMNG content from GBIF. GBIF unfortunately is using a version of IRMNG from 2011 that shows a large number of invalid taxa as being accepted. These were corrected in later versions of IRMNG (including the one on the GNACLR site, from 2012), but we don't have a script yet to use any such later version.

To compensate for this problem Stephen added code to the GBIF preprocessing script to suppress all names attributed to the IRMNG homonym list, and all descendents of such names/taxa. For OTT 2.1 I have changed the script to only suppress such names when the taxon has no children, and that helps in many cases, e.g. Amphibamus and many others in the above lists.
(The IPNI issue is a red herring; Seymouria the amphibian appears just fine in OTT 2.1.)

This change for OTT 2.1 is only a halfway measure since it doesn't help us get newer IRMNG taxa such as Osteolepiformes.

If there is a Darwin Core archive for a post-2011 version of IRMNG, then GBIF should be notified so that they can pick it up and incorporate it into their taxonomy. They just released a new taxonomy last week and it still had the 2011 version of IRMNG.

I haven't decided whether to process IRMNG specially or just continue to rely on GBIF. If GBIF gets a newer version then any work we do to special-case IRMNG in our system will be redundant. Obviously that's my preference. If this IRMNG/GBIF coordination won't be happening any time soon, we will want to get a recent version of IRMNG and write a little script to prepare it for entry into our system. Laura may have a recent version, but I don't, other than the the GNACLR one.

—
Reply to this email directly or view it on GitHub.

[jar398]

OTT 2.1 seems to fix most of these problems, although I didn't check every name.
It excludes Apateon, which is both a synonym.
I'm still working out the kinks but I think it's better than it was.
Jonathan

[jar398]

I checked the status of these in OTT 2.5 draft 1, which of course now has IRMNG genera. Most of them are there. In the cases they're not, it's because they're not known in any of our feeder taxonomies. We will have to add new taxonomy sources, or add to the taxonomy as patches.

Of course the decision was made to hide extinct taxa for the time being. We can re-expose them at any time, at the cost of clutter in the synthetic tree due to the large number if incertae sedis fossils, but a good strategy for dealing with extinct taxa has not been worked out yet. I suggest keeping in touch with Stephen on this issue, since it's up to him what goes into synthesis.

OK if I close this issue?

(4944667=gbif:4835359 Agriochoerus)
(4943853=gbif:3239419 Arctocyon)
(4942612=gbif:3240256 Basilosaurus)
(4944725=gbif:4835695 Cainotherium)
(4944704=gbif:4835498 Cebochoerus)
(5327129=irmng:11066 Desmostylia)
** No taxon found with this name: Diacodexis pakist
-- misspelling for Diacodexis pakistanensis ?
** No taxon found with this name: Diacodexis wasatch
-- not in any input taxonomy
** No taxon found with this name: Dissacus navajovious
-- not in any input taxonomy
(4942385=gbif:4974058. Dissacus praenuntius)
(4944775=gbif:4835391 Elomeryx)
(4944916=gbif:3240378 Entelodontidae)
(4942553=gbif:4832598 Georgiacetus)
(4944543=gbif:4576254 Gobiohyus)
** No taxon found with this name: Hapalodectes hetang
-- misspelling for Hapalodectes hetangensis ?
** No taxon found with this name: Hapalodectes leptogna
-- misspelling for Hapalodectes leptognathus ?
(4942373=gbif:4832492 Harpagolestes)
(4944618=gbif:4835648 Heteromeryx)
(4944869=gbif:4834785 Homacodon)
(4945069=gbif:5428926 Hyopsodus)
(4942678=gbif:3239467 Hyracotherium)
(4943486=gbif:3239371 Leptictidae)
(4944634=gbif:4835656 Leptoreodon)
(4945579=gbif:4829880 Meniscotherium)
(4944458=gbif:4835269 Merycoidodon)
(4942379=gbif:4832495 Mesonyx)
(4944612=gbif:4835644. Mixtotherium)
** No taxon found with this name: Mongolian Dissacus
-- not in any input taxonomy
(4942363=gbif:4974056. Pachyaena gigantea)
(4942364=gbif:4974057. Pachyaena ossifraga)
** No taxon found with this name: Palaeoryctoidea
-- not in any input taxonomy
(4945569=gbif:3239508 Phenacodus)
(4941495=gbif:3239538 Plesiadapidae)
(4942203=gbif:4835769 Poebrotherium)
** Ambiguous taxon name: Protoceras
(4942551=gbif:4832593 Protocetus)
(4942504=gbif:4832544 Remingtonocetus)
(4942371=gbif:4832493 Sinonyx)
(4942399=gbif:4832482 Synoplotherium)
(4942025=gbif:4833267 Vulpavus)

(5326025=irmng:12418 Aistopoda)
(4948603=gbif:4815794. Amphibamus)
** Ambiguous taxon name: Apateon
(4948118=gbif:3239057 Captorhinidae)
(4948597=gbif:4815799 Doleserpeton)
** Ambiguous taxon name: Eryops
(5326018=irmng:11819 Lysorophia)
(4948567=gbif:4816269. Microbrachis)
(5326003=irmng:10147 Nectridea)
(5320230=irmng:12292 Osteolepiformes)
(4946157=gbif:3238968 Procolophonidae)
(4948517=gbif:3241196 Seymouria)
** No taxon found with this name: Synapsida
-- not in any input taxonomy
(4948606=gbif:4975520 Tersomius)
(4948636=gbif:3241217. Triadobatrachus)

[jar398]

I asked if I could close the issue and got no reply, so will close. Reopen if neccessary.