Integrate paper metadata handling with Wikidata

[Daniel-Mietchen]

This post is inspired by the BibTeX from Wikidata functionality described in https://larsgw.blogspot.de/2016/09/citationjs-on-command-line.html .

Some thoughts on how to integrate ContentMine's paper metadata handling with Wikidata:

• if a ContentMine pipeline (or any reference file in BibTeX or similar format, for that matter) touches bibliographic metadata of scholarly articles, check whether Wikidata items for these articles already exist (e.g. via P932, P356, P698).
  • if yes, it might simply trigger an integrity check of these metadata, perhaps identify the main topic (P921) or do nothing for the moment
  • if no, it should start the missing items with at least some basic properties (e.g. P31:Q13442814 and the respective value for a persistent identifier). If this would leave the items incomplete with respect to Wikidata's data model for scholarly articles, the missing pieces could be handled by the mostly existing pipelines around constraint violations.
• in addition to the existing ContentMine pipelines to search by dictionaries, it might be interesting to have some functionality to search the literature (across all or selected dictionaries) by contributions from particular authors, institutions, journals, dates or some such, with which Wikidata could help
• What about running ContentMine over Wikipedia dumps to identify facts?
  • if these facts are referenced on Wikipedia to scholarly sources, ContentMine could check whether the indicated sources actually support the statement, and flag cases where that's not clear
  • if the Wikipedia statements lack scholarly references, ContentMine might be able to find some
  • as above, the metadata of the scholarly references would go to Wikidata, from where it might be pulled into the respective Wikipedia article by way of some variant of Module:Cite.

[tarrow]

These are definitely interesting ideas.

The first idea appears to me to be the most readily actionable at the moment. What sort of work flow do you envisage for this? Hope this doesn't seem like a barrage of questions but I just want to check I've understood this all right.

Should we build a bot and seek approval from the community to add these items? In the call we discussed principally interacting via the primary-sources tool although we mostly talked about 'facts' not paper metadata. Perhaps given that this data is already 'curated' by either the NCBI or CrossRef this isn't such a problem.

Which metadata should we consider adding? If we are looking at all the new publications on a given day should these all be added to Wikidata? My impression from wikicite was that this kind of blanket adding where there is neither a structural need or a lack of real notability for a given publication should be avoided (and perhaps go into librarybase instead?).

One of the places I started with librarybase before was only importing works that were referenced on enwiki (but we could choose any/all wikis)?

[Daniel-Mietchen]

I think having Wikidata entries for all works cited from enwiki is reasonable, and expansion to all works cited by any Wikimedia project (at least from their content namespaces) should come as soon as possible thereafter.

Blanket addition beyond that may cause problems but still makes sense in the long run, as that would contribute to the goal of turning Wikidata into an open citation graph. So anything cited anywhere from "within scope" would eventually get a Wikidata item, and after some time, I could well imagine encouraging people to upload their BibTeX files to some tool on Wikimedia labs that would then check these files against the Wikidata corpus and add info / flag inconsistencies as needed.

Perhaps we can start by sharing in a standard fashion the publications that CM has read on a given day, perhaps along with things mined from them? We could then go over that feed and hopefully become more specific about the respective workflows for Wikidata/ Librarybase, and demo things with Zika.

[petermr]

I think these are all in scope for the WikiFactMine project - it will
depend on details.

On Mon, Oct 10, 2016 at 4:46 AM, Daniel Mietchen [email protected]
wrote:

This post is inspired by the BibTeX from Wikidata functionality described
in https://larsgw.blogspot.de/2016/09/citationjs-on-command-line.html .

Yes - Lars has done a super job.

Some thoughts on how to integrate ContentMine's paper metadata handling
with Wikidata:

• if a ContentMine pipeline (or any reference file in BibTeX or
  similar format, for that matter) touches bibliographic metadata of
  scholarly articles, check whether Wikidata items for these articles already
  exist (e.g. via P932, P356, P698).
  • if yes, it might simply trigger an integrity check of these
    metadata, perhaps identify the main topic (P921) or do nothing for the
    moment
  • if no, it should start the missing items with at least some basic
    properties (e.g. P31:Q13442814 and the respective value for a persistent
    identifier). If this would leave the items incomplete with respect to
    Wikidata's data model for scholarly articles, the missing pieces could be
    handled by the mostly existing pipelines around constraint violations.

I think this is a great place to start learning about Wikidata and
normalized metadata. Because the scholarly literature is not pr

• • in addition to the existing ContentMine pipelines to search by
    dictionaries, it might be interesting to have some functionality to search
    the literature (across all or selected dictionaries) by contributions from
    particular authors, institutions, journals, dates or some such, with which
    Wikidata could help

Yes - bibliography is starting to emerge as critical and I think we can
and should address it. We can't do all of it, but it needs ot be integrated
into the facts.

• What about running ContentMine over Wikipedia dumps to identify
  facts?
  • if these facts are referenced on Wikipedia to scholarly sources,
    ContentMine could check whether the indicated sources actually support the
    statement, and flag cases where that's not clear
  • if the Wikipedia statements lack scholarly references,
    ContentMine might be able to find some
  • as above, the metadata of the scholarly references would go to
    Wikidata, from where it might be pulled into the respective Wikipedia
    article by way of some variant of Module:Cite.

I'll bounce this around with Magnus

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#126, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAsxSx2nXGs7meBHAVdzPYuUsVAJNgKYks5qybUKgaJpZM4KSNUs
.

Peter Murray-Rust
Reader in Molecular Informatics
Unilever Centre, Dep. Of Chemistry
University of Cambridge
CB2 1EW, UK
+44-1223-763069