Skip to main content

Oh Crossref

If you care about open peer review metadata (and, I mean, who doesn't), please consider taking a small action today.
Published onSep 26, 2022
Oh Crossref

Let me preface this by saying I really like and respect Crossref. The people I’ve met and corresponded with who work there are universally nice and well meaning. And I think its potential is drastically underrated, mostly due to its legacy tech stack. I mean, it’s basically living the semantic web/web3 dream as a giant, free-to-query database in the sky that allows machines to understand what’s (supposed to be) living at a given URI independent of the content itself — and, increasingly, what other people are saying about that URI. That’s a pretty amazing feat.

But woof, where it falls short, it really falls short, and ends up hampering innovation as a result. Today, I learned that the service only supports depositing peer reviews with isReviewOf relationships to content already registered at Crossref. This means that if you review, for example, an arXiv preprint (arXiv uses DataCite to provide DOIs), and you want to publish it openly, you have deposit that peer review to Crossref without the metadata link to the reviewed content.1

This limitation will have a stifling effect on the growing open peer review movement, especially when it comes to preprint reviews, by making it harder for researchers to get credit for reviewing work openly, for authors to find feedback about their work, and for researchers who study research to understand the impacts and growth of open review. Worse still, the Crossref team tells me that this is not even close to a high priority for them, meaning it will most likely be years before we see it implemented, if we ever do. As a result, even as movements like ASAPBio’s #PublishYourReviews campaign, open evaluation platforms like eLife’s Sciety, and the joint Docmaps effort2 to capture editorial processes gain steam, they will be limited by the failure of key infrastructure to support their innovations.

As a product manager, I empathize with Crossref’s prioritization and legacy tech management challenges. So I would understand the decision to deprioritize these types of efforts if this was simply not inline with Crossref’s mission (or documented at all), or if there was no community support for the idea. But just this June, Crossref made a big to-do about their vision for supporting what they call the “Research Nexus,” which is the effort to transparently link together and understanding the full lifecycle and impact of research outputs. Open review is exactly the type of effort that they claim to want to support, and they have the groundswell of community support for it mentioned above forming. So it’s a little strange that this relatively small request — to add support for something already in their schema, no less! — wouldn’t be a priority.

Luckily, the Crossref team is very responsive to feedback, and have kindly opened a feature request where you can (kindly, but firmly!) ask that they support open review.

So, my ask for you to day is to visit that feature request and add a comment in support of prioritizing the feature. Here’s a simple template for adding a comment in the most impactful way:

  • Your name

  • Any organizations you represent, particularly if they are Crossref members or users

  • Why you want them to fully support peer review relationships — making special note if this support is critical for you or your organization’s work

  • Anything the lack of support prevents you from doing today

  • Any general thoughts about the need to support open peer review

Thanks y’all. We’ll get there. And yes, I am investigating whether DataCite fully supports peer review relationships as an alternative. If anyone knows for certain, please leave a note here.

Comments
2
Rupert Gatti:

As I understand the CrossRef Schema - it IS possible to tag a work as aReviewOf a work that has an arXiv id -

https://data.crossref.org/reports/help/schema_doc/5.3.1/relations_xsd.html#inter_work_relation

provides a relationship type "isReviewOf"

and the attributes allow an array of identifiers: https://data.crossref.org/reports/help/schema_doc/5.3.1/relations_xsd.html#relations_type.atts

which include arXiv ids.

I have never actually tried it, but on the face of it would appear you can deposit a review of an arXiv working paper and include the relationship within the metadata. Of course I might have also completely misunderstood the issue!

Gabriel Stein:

Hi Rupert — per the schema, it should be possible. But Crossref has confirmed that it is not in fact currently possible, hence the post :).

Chris Wilkinson:

DataCite has `relatedIdentifiers`, which allows for a wide range of identifiers (https://schema.datacite.org/meta/kernel-4.4/doc/DataCite-MetadataKernel_v4.4.pdf#page=24)

For example, https://api.datacite.org/dois/10.5281/zenodo.7017120 is a review of an arXiv article:

curl "https://api.datacite.org/dois/10.5281/zenodo.7017120" | jq .data.attributes.relatedIdentifiers

Related, Zenodo doesn't allow us to use the `PeerReview` resource type. I opened https://github.com/zenodo/zenodo/issues/2295, but haven't had a response.

Gabriel Stein:

Thanks Chris! Any evidence of the relationship accepting identifier types other than doi, or non-datacite dois that you can find? I ask because per crossref’s docs and schema, they should support this, too. They just don’t in reality.

+ 2 more...