geocompr.bib: attempt to add URL of r-spatial.org/book

[florisvdh]

I'm not sure whether this will actually result in the URL appearing in the bibliography; should depend on the applied citation style.

What follows are two sidenotes 😉 . Feel free to make them into issues, or whatever!

Is the citation style actually set for HTML? From the Pandoc manual, it seems that biblio-style (currently in index.Rmd YAML header) is for LaTeX only:

biblio-style
bibliography style, when used with --natbib and --biblatex

For HTML, one could use the csl YAML key.

Apart from that, CSL-JSON (*.json) or CSL-YAML (*.yaml) formats are actually better suited for storing bibliographic data when using Pandoc itself (not natbib) for producing citations & bibliography. It then uses its own CSL processor, pandoc-citeproc, which AFAIK is always used in case of HTML, and will by default be used for all formats including PDF. CSL-JSON / CSL-YAML are the preferred formats to use with pandoc-citeproc (with CSL-JSON being the official format for CSL, CSL-YAML is a one-to-one Pandoc-only converted format which is more human-readable/editable than json). Some more background in inbo/tutorials#207 (comment).

BibTeX (*.bib) is supported by pandoc-citeproc, but the processing is necessarily lossy because the BibTeX format is less standardized than CSL-JSON.

One way to create json/yaml from another file format, is by using RStudio in VME mode. Or by using pandoc-citeproc on the command line (file conversion). Probably there are ways to do it from the R console as well, perhaps with rbbt or rmarkdown (didn't check).

[Nowosad]

Thanks for this change and (even more!) for the discussion.

1. Regarding the change you made - we are using the https://www.zotero.org/groups/418217/energy-and-transport/collections/9K6FRP6N/items/MJUZR4QA/collection zotero library to store the geocompr references. I have added the book url there and updated our bib file.
2. You are probably right - we do not use the csl argument yet. Do you have any preferable citation style?
3. Should the bib-json change looks like this - https://github.com/Robinlovelace/geocompr/tree/bib? I saved all of our references as .json in Zotero.
4. I do not have a great "bibliography" workflow for geocompr. Currently, I add a new reference to Zotero, save it manually into the geocompr.bib file using the BBT addon of Zotero, and push the change to the GitHub repo. Do you have any advice on how to improve this workflow?

@Robinlovelace your thoughts could be also important here.

[florisvdh]

Hi @Nowosad if you're prepared to try RStudio VME (beware, it may cause various Rmd syntax changes depending on project settings) I believe it can be done directly from there (not using Zotero groups myself however, and -admittedly- not VME either). Then that surely is the easiest way. We made a tutorial which may shed some further light (https://inbo.github.io/tutorials/tutorials/r_citations_markdown/).

For manual insertion of bibliographic records from Zotero I use the RStudio addin of rbbt, which detects the file type you're in. E.g. if you're in a yaml file, it will insert the Zotero references that are selected in the Zotero GUI in the right format:

result:

Your json file looks fine; if you do it manually with Zotero, always export as 'Better CSL JSON' (or 'Better CSL YAML') (from the Better BibTeX addon). I believe with rbbt the json output can also be made 'pretty', i.e. multiline. My preference is CSL-YAML since it's always multiline, far less punctuation (as above), hence more easy to edit directly if needed (like BibTeX, one could argue).

The RStudio VME 'insert citation' addon is really cool though; it can read from various sources (including Zotero) while inserting a citation, meanwhile amending your existing bibliography file - not just overwriting the whole thing. So it is perfect for collaborative setups. BTW there's a pending request in the RStudio repo issues to make the addon available in Rmd source mode.

[florisvdh]

we do not use the csl argument yet. Do you have any preferable citation style?

I leave that totally to the author's preferences 😄

[florisvdh]

Zotero Groups are supported in RStudio VME: https://rstudio.github.io/visual-markdown-editing/citations.html#group-libraries

[Nowosad]

Thanks! I am definitely not ready for RStudio VME - I have tried it but found it slow on my computer. Also, after writing many documents in the Rmd source mode, I got used to it. I will wait for the addon mentioned above.

[Nowosad]

we do not use the csl argument yet. Do you have any preferable citation style?

I leave that totally to the author's preferences smile

@Robinlovelace @jannes-m do you have any favorite csl?

[Nowosad]

@florisvdh One important thing I forgot to ask before. Could we use CSL-JSON / CSL-YAML in LaTeX? We need both html and pdf versions for the Geocomputation with R book...

[florisvdh]

Could we use CSL-JSON / CSL-YAML in LaTeX?

Yes it's no problem as long as you use Pandoc as a front-end to LaTeX (e.g. by using rmarkdown 🙂) and it will give you consistency between outputs.

For PDF output, Pandoc processes these files with its citeproc CSL-processor and 'renders' the citations + bibliography in TeX already (as it would in HTML), before passing the TeX file to LaTeX in order to make the PDF. So this approach guarantees consistent results between HTML, PDF and other formats, and the process does not need any conversion of CSL-JSON/YAML to another format like bib.

Just make sure that the PDF bibliography is not managed by the TeX packages natbib or biblatex. By default it won't be, since such thing needs to be told explicitly to Pandoc (e.g. with a YAML key in Rmd, see https://bookdown.org/yihui/rmarkdown/pdf-document.html#latex-packages-for-citations).

So it's only when making PDF output with natbib or biblatex that it's more appropriate to manage bibliographic records in a bib(la)tex file directly, and conform to the requirements for these packages. But then you will see slightly different results in HTML if you want to have the latter as well, because Pandoc must first translate the BibTeX records to CSL-JSON anyway for other output types, which is not a simple one-to-one translation. Apart from that, differences can also arise in the output because a 100% match is needed between the specific CSL style file and the style file for natbib or biblatex. The CSL style files are available at https://github.com/citation-style-language/styles - and searchable from https://www.zotero.org/styles. See https://citationstyles.org/authors/ for more information.

[Nowosad]

Thanks! Currently, we use natbib for pdfs. I will need to try out alternatives you mentioned.

[Robinlovelace]

@Robinlovelace @jannes-m do you have any favorite csl?

This one is pretty good: https://github.com/citation-style-language/styles/blob/master/elsevier-harvard.csl

[Robinlovelace]

In terms of auto update: you can set Zotero to auto sync and auto update .bib files so it updates automatically, that's what I do.

[Robinlovelace]

This is definitely a good change so going to merge. However, it should automatically updated after changing the online Zotero file. I think I've discovered why this has not happened, for some reason, default: no for "Add URLs to BibTeX export": https://retorque.re/zotero-better-bibtex/installation/preferences/export/

[Robinlovelace]

I think this change should fix it, described in the link above:

[Robinlovelace]

Previously it was 'no'.

[florisvdh]

I think I've discovered why this has not happened

Yes, in preparing this PR I had to export as 'Better BibLaTex' from my Zotero record for this reference. Only then the URL was included.

[florisvdh]

To get more automation ('syncing') for CSL-JSON and avoid a Zotero->BibTeX->CSL route, you could try the following in the YAML header:

bibliography: http://127.0.0.1:23119/better-bibtex/collection?[collectionID].csljson

See https://retorque.re/zotero-better-bibtex/exporting/pull/

To collect just specific Zotero records, there's the JSON-RPC call (https://retorque.re/zotero-better-bibtex/exporting/json-rpc/) which the rbbt package uses.

[florisvdh]

To get more automation ('syncing') for CSL-JSON and avoid a Zotero->BibTeX->CSL route, you could try the following in the YAML header:

bibliography: http://127.0.0.1:23119/better-bibtex/collection?[collectionID].csljson

See https://retorque.re/zotero-better-bibtex/exporting/pull/

The URL can also be obtained interactively in Zotero:

right-click menu on group library collection:

result:

However for me this only succeeded for collections in 'My Library' (demoed below). curl hangs with group library / group library collection - maybe a bug since it is documented to work for group libraries as well. The group library I tested this with is very old though. Does it work in your case?

$ curl http://127.0.0.1:23119/better-bibtex/export/collection?/1/ZJVNXTQ4.json
[
  {"id":"knotters_estimating_1997","author":[{"family":"Knotters","given":"M"},{"family":"Walsum","given":"P. E. V","non-dropping-particle":"van"}],"container-title":"Journal of Hydrology","container-title-short":"JHydrol","DOI":"10.1016/S0022-1694(96)03278-7","ISSN":"0022-1694","issue":"1","issued":{"date-parts":[[1997,10,1]]},"page":"25-46","source":"ScienceDirect","title":"Estimating fluctuation quantities from time series of water-table depths using models with a stochastic component","type":"article-journal","URL":"http://www.sciencedirect.com/science/article/pii/S0022169496032787","volume":"197"},
  {"id":"van_der_sluijs_water_1985","accessed":{"date-parts":[[2018,9,6]]},"author":[{"family":"Van der Sluijs","given":"P."},{"family":"De Gruijter","given":"J. J."}],"container-title":"Agricultural Water Management","container-title-short":"Agricultural Water Management","DOI":"10.1016/0378-3774(85)90001-0","ISSN":"0378-3774","issue":"2","issued":{"date-parts":[[1985,9,1]]},"page":"109-125","source":"ScienceDirect","title":"Water table classes: A method to describe seasonal fluctuation and duration of water tables on Dutch soil maps","title-short":"Water table classes","type":"article-journal","URL":"http://www.sciencedirect.com/science/article/pii/0378377485900010","volume":"10"},
  {"id":"van_heesen_presentation_1970","accessed":{"date-parts":[[2018,9,6]]},"author":[{"family":"Van Heesen","given":"H. C."}],"container-title":"Geoderma","container-title-short":"Geoderma","DOI":"10.1016/0016-7061(70)90006-6","ISSN":"0016-7061","issue":"3","issued":{"date-parts":[[1970,9,1]]},"page":"257-278","source":"ScienceDirect","title":"Presentation of the seasonal fluctuation of the water table on soil maps","type":"article-journal","URL":"http://www.sciencedirect.com/science/article/pii/0016706170900066","volume":"4"}
]

[Robinlovelace]

That looks like really useful functionality, thanks for sharing! For now my plan is to go with the "if not broken don't fix" philosophy but confident that this functionality will come in handy in the future. Many thanks!

[florisvdh]

It was a pleasure to provide some hints @Robinlovelace! Just be aware that for HTML citations & bibliography you currently do a sort of roundtrip Zotero CSL -> BibTeX -> CSL-JSON. While BBT and Pandoc (respectively) do the best possible thing here, some differences between HTML & PDF bibliography may occur as a consequence. Of course it's up to you decide whether that's a problem. More background in https://retorque.re/zotero-better-bibtex/exporting/pandoc/#use-csl-not-bibtex-with-pandoc.

[Nowosad]

We will need to think about it (especially due to the fact we need to have both HTML and PDF version).

That being said - I learned a lot of new things in this discussion. Thank you a lot @florisvdh!

[Robinlovelace]

Update on this, the hyperlinks are in the references section: https://geocompr.robinlovelace.net/references.html