Citing external software

[lkastner]

Is your feature request related to a problem? Please describe.
The main problem is that mathematical software is used in a lot of research, but it is often not cited. With Oscar this problem becomes more intricate as we attach a lot of software in the "background" which the user never notices, like cohomCalg, TOPCOM, lrs, cdd, normaliz, just to name a few (that I know of). Funding for these projects and their authors also depends on citations of their software so this lack of citations has real consequences.

Describe the solution you'd like
A way to collect citations for all software packages involved in an Oscar session. There are different ways to go about this, polymake e.g. annotates every object with a citation whenever external software is used in a rule. But in Oscar this principle might not be feasible.

Ideally the user should be able to get

• a list of all external (and maybe internal) software used
• (optionally) for every entry the bibtex (we could make a script that gets this from swmath, then we would not have to curate these)
• (optionally) the location where the external software was used.

Describe alternatives you've considered
Currently there is no way to get these citations except for reading the code.

Additional context
What about the cornerstones? Do they count as part of Oscar or should they be individually cited as well?

I am happy to implement something, but I would very much appreciate hearing your thoughts and ideas beforehand.

[thofma]

Can you show an example of how this looks like in action in polymake?

[lkastner]

This is what happens inside of polymake:

polytope > $c = cube(2);

polytope > $cc = new Polytope(POINTS=>$c->VERTICES);

polytope > print $cc->FACETS;
polymake: used package ppl
  The Parma Polyhedra Library ([[wiki:external_software#PPL]]): A C++ library for convex polyhedra
  and other numerical abstractions.
  http://www.cs.unipr.it/ppl/

1 1 0
1 0 1
1 -1 0
1 0 -1

polytope > save($cc, "/tmp/cube");

I made this file pretty and for some reason github does not like to have a .json, so it is a .json.txt instead:
pretty.json.txt
This now has a section

"_info": {
        "credits": {
            "ppl": "  The Parma Polyhedra Library ([[wiki:external_software#PPL]]): A C++ library for convex polyhedra\n  and other numerical abstractions.\n  http://www.cs.unipr.it/ppl/\n"
        }
    },

[aelzenaar]

I notice that you do not seem to be saving the version of the software used - this is important for reproducibility. Also, perhaps it might be useful to have a switch to change between just logging a list of programs used (with citations) and a more full log (which maybe lists every call into an external library) - this could be useful for debugging (if your program gives a wrong result, you might not be sure if this is an error in your code, in the Oscar layer, or in the base software).

[fingolfin]

I think recording the precise versions of all involved software is somewhat orthogonal to what the goal is here. For reproducibility, we already have Julia's Manifest.toml, which records these versions and more -- thanks to the fact that most of our external software is included via dedicated packages resp. via JLLs, that captures most of it. Tracking used GAP packages is a noticeable, err, gap here (pun... intended... I guess?). But even those will eventually be tracked via Julia packages.

So, while I don't object to also printing versions with the credits, I don't think they should be used for reproducibility (so in particular, nice human readable versions should be used, and not e.g. the precise but obfuscated ones used by some of our JLLs)

[lkastner]

I made a rought sketch at
https://github.com/lkastner/jupyter-notebooks/blob/main/oscar_credits.ipynb
Of course the print_credits global variable would be replaced by something along the lines of the unicode switch. Maybe this can serve as a basis for this (non-)discussion?

Ping @fingolfin

[lkastner]

This is based on the @attr macro of AbstractAlgebra.jl and ofc I don't understand everything I copied from there, so forgive me if I use something wrong.

[thofma]

Interesting. I wonder what the overhead is. In your example:

@annotate_credits creditsA function g(x::Int)
    2*x
end

Could you do a using BenchmarkTools; x = Ref{Int}(2); @benchmark g($x[]) and then do a similar thing for the non-annotated version?

The tracing here effectively means recording every function call in a global dictionary. This might be slow. If we do something 'tracing' like that, we would have to put it on every function in e.g. Nemo.jl (to get credits for flint/arb) or Singular.jl. But the annotation must go at every method definition, since otherwise I am not sure we need it all.

P.S.: There is a way to make this choice between tracing and not-tracing a compile time option. By this I mean that something like print_credit = true (or something similar) would redefine the methods to include the tracing. The version of the method with print_credit = false would be free of the tracing. The only(?) disadvantage is, that once the flag print_credit is changed, everything would have to be recompiled, making it a big laggy for the REPL.

[lkastner]

I added the benchmarks and it looks pretty grim. But this seems not to be related to the print_credits or anything, even if I comment out the entire content of the default_display_credits, the benchmarks stay the same.

On the other hand I'd raise the question whether we really want to annotate everything, even the Oscar cornerstones. My main concern was more with external software that does not profit from Oscar funding, but nevertheless is used, like e.g. TOPCOM and cohomCalg in the ToricVarieties section.

[fingolfin]

That's absolutely true. As I mentioned elsewhere, also GAP packages (and possibly polymake "applications" (did I use that term right?) and certain Singular libraries maybe should get extra credit.

Access to these specialized dependencies is indeed more restricted, so perhaps less problematic to track in terms of performance...

[fieker]

There are several different problems here and, as far as I can see it, some canot be solved by any pure software solution...
We can (efficient or not, nice or not) devise a method to record all software used in a given computation (for an arbitrary given value of "all"). However, what we cannot solve (at least I do not see anyhting here) is if the software used is essential or trivial - in this case.
As an example, "all" of Oscar is based in flint and gmp. Suppose 90% of the total time is in multiplications. Does this automatically mean flint deserves praise? Or gmp? both? neither?

• it could be we're only doing (in the end) 60 bit operations, then
  • one should recode using Int
  • gmp is not involved
  • fmpz makes it bareable
• it could be that we're using insanely large operations, thus the special flint-fft, thus flint is crucial here and deserves every bit of credit
• it could be that we're using 120 bit operations, thus one should recode...
• it could be that this is lets say 1000 bit operations, and gmp is crucial.
  (the use of integer multiplication here is not important - I can make similar examples everywhere)

To some degree it doesn't matter as credit should go to the tools used, however, in a paper I would usually not give a reference to basic results...

To make things more interesting, if Oscar is successful, credit might have to be attachend essentially to individual functions as the result of an entire PhD might just be a single non-trivial function...
Thus:

• any list has to be vetted by s.o. how KNOWS
• a list that is complete will be so long that no-one is using it
• a list that is too short will piss off anyone who is not credited

A completely orthoginal problem is then reproducibility. As Max said, here the Project.toml and jll already solve most of it.