Which packages to load by default?

[olexandr-konovalov]

This came from testing https://github.com/gap-system/gap-container today. We have started to discuss this with @markuspf - perhaps worth to create an issue here.

When the core GAP system is started and packages are not present (except GAPDoc), a series of warnings is displayed:

$ gap   
 *********   GAP 4.8.5, 25-Sep-2016, build of 2016-09-27 14:15:44 (UTC)
 *  GAP  *   http://www.gap-system.org
 *********   Architecture: x86_64-pc-linux-gnu-gcc-default64
 Libs used:  gmp, readline
 Loading the library and packages ...
#I  autpgrp package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  alnuth package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  crisp package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  ctbllib package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  factint package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  fga package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  irredsol package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  laguna package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  polenta package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  polycyclic package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  resclasses package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  sophus package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  tomlib package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
 Components: trans 1.0, prim 2.1, small* 1.0, id* 1.0
 Packages:   GAPDoc 1.5.1
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> 

First of all, if you're using the GAP core without packages and want to avoid these warnings, start GAP with -A option.

Second, why this list of packages is as such? This is defined in lib/package.gi:

# The packages to load during startup can be specified via a user preference.
DeclareUserPreference( rec(
  name:= "PackagesToLoad",
  description:= [
    "A list of names of packages which should be loaded during startup. \
For backwards compatibility, the default lists most of packages \
that were autoloaded in GAP 4.4 (add or remove packages as you like)."
    ],
  default:= [ "autpgrp", "alnuth", "crisp", "ctbllib", "factint", "fga", 
              "irredsol", "laguna", "polenta", "polycyclic", "resclasses", 
              "sophus", "tomlib" ],
  values:= function() return RecNames( GAPInfo.PackagesInfo ); end,
  multi:= true,
  ) );

Indeed the comment is right in saying "most of packages" as in a version of GAP 4.4.12 that I've just compiled, I have the following packaged loaded by default:

AClib 1.1, Polycyclic 2.8.1, Alnuth 2.3.1, AutPGrp 1.4, CrystCat 1.1.3, Cryst 4.1.6, CRISP 1.3.4, CTblLib 1.1.3, TomLib 1.1.4, FactInt 1.5.3, GAPDoc 1.3, FGA 1.1.0.1, IRREDSOL 1.2, LAGUNA 3.5.0, Sophus 1.23, Polenta 1.2.7, ResClasses 3.0.0

so AClib and CrystCat were omitted.

Third, do we want to revise this list? IIRC the intention was to keep it as such for backwards compatibility in GAP 4.5, and return to it later. IMHO we should only revise it by reducing it (either by not loading a package or by making it needed to run GAP), and if we decide why some packages should stay in this user preference then it may be good to keep that decision recorded under this issue. I've created a checklist and suggest to collect comments on individual packages under this issue.

• autpgrp
• alnuth
• atlasrep
• crisp
• ctbllib
• factint
• fga
• irredsol
• laguna : ok to remove from packages loaded by default
• polenta
• polycyclic
• resclasses
• sophus
• tomlib

I think these packages fall into two categories: those which may be used by the library if they are marked for loading (atlasrep, though it is not in the default list, and also factint and tomlib) and the rest.

(Updated: the last paragraph seems unclear because any package is used by the library. Here is what I had in mind: sometimes, we can use ShowPackageVariables to detect e.g. that the package installs new method for an operation defined in the library, and in this case we can decide when the method from the package is used from a well-defined mechanisms. But other cases when the library calls IsPackageMarkedForLoading and then calculation goes one way or another dependently on the availability of the package, can not be decided looking at the package - they can be figured out only if you look at the corresponding library code. That's the main difference between these two cases).

[olexandr-konovalov]

I will start with my package LAGUNA - it was added to GAP distribution when the author was able to decide whether it should be autoloaded or not. My thoughts were that it installs a method for Units for a group ring in case it is a modular group algebra of a finite p-group over a field of p elements, speeds up some operations with p-elements of unit groups of other group algebras, and also speeds up calculations of some of the Lie properties of group algebras. I think all these scenarios are quite specific, and it may be made excluded from the default PackagesToLoad preference.

[james-d-mitchell]

I'd be happy to see all of the packages except those used in the library that you mention (atlasrep, factint and tomlib) excluded from autoloading. I usually explicitly disable autoloading of all these packages anyway in my gap.ini.

[fingolfin]

I think autpgrp is also explicitly loaded (at least it used to be).

For some of the other packages, they fall into slightly different categories for me. For example, polycyclic and polenta provide a new type of groups (PcpGroups), and unless you explicitly create such groups, you'll probably never notice their absence.

In contrast, fga enhances functionality for freegroups. In the past, I had several instances where computations in free groups only worked well because fga was loaded -- and I discovered this only when by chance I run my computations on somebody else's machine, and was surprised that they quickly got stuck in an infinite loop, because that user had removed fga from autoload.

I guess what I am trying to say is this: For some (but probably not all) of these packages, removing them from autoload may cause computations to fail / hang / take longer than with core GAP, and the user may not understand why.

This is why changing this list is somewhat "dangerous" and should be handled with care.

That said, I'd also prefer if we loaded as few of them as possible -- but the hard part is to decide what "possible" means ;-).

[olexandr-konovalov]

My gut feeling is that the list could be reduced to atlasrep, autpgrp, factint, fga, and tomlib. The closure of their needed/suggested will also lead to loading browse, ctbllib and io. I have also seen serious performance problems without autpgrp and fga.

The practical implementation of this could go along removing one package at a time, evaluating Jenkins tests, and then going to the next package, in some order (I'd probably save polycyclic to the last iteration).

[hulpke]

While I have some opinion on what packages are crucial (my autoload list is autpgrp, alnuth, ctbllib,
factint, irredsol, tomlib, transgrp), I agee with @fingolfin 's argument -- unless autoloading of the current packages causes problems, we should be careful in removing these, in particular if they quietly improve performance.

How about simply rewriting the error message:

The packages: autpgrp, factint, ....
were set for autoloading in gap.ini but are not available (see '??gap.ini')

for a more gentle fail.

[ChrisJefferson]

I agree that in general we should leave more packages in, particularly when they extend built-in functionality.

Advanced users can disable packages if they don't want them -- basic users (and not-so-basic, like me) don't know which packages to load to make built-in functionality faster, or why code they have been using for 5 years has suddenly stopped working, or got much slower.

What is the advantage to less auto-loading? Faster GAP start time?

[james-d-mitchell]

@ChrisJefferson, @fingolfin This sounds like an argument for extending the list of autoloaded packages. Many packages extend the built-in functionality, and some calculations only become feasible when certain packages (not currently in the list of autoloaded packages) are loaded.

[Stefan-Kohl]

I think it would make sense to autoload by default precisely all packages from which a GAP user may benefit significantly even without knowing about the existence of the particular package.

[olexandr-konovalov]

@Stefan-Kohl thanks - I've created this issue exactly with the intention to collect relevant details of the packages explaining why they may fit into such category. Maybe you could add a few words on resclasses, please?

[Stefan-Kohl]

@alex-konovalov I think ResClasses is a case of doubt -- a GAP user may benefit from ResClasses without knowing about the package -- but significantly enough? -- I don't know. As a number of utility functions have been moved to the Utils package, maybe the benefit now is less than it was before. For this reason I probably wouldn't put ResClasses on the default autoload list if it wouldn't be there already. On the other hand, removing a package from the list may always cause code not to work any more which worked before -- so unless there are particular reasons to remove ResClasses from the autoload list, I don't see the benefit of doing so.

[olexandr-konovalov]

@hulpke in #1135, I've looked what happens if default:= [ "autpgrp", "alnuth", "ctbllib", "factint", "fga", "irredsol", "tomlib" ]. You've mentioned that you keep alnuth in your autoload list - what are the particular use cases you keep in mind? IIRC, it's not usable if PARI/GP is not installed.

[Stefan-Kohl]

I think packages which cannot be used without additional software which is not distributed with GAP should not be loaded by default. -- That said, of course removing a package from the list always needs some care, in order to avoid breaking anything.

[hulpke]

@alex-konovalov The packages in my list were selected by the criteria:

• No banner
• I might want to use it, or believe it should be part of the main system and not a package. (so transgrp and primgrp will be added)
• It hasn't caused problems in the past by overriding methods
• Nostalgia (I reviewed the thesis on which alnuth was based)

At least bribery did not make the list of criteria...