It takes a long time to open the history page of a folder if the history is very long

[tonzhao]

Is your feature request related to a problem? Please describe.
It takes a long time to open the history page of a folder if the history is very long.
Because OpenGrok incrementally updates the history cache based on the modification time of the file, and the modification of files in subfolders does not affect the timestamp of the top-level folder, so OpenGrok dynamically loads the history information of the folder through JGit. For a folder with 65K histories, OpenGrok-1.7.21 takes about 2 minutes to load all the history information.

Describe the solution you'd like
Since the history page is displayed in pagination, server-side pagination should greatly improve the page loading speed and user experience.

Describe alternatives you've considered
None

Additional context
None

[vladak]

Firstly, there is no history cache for directories (#1704 - not easy to do) which means that when retrieving history for a directory, HistoryGuru allways falls back to the repository method.

The problem is that the webapp retrieves the history as a whole:

opengrok/opengrok-web/src/main/webapp/history.jsp

Line 86 in b269f53

hist = HistoryGuru.getInstance().getHistoryUI(file);

This means that the complete history of given directory is gathered first into the History object and then passed to the JSP, hence the delay. The History object is also dragged around in the session:

opengrok/opengrok-web/src/main/webapp/history.jsp

Line 101 in b269f53

request.setAttribute("history.jsp-hist", hist);

causing significant heap usage as noted in #3541.

To address this, the method needs to be replaced by some sort of history reader or a stream or iterator that would keep the state in the session however would allow to return data (HistoryEntry objects) on the fly. For repositories that execute external commands (like Mercurial), that would mean keeping the process around until the session expires or the command finishes (the history entries are drained). Similar approach should be then applicable when reading from the history cache (once the format allows it - #3539).

[vladak]

The UX aspect also needs to be taken into account. With iterator, the number of history entries to be displayed will not be known before hand so there would have to be the double arrow link >> preceded with page numbers already visited. The final number can be displayed only when the iterator is completed. Also, since the history.jsp code to display the history entries does not really make much difference between the directory and file mode, it will have to be carefully adapted, until the history cache interface allows for iterative access too. Or simulate iterative access on History level. Or simply unify the paging behavior, i.e. do not display expectant page numbers even for file history.

Anyhow, solely the iterator is not applicable to the paging as it should be able possible to move backwards. There needs to be some caching of history entries per session, likely limited to some amount to avoid the situation described in #3541. Currently the History object is stored in the request (as the history.jsp-hist attribute), not in the session.