Save manifest files separate from bundles for PP portlet inspection

[john-thomas-dotcms]

Parent Issue

No response

User Story

As an administrator or support engineer, I want to be able to view the contents of a bundle that was pushed even after the bundle is deleted from the disk, so I can investigate PP issues and operation without taking large amounts of disk space to store full bundles.

Currently:

• Every time a bundle is pushed, information about it is stored in 2 places:
  • Information (metadata) about the push is stored in the DB.
  • The full bundle (.tar.gz) is stored on the disk.
• When the PP portlet loads push history, it reads the list of bundles from the DB
• If the user clicks on a bundle in the list to find more information on it, it opens the bundle and retrieves the manifest file from the bundle.

Problem:

• Bundles can take a huge amount of disk space.
• Because bundles take so much space, for cloud customers we delete bundles after a short time (3-7 days).
• After a bundle has been deleted from the disk, the PP portlet will show the bundle in the list, but the user will not be able to view any information about the bundle.
  • Attempts to open the bundle information will generate an error.

Solution:

• When saving the bundle to the disk, also save a copy of just the manifest to the disk.
  • A copy of the manifest will also still be saved within the bundle; the separate copy saved to disk will be an unpackaged duplicate.
• Change the PP portlet to retrieve information about bundles from the unpackaged manifest files, rather than from the full bundles.

This means that to save disk space, the full bundles can be deleted, but the unpackaged manifests can be left on disk. And that means that the PP portlet will still be able to retrieve information about old bundles after the (disk-hogging) bundles are gone.

Acceptance Criteria

1. When a user clicks on an item in the PP portlet history list:

   • If the unpackaged manifest file exists:
     • It's opened to retrieve the bundle information.
   • If the unpackaged manifest file does not exist, but the full manifest does exist:
     • The full bundle is opened to retrieve the bundle information.
     • Note: The manifest file should not be saved as an unpackaged manifest file in this case.
       • This would cause inconsistent and support-ticket-generating behavior.
   • If neither of the files exist:
     • An error message should be displayed, Information about this bundle is no longer available.

2. When the "Download Manifest" button in the bundle information popup is clicked:

   • The unpackaged manifest is downloaded.

3. All operations in item 1, above, work on both sender and receiver

   • The bundle manifest is extracted and saved as an unpackaged manifest.

4. When a bundle is deleted from dotCMS:

   • Both the full bundle and the unpackaged manifest file for that bundle are also deleted.

Proposed Objective

Customer Support

Proposed Priority

Priority 3 - Average

External Links... Slack Conversations, Support Tickets, Figma Designs, etc.

• Problem and Mitigation (public channel): https://dotcms.slack.com/archives/C028Z3R2D/p1724085444893659
• Implementation Discussion (private team channel): https://dotcms.slack.com/archives/C058GCQ7T39/p1724087650078899

Assumptions & Initiation Needs

No response

Quality Assurance Notes & Workarounds

No response

Sub-Tasks & Estimates

No response