PLIP: Restructure CMFPlone static resources

[thet]

Proposer : Johannes Raggam

Seconder : None by yet

PLIP configuration: https://github.com/plone/buildout.coredev/blob/5.2/plips/plip-1653-staticresources.cfg

Repos:

• PLIP 1653 - Restructure static resources #2542
• PLIP 1653 - Restructure static resources mockup#874
• PLIP 1653 - Restructure static resources plone.app.theming#149
• PLIP 1653 - Restructure static resources plone.app.upgrade#184
• PLIP 1653 - Restructure static resources plone.resourceeditor#22
• https://github.com/plone/plone.staticresources

Abstract

This PLIP is about the separation of Plone's static resources from CMFPlone.

Plone's own bower components directory (Products.CMFPlone.static.components) is hard to understand, maintain and keep up to date.

While all patterns reside in mockup the toolbar pattern and After the deadline TinyMCE plugin are still kept in CMFPlone.

The static resources take a lot of disc space, fill up the git history and make searching in files cumbersome.

This PLIP proposes to seperate those static resources from CMFPlone into a dedicated package plone.staticresources.

Motivation

• Complexity - many related resources are spread over different packages and directories. This makes the current structure hard to understand.

• Maintainability - It's cumbersome, errorprone and easy to forget to keep bower.json in CMFPlone and mockup in sync.

• Depending on a specific version of static resource bundles (via bower, npm or python version specifiers) would even be easier if we do not maintain two similar repositories.

• The process of upgrading CMFPlone static resources and just keeping the minimal necessary set excluding unused thing like test and demo files is difficult (includes maintaining the .gitignore file)

• The static resources fill up a lot of space in CMFPlone. The static directory counts currently 91MB.

• Each change to the static directory (e.g. updating the components directory) adds to the git history.

• Searching within files is cumbersome - if you do not exclude the static directory or explicitly include only python files, most of the hits come from the static directory.

• All Plone related patterns live in mockup - so should the toolbar pattern. It's more likely that it gets more attention than being in CMFPlone. And mockup (or maybe even more patternslib) IS the best-practice resource of creating patterns.

Proposal & Implementation

• Create a new package plone.staticresources.

• Move the toolbar pattern to mockup.

• Create a npm package for mockup.

• Create a npm package for patternslib.

• Manage those two dependencies and all other via npm.

• Git-include the node_modules resource directory.

• Register necessary resources as plone static resources or browser resources, like it was done in CMFPlone.

• Register all resources via registry.xml, like it was done in CMFPlone.

• These changes include the plone and plone-logged-in bundles also.

• Remove the resource registration in registry.xml, the plone static resource and browser resource registrations as well as all resource files from CMFPlone.

• Remove resource registrations from mockup, as this is now done in plone.staticresources and make mockup a JS-only package by removing all Python related setup code.

• Raise versions for CMFPlone to 5.2 and mockup to 3.0.

Risks

Low risks, as all resource paths keep the same (++resource++mockup, ++resource++plone).

The node_modules directory should be provided as components, as like the current bower directory as like the current bower directory, to be able to keep the same paths like before.

Metadata

Updated version from 2017-02-21.
/cc @davilima6 @agitator @plone/framework-team

[davilima6]

This is so important I believe it should be considered a blocker for Plone 5.1.

[thet]

@davilima6 I also want to have this in 5.1. It's quite a big change, even if it does not break backwards compatibility. 5.1 should be the "bugfix" release, where major dot-zero release oddities are resolved.

Eventually and alternatively to commiting the bower_components to Github we could add it to the egg release only. That requires an additional buildout step to run npm install in the mockup directory, when using the mockup development version in Plone.

[jensens]

@thet semantical versioning: With X.Y.Z a Y indicates added features w/o breaking backward compatibility. Z is a bugfix release.

[jensens]

FWT added this PLIP to its list in "submitted" state. It will be voted on now.

[gforcada]

@thet wouldn't be a better approach to get a buildout file on the plips folder in buildout.coredev? This way you can commit right away and your configuration changes and updates are always running with latest changes.

I can create a jenkins job for it as soon as the buildout file is there 👍

[thet]

here it is: https://github.com/plone/buildout.coredev/blob/5.2/plips/plip-1653-staticresources.cfg

thanks for setting up a jenkins job!

[gforcada]

@thet there it is: https://jenkins.plone.org/view/PLIPs/job/plip-plone-staticresources/

[thet]

@jensens #1653 should soon be ready to review. I just need to get the PLIP jenkins job running and fix tests, if they fail. PLIP jenkins job runner needs a fix, see: plone/buildout.coredev#549

[gforcada]

@thet PLIP jobs, hopefully fixed and now with 3.6 and 3.7 support :-)

[thet]

OMG! https://jenkins.plone.org/view/PLIPs/job/plip-plone-staticresources-2.7/5/ passed! After two runs with flaky robot tests.
Now testing for 3.6 and 3.7

[thet]

3.7 passed: https://jenkins.plone.org/view/PLIPs/job/plip-plone-staticresources-3.7/3/
3.6 failed with another single error. each test run something different...

[thet]

3.6 passed: https://jenkins.plone.org/view/PLIPs/job/plip-plone-staticresources-3.6/

ready to review.

There are two more things I want to do, but that can probably also done after review:

• Add an extra profile+uninstall profile for Patternslib patterns.
• Resolve security alerts: https://github.com/plone/plone.staticresources/network/alerts But since this is more a Bug with current mockup, it's OK to solve it after review/merging - IMO.

[ale-rt]

Added an initial review: https://github.com/plone/buildout.coredev/blob/5.2/plips/reviews/plip-1653-review-ale-rt.rst
Feel free to comment :)

CC @thet @jensens

[gforcada]

@thet just thinking out loud, so don't put too much time thinking on it: if mockup and patternslib are released as NPM, would it be possible at some point to create a complete plain theme without @datakurre webpack plugin, by just npm installing these two packages (and whatever else might be needed)? Or this is out of the scope of this PLIP (but could be proposed as another PLIP to build on top of this one?)

[ale-rt]

Relevant branches merged

[jensens]

so, whats left? Some bits of documentation are missing?

[ale-rt]

Yep documentation.
I also noticed that the plone-compile-resources is not generated by the coredev buildout.
I am checking this issue now.

[thet]

This is already a reality since 5.2.