Vocabulary Roadmap (work in progress)

[possumbilities]

Notice: As there is still much to migrate/change, this Roadmap is considered a work in progress and is still subject to change.

---

This Roadmap is built from an older draft, created within a different context of needs and requirements

• Vocabulary Roadmap 2022.Q4-2023.Q1 (work in progress) #2

---

TOC

• Summary
• History
• Current Challenges
• Objectives
• Unknowns (WIP)
• Related Resources

Summary

Vocabulary has become sprawling, out of date, difficult to maintain, mismatched in documentation, and overly complex. It needs to be reconsidered from multiple angles, and given clarity not just in structure, but also approach.

Goals

1. To reasonably reduce the number of repositories in order to remove unnecessary complexity, and increase maintainability of Vocabulary itself.
2. To solidify a stable enough "foundation" for work to continue with Vocabulary.
3. To update and create consistency among projects where some variation of Vocabulary is a dependency.

Outcomes (proposed)

Code

• creativecommons/vocabulary o--o @creativecommons/vocabulary
• creativecommons/vocabulary-vue o--o @creativecommons/vocabulary-vue

Project Structure / Mechanics as a Design System

- Vocabulary 
	- Design Library (Figma), the first order ref point for all design concepts in Vocabulary
		- Tokens
		- Typography
		- Contexts (behavioral, structural)
		- Components (General non-fw specific components)
	- Fonts (Typefaces), which provides typefaces for use in the system
	- Icons (SVGs), which provides a spritekit for use in the system

	- Native Code implementation, the __source of truth__ for all designs in Vocabulary
		- HTML markup for Contexts and Components, which provides base markup for all components/pages
		- CSS library and assets, which provides base CSS styles for all components.
		- JS library and helpers, which provides a helpful base for all components.

- Vue Vocabulary (Vue.js library), 
	- which provides Vue components __derived from__ Vocabulary

Why?

Most of the issues with Vocabulary sit within two categories:

1. Original scope misaligned with current scope
2. The Consequences of time

The intended original scale and scope of Vocabulary has shifted dramatically since its original creation, creating conflicting objectives and philosophies.

In many current situations code implementations don't necessarily need a node package to implement Vocabulary, they just need a link to a relevant vocabulary.css file hosted on a CDN, and a reference spec for how markup should be output and what adjacent behaviors it should follow.

More complex setups just need a straight-forward Vue Components library that has Vocabulary itself as a dependency.

From a repository standpoint, Vocabulary started as separate repositories, then became a monorepo, and then back to separate repositories in some instances--and the original monorepo in others.

This project seeks to move the core of Vocabulary back to a singular repository, and keep the Vocabulary Vue Components implementation as its own repository. There is insufficient documentation for why Vocabulary was split in precisely the ways it was originally, but some of its difficulty in being maintained either as separate repositories or as a monorepo is in how the project is structured overall. The newly proposed structure seeks to address these issues.

The Current Challenges section does a good job of elaborating further.

History

Note: The creativecommons/cc-assets repository and package are included here for completeness, but they were never fully integrated into the Vocabulary project.

Vocabulary Multi-Repo 2019-2020

Repositories and JavaScript Packages

Repository	JavaScript Package
creativecommons/cc-assets	@creativecommons/cc-assets
creativecommons/fonts	@creativecommons/fonts
creativecommons/vocabulary	@creativecommons/vocabulary
creativecommons/vue-vocabulary	@creativecommons/vue-vocabulary

Vocabulary Monorepo 2020-2022

[RFC] Move vocabulary packages to a monorepo · Issue creativecommons/vocabulary#612

Dedicated repositories archived

Old Repository		New Repository
creativecommons/fonts	➡️	cc-archive/fonts
creativecommons/vue-vocabulary	➡️	cc-archive/vue-vocabulary

Repositories and JavaScript Packages

creativecommons Repo	Directory	JavaScript Package
cc-assets		@creativecommons/cc-assets
vocabulary	./packages/fonts ./packages/vocabulary ./packages/vue-vocabulary	@creativecommons/fonts @creativecommons/vocabulary @creativecommons/vue-vocabulary

Vocabulary Incomplete Re-Split 2022

Split Vocabulary packages into separate projects · Issue creativecommons/vocabulary#1049

Repository directories migrated to new repositories

Vocabulary Directory		New Repository
./packages/fonts	➡️	creativecommons/vocabulary-fonts
./packages/vocabulary	➡️	creativecommons/vocabulary-styles
./packages/vue-vocabulary	➡️	creativecommons/vocabulary-components

Repositories and JavaScript Packages

Due to the incomplete re-split, all of the packages remain in use:

creativecommons Repo	Directory	JavaScript Package
cc-assets		@creativecommons/cc-assets
vocabulary	./packages/fonts ./packages/vocabulary ./packages/vue-vocabulary	@creativecommons/fonts @creativecommons/vocabulary @creativecommons/vue-vocabulary
vocabulary-components		@creativecommons/vocabulary-components
vocabulary-fonts		@creativecommons/vocabulary-fonts
vocabulary-styles		@creativecommons/vocabulary-styles

Current challenges

• Code Rot / Sprawl / Complexity / Performance
• Change in Objective / Scope
• Incomplete Documentation

Code Rot / Sprawl / Complexity / Performance

• As codebases are utilizing two entirely different implementations of Vocabulary, and the original Vocabulary monorepo is suspended many use cases of Vocabulary are very out of date.
• There's cruft left over in various repositories from the split/mergers that have occurred reducing clarity.
• Many of the CD workflows are broken or possibly not needed. (further investigation needed)
• The use of icons as a font is not a sound Accessible approach, and should largely be replaced with some form of SVG implementation
• fonts exist as a package of their own, when there is zero known use for them to be implemented without vocabulary itself.
• the vocabulary styles follow the same practice as fonts. (In some cases vocabulary-styles was the original vocabulary repo).
• unpkg has latency issues, especially if a specific version of a file is not required.
• the use of bulma.js adds a further layer of abstraction that doesn't actually normalize, but instead resets the default behavior of elements.
  • creativecommons/vocabulary#1037
  • bulmajs
• storybook is useful, but also complex. It needs to be reproached with more clarity, or replaced.
  • creativecommons/vocabulary#630
  • creativecommons/vocabulary#905

Change in Objective / Scope

Node (NPM) as a CDN Pipeline to unpkg

Originally, Vocabulary was built as a robust design system to be used from an application-centric focus. This meant that there were certain challenges when implementing it within setups such as WordPress, since the main core focus was from compatibility with the Node System, and more specifically with Vue.js.

Since then, the focus of Vocabulary use-cases are much more WordPress / Static site centric, with app uses as the secondary priority.

In this way, currently, NPM's major function, and by proxy these repositories existing as Node packages, is to publish them easily to a CDN (unpkg), not to necessarily be consumed as as dependency in Node projects. Individual files (primarily the font and css files) are then often used as <link> includes in relevant codebases.

This isn't entirely untenable, but there are likely alternative CDN solutions for this use-case that don't primarily rely on the NPM ecosystem or on unpkg.

Source of truth and conflicting design system mechanics

From the Storybook documentation:

The overall Vocabulary project consists of:

• Design Library (Figma), the source of truth for all designs in Vocabulary
• Fonts (Typefaces), which provides typefaces and the main icon font for use in the system
• Vocabulary (CSS library and assets), which provides base CSS styles for all components.
• Vue Vocabulary (Vue.js library), which provides Vue components derived from Vocabulary

A design system that's primarily used within code, needs its source of truth to be actual code, and more specifically the meaning and presentation, and often behavior of various design elements. A program like Figma can somewhat provide the presentation part of the source of truth, but as it is more visually focused it is unreliable for the actual code (CSS) implementation of that presentation, and even less so for the meaning, which is mostly derived from the HTML markup. As for behavior, the results can largely be mixed from relying on Figma. This means resting the source of truth on Figma results in the presentation being incomplete, the behavior being inconsistent, and the meaning having to be derived elsewhere.

As a result, coming from an app-centric focus, the Vue.js Components and Vocabulary Styles ended up being closer to the actual original "source of truth" for meaning + some behavior + presentation, and this meant that Vue and by proxy Node conventions set the tone and direction of how and where Vocabulary could/should be used. This likely made solid sense considering the original focus and the original use of Vue.js in a larger application-centric approach.

This doesn't make as much stable sense in the present, where WordPress themes largely need a CSS file of appropriate styles, and a directed documented implementation guide for how the markup should be output. The most direct implementation being as Gutenberg blocks on the b/e and as PHP code for the f/e rendering

There is also the possibility of the WordPress theme utilizing Vue.js Components into the mix, which paired with the backend editor (Gutenberg) being in React.js may introduce unnecessary complexity.

The Vue.js components have a use, but they should be an implementation of a source of truth not the original source of truth. The current configuration leans too heavily on the latter.

Change in scale

At greater scale (multiple apps, higher number of engineers + teams) high reliability on a f/e framework (Vue.js, React.js, etc.) as the majority source of truth for components can be worth any complexities or missing pillars that come along because they help to not just enforce a Component, but to enforce that component's markup in a more direct, but also abstract pathway.

When the scale is not quite as large, these tradeoffs can come at a much higher cost. Additionally, if in the future there is a need to pivot away from utilizing a specific framework as the source of truth, then the migration path can be challenging since there is no framework agnostic source of truth to go back to as reference.

In general, if the project isn't going to have the kind of scale of a larger SAAS/PAAS company, it can make more sense for a Design System to first exist as a singular source of truth of components in as native of a format as possible (HTML, CSS, helper JS). There is some documentation of this within storybook, but it is inconsistent, some of the markup relying only on Vue markup.

Then, you can have various implementations in frameworks as they become relevant (Vue.js, React.js, etc.). This provides a stable source of truth that will remain relevant regardless of needed implementation. Changes to core Vocabulary can then cascade down to the individualistic implementations with their own specific nuances.

There are many sound reasons to pick one approach over another, or to pursue additional routes. However, much has changed since the original creation of Vocabulary, most importantly its use requirements, which include its scale. This has resulted in a high degree of complexity and code rot, which stands in the way of stable maintainability and accurate documentation.

Incomplete Documentation

• The CI/CD pipeline and release process isn't documented in a cohesive location or format.
  • creativecommons/vocabulary#602
• How or where Vocabulary is used in various codebases isn't thoroughly documented, some repositories having overlapping versions

Objectives (work in progress)

Setup

• creativecommons/vocabulary
  • draft Issue outlining roadmap of changes / 2022.Q4 - 2023.Q1 Roadmap
  • paragraph for each phase breakout
  • adjacent repo specific issues needed?

• tag save points: vocabulary-2022.Q1-hybrid
  • core repositories:
    • creativecommons/vocabulary
    • creativecommons/cc-assets
    • creativecommons/vocabulary-components
    • creativecommons/vocabulary-fonts
    • creativecommons/vocabulary-styles
  • downstream repositories?: (decided to not tag these, as they may receive work outside this scope of this project, and any benefit a Tag would have here to cycle backward would be easily negated by that work)
    • creativecommons/creativecommons-certificate
    • creativecommons/wp-theme-openglam
    • creativecommons/creativecommons.github.io-source
    • creativecommons/wp-theme-summit
    • creativecommons/cc-global-components
    • creativecommons/cc-legal-tools-data
    • creativecommons/cc-legal-tools-app
    • creativecommons/creativecommons-base
    • creativecommons/chooser

Phase 1

Involves the creativecommons/vocabulary repository once again becoming the central hub of work, after several repositories have been combined, their functionality and associated NPM packages will be updated to reflect those changes, followed by changes with any projects which utilize those repositories as a dependency in some way.

1.1 Setup - creativecommons/vocabulary

• Disable GH Workflows

1.2 Return a stable set of src files to work from - creativecommons/vocabulary

• deprecate existing files in /src by moving to /src/_deprecated/ for later reference
• track down and find the last stable versions of the creativecommons/vocabulary/packages/vocabulary/src/styles` directory
• combine last stable style files in this repository with any relevant changes made to those same files in the creativecommons/vocabulary-styles repo.
• place returned style files in /src/scss/ directory
• track down and find last stable versions of the creativecommons/vocabulary/packages/fonts/src/ files
• place returned font files in /src/fonts/ directory
• vendorize bulma into /src/scss/vendor/ to be more hermetic in builds and set foundation to deprecate later

1.3 Make a versioning decision - creativecommons/vocabulary

• determine which versioning to continue with semver vs calver: answer: semver
• determine which number/version the project should continue from : answer: 'v2.0.0-alpha`

1.4 Node cleanup - creativecommons/vocabulary

• reset package.json and package-lock.json files removing dependencies
• correct any relevant information in the package.json file meta data
• research the implications or routes for keeping the @creativecommons/vocabulary namespace
• if not possible to keep vocabulary namespace, what is the alternative? @creativecommons/vocabulary-core
• update relevant package information to reflect prior changes

1.5 Prepare and move forward with pre-release - creativecommons/vocabulary

• make a public /dist/css directory to house the vocabulary.css and vocabulary.css.map files going forward
• successfully build new vocabulary.css file in /dist/css/
• prepare 2.0.0-alpha pre-release
• tag 2.0.0-alpha pre-release

⚡ [milestone] - stable MVP foundation reconstructed with as much git history intact across relevant repositories.

1.6 Publish 2.0.0-alpha pre-release - creativecommons/vocabulary

• publish to npm (if relevant) as @creativecommons/vocabulary-core
• publish to unpkg (if relevant)
• decided to not publish to npm until at least a stable v2.0.0 release, possibly not at all (depending)

1.7 Ensure repo follows current Repo Guidelines - creativecommons/vocabulary

• /.github/CODEOWNERS
• /.cc-metadata.yml
• /CODE_OF_CONDUCT.md
• /LICENSE
• /CONTRIBUTING.md
• /README.md
• CI and code style linters that run automatically whenever new code is pushed (if applicable to the project)
• At least a couple of automated tests (if applicable to the project)

1.8 Make determination on remaining repos

• keep cc-assets as standalone repo
• rename creativecommons/vocabulary-fonts --> creativecommons/cc-fonts
• archive creativecommons/vocabulary-styles now that relevant code is merged into creativecommons/vocabulary
• further research best route for creativecommons/vocabulary-components; rename, new repo, archive

1.9 +

• creativecommons/vocabulary#1092
• creativecommons/vocabulary#1093

[After several weeks of archaeology the original roadmap proposed for Phase 1 needs further consideration and updates on the best route forward.]

Phase 2 CDN Reconsideration

Vocabulary should not rely on being primarily a node package published to and served via npm/unpkg. This can remain secondarily, especially for node ecosystem specific projects, but for any other use cases that just need a <link> reference to utilize vocabulary there are better approaches, namely a simplistic CDN deployment pipeline that is not unpkg.

• explore alternatives to unkpg
• establish baseline requirements (versioning, serving location scale, etc.)
• unpkg/@creativecommons/vocabulary
  • vocabulary.css --> [custom cdn]
  • vocabulary.js --> [custom cdn]

Phase 3 - Refactor Vocabulary to be proper source of truth

Much of the work thus far has mostly been to get the repositories reduced, clarify the code, and make the CDN setup more robust. However, Vocabulary itself still needs the bulk of its code to be refactored. There is a strong likelihood that bulma needs to be removed/replaced, and that storybook needs a reconsideration.

• creativecommons/vocabulary --> [refactor transforms as proper design system hub, remove bulma]
• creativecommons/vocabulary --> [refactor builds css + js files] --> outputs to non unpkg CDN
• creativecommons/vocabulary-vue --> [refactor with vocabulary dependency, matching new spec}
• creativecommons/vocabulary --> [rewrite Storybook (or equivalent) to be new system explorer]

Future Considerations

• creativecommons/vocabulary-web-components --> [new repo?]
• creativecommons/vocabulary-react --> [new repo?]
• creativecommons/vocabulary
  • vocabulary-web-components.js --> [custom cdn?]

Unknowns (WIP)

• The use of storybook as a documentation/site tool may need to be replaced or possibly just changed in structure or use, but not enough is known to make a solid call
• Tests have suffered from issues dealing with mostly time and code rot, the shifting scope of Vocabulary also impacts the expected outcomes of various functionality, as well as the possibility of changing up tooling. Which means Tests overall are in some instances intentionally disabled, in others broken from the side effects of time, and will likely need a revisit as things shift within Vocabulary's new scope. All this to stay, knowing exactly what to do with tests is going to be messy. They may need to be taken out entirely, and then reconsidered in a full refactor.
• It would be handy to run tests through something like Browserstack, no idea just yet if an equivalent exists in the current setup, or if there's solid reason to do so down the line
• Establishing the correct versioning scheme to follow going forward (semver vs calver). Using calver with the two digit month breaks on npm/unpkg as it removes the extraneous 0 to be compatible with semver (which it expects). To account for this, and still use calver means dropping the extraneous 0 on all js projects, but then across the board versioning in some projects has the zero and in others does not, creating inconsistency
  • creativecommons/vocabulary#588
  • Coding standards creativecommons-base#2 (review)
  • https://github.com/creativecommons/tech-support/issues/833
• The creativecommons/cc-assets repository has items licensed under the Creative Commons Trademark policy, bundling them in with another repo will require some LICENSE language delineating that they are not necessarily licensed the same as the rest of the repo or its assets.
  • At current the logos from creativecommons/cc-assets are included within the creativecommons/vocabulary repo already, but there is no delineation made about licensing in Vocabulary's LICENSE file, or its README.md file in regards to those items.

Related Resources

• Wikimedia Style Guide
• Wikimedia Style Guide (on GitHub)
• Polaris by Shopify
• Polaris on GitHub
• VuePress
• GitHub Pages
• Twitter Bootstrap