Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRM-21179 - Provide copies of dc.js, etal, for use by extensions #11005

Merged
merged 4 commits into from
Oct 9, 2017

Conversation

totten
Copy link
Member

@totten totten commented Sep 20, 2017

Overview

This PR adds a new asset, visual-bundle.{js,css}, which combines together dc.js, d3.js and crossfilter.js. The aim is to build on key elements of Civisualize in a way that allows broader community-of-practice. The libraries and versions are carefully chosen to match Civisualize.

visual-bundle.{js,css} is provided as a resource for developers -- but this PR doesn't actively use the resource within core.

Note: This is the product of a fairly long conversation with @colemanw, @jamienovick, and @guanhuan about how to manage the dependencies on these libraries. I'll try to recapture that conversation below.

Before

  • civicrm-core includes a copy of d3 v3.4.x. It can be manually loaded as [civicrm.bower]/d3/d3.js.
  • civisualize includes a copy of d3 v3.5.x, dc v2.1.x, and crossfilter v1.3.x.

After

  • For backward-compatibility, all previous copies of d3, etal, are still available.
  • Additionally, civicrm-core provides an optional visual-bundle.{js,css}:
    • It combines d3 v3.5.x, dc v2.1.x, and crossfilter v1.3.x. (These basically match civisualize, but they include small patch-level updates.)
    • To allow visual-bundle.js to coexist with other libraries, all JS services are namespaced (much like CRM.$ or CRM._). Speciifcally:
      • d3 => CRM.visual.d3
      • dc => CRM.visual.dc
      • crossfilter => CRM.visual.crossfilter
    • The library is not loaded by default. To load it, one should lookup the URL (Civi::service('asset_manager')->getUrl('visual-bundle.js')) and include that file.

Comments

Several points were raised during the previous discussion. (Note: I'm acting as a rapporteur in trying to capture these points -- not every person agrees completely with every point, but mostly they're valid points.)

  • We'd like to support a "community of practice" that stays aligned with Civisualize.
  • Civisualize is an extension which bundles together these elements:
    • (a) JS Libraries: dc.js and d3.js and crossfilter.js
    • (b) Data Loading: Some PHP/Smarty/JS helpers for loading data.
    • (c) Report UIs: A set of specific reports for contributions, memberships, etc -- and various decisions about UX/navigation/permissions.
  • For CiviCase v5 dashboard, the visualizations should use the same JS libraries/concepts, but the pageflows and organizational contexts can be rather different from a typical Civisualize deployment. (To wit: they don't really use contribution/membership reporting.)
  • There are several ways to handle distribution/bundling of the JS libraries in a way which would satisfy these points. No approach is perfect -- they all require some kind of compromise. These include:
    • Punt to downstream: Don't bundle the JS libraries. Instead, prompt the sysadmin to do so. This avoids making any real commitment, but it creates dependency-hell for sysadmins, and it leaves a lot of ambiguity for devs.
    • Depend on Civisualize: To get the full value of org.civicrm.civicase, you should also install Civisualize. This doesn't require any new packaging work, but it exposes pure-CiviCase users to the contribution/membership/etc reporting.
    • Ask to split Civisualize into smaller extensions: Ex civisualize_reports and civisualize_sdk. However, this is a big ask, and it creates a significant impact on sysadmins when they upgrade, and we still have issues with dependency management in core.
    • Convert to composer for distribution: This is potentially elegant, but it puts major design questions and downstream impacts on the critical-path for CiviCase.
    • Copy/move the JS libraries to civicrm-core: This allows multiple extensions to load the same copy+versions of the libraries. However, it puts core in the hotseat if anyone wants to upgrade or mix-in different versions.
  • This PR implements the last one. To address the concern about putting core in the hotseat for library upgrades, it includes a few measures:
    • These copies of dc/d3/crossfilter are namespaced under CRM.visual.*.
    • The visual-bundle.js is not loaded by default. It's purely opt-in.
    • If a future major upgrade is a required (eg d3 v4 plus some future version of dc that's compatible with it), then that can be packaged separately (eg visual-bundle-2.js and CRM.visual2.*). The visual-bundle.js and visual-bundle-2.js can coexist.

*/

/**
* This class defines teh `visual-bundle.js` asset, which combines `dc.js`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

teh typo

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

teh world's best typo

This defines an asset named `visual-bundle.js`.  It creates a custom build
of `dc.js` + `crossfilter.js` + `d3.js` which loads all the services into
the namespace `CRM.visual.{foo}`, which allows it to coexist with other
copies of the library.
@totten totten force-pushed the master-visual-bundle branch 2 times, most recently from 5250630 to 3229d04 Compare September 22, 2017 18:56
…her ext's

== Before ==

In `hook_civicrm_angularModules`, the `js` resources are assumed
to be file-blobs relative to the declared extension.

== After ==

In `hook_civicrm_angularModules`, the `js` resources can be:

 * `assetBuilder://NAME?PARAMS`; in which case the JS file is loaded from the `asset_builder` service
 * `ext://EXTKEY/FILE`; in which case the JS file is loaded form another extension
 * Otherwise, the resource is assumed to be file-blob relative to the declared extension
@totten totten changed the title (WIP) Provide copies of dc.js, etal, for use by extensions CRM-21179 - Provide copies of dc.js, etal, for use by extensions Sep 25, 2017
@colemanw
Copy link
Member

@totten - IMO we should not be newly adopting old versions of libraries without good reason. And since you are continuing to include the old version as before to maintain Civivisualize support, I don't see the reason our new visual bundle shouldn't be up-to-date.
When @tttp updates his extension it can take advantage of the newest libraries, but for now it should continue to work, correct?

@jamienovick
Copy link

Agree. Lets go new. Also any views of using c3 as its a nice wrapper for d3 and looks nice? http://c3js.org/ ?

@totten
Copy link
Member Author

totten commented Sep 27, 2017

@colemanw This uses the latest stable version of dc (2.1.x) along with its dependencies.

d3 v3=>v4 is a major change -- more akin to Drupal 6=>7 or Angular 1=>2. Both dc and c3 are based on d3 v3.x. They are not currently compatible with d3 v4.x. (See eg this and this and this.)

@jamienovick We had mockups and discussions in cases about trade-offs between dc.js and c3.js. Recap:

  • dc.js keeps us aligned with Civisualize - so that we're using the same concepts/terminology/etc. It's better for implementing interactive/multipart panels. Trade-off: it's documentation is more arcane.
  • c3.js has nicer documentation and more stylized defaults. Trade-off: it's harder to undo/remove bits that you don't want (which posed immediate issues in their donut chart), and it diverges from Civisualize community-of-practice.

@tttp
Copy link
Contributor

tttp commented Sep 27, 2017 via email

@totten
Copy link
Member Author

totten commented Sep 28, 2017

Good impulse. For some context -- the impetus here is to support CRM-21179 and civicrm/org.civicrm.civicase#45 (which is now already implemented on the dc 2.1 stack). Previously, I wrote a few fiddles to compare those CiviCase charts with dc and c3 -- and shared with the other CiviCase folks. This is how I came to the opinions ^^^.

The fiddles were:

  • Donut chart: dc and c3
  • Horizontal bar chart: dc and c3

Note that those fiddles are not apples-to-apples -- I continued iterating on the dc examples to get closer to the mockup. The c3 example is closer to stock. I did investigate making the c3 examples closer to the mockup -- and found them slightly-easier or slightly-harder, depending on which compromises one is arbitrarily (un)willing to make. It's basically a wash IMHO, except for the points from earlier: dc is more pithy with interactive/multipart dashboards, and c3 has clearer docs, but c3 unnecessarily splits our community-of-practice.

@colemanw colemanw merged commit 4d2aea5 into civicrm:master Oct 9, 2017
@totten totten deleted the master-visual-bundle branch October 9, 2017 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants