line/area chart transitions are weird when there are both updates and adds

[gordonwoodhull]

looks like adds/updates are out of sequence.

example: change the time range on the stock example. just moving forward forward/backward in time looks mostly okay, but changing the duration draws some lines first, some lines later, and the fill seems to follow the lines.

[gordonwoodhull]

I am not sure if it's related, but the same sort of out-of-order adds/updates is present in the series chart example.

[gordonwoodhull]

this appears to be a regression from 1.6, where there is a little bit of distortion but not the lines and fill moving at different times

[gordonwoodhull]

I think a good part of the weirdness is there doesn't seem to be any data join for the points in the line - neither key nor index-based. See bostock's path transitions.

It just transitions whatever path/area/points are there to the next one. The only join is on the stacks.

[gordonwoodhull]

Correction: the dots are joined, but not the lines or areas.

[linden-dg]

I'm very new to dc.js, but this is the one of the first things I noticed - the transitions on line and stacked area charts don't look right, particularly when using a rangeChart to filter the date range (I've been using them primarily for timeseries charts).

(If you've already seen Mike Bostock's path transition page then you probably know all this, but I’m writing it for my peace of mind!)

I think what's happening is that the chart is going through the enter/exit process, then updating the 'd' attribute for the remaining points to match the new filtered date range.
Using an example- if the size of the filtered range doesn't change (i.e. the rangeChart brush stays at the same zoom, but is shifted left or right), then from d3's perspective there are no new points, and you’re just changing the position of the existing points. This means that rather than shifting the points left or right to match the new range, the points just move up and down.
If new data points are added (e.g. the zoom level of the rangeChart brush is changed), then the new points are added to the map before the transition is called. Because they’re new points, they don’t have an existing position to transition from, so they’re just placed at the post-transition position (I think). This causes the ugly transition where the new points are in the post-transition position, while the existing points shift from their pre-transition position to the post-positon, causing the lines and shaded area to go all over the place.

What the chart should do is enter the new data points outside the filtered range (e.g. hidden outside of the clip path/svg), then translate the chart to match the new range, then exit/remove the old data points outside of the range (which is now hidden off screen). This would give a clean transition from left to right (or in and out), and maintain object constancy.
Strangely enough it looks like the axis already uses this transition structure (though I could be wrong).

Hopefully that makes sense. I tried to mess around with the source code to get the transition working, but broke it spectacularly! Once I’m more familiar with the dc.js code I can give it another crack though.

[gordonwoodhull]

Thanks @ldalegandar, I agree with this assessment and it is helpful that you described it so well.

The reason why it works for the axes is that those are part of d3 itself, so of course mbostock got it right. The transitions in dc.js were written by a variety of people, probably none as expert as he.

It is probably a simple fix, once we figure out what to do... any help would be appreciated!

[linden-dg]

So after spending ages trying to get my head around this, I realised the range chart doesn't actually need to apply a filter to the focus chart - it only need to hide the data points outside of the selected domain/date range (i.e. they're still contained in the svg, but hidden outside the clippath). This means the focus chart is essentially drawn once and the zoom/brush just updates what is seen on the page.

Long story short, this can be easily done by removing the domainFilter in the dc.stackMixin. To see it in action, comment out line 3110:

layer.values = layer.values.filter(domainFilter());

Removing this filter does break elasticY as the maximum doesn't change, because we're no longer filtering the values. We'd need to update the elasticY code to only look at values within the current domain. Working on that now.

Haven't seen anything else break, but I haven't thoroughly tested it yet either.

[linden-dg]

And looks like if you add the domainFilter back into the yAxisMax calculation then elasticY works again.

At least for the stackMixin, changing:

function flattenStack() {
        return _chart.data().reduce(function (all, layer) {
            return all.concat(layer.values);
        }, []);
    }

To this fixes the elasticY:

function flattenStack() {
        return _chart.data().reduce(function (all, layer) {
            return all.concat(layer.values.filter(domainFilter()));
        }, []);
    }

(flattenStack is called in the _chart.yAxisMax function).

[gordonwoodhull]

@ldalegandar, that's an interesting fix. I wonder if it is necessary to change the data in order to fix the transition.

[monfera]

@gordonwoodhull re your previous comment on this ticket, the artifacts seem related to the ones documented here: http://bost.ocks.org/mike/path/ - sorry if it's trivial or unrelated.

[gordonwoodhull]

@monfera, yep, I linked to that above. Thanks.

[monfera]

@gordonwoodhull You're right! I missed it on a quick scan.

[dahlbyk]

Perhaps relevant to folks who find this:

#949 (comment): Until this is fixed properly, there is a flag evadeDomainFilter which just skips filtering of points. It's available in dc.js 2.0.4 and 2.1.7.

[gordonwoodhull]

Note that while evadeDomainFilter avoids the problem with updates and adds, removes still will cause extra animations.