Data disappearing while zooming/panning when y-values are not ordered by x

[gjpc]

Modify the Scatter plot demo thus:

add a line 99:

    double xv = (double) (arc4random_uniform(count)) + 3;

modify the next line which is now 100:

    [yVals1 addObject:[[ChartDataEntry alloc] initWithValue:val xIndex: xv]];

Start panning and zooming and many of the data points will not be rendered. Also, the value labels are frequently not rendered.

The problem is in ScatterChartRenderer.swift this code appears in two places:

            if (!viewPortHandler.isInBoundsRight(point.x))
            {
                break;
            }

This code at lines 91 and 217 assume that the X data is ordered. Removing the conditional break's fixes the problem.
in fork: https://github.com/gjpc/ios-charts

Patch:

diff --git a/Charts/Classes/Renderers/ScatterChartRenderer.swift b/Charts/Classes/Renderers/ScatterChartRenderer.swift
index 073e45c..f688630 100644
--- a/Charts/Classes/Renderers/ScatterChartRenderer.swift
+++ b/Charts/Classes/Renderers/ScatterChartRenderer.swift
@@ -87,12 +87,7 @@ public class ScatterChartRenderer: ChartDataRendererBase
             point.x = CGFloat(e.xIndex);
             point.y = CGFloat(e.value) * phaseY;
             point = CGPointApplyAffineTransform(point, valueToPixelMatrix);            
-            
-            if (!viewPortHandler.isInBoundsRight(point.x))
-            {
-                break;
-            }
-            
+                        
             if (!viewPortHandler.isInBoundsLeft(point.x) || !viewPortHandler.isInBoundsY(point.y))
             {
                 continue;
@@ -213,12 +208,7 @@ public class ScatterChartRenderer: ChartDataRendererBase
                 var lineHeight = valueFont.lineHeight;

                 for (var j = 0, count = Int(ceil(CGFloat(positions.count) * _animator.phaseX)); j < count; j++)
-                {
-                    if (!viewPortHandler.isInBoundsRight(positions[j].x))
-                    {
-                        break;
-                    }
-                    
+                {                   
                     // make sure the lines don't do shitty things outside bounds
                     if ((!viewPortHandler.isInBoundsLeft(positions[j].x)
                         || !viewPortHandler.isInBoundsY(positions[j].y)))

[gjpc]

Though the patch allows viewing and interaction with a non x ordered scatter data set, utility functions such as entryIndex expect ordered x data and fail. So the patch is only good if you are just displaying the data. There might be other package architectural dependencies on ordered x data. Anyone in the project care to comment?

However given issue #82 the bad behavior may be rooted in other bugs.

[danielgindi]

Actually, many functions in the project depends on the y values being ordered on the x axis. That's because in many places we know that we can stop at a certain point, and not loop through more data. This is done for either rendering or even just preventing calculations on data that's our of the viewport.

I'll continue a discussion on this with Phil, in the other issue on the MPAndroidChart repo. PhilJay/MPAndroidChart#718.

You can follow up on it there, and when we have a conclusion I'll give an update here.

[gjpc]

Yes, I have noticed that and I would think in many non-biological areas it would be fine, it's a good optimization. Perhaps setting a flag in a DataSet object's first scan of the data to let the renderers know it aint so. It may be advisable to to go ahead and set the flag when realtime data on the x axis manipulation occurs.

[danielgindi]

We've come to a conclusion that the values must be ordered - as it is considerably less expensive to sort the data by x-index before passing to the chart, than to loop over the whole data and do the bounds-checking for each and every point.

So this is going to be closed for now. If there's a request for a specific special feature please open a relevant issue :-)

[gjpc]

If anyone wants a scatter chart that actually works they can go to my branch....

[danielgindi]

Or work correctly with the API. The scatter chart works.

[gjpc]

It does not :P

[micnguyen]

@gjpc Hi, I've tried implementing your changes to remove the condition checking on my project that suffers this issue as well, but it does not seem to fix it either. Have any advice?

[gjpc]

@micnguyen I need a bit more information before I may offer any advice.

[gjpc]

@micnguyen I am not sure you implemented my changes or @danielgindi branch of my changes that may or may not contain my changes properly implemented. I just pushed a major enhancement to the Independent Scatter Chart demo to illustrate exactly what an Independent scatter plot does. Please go to my branch and see it that solves your problems.

[micnguyen]

@gjpc I basically created a time series charge with lots of x-axis intervals (spaced by 1 minute). So there will data points at say, interval 1 and another data point at interval 61 if they are an hour apart. The issue I have been seeing is if you zoom until a point is off the screen, the line that connects them disappears. I assumed your change of removing the conditionals on line 91 and 217 fixed that - or have you tackled another issue that I may be mistaking?

[gjpc]

@micnguyen I chose not to draw a line unless both points are visible on the screen. I find it easier to zoom out to find the next dot's position, panning and then re-zooming a more natural mode of analysis. In a particularly dense data set I found too many lines running off the screen in so many different directions obscured the data under study. I'll consider adding a flag to allow a line to the off-screen points.

[micnguyen]

@gjpc Our testers and myself personally have found the disappearing lines odd behaviour. In our situation, we only have 1 data set that all have unique x-y values. The lines that connect said data points do provide some information to the user (especially if your data points do not have large explicit markers). I hope the changes to achieve that won't be too much work - thank you.

[gjpc]

@micnguyen it is a trivial fix if you don't mind the line transversing to the off screen data point crossing the axis lines. It is more work to stop it at the axis, but there is a certain amount of positive feedback in indicating the data exists, but it is out of the view port.

What do you think?

[micnguyen]

@gjpc It looks great! It appearing out of the axis lines is rather unfortunate and undesirable however. But if the lines and data points don't disappear when out of the viewport, that's great.

[gjpc]

@micnguyen You are right, pickup the finish at https://github.com/gjpc/ios-charts

[gjpc]

@micnguyen I don't know if this will effect you, but I just did a rebase and force push to get rid of a previous commit error