EMF file support #152

antoinerg · 2018-11-27T19:21:17Z

This PR is a work in progress adding EMF support to Orca. After testing Inkscape, unoconv and svg2vector I went with Inkscape because it gives the best results.

The biggest challenge with converting SVG to EMF have to do with opacity. Although EMF supports full transparency, it does not support opacity between 0 and 1. This is a problem because:

We have a lot of rectangles in our figures with fill: rgba(0,0,0,0). Inkscape throws away the alpha channel and is then left with a black rectangle instead of a transparent one! Example:
We overlay a semi-transparent PNG of WebGL figures on top of SVG axes.

Finally, Inkscape doesn't handle well the way we define our SVG linear gradients:

Those issues can be addressed by cleaning the SVG prior to feeding it to Inkscape as I demonstrate here.

In commit 035ed22, I fix most of the black rectangles by removing transparent rectangles, properly convert the gradient in EMF by changing the gradientUnits of our SVG linearGradient element and finally manually do alpha compositing for each pixel of the PNG images of WebGL traces.

You can find a comparison of Orca's PNG and Orca's EMF converted to PNG at https://gist.github.com/antoinerg/680ce690b039fbcb49b976de1d28e1c6

To showcase the progress so far, the above figures now looks like this in EMF:

etpinard · 2018-11-29T16:37:54Z

.circleci/config.yml

@@ -61,7 +61,7 @@ jobs:
          name: Install deps
          command: |
            npm install
-            sudo apt-get install poppler-utils libgconf-2-4
+            sudo apt-get update && sudo apt-get install inkscape poppler-utils libgconf-2-4


minor 🐄

sudo apt-get update sudo apt-get install inkscape poppler-utils libgconf-2-4

etpinard · 2018-11-29T16:38:29Z

package-lock.json

@@ -143,7 +143,7 @@
        "string-width": {
          "version": "2.1.1",
          "resolved": "https://registry.npmjs.org/string-width/-/string-width-2.1.1.tgz",
-          "integrity": "sha512-nOqH59deCq9SRHlxq1Aw85Jnt4w6KvLKqWVik6oA9ZklXLNIOlqg4F2yrT1MVaTjAqvVwdfeZ7w7aCvJD7ugkw==",
+          "integrity": "sha1-q5Pyeo3BPSjKyBXEYhQ6bZASrp4=",


Hmm. Which npm version are you using?

I'm using 6.2.0. I should probably upgrade. What are you using?

yeah, upgrading would be best.

etpinard · 2018-11-29T16:46:39Z

src/util/inkscape.js

+    })
+  }
+
+  cleanSvg (svg) {


Oh wow. That's some fancy regex work!

How you tried using something like xml2.js to loop over all those SVG nodes? It should make things easier to maintain.

I was pondering whether to use regex vs parsing the XML. The latter is certainly less error-prone.

Now that I know what cleaning needs to be done, I will look into xml2.js!

etpinard · 2018-11-29T17:04:59Z

@antoinerg Thanks for making https://gist.github.com/antoinerg/680ce690b039fbcb49b976de1d28e1c6 - the results look pretty good!

I've noticed that fill from carpets traces and some contour traces (the ones coloring constraint regions?) appear black. Is that something that can be fixed?

Ideally, I would prefer not using regexes to do all the heavy compatibility work before the SVG -> EMF conversion. Using something like xml2js to parse the SVG, then looping over all the nodes checking for fill-opacity, stroke-opacity and opacity attributes and then applying the conversion sounds worth the try.

We could even move this compatibility logic to plotly.js' Snapshot.toSVG - which already does a few similar things. I'm not sure if we should be adding a Plotly.toImage flag (e.g. noOpacity) or add a "format" (e.g. svg-for-emf)?

Thinking about this again, I think adding a noOpacity flag in plotly.js makes sense. There are probably other image formats out there that don't fully support an alpha channel. Now this doesn't solve all the EMF/Inkscape issues; perhaps these remaining issues could be handling in Orca? What do you think @antoinerg ?

antoinerg · 2018-11-29T18:36:23Z

@etpinard thank you for the review!

I've noticed that fill from carpets traces and some contour traces (the ones coloring constraint regions?) appear black. Is that something that can be fixed?

I decided to wait for your feedback before looking into those. There are also somegl2d traces that aren't perfect. I will let you know what we can do with those!

Ideally, I would prefer not using regexes to do all the heavy compatibility work before the SVG -> EMF conversion. Using something like xml2js to parse the SVG, then looping over all the nodes checking for fill-opacity, stroke-opacity and opacity attributes and then applying the conversion sounds worth the try.

I agree that regexes feel like a hack and are error-prone. I'll look into xml2js.

Thinking about this again, I think adding a noOpacity flag in plotly.js makes sense. There are probably other image formats out there that don't fully support an alpha channel.

I was wondering the same thing: how much cleaning should be done in plotly.js vs Orca. A noOpacity flag in plotly.js could be nice and we have a DOM over there: no need for xml2js. I need to think about it 🤔

Now this doesn't solve all the EMF/Inkscape issues; perhaps these remaining issues could be handling in Orca?

Yes, some of the fixes for Inkscape's SVG-to-EMF glitches should definitely stay in Orca (ex. changing how we define SVG linearGradients) . One good reason for this is that those fixes might depend on Inkscape's version (something we should always actually check in Orca prior to cleaning up!).

antoinerg · 2018-12-03T19:10:08Z

I've noticed that fill from carpets traces and some contour traces (the ones coloring constraint regions?) appear black. Is that something that can be fixed?

Carpets

After removing the semi-transparent fills, the carpets now look like:

Contours

About conversion problems with contours, are you referring to the result below?

If so, I need to investigate the clipping. I suspect it has to do with the way we define our units (similarly to how we fixed the linearGradient.

gl2d

Unfortunatley, those one are much harder to deal with because they were designed with transparency in mind: they have SVG axes on top of which there are a transparent PNGs of the WebGL contexts. As you know by now, EMF doesn't support transparency very well. Not sure how we could handle this without significant work/modification to the SVG. See this comment for details on EMF and transparency in Inkscape's bug tracker: https://bugs.launchpad.net/inkscape/+bug/1419686/comments/3

antoinerg · 2018-12-03T20:05:13Z

Results from commit 1ecb2f4: https://gist.github.com/antoinerg/e3b7d7b7b97bbff9e0d84c769f0a4e91

antoinerg · 2018-12-04T20:18:13Z

Results for commit 5432b59: https://gist.github.com/antoinerg/1637f08d26664965b6d942e53d56e7a3

Improvement to gl3d: paper background color is taken into account when doing alpha compositing:

etpinard · 2018-12-06T14:59:48Z

@antoinerg Thanks for fixing our tests!

Now, it looks like the AppVeyor runs have been failing since 7dddce5 where jsdom got bumped down to accomodate Node@6. 😡

antoinerg · 2018-12-06T16:14:05Z

Commit 212ed21 (which has a bumped down jsdom) actually passed tests on AppVeyor so hopefully the problem is not with the new dependencies introduced by this PR. Let's trigger a new build and see if it passes.

etpinard · 2018-12-06T17:31:12Z

Ok. Something is up with AppVeyor. I pushed an empty commit off master, and AppVeyor fails there too:

https://ci.appveyor.com/project/AppVeyorDashAdmin/image-exporter/builds/20823193

So, I would be ok merging this thing as is. Let me review it one last time.

antoinerg · 2018-12-06T17:34:07Z

@etpinard There's also one last important to do: update the deployment's Dockerfile. Should I submit a separate PR for this one?

etpinard · 2018-12-06T17:34:55Z

There's also one last important to do: update the deployment's Dockerfile.

Ah good point. Let's do this in this PR!

etpinard · 2018-12-06T17:37:48Z

src/util/inkscape.js

+
+    // Get background color from figure's definition
+    var bgColor
+    if (opts.figure.layout.paper_bgcolor) {


figure.layout might not always be defined (unless orca is doing something hacky I don't remember), so I would prefer:

if ((opts.figure.layout || {}).paper_bgcolor) { /* */ }

to stay safe.

Change made in commit e4ab53d

etpinard · 2018-12-06T17:39:03Z

src/util/inkscape.js

+  }
+
+  cleanSvg (svg, bgColor) {
+    const fragment = JSDOM.fragment(svg)


Nice find! This is probably faster than DOMParser

etpinard · 2018-12-06T17:40:27Z

src/util/inkscape.js

+      var m = style.match(fillOpacityZero)
+
+      if (m) {
+        var rgbFill = m[1] + `fill: rgb(${bgColor[0]},${bgColor[1]},${bgColor[2]});`


🐄 Let's make this one single ES6 string: ${m[1]} fill: rgb ... ;

Fixed in e76c52b

etpinard · 2018-12-06T17:43:36Z

Nice job. The new jsdom-based logic looks fantastic.

https://gist.github.com/antoinerg/1637f08d26664965b6d942e53d56e7a3 is giving me 500 though, is that just Github acting up?

antoinerg · 2018-12-06T17:57:33Z

https://gist.github.com/antoinerg/1637f08d26664965b6d942e53d56e7a3 is giving me 500 though, is that just Github acting up?

It's probably Github acting up because the gist is large. Maybe git clone it:

git clone [email protected]:1637f08d26664965b6d942e53d56e7a3.git

By the way, I really like sxiv to check the images. sxiv *.png and the nkeys n and p to see the next and previous image. Press enter to see preview of many images.

antoinerg · 2018-12-06T18:23:31Z

Results in PNG for latest commit e76c52b:

git clone [email protected]:632c26b518988f63bb766453bf850c9c.git

Basically, gl2d and sankey have conversion issues. Finally, some contour images don't get clipped properly. The other ones look pretty good however !

etpinard · 2018-12-06T18:58:50Z

deployment/Dockerfile

@@ -1,4 +1,4 @@
-FROM ubuntu:xenial
+FROM ubuntu:bionic


Hmm. We'll have to test that thoroughly on stage. Is this required?

It's definitely the path of least resistance to get the latest version of Inkscape (0.92.3 which is also the version I used in my testing). Otherwise we'd have to compile Inkscape in the container.

Using a ppa shouldn't be too bad:

http://ubuntuhandbook.org/index.php/2017/01/install-inkscape-0-92-ppa-ubuntu-16-04-16-10-14-04/

I'm not against bumping Ubuntu, but we can't just casually bump it here w/o some thorough manual testing.

I understand your concern. Let me try to compile Inkscape or install from a deb package.

Using a ppa shouldn't be too bad:

That did the trick thank you 👍

Commit b6daae1 reverted our container to ubuntu:xenial and install Inkscape from a PPA

amazing 🎉

Thanks very much!

etpinard · 2018-12-06T19:05:10Z

deployment/Dockerfile

@@ -90,7 +90,7 @@ RUN apt-get update -y && apt-get install -y subversion && \
 RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
    sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list' && \
    apt-get update -y && \
-    apt-get install -y google-chrome-stable xvfb poppler-utils git && \
+    apt-get install -y google-chrome-stable xvfb poppler-utils inkscape git && \


Have you tried running that locally?

If I remember correctly:

docker build . -f deployment/Dockerfile -t tmp-inkscape # check if ok with docker images docker run tmp-inkscape -d -p 9010:9010 tmp-inkscape # then curl localhost:9010 ...

Default port is 9091

The following works for me!

docker build . -f deployment/Dockerfile -t tmp-inkscape # check if ok with docker images docker run tmp-inkscape -d -p 9091:9091 tmp-inkscape # then cd plotly/plotly.js/test/image/mocks/ curl -XPOST -d @14.json localhost:9091 > /tmp/14.png

etpinard · 2018-12-06T22:57:12Z

Results in PNG for latest commit e76c52b:
git clone [email protected]:632c26b518988f63bb766453bf850c9c.git

Thanks!

The results are very impressive!

Apart from all gl2d and splom mocks, I noticed:

we could probably improve the look of rangesliders e.g. we could try removing the shaded layers completely instead of having them show up in black
category_autorange has shapes with fill-color: rgb(0,0,0) and fill-opacity: 0, sounds like we could simply remove fill-color here to get better results
contour_log and contour_autorange are off like you mentioned in EMF file support #152 (comment)
legend_horizontal_autowrap has a border around the legend items
all sankey mocks look odd

All in all, out of 600+ mocks, that's not bad.

I'm ready to 💃

@antoinerg you can either:

try to fixup a few of the items above before merging, or
merge this and open a new orca issue pasting that list of open items.

About the AppVeyor failure, we should try running the conda pack steps on a Windows laptop.

antoinerg · 2018-12-06T23:17:44Z

@antoinerg you can either:

try to fixup a few of the items above before merging, or

merge this and open a new orca issue pasting that list of open items.

@etpinard, I think I will go with the second course of action so others can benefit as soon as possible from the tests that we fixed in this branch. For example this PR @tarzzz just submitted #165 fails :(

About the AppVeyor failure, we should try running the conda pack steps on a Windows laptop.

Is this blocking? If so, I will test it first thing tomorrow!

Thank you for the thorough review and for finding all the discrepancies 🕵️‍♂️! I will fix them in a separate issue as you suggested 🎉

etpinard · 2018-12-06T23:20:07Z

Is this blocking? I

Not blocking for this PR, but we should try to fix it ASAP to not delay any upcoming orca releases.

Opening an issue for now would be 👌

antoinerg added 3 commits November 26, 2018 19:23

initial support for EMF file format thanks to Inkscape

5666916

remove PATCH version from docker image name to get updates

e823274

fix regular expression to match svg attributes with dash

035ed22

antoinerg added status: in progress status: discussion needed labels Nov 27, 2018

antoinerg added 3 commits November 27, 2018 15:28

circleci: debian-jessie -> debien-strecth to get inkscape 0.9x

40437c4

properly check test-mock.emf size and fix lint

25f0bb6

missing deps, need to update package lists

e2f2be7

etpinard reviewed Nov 29, 2018

View reviewed changes

minor code style fix

5ed1198

check Inkscape's version, provide meaning error message when --verbose

fc8c755

parse SVG using jsdom instead of regexes

1ecb2f4

antoinerg added 4 commits December 3, 2018 16:25

jsdom 13.0.0 -> 11.12.0 to support node.js v6

7dddce5

find image elements using jsdom

6b1a1fc

gl3d: use bg color from figure.layout if provided

d4a1bfa

fix lint

5432b59

fix Inkscape unit test

212ed21

bpostlethwaite self-requested a review December 5, 2018 17:03

antoinerg added 4 commits December 5, 2018 17:50

relax constraint for Inkscape's version: now requires gte 0.92.x

37747b1

use different port for orca_serve_graph-only_test.js

315a7d6

npm install with latest npm version

48e1e11

teardown server last, orca should reply 404 instead of ECONNREFUSED

c1822e2

cache node_modules only if package-lock.json didn't change

ffc9bfd

etpinard reviewed Dec 6, 2018

View reviewed changes

install Inkscape in deployment container

e24f167

improve code readability and style

e76c52b

antoinerg removed status: discussion needed status: in progress labels Dec 6, 2018

etpinard reviewed Dec 6, 2018

View reviewed changes

antoinerg added 3 commits December 6, 2018 16:23

safer code which ensures there is a figure.layout object

e4ab53d

revert to ubuntu:xenial, install latest Inkscape from a PPA

b6daae1

run apt-get update prior to installing Inkscape to get latest version

f117d36

antoinerg merged commit 36e2668 into master Dec 7, 2018

antoinerg deleted the inkscape branch December 7, 2018 16:35

antoinerg mentioned this pull request Dec 7, 2018

Improve EMF support #166

Open

5 tasks

etpinard added this to the v1.2.0 milestone Dec 29, 2018

antoinerg mentioned this pull request Mar 20, 2019

EMF - problems with colorscale #218

Closed

antoinerg mentioned this pull request Dec 18, 2019

Plotly.toImage: add format to remove all semi-transparent elements plotly/plotly.js#4433

Closed

EMF file support #152

EMF file support #152

Conversation

antoinerg commented Nov 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

etpinard commented Nov 29, 2018

antoinerg commented Nov 29, 2018

antoinerg commented Dec 3, 2018

Carpets

Contours

gl2d

antoinerg commented Dec 3, 2018

antoinerg commented Dec 4, 2018 • edited Loading

etpinard commented Dec 6, 2018

antoinerg commented Dec 6, 2018

etpinard commented Dec 6, 2018

antoinerg commented Dec 6, 2018

etpinard commented Dec 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

etpinard Dec 6, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

etpinard commented Dec 6, 2018

antoinerg commented Dec 6, 2018

antoinerg commented Dec 6, 2018 • edited Loading

Choose a reason for hiding this comment

antoinerg Dec 6, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

etpinard Dec 6, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antoinerg Dec 6, 2018 • edited Loading

Choose a reason for hiding this comment

etpinard commented Dec 6, 2018

antoinerg commented Dec 6, 2018

etpinard commented Dec 6, 2018 • edited Loading

antoinerg commented Nov 27, 2018 •

edited

Loading

antoinerg commented Dec 4, 2018 •

edited

Loading

etpinard Dec 6, 2018 •

edited

Loading

antoinerg commented Dec 6, 2018 •

edited

Loading

antoinerg Dec 6, 2018 •

edited

Loading

etpinard Dec 6, 2018 •

edited

Loading

antoinerg Dec 6, 2018 •

edited

Loading

etpinard commented Dec 6, 2018 •

edited

Loading