ImageGraph: export reports as graph

[anonymous-matomo-user]

See http://piwik.org/docs/analytics-api/metadata/#toc-static-image-graphs

This plugin renders graphs from piwik reports.

Every report with existing columns 'label' and 'nb_visits' is applicable, at this time.
The label values are taken for the abscissa and 'nb_visists' values are drawn as value.

Such reports could be drawn as :
a. basic line
b. basic bar
c. 3d pie

The pChart Framework (http://pchart.sourceforge.net/) is used to render graphs as png.

[anonymous-matomo-user]

Attachment: plugin source including pChart framework
ImageGraphPlugin.zip

[anonymous-matomo-user]

Attachment: Example of continent report drawn as bar graph
Continent_BasicBar.png

[anonymous-matomo-user]

Attachment: Example of continent report drawn as line graph
Continent_BasicLine.png

[anonymous-matomo-user]

Attachment: Example of coutry report drawn as 3d pie graph
Country_3dPie.png

[anonymous-matomo-user]

Attachment: Example of OS report drawn as 3d pie graph
OS_3dPie.png

[anonymous-matomo-user]

Attachment: Example of visitsperlocaltime report drawn as bar graph
VisitsPerLocalTime_BasicBar.png

[robocoder]

arni: very nice.

This is a great start to generating graphs on the server side. Other use cases include:

• inclusion in pdf reports
• non-Flash charting (Non-Flash Charting & Tables #804)

[anonymous-matomo-user]

Attachment:
API.php

[anonymous-matomo-user]

On some php installations the pChart Framework throws an Error, while ErrorReporting is set to "GD".

For first fix, I set the ErrorReporting of pChart to off.

See the new API.php at Line 117

[anonymous-matomo-user]

update: You can now set the column of report which is used as ordinate

[anonymous-matomo-user]

Attachment: New Source of ImageGraph including pCart-Framwork
ImageGraph.zip

[anonymous-matomo-user]

I tried to get ImageGraph to work but failed. Can you provide a few more examples of which reports work? Not just the PNG but the URL you use to generate the image?

I tried clicking on the example URL Piwik generates, got an error "uninitialized string offset 0/1 in pChart/pData.class", see
http://en.qi-hardware.com/piwik/index.php?module=API&method=ImageGraph.get&idSite=1&period=day&date=yesterday&apiModule=UserCountry&apiAction=getCountry&format=Html

Then I tried to get a simple line chart for unique visitors in last 30 days, got another error "undefined index dimension/processedMetrics in API.php", see
http://en.qi-hardware.com/piwik/index.php?module=API&method=ImageGraph.get&idSite=1&period=day&date=last30&apiModule=VisitsSummary&apiAction=get&format=Html

To get the public URLs to work, I commented out Piwik::checkUserIsNotAnonymous() and Piwik::checkUserHasViewAccess($idSite) in API.php - is that a security risk? (my data is all public anyway, I just don't want the server to be hijacked :-))

My goal is the line chart for unique visitors in last 30 days. But I'm a little lost in the various tables, metadata, etc. Any help or more documentation for ImageGraph (example URLs) is very appreciated.
Thanks,
Wolfgang

[anonymous-matomo-user]

Replying to wolfgang:

I tried to get ImageGraph to work but failed. Can you provide a few more examples of which reports work? Not just the PNG but the URL you use to generate the image?

The ImageGraph should work with all reports, providing the label column and the dimension column when calling the API.getMetadata.

I tried clicking on the example URL Piwik generates, got an error "uninitialized string offset 0/1 in pChart/pData.class", see
http://en.qi-hardware.com/piwik/index.php?module=API&method=ImageGraph.get&idSite=1&period=day&date=yesterday&apiModule=UserCountry&apiAction=getCountry&format=Html

Im not sure, why this is not working on your copy.
This link works fine in my copy.
Could you please post your php and piwik version?

Then I tried to get a simple line chart for unique visitors in last 30 days, got another error "undefined index dimension/processedMetrics in API.php", see
http://en.qi-hardware.com/piwik/index.php?module=API&method=ImageGraph.get&idSite=1&period=day&date=last30&apiModule=VisitsSummary&apiAction=get&format=Html

This will not work, because the VisitsSummary.get doesnt provide the label column.

To get the public URLs to work, I commented out Piwik::checkUserIsNotAnonymous() and Piwik::checkUserHasViewAccess($idSite) in API.php - is that a security risk? (my data is all public anyway, I just don't want the server to be hijacked :-))

Im not very familiar with piwik, too.
So I dont know if this is a security risk.

My goal is the line chart for unique visitors in last 30 days. But I'm a little lost in the various tables, metadata, etc. Any help or more documentation for ImageGraph (example URLs) is very appreciated.

To get a report for the unique visitor of the last days, that could be drawn by the ImageGraph-Module you will need another plugin, which I will publish in the next few days.

Some links that should work:

index.php?module=API&method=ImageGraph.get&idSite=1&period=day&date=yesterday&apiModule=UserCountry&apiAction=getCountry&format=xml&token_auth=

index.php?module=API&method=ImageGraph.get&idSite=1&period=day&date=yesterday&apiModule=UserCountry&apiAction=getContinent&format=xml&token_auth=

index.php?module=API&method=ImageGraph.get&idSite=1&period=week&date=yesterday&apiModule=UserSettings&apiAction=getOS&format=xml&token_auth=

Thanks,
Wolfgang

[anonymous-matomo-user]

(same wolfgang, forgot password and password reminder emails are not being sent out...)

Thanks for your help!
But... :-)

Some links that should work:
index.php?module=API&method=ImageGraph.get&idSite=1&period=day&date=yesterdayapiModule=UserCountry&apiAction=getCountry&format=xml&token_auth=

That is pretty much exactly the link I posted as not working, and if you would have clicked on it you would see the entire (long) error which I didn't want to paste here because you just need to click on it to see it.

Im not sure, why this is not working on your copy. This link works fine in
my copy. Could you please post your php and piwik version?

Piwik 1.0, php 5.3.2
Can you click on my link? Maybe the error message tells you more than it tells me. It seems to be related to the actual ImageGraph plugin, and/or Piwik table data.
Here's the link again
http://en.qi-hardware.com/piwik/index.php?module=API&method=ImageGraph.get&idSite=1&period=day&date=yesterday&apiModule=UserCountry&apiAction=getCountry&format=Html

To get a report for the unique visitor of the last days, that could
be drawn by the ImageGraph-Module you will need another plugin,
which I will publish in the next few days.

Oh great, I will be very much looking forward to it. When I find some time, I will dig deeper into my actual bug, which as I say is still there...

Some links that should work:

No they don't work. All 3 of them don't work. That is the bug I was trying to tell you about. You can click on my link then you see it. I will try to dig deeper if I can.

Thanks a lot for your help, good luck with this plugin and the new one, I think it's really cool!
Wolfgang

[anonymous-matomo-user]

Hi Wolfgang.

Ive clicked on your posted links, but I cant reproduce them on my system.
Ive fixed some Code in the pChart-Framework.
Could you please try this new version?

Thanks arni

Replying to wolfgang2:

(same wolfgang, forgot password and password reminder emails are not being sent out...)

Thanks for your help!
But... :-)

Some links that should work:
index.php?module=API&method=ImageGraph.get&idSite=1&period=day&date=yesterdayapiModule=UserCountry&apiAction=getCountry&format=xml&token_auth=

That is pretty much exactly the link I posted as not working, and if you would have clicked on it you would see the entire (long) error which I didn't want to paste here because you just need to click on it to see it.

Im not sure, why this is not working on your copy. This link works fine in
my copy. Could you please post your php and piwik version?

Piwik 1.0, php 5.3.2
Can you click on my link? Maybe the error message tells you more than it tells me. It seems to be related to the actual ImageGraph plugin, and/or Piwik table data.
Here's the link again
http://en.qi-hardware.com/piwik/index.php?module=API&method=ImageGraph.get&idSite=1&period=day&date=yesterday&apiModule=UserCountry&apiAction=getCountry&format=Html

To get a report for the unique visitor of the last days, that could
be drawn by the ImageGraph-Module you will need another plugin,
which I will publish in the next few days.

Oh great, I will be very much looking forward to it. When I find some time, I will dig deeper into my actual bug, which as I say is still there...

Some links that should work:

No they don't work. All 3 of them don't work. That is the bug I was trying to tell you about. You can click on my link then you see it. I will try to dig deeper if I can.

Thanks a lot for your help, good luck with this plugin and the new one, I think it's really cool!
Wolfgang

[anonymous-matomo-user]

Attachment: Fixed Code in included pChart - Framework
ImageGraph.2.zip

[anonymous-matomo-user]

wow, yes! I looked at what you changed and would not have thought that it makes a difference, but the country chart works now! (you can click on the old link)

Strange though, in pChart.class, you change a $this->Data to $this->data (lower-case), even though everywhere else it's always upper-cased. It looks like you introduce a bug but you fix it... Maybe there are both a lower-case and upper-case data/Data somewhere?

Anyway, you know what you are doing. Country graph is working for me now, and I am eagerly awaiting the one you mentioned that could display unique visitors.
Thanks a lot for your fix! Wolfgang

[anonymous-matomo-user]

Hi Wolfgang.

You can check now the ticket #1773 to install the VisitsSummaryEvolution plugin.
After that you can test the link

index.php?module=API&method=ImageGraph.get&idSite=1&period=month&date=yesterday&apiModule=VisitsSummaryEvolution&apiAction=getVisitsSummaryDailyEvolution

You should receive a line chart displaying the visitors of each day for the current month.

arni

Replying to wolfgang2:

wow, yes! I looked at what you changed and would not have thought that it makes a difference, but the country chart works now! (you can click on the old link)

Strange though, in pChart.class, you change a $this->Data to $this->data (lower-case), even though everywhere else it's always upper-cased. It looks like you introduce a bug but you fix it... Maybe there are both a lower-case and upper-case data/Data somewhere?

Anyway, you know what you are doing. Country graph is working for me now, and I am eagerly awaiting the one you mentioned that could display unique visitors.
Thanks a lot for your fix! Wolfgang

[mattab]

arni, are you interested to get this code contributed to core? I would like to do a full code review, if you confirm that you are ready for it, I'll get started :)

It looks like a great start!

[anonymous-matomo-user]

Yeah. Of course.

Replying to matt:

arni, are you interested to get this code contributed to core? I would like to do a full code review, if you confirm that you are ready for it, I'll get started :)

It looks like a great start!

[robocoder]

The pChart library contains Microsoft fonts (i.e., tahoma.ttf in pChart 1.27d and verdana.ttf in pChart 2) that don't allow for this form of redistribution. The font should be removed from the package. (The user can be instructed to install the missing font, if required.)

[sgiehl]

Is anyone still working on this plugin?
It would be nice to use something like that for Piwik Mobile in the future.

[anonymous-matomo-user]

I will move all fonts out, except of GeoSansLight. This should be ok. or am I wrong?!

Replying to vipsoft:

The pChart library contains Microsoft fonts (i.e., tahoma.ttf in pChart 1.27d and verdana.ttf in pChart 2) that don't allow for this form of redistribution. The font should be removed from the package. (The user can be instructed to install the missing font, if required.)

[anonymous-matomo-user]

Im on. Just a bit busy at this time.
I think, I will upload a new version in the next two weeks.

Replying to SteveG:

Is anyone still working on this plugin?
It would be nice to use something like that for Piwik Mobile in the future.

[robocoder]

If labels are translated, what's the coverage wrt Piwik translations? Is there a fallback to English?

[anonymous-matomo-user]

Replying to vipsoft:

If labels are translated, what's the coverage wrt Piwik translations? Is there a fallback to English?

I do not really understand your answer.

[mattab]

vipsoft, the ImageGraph plugin is using the Metadata API so it doesn't deal with any translation it is all automatic

arni, I have sent you an email for discussing future of the plugin and how we can integrate it

[robocoder]

Sorry, I was posting from my iPhone. I'll try to be clearer.

Since ImageGraph doesn't deal with translations directly, then there's no fallback to English if characters used by a (translated) label can't be rendered using the available font(s).

For example, GeoSansLight appears to be limited to Latin-1 characters (accented characters) and some special symbols (e.g., Euro). It would not be able to render the translated labels for Amharic, Arabic, Belarusian, Bulgarian, Hellenic, Hebrew, Japanese, Georgian, Korean, Russian, Telugu, Thai, Ukranian, or Chinese.

[robocoder]

WRT maintenance of third-party libraries:

• pChart 1.27d is the last of the 1.x series and no longer supported. It is GPLv1 or later.
• pChart 2 is the current stable dev branch. It is available under either a GPLv3 or commercial license.
• Unlike tcpdf, both versions of pChart requires the server have the freetype library (e.g., /usr/lib/libfreetype.so) since it calls the gd extension's imagettf* functions.

[mattab]

I have suggested to Arni the following features:

• the graphs should look as close as possible to the existing flash graph design: same default color, same font size, same vertical grids, etc.
• possibility to customize by GET Parameter the size (width/height) of the graph
• possibility to customize the colors, font-size, etc. by GET Parameter as well.
• default color, font-size etc. would be set as a class const DEFAULT_COLOR = '553344'; etc.
• We don't need the 3D pie in fact because it is difficult to see the data
• the supported graph types should be: evolution line chart, 2D pie chart, vertical bar graph.
• to support Evolution line chart (to plot for example Number of visits over the last 100 days), the graph will be drawn when there is no dimension. This corresponds to the "Simple metrics reports" in http://piwik.org/docs/analytics-api/metadata/#toc-listing-all-the-metadata-api-functions
• produce a page that will plot a graph of each type, for ALL reports, by simply looping over available metadata reports using the metadata function listing all functions
  See for example a script a little bit similar to list all widgets in piwik/misc/iframeWidget_localhost.php
  The page will be used as a visual test that all graphs look good, and we can easily open this page after changing some code, to check for regression :)
• make sure that the lib used, pCharts, is not modified so we can later upgrade the lib without problem
• reuse the fonts from the TCPDF directory, because they are already there. Also, there is a bit of code already in TCPDF to choose the best font based on laanguage (so it works in japanese, chinese russian etc.). Don't worry about refactoring TCPDF I can help with this, so we can have a Piwik helper function that will return the font path based on the parameter $language (that will be used in both TCPDF and ImageGraph)

one update

• the Evolution chart (line chart X/Y over time) is available only for reports without dimensions (ie. to show a metric evolving over time).
• the Vertical bar graph (and other graphs) are only available for reports with a dimension, to show a given set of dimensions at a given time

Arni confirmed he is interested in finishing the work so this is great news!! then we shall modify PDF reports and of course, Piwik Mobile :)

[julienmoumne]

(In [5582]) * fixes #2706, #2828, #2704, refs #1721, #2637, #2711, #2318, #71 : horizontal static graph implemented

• fixes Static Graph: better graphs when date range period parameter #2788, refs Metadata improvements to generate static graph image URL #2670, ImageGraph: export reports as graph #1721, #2637, #2711 : default graph type logic moved to ImageGraph API, improved date/period logic, new parameter graphs_default_period_to_plot_when_period_range
• fixes Graphs should support Unicode characters & other improvements #2704, graph image cutoff first digits when Y axis has decimals #2804, refs ImageGraph: export reports as graph #1721 : pChart updated to 2.1.3, pChart code removed from Piwik code, OOP refactoring, support for unifont.ttf if present in ImageGraph/fonts, testAllSizes now uses report metadata ImageGraphUrl
• refs Plugin to provide PDF export of data #5491 : space between report title and report table reduced to avoid page overflow
• refs ImageGraph: graphs should display % values #2829 : TODO display percentages
• r5544, r5547, r5549 merged

[julienmoumne]

(In [5596]) refs #1721

• dimension-less graphs with no data now display 'No data for this graph'
• regression in graph filename
• narrowing down Piwik_ImageGraph_StaticGraph_Exception requirements to handle early exceptions
• increasing min space between horizontal values in horizontal graphs

[julienmoumne]

(In [5597]) refs #1721

• adding PDF/HTML reports image size to testAllSizes

[julienmoumne]

(In [5601]) refs #1721

• exception now thrown for dimension-less report without multiple periods
• enhanced logo positioning for horizontal graphs
• enhanced legend positioning for all graphs

[julienmoumne]

(In [5634]) refs #1721

• display 'No data for this graph.' when the report does not contain at least one value different than 0
• Piwik_ImageGraph_StaticGraph_Exception now renders an image with the size specified by the width and height parameters
• legend positioning now works for all font sizes
• enhanced label truncation for horizontal graphs

[julienmoumne]

(In [6112]) refs #1721 - wrong package in php doc

[julienmoumne]

(In [6947]) refs #1721 refs #3013

• show label icons in evolution graph
• smaller legend font size
• draw colored shadows in legend
• vertical legend for evolution graphs
• enhanced label skipping for daily evolution graph

[julienmoumne]

(In [6951]) refs #1721 refs #3013

• better legend font sizes
• legend shadow only when plotting multiple series
• more shadow padding

[julienmoumne]

(In [7081]) refs #1721

• new API parameter : $legendFontSize
• increased legend font size

[julienmoumne]

(In [7139]) refs #1721 increase max width & height for tablet screens