PDFReports: TCPDF temporary subject to open_basedir restriction

[EPinci]

piwik\libs\tcpdf\config\tcpdf_config.php defines K_PATH_CACHE defaulting to the non existing folder piwik\libs\tcpdf\cache.

On most system this will cause tempnam functions to failback to system wide temp folder (as opposed to the expected piwik\tmp\cache).
On Windows system this will default to C:\Windows\temp that is often (very) out of the open_basedir causing the PDF generation to fail with stack dump.

Tcpdf cache folder should default to piwik installation cache folder.

[robocoder]

Another side-effect of falling back to the system wide temp folder is running into open basedir restrictions.

[anonymous-matomo-user]

PDFReport Cache Folders are not in Piwik 1.0 installation package

libs/tcpdf/cache
libs/tcpdf/images

please put those folders in package.

greetings

[mattab]

machoyer, why is it important to put these folders in Piwik?

epinci, does TCPDF currently use cache? if so, did you test a K_PATH_CACHE value that would work and use piwik/tmp/ folders? Thanks

[anonymous-matomo-user]

See #5491?replyto=36#comment - this was the only way for mee to avoid this error -> #5491?replyto=33#comment

Maybe the K_PATH_CACHE in piwik\libs\tcpdf\config\tcpdf_config.php isn't set correctly. If yes, creating those folders in installation package would be unnecessary.

[mattab]

I'm not sure how to fix this issue, since the K_PATH_CACHE should be piwik/tmp/ probably, but I wouldn't want to modify the tcpdf_config.php - it looks like TCPDF doesn't allow to modify the value appart from editing this file directly (if we define it upstream, there will be a PHP error "CONSTANT already defined"...).

[mattab]

If someone experiences this issue please comment and we can try to fix the path, since I'm unable to reproduce.

[anonymous-matomo-user]

I haven't had the PDF Reports activated until I had upgraded to piwik v1.1.1

The problem appears everytime I try to download the PDF report. Trying to send it via mail hangs at "Loading Data".

Backtrace:
There is an error. Please report the message and full backtrace in the Piwik forums.

Warning: imagepng() href='function.imagepng'>function.imagepng</a>: open_basedir restriction in effect. File(/tmp/jpg_SNJixl) is not within the allowed path(s): (/var/www/web352/html/:/var/www/web352/phptmp/:/var/www/web352/files/:/var/www/web352/atd/:/usr/share/php/) in /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php on line 7545

Backtrace -->
#0 Piwik_ErrorHandler(2, imagepng() href='function.imagepng'>function.imagepng</a>: open_basedir restriction in effect. File(/tmp/jpg_SNJixl) is not within the allowed path(s): (/var/www/web352/html/:/var/www/web352/phptmp/:/var/www/web352/files/:/var/www/web352/atd/:/usr/share/php/), /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php, 7545, Array ([=> Resource id #278,tempname => /tmp/jpg_SNJixl)) called at [imagepng(Resource id #278, /tmp/jpg_SNJixl) called at /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php:7545#2 TCPDF->_toPNG(Resource id #278) called at [TCPDF->Image(/var/www/web352/html/piwik/plugins/UserSettings/images/browsers/FF.gif, 12, 39,53875, 4) called at /var/www/web352/html/piwik/plugins/PDFReports/PDFRenderer.php:240#4 Piwik_PDFReports_PDFRenderer->paintReportTable() called at [Piwik_PDFReports_PDFRenderer->paintReport() called at /var/www/web352/html/piwik/plugins/PDFReports/API.php:284#6 Piwik_PDFReports_API->generateReport(1, 2011-01-08, 2, , 1, day) called at [call_user_func_array(Array (0 => Piwik_PDFReports_API Object ([=> Array ()),1 => generateReport), Array ([=> 1,1 => 2011-01-08,[=> 2,3 => ,[=> 1,5 => day)) called at [Piwik_API_Proxy->call(Piwik_PDFReports_API, generateReport, Array (token_auth => ________________,[=> API,action => index,[=> 2,period => day,[=> 2011-01-08,method => PDFReports.generateReport,[=> 1,outputType => 1,[=> 50)) called at /var/www/web352/html/piwik/core/API/Request.php:117#9 Piwik_API_Request->process() called at [Piwik_API_Controller->index() called at (null):0#11 call_user_func_array(Array ([=> Piwik_API_Controller Object ( => API,[=> , => ,[=> 2, => ),[=> index), Array ()) called at /var/www/web352/html/piwik/core/FrontController.php:125#12 Piwik_FrontController->dispatch() called at [/var/www/web352/html/piwik/index.php:60]

There is an error. Please report the message and full backtrace in the Piwik forums.

Warning: imagepng() href='function.imagepng'>function.imagepng</a>: Invalid filename in /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php on line 7545

Backtrace -->
#0 Piwik_ErrorHandler(2, imagepng() href='function.imagepng'>function.imagepng</a>: Invalid filename, /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php, 7545, Array ([=> Resource id #278,tempname => /tmp/jpg_SNJixl)) called at [imagepng(Resource id #278, /tmp/jpg_SNJixl) called at /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php:7545#2 TCPDF->_toPNG(Resource id #278) called at [TCPDF->Image(/var/www/web352/html/piwik/plugins/UserSettings/images/browsers/FF.gif, 12, 39,53875, 4) called at /var/www/web352/html/piwik/plugins/PDFReports/PDFRenderer.php:240#4 Piwik_PDFReports_PDFRenderer->paintReportTable() called at [Piwik_PDFReports_PDFRenderer->paintReport() called at /var/www/web352/html/piwik/plugins/PDFReports/API.php:284#6 Piwik_PDFReports_API->generateReport(1, 2011-01-08, 2, , 1, day) called at [call_user_func_array(Array (0 => Piwik_PDFReports_API Object ([=> Array ()),1 => generateReport), Array ([=> 1,1 => 2011-01-08,[=> 2,3 => ,[=> 1,5 => day)) called at [Piwik_API_Proxy->call(Piwik_PDFReports_API, generateReport, Array (token_auth => ________________,[=> API,action => index,[=> 2,period => day,[=> 2011-01-08,method => PDFReports.generateReport,[=> 1,outputType => 1,[=> 50)) called at /var/www/web352/html/piwik/core/API/Request.php:117#9 Piwik_API_Request->process() called at [Piwik_API_Controller->index() called at (null):0#11 call_user_func_array(Array ([=> Piwik_API_Controller Object ( => API,[=> , => ,[=> 2, => ),[=> index), Array ()) called at /var/www/web352/html/piwik/core/FrontController.php:125#12 Piwik_FrontController->dispatch() called at [/var/www/web352/html/piwik/index.php:60]

There is an error. Please report the message and full backtrace in the Piwik forums.

Warning: fopen() href='function.fopen'>function.fopen</a>: open_basedir restriction in effect. File(/tmp/jpg_SNJixl) is not within the allowed path(s): (/var/www/web352/html/:/var/www/web352/phptmp/:/var/www/web352/files/:/var/www/web352/atd/:/usr/share/php/) in /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php on line 7611

Backtrace -->
#0 Piwik_ErrorHandler(2, fopen() href='function.fopen'>function.fopen</a>: open_basedir restriction in effect. File(/tmp/jpg_SNJixl) is not within the allowed path(s): (/var/www/web352/html/:/var/www/web352/phptmp/:/var/www/web352/files/:/var/www/web352/atd/:/usr/share/php/), /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php, 7611, Array ([=> /tmp/jpg_SNJixl)) called at (null):0#1 fopen(/tmp/jpg_SNJixl, rb) called at [TCPDF->_parsepng(/tmp/jpg_SNJixl) called at /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php:7549#3 TCPDF->_toPNG(Resource id #278) called at [TCPDF->Image(/var/www/web352/html/piwik/plugins/UserSettings/images/browsers/FF.gif, 12, 39,53875, 4) called at /var/www/web352/html/piwik/plugins/PDFReports/PDFRenderer.php:240#5 Piwik_PDFReports_PDFRenderer->paintReportTable() called at [Piwik_PDFReports_PDFRenderer->paintReport() called at /var/www/web352/html/piwik/plugins/PDFReports/API.php:284#7 Piwik_PDFReports_API->generateReport(1, 2011-01-08, 2, , 1, day) called at [call_user_func_array(Array (0 => Piwik_PDFReports_API Object ([=> Array ()),1 => generateReport), Array ([=> 1,1 => 2011-01-08,[=> 2,3 => ,[=> 1,5 => day)) called at [Piwik_API_Proxy->call(Piwik_PDFReports_API, generateReport, Array (token_auth => ________________,[=> API,action => index,[=> 2,period => day,[=> 2011-01-08,method => PDFReports.generateReport,[=> 1,outputType => 1,[=> 50)) called at /var/www/web352/html/piwik/core/API/Request.php:117#10 Piwik_API_Request->process() called at [Piwik_API_Controller->index() called at (null):0#12 call_user_func_array(Array ([=> Piwik_API_Controller Object ( => API,[=> , => ,[=> 2, => ),[=> index), Array ()) called at /var/www/web352/html/piwik/core/FrontController.php:125#13 Piwik_FrontController->dispatch() called at [/var/www/web352/html/piwik/index.php:60]

There is an error. Please report the message and full backtrace in the Piwik forums.

Warning: fopen(/tmp/jpg_SNJixl) href='function.fopen'>function.fopen</a>: failed to open stream: Die Operation ist nicht erlaubt in /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php on line 7611

Backtrace -->
#0 Piwik_ErrorHandler(2, fopen(/tmp/jpg_SNJixl) href='function.fopen'>function.fopen</a>: failed to open stream: Die Operation ist nicht erlaubt, /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php, 7611, Array ([=> /tmp/jpg_SNJixl)) called at (null):0#1 fopen(/tmp/jpg_SNJixl, rb) called at [TCPDF->_parsepng(/tmp/jpg_SNJixl) called at /var/www/web352/html/piwik/libs/tcpdf/tcpdf.php:7549#3 TCPDF->_toPNG(Resource id #278) called at [TCPDF->Image(/var/www/web352/html/piwik/plugins/UserSettings/images/browsers/FF.gif, 12, 39,53875, 4) called at /var/www/web352/html/piwik/plugins/PDFReports/PDFRenderer.php:240#5 Piwik_PDFReports_PDFRenderer->paintReportTable() called at [Piwik_PDFReports_PDFRenderer->paintReport() called at /var/www/web352/html/piwik/plugins/PDFReports/API.php:284#7 Piwik_PDFReports_API->generateReport(1, 2011-01-08, 2, , 1, day) called at [call_user_func_array(Array (0 => Piwik_PDFReports_API Object ([=> Array ()),1 => generateReport), Array ([=> 1,1 => 2011-01-08,[=> 2,3 => ,[=> 1,5 => day)) called at [Piwik_API_Proxy->call(Piwik_PDFReports_API, generateReport, Array (token_auth => ________________,[=> API,action => index,[=> 2,period => day,[=> 2011-01-08,method => PDFReports.generateReport,[=> 1,outputType => 1,[=> 50)) called at /var/www/web352/html/piwik/core/API/Request.php:117#10 Piwik_API_Request->process() called at [Piwik_API_Controller->index() called at (null):0#12 call_user_func_array(Array ([=> Piwik_API_Controller Object ( => API,[=> , => ,[=> 2, => ),[=> index), Array ()) called at /var/www/web352/html/piwik/core/FrontController.php:125#13 Piwik_FrontController->dispatch() called at [/var/www/web352/html/piwik/index.php:60]

TCPDF ERROR: Can't open image file: /tmp/jpg_SNJixl

[anonymous-matomo-user]

Please set the priority of this bug higher. It effects every installation that has open_basedir restrictions. AFAIK, using open_basedir is a well established php security practice, I'm surprised that at least one of the developers (matt) isn't using it. You hit this bug every time that you upgrade because a new libs/tcpdf directory without the cache and images directories gets created. The problem is not the cache, but the images directory. tcpdf creates temporary png files for inclusion in the PDF report and that fails because the images subdirectory doesn't exist.
A simple solution is to create this or both directories (not sure if the cache directory is necessary at all for piwik) or just have them included in the tarball.
I agree that reusing the piwik tmp dir is a cleaner solution, but at least for the images directory I don't think it matters where it is as the files get deleted right after use and don't clobber the directory.
Thanks!

[mattab]

bolero, if you are able to provide a patch it would help! thank you

[anonymous-matomo-user]

Hm, the site seems to experience problems today. I wasn't able to submit or access the tracker for some hours.

I didn't change any code. The problem appeared after the 1.1.1 upgrade and so I searched the forum and found the problem and the solution in the German forum. I'm surprised that it wasn't mentioned in the English forum. Anyway, here's the link: http://forum.piwik.org/read.php?5,53811

The solution is as I mentioned: create the cache and images directories within the tcpdf root with appropriate rights, e.g. in our case

drwxr-xr-x  6 apache apache   4096 Jan 11 16:15 .
drwxr-xr-x 21 apache web11    4096 Jan  7 14:57 ..
-rw-r--r--  1 apache apache   7785 Jan  7 14:57 2dbarcodes.php
-rw-r--r--  1 apache apache  59791 Jan  7 14:57 barcodes.php
drwxr-xr-x  2 apache apache   4096 Jan 11 16:16 cache
-rw-r--r--  1 apache apache  76325 Jan  7 14:57 CHANGELOG.TXT
drwxr-xr-x  3 apache apache   4096 Nov  5 15:38 config
drwxr-xr-x  2 apache apache   4096 Jan  7 14:57 fonts
-rw-r--r--  1 apache apache  35147 Jan  7 14:57 gpl.txt
-rw-r--r--  1 apache apache   5499 Jan  7 14:57 htmlcolors.php
drwxr-xr-x  2 apache apache   4096 Jan 11 16:15 images
-rw-r--r--  1 apache apache   7651 Jan  7 14:57 lgpl-3.0.txt
-rw-r--r--  1 apache apache  53738 Jan  7 14:57 pdf417.php
-rw-r--r--  1 apache apache  80058 Jan  7 14:57 qrcode.php
-rw-r--r--  1 apache apache   3839 Jan  7 14:57 README.TXT
-rw-r--r--  1 apache apache   2153 Jan  7 14:57 spotcolors.php
-rw-r--r--  1 apache apache   2290 Jan  7 14:57 tcpdf.crt
-rw-r--r--  1 apache apache   1286 Jan  7 14:57 tcpdf.fdf
-rw-r--r--  1 apache apache   1749 Jan  7 14:57 tcpdf.p12
-rw-r--r--  1 apache apache 950467 Jan  7 14:57 tcpdf.php
-rw-r--r--  1 apache apache 227828 Jan  7 14:57 unicode_data.php

so, simply adding the directories to the tcpdf install source should suffice. Maybe with 775 or 777 permissions, as 755 will probably not be sufficient for most installations.

[robocoder]

I'll look into a proper fix. We don't want temporary files created in core, libs, or plugins. If code is shared between multiple installations, there's the potential for conflict.

[anonymous-matomo-user]

Thanks!

[anonymous-matomo-user]

I've taken a look at my own tcpdf installation and found that it contains an images directory and also a cache directory. And both directories have content. Mostly images used for the examples and the tcpdf logo. So, the install source contains those directories. You must be removing them because you don't need the examples etc.
The tcpdf included in piwik doesn't contain any docs or examples. Don't get me wrong, but I wonder if this might not be a violation of the license. Nicola also recently changed the license from GPL2 to GPL3 plus a small addendum and insists on on compliance to the letter. I would check with him if it is ok to distribute tcpdf in this form.

[robocoder]

We've already done an extensive license review. This was a pre-requisite to submitting Piwik to the FSF directory. http://directory.fsf.org/project/piwik/

TCPDF is actually LGPL v3. The LGPL terms are written as an addition to the GPL ... that's why you see both gpl.txt and lgpl-3.0.txt in the folder. LGPLv3 is compatible with GPLv3 license used by Piwik.

Both licenses expressly allow derivatives (by addition, modification, omission, etc). The license requires that we provide source to what we distribute, so we are in compliance. On top of that, we preserve attribution and include a URL to the project in ./LEGALNOTICE.

[robocoder]

Ok, the proposed fix:

• define K_TCPDF_EXTERNAL_CONFIG, to ignore the tcpdf/config/* settings; downside is that there's a lot to define
• create tmp/tcpdf/{cache|images}
• add Proxy method to download the generated PDF (similar to minified CSS/JS assets)

[mattab]

downside is that there's a lot to define
what is the downside exactly? do we have to copy paste some of their code?

tmp/tcpdf/
that will be 2 more directories to give write access to? Maybe they could be written in a single tmp/pdf/ directory to keep things simple?

new Proxy method
currently the API itself acts as a proxy:

        case self::OUTPUT_PDF_DOWNLOAD:
            $outputFilename = "$websiteName - $prettyDate - $description.pdf";
            $flagOutput = 'D';
        break;
        default:
        case self::OUTPUT_PDF_INLINE_IN_BROWSER:
            $flagOutput = 'I';
        break;
    }
    $pdf->Output($outputFilename, $flagOutput);

so I dont think we need a new proxy method

[mattab]

I think there is a new occurence of this bug maybe: http://forum.piwik.org/read.php?2,72150

[robocoder]

re: comment:18 -- that's the same as comment:8

[mattab]

Creating the empty directories in tcpdf is not enough since they also require write persmissions. Not sure if we can chmod during install, or maybe throw an error when openbasedir restrictions are in place?

[robocoder]

We shouldn't create tmp files in libs because:

• it's inconsistent with the rest of Piwik (plus, checkDirectoriesWritable() uses PIWIK_USER_PATH)
• it would add yet another directory to exclude when a user makes a backup or runs an IDS scan

[anonymous-matomo-user]

In case you do not want to create temporary files there then tcpdf should create them in the tmp directory the virtual host uses or the tmp directory piwik uses. The only option then is to change the define ('K_PATH_IMAGES', K_PATH_MAIN.'images/'); or ask Nicola for another way of setting his config options.

[robocoder]

I'll follow-up upstream ... it would be nice to have something simpler than comment:16.

[robocoder]

(In [4210]) fixes #1656 - custom config file to override K_PATH_CACHE and K_PATH_IMAGES

• also update to tcpdf 5.9.062

[robocoder]

(In [4212]) refs #1656 - fix applied upstream

[robocoder]

(In [4213]) refs #1656 - revert part of r4212 back to mirror upstream; the "fix applied upstream" is in reference to r3587

[mattab]

PDF export is broken, is it workign for you?

Warning: opendir(D:/piwik/svn/trunk/plugins/PDFReports/fonts/) href='function.opendir'>function.opendir</a>: failed to open dir: No error in D:\piwik\svn\trunk\libs\tcpdf\tcpdf.php on line 4716

Warning: readdir(): supplied argument is not a valid Directory resource in D:\piwik\svn\trunk\libs\tcpdf\tcpdf.php on line 4717

[robocoder]

Can't check right now, but I know where the problem is. I'll fix when I get back home. Thanks.

[mattab]

Cool, the problem is only with the fonts directory (once I copied over from tcpdf/fonts where it was expecting it in plugins/PDFReports/fonts PDF generation was working)

[robocoder]

(In [4224]) fixes #1656

[robocoder]

Update: we'll have to continue using the custom config file in plugins/PDFReports/config because the patch I submitted upstream was rejected.

[rtn]

drwxr-xr-x 6 apache apache 4096 Jan 11 16:15 .
drwxr-xr-x 21 apache web11 4096 Jan 7 14:57 ..

That's scarry!!! This is not definitely written with web security in mind. I just fetched the source and try to install piwik but cannot do so without putting my web server in a risky situation.

1. apache or user that web engine run as should not have full privilege to the web directories
2. session directory should not be under the webroot
3. cache directory should not be under the webroot

I have not find a config variable or method in piwik to put my tmp directory and config directory outside the web root yet; so, i am not using this for web security reason. on older version of piwik (maybe 4-5 years back), i was able to put the config outside the web root.

i found this: #8120

web security is definitely not what piwik wants to deal with. thomaszbz made an excellent point but seems to be shutdown for giving an excellent security critic.