Provide 'Non Javascript Tracking tag' and document Tracking push API to piwik.php

[mattab]

This is related to #134, but is not duplicate. #134 is about bulk loading logs into Piwik, this ticket is a first step:

• Propose non js tracking tag (simple image tag) in the 'display JS' page. Document limitations (missing parameters and bots tracked like normal users)

• Clarify that <noscript> still doesn't track anything

• Publish the full Tracking API to piwik.php

  • At first, PHP client that returns the URL to piwik.php with parameters (users can use in a dynamically built <img src=""> tag)
  • the client will allow to define optional parameters such as: user agent, IP, accepted languages, server time, that when specified, will overwrite the systems defaults.
  • this API can also do the http request itself (rather than returning the URL to put in a img src). This will make it possible to track interactions from any software or system that doesn't execute Javascript.
  • We must be careful in the future with 1st party cookies (Duplicate 1st party cookie #557), the client itself will have to set cookies (as well as forwarding any cookie set by piwik.php)

• we can then use this API to build simple integration test.
  For example, we can generate 2 visits with known keywords/resolution/website/countries, etc. Then call the API that will archive data and return reports. First time we would save the XML reports in some directory. Next times, the output would be compared to the saved XML files, and if there is a difference, test will fail. Developper can then manually check the diff between outputs and overwrite the 'expected' outputs if they are indeed expected (and so if the format/algorithms/data changed).

  By using this technique, I hope that we can implement pretty easily a very good generic test coverage for Piwik, before the 1.0 release.

[mattab]

(In [2430]) Refs #1446

[mattab]

in [2429]

Introducing documented piwik.php PHP Tracking client.
Also adding integration test suite in place.

    * Adding new mechanism to trigger known tracking requests, then call all API methods get* and compare XML to the previous 'expected' XML.
    * make sure overwritten config files are not read during tests. Use defaults from global.ini.php.
    * piwik.php Tracker allow calling script to force Datetime and user IP for a specific request.
    * Wrote 3 main tests
          o wrong tracking requests should not return error and should not record anything
          o standard one visitor, 2 visits, with page view, download, outlink and goal tracked
          o Added integration test as an example in the ExampleAPI plugin (pretty simple to add testing code) 

TODO

    * Finish PiwikTracker and show it in UI
    * Show how to use image based tracker in UI
    * Add more tests (multi periods and multi sites) in Main.test.php
    * Fix failing Config.test.php test (when ran via all_tests.php doesn't fail otherwise) 

[mattab]

(In [2431]) Now testing various API exports for ExampleAPI plugins, for all API formats: xml, csv, tsv, json, etc.
Any breaking or modifying change in the renderer or data conversion should trigger a test fail.

Improved parameters in ExampleAPI API.php
Refs #1446

[mattab]

(In [2432]) Refs #1446 Fixing failing test due to Windows EOL and headers already sent error. Also not testing 'format=original'.

[mattab]

(In [2436]) Fixing typos in tests Refs #1446

[mattab]

(In [2441]) Refs #1446

• Adding test covering tracking on multiple websites, test requesting all periods

[mattab]

(In [2442]) Fixing bug in Archiving found with this new test suite (max_actions was incorrectly summed during archiving, rather than max() operation)
Updating expected file as naming convention changed in previous commit
Refs #1446

[mattab]

(In [2453]) Fixes #1446

• refreshed whole tracking code page. Refactored code to reuse in Installation and in the piwik.org docs at http://piwik.org/docs/tracking-api/ (upcoming)
• inviting externals devs to submit the Tracking API Client in their languages (currently in PHP, could be in Perl, python, java etc)
• Comments in PiwikTracker
• Removed unused admin UI strings

[halfdan]

Regression: Getting wrong idsite in alternative tracking code for idsite > 1. Screenshot attached.

[halfdan]

Attachment:
WrongidSite.png

[halfdan]

On a second view: Duplicate http:// showing in alternative tracking code.

[mattab]

(In [2457]) Refs #1446
Fixes idSite display and duplicate http://

[mattab]

(In [2472]) Refs #1446 Refs #818

• Fixing unit tests on local box by ignoring ExampleAPI.getPiwikVersion which would fail at every new version

Note: on my box, when I run all_tests.php, I don't see the green bar at the bottom of the page anymore. It simply displays the Time and Memory delta. I didn't see any commit that coudl have broken this so I'm a bit confused as to why the green/red bar doesn't display anymore?

[robocoder]

installer is broken:

• Installation_JsTag is untranslated (appears in the list of steps)
• on index.php?action=displayJavascriptCode&module=Installation, there's a flash of unstylized text, band then only the logo and steps are shown. A manual refresh then seems to work.

[robocoder]

Attachment:
Screenshot.png

[mattab]

(In [2482]) Fixes #1446: translating step name + fixing smarty error

[mattab]

(In [2495]) Refs #1446
Integration tests were often different simply because the internal idsubdatatable in the response was different. However. this idsubtable is really just an internal ID and it is not expected that this ID is the same across several archiving processes. Therefore integration tests will now set a special flag to remove the ID from the response, to minimize noise.