Module HAR #263

william-p · 2014-03-25T14:35:46Z

Add har module for export HAR data in external file.
I use netsniff.js for convert to HAR format.
(#64)

macbre · 2014-03-25T15:16:01Z

Please update README.md file as well

macbre · 2014-03-25T15:17:48Z

modules/har/har.js

+ */
+
+if (!Date.prototype.toISOString) {
+    Date.prototype.toISOString = function () {


phantomjs> console.log(typeof Date.prototype.toISOString); function

Polyfill not needed here :)

…ntext

william-p · 2014-03-25T15:24:34Z

Done ;)

macbre · 2014-03-25T16:43:38Z

modules/har/har.js

+            version: '1.2',
+            creator: {
+                name: "Phantomas - HAR",
+                version: VERSION


I'd use phantomas version here to ease debugging

macbre · 2014-03-25T16:44:03Z

One tiny little detail left and we're ready to merge it :)

william-p · 2014-03-25T16:58:51Z

You are right 😄
How to get Phantomas version properly ?
phantomas.version is undefined :/

For #263

macbre · 2014-03-25T18:35:19Z

@william-p, good point. I've added phantomas.getVersion function in 3ca06c7

william-p · 2014-03-25T18:55:31Z

Okay, now i'm first to use it 😄

macbre · 2014-03-25T19:04:50Z

modules/har/har.js

+                id: address,
+                title: title,
+                pageTimings: {
+                    onLoad: endTime.getTime() - startTime.getTime()


https://dvcs.w3.org/hg/webperf/raw-file/tip/specs/HAR/Overview.html#sec-object-types-pageTimings

HAR spec mentions the optional onContentLoad property that "represents DOMContentLoad event or document.readyState == interactive". Worth adding?

cvan · 2014-03-26T00:32:22Z

modules/har/har.js

+        phantomas.log('Convert HAR to JSON');
+        var dump = JSON.stringify(har);
+
+        phantomas.log('Write HAR in \'%s\'', path);


wrap with double quotes so you don't have to escape single quotes

cvan · 2014-03-26T00:33:38Z

HAR support is awesome to see! it'd be nice if we could identify which resources are inline à la https://github.com/cvan/phantomHAR/blob/master/phantomhar.js#L232 but that's an enhancement

cvan · 2014-03-26T00:35:25Z

modules/har/har.js

+                headers: endReply.headers,
+                redirectURL: "",
+                headersSize: -1,
+                bodySize: startReply.bodySize,


this is not accurate but that's phantomJS' fault. In a project of mine, I actually request each resource separately to get the accurate number of bytes transferred (and the content of each response). It's a pain.

For macbre#263

william-p · 2014-03-26T09:53:15Z

@cvan, thanks for your comments, i appreciate.
Sorry for code style, major part of code is a wrapper around original ntsniff but you are right. I will work for improve this.
Maybe the best solution is to use your HAR code ? it seems more efficient.
What do you think ?

macbre · 2014-03-26T10:37:18Z

@william-p, it's actually a good idea to re-use @cvan code given the fact that phantomas already provides information such as type of response's content.

william-p · 2014-03-26T10:42:16Z

@macbre, i'm agree :) you are ok with simple copy/paste with origin in comments ?

macbre · 2014-03-26T11:48:46Z

Copy&paste - no :) As I said previously, phantomas provides most of the information for each request / response that phantom-HAR generates. There's no need to copy & paste the whole code.

william-p · 2014-03-26T11:52:13Z

Of course, I talked about createHAR function and utils if need ...

macbre · 2014-03-26T11:54:53Z

In that case, I'm all for it :)

william-p · 2014-03-26T14:34:42Z

nb: inline is not supported in this commit

william-p · 2014-03-26T15:32:41Z

Humm, module jsErrors raise a error in report ...

"jsErrors": [
    "SECURITY_ERR: DOM Exception 18: An attempt was made to break through the security policy of the user agent. - unknown fn: phantomjs://webpage.evaluate() @ 2 / unknown fn: phantomjs://webpage.evaluate() @ 3 / unknown fn: phantomjs://webpage.evaluate() @ 3"
]

Edit:
Ok, localStorage.clear() is the cause.

cvan · 2014-03-26T19:18:31Z

Hmm. I understand the reason behind reusing phantomHAR code, but adding "custom" properties to PhantomJS page object is something I'd like to avoid. Can a "local" module variable / variables be used instead?

The issue becomes unless you're keeping track of the rendered DOM, you don't know for example which images were inline and which were defined in stylesheets.

I would recommend skipping on that part; that can be an enhancement.

Ideally, @cvan can publish phantomHAR to npm and expose CommonJS module that can be reused in phantomas code.

I would not recommend reusing phantomHAR code, especially for this part.

I've been reverse-engineering Chrome's HAR builder. And I've been working on a patch for my project (cvan/fastHAR-api) that generates the correct values for the following:

response.headersSize (length of raw headers)
response.bodySize (transfer size - response.headersSize)
response.content.size (length of response body)
response.content.compression (response.bodySize - response.content.size)

I'll be sure to open a pull request once I'm done.

Nice work, guys!

macbre · 2014-03-26T20:51:27Z

The issue becomes unless you're keeping track of the rendered DOM, you don't know for example > which images were inline and which were defined in stylesheets.

I would recommend skipping on that part; that can be an enhancement.

@cvan, I do not quite get you here. I was referring to an array that stores a list of requests that is a stored as page object member instead of being a variable in module's scope.

That's my only concern regarding code style and good practices in general. Once this one is solved, we're ready to merge...

@cvan, looking forward to your pull request :)

william-p · 2014-03-26T21:05:57Z

Okay, until something better, tomorrow (France) i will make a patch for use own scope in place of page variable.

@cvan, i'm waiting with impatience your enhancements ;)

Nice work too 😄

macbre · 2014-03-26T21:56:28Z

@william-p, merci :)

macbre · 2014-03-27T17:55:10Z

@william-p, this one is great 👍 Thanks @cvan for the feedback.

Module HAR

william-p · 2014-03-27T17:56:59Z

Nice job guys, thanks for merge :)

william-p added 8 commits February 24, 2014 18:08

Add HAR module, first release

803131e

add: fill endTime if event 'loadFinished' was not fired

d854956

check contentType

c8ed0bb

add use fs for write har dump (--har <path>)

f14204d

Merge branch 'master' into module-har

ddc7819

add har param

f3748eb

fix js style

c632fcb

fix js style again

d3088b1

macbre reviewed Mar 25, 2014
View reviewed changes

william-p added 2 commits March 25, 2014 16:22

update with har parameter

0036a36

remove Date.prototype.toISOString function, useless in phantomjs co…

4b49d6d

…ntext

macbre reviewed Mar 25, 2014
View reviewed changes

macbre added a commit that referenced this pull request Mar 25, 2014

Expose phantomas version via getVersion

3ca06c7

For #263

william-p added 2 commits March 25, 2014 19:38

Use version of Phantomas in 'creator' field

7f45e2c

Merge branch 'master' into module-har

8e6daa7

macbre added this to the v1.0 milestone Mar 25, 2014

macbre added the enhancement label Mar 25, 2014

macbre reviewed Mar 25, 2014
View reviewed changes

cvan reviewed Mar 26, 2014
View reviewed changes

vgangan pushed a commit to vgangan/phantomas that referenced this pull request Mar 26, 2014

Expose phantomas version via getVersion

87fb176

For macbre#263

Replace 'netsniff.js' by 'phantomHAR' code

2ddef09

Remove 'localStorage.clear()'

d06f915

william-p added 2 commits March 27, 2014 12:41

Remove useless getType function

09891a4

Use own scope for page variable

713bbcb

macbre added a commit that referenced this pull request Mar 27, 2014

Merge pull request #263 from william-p/module-har

50d980f

Module HAR

macbre merged commit 50d980f into macbre:master Mar 27, 2014

william-p deleted the module-har branch March 27, 2014 17:57

macbre mentioned this pull request Mar 27, 2014

Add DOMContentLoaded information to HAR #266

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Module HAR #263

Module HAR #263

william-p commented Mar 25, 2014

macbre commented Mar 25, 2014

macbre Mar 25, 2014

william-p commented Mar 25, 2014

macbre Mar 25, 2014

macbre commented Mar 25, 2014

william-p commented Mar 25, 2014

macbre commented Mar 25, 2014

william-p commented Mar 25, 2014

macbre Mar 25, 2014

cvan Mar 26, 2014

cvan commented Mar 26, 2014

cvan Mar 26, 2014

william-p commented Mar 26, 2014

macbre commented Mar 26, 2014

william-p commented Mar 26, 2014

macbre commented Mar 26, 2014

william-p commented Mar 26, 2014

macbre commented Mar 26, 2014

william-p commented Mar 26, 2014

william-p commented Mar 26, 2014

cvan commented Mar 26, 2014

macbre commented Mar 26, 2014

william-p commented Mar 26, 2014

macbre commented Mar 26, 2014

macbre commented Mar 27, 2014

william-p commented Mar 27, 2014

Module HAR #263

Module HAR #263

Conversation

william-p commented Mar 25, 2014

macbre commented Mar 25, 2014

macbre Mar 25, 2014

Choose a reason for hiding this comment

william-p commented Mar 25, 2014

macbre Mar 25, 2014

Choose a reason for hiding this comment

macbre commented Mar 25, 2014

william-p commented Mar 25, 2014

macbre commented Mar 25, 2014

william-p commented Mar 25, 2014

macbre Mar 25, 2014

Choose a reason for hiding this comment

cvan Mar 26, 2014

Choose a reason for hiding this comment

cvan commented Mar 26, 2014

cvan Mar 26, 2014

Choose a reason for hiding this comment

william-p commented Mar 26, 2014

macbre commented Mar 26, 2014

william-p commented Mar 26, 2014

macbre commented Mar 26, 2014

william-p commented Mar 26, 2014

macbre commented Mar 26, 2014

william-p commented Mar 26, 2014

william-p commented Mar 26, 2014

cvan commented Mar 26, 2014

macbre commented Mar 26, 2014

william-p commented Mar 26, 2014

macbre commented Mar 26, 2014

macbre commented Mar 27, 2014

william-p commented Mar 27, 2014