[CLOSED] Some integration and extension tests fail when running all

[core-ai-bot]

Issue by jasonsanjose
Thursday Apr 04, 2013 at 18:50 GMT
Originally opened as adobe/brackets#3343

---

When running grunt test-integration --suite=all, the following suites see intermittent failures: Live Development, StaticServer, NodeConnection. All of these suites rely on NodeConnection. It's not clear why they pass in isolation, but fail in the overall "all" suite. Below is a sample of the failures due to timeout.

        "Live Development": {
            "id": 140,
            "name": "Live Development",
            "specCount": 12,
            "passedCount": 6,
            "failedCount": 6,
            "specs": [
                {
                    "name": "Live Development Live Development startup and shutdown should return a ready socket on Inspector.connect and close the socket on Inspector.disconnect.",
                    "description": "should return a ready socket on Inspector.connect and close the socket on Inspector.disconnect",
                    "passed": false,
                    "messages": [
                        "timeout: timed out after 5000 msec waiting for success Inspector.connectToURL"
                    ]
                },
                {
                    "name": "Live Development CSS Editing should establish a browser connection for an opened html file.",
                    "description": "should establish a browser connection for an opened html file",
                    "passed": false,
                    "messages": [
                        "timeout: timed out after 10000 msec waiting for Waiting for browser to become active"
                    ]
                },
                {
                    "name": "Live Development CSS Editing should push changes through the browser connection.",
                    "description": "should push changes through the browser connection",
                    "passed": false,
                    "messages": [
                        "timeout: timed out after 10000 msec waiting for Waiting for browser to become active"
                    ]
                },
                {
                    "name": "Live Development CSS Editing should ignore query strings in linked CSS file hrefs.",
                    "description": "should ignore query strings in linked CSS file hrefs",
                    "passed": false,
                    "messages": [
                        "timeout: timed out after 10000 msec waiting for Waiting for browser to become active"
                    ]
                },
                {
                    "name": "Live Development CSS Editing should push in memory css changes made before the session starts.",
                    "description": "should push in memory css changes made before the session starts",
                    "passed": false,
                    "messages": [
                        "timeout: timed out after 10000 msec waiting for Waiting for LiveDevelopment document"
                    ]
                },
                {
                    "name": "Live Development CSS Editing should reapply in-memory css changes after saving changes in html document.",
                    "description": "should reapply in-memory css changes after saving changes in html document",
                    "passed": false,
                    "messages": [
                        "timeout: timed out after 2000 msec waiting for success LiveDevelopment.open()"
                    ]
                }
            ]
        },

        "StaticServer": {
            "id": 148,
            "name": "StaticServer",
            "specCount": 14,
            "passedCount": 10,
            "failedCount": 4,
            "specs": [
                {
                    "name": "StaticServer StaticServerDomain should trigger an event when a file path is requested.",
                    "description": "should trigger an event when a file path is requested",
                    "passed": false,
                    "messages": [
                        "Error: Expected '<!doctype html>\n<html data-brackets-id='5'>\n<head data-brackets-id='2'>\n<meta charset=\"UTF-8\">\n<title data-brackets-id='1'>Simple Test</title>\n<link rel=\"stylesheet\" href=\"simpleShared.css\">\n<link rel=\"stylesheet\" href=\"simple1.css\">\n</head>\n\n<body data-brackets-id='4'>\n<p data-brackets-id='3' id=\"testId\" class=\"testClass\">Brackets is awesome!</p>\n</body>\n</html>\n' to be 'This is a file in folder 1.'.\n    at new jasmine.ExpectationResult (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:102:32)\n    at null.toBe (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1194:29)\n    at path (file:///C:/Users/jasonsj/Github/brackets/src/extensions/default/StaticServer/unittests.js:266:34)\n    at jasmine.Block.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1024:15)\n    at jasmine.Queue.next_ (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1842:31)\n    at onComplete (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1838:18)\n    at jasmine.WaitsForBlock.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:2322:5)\n    at file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:2336:12",
                        "Error: Expected 71 to be greater than 500.\n    at new jasmine.ExpectationResult (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:102:32)\n    at null.toBeGreaterThan (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1194:29)\n    at path (file:///C:/Users/jasonsj/Github/brackets/src/extensions/default/StaticServer/unittests.js:269:37)\n    at jasmine.Block.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1024:15)\n    at jasmine.Queue.next_ (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1842:31)\n    at onComplete (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1838:18)\n    at jasmine.WaitsForBlock.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:2322:5)\n    at file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:2336:12"
                    ]
                },
                {
                    "name": "StaticServer StaticServerDomain should send static file contents after canceling a filter request.",
                    "description": "should send static file contents after canceling a filter request",
                    "passed": false,
                    "messages": [
                        "Error: Expected '<!doctype html>\n<html data-brackets-id='5'>\n<head data-brackets-id='2'>\n<meta charset=\"UTF-8\">\n<title data-brackets-id='1'>Simple Test</title>\n<link rel=\"stylesheet\" href=\"simpleShared.css\">\n<link rel=\"stylesheet\" href=\"simple1.css\">\n</head>\n\n<body data-brackets-id='4'>\n<p data-brackets-id='3' id=\"testId\" class=\"testClass\">Brackets is awesome!</p>\n</body>\n</html>\n' to be 'This is a file in folder 1.'.\n    at new jasmine.ExpectationResult (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:102:32)\n    at null.toBe (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1194:29)\n    at path (file:///C:/Users/jasonsj/Github/brackets/src/extensions/default/StaticServer/unittests.js:316:34)\n    at jasmine.Block.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1024:15)\n    at jasmine.Queue.next_ (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1842:31)\n    at onComplete (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1838:18)\n    at jasmine.WaitsForBlock.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:2322:5)\n    at file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:2336:12"
                    ]
                },
                {
                    "name": "StaticServer StaticServerDomain should override the static file server response with a new response body.",
                    "description": "should override the static file server response with a new response body",
                    "passed": false,
                    "messages": [
                        "Error: Expected '<!doctype html>\n<html data-brackets-id='5'>\n<head data-brackets-id='2'>\n<meta charset=\"UTF-8\">\n<title data-brackets-id='1'>Simple Test</title>\n<link rel=\"stylesheet\" href=\"simpleShared.css\">\n<link rel=\"stylesheet\" href=\"simple1.css\">\n</head>\n\n<body data-brackets-id='4'>\n<p data-brackets-id='3' id=\"testId\" class=\"testClass\">Brackets is awesome!</p>\n</body>\n</html>\n' to be 'custom response'.\n    at new jasmine.ExpectationResult (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:102:32)\n    at null.toBe (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1194:29)\n    at path (file:///C:/Users/jasonsj/Github/brackets/src/extensions/default/StaticServer/unittests.js:360:34)\n    at jasmine.Block.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1024:15)\n    at jasmine.Queue.next_ (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1842:31)\n    at goAgain (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1832:18)"
                    ]
                },
                {
                    "name": "StaticServer StaticServerDomain should ignore multiple responses for the same request.",
                    "description": "should ignore multiple responses for the same request",
                    "passed": false,
                    "messages": [
                        "Error: Expected 2 to be 1.\n    at new jasmine.ExpectationResult (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:102:32)\n    at null.toBe (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1194:29)\n    at path (file:///C:/Users/jasonsj/Github/brackets/src/extensions/default/StaticServer/unittests.js:405:41)\n    at jasmine.Block.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1024:15)\n    at jasmine.Queue.next_ (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1842:31)\n    at onComplete (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1838:18)\n    at jasmine.WaitsForBlock.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:2322:5)\n    at file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:2336:12",
                        "Error: Expected '<!doctype html>\n<html data-brackets-id='5'>\n<head data-brackets-id='2'>\n<meta charset=\"UTF-8\">\n<title data-brackets-id='1'>Simple Test</title>\n<link rel=\"stylesheet\" href=\"simpleShared.css\">\n<link rel=\"stylesheet\" href=\"simple1.css\">\n</head>\n\n<body data-brackets-id='4'>\n<p data-brackets-id='3' id=\"testId\" class=\"testClass\">Brackets is awesome!</p>\n</body>\n</html>\n' to be 'good response'.\n    at new jasmine.ExpectationResult (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:102:32)\n    at null.toBe (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1194:29)\n    at path (file:///C:/Users/jasonsj/Github/brackets/src/extensions/default/StaticServer/unittests.js:410:34)\n    at jasmine.Block.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1024:15)\n    at jasmine.Queue.next_ (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1842:31)\n    at onComplete (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:1838:18)\n    at jasmine.WaitsForBlock.execute (file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:2322:5)\n    at file:///C:/Users/jasonsj/Github/brackets/test/thirdparty/jasmine-core/jasmine.js:2336:12"
                    ]
                }
            ]
        },

[core-ai-bot]

Comment by RaymondLim
Thursday Apr 04, 2013 at 22:51 GMT

---

Not sure it's only me. I can consistently reproduce one unit test failure both on my win7 and mac 10.8.2. The unit test failing is "Live Development startup and shutdown".

[core-ai-bot]

Comment by njx
Monday Apr 08, 2013 at 18:20 GMT

---

Reviewed. To@jasonsanjose for sprint 24.

[core-ai-bot]

Comment by redmunds
Friday Apr 26, 2013 at 16:52 GMT

---

FBNC for@jasonsanjose

[core-ai-bot]

Comment by jasonsanjose
Friday Apr 26, 2013 at 17:00 GMT

---

Removed FBNC. #3628 was just a first step in re-arranging how we run tests on Jenkins. They are still failing more often than not. The new Jenkins jobs give us some more flexibility in how we run the tests and how we capture the results.

[core-ai-bot]

Comment by jasonsanjose
Friday Apr 26, 2013 at 17:03 GMT

---

Removing Sprint 24 milestone

[core-ai-bot]

Comment by njx
Tuesday May 21, 2013 at 17:15 GMT

---

Moved to sprint 26

[core-ai-bot]

Comment by jasonsanjose
Friday Jun 07, 2013 at 17:12 GMT

---

Too late to land for sprint 26. Punting to sprint 27.