Verify that the criteria for advancing the phase for test plans has b…

…een met by referencing required reports for a phase (#722) * initial model, migration, and seeder * seeders test plan ids * test plan service * fixing migration * updating migrations to have foreign keys * Implementing Howard's feedback * Start to migrations and functions to support the DB change for further support of 648 * Update migrations and resolvers to update the individual test plan reports so they can be a part of the same testplanversion going through the candidate phase * Add phase to TestPlanVersion * Update resolver and logic for providing content on the reports page * Update query and resolvers for single report pages * Update query and resolvers for test plan version mutations * Support for CandidateTests functionality and returned other basic functionality which will be removed in the future such as individual test plan report updating * Restored viewing of CandidateTestPlanRun * Fix graphql.schema * Update migrations * Remove unused references * Update tests * Rename TestPlanVersion date fields with 'status' in name to 'phase' * Add approvedAt column to TestPlanReport * Rename candidateStatusReachedAt to approvedAt * Update migration to not use server specific function * Update migration to be more clear and help in functional priority of following migrations * Start draft of new Data Management page * Add links to single-page view of the plans * Update migration to prevent testPlanVersions not in the DRAFT phase having a draftPhaseReachedAt value * Update migration to prevent testPlanVersions not in the DRAFT phase having a draftPhaseReachedAt value and set the phase to RD if not * Update content of cells * Show remaining data content for the Data Management Row * Update the updatePhaseResolver * Added checks to account for how the phase promotions will be done * Making changes to account for using data from earlier runs when promoting a test plan version * Implementation of preserving data with earlier version of test plan when the runs for a target doesn't exist * Cleanup, comments and refactoring * Sunset earlier test plan version during phase update * Update TestManagement references to DataManagement * Update Test Plan, Covered At and Overall Status columns based on latest mockups * Implemented P0 styling items from latest mockups * Styling updates * Remove old version of test management content * Add proptypes check * Update Data Management page to allow access without being an admin * Hide admin actions on data management page * Adjust column sizes * Update tests * Update condition for when 'Advance to Recommended' button is shown * Restrict condition for phase transition * Editorial changes * Editorial changes * Cleanup TODOs * Address rare error condition * Add confirmation dialog when advancing test plan version; reduce values being requested * Unique test plan report combinations shown in advance confirmation dialog * Update test * Update tests * Update focus points when advancing phases * Update migration to make purpose clearer * Add validation for candidate phase * Set default sorting for data management table * Add beginning of validation for Recomended phase * Create /test-review endpoint for displaying what's available at aria-at.netlify in-app * Update Data Management VERSION_STRING link to point to template generated at /test-review * Fix sorting related bug with ManageTestQueue component * Remove required reports logic from updatePhaseResolver; needs to be moved to its own PR * Update test * Stop checks to see if testPlanReports exist which use an AT. Instead, show all currently known ats in this Covered ATs column * Reverse logic for showing the latest versions for a test plan * Re-evaluate sorting logic for DataManagement default phase sorting and applicable Test Plan Versions shown for ManageTestQueue component * Finish verification for recommended phase * Fix failing tests except the datamanagement tests * Add beginning of validation for Recomended phase * Finish verification for recommended phase * Fix failing tests except the datamanagement tests * Update AtBrowsers and everything that has to do with it * Correcting merge conflict lines * Fix updatePhaseResolver * Fix failing tests * Remove comment * Fix function signiture --------- Co-authored-by: Erika Miguel <[email protected]> Co-authored-by: Howard Edwards <[email protected]>
w3c · Aug 15, 2023 · 527e78f · 527e78f
1 parent e7d6fb5
commit 527e78f
Show file tree

Hide file tree

Showing 16 changed files with 232 additions and 44 deletions.
diff --git a/server/graphql-schema.js b/server/graphql-schema.js
@@ -87,6 +87,14 @@ const graphqlSchema = gql`
         The Ats which can be run with the specific browser, for example, Jaws can be run with Chrome but not Safari, and Safari works with VoiceOver only.
         """
         ats: [At]!
+        """
+        The Ats which are required to move a TestPlanVersion to CANDIDATE phase.
+        """
+        candidateAts: [At]!
+        """
+        The Ats which are required to move a TestPlanVersion to RECOMMENDED phase.
+        """
+        recommendedAts: [At]!
     }
 
     """
@@ -153,6 +161,14 @@ const graphqlSchema = gql`
         The browsers which can run the At, for example, Safari can run VoiceOver but not Jaws because Jaws is Windows only.
         """
         browsers: [Browser]!
+        """
+        The browsers which are required to move a TestPlanVersion to CANDIDATE phase.
+        """
+        candidateBrowsers: [Browser]!
+        """
+        The browsers which are required to move a TestPlanVersion to RECOMMENDED phase.
+        """
+        recommendedBrowsers: [Browser]!
     }
 
     """

diff --git a/server/migrations/20230523163855-addColumnsToAtBrowsers.js b/server/migrations/20230523163855-addColumnsToAtBrowsers.js
@@ -0,0 +1,39 @@
+'use strict';
+
+/** @type {import('sequelize-cli').Migration} */
+module.exports = {
+    async up(queryInterface, Sequelize) {
+        return queryInterface.sequelize.transaction(async transaction => {
+            await queryInterface.addColumn(
+                'AtBrowsers',
+                'isCandidate',
+                {
+                    type: Sequelize.DataTypes.BOOLEAN,
+                    allowNull: false
+                },
+                { transaction }
+            );
+
+            await queryInterface.addColumn(
+                'AtBrowsers',
+                'isRecommended',
+                {
+                    type: Sequelize.DataTypes.BOOLEAN,
+                    allowNull: false
+                },
+                { transaction }
+            );
+        });
+    },
+
+    async down(queryInterface /* , Sequelize */) {
+        return queryInterface.sequelize.transaction(async transaction => {
+            await queryInterface.removeColumn('AtBrowsers', 'isCandidate', {
+                transaction
+            });
+            await queryInterface.removeColumn('AtBrowsers', 'isRecommended', {
+                transaction
+            });
+        });
+    }
+};
diff --git a/server/models/AtBrowsers.js b/server/models/AtBrowsers.js
@@ -19,6 +19,14 @@ module.exports = function (sequelize, DataTypes) {
                     model: 'Browser',
                     key: 'id'
                 }
+            },
+            isCandidate: {
+                type: DataTypes.BOOLEAN,
+                allowNull: false
+            },
+            isRecommended: {
+                type: DataTypes.BOOLEAN,
+                allowNull: false
             }
         },
         {

diff --git a/server/models/loaders/AtLoader.js b/server/models/loaders/AtLoader.js
@@ -17,6 +17,21 @@ const AtLoader = () => {
 
             ats = await activePromise;
 
+            // Sort date of atVersions subarray in desc order by releasedAt date
+            ats.forEach(item =>
+                item.atVersions.sort((a, b) => b.releasedAt - a.releasedAt)
+            );
+
+            ats = ats.map(at => ({
+                ...at.dataValues,
+                candidateBrowsers: at.browsers.filter(
+                    browser => browser.AtBrowsers.isCandidate
+                ),
+                recommendedBrowsers: at.browsers.filter(
+                    browser => browser.AtBrowsers.isRecommended
+                )
+            }));
+
             return ats;
         }
     };

diff --git a/server/models/loaders/BrowserLoader.js b/server/models/loaders/BrowserLoader.js
@@ -18,6 +18,16 @@ const BrowserLoader = () => {
 
             browsers = await activePromise;
 
+            browsers = browsers.map(browser => ({
+                ...browser.dataValues,
+                candidateAts: browser.ats.filter(
+                    at => at.AtBrowsers.isCandidate
+                ),
+                recommendedAts: browser.ats.filter(
+                    at => at.AtBrowsers.isRecommended
+                )
+            }));
+
             return browsers;
         }
     };

diff --git a/server/resolvers/TestPlanVersionOperations/updatePhaseResolver.js b/server/resolvers/TestPlanVersionOperations/updatePhaseResolver.js
@@ -253,6 +253,45 @@ const updatePhaseResolver = async (
         throw new Error('No test plan reports found.');
     }
 
+    if (phase === 'CANDIDATE' || phase === 'RECOMMENDED') {
+        const reportsByAtAndBrowser = {};
+
+        testPlanReports.forEach(testPlanReport => {
+            const { at, browser } = testPlanReport;
+            if (!reportsByAtAndBrowser[at.id]) {
+                reportsByAtAndBrowser[at.id] = {};
+            }
+
+            reportsByAtAndBrowser[at.id][browser.id] = testPlanReport;
+        });
+
+        const ats = await context.atLoader.getAll();
+
+        const missingAtBrowserCombinations = [];
+
+        ats.forEach(at => {
+            const browsers =
+                phase === 'CANDIDATE'
+                    ? at.candidateBrowsers
+                    : at.recommendedBrowsers;
+            browsers.forEach(browser => {
+                if (!reportsByAtAndBrowser[at.id]?.[browser.id]) {
+                    missingAtBrowserCombinations.push(
+                        `${at.name} and ${browser.name}`
+                    );
+                }
+            });
+        });
+
+        if (missingAtBrowserCombinations.length) {
+            throw new Error(
+                `Cannot set phase to ${phase.toLowerCase()} because the following` +
+                    ` required reports have not been collected:` +
+                    ` ${missingAtBrowserCombinations.join(', ')}.`
+            );
+        }
+    }
+
     for (const testPlanReport of testPlanReports) {
         const runnableTests = runnableTestsResolver(testPlanReport);
         let updateParams = {};

diff --git a/server/resolvers/atsResolver.js b/server/resolvers/atsResolver.js
@@ -1,22 +1,5 @@
-const { getAts } = require('../models/services/AtService');
-
-const atsResolver = async () => {
-    const ats = await getAts(
-        undefined,
-        undefined,
-        undefined,
-        undefined,
-        undefined,
-        undefined,
-        {
-            order: [['name', 'asc']]
-        }
-    );
-    // Sort date of atVersions subarray in desc order by releasedAt date
-    ats.forEach(item =>
-        item.atVersions.sort((a, b) => b.releasedAt - a.releasedAt)
-    );
-    return ats;
+const atsResolver = async (_, __, context) => {
+    return context.atLoader.getAll();
 };
 
 module.exports = atsResolver;
diff --git a/server/resolvers/browsersResolver.js b/server/resolvers/browsersResolver.js
@@ -1,9 +1,5 @@
-const { getBrowsers } = require('../models/services/BrowserService');
-
-const browsersResolver = () => {
-    return getBrowsers(undefined, undefined, undefined, undefined, undefined, {
-        order: [['name', 'asc']]
-    });
+const browsersResolver = async (_, __, context) => {
+    return context.browserLoader.getAll();
 };
 
 module.exports = browsersResolver;
diff --git a/server/scripts/populate-test-data/index.js b/server/scripts/populate-test-data/index.js
@@ -102,6 +102,42 @@ const populateTestDatabase = async () => {
         'completeAndPassing'
     ]);
 
+    await populateFakeTestResults(8, [
+        'completeAndPassing',
+        'completeAndPassing',
+        'completeAndPassing'
+    ]);
+
+    await populateFakeTestResults(9, [
+        'completeAndPassing',
+        'completeAndPassing',
+        'completeAndPassing'
+    ]);
+
+    await populateFakeTestResults(10, [
+        'completeAndPassing',
+        'completeAndPassing',
+        'completeAndPassing'
+    ]);
+
+    await populateFakeTestResults(11, [
+        'completeAndPassing',
+        'completeAndPassing',
+        'completeAndPassing'
+    ]);
+
+    await populateFakeTestResults(12, [
+        'completeAndPassing',
+        'completeAndPassing',
+        'completeAndPassing'
+    ]);
+
+    await populateFakeTestResults(13, [
+        'completeAndPassing',
+        'completeAndPassing',
+        'completeAndPassing'
+    ]);
+
     console.info(
         'Successfully populated. Please wait a moment for the process to close.'
     );

diff --git a/server/scripts/populate-test-data/pg_dump_2021_05_test_data.sql b/server/scripts/populate-test-data/pg_dump_2021_05_test_data.sql
@@ -71,10 +71,16 @@ INSERT INTO "AtMode" ("atId", name) VALUES (3, 'INTERACTION');
 INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId") VALUES (1, 'DRAFT', get_test_plan_version_id(text 'Toggle Button'), '2021-05-14 14:18:23.602-05', 1, 2);
 INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId") VALUES (2, 'DRAFT', get_test_plan_version_id(text 'Select Only Combobox Example'), '2021-05-14 14:18:23.602-05', 2, 1);
 INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (3, 'CANDIDATE', get_test_plan_version_id(text 'Modal Dialog Example'), '2021-05-14 14:18:23.602-05', 1, 2, 'READY');
-INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (4, 'CANDIDATE', get_test_plan_version_id(text 'Modal Dialog Example'), '2021-05-14 14:18:23.602-05', 2, 1, 'READY');
+INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (4, 'CANDIDATE', get_test_plan_version_id(text 'Modal Dialog Example'), '2021-05-14 14:18:23.602-05', 2, 2, 'READY');
 INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (5, 'CANDIDATE', get_test_plan_version_id(text 'Modal Dialog Example'), '2021-05-14 14:18:23.602-05', 3, 3, 'READY');
 INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (6, 'CANDIDATE', get_test_plan_version_id(text 'Checkbox Example (Mixed-State)'), '2021-05-14 14:18:23.602-05', 3, 3, 'READY');
 INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId") VALUES (7, 'DRAFT', get_test_plan_version_id(text 'Alert Example'), '2021-05-14 14:18:23.602-05', 3, 1);
+INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (8, 'CANDIDATE', get_test_plan_version_id(text 'Modal Dialog Example'), '2021-05-14 14:18:23.602-05', 1, 1, 'READY');
+INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (9, 'CANDIDATE', get_test_plan_version_id(text 'Modal Dialog Example'), '2021-05-14 14:18:23.602-05', 2, 1, 'READY');
+INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (10, 'CANDIDATE', get_test_plan_version_id(text 'Modal Dialog Example'), '2021-05-14 14:18:23.602-05', 3, 1, 'READY');
+INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (11, 'CANDIDATE', get_test_plan_version_id(text 'Modal Dialog Example'), '2021-05-14 14:18:23.602-05', 3, 2, 'READY');
+INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (12, 'CANDIDATE', get_test_plan_version_id(text 'Checkbox Example (Mixed-State)'), '2021-05-14 14:18:23.602-05', 1, 2, 'READY');
+INSERT INTO "TestPlanReport" (id, "status", "testPlanVersionId", "createdAt", "atId", "browserId", "vendorReviewStatus") VALUES (13, 'CANDIDATE', get_test_plan_version_id(text 'Checkbox Example (Mixed-State)'), '2021-05-14 14:18:23.602-05', 2, 2, 'READY');
 
 --
 -- Data for Name: TestPlanVersion; Type: TABLE DATA; Schema: public; Owner: atr
@@ -111,6 +117,12 @@ INSERT INTO "TestPlanRun" (id, "testerUserId", "testPlanReportId", "testResults"
 INSERT INTO "TestPlanRun" (id, "testerUserId", "testPlanReportId", "testResults") VALUES (5, 1, 4, '[]');
 INSERT INTO "TestPlanRun" (id, "testerUserId", "testPlanReportId", "testResults") VALUES (6, 1, 5, '[]');
 INSERT INTO "TestPlanRun" (id, "testerUserId", "testPlanReportId", "testResults") VALUES (7, 2, 6, '[]');
+INSERT INTO "TestPlanRun" (id, "testerUserId", "testPlanReportId", "testResults") VALUES (8, 1, 8, '[]');
+INSERT INTO "TestPlanRun" (id, "testerUserId", "testPlanReportId", "testResults") VALUES (9, 1, 9, '[]');
+INSERT INTO "TestPlanRun" (id, "testerUserId", "testPlanReportId", "testResults") VALUES (10, 1, 10, '[]');
+INSERT INTO "TestPlanRun" (id, "testerUserId", "testPlanReportId", "testResults") VALUES (11, 1, 11, '[]');
+INSERT INTO "TestPlanRun" (id, "testerUserId", "testPlanReportId", "testResults") VALUES (12, 1, 12, '[]');
+INSERT INTO "TestPlanRun" (id, "testerUserId", "testPlanReportId", "testResults") VALUES (13, 1, 13, '[]');
 
 --
 -- Name: At_id_seq; Type: SEQUENCE SET; Schema: public; Owner: atr

diff --git a/server/seeders/20230622202911-addAtBrowser.js b/server/seeders/20230622202911-addAtBrowser.js
@@ -9,16 +9,18 @@ module.exports = {
         const firefoxBrowserId = 1;
         const chromeBrowserId = 2;
         const safariBrowserId = 3;
+
         return queryInterface.bulkInsert(
             'AtBrowsers',
+            // prettier-ignore
             [
-                { atId: jawsAtId, browserId: firefoxBrowserId },
-                { atId: jawsAtId, browserId: chromeBrowserId },
-                { atId: nvdaAtId, browserId: firefoxBrowserId },
-                { atId: nvdaAtId, browserId: chromeBrowserId },
-                { atId: voiceOverAtId, browserId: safariBrowserId },
-                { atId: voiceOverAtId, browserId: firefoxBrowserId },
-                { atId: voiceOverAtId, browserId: chromeBrowserId }
+                { atId: jawsAtId, browserId: firefoxBrowserId, isCandidate: false, isRecommended: true },
+                { atId: jawsAtId, browserId: chromeBrowserId, isCandidate: true, isRecommended: true },
+                { atId: nvdaAtId, browserId: firefoxBrowserId, isCandidate: false, isRecommended: true },
+                { atId: nvdaAtId, browserId: chromeBrowserId, isCandidate: true, isRecommended: true },
+                { atId: voiceOverAtId, browserId: safariBrowserId, isCandidate: true, isRecommended: true },
+                { atId: voiceOverAtId, browserId: firefoxBrowserId, isCandidate: false, isRecommended: false },
+                { atId: voiceOverAtId, browserId: chromeBrowserId, isCandidate: false, isRecommended: true }
             ],
             {}
         );

diff --git a/server/tests/integration/embed.test.js b/server/tests/integration/embed.test.js
@@ -28,12 +28,14 @@ describe('embed', () => {
         // Load the iframe, twice, one with a normal load and a second time from
         // the cache
         const initialLoadTimeStart = Number(new Date());
-        const res = await sessionAgent.get('/embed/reports/modal-dialog');
+        const res = await sessionAgent.get('/embed/reports/checkbox-tri-state');
         const initialLoadTimeEnd = Number(new Date());
         const initialLoadTime = initialLoadTimeEnd - initialLoadTimeStart;
 
         const cachedTimeStart = Number(new Date());
-        const res2 = await sessionAgent.get('/embed/reports/modal-dialog');
+        const res2 = await sessionAgent.get(
+            '/embed/reports/checkbox-tri-state'
+        );
         const cachedTimeEnd = Number(new Date());
         const cachedTime = cachedTimeEnd - cachedTimeStart;
 
@@ -43,10 +45,11 @@ describe('embed', () => {
         const nonWarning = screen.queryByText('Recommended Report');
         const warning = screen.queryByText('Warning! Unapproved Report');
         const unsupportedAtBrowserCombination =
-            screen.getAllByText('Not Applicable');
-        const futureSupportedAtBrowserCombination = screen.getAllByText(
+            screen.queryAllByText('Not Applicable');
+        const futureSupportedAtBrowserCombination = screen.queryAllByText(
             'Data Not Yet Available'
         );
+
         const nonWarningContents = screen.queryByText(
             'The information in this report is generated from candidate tests',
             { exact: false }
@@ -70,8 +73,8 @@ describe('embed', () => {
         expect(initialLoadTime / 10).toBeGreaterThan(cachedTime);
         expect(nonWarning || warning).toBeTruthy();
         expect(nonWarningContents || warningContents).toBeTruthy();
-        expect(unsupportedAtBrowserCombination).toBeTruthy();
-        expect(futureSupportedAtBrowserCombination).toBeTruthy();
+        expect(unsupportedAtBrowserCombination.length).not.toBe(0);
+        expect(futureSupportedAtBrowserCombination.length).not.toBe(0);
         expect(viewReportButton).toBeTruthy();
         expect(viewReportButtonOnClick).toMatch(
             // Onclick should be like the following:
@@ -80,7 +83,7 @@ describe('embed', () => {
         );
         expect(copyEmbedButton).toBeTruthy();
         expect(copyEmbedButtonOnClick).toMatch(
-            /announceCopied\('https?:\/\/[\w.:]+\/embed\/reports\/modal-dialog'\)/
+            /announceCopied\('https?:\/\/[\w.:]+\/embed\/reports\/checkbox-tri-state'\)/
         );
         expect(cellWithData).toBeTruthy();
     });