Fix #1273: Add tests for hints and solutions #1276
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
empty test suite.
… training controller
…ler-interface Conflicts: domain/src/main/java/org/oppia/domain/question/QuestionAssessmentProgressController.kt domain/src/main/java/org/oppia/domain/question/QuestionTrainingController.kt domain/src/test/java/org/oppia/domain/question/QuestionTrainingControllerTest.kt model/src/main/proto/question.proto
This includes some basic refactoring of internal structures used by the exploration progress controller to share common functionality between the two progress controllers. There's still some duplication, but this seems like a reasonable split since there's likely to be further differences in the progress controllers in the future. The question assessment progress controller tests pass, but no new ones have yet been added to thoroughly test the implementation.
ExplorationProgressController. They haven't yet been verified as correct.
…terface Conflicts: domain/src/main/assets/sample_questions.json domain/src/main/java/org/oppia/domain/question/QuestionAssessmentProgressController.kt domain/src/main/java/org/oppia/domain/question/QuestionTrainingController.kt domain/src/main/java/org/oppia/domain/util/JsonAssetRetriever.kt domain/src/test/java/org/oppia/domain/question/QuestionTrainingControllerTest.kt
BenHenning
changed the title
|
Thanks @veena14cs! FYI everyone else: I'm going to go ahead and merge this. Please follow up with me if you have any concerns. |
|
NB: CircleCI was passing on this branch, see:
I'm going to update with develop & re-run the GitHub actions tests per requirements, but I may force the PR submission given that past CircleCI tests are indeed passing. |
aggarwalpulkit596
approved these changes
Jun 26, 2020
This was referenced Jun 28, 2020
|
Reminder to self to update this with the latest develop. |
Conflicts: app/src/main/java/org/oppia/app/player/state/testing/StateFragmentTestActivity.kt
|
This is now up-to-date with the latest develop, and the tests have been updated to pass again with recent hints & solutions changes. |
BenHenning
added a commit
that referenced
this pull request
Aug 12, 2020
This piggybacks off of the solution introduced in #1276 for Robolectric. That PR allows Robolectric tests in app module to share dependencies with production components by creating a test application & telling Robolectric to use it instead of OppiaApplication via a @config annotation. This PR achieves the same thing by using a custom test runner that reads the same annotation and forces Espresso & instrumentation to use the test application instead of the default OppiaApplication. This setup will be easier once #59 is finished since we can specify the application in per-test manifests that both Robolectric & Espresso will respect. Note that while this lets the same test coroutine dispatchers work in both production & test code for Espresso, some additional work was needed to ensure the coroutines behave correctly. In particular, the coroutines use a fake system clock in Robolectric that requires explicit synchronization points in the tests to allow the clock to move forward & properly coordinate execution between background & main thread operations. However, in Espresso, since everything is using a real clock an idling resource is the preferred way to synchronize execution: it allows the background coroutines to operate in real-time much like they would in production, and then notify Espresso when completed. The test dispatchers API is now setup to support both synchronization mechanisms for both Robolectric & Espresso (the idling resource does nothing on Robolectric and the force synchronization effectively does nothing on Espresso). The first test being demonstrated as now stable is SplashActivityTest (as part of downstream work in #1397.
BenHenning
added a commit
that referenced
this pull request
Aug 12, 2020
* Introduce test coroutine dispatchers support in Espresso. This piggybacks off of the solution introduced in #1276 for Robolectric. That PR allows Robolectric tests in app module to share dependencies with production components by creating a test application & telling Robolectric to use it instead of OppiaApplication via a @config annotation. This PR achieves the same thing by using a custom test runner that reads the same annotation and forces Espresso & instrumentation to use the test application instead of the default OppiaApplication. This setup will be easier once #59 is finished since we can specify the application in per-test manifests that both Robolectric & Espresso will respect. Note that while this lets the same test coroutine dispatchers work in both production & test code for Espresso, some additional work was needed to ensure the coroutines behave correctly. In particular, the coroutines use a fake system clock in Robolectric that requires explicit synchronization points in the tests to allow the clock to move forward & properly coordinate execution between background & main thread operations. However, in Espresso, since everything is using a real clock an idling resource is the preferred way to synchronize execution: it allows the background coroutines to operate in real-time much like they would in production, and then notify Espresso when completed. The test dispatchers API is now setup to support both synchronization mechanisms for both Robolectric & Espresso (the idling resource does nothing on Robolectric and the force synchronization effectively does nothing on Espresso). The first test being demonstrated as now stable is SplashActivityTest (as part of downstream work in #1397. * Revert "Fixes #941: Add radar effect in Hints and solution (#1475)" This reverts commit 41eb10b. * Add thorough documentation for new dispatchers. * Clean up comments & add additional documentation. * Fix lint errors. * Fix broken test after changes to FakeSystemClock.
BenHenning
added a commit
that referenced
this pull request
Aug 12, 2020
* Introduce test coroutine dispatchers support in Espresso. This piggybacks off of the solution introduced in #1276 for Robolectric. That PR allows Robolectric tests in app module to share dependencies with production components by creating a test application & telling Robolectric to use it instead of OppiaApplication via a @config annotation. This PR achieves the same thing by using a custom test runner that reads the same annotation and forces Espresso & instrumentation to use the test application instead of the default OppiaApplication. This setup will be easier once #59 is finished since we can specify the application in per-test manifests that both Robolectric & Espresso will respect. Note that while this lets the same test coroutine dispatchers work in both production & test code for Espresso, some additional work was needed to ensure the coroutines behave correctly. In particular, the coroutines use a fake system clock in Robolectric that requires explicit synchronization points in the tests to allow the clock to move forward & properly coordinate execution between background & main thread operations. However, in Espresso, since everything is using a real clock an idling resource is the preferred way to synchronize execution: it allows the background coroutines to operate in real-time much like they would in production, and then notify Espresso when completed. The test dispatchers API is now setup to support both synchronization mechanisms for both Robolectric & Espresso (the idling resource does nothing on Robolectric and the force synchronization effectively does nothing on Espresso). The first test being demonstrated as now stable is SplashActivityTest (as part of downstream work in #1397. * Revert "Fixes #941: Add radar effect in Hints and solution (#1475)" This reverts commit 41eb10b. * Add thorough documentation for new dispatchers. * Broadly clean up test dispatchers. TestCoroutineDispatchers have been used in a few heavy Robolectric tests for a couple months now, without issue. This change broadly migrates all test dispatchers over to TestCoroutineDispatchers and TestDispatcherModule, resulting in significant and broad cleanups. This change also includes broadly removing now-unnecessary uses of various Kotlin coroutine annotations, and runBlockingTest() since TestCoroutineDispatchers replaces this functionality & does it more correctly. Some things to note: 1. This change also includes moving ProfileTestHelper to the testing module since it's been changed to rely on TestCoroutineDispatcher (which improves downstream test stability in the app module). 2. MainThreadExecutor was removed as the test disaptcher in StateFragmentTest and HomeActivityTest which may cause failures in these tests. Since neither test is part of the non-flaky test run, this is considered fine here. 3. Notice that runCurrent() has been moved around or added in a bunch of places--this is to be expected. TestCoroutineDispatchers always operates in a paused state, whereas before all test dispatchers were generally operating in a running state. These additional calls are needed to synchronize access. In some special cases, an additional coroutine dispatcher is being used to demonstrate blocking functionality without interfering with TestCoroutineDispatcher functionality. Also, note that missing runCurrent() in app module tests that depend on waitFor() can result in long failures (30 seconds for the timeout), so long-term tests should opt to avoid using waitFor() since TestCoroutineDispatchers also has an idling resource for Espresso use cases. * Update all tests to use paused looper with Robolectric. This is the recommended mode to run all Robolectric tests, and it interoperates better with TestCoroutineDispatchers. Note that this commit also includes some functional changes to AudioPlayerControllerTest. Before, the audio track being played was 1 second long, and 1 second needs to be waited before audio is detected as playing. These race against each otehr in paused looper mode, so extending the track duration to 2 seconds fixes the situation. I also made some minor cleanups elsewhere in the test. * Add testing package to CI & ktlint. * Clean up comments & add additional documentation. * Fix lint errors. * Fix broken test after changes to FakeSystemClock. * Fix linter failures. * Fix broken BindableAdapterTest post update with develop. The adapter test requires synchronization when running with a paused looper mode to make sure state is propagated when desired. * Clean up new test post-merge.
BenHenning
added a commit
that referenced
this pull request
Aug 17, 2020
* First phase of introducing automatic app expiration after a fixed amount of time (as controlled by a manifest-defined constant) This approach is based off of the existing onboarding flow, except it refactors that to be a bit more general. Rather than it focusing on whether the user has completed the onboarding experience, it focuses on what the user's app opening experience should be: onboarding, profile chooser, or app expiration dialog. * Post-merge fixes & update AppStartupStateControllerTest to work with the new expiration date check. * Add additional flag for turning auto expiration on/off rather than just using the date value. * Add tests for AppStartupStateControllerTest, and prepare for stabilizing SplashActivityTest. * Post-merge fixes. * Introduce test coroutine dispatchers support in Espresso. This piggybacks off of the solution introduced in #1276 for Robolectric. That PR allows Robolectric tests in app module to share dependencies with production components by creating a test application & telling Robolectric to use it instead of OppiaApplication via a @config annotation. This PR achieves the same thing by using a custom test runner that reads the same annotation and forces Espresso & instrumentation to use the test application instead of the default OppiaApplication. This setup will be easier once #59 is finished since we can specify the application in per-test manifests that both Robolectric & Espresso will respect. Note that while this lets the same test coroutine dispatchers work in both production & test code for Espresso, some additional work was needed to ensure the coroutines behave correctly. In particular, the coroutines use a fake system clock in Robolectric that requires explicit synchronization points in the tests to allow the clock to move forward & properly coordinate execution between background & main thread operations. However, in Espresso, since everything is using a real clock an idling resource is the preferred way to synchronize execution: it allows the background coroutines to operate in real-time much like they would in production, and then notify Espresso when completed. The test dispatchers API is now setup to support both synchronization mechanisms for both Robolectric & Espresso (the idling resource does nothing on Robolectric and the force synchronization effectively does nothing on Espresso). The first test being demonstrated as now stable is SplashActivityTest (as part of downstream work in #1397. * Revert "Fixes #941: Add radar effect in Hints and solution (#1475)" This reverts commit 41eb10b. * Add thorough documentation for new dispatchers. * Clean up comments & add additional documentation. * Fix lint errors. * Fix broken test after changes to FakeSystemClock. * Add splash tests to CI. * Add tests for new SplashActivity behavior. Includes adding an abstraction for metadata checking. Also, clean up the new dialog fragment by moving hardcoded strings to strings.xml & add missing documentation. * Fix broken AppStartupStateControllerTest. * Lint fixes. * Fix one missed conflict. * Post-merge fix.
BenHenning
added a commit
that referenced
this pull request
Sep 3, 2020
) * Introduce test coroutine dispatchers support in Espresso. This piggybacks off of the solution introduced in #1276 for Robolectric. That PR allows Robolectric tests in app module to share dependencies with production components by creating a test application & telling Robolectric to use it instead of OppiaApplication via a @config annotation. This PR achieves the same thing by using a custom test runner that reads the same annotation and forces Espresso & instrumentation to use the test application instead of the default OppiaApplication. This setup will be easier once #59 is finished since we can specify the application in per-test manifests that both Robolectric & Espresso will respect. Note that while this lets the same test coroutine dispatchers work in both production & test code for Espresso, some additional work was needed to ensure the coroutines behave correctly. In particular, the coroutines use a fake system clock in Robolectric that requires explicit synchronization points in the tests to allow the clock to move forward & properly coordinate execution between background & main thread operations. However, in Espresso, since everything is using a real clock an idling resource is the preferred way to synchronize execution: it allows the background coroutines to operate in real-time much like they would in production, and then notify Espresso when completed. The test dispatchers API is now setup to support both synchronization mechanisms for both Robolectric & Espresso (the idling resource does nothing on Robolectric and the force synchronization effectively does nothing on Espresso). The first test being demonstrated as now stable is SplashActivityTest (as part of downstream work in #1397. * Revert "Fixes #941: Add radar effect in Hints and solution (#1475)" This reverts commit 41eb10b. * Stabilize StateFragmentTest such that it passes on both Robolectric and Espresso. Note that some issues were found during this: #1612 (#1611 was found a few weeks ago, but it also affects these tests). To ensure the tests can still be run, a @runon annotation was added to allow tests to target specific test platforms. The tests that currently fail on Robolectric due to #1611 and #1612 are disabled for that platform. The test suite as a whole has been verified to pass in its current state on both Robolectric and Espresso (on a Pixel XL). The aim of this PR is to actually enable critical state fragment tests in CI, so both StateFragmentTest and StateFragmentLocalTest are being enabled in GitHub actions. * Enable StateFragmentTest (Robolectric) & StateFragmentLocalTest for CI. * Add thorough documentation for new dispatchers. * Clean up comments & add additional documentation. * Fix lint errors. * Fix broken test after changes to FakeSystemClock. * Fix linter errors. * Use a custom executor service for Glide requests that coordinates with Oppia's test dispatchers. Note that this does not actually introduce the service--that will happen in a new branch. * Introduce new executor service which allows interop with Kotlin coroutines, plus a test to verify that it fundamentally follows one interpretation of ExecutorService's API. * Fix flaky timeout tests by improving cancellation cooperation for invokeAny() and provide longer timeouts for tests that are CPU-sensitive. * Add documentation & clean up unused code. * Lint fixes. * Significantly reorganize invokeAll() to try and make it more cooperative for cancellation, and increase timeout times in tests to reduce flakiness for time-sensitive tests. Some tests are remaining flaky, so ignoring those. Re-add maybeWithTimeoutOrNull since it actually was needed. * Lint fixes. * Post-merge module fixes. * Post-merge fixes with ratio input & add a TODO to improve speed of the new coroutine executor service. * Revert "Fixes part of #40 & #42: Generalisation Highfi Mobile Portrait + Landscape - Buttons (#1653)" This reverts commit 1bb1ffa. * Ensure terminated tasks do not interfere with one another (timeouts should happen individually for each task during termination). This fixes a failure observed in StateFragmentLocalTest in #1630. * Ignore failing tests until #1769 is resolved. * Fix awaitTermination & improve test. Improve stack trace for test dispatcher timeouts. * Fix slow & broken tests in Robolectric for StateFragmentLocalTest. * Add missing deps for StateFragmentLocalTest. * Address reviewer comments.
BenHenning
added a commit
that referenced
this pull request
Sep 3, 2020
* Add support for showing concept cards in feedback, and add a concept card as one of the remediation pathways for 'the meaning of equal parts' lesson. * Introduce test coroutine dispatchers support in Espresso. This piggybacks off of the solution introduced in #1276 for Robolectric. That PR allows Robolectric tests in app module to share dependencies with production components by creating a test application & telling Robolectric to use it instead of OppiaApplication via a @config annotation. This PR achieves the same thing by using a custom test runner that reads the same annotation and forces Espresso & instrumentation to use the test application instead of the default OppiaApplication. This setup will be easier once #59 is finished since we can specify the application in per-test manifests that both Robolectric & Espresso will respect. Note that while this lets the same test coroutine dispatchers work in both production & test code for Espresso, some additional work was needed to ensure the coroutines behave correctly. In particular, the coroutines use a fake system clock in Robolectric that requires explicit synchronization points in the tests to allow the clock to move forward & properly coordinate execution between background & main thread operations. However, in Espresso, since everything is using a real clock an idling resource is the preferred way to synchronize execution: it allows the background coroutines to operate in real-time much like they would in production, and then notify Espresso when completed. The test dispatchers API is now setup to support both synchronization mechanisms for both Robolectric & Espresso (the idling resource does nothing on Robolectric and the force synchronization effectively does nothing on Espresso). The first test being demonstrated as now stable is SplashActivityTest (as part of downstream work in #1397. * Revert "Fixes #941: Add radar effect in Hints and solution (#1475)" This reverts commit 41eb10b. * Stabilize StateFragmentTest such that it passes on both Robolectric and Espresso. Note that some issues were found during this: #1612 (#1611 was found a few weeks ago, but it also affects these tests). To ensure the tests can still be run, a @runon annotation was added to allow tests to target specific test platforms. The tests that currently fail on Robolectric due to #1611 and #1612 are disabled for that platform. The test suite as a whole has been verified to pass in its current state on both Robolectric and Espresso (on a Pixel XL). The aim of this PR is to actually enable critical state fragment tests in CI, so both StateFragmentTest and StateFragmentLocalTest are being enabled in GitHub actions. * Enable StateFragmentTest (Robolectric) & StateFragmentLocalTest for CI. * Add thorough documentation for new dispatchers. * Clean up comments & add additional documentation. * Fix lint errors. * Fix broken test after changes to FakeSystemClock. * Fix linter errors. * Update test lesson to include references to concept cards. * Lint fixes & use HtmlCompat instead of Html. * Add support for the newer & finalized tag format. * Lint fixes. * Use a custom executor service for Glide requests that coordinates with Oppia's test dispatchers. Note that this does not actually introduce the service--that will happen in a new branch. * Introduce new executor service which allows interop with Kotlin coroutines, plus a test to verify that it fundamentally follows one interpretation of ExecutorService's API. * Fix flaky timeout tests by improving cancellation cooperation for invokeAny() and provide longer timeouts for tests that are CPU-sensitive. * Add documentation & clean up unused code. * Lint fixes. * Significantly reorganize invokeAll() to try and make it more cooperative for cancellation, and increase timeout times in tests to reduce flakiness for time-sensitive tests. Some tests are remaining flaky, so ignoring those. Re-add maybeWithTimeoutOrNull since it actually was needed. * Lint fixes. * Post-merge module fixes. * Post-merge fixes with ratio input & add a TODO to improve speed of the new coroutine executor service. * Revert "Fixes part of #40 & #42: Generalisation Highfi Mobile Portrait + Landscape - Buttons (#1653)" This reverts commit 1bb1ffa. * Ensure terminated tasks do not interfere with one another (timeouts should happen individually for each task during termination). This fixes a failure observed in StateFragmentLocalTest in #1630. * Ignore failing tests until #1769 is resolved. * Fix awaitTermination & improve test. Improve stack trace for test dispatcher timeouts. * Fix slow & broken tests in Robolectric for StateFragmentLocalTest. * Add missing deps for StateFragmentLocalTest. * Address TODOs (including adding support for list tags which replaces the old handler & adds nested custom tag support), and add tests. * Lint fixes. * Address reviewer comments. * Address review comments. Fix new concept card tests on Espresso & add landscape versions (required configuring hints to show quickly to avoid delaying the test, and fixing a bug in the espresso test dispatcher). Add support for disabling concept cards of they aren't enabled for parsing particular HTML (the default behavior is to ignore the custom tag). * Add support for concept cards in questions. Note that it's not clear how to test verifying that pressing the exit button closes the concept card since the exit button is part of the dialog's toolbar. * Fix image-breaking duplicated code in HtmlParser, fix a paragraph parsing issue in BulletTagHandler, and add tests for CustomHtmlContentHandler. * Lint fixes.
prayutsu
pushed a commit
to prayutsu/oppia-android
that referenced
this pull request
Sep 3, 2020
…#1764] (oppia#1630) * Introduce test coroutine dispatchers support in Espresso. This piggybacks off of the solution introduced in oppia#1276 for Robolectric. That PR allows Robolectric tests in app module to share dependencies with production components by creating a test application & telling Robolectric to use it instead of OppiaApplication via a @config annotation. This PR achieves the same thing by using a custom test runner that reads the same annotation and forces Espresso & instrumentation to use the test application instead of the default OppiaApplication. This setup will be easier once oppia#59 is finished since we can specify the application in per-test manifests that both Robolectric & Espresso will respect. Note that while this lets the same test coroutine dispatchers work in both production & test code for Espresso, some additional work was needed to ensure the coroutines behave correctly. In particular, the coroutines use a fake system clock in Robolectric that requires explicit synchronization points in the tests to allow the clock to move forward & properly coordinate execution between background & main thread operations. However, in Espresso, since everything is using a real clock an idling resource is the preferred way to synchronize execution: it allows the background coroutines to operate in real-time much like they would in production, and then notify Espresso when completed. The test dispatchers API is now setup to support both synchronization mechanisms for both Robolectric & Espresso (the idling resource does nothing on Robolectric and the force synchronization effectively does nothing on Espresso). The first test being demonstrated as now stable is SplashActivityTest (as part of downstream work in oppia#1397. * Revert "Fixes oppia#941: Add radar effect in Hints and solution (oppia#1475)" This reverts commit 41eb10b. * Stabilize StateFragmentTest such that it passes on both Robolectric and Espresso. Note that some issues were found during this: oppia#1612 (oppia#1611 was found a few weeks ago, but it also affects these tests). To ensure the tests can still be run, a @runon annotation was added to allow tests to target specific test platforms. The tests that currently fail on Robolectric due to oppia#1611 and oppia#1612 are disabled for that platform. The test suite as a whole has been verified to pass in its current state on both Robolectric and Espresso (on a Pixel XL). The aim of this PR is to actually enable critical state fragment tests in CI, so both StateFragmentTest and StateFragmentLocalTest are being enabled in GitHub actions. * Enable StateFragmentTest (Robolectric) & StateFragmentLocalTest for CI. * Add thorough documentation for new dispatchers. * Clean up comments & add additional documentation. * Fix lint errors. * Fix broken test after changes to FakeSystemClock. * Fix linter errors. * Use a custom executor service for Glide requests that coordinates with Oppia's test dispatchers. Note that this does not actually introduce the service--that will happen in a new branch. * Introduce new executor service which allows interop with Kotlin coroutines, plus a test to verify that it fundamentally follows one interpretation of ExecutorService's API. * Fix flaky timeout tests by improving cancellation cooperation for invokeAny() and provide longer timeouts for tests that are CPU-sensitive. * Add documentation & clean up unused code. * Lint fixes. * Significantly reorganize invokeAll() to try and make it more cooperative for cancellation, and increase timeout times in tests to reduce flakiness for time-sensitive tests. Some tests are remaining flaky, so ignoring those. Re-add maybeWithTimeoutOrNull since it actually was needed. * Lint fixes. * Post-merge module fixes. * Post-merge fixes with ratio input & add a TODO to improve speed of the new coroutine executor service. * Revert "Fixes part of oppia#40 & oppia#42: Generalisation Highfi Mobile Portrait + Landscape - Buttons (oppia#1653)" This reverts commit 1bb1ffa. * Ensure terminated tasks do not interfere with one another (timeouts should happen individually for each task during termination). This fixes a failure observed in StateFragmentLocalTest in oppia#1630. * Ignore failing tests until oppia#1769 is resolved. * Fix awaitTermination & improve test. Improve stack trace for test dispatcher timeouts. * Fix slow & broken tests in Robolectric for StateFragmentLocalTest. * Add missing deps for StateFragmentLocalTest. * Address reviewer comments.
prayutsu
pushed a commit
to prayutsu/oppia-android
that referenced
this pull request
Sep 3, 2020
* Add support for showing concept cards in feedback, and add a concept card as one of the remediation pathways for 'the meaning of equal parts' lesson. * Introduce test coroutine dispatchers support in Espresso. This piggybacks off of the solution introduced in oppia#1276 for Robolectric. That PR allows Robolectric tests in app module to share dependencies with production components by creating a test application & telling Robolectric to use it instead of OppiaApplication via a @config annotation. This PR achieves the same thing by using a custom test runner that reads the same annotation and forces Espresso & instrumentation to use the test application instead of the default OppiaApplication. This setup will be easier once oppia#59 is finished since we can specify the application in per-test manifests that both Robolectric & Espresso will respect. Note that while this lets the same test coroutine dispatchers work in both production & test code for Espresso, some additional work was needed to ensure the coroutines behave correctly. In particular, the coroutines use a fake system clock in Robolectric that requires explicit synchronization points in the tests to allow the clock to move forward & properly coordinate execution between background & main thread operations. However, in Espresso, since everything is using a real clock an idling resource is the preferred way to synchronize execution: it allows the background coroutines to operate in real-time much like they would in production, and then notify Espresso when completed. The test dispatchers API is now setup to support both synchronization mechanisms for both Robolectric & Espresso (the idling resource does nothing on Robolectric and the force synchronization effectively does nothing on Espresso). The first test being demonstrated as now stable is SplashActivityTest (as part of downstream work in oppia#1397. * Revert "Fixes oppia#941: Add radar effect in Hints and solution (oppia#1475)" This reverts commit 41eb10b. * Stabilize StateFragmentTest such that it passes on both Robolectric and Espresso. Note that some issues were found during this: oppia#1612 (oppia#1611 was found a few weeks ago, but it also affects these tests). To ensure the tests can still be run, a @runon annotation was added to allow tests to target specific test platforms. The tests that currently fail on Robolectric due to oppia#1611 and oppia#1612 are disabled for that platform. The test suite as a whole has been verified to pass in its current state on both Robolectric and Espresso (on a Pixel XL). The aim of this PR is to actually enable critical state fragment tests in CI, so both StateFragmentTest and StateFragmentLocalTest are being enabled in GitHub actions. * Enable StateFragmentTest (Robolectric) & StateFragmentLocalTest for CI. * Add thorough documentation for new dispatchers. * Clean up comments & add additional documentation. * Fix lint errors. * Fix broken test after changes to FakeSystemClock. * Fix linter errors. * Update test lesson to include references to concept cards. * Lint fixes & use HtmlCompat instead of Html. * Add support for the newer & finalized tag format. * Lint fixes. * Use a custom executor service for Glide requests that coordinates with Oppia's test dispatchers. Note that this does not actually introduce the service--that will happen in a new branch. * Introduce new executor service which allows interop with Kotlin coroutines, plus a test to verify that it fundamentally follows one interpretation of ExecutorService's API. * Fix flaky timeout tests by improving cancellation cooperation for invokeAny() and provide longer timeouts for tests that are CPU-sensitive. * Add documentation & clean up unused code. * Lint fixes. * Significantly reorganize invokeAll() to try and make it more cooperative for cancellation, and increase timeout times in tests to reduce flakiness for time-sensitive tests. Some tests are remaining flaky, so ignoring those. Re-add maybeWithTimeoutOrNull since it actually was needed. * Lint fixes. * Post-merge module fixes. * Post-merge fixes with ratio input & add a TODO to improve speed of the new coroutine executor service. * Revert "Fixes part of oppia#40 & oppia#42: Generalisation Highfi Mobile Portrait + Landscape - Buttons (oppia#1653)" This reverts commit 1bb1ffa. * Ensure terminated tasks do not interfere with one another (timeouts should happen individually for each task during termination). This fixes a failure observed in StateFragmentLocalTest in oppia#1630. * Ignore failing tests until oppia#1769 is resolved. * Fix awaitTermination & improve test. Improve stack trace for test dispatcher timeouts. * Fix slow & broken tests in Robolectric for StateFragmentLocalTest. * Add missing deps for StateFragmentLocalTest. * Address TODOs (including adding support for list tags which replaces the old handler & adds nested custom tag support), and add tests. * Lint fixes. * Address reviewer comments. * Address review comments. Fix new concept card tests on Espresso & add landscape versions (required configuring hints to show quickly to avoid delaying the test, and fixing a bug in the espresso test dispatcher). Add support for disabling concept cards of they aren't enabled for parsing particular HTML (the default behavior is to ignore the custom tag). * Add support for concept cards in questions. Note that it's not clear how to test verifying that pressing the exit button closes the concept card since the exit button is part of the dialog's toolbar. * Fix image-breaking duplicated code in HtmlParser, fix a paragraph parsing issue in BulletTagHandler, and add tests for CustomHtmlContentHandler. * Lint fixes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #1273.
This introduces tests for hints & solutions. Note that these are Robolectric-only tests because a lot of UI screens need to be quickly driven through, and we need to potentially wait long periods of time (so utilizing fake time is really helpful). Consider that locally, these tests seem to run in about ~40 seconds. Some of the tests are verifying scenarios where the user is waiting 60 seconds for a hint to show, and may go through multiple states with questions in order to reach a desired state. These tests heavily rely on the time-related fixes introduced in #1275.
This PR relies on one of the fractions explorations for driving different hints & solutions flows. To keep things simple, it actually drives through the exploration like a user. In the future, we may want to create dedicated explorations for the different scenarios being tested here.
Note that this PR also is the first test that utilizes proper test dependency replacement for UI components (support added in #1275). This is needed in order to switch the production coroutine dispatchers with test ones that can allow coordination with the fake clock. Subsequently, these tests are not compatible with Espresso which is why they aren't living under sharedTest.
Note also that this new test introduces a nice
StateViewHolderTypeMatcherfor quickly scrolling between the elements of the state & questions recycler views. We may want to utilize this in future tests.