Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finalize initial set of User Accounts tests #39

Merged
merged 2 commits into from
Sep 19, 2017
Merged

Finalize initial set of User Accounts tests #39

merged 2 commits into from
Sep 19, 2017

Conversation

seav
Copy link
Contributor

@seav seav commented Sep 12, 2017

Proposed changes in this pull request

This PR adds automated functional tests for most User Accounts test cases as described in the Functional Test Cases spreadsheet.

  • Bump the package version to 0.2.1.
  • All User Accounts test cases have tests except for the following:
    • R3, R4, U2, U3, U4, U7, U10, U11 – these require a way to check emails sent by the platform. We may be able to automate these in the future but not for now.
    • R4 and R5 – these require a wait time of 48 hours which is not feasible for an automated test.
  • The automated test for test cases U15 to U18 (corresponding to the user avatar cases) is marked with the pytest upload marker. Test cases that perform file upload cannot be performed on BrowserStack using our own supplied files. (See BrowserStack's docs.) The plan is to run this test with the local ChromeDriver on the Travis VM instead of using BrowserStack and the pytest marker allows us to target this test.
  • This PR contains a single generic user account test fixture. (See cadasta/test/fixtures/main.json.) This will be the test account used for the test cases where an existing user account is needed (R7, R10, all L*, and all U* except for U9).
  • The tests are designed as much as possible to return the database state back to before the test was run so that the test suite can be run against a server multiple times and with tests in any order. Therefore many tests perform a reversion step after the test script has been performed. For example, after testing that the user can update their username, the username is again updated back to the original value.
  • The test user account usernames have the prefix functest_ for easier identification in the database. Furthermore, temporary or throwaway user account usernames (such as those created by the user account creation tests) are prefixed with the prefix functest_tmp_. After running the test suite, deleting these temporary user accounts should result in the database only containing the original test fixtures.
  • The tests make good use of pytest fixture features. See the pytest docs for more information.

When should this PR be merged

Before Sprint 21 QA day. (So that we don't manually test these cases anymore.)

Risks

No risks foreseen. This PR should only affect QA and not the actual platform function.

Follow-up actions

Approve and merge the corresponding PR on the cadasta-platform repo. As much as possible, this should be done right after this PR is merged.

Checklist (for reviewing)

General

  • Is this PR explained thoroughly? All code changes must be accounted for in the PR description.
  • Is the PR labeled correctly? It should have the migration label if a new migration is added.
  • Is the risk level assessment sufficient? The risks section should contain all risks that might be introduced with the PR and which actions we need to take to mitigate these risks. Possible risks are database migrations, new libraries that need to be installed or changes to deployment scripts.

Functionality

  • Are all requirements met? Compare implemented functionality with the requirements specification.
  • Does the UI work as expected? There should be no Javascript errors in the console; all resources should load. There should be no unexpected errors. Deliberately try to break the feature to find out if there are corner cases that are not handled.

Code

  • Do you fully understand the introduced changes to the code? If not ask for clarification, it might uncover ways to solve a problem in a more elegant and efficient way.
  • Does the PR introduce any inefficient database requests? Use the debug server to check for duplicate requests.
  • Are all necessary strings marked for translation? All strings that are exposed to users via the UI must be marked for translation.

Tests

  • Are there sufficient test cases? Ensure that all components are tested individually; models, forms, and serializers should be tested in isolation even if a test for a view covers these components.
  • If this is a bug fix, are tests for the issue in place? There must be a test case for the bug to ensure the issue won’t regress. Make sure that the tests break without the new code to fix the issue.
  • If this is a new feature or a significant change to an existing feature? has the manual testing spreadsheet been updated with instructions for manual testing?

Security

  • Confirm this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets.
  • Are all UI and API inputs run through forms or serializers?
  • Are all external inputs validated and sanitized appropriately?
  • Does all branching logic have a default case?
  • Does this solution handle outliers and edge cases gracefully?
  • Are all external communications secured and restricted to SSL?

Documentation

  • Are changes to the UI documented in the platform docs? If this PR introduces new platform site functionality or changes existing ones, the changes must be documented in the Cadasta Platform Documentation.
  • Are changes to the API documented in the API docs? If this PR introduces new API functionality or changes existing ones, the changes must be documented in the API docs.
  • Are reusable components documented? If this PR introduces components that are relevant to other developers (for instance a mixin for a view or a generic form) they should be documented in the Wiki.

Copy link
Member

@oliverroick oliverroick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can this branch using the functest/accounts branch on cadasta-platform I'm getting selenium.common.exceptions.TimeoutException for the following tests:

  • ::TestLogin::test_user_can_login_by_email
  • ::TestUpdating::test_user_can_change_password
  • ::TestUpdating::test_user_cannot_change_password_without_current_password
  • ::TestUpdating::test_correct_user_info_is_shown
  • ::TestUpdating::test_user_cannot_update_profile_without_password
  • ::TestUpdating::test_user_can_update_username
  • ::TestUpdating::test_user_can_update_full_name
  • ::TestUpdating::test_user_can_update_preferred_language
  • ::TestUpdating::test_user_can_update_preferred_measurement_system
  • ::TestUpdating::test_user_avatar

Anything I'm missing?

@seav
Copy link
Contributor Author

seav commented Sep 18, 2017

I can only guess but I think the user account fixture is missing in your VM DB. Can you temporarily change the requirements/dev.txt file so that the cadasta-test repository points to the accounts branch instead of master? Note that the cadasta-platform functest/accounts branch assumes that this PR is already merged before it will successfully run as is. Before then we need to temporarily point to the accounts branch on the cadasta-test repo.

@oliverroick
Copy link
Member

Can you temporarily change the requirements/dev.txt file so that the cadasta-test repository points to the accounts branch instead of master?

That's what I did.

@seav
Copy link
Contributor Author

seav commented Sep 18, 2017

OK. Can you uninstall and then reinstall the cadasta-test package? I just pushed a commit a while ago to the cadasta-test accounts branch to add the now-existing created_date and last_updated fields to the user account fixture due to the now-merged PR 1705 in the cadasta-platform repo. Without this commit, the fixture will fail to load.

Copy link
Member

@oliverroick oliverroick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I reprovisioned a VM using cadasta-test/accounts and loaded the fixtures. The tests are now passing.

We definitely we documentation about how to run the functional tests locally because it's not exactly a smooth and straightforward process.

@seav
Copy link
Contributor Author

seav commented Sep 19, 2017

OK. I will take time to update the documentation next week.

@seav seav merged commit e5db4d0 into master Sep 19, 2017
@seav seav deleted the accounts branch September 19, 2017 12:14
seav pushed a commit that referenced this pull request Sep 20, 2017
@seav seav mentioned this pull request Sep 20, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants