Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Initial Quality Assurance guidelines/flow #94

Open
wants to merge 24 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
77f3097
Start of quality assurance section
Nov 1, 2018
cb1628b
Further developed QA Process
Nov 26, 2018
2592b3f
Start rewording manual testing section for initial push
Dec 11, 2018
9eee1ef
Automated testing copy
Dec 11, 2018
1b26f26
Merge pull request #1 from DivineOmega/feature/quality-assurance
jaredkove Dec 11, 2018
785a84e
/quality-assurance:
Dec 11, 2018
7556697
Merge branch 'master' into feature/quality-assurance
jaredkove Dec 11, 2018
055cd24
Update quality-assurance/core.md
ash123456789 Dec 11, 2018
cf43017
Update quality-assurance/core.md
ash123456789 Dec 11, 2018
ee86165
Update quality-assurance/core.md
dextermb Dec 11, 2018
3072e8e
Update quality-assurance/core.md
dextermb Dec 11, 2018
1de8120
Update quality-assurance/core.md
dextermb Dec 11, 2018
8830ee9
Update quality-assurance/core.md
dextermb Dec 11, 2018
c59d910
Update quality-assurance/core.md
dextermb Dec 11, 2018
c826658
Update quality-assurance/core.md
dextermb Dec 11, 2018
214d47c
Update quality-assurance/core.md
dextermb Dec 11, 2018
cf5355d
Update quality-assurance/core.md
dextermb Dec 11, 2018
a562f75
Remove class clause line
jaredkove Dec 11, 2018
437da6c
Update quality-assurance/core.md
DivineOmega Dec 11, 2018
7969224
Update quality-assurance/core.md
ash123456789 Dec 12, 2018
f6f8cf3
Update quality-assurance/core.md
DivineOmega Dec 12, 2018
ef37320
Removed duplication of Git Workflow
Dec 20, 2018
2b6e138
Merge branch 'master' into feature/quality-assurance
DivineOmega Aug 14, 2019
ae57017
Merge branch 'master' into feature/quality-assurance
DivineOmega Aug 15, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ We work with a number of different software platforms. Each has their own set of
* 🗂 [Databases](general/databases.md)
* 🛠 [DevOps](devops/devops.md)
* 🔢 [Style Guides](styleguides/styleguides.md)
* 🚦 [Quality Assurance](quality-assurance/core.md)
* 📖 [Open Source Projects](general/open-source.md)
* ✏️ Design
* [✍️ Design Tools and Resources](design/design-tools.md)
Expand Down
193 changes: 193 additions & 0 deletions quality-assurance/core.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
# 🚦 Langley Foxall Quality Assurance

When building applications that run an entire business end-to-end - it's important that we adhere to high quality
assurance standards in order to consistently deliver a high quality software project.

There are different parts/processes to quality assurance as a whole, and these are not limited to just manual testing and
automated testing.

* [What makes a good web application?](#what-makes-a-good-web-application)
* [Blackbox](#blackbox)
* [Whitebox](#whitebox)
* [Processes](#processes)
* [Automated Testing](#automated-testing)
* [Manual Testing](#manual-testing)

## What makes a good web application?
The following is a checklist of things to think about both during development of a product and during the quality
assurance phase. They should be considered guidelines, and not a strict set of requirements, as not all points will be applicable or appropriate for all applications.

### Blackbox
**General**
* All UI scales well and looks good on small form factor devices
* All errors are handled and UI reacts appropriately
* Errors are clear in meaning & reflect what happened
* Activity indicators are used when loading data asynchronously
* Empty information is handled correctly in UI (optional fields etc)
* Long running operations give a indication of progress if possible
* Breadcrumbs are used where appropriate

**Navbar**
* UI reflects the current active page
* An icon that is appropriate for the page (if applicable)

**Forms**
* Invalid inputs correctly trigger validation errors
* All form inputs have associated labels
* Appropriate validation for different inputs (phone number, postcode)
* The forms are not too long & and are grouped logically
* The user is alerted of the form lifecycle (success, error)
* When modals are used in the context of updates, ensure inputs are correctly pre populated.
* Autofill is disabled on inputs where appropriate
* Inputs have correct type property (emails have email type, phone numbers have tel type)

**Modals**
* Modals are animated on both display and dismiss
* Modals are used where appropriate over pages

**Formatting**
* Dates are formatted correctly for the appropriate audience.
* Monetary values are formatted correctly for the appropriate audience
* Phone numbers are formatted correctly for the appropriate audience

**Pagination**
* UI correctly indicates current page
* UI shows at least 2 page tabs ahead and below the current page

**Login & Auth**
* Users can only access pages they are permitted access to.
* Authentication when they are logged in
* Login, registration (if applicable) and logout works as expected.
* Reset password works as expected

**Registration**
* Email addresses are verified & unique (if applicable)
* Passwords meet minimum standards (min 8 chars)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never wrote this for explicitly Laravel - it's just a general web assurance guide to ensure that the projects we write for Web, irregardless of technology, all follow the same basic principles, the steps outlined can be applied to any web stack.

Copy link
Contributor

@DivineOmega DivineOmega Dec 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do something similar to the following.

Suggested change
* Passwords meet minimum standards (min 8 chars)
* Passwords meet minimum standards, as defined by [NIST Special Publication 800-63B section 5](https://pages.nist.gov/800-63-3/sp800-63b.html#sec5). If using Laravel, ensure the [Laravel NIST Password Rules](https://github.com/langleyfoxall/laravel-nist-password-rules) is integrated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm against blanket enforcing the NIST rules due to their complexities and it would cause a significant barrier to entry for social apps (registration processes) and uptake to those platforms.

For that reason I think a lower level as a minimum (8 Chars) is enough, but its preferable in our enterprise software to opt for NIST specification.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Passwords meet minimum standards (min 8 chars)
* For enterprise software, passwords should preferably meet standards defined by [NIST Special Publication 800-63B section 5](https://pages.nist.gov/800-63-3/sp800-63b.html#sec5). If using Laravel, ensure the [Laravel NIST Password Rules](https://github.com/langleyfoxall/laravel-nist-password-rules) is integrated.

@wheatleyjj New suggestion based on your feedback.

* Passwords should not have maximum length restrictions (allow at least 255 characters).

**Emails**
* Emails look good on small form factor devices
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth mentioning ensuring the HTML and CSS in the email works on the popular clients if custom HTML and CSS is being written. There are good resources such as this guide which lets you search by email clients.

* Ensure correct subject & sender name
* Correct email is sent, information is conveyed appropriately
* Emails are received in a timely manner
* Emails come from the appropriate email (often the clients)
* Links point to the correct URIs

**Push Notifications**
* Push notifications are received in a timely manner
* They show correct & relevant information

**Permissions Roles**
* Ensure permissions & roles match up to possible actions (only admins can create users etc)

### Whitebox
**General**
* Filenames must follow standard coding conventions (UserController.php)
* Classes must follow standard coding conventions (UserController)
* Variables should follow camelCase & be descriptive (myName)
* All long running operations (at your discretion) are queued
jaredkove marked this conversation as resolved.
Show resolved Hide resolved
* APIs follow REST architecture as closely as possible and implement our [API Response](https://github.com/langleyfoxall/helpers-laravel#apiresponse)
jaredkove marked this conversation as resolved.
Show resolved Hide resolved
* API responses hide sensitive information outside of what is absolutely necessary.
* Auth middleware is setup & working correctly
* Event system is used for method side effects (Audit trail etc)
* Laravel Notifications are used to notify the user of system events (subscription expired, new friend request etc)
* Sensitive values are either hashed or encrypted (when required by law or common sense)
* Third party API integrations that can & will be re-used across projects should be implemented as external packages,
assuming no existing open-source integration exists, to allow for independent testing and code reuse if appropriate.
* Ensure third party API keys (e.g. SES keys) are not present within .env.example and other relevant files outside of the
production / development environments.
* If Travis CI is being used, ensure third party API keys are input directly into Travis CI's website relevant to the
project and make sure "Display value in build log" is not checked.
* Log viewers are hidden behind a protected route

**Controllers**
* Be on the lookout for duplicated code - recommend refactoring into helper classes or models if so
* Ensure all requests are validated either through a validator or custom request
* Errors are handled correctly in try catch blocks
* Responses are returned in a consistent & logical format
* Only appropriate & relevant information is returned
* When returning JSON ensure the returned JSON conforms to the [JSON spec](http://json-schema.org/specification.html)
* Appropriate status codes are set
jaredkove marked this conversation as resolved.
Show resolved Hide resolved

**Emails**
* Emails use the SES driver unless agreed beforehand
* All emails are queued up for sending
jaredkove marked this conversation as resolved.
Show resolved Hide resolved
* Only relevant information is passed to the mailable instance

**Views**
* Loading of assets (JS/HTML) use the `asset` global helper
* Blade layout system is used where appropriate (@extends, @yield)
* Components are split out logically where appropriate, make use of Laravel's powerful blade templating system if
appropriate (clientCard, navbar etc)

**Service Providers**
* Service Providers are utilised where appropriate to bind classes into the container (API interfaces etc)

**File Storage**
* Files are stored on S3 unless requested by the client to be stored locally or agreed beforehand
* Ensure appropriate permissions on files in applicable (sensitive documents are private, avatars are public etc)

**Queues**
* Queues use Redis unless agreed beforehand
* Ensure appropriate measures are in place for handling failed jobs

**Broadcasting**
* Broadcasting uses Laravel Echo unless agreed beforehand
* Ensure appropriate authentication & security for each channel

**Database**
* All migrations use foreign keys for relations to ensure database integrity
* SQL data types are appropriate
* Monetary values should either be stored in pence as an INTEGER or as a DECIMAL - Never a FLOAT
* Large tables that are queried often are indexed in order to keep database speed up

## Processes
### Automated Testing

Automated testing should be implemented in a way to confirm functionality specified by the client.
To this end, automated tests should be created to confirm the project user stories and related
business logic is satisfied and correct.

#### Unit Testing

Our current policy is that unit tests should be used for:

* Testing of project business logic / rules, that are able to be sufficient abstracted
* Testing of packages, including our open source work

#### Integration, Browser & End-to-End Testing

Integration, browser and end-to-end tests should be used for:

* Testing of project functionality related to client specified user stories
* Testing of critical path functionality, such as the register, login, and checkout process of an
ecommerce site

Since end-to-end tests are slower than browser tests and browser tests are slower than integration
tests, you should ideally aim to use the fastest possible test type, if appropriate.
### Manual Testing
#### 1 - Developer Testing
During the development of the project, developers will create branches as described in our git work flow documentation.
After the initial feature development - the developer should manually test their feature after writing passing
[automation tests](#automated-testing). This phase should also include a check that ensures the feature does not infringe
any of the items listed in the ["What makes a good web application?"](#what-makes-a-good-web-application) section.

Once the feature has the thumbs up from the developer - that is when the feature can continue through our
[Git Workflow](/devops/git/workflow.html)

The developers should log any bugs found in this phase into JIRA to improve their ratio of bugs they find VS bugs that
get through to the QA team/customer.

#### 2 - Sprint Testing (On the master branch)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A possible problem count occur here if QA is testing on the master branch, and development continues on with development of the next sprint in the mean time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should mark a version/end of sprint branch that the QA team uses?

Thoughts? @dextermb @AlexCatch @NilesB @DivineOmega @robmilward

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wheatleyjj What about testing on staging? We should be aiming to push a version to staging at the end of any sprint so it makes sense to do QA on that branch.

Copy link
Contributor

@ash123456789 ash123456789 Dec 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @AlexCatch about testing on staging since that is essentially a development environment anyway.

Copy link
Contributor

@DivineOmega DivineOmega Dec 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've worked on projects previously with this kind of QA process in place.

At the start of a sprint, a sprint-1 branch would be created. After the sprint is over, QA would begin testing on sprint-1. At the same time a sprint-2 branch would be created (from sprint-1) to allow developers to continue development immediately.

Once QA was happy with sprint-1 it would be merged into mater, and the process would repeat.

Copy link
Contributor Author

@jaredkove jaredkove Dec 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ash123456789 @AlexCatch I think that testing on the customers environment would cause annoyances to the client if they're using it regularly.

Ways around this are sprint branches as I and @DivineOmega suggested or having a development environment - but that comes with another layer of grief. I'm also thinking experienced QA testers will have good knowledge of automation tests so would be able to look into both the Blackbox/Whitebox sections of this document more easily if they have it running locally.

Thumbs up if you agree?

Copy link
Contributor

@DivineOmega DivineOmega Jan 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wheatleyjj Agree with the sprint branches and/or development server idea. 👍

At the end of each sprint the Lead Developer on the project should arrange some time with a member of the QA team to
manually test all completed user stories in that sprint.

They should look to give the functionality a complete test, using any available acceptance criteria provided with the
story. Testers should also ensure that the UX seems correct, that all inputs are validated correctly and data is
retained where appropriate.

#### 3 - S2P Testing (Staging server)
Staging to production testing should be focused around the user flow of the system.

Users should start with a fresh, clear database and work on the journey of the system end-to-end. Using the different
user types, the tests should ensure that the customers workflow is satisfied from input to output.