Versions Compared

Old Version 6

changes.mady.by.user Mwayanjana Bolokonya

Saved on

compared with

New Version Current

changes.mady.by.user Lite Farm

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

For the last 6 months, there has been a Cypress happy path e2e test implemented in the LiteFarm codebase and it is set up to run on a GitHub hosted runner on pull requests to the integration branch. The statement coverage for this test is currently just under 60%, and it was is the QA team’s goal to progressively increase this coverage metric.

Goals

Tests should be refactored to be more flexible (guidance from Mika Saryan). Some goals to aim for with our tests:

  1. The tests should be organized into smaller chunks with specific functionality: create task, create account, create farm, etc

  2. The architecture has to be designed to allow flexibility in how you run the tests

    1. Only run a single language

    2. Allow to run only "fast" tests. You can tag the tests as fast and slow, this allows to run subset of the tests (e.g. when testing a commit vs integration into master)

  3. Use "fixtures" (they may not be called this in Cypress). There is concept of fixtures in automated testing, which allow to do certain pre-requisites before running a test (e.g. create an account). This allows you to run 1 test or 10 tests, but create an account only once.

  4. Eliminate any Sleep or fixed time based Wait. Usually this is the first thing that causes flaky and slow tests

  5. Add profiling. You need to know how long each test takes which will allow to figure out bottlenecks in the test or in the backend

Issues with the current implementation

...

  1. An upgrade to a larger Github hosted runner(https://docs.github.com/en/actions/using-github-hosted-runners/using-larger-runners ) to explore whether lack of resources is the cause of flakiness of Cypress tests in the CI/CD pipeline

  2. Run the app using Docker, if the above does not yield any positive results

  3. Ensuring that engineers fix tests that become broken

  4. Standardized .env files across developers local environments and test environments

  5. Breaking test suites down into smaller, focused test suites. Aim to keep each test suite focused on a specific area or feature of your application. Avoid creating overly large or complex test suites that test multiple unrelated features. Smaller, focused test suites are easier to manage, debug, and maintain.