Versions Compared

Old Version 2

changes.mady.by.user Mwayanjana Bolokonya

Saved on

compared with

New Version 3

changes.mady.by.user Lite Farm

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Limited test coverage: Running E2E tests on a pre-push hook may not cover all possible scenarios and edge cases that can occur in a production environment. It is important to ensure that the tests cover a wide range of scenarios and edge cases to ensure that the code is thoroughly tested.(e.g. minIO server on local)

  2. Environment inconsistencies: Developers may encounter environment inconsistencies when running automated tests in their local environment, which can lead to false positives or false negatives.

...

  1. Increased deployment time: Running E2E tests during deployment can increase the time it takes to deploy new code changes. This is because E2E tests typically take longer to run than unit tests or integration tests. This increased deployment time can cause delays in delivering new features or updates to users.

  2. Limited customization: GitHub Actions provide a limited set of customization options for E2E testing. Teams may have specific requirements or preferences for how their E2E tests are run that cannot be accommodated within the constraints of a GitHub Action workflow.

  3. Security: Running E2E tests in the production environment using a GitHub Action workflow may raise security concerns, as it requires giving access to the production environment to external systems.

Solution 3

Optimise the CI runner and test environment Create a custom Github runner to run on a Digital Ocean server to more accurately mimic the production beta environment

Pros:

  1. Customization: Running E2E tests on a dedicated CI runner provides more flexibility and customization options. Teams can configure the environment, tools, and dependencies to suit their specific needs.

  2. Control: Running E2E tests on a dedicated CI runner provides greater control over the testing process. Teams can manage the resources used for testing, and ensure that external dependencies are available and configured correctly.

  3. Security: Running E2E tests on a dedicated CI runner provides an additional layer of security, as it allows teams to isolate the testing environment from the production environment.

...