Objective

The UI codebase for Litefarm is currently using Redux as a solution for global state management. To handle fetching data from the API and storing it in the store, we’re using the Redux-Saga library. Sagas are functions that act as separate “threads” within the application, listening for specific actions, running asynchronous tasks in response and updating the state.

Although sagas used to be a common approach for data fetching, currently Redux documentation itself proposes an alternative called RTK Query, which involves less boilerplate and should be easier to read and maintain than sagas.

The goal of this RFC is to articulate the motivation and reasons why moving towards RTK Query would be desirable, and delineate a clear plan for implementing a migration.

Motivation

• On our latest release (November 2023), we had around a dozen bugs related to “white screens”, which were caused by attempting to read properties from undefined objects. These objects were entities in the Redux store, and a common underlying cause were issues around handling re-fetching of the data on certain situations such as switching farms. With Redux-Saga, all of the logic for fetching data, caching it and re-fetching it has to be written manually, while RTK Query automatically handles this logic.

• The current official documentation for Redux strongly recommends against using sagas for data fetching, and instead supports the use of RTK Query as the default choice for this use case (see https://redux.js.org/usage/side-effects-approaches#recommendations ).

• The amount of code that needs to be written manually to do data fetching with sagas means any time we add a new endpoint to our API and need to fetch the data on the UI, we have to touch on or implement multiple different files:

  • Actions, where the action which will be captured by the saga is defined.

  • Constants, where the action names are defined.

  • Saga, where the actual call to the endpoint happens.

  • Slices, to add selector functions to read the new data from the store.

  • Components, where we dispatch the actions and read the data from selectors.

RTK Query, as the Redux docs mention, “replaces the need to write any actions, thunks, reducers, selectors, or effects to manage data fetching”. There’s only one API slice where the new endpoint needs to be added, and then the RTK Query hooks can be called directly in the components to fetch and read the data.

• Less manually written code also means less complexity, and in turn code that’s easier to read, maintain, and requires a learning curve that’s a lot less steep for contributors or new members of the team.

User Benefit

Although the underlying change from sagas to RTK Query should be transparent to users in terms of functionality, there should be various direct sources of end user benefit from this change:

• Better code quality and in turn less likelihood of catastrophic bugs resulting in white screens, which require refreshing the page to be fixed or even worse, wiping the cache (which a lot of users likely don’t do often), resulting in a highly frustrating user experience.

• Better velocity to deliver new features to end users, since less time is spent implementing fetching for new data or fixing bugs related to data fetching.

• Hopefully better load times for pages in the app, since it should now be much easier to track down what data is actually required by each page and make sure that we don’t fetch data we don’t actually need, as well as ensure that re-fetching is handled optimally so no recurrent calls are done that aren’t required.

Design Proposal

Alternatives Considered

The Redux documentation includes a thorough comparison between RTK Query and other common alternatives used for data fetching, such as React Query, Apollo and urql (see Comparison with Other Tools | Redux Toolkit ).

Both Apollo and urql are suited for GraphQL APIs, which means they don’t fit our needs. Between RTK Query and react query, although both offer a similar set of features, RTK Query is better suited to interact with a Redux store. Since we already heavily use Redux, migrating to RTK Query should be simpler and have better supportive documentation than incorporating React Query, and have the advantage of showing changes in state over time in the Redux dev tools for easier debugging.

In terms of comparing RTK Query with other approaches for side effects handling such as sagas and thunks, as covered in the previous sections, we have already experienced issues with those alternatives which are the main drive for wanting to migrate to RTK Query. The official Redux recommendations also support the migration and advice against using these other approaches for the specific use case of data fetching.

Performance Implications

Using RTK Query will handle caching of the data for us and make sure we only re-fetch the data when the payload or query parameters change, avoiding unnecessary recurrent calls to the API. This should mean better performance for the UI by decreasing load times for pages, and less strain on the API side.

Dependencies

This proposal requires adding a new dependency to the codebase, which is the package for RTK Query itself. RTK Query, although optional, is part of the Redux Toolkit package which we already use, and which is currently well maintained by the Redux team itself. The Redux Toolkit package has 10K stars in GitHub, and its most recent update was last month.

The Redux Toolkit package is open source under a MIT License, which means if at some point it was deprecated or maintenance stopped, the risk to LiteFarm would be minimal since we would be able to fork the repository and continue using the library while evaluating alternatives.

Since we are already using Redux Toolkit, per the Redux documentation the increased bundle size for adding RTK Query will be ~9 KB for the package itself and ~2 KB for the hooks, so around 11 KB. In comparison, React Query has a bundle size of around 50 KB, so in terms of size when Redux Toolkit is already in use RTK Query is more convenient. Although this increased bundle size should slightly increase load times, the added benefit of easier code maintenance and 

Engineering Impact

There’ll be a cost to migrating all of our current sagas to RTK Query, but the time spent should be considered an investment for easier code maintainability and less time spent debugging, fixing bugs and adding new code in the future.

There are about 50 saga files currently in our repository. Most of them are small enough that they could likely be migrated pretty quickly, with some additional time for testing for regressions, while some of them are more bloated and will require more time. At a very rough guesstimate of 1 dev day per file, the entire migration would require about 50 dev days (although we probably wouldn’t migrate saga by saga, but rather component by component, but that’s a lot harder to estimate). However, we would not invest in this migration all at once but do it gradually over time. The saga approach and the RTK Query approach can co-exist for as long as is needed while we execute the migration.

User Impact

There should be no user impact from this feature and the rollout should be transparent to users. This means as mentioned previously we can migrate gradually and ship the changes out as we go, without affecting the release schedule that’s planned.

Tutorials and Examples

This PR illustrates the migration of one page, the Finances Overview, from sagas to RTK Query.

Some key benefits from the migration found throughout the process of implementing it:

• Instead of having components get the data it needs from the store through selectors, they now get the data they need from the RTK Query hooks. This means the library automatically handles calling the right endpoints if they haven’t been called before, and no matter which order the user navigates through the app we can be sure the data will be fetched when required.

• Since we don’t manually specify when to fetch the data, the main Finances component now only shows the loader the first time you navigate to it or when you switch farms (switching farms invalidates the cache since endpoints receive the farm ID as part of the URL). The rest of the time, there’s no waiting time for the page to load since RTK Query handles retrieving the data from the cache instead of re-fetching it.

• Lots less code to fetch data. Instead of having to add a constant + an action + a saga, adding a new endpoint now requires three or four lines added to the apiSlice file.

• Using cache tags allows us to specify how queries and mutations are linked, and let RTK Query automatically handle re-fetching entities if we have performed a mutation adding a new instance of that entity. E.g.: if we add a new expense, because of the way tags are specified, expenses will be automatically re-fetched to reflect the new state.

Some cons encountered throughout the process:

• We heavily use Redux selectors throughout the app, and sometimes these selectors have logic in them such as sorting or filtering. When switching to RTK Query, we now need to use the selectFromResult option in the query hooks which is the equivalent of selectors. This means re-writing the logic that exists in selectors today, although it should be fairly easy to port it to selectFromResult.

Potential improvements:

• Currently all of the endpoints are specified in the apiSlice file. If we find that this file gets too bloated and hard to read, there's a way to split the endpoints in different files Code Splitting | Redux Toolkit.

• There’s some logic used in the PR in selectFromResult functions that could be abstracted further to reuse it in other components.

Steps to Migrate

My suggestion is that we migrate page by page. Picking any page in the app, the steps to follow would be roughly:

• Take a look at the main index.js file for that page. Are there any selectors there or anything that's being fetched from the Redux store that is in fact data from the API? If so, trace back the saga that does the data fetching.

• Add the corresponding endpoints to apiSlice.js, and return the right hooks. Make sure you add a tag type if you're adding an entity for the first time, and that you specify that tag in the providesTags option in the query. This will ensure that when adding mutations related to this query later, RTK Query knows when to invalidate the cache and re-fetch the data.

• Replace the selectors in the component file with calls to the hooks returned by apiSlice. If the selectors had any specific logic for filtering out results, you'll also need a selectFromResult function to replace that logic.

• If there are any actions dispatched in the component to fetch data from the API, remove them.

• Are there any mutations performed in the component, such as adding or deleting an entity? Trace back the saga that does the mutation.

• In the same manner, add the corresponding endpoints to apiSlice.js and return the mutation hooks. Make sure you add the right tags in the invalidatesTags option, to force RTK Query to re-fetch data after the mutation is done.

• In the component, where an action is dispatched to execute a mutation, replace that with the function returned by the mutation hook. Add any error handling that previously existed in the saga.

• If there are any components that are accessed from this page and that would affect how the data is fetched in this page, migrate them as well. For example, in Finances Overview, even though adding an expense or revenue is done in different pages, those mutations affect the transaction list displayed in the overview – so I migrated them altogether.

• If the sagas you replaced are no longer used by any component, remove them and their corresponding actions. Same for selectors.

API Shortcomings

While implementing the migration example above, I encountered some shortcomings in our API which we could improve as we continue migrating:

• We currently have several selectors on the webapp side that take care of filtering out deleted results. Although we do soft deletes, presumably a deleted entity would never need to be consumed by the UI since from the user’s perspective it’s gone. We could implement filtering out deleted entities in the endpoints themselves, on the server side, to remove this responsibility from the UI layer.

• Our endpoint URLs don’t follow the REST paradigm. For example, to fetch all sales within a farm, we use /expense/farm/:farm_id. This URL can be understood as "a farm is a nested entity within an expense", but it's actually the other way round, and the correct URL would be /farms/:farm_id/expenses. Alternatively we could use /expenses?farm_id=:farm_id, meaning get all the expenses filtered by this farm ID.

• The case described above is not consistent throughout the API, with some endpoints  following the pattern above, some being /sale/:farm_id. PATCH and DELETE endpoints don't include a farm_id at all. We should try to keep a consistent pattern across the URLs, and for PATCH and DELETE requests, we should be checking if the user has access to the farm in the same way we do for requests to fetch or to add. I'd argue that since deleting is a destructive action, permissions should be even more heavily enforced there than for fetching.

• If we used a pattern like /farms/:farm_id/expenses, we could have a common router for every route checking for farm access, and then the nested more specific routers for each entity.

• Note that in the example used above (/farms/:farm_id/expenses), nouns are specified in plural -- although that detail is somewhat debatable, this article illustrates the reasons why using plural could be preferable.

Sources

Redux and Redux Toolkit documentation:

• https://redux.js.org/usage/side-effects-approaches

• RTK Query Overview | Redux Toolkit

• Migrating to Modern Redux | Redux

Bundle size for React query:

• https://bundlephobia.com/package/react-query@3.39.3

Template for this RFC:

• https://github.com/tensorflow/community/blob/master/rfcs/yyyymmdd-rfc-template.md