Isomorphic JavaScript at LiteFarm

Isomorphic JavaScript is JavaScript that can run in both the browser and in Node.js on our back-end. The primary advantage of isomorphism is that business logic can be shared between the front-end and back-end, reducing duplication of code. This reduces errors due to code getting out of sync (for example when the validation for the sensor upload csv was different on the front-end and the back-end), and reduces the developer effort required to implement full-stack features. Given the strong benefits of introducing Isomorphism into the code base, this document will explore how to implement it.

Updating our tooling to support shared code

The easiest way to update our monorepo to support shared code is to add a package to the packages directory. For example, this directory could be structured as:

With this setup, only minor changes would have to be made to our run/build scripts and programs. Specifically, we would need to configure nodemon to watch the code in the shared directory as well. This can be done by adding a nodemonConfig to the package.json file in the api directory. This nodemonConfig can look like:

The other changes that would need to be made would be to the webapp and api Dockerfiles, as those would need to copy the shared code over into the container before they are built.

Updating our code to support shared code

Option #4 from the below list was selected and as of September 2022, the api has been updated to ES6. The following section is being kept for historical context.

The major issue with updating our code to support isomorphism is that the webapp package uses ES6 modules and imports, while the api package uses CommonJS modules and imports. This means that by default, the shared code will not easily support being imported into both the webapp and api. If we write the shared code as ES6 modules with the .mjs extension then it can be imported into the CommonJS modules on the back-end with an asynchronous import() function, however this would lead to us having both synchronous and asynchronous imports in the same file, and would lead to added complexity as a lot more of the codebase would need to be asynchronous by default. If we make the shared code as CommonJS modules, then we can use the import * as foo from 'foo' syntax to import it into ES6, but if there were any calls to require() another file, then the import would fail - this would mean that every single piece of shared logic would need to be in self contained files.

Given these challenges of ES6/CommonJS interop there are 4 approaches which I think offer a good balance of maintaining our current tooling/code as much as possible while also allowing us to adopt isomorphism:

1. We progressively adopt isomorphism where it is needed/wanted in the codebase using ES6 modules for shared code and converting back-end files to ES6 where needed. Whenever a file in the api package needs to use the shared code, we would switch its extension to be .mjs, and then we would refactor every file which require()s the file to use an asynchronous import() call instead. This would mean that we could adopt this pattern (and ES6) very incrementally, however it would also greatly increase the complexity of working with imports on the back-end, as they could either be synchronous require() calls or asynchronous import() calls, and whenever they are asynchronous, the file will need to be updated to handle that appropriately.

2. We progressively adopt isomorphism where it is needed/wanted in the codebase using ES6 modules for shared code without converting back-end files to ES6. Whenever a file in the api package needs to use the shared code, we would refactor that file to handle the asynchronous import() call, and leave everything else unchanged in other files. This would mean that we could adopt this pattern (but not ES6) very incrementally. However, once again this will lead to increased developer complexity in terms of the imports into files, if in a more contained scope.

3. We progressively adopt isomorphism where it is needed/wanted in the codebase using CommonJS modules. Whenever logic needs to be shared, all of the logic (and any code it depends on) would be placed within one file and then imported using the appropriate syntax into the front-end and back-end code. This would mean that any shared code cannot rely on any external packages, or other files within the project. This could once again lead to code duplication as we start to duplicate logic between different files in the shared directory, reducing the benefits gained from this approach.

4. We adopt isomorphism with ES6 modules in one go, by converting the api directory to support ES6 instead of CommonJS. We would update the type field in packages/api/package.json to make all of the files run as ES6 modules, which would mean that our front-end and back-end code are now consistent with each other. The risk here is that this is a very large change, and would lead to a massive PR. However, as this would only be changing the syntax of the imports (and none of the logic throughout the app), much of the risk could be mitigated by making the changes and then deploying the branch with the changes to avocado and running a mini sanity to see if anything broke. Between the mini sanity and the usual pre-release sanity, we would be able to catch any errors that may occur.

Given that ES modules are the standard for JavaScript instead of CommonJS, and that the changes would only be made to the way in which code is imported into the api files, I think that the best approach for LiteFarm is to just make the change in one go, rather than having a slow conversion between the two that increases complexity for developers, and will likely never finish without a concerted effort to be completed.