Technical notes

Owned by Anto Sgarlatta
Last updated: Dec 13, 2024
6 min read

Overview

We will be splitting the project into four mini-projects, and push out a release for each one of them:

Task breakdown [WIP]

Note: This is a work in progress and has to be revisited with final designs and more information. The architecture design reflects initial thoughts and will be refined, and the estimations are very rough guesses that will also change as we improve the breakdown and learn more.

Sensor management

Database

Current state

Proposal

Tasks

  • Drop partner_reading_type table

  • Create inbound_integration_type enum table

    • id

    • type (SENSOR_READING, IRRIGATION_PRESCRIPTION)

  • Change farm_external_integration name to farm_inbound_integration, add type_id column for integration type

  • Add sensor_manufacturer table

    • id

    • name

    • farm_id (need to be able to add custom brands)

    • integrating_partner_id (for Ensemble Scientific, link to integration table)

  • sensor_reading_type should be enum table containing the type of readings

    • id

    • key

  • Create sensor_sensor_reading_type_relationship table (one to many)

    • sensor_id (FK to sensor)

    • sensor_reading_type_id (FK to sensor_reading_type)

  • Create sensor_profile table (a profile is a set of sensors on the same location)

    • id

    • location_id

    • field_location_id (associated with a field)

  • Drop partner_id and location_id columns in sensor table, add

    • sensor_manufacturer_id (FK to sensor_manufacturer)

    • sensor_profile_id (FK to sensor_profile)

  • Scalability:

    • Make the sensor_reading table partitioned by date from the start

    • Setup pg_partman to create partitions by year or month on the sensor_reading table

      • How frequent will the sensor readings be?

      • How many readings do we want to retain?

      • What will be the display options for the chart on the UI?

Estimation: 2 dev weeks

Back-end

Current endpoints

router.get('/:farm_id', SensorController.getSensorsByFarmId); router.post( '/', checkScope(['add:sensors']), upload.single('sensors'), SensorController.addSensors, ); router.delete('/:location_id', SensorController.deleteSensor); router.patch('/:location_id', SensorController.updateSensorbyID); router.post( '/reading/partner/:partner_id/farm/:farm_id', validateRequest, SensorController.addReading, ); router.get('/:location_id/reading', SensorController.getAllReadingsByLocationId); router.get('/reading/farm/:farm_id', SensorController.getReadingsByFarmId); router.post('/reading/invalidate', SensorController.invalidateReadings); router.post( '/ ', hasFarmAccess({ body: 'location_id' }), checkScope(['delete:fields']), validateLocationDependency, SensorController.retireSensor, ); router.get('/:location_id/reading_type', SensorController.getSensorReadingTypes); router.get('/farm/:farm_id/reading_type', SensorController.getAllSensorReadingTypes); router.get('/partner/:partner_id/brand_name', SensorController.getBrandName); router.post( '/reading/visualization', hasFarmAccess({ body: 'farm_id' }), hasFarmAccess({ body: 'locationIds' }), SensorController.getAllSensorReadingsByLocationIds, );

 

Proposal

  • GET /sensor - Retrieve all sensors for a farm

  • POST /sensor - Add a new sensor or sensors (should receive an array, similar to Animals implementation)

  • PATCH /sensor/:id - Edit a sensor

  • DELETE /sensor/:id - Delete a sensor

  • GET /sensor/:id/reading - Get all readings for a sensor

  • POST /sensor/:id/reading  -  Post a sensor reading, will be used by external partner (this is what we should be registering for them as a , one per sensor)

  • GET /sensor/reading_type - Get sensor reading types enum

  • GET /sensor/manufacturer - Get sensor manufacturers enum

 

Tasks

  • Refactor routes

    • Similar to animals project, farm_id will be read from header and not included in routes

  • Refactor models following DB migration

  • Refactor controller

    • Refactor method to add a sensor

      • Improve code readability

      • Depending on what the addition flow looks like, we might have to enable receiving a list of sensors (similar to animals)?

    • Remove support for CSV parsing

    • Refactor method to patch and delete sensors, improve code readability

    • Refactor ensemble util to a service, improve code readability

    • Remove methods to get readings by farm and by location

    • Add method to get readings by sensor ID

    • Add methods to get sensor manufacturers and reading types enum

    • Add method to add sensor readings

      • Should parse incoming data from partner and store it appropriately.

Estimation: 10 dev weeks

Front end

Current state

Proposal

TODO: Link designs

Tasks

  • [Nice to have] Implement “Add to map” redesign

    • Drawer component should now have the option to open from the side (might be implemented for Beta support?)

    • Set “responsiveModal” prop in Drawer to false in MapDrawer component

  • [Nice to have] Redesign map footer

  • Refactor MapDrawer out of MapFooter and into Map component

  • Refactor MapDrawer to display flows for different areas/lines/points

    • Should support different steps on top of current add/filter mode

    • Will only support displaying Sensor flow in drawer for now (possibly redo Field, see Nice to have below)

  • [Nice to have] Implement Redesign of Field details view

  • Add Sensors tab to Field details and display profile/sensors

  • Refactor sensor saga to RTK Query

  • Implement sensor addition flow

    • Will need to use multistep form

  • Add “view” mode to MapDrawer

    • Will allow displaying details for the sensor within the drawer

  • Implement sensor details tab

  • Implement sensor readings tab

    • Library for chart display?

    • Remove sensor readings from location selection modal?

  • Implement sensor edit flow

  • Implement sensor deletion flow

Estimation: 12 dev weeks

 

Pivot management

Database

TODO: Proposed state with diagrams

Tasks

  • Create pivot_manufacturer table

    • id

    • name

    • farm_id (FK to farm)

    • Integration_id (FK to integration)

  • Create pivot table

    • id

    • location_id

    • field_location_id

    • radius

    • power_source

    • flow_rate

    • pivot_manufacturer_id

    • external_id

    • approved

    • deleted

Estimation: 1 dev week

Back end

Proposed endpoints

  • GET /pivot - Retrieve all sensors for a farm

  • POST /pivot - Add a new pivot

  • PATCH /pivot/:id - Edit a pivot

  • DELETE /pivot/:id - Delete a pivot

  • GET /pivot/manufacturer - Get pivot manufacturers enum

Tasks

  • Create routes

  • Create models

    • Add pivot relation mapping to location model

  • Create pivot controller, with methods

    • Getting all pivots for a farm

    • Adding a pivot

    • Editing a pivot

    • Deleting a pivot

    • Getting all pivot manufacturers

Estimation: 6 dev weeks

Front end

TODO: Add designs

Tasks

  • Add pivot support to the location slice.

  • Add RTK Query support for pivot management.

  • Display pivot point on map.

  • Implement adding pivot flow.

    •  Should be able to reuse the same drawer structure put together for sensors.

  • Implement pivot details view.

  • Display pivot in Sensors tab in Field details view.

  • Implement editing a pivot.

  • Implement deleting a pivot.

Estimation: 10 dev weeks

Irrigation prescription management

Database

TODO: Add diagram

Tasks

  • Create irrigation_prescription_settings table (or include these attributes in pivot itself?)

  • Create irrigation_prescription_request table (store requests to create a prescription on demand from user)

    • id

    • pivot_id 

    • created_at

  • Create irrigation_prescription table (keep track of irrigation files created)

    • id

    • pivot_id  (FK to pivot)

    • figure_id

    • created_at

    • document_id (store privately for farm, might be several documents if there are several files)

    • irrigation_prescription_request_id (FK to irrigation_prescription_request, null if automatically generated)

Back end

Proposal

  • GET /irrigation_prescription - Get all prescriptions for a farm

  • POST /irrigation_prescription_request - Generate a new request for a prescription

  • POST /irrigation_prescription - Generate a new prescription

  • PATCH /irrigation_prescription/:id - Used to approve the prescription

  • GET /irrigation_prescription_settings?pivot_id=:id - Get irrigation prescription settings for a pivot

  • PATCH  /irrigation_prescription_settings:id - Edit settings

Tasks

  • Create routes

  • Create models

  • Update pivot controller so that when a pivot is added, an integration of type IRRIGATION_PRESCRIPTION is stored and registered with the partner (would we send the irrigation prescription settings to the partner at this point? - TBD how we register that)

  • Create irrigation prescription controller, with methods

    • Getting all prescriptions for a farm

    • Requesting a prescription

      • Used by the UI to request a prescription on demand, will need to make a call to partner’s API

    • Generating a new prescription

      • Used by the partner as a webhook to post a prescription to.

      • Should use prescription settings.

      • When we receive the prescription we should store the file and/or add the document record.

    • Approving a prescription

      • Patching approved attribute on prescription

      • On approval, send to the pivot?

    • Getting irrigation prescription settings for a pivot

    • Editing irrigation prescription settings

  • Add a method sending a notification when an irrigation prescription is posted.

Estimation: 12 dev weeks

Front end

TODO: Add designs

Tasks

  • Add RTK Query support for irrigation prescriptions.

  • Add step to input irrigation prescription settings to pivot addition flow.

  • Display irrigation prescription settings in field details view.

  • Add capability to edit irrigation prescription settings.

  • Add irrigation prescription generation CTA and generate a prescription request on click.

  • Add support for new notification (translation strings).

  • Ensure irrigation prescription files are displayed in the Documents section.

Estimation: 6 dev weeks

TODO: Weather & soil type integrations and map enhancementes

Weekly execution plan [WIP]

Gantt diagram to be added here with proposal on how to assign resources to tasks over time to meet the deadline.