Technical notes
- Anto Sgarlatta
Overview
We will be splitting the project into four mini-projects, and push out a release for each one of them:
Sensors management https://lite-farm.atlassian.net/browse/LF-4491
Pivots management https://lite-farm.atlassian.net/browse/LF-4499
Irrigation prescriptions management https://lite-farm.atlassian.net/browse/LF-4499 (this user story still needs to be split up in Jira)
Map enhancements https://lite-farm.atlassian.net/browse/LF-4508
Task breakdown [WIP]
Note: This is a work in progress and has to be revisited with final designs and more information. The architecture design reflects initial thoughts and will be refined, and the estimations are very rough guesses that will also change as we improve the breakdown and learn more.
Sensor management
Database
Current state
Proposal
Tasks
Drop partner_reading_type table
Create inbound_integration_type enum table
id
type (SENSOR_READING, IRRIGATION_PRESCRIPTION)
Change farm_external_integration name to farm_inbound_integration, add type_id column for integration type
Add sensor_manufacturer table
id
name
farm_id (need to be able to add custom brands)
integrating_partner_id (for Ensemble Scientific, link to integration table)
sensor_reading_type should be enum table containing the type of readings
id
key
Create sensor_sensor_reading_type_relationship table (one to many)
sensor_id (FK to sensor)
sensor_reading_type_id (FK to sensor_reading_type)
Create sensor_array table (a profile is a set of sensors on the same location)
id
name
location_id
field_location_id (associated with a field)
Drop partner_id and location_id columns in sensor table, add
sensor_manufacturer_id (FK to sensor_manufacturer)
sensor_profile_id (FK to sensor_profile)
name
retired
deleted
Scalability:
Make the sensor_reading table partitioned by date from the start
Setup pg_partman to create partitions by year or month on the sensor_reading table
How frequent will the sensor readings be?
How many readings do we want to retain?
What will be the display options for the chart on the UI?
Estimation: 2 dev weeks
Back-end
Current endpoints
router.get('/:farm_id', SensorController.getSensorsByFarmId);
router.post(
'/',
checkScope(['add:sensors']),
upload.single('sensors'),
SensorController.addSensors,
);
router.delete('/:location_id', SensorController.deleteSensor);
router.patch('/:location_id', SensorController.updateSensorbyID);
router.post(
'/reading/partner/:partner_id/farm/:farm_id',
validateRequest,
SensorController.addReading,
);
router.get('/:location_id/reading', SensorController.getAllReadingsByLocationId);
router.get('/reading/farm/:farm_id', SensorController.getReadingsByFarmId);
router.post('/reading/invalidate', SensorController.invalidateReadings);
router.post(
'/ ',
hasFarmAccess({ body: 'location_id' }),
checkScope(['delete:fields']),
validateLocationDependency,
SensorController.retireSensor,
);
router.get('/:location_id/reading_type', SensorController.getSensorReadingTypes);
router.get('/farm/:farm_id/reading_type', SensorController.getAllSensorReadingTypes);
router.get('/partner/:partner_id/brand_name', SensorController.getBrandName);
router.post(
'/reading/visualization',
hasFarmAccess({ body: 'farm_id' }),
hasFarmAccess({ body: 'locationIds' }),
SensorController.getAllSensorReadingsByLocationIds,
);
Proposal
GET /sensor_array - Retrieve all sensor arrays for a farm
POST /sensor_array - Add a new sensor array or arrays with their corresponding sensors (should receive an array, similar to Animals implementation)
PATCH /sensor_array/:id - Edit a sensor array. Deleting a sensor should also happen through this endpoint.
DELETE /sensor_array/:id - Delete a sensor array.
GET /sensor_array/:id/reading - Get all readings for a sensor array
POST /sensor/:id/reading - Post a sensor reading, will be used by external partner (this is what we should be registering for them as a , one per sensor)
GET /sensor/reading_type - Get sensor reading types enum
GET /sensor/manufacturer - Get sensor manufacturers enum
Tasks
Refactor routes
Similar to animals project, farm_id will be read from header and not included in routes
Refactor models following DB migration
Refactor controller
Refactor method to add a sensor
Improve code readability
Should be directed towards adding sensor arrays now
Enable adding an array of sensor arrays
Remove support for CSV parsing
Refactor method to patch and delete sensors, improve code readability and direct towards patching and deleting sensor arrays
Refactor ensemble util to a service, improve code readability
Remove methods to get readings by farm and by location
Add method to get readings by sensor array ID
Add methods to get sensor manufacturers and reading types enum
Add method to add sensor readings
Should parse incoming data from partner and store it appropriately.
[Nice to have] Get field for a specific location
https://developers.google.com/maps/documentation/javascript/examples/poly-containsLocation }
If this works this method can be called on sensor array addition to automatically detect the field and link it instead of having the user specify it manually
Estimation: 10 dev weeks
Front end
Current state
Proposal
Tasks
[Nice to have] Implement “Add to map” redesign
Drawer component should now have the option to open from the side (might be implemented for Beta support?)
Set “responsiveModal” prop in Drawer to false in MapDrawer component
[Nice to have] Redesign map footer
[Nice to have] Implement Redesign of Field details view
Add Field technology tab to Field details and display profile/sensors
Refactor sensor saga to RTK Query
Implement sensor addition flow
Will need to use multistep form
Implement sensor details tab
Implement sensor data tab
Library for chart display?
Remove sensor readings from location selection modal?
Implement sensor edit flow
Implement sensor deletion flow
Estimation: 12 dev weeks
Pivot management
Database
TODO: Proposed state with diagrams
Tasks
Create pivot_manufacturer table
id
name
farm_id (FK to farm)
Integration_id (FK to integration)
Create pivot table
id
location_id
field_location_id
radius
power_source
flow_rate
pivot_manufacturer_id
external_id
approved
deleted
Estimation: 1 dev week
Back end
Proposed endpoints
GET /pivot - Retrieve all sensors for a farm
POST /pivot - Add a new pivot
PATCH /pivot/:id - Edit a pivot
DELETE /pivot/:id - Delete a pivot
GET /pivot/manufacturer - Get pivot manufacturers enum
Tasks
Create routes
Create models
Add pivot relation mapping to location model
Create pivot controller, with methods
Getting all pivots for a farm
Adding a pivot
Editing a pivot
Deleting a pivot
Getting all pivot manufacturers
Estimation: 6 dev weeks
Front end
TODO: Add designs
Tasks
Add pivot support to the location slice.
Add RTK Query support for pivot management.
Display pivot point on map.
Implement adding pivot flow.
Should be able to reuse the same drawer structure put together for sensors.
Implement pivot details view.
Display pivot in Sensors tab in Field details view.
Implement editing a pivot.
Implement deleting a pivot.
Estimation: 10 dev weeks
Irrigation prescription management
Database
TODO: Add diagram
Tasks
Create irrigation_prescription_settings table (or include these attributes in pivot itself?)
Create irrigation_prescription_request table (store requests to create a prescription on demand from user)
id
pivot_id
created_at
Create irrigation_prescription table (keep track of irrigation files created)
id
pivot_id (FK to pivot)
figure_id
created_at
document_id (store privately for farm, might be several documents if there are several files)
irrigation_prescription_request_id (FK to irrigation_prescription_request, null if automatically generated)
Back end
Proposal
GET /irrigation_prescription - Get all prescriptions for a farm
POST /irrigation_prescription_request - Generate a new request for a prescription
POST /irrigation_prescription - Generate a new prescription
PATCH /irrigation_prescription/:id - Used to approve the prescription
GET /irrigation_prescription_settings?pivot_id=:id - Get irrigation prescription settings for a pivot
PATCH /irrigation_prescription_settings:id - Edit settings
Tasks
Create routes
Create models
Update pivot controller so that when a pivot is added, an integration of type IRRIGATION_PRESCRIPTION is stored and registered with the partner (would we send the irrigation prescription settings to the partner at this point? - TBD how we register that)
Create irrigation prescription controller, with methods
Getting all prescriptions for a farm
Requesting a prescription
Used by the UI to request a prescription on demand, will need to make a call to partner’s API
Generating a new prescription
Used by the partner as a webhook to post a prescription to.
Should use prescription settings.
When we receive the prescription we should store the file and/or add the document record.
Approving a prescription
Patching approved attribute on prescription
On approval, send to the pivot?
Getting irrigation prescription settings for a pivot
Editing irrigation prescription settings
Add a method sending a notification when an irrigation prescription is posted.
Estimation: 12 dev weeks
Front end
TODO: Add designs
Tasks
Add RTK Query support for irrigation prescriptions.
Add step to input irrigation prescription settings to pivot addition flow.
Display irrigation prescription settings in field details view.
Add capability to edit irrigation prescription settings.
Add irrigation prescription generation CTA and generate a prescription request on click.
Add support for new notification (translation strings).
Ensure irrigation prescription files are displayed in the Documents section.
Estimation: 6 dev weeks
TODO: Weather & soil type integrations and map enhancementes
Weekly execution plan [WIP]
Gantt diagram to be added here with proposal on how to assign resources to tasks over time to meet the deadline.