-
Type:
New Feature
-
Resolution: Fixed
-
Priority:
Major
-
Component/s: Cloud Integrations
-
None
Sometimes data in vendor clouds changes over time, and the current poll-based Cloud Datum Stream integration has no visibility into the changes, and so the collected data in SolarNetwork can get out-of-sync with that in the vendor's cloud.
This ticket is to add a configurable "validation" task to compliment the existing poll task. The validation task should:
- allow any number of tasks can be configured per cloud datum stream
- each task has a configurable time offset, that specifies how far back in time to validate the data at
- when the task executes, it should download a sample of data starting at the configured time offset, and compare that to what exists in SolarNetwork. If any differences are found, the data in SolarNetwork should be replaced by the new data
- the validation should continue checking data samples, looking forward in time, updating the data in SolarNetwork as long as the discovered data differs from what SolarNetwork has
- The validation task should not check past the "start date" of any associated poll task on the same stream, nor go past the current date
- It might be necessary to create auxiliary reset records on either side of any updated time range, if a mis-match in the accumulated properties is discovered. Ideally this should not be necessary, and should be avoided if possible.
The imagined use of this validation task would be to configure one or more checks on datum streams, each of increasing time offset, so that a set of rolling validation checks occur over time.