Table of Contents
...
SensorThings Entity | Description | Example: NMBGMR Aquifer Monitoring Well | |||
|---|---|---|---|---|---|
Metadata | Location | A unique coordinate or area on the surface of the earth | Location in latitude and longitude or UTM easting and Northing (UTM Zone 13, NAD83) | Street Address (possibly with associated latitude and longitude). (e.g. 3960 PRINCE ST) | Location in easting and northing (UTM NAD83 in meters) |
Thing | Some real-world thing with which one or more Sensors are associated | Well Point ID WL-0150 | Sample Pt RT236I | Point of Diversion POD Number A 00008 AS | |
Datastream | A collection of Observations about an ObservedProperty produced by a Sensor associated with a Thing | Time series, Hydrograph | Sample Results | Meter Readings (Quarterly) | |
Datastream/observationType | The type of observation, codified in the Observations and Measurements data standard. Types include Categorical (defined text), Count (integer), Measurement (continuous number), Observation (free text), and TruthObservation (True/False) | Measurement | Categorical or TruthObservation | Measurement | |
Datastream/unitOfMeasurement | A three-item definition of the unit of measurement, including its name, symbol, and link to the definition (preferably to one provided in an established ontology such as http://unitsofmeasure.org/ucum.html or http://qudt.org/) | feet (e.g. http://qudt.org/vocab/unit/FT) | TCR Result | Acre-Feet (e.g. http://qudt.org/vocab/unit/AC-FT) | |
Sensor | The procedure used to provide a Datastream. Can be a particular data recording device model, or a defined procedure followed by a human observer. If applicable, a specific instance (e.g. a sensor model and serial number) | Steel-tape measurement; Continuous acoustic sounder | 9223B-PA (https://www.standardmethods.org/doi/10.2105/SMWW.2882.194) | MCCROMETER Diversion Meter-Meter Number 17147 | |
ObservedProperty | The raw or processed phenomenon (quantitative or qualitative) being measured for the Datastream. Preferably including a link to a definition provided by an established ontology or controlled vocabulary such as the ODM2 Controlled Vocabularies or http://qudt.org/) | Depth to Water Below Ground Surface (BGS) | Analyte (e.g. Coliform (TCR) (3100)) | Mtr Amount | |
OPTIONAL: FeatureOfInterest | The real-world feature that the Observations are about. This may be different from the Thing on which the Sensor is mounted. Can include a JSON-formatted point location or a polygon or collections thereof. | Formation (e.g. https://maps.nmt.edu/maps/data/hydrograph/formation_lu) | Public Water System (head office location or service area boundary) (e.g. Albuquerque Water System PWSID NM3510701) | Water Right (set of relevant points of diversion) | |
Data | Observation | A single measurement value including the result, time values, and other metadata. Information on the ObservedProperty that was measured by what Sensor is provided by the Datastream these observations are in. Features of Interest are linked for each observation as well. Observations are linked to (collected in) Datastreams | Depth Measurement | Sample (e.g. 763391) | Meter Reading |
Observation/result | The actual measured value, with valid values defined in observationType and units defined in unitsOfMeasurement, both provided by Datastream | Depth (e.g. 337.08) | Sample Result (P (Positive/ Coliform found) A (Negative/ Coliform not found)) | Mtr Amount (e.g. 107.948) | |
Observation/phenomenonTime | The date+time (or interval) in ISO 8601 format (YYYY-MM-DDT:HH:MM:SS-Z) when the observation occured | 2019-01-31 00:00:00 | MP (Monitoring Period) (e.g. 01-01-2020 to 01-31-2020) | 1/20/2017-04/05/2017 (Quarterly period for which volume was measured) | |
OPTIONAL: Observation/resultTime | The date+time that the result was generated. May be the same as phenomenonTime | Date (e.g 01-06-2020) | 04/05/2017 (date of meter reading) | ||
OPTIONAL: Observation/validTime | The date+time interval during which the Observation can be used (often used for provisional values that are replaced by QA/QC’d observations) | ||||
OPTIONAL: Observation/resultQuality | A description of the result Quality. Will vary according to agency practice. Can use ODM2 controlled vocabulary for data quality types as a guide. | Precision (e.g. “within two hundredths of a foot”) |
...
All three options can be made to run through cloud-native stacks such as Google App Engine or through any local physical or virtual machine environment through containerization paradigms such as Docker and Kubernetes. The fastest way to set upa simple SensorThings up a simple SensorThings instance is by using Docker and Docker Compose. Docker containers are virtual environments that can run on any host operating system and contain only the software and resources necessary to run a desired application. Docker Compose is a framework to run multiple Docker containers that are linked together in a virtual network on the same host. Below is a a simple process for setting up SensorThings API on a local machine or a cloud VM using FROST-Server Docker Containers.
Install Docker, using instructions depending on the operating system (can be Windows, macOS, Ubuntu, Debian, CentOS, Fedora).
Install Docker Compose, again using instructions for your operating system
Download this FROST-Server docker-compose.yaml configuration file into a directory of your choosing (e.g. /User/Home/SensorThings). This file gives the Docker engine instructions on which pre-configured virtual environments to download from Docker Hub (in this case, a CentOS
In the console (Linux), Terminal (macOS) or PowerShell (Windows) navigate to the directory where docker-compose.yaml is, and run docker-compose up, e.g.
Code Block cd ~/User/Home/SensorThings docker-compose up
5. SensorThings can be accessed by navigating to the browser on the machine to http://localhost:8080/FROST-Server/v1.1/
Detailed documentation on how to interact with and configure FROST-Server can be found on the FROST documentation site. Additional security measures, including authentication and enabling https are best done with a proxy server such as Apache 2, Nginx, or Caddy 2.
Step 2: Uploading data to SensorThings
Regardless of which particular version of SensorThings API is set up, uploading data to the SensorThings database is the same, and involves three general steps:
Mapping the data to the SensorThings data model
Extracting the data from the Agency data system into a JSON form mapped to the data model
Uploading the extracted and JSON-formatted data into SensorThings via HTTP POST requests, as documented here
An example Python workflow for the above three steps from the NMBGMR groundwater level and quality database is available here: https://github.com/NMBGMR/WDIETL/tree/master/etl Similar workflows can be constructed with any scripting language capable of making HTTP requests, making SQL-type queries from database connections and/or importing and manipulating data from tabular data sources such as csv, tsv, and excel files. Common options include Java, Javascript, PHP, R, Shell script, etc.
...