...
Starting point. I am a data provider and have made the required metaupload.yml and obs.csv files. file. Upload Formats
Digression: WDI potentially should provide tools or guidance or the option to upload a non-normalized xls or csv file. This may be more conducive to agencies existing work flows. However the toolchain we are designed here is assuming the starting point is meta.yml and obs.csvupload.yml
update: 5/17/20 added upload.csv compatible format. currently exact headers are required, no column order enforced. also added a csvextractor to convert the csv file to a set of yaml files.
2. Upload. Login into the WDI upload app. submit your metaupload.yml and obs.csv file
another option is provide a web form at this point that will generate a meta.yml file for the user.
the user could select from user defined meta.yml.templates. These templates would simply prepopulate the entry form if selected, so the user could easily override existing settings.
3. meta.yml and obs.csv upload.yaml loaded into Clowder Icebox.
...
5. WDI triggers a ST extraction.
extractor reads in metaupload.yml yaml and obs.csv and uploads records to a ST instance via the ST API.
setup a dockerized extractor image. launch a cluster of extractor containers.
