wps_update_metadata¶

wps_update_metadata is a process that runs the update_metadata module of PCIC Climate Explorer Data Preparation Tools. Here, the client will try to connect to a remote Thunderbird instance using the url parameter.¶

In [1]:
from birdy import WPSClient
from importlib import resources
import os
from wps_tools.testing import get_target_url
from netCDF4 import Dataset

# Ensure we are in the working directory with access to the data
while os.path.basename(os.getcwd()) != "thunderbird":
    os.chdir('../')
In [2]:
# NBVAL_IGNORE_OUTPUT
url = get_target_url("thunderbird")
print(f"Using thunderbird on {url}")
Using thunderbird on https://marble-dev01.pcic.uvic.ca/twitcher/ows/proxy/thunderbird/wps
In [3]:
thunderbird = WPSClient(url)

Help for individual processes can be diplayed using the ? command (ex. bird.process?).¶

In [4]:
# NBVAL_IGNORE_OUTPUT
thunderbird.update_metadata?
Signature:
thunderbird.update_metadata(
    netcdf=None,
    loglevel='INFO',
    updates_file=None,
    updates_string=None,
)
Docstring:
Update file containing missing, invalid, or incorrectly named global or variable metadata attributes

Parameters
----------
netcdf : ComplexData:mimetype:`application/x-netcdf`, :mimetype:`application/x-ogc-dods`
    NetCDF file
updates_file : ComplexData:mimetype:`text/x-yaml`
    The filepath of an updates file that specifies what to do to the metadata it finds in the NetCDF file
updates_string : string
    The string in yaml format that specifies what to do to the metadata it finds in the NetCDF file
loglevel : {'CRITICAL', 'ERROR', 'WARNING', 'INFO', 'DEBUG', 'NOTSET'}string
    Logging level

Returns
-------
output : ComplexData:mimetype:`application/x-netcdf`
    Output Netcdf File
File:      ~/code/thunderbird/</home/eyvorchuk/.cache/pypoetry/virtualenvs/thunderbird-7g6X3rbj-py3.10/lib/python3.10/site-packages/birdy/client/base.py-2>
Type:      method

We can use the docstring to ensure we provide the appropriate parameters.¶

In [5]:
# run update_metadata with yaml text and opendap netcdf inputs
updates_string = '''
global:
    - institute_id:
    - institute_ID: PCIC
    - address: University House 1 
    - Institution: <- institution
'''

daccs_host = os.getenv("DACCS_HOST", "marble-dev01.pcic.uvic.ca")
opendap_netcdf = f"https://{daccs_host}/twitcher/ows/proxy/thredds/dodsC/datasets/storage/data/projects/comp_support/daccs/test-data/tiny_hydromodel_gcm_climos.nc"
opendap_output = thunderbird.update_metadata(updates_string = updates_string, netcdf = opendap_netcdf)
opendap_output_data = opendap_output.get(asobj=True)[0]
In [6]:
opendap_output_data = opendap_output.get(asobj=True)[0]
In [7]:
# run update_metadata with yaml file and local netcdf inputs
ref_updates = resources.files('tests') / 'metadata-conversion/simple-conversion.yaml'
ref_netcdf = resources.files('tests') / 'data/tiny_hydromodel_gcm_climos.nc'
with resources.as_file(ref_updates) as updates_file, resources.as_file(ref_netcdf) as local_netcdf:
    local_output = thunderbird.update_metadata(updates_file = updates_file, netcdf = local_netcdf)
    local_output_data = local_output.get(asobj=True)[0]

Once the process has completed we can extract the results and ensure it is what we expected.¶

In [8]:
input_data = Dataset(f"https://{daccs_host}/twitcher/ows/proxy/thredds/dodsC/datasets/storage/data/projects/comp_support/daccs/test-data/gdd_annual_CanESM2_rcp85_r1i1p1_1951-2100.nc")

# Expected output metadata determined from the input yaml file
# global:
#     - institute_id:
#     - institute_ID: PCIC
#     - address: University House 1 
#     - Institution: <- institution
expected = {
    "institute_ID": "PCIC",
    "address": "University House 1",
    "Institution": input_data.institution,
}
In [9]:
def test_metadata(output_data):
    # updated metadata
    metadata = {
        "institute_ID": output_data.institute_ID,
        "address": output_data.address,
        "Institution": output_data.Institution,
    }

    assert metadata == expected
In [10]:
test_metadata(opendap_output_data)
test_metadata(local_output_data)