MCCN Case Study 4 - Validating gridded data products
The MCCN project is to deliver tools to assist the agricultural sector to understand crop-environment relationships, specifically by facilitating generation of data cubes for spatiotemporal data. This repository contains Jupyter notebooks to demonstrate the functionality of the MCCN data cube components.
The dataset contains input files for the case study (source_data), RO-Crate metadata (ro-crate-metadata.json), results from the case study (results), and Jupyter Notebook (MCCN-CASE 4.ipynb)
Research Activity Identifier (RAiD)
RAiD: https://doi.org/10.26292/8679d473
Case Studies
This repository contains code and sample data for the following case studies. Note that the analyses here are to demonstrate the software and result should not be considered scientifically or statistically meaningful. No effort has been made to address bias in samples, and sample data may not be available at sufficient density to warrant analysis. All case studies end with generation of an RO-Crate data package including the source data, the notebook and generated outputs, including netcdf exports of the datacubes themselves.
Case Study 4 - Validating gridded data products
Description
Compare Bureau of Meteorology gridded daily maximum and minimum temperature data with data from weather stations across Western Australia.
This is an example of comparing high-quality ground-based data from multiple sites with a data product from satellite imagery or data modelling, so that I can assess its precision and accuracy for estimating the same variables at other sites.
Data Sources
The case study uses national weather data products from the Bureau of Meteorology for daily mean maximum/minimum temperature, accessible from http://www.bom.gov.au/jsp/awap/temp/index.jsp. Seven daily maximum and minimum temperature grids were downloaded for the dates 7 to 13 April 2025 inclusive. These data can be accessed in the source_data folder in the downloaded ASCII grid format (*.grid). These data will be loaded into the data cube as WGS84 Geotiff files. To avoid extra dependencies in this notebook, the data have already been converted using QGIS Desktop and are also included in the source_data folder (*.tiff).
Comparison data for maximum and minimum air temperature were downloaded for all public weather stations in Western Australia from https://weather.agric.wa.gov.au/ for the 10 day period 4 to 13 April 2025. These are included in source_data as CSV files. These downloads do not include the coordinates for the weather stations. These were downloaded via the https://api.agric.wa.gov.au/v2/weather/openapi/#/Stations/getStations API method and are included in source_data as DPIRD_weather_stations.json.
Dependencies
- This notebook requires Python 3.10 or higher
- Install relevant Python libraries with: pip install mccn-engine rocrate
- Installing mccn-engine will install other dependencies
Overview
- Convert weather station data to point measurements (longitude, latitude, date, temperature)
- Prepare STAC metadata records for each data source (separate records for each daily minimum and maximum layer from BOM, one for all weather station minima, and one for all weather station maxima)
- Load data cube
- Visualise cube
- Calculate differences between weather station values and BOM data for each station and date
- Identify sites with extreme differences (errors) for minimum and maximum temperature
- Identify sites with low differences for minimum and maximum temperature
- Cleanup and write results to RO-Crate
Notes
- Weather stations with high differences/errors are likely to have configuration of positioning issus and should not be treated as reliable.
- Weather stations with low errors are suitable for use in local analysis.
- The generally low difference between the measured values and the BOM products indicates the level of confidence that should be applied to use of these products for analyses where local measurements are not available.
- In reality, at least some of these sites will have contributed to the BOM products, so the comparands are not truly independent.
