Skip to content

Using the examples

Jump to bottom
Peter Isaac edited this page Jun 11, 2022 · 7 revisions

Using the Examples

General

There are example files for 3 OzFlux/TERN Ecosystem Processes sites (Calperum, Cumberland Plain and Loxton). They are available from GitHub, see https://github.com/OzFlux/Examples.git. See the Downloading section later on this page for instructions on how to download the example files.

These files are intended to be used as examples for using PyFluxPro. Each ZIP file contains all of the files required to complete processing the site data from ingestion (L1) to partitioning (L6), including L1 data files (as Excel workbooks), control files for all levels of processing and alternate data (AWS, ACCESS and ERA5) for gap filing meteorological data at L4. Each site highlights the different ways that PyFluxPro works.

Loxton

This site was an irrigated almond orchard in South Australia, not far from the town of Renmark. The data period is just less than 1 year but the data is of very high quality and this makes it an excellent example site. The site coordinates are -34.470328, 140.655384.

The eddy covariance system consisted of a CSAT3 sonic anemometer, an Li-7500 infra-red gas analyser (IRGA) and a CR3000 data logger. The turbulence data was processed using EddyPro to calculate the 30 minute fluxes. The EddyPro output and biomet data has been assembled into an Excel workbook, Loxton_L1.xls, in the Loxton/Data/Tower folder. The short period of the Loxton data means that it can be processed in a single block and does not need to be split into individual years.

Calperum

This site is over a regenerating mallee ecosystem in South Australia near the town of Renmark. The site details can be seen at https://www.ozflux.org.au/monitoringsites/calperum/index.html. The site coordinates are -34.002060, 140.589120.

The files for this site offer 3 example processing paths:

  1. L1 to L3 processing of data for 2016 using the typical OzFlux data path (PFP calculates fluxes from covariances at L3), see the control files in PFP_examples/Calperum/controlfiles/OxFlux and the L1 workbook in PFP_examples/Calperum/Data/OzFlux/2016/.
  2. L1 to L3 processing of data for 2016 using the output from EddyPro (fluxes calculated by EddyPro and read in at L1), see the control files in PFP_examples/Calperum/controlfiles/EddyPro/ and the L1 workbook in PFP_examples/Calperum/Data/EddyPro/2016/.
  3. L4 to L6 processing of data for 2010 to 2020 using L3 data downloaded from the OzFlux Data Portal (https://data.ozflux.org.au/portal/pub/viewColDetails.jspx?collection.id=1882712&collection.owner.id=703&viewType=anonymous). See the control files in PFP_examples/Calperum/controlfiles/ and the L3 netCDF files in PFP_examples/Calperum/Data/Tower/.

Cumberland Plain

This site is over a patch of remnant native vegetation in Western Sydney close to the Hawkesbury River. The site details can be seen at https://www.ozflux.org.au/monitoringsites/cumberlandplain/index.html. The site coordinates are -33.615280, 150.723610.

The files for this site offer an example of processing data originally from a Campbell system in 2014 with a change to a SmartFlux system in October 2018. Fluxes for the period 2014 to October 2018 were calculated by EddyPro and are read in at L1. This site also demonstrates the inclusion of a CO2 storage term calculated from measurements of the CO2 concentration at different heights within the canopy.

This is a complex site with several changes in data format over the years. It is intended to show the ability of PyFluxPro to handle this level of complexity. This includes using PyFluxPro to handle changes in statistics, for example the Campbell system outputs CO2 variance in mg^2/m^6 but the SmartFlux system use mmol^2/m^6. PyFluxPro is able to convert these units to produce a homogeneous time series despite the change in data collection system.

Downloading

You can download the example files for a site by following these steps:

  1. Go to the Examples repository on GitHub, see https://github.com/OzFlux/Examples.git.
  2. Click on the ZIP file for a site. GitHub will open a page saying it "can't show files this big".
  3. Click on the Download button at the top right corner of the window displaying this message.
  4. The file will start downloading after a delay of several seconds.

A note about folders

The control files for each site in the examples assume a particular folder structure. You're welcome to use any folder structure that makes sense to you but if you stick with the one described below then you wont have to change anything in the example control files in order to run them.

The screenshot below shows the default folder structure used in the example files.

The main points are:

  1. The example files for each site are in separate folders under the PFP_examples folder.
  2. The PFP_examples and PyFluxPro folders are at the same level.
  3. Keep your own data and control files separate from both PFP_examples and PyFluxPro. In the example shown above, your sites go in the mySites folder which is also at the same level as the PyFluxPro folder.

If you use the folder structure described above then you can use the example control files with modification.

Running the examples

We will use the level 1 (L1) processing for Loxton as a basic demonstration for how to use the examples. A more detailed description of how to use the examples can be found in the PyFluxPro wiki.

Before you start, make sure:

  1. You have installed PyFluxPro and checked that it starts correctly.
  2. You have installed the example files for Loxton using the folder structure shown above.

To run the L1 processing for Loxton, follow these steps:

  1. Open a command window (Windows) or a terminal (macOS and Linux), use cd to navigate to your PyFluxPro folder and start PyFluxPro by typing:
    1. Windows;
      1. If you installed PyFluxPro from the single file installer, just type PyFluxPro at the command line.
      2. If you installed PyFluxPro by cloning from the GitHub repository, type "python PyFluxPro" at the command line or just "pfp".
    2. macOS;
      1. If you installed PyFluxPro by uncompressing the ZIP file, just type ./PyFluxPro at the command line.
      2. If you installed PyFluxPro by cloning from the GitHub repository, type "python3 PyFluxPro" at the command line or just "./pfp".
    3. Linux;
      1. You know what to do ...
  2. When the PyFluxPro GUI opens, use the File/Open menu option to open the Loxton L1 control file (L1.txt), see the screenshot below;
  3. Once the Loxton L1 control file has loaded in the GUI, take a few minutes to review its contents. The PyFluxPro wiki entry on L1 (see https://github.com/OzFlux/PyFluxPro/wiki/Level-1) has a description of the L1 control file contents. Below is a screenshot of the Loxton L1 control file loaded into the GUI.
  4. If you have followed the folder structure suggested above then this control file will run without changes. If you have used a different folder structure, you will need to change the file_path entry in the Files section so that is refers to the folder containing the Loxton L1 workbook. A right click on the file_path entry (in the Value column) will allow you to browse to the appropriate folder.
  5. To run the L1 processing for Loxton, use the Run/Current menu option, see below:
  6. If the Loxton L1 processing runs successfully, you should see the output below:

That's it, you're all done for Loxton L1. Now try the same process for Loxton L2 to L3. Then try Utilities/Climatology and Utilities/u threshold/CPD (McHugh)* to run the climatology (needed for L4) and the CPD u* threshold detection routine (needed for L5). Then run L4 to L6. Try the other example sites, they all do something different that displays the various features of PyFlyxPro.

The PyFluxPro Wiki, see https://github.com/OzFlux/PyFluxPro/wiki, has more information on the processing.

  1. Home
  2. FAQ
  3. Installing PyFluxPro
    1. General
    2. Single File Installers
    3. Installing from GitHub
    4. Running PyFluxPro
    5. Updating PyFluxPro
    6. Removing PyFluxPro
  4. Using the Examples
  5. Overview of PyFluxPro
    1. General
    2. Processing Levels
    3. Control Files
    4. Typical Workflow using PyFluxPro
    5. A Note About Time
  6. The PyFluxPro GUI
  7. Editing control files
  8. Variable names and attributes
    1. Introduction
    2. Variable naming: rules
    3. Variable attributes: units, long_name and standard_name
    4. Variable attributes: instrument
  9. Quality control flag definitions
  10. Level 1
    1. Overview
    2. The L1 Control File
      1. Files Section
      2. Global Section
        1. A note about time_zone
      3. Variables Section
    3. Running L1
    4. Output from Running L1
  11. Level 2
    1. Overview
    2. The Quality Control Checks
    3. The L2 Control File
      1. The Files Section
      2. The Options Section
      3. The Variables Section
      4. The Plots Section
    4. Running L2
    5. Output from Running L2
  12. Level 3
    1. Overview
    2. A Little History
    3. L3 Processing Steps
    4. The L3 Control File
      1. The Files Section
      2. The Options Section
      3. The Soil Section
      4. The Massman Section
      5. The Variables Section
      6. The Plots Section
    5. Running L3
    6. Output from Running L3
  13. Level 4
    1. Overview
    2. Background
    3. The L4 Control File
      1. Files Section
      2. Options Section
      3. Drivers Section
    4. Running L4
    5. Output from Running L4
  14. Level 5
    1. Overview
    2. Background
    3. The L5 Control File
      1. Files Section
      2. Imports Section
      3. Options Section
      4. u* Threshold Section
      5. Fluxes Section
    4. Running L5
    5. Output from Running L5
  15. Level 6
Clone this wiki locally