-
Notifications
You must be signed in to change notification settings - Fork 31
Home
NEMOSIS is a simple tool for creating datasets using publicly available information about the Australian National Electricity Market (NEM).
The quickest way to get started using NEMOSIS is to download the latest release of the graphical user interface as an executable file. Want to dig into the source code or use the underlying python functions in your own development? Then download the full repository.
When a query is made using NEMOSIS the relevant data files are automatically downloaded and the program builds a composite csv and feather based data cache on your computer. This means the first time you query a particular table or request data from a particular month the request will execute slowly, any repeated access to the same data will be much quicker.
After downloading the graphical user interface the first thing to do is create two new folders one for NEMOSIS to build the data cache in and one where you would like the outputs from NEMOSIS saved. When you open the interface provide the paths of these folders in the relevant fields, see worked examples.
Making a query is simple:
- Name the query
- Provide a start time and an end time. NEMOSIS filters results so that only data relevant to the dispatch of the NEM between the start and end time is returned. Note the start time is exclusive and the end time is inclusive.
- Select the table you would like to request information from. As you select different tables, the column names of the table are displayed to the right, this lets you explore what information is contained within in each table.
- Select the columns you would like included in your results. Choosing less columns limits the information returned but reduces files sizes. You should at least select the primary key columns for each table, these will appear selected when a table is chosen.
- Add filters, NEMOSIS also includes a basic filtering capability, as you select columns the option to filter some columns will also appear. Filter on a column by typing the filter value into the entry box immediately below the filter label, then press enter to add this value to the filter options and then finally select the filter options you would like applied. Filtering operates simply by matching strings, i.e only rows with string values exactly matching one of those select in filter will be returned. If no options are selected for a filter then it is not applied.
- When your happy with you query press the run queries button.
Multiple queries can be added and these will be executed sequentially down the page.
Merging or join tables is the process of matching rows of data based on specified columns, if the entries in all the specified columns match then the two rows from different tables will be merged into a single row. Two previous queries can be merged to create a new data set i.e if you wanted to investigate generator behaviour according to fuel type then you might want to merge the "DISPATCHLOAD" table with the "Generators and Scheduled Loads" table to produce a data set that contained both dispatch information and fuel type information. To find out more about how merging or join tables works researching 'inner', 'left' and 'right' joins is good place to start. Creating a merge is easy
- Name the merge
- Provide the name of the query result to put on the left side of the merge
- Provide the name of the query result to put on the right side of the merge
- Select the join type
- Add and select the columns to base the merge on from the left hand query
- Add and select the columns to base the merge on from the right hand query. Note columns will be matched according to their order in the list.