Tables: curated catalog and escape hatch

Curated tables

fetch() accepts curated tables spanning four families; each is wired to its locations in every tier and filename era:

  • Prices & demand — DISPATCHPRICE, TRADINGPRICE, DISPATCHREGIONSUM, TRADINGINTERCONNECT, DISPATCHINTERCONNECTORRES

  • Generation & SCADA — DISPATCH_UNIT_SCADA, DISPATCHLOAD, ROOFTOP_PV_ACTUAL

  • Forecasts — P5MIN_REGIONSOLUTION, P5MIN_INTERCONNECTORSOLN, PREDISPATCHPRICE, PREDISPATCHREGIONSUM, PREDISPATCHLOAD

  • Bids & offers — BIDDAYOFFER_D, BIDPEROFFER_D, BIDDAYOFFER, BIDPEROFFER (plus pre-2021 TRADINGREGIONSUM for history)

List them programmatically with tables():

import nemdatatools as ndt

ndt.tables()

Inspecting availability

availability() describes where a curated table lives and any known holes in its history:

info = ndt.availability("BIDPEROFFER_D")
for gap in info["gaps"]:
    print(f"gap {gap['from']} -> {gap['to']}: {gap['reason']}")

The result names the MMSDM subdirectory and filenames in both eras, the Reports CURRENT and ARCHIVE packages, and each gap’s bounds and reason. Requesting a range that crosses a gap raises AvailabilityGapError naming the substitute table, instead of returning silently partial data.

Region filtering

Tables with a region column accept a regions= filter using the five NEM region identifiers (available as NEM_REGIONS): NSW1, QLD1, SA1, TAS1, VIC1.

prices = ndt.fetch("DISPATCHPRICE", "2026/06/01", "2026/06/07", regions=["QLD1"])

Escape hatch: any MMSDM table

Every other MMSDM table — roughly 236 of them — is reachable through fetch_mmsdm_table():

gencon = ndt.fetch_mmsdm_table("GENCONDATA", "2026/05/01", "2026/05/31")

Files are still discovered by listing-and-matching in both filename eras and all FILEnn parts are fetched, but no Reports stitching, gap checking, or de-duplication is applied — the newest data available is the newest MMSDM monthly snapshot (~6 weeks behind). The time column is auto-detected for range filtering (SETTLEMENTDATE, INTERVAL_DATETIME, or DATETIME); when none is recognised, all rows of the fetched months are returned.

Snapshot subdirectories other than the default DATA are selected with subdir= — P5MIN_ALL_DATA and PREDISP_ALL_DATA hold the complete 5-minute pre-dispatch and pre-dispatch runs.

Aggregated price and demand

fetch_price_and_demand() fetches AEMO’s aggregated 5-minute price and demand CSVs by region — a lightweight alternative to DISPATCHPRICE/DISPATCHREGIONSUM when only regional price and demand are needed:

pd_data = ndt.fetch_price_and_demand("2024/01/01", "2024/12/31", ["NSW1"])

It returns REGION, TOTALDEMAND, RRP, and PERIODTYPE indexed by SETTLEMENTDATE.