NEMDataTools

Overview

  • NEMDataTools Documentation
    • Overview
    • Installation
      • From PyPI (Recommended)
      • From TestPyPI (Pre-releases)
      • From Source (Development)
      • Requirements
    • Quick Start
    • Core Features
    • Development Status
      • ✅ Completed Features
      • 🚧 In Progress
      • 📋 Tested Data Types
    • Documentation Structure
    • API Reference
      • Core Functions
    • License
  • NEM Data Profile
    • NEM Data Types
      • MMSDM URL Structure
      • Special Cases
    • AEMO Data Tables
    • Regional Reference Nodes
    • Data Access Frequency
    • Key Field Descriptions
      • Common Fields
      • Generation and Dispatch Fields
      • Bidding Fields
    • Data Model Relationships
    • Recent Changes
    • NEMDataTools Implementation Status
      • ✅ Fully Supported Data Types
      • ⚠️ Framework Ready (Implementation Complete, Testing Pending)
      • 📋 Configured But Not Yet Tested
    • Data Quality Considerations

Developer Guide

  • NEMDataTools Project Board
    • Phase 1: Project Setup (Milestones 1-2) ✅ COMPLETED
      • Milestone 1: Development Environment Setup ✅ COMPLETED
      • Milestone 2: Core Module Skeletons ✅ COMPLETED
    • Phase 2: Core Functionality Implementation (Milestones 3-4)
      • Milestone 3: Time Utilities and Cache Management
      • Milestone 4: Data Downloading
    • Phase 3: Data Processing (Milestones 5-6)
      • Milestone 5: Basic Data Processing
      • Milestone 6: Advanced Processing
    • Phase 4: Documentation and Examples (Milestone 7)
    • Phase 5: Quality Assurance and Release (Milestone 8)
    • Phase 6: Continuous Improvement (Ongoing)
    • Development Practices
  • NEMDataTools Implementation Status
    • Core Components
      • 1. Downloader Module
      • 2. Cache Manager ✅ IMPLEMENTED
      • 3. Time Utilities ✅ IMPLEMENTED
      • 4. Data Processor
      • 5. Data Type Handlers ✅ IMPLEMENTED
      • 6. Batch Commands ✅ IMPLEMENTED
        • Implemented Functionalities:
        • Implementation Details:
        • Example Usage:
        • Key Parameters:
        • Error Handling Strategy:
    • Development Timeline
    • Implementation Details
      • AEMO Data Access
      • Caching Strategy
      • Error Handling
    • Future Extensions
  • Quick Start Guide: Setting Up NEMDataTools with UV
    • Prerequisites
    • Step 1: Install UV
    • Step 2: Clone and Setup the Project
    • Step 3: Create and Activate Virtual Environment
    • Step 4: Setup Configuration Files
      • 1. .uv.toml
      • 2. pyproject.toml
    • Step 5: Install Dependencies with UV
    • Step 6: Implement Core Modules
    • Step 7: Run Tests
    • Step 8: Code Formatting and Linting
    • Step 9: Build Documentation
    • Automation Script
    • Next Steps
  • Using UV for Dependency Management in NEMDataTools
    • Installing UV
    • Project Setup with UV
      • 1. Create Project Structure
      • 2. Create Virtual Environment with UV
      • 3. Initialize Dependencies with UV
      • 4. UV Configuration
    • Development Workflow with UV
      • Installing New Dependencies
      • Updating Dependencies
      • Managing Specific Versions
      • Locking Dependencies
    • Integration with CI/CD
    • Performance Considerations
    • Troubleshooting
  • Adding pre-commit to NEMDataTools
    • Step 1: Update pyproject.toml to include pre-commit
    • Step 2: Install pre-commit using UV
    • Step 3: Create a pre-commit configuration
    • Step 4: Install the pre-commit hooks
    • Step 5: Run pre-commit manually (optional)
    • Using pre-commit with UV
    • Integration with your development workflow
    • Customizing pre-commit configuration
    • Commit message convention with commitizen
    • Troubleshooting pre-commit
    • Additional Tips
  • Commitizen Conventional Commit Guide
    • What is Commitizen?
    • Commit Message Format
      • Types
      • Scope
      • Description
      • Examples
    • Creating Commits with Commitizen
      • Interactive Mode
      • Conventional git commit
    • Breaking Changes
    • Referencing Issues
    • Benefits of Using Commitizen
    • Commitizen CLI Commands
    • Integration with NEMDataTools
    • Customizing Commitizen
NEMDataTools
  • Search

© Copyright 2025, Zhipeng He.

Built with Sphinx using a theme provided by Read the Docs.