Skip to content

asterixix/entsoe-fetch-tool

BranchesTags

Repository files navigation

ENTSO-E Data Fetch Tool

A Python tool for fetching electricity market data from the ENTSO-E Transparency Platform RESTful API. This tool provides an interactive command-line interface to download various types of energy market data including load forecasts, generation data, prices, and cross-border flows.

Table of Contents

Features

  • ✅ Interactive command-line interface
  • ✅ Support for 12+ different data types (load, generation, prices, etc.)
  • ✅ Automatic data chunking for large date ranges
  • ✅ Comprehensive error handling with helpful messages
  • ✅ Data saved to organized CSV files in a data/ folder
  • ✅ Support for filtering by production type (PSR)
  • ✅ Environment variable support for API key management
  • ✅ Detailed logging of API requests and responses
  • ✅ Automatic retry logic for failed requests

Prerequisites

  • Python 3.7 or higher
  • pip (Python package installer)
  • Visual Studio Code (recommended) or any text editor

Installation

  1. Clone or download this repository:

    git clone https://github.com/yourusername/entsoe-fetch-tool.git
cd entsoe-fetch-tool

  2. Create a virtual environment (recommended):

    # Windows
python -m venv venv
venv\Scripts\activate

# macOS/Linux
python3 -m venv venv
source venv/bin/activate

  3. Install required packages:

    pip install -r requirements.txt

    If requirements.txt doesn't exist, install manually:

    pip install requests pandas python-dotenv

Getting an API Key

To access the ENTSO-E Transparency Platform API, you need a security token. Follow these steps:

  1. Register on ENTSO-E Transparency Platform:

  2. Request API Access:

    • Send an email to transparency@entsoe.eu
    • Subject: "Restful API access"
    • Body: Include the email address you used for registration
    • Example email: 
      Subject: Restful API access

Dear ENTSO-E Transparency Team,

I would like to request access to the RESTful API.
My registered email address is: your.email@example.com

Thank you.

  3. Generate Your Token:

    • After approval (usually within 1-2 business days), log in to your account
    • Go to "My Account Settings"
    • Click on "Generate a new token"
    • Copy and save your token securely

  4. Configure the Tool:

    • When you first run the tool, it will prompt you to enter your API key
    • The key will be saved in a .env file for future use
    • Alternatively, create a .env file manually: 
      ENTSOE_API_KEY=your-api-key-here

Usage

Running in Visual Studio Code

  1. Open the project folder in VS Code:

    • File → Open Folder → Select the entsoe-fetch-tool folder

  2. Open a terminal in VS Code:

    • Terminal → New Terminal
    • Or use `Ctrl+`` (backtick)

  3. Activate virtual environment (if created):

    # Windows
venv\Scripts\activate

# macOS/Linux
source venv/bin/activate

  4. Run the tool:

    python entsoe-fetch-tool.py

Basic Usage Example

$ python entsoe-fetch-tool.py

=== ENTSO-E Data Fetcher ===
Using API key: c905e7b6-e...0b2a

Do you want to change the API key? (y/N): n

=== ENTSO-E Data Fetcher ===

Available countries:
AL         AT         BA         BE         BG         BY        
CH         CZ         DE         DE_50HZ    DE_AMPRION DE_LU     
...

Enter country code (e.g., PL, DE, FR): DE

Available data types:

LOAD DATA:
1:  System total load forecast - Forecasted electricity consumption
2:  Actual total load - Real-time electricity consumption data

GENERATION DATA:
3:  Wind and solar forecast - Renewable energy predictions (limited availability)
4:  Wind and solar forecast (alt) - Alternative forecast format
5:  Actual generation per type - Real-time generation by fuel type
...

Select document type (1-12): 2

Enter start date (YYYY-MM-DD): 2024-01-01
Enter end date (YYYY-MM-DD): 2024-01-07

Fetching data...
✓ Success!
Data saved to: data/entsoe_data_DE_20240101_20240107.csv

Available Data Types

Load Data

  1. System total load forecast (A65) - Day-ahead and week-ahead load forecasts
  2. Actual total load (A70) - Real-time electricity consumption

Generation Data

  1. Wind and solar forecast (A74) - Limited availability
  2. Wind and solar forecast (alt) (A69) - Alternative format
  3. Actual generation per type (A75) - Generation by fuel type
  4. Actual generation (A73) - Detailed per-unit generation
  5. Generation forecast (A71) - Day-ahead schedules
  6. Installed capacity (A68) - Available generation capacity

Market Data

  1. Day-ahead prices (A44) - Electricity market prices

Transmission Data

  1. Transfer capacity (A61) - Cross-border transmission limits
  2. Cross-border flow (A11) - Actual electricity flows
  3. Commercial exchanges (A09) - Scheduled cross-border trades

Supported Countries

The tool supports all European countries in the ENTSO-E system:

  • Western Europe: AT, BE, CH, DE, FR, LU, NL
  • Southern Europe: ES, GR, IT, MT, PT
  • Northern Europe: DK, FI, NO, SE
  • Eastern Europe: BG, CZ, HU, PL, RO, SK
  • Baltic States: EE, LT, LV
  • Balkans: AL, BA, HR, ME, MK, RS, SI, XK
  • Others: BY, GB, IE, RU, TR, UA

Some countries have multiple zones (e.g., DE_50HZ, IT_NORD, NO_1-5).

Common Issues and Solutions

1. Error 999: No Data Available

Problem: API returns error code 999 Solution:

  • The selected data type is not available for the country/period
  • Try a different data type or country
  • Check the availability notes in the tool

2. Error 401: Unauthorized

Problem: Invalid or missing API key Solution:

  • Verify your API key is correct
  • Ensure you have API access approved by ENTSO-E
  • Check the .env file contains the correct key

3. Error 429: Rate Limited

Problem: Too many requests Solution:

  • The tool automatically waits 10 seconds
  • Avoid running multiple instances simultaneously
  • ENTSO-E limits: 400 requests per minute

4. No Data Returned

Problem: Query successful but no data points Solution:

  • Try a shorter date range (1-7 days initially)
  • Some data types have specific time limitations
  • Historical data may not be available for all periods

5. XML Parsing Errors

Problem: Cannot parse response Solution:

  • Check your internet connection
  • Verify the API endpoint is accessible
  • Report persistent issues

API Limitations

Time Range Limits

  • Most queries: Maximum 1 year
  • Generation per unit (A73): Maximum 1 day
  • Balancing data: Maximum 100 days
  • Default chunk size: 7 days (automatic splitting)

Response Limits

  • Maximum 100 documents per query
  • Large datasets may be paginated
  • Some responses delivered as ZIP files

Rate Limits

  • 400 requests per minute per user
  • Automatic rate limiting built into the tool

Data Availability

  • Not all data types available for all countries
  • Historical data coverage varies
  • Real-time data typically has 5-15 minute delay

Development

Project Structure

entsoe-fetch-tool/
├── entsoe-fetch-tool.py    # Main script
├── data/                   # Output folder (created automatically)
├── .env                    # API key configuration (create this)
├── .env.example            # Example configuration
├── .gitignore              # Git ignore file
├── README.md               # This file
├── CONTRIBUTING.md         # Contribution guidelines
├── LICENSE.md              # MIT License
├── SECURITY.md             # Security policy
└── requirements.txt        # Python dependencies

Environment Variables

  • ENTSOE_API_KEY: Your ENTSO-E API token

Debugging in VS Code

  1. Create .vscode/launch.json:

    {
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Python: Current File",
      "type": "python",
      "request": "launch",
      "program": "${file}",
      "console": "integratedTerminal"
    }
  ]
}

  2. Set breakpoints by clicking left of line numbers

  3. Press F5 to start debugging

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Security

For security concerns, please see SECURITY.md.

License

This project is licensed under the MIT License - see LICENSE.md for details.

References

Acknowledgments

This tool is based on the official ENTSO-E Transparency Platform API. ENTSO-E is the European Network of Transmission System Operators for Electricity.

About

Fetching electricity market data from the ENTSO-E Transparency Platform RESTful API

transparency.entsoe.eu/content/static_content/Static%20content/web%20api/Guide.html

Topics

python data rest-api fetch-api energy-data entso-e

Resources

Readme

License

MIT license

Contributing

Contributing

Security policy

Security policy
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Languages