Skip to content

Deprecated Apache Beam Version Causing Error in weather-dl tool. #362

@dabhicusp

Description

@dabhicusp

I encountered an issue while using the following command to download data from cds using weather-dl tool. Since this is an urgent task, I would greatly appreciate any assistance in resolving this error.

command used : python "weather_dl/weather-dl" weather-tools/config_files/*.cfg --runner DataflowRunner --project $PROJECT --region $REGION --temp_location "gs://$BUCKET/tmp/" --disk_size_gb 260 --sdk_container_image "gcr.io/*****/miniconda3-beam:weather-tools-with-aria2" --experiment use_runner_v2

I performed the above command in a fresh environment of weather-tools, and within the config_files directory, there is only one file with the following content:

[parameters]
client=cds
dataset=reanalysis-era5-complete
target_path=gs://gcp-public-data-arco-era5/raw/ERA5GRIB/HRES/Daily/{date:%%Y/%%Y%%m%%d}_hres_dve.grb2
partition_keys=
    date

[parameters.sahil]
api_url=https://cds.climate.copernicus.eu/api/v2
api_key=****:****-****-****

[selection]
class=ea
stream=oper
expver=1
type=an
levtype=ml
levelist=1/to/137
date=2023-04-01/to/2023-04-30
time=00/to/23
param=138/155

while researching the above issue I discovered that the version of Apache Beam used in tool is deprecated.

The error I got is ::
e1
e2
e3
e4
e5

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions