A decision theater application for exploring water management scenarios in the Yellow River Basin (YRB), combining Python-based data processing with a React frontend for interactive visualization.
- 中文用户手册 - 完整的中文使用指南
- English User Guide - Complete English usage guide
For end users, please refer to the user guides above. The technical documentation below is for developers and system administrators.
注意: 本项目使用 Google Release Please 进行自动化版本管理。版本号由提交消息自动生成，请勿手动修改。
当前版本: v1.2.0 → v2.0.0 (待发布)
- ✅ Page 1: Introduction - Yellow River Basin overview with river analysis bubble charts
- ✅ Page 2: Climate Change Impact Analysis - Complete real data integration with SNWTP toggle
- ✅ Page 3: Demographics & Domestic Water - Multi-scenario analysis with peak year detection
- ✅ Page 4: Water Demand Analysis - Irrigation and production water demand with parameter sliders
- ✅ Page 5: Water Composition Analysis - Water demand composition trends and total demand analysis
- ✅ Page 6: Ecological Water Flow - Threshold comparison with SNWTP impact analysis
- ✅ Page 7: Water Stress Index Analysis - Global scenario selection and water stress monitoring with threshold monitors
- 🔧 Complete Real Data Integration: All 7 pages now use actual backend data
- 🌐 Global Scenario Selection: Moved to Page 7 for better user flow
- 📊 Now vs Future Comparison Panels: Added comparison panels for Pages 5 & 6
- 🎨 Optimized UI: Improved navigation width and unified page labeling
- 📈 Advanced Analytics: Multi-scenario uncertainty visualization with confidence intervals
- 🧹 Code Cleanup: Removed redundant components and optimized file structure
┌─────────────────────────────────────────────────────────────────┐
│ Frontend (viz/) │
│ React + Vite + Plotly.js │
│ • Interactive charts and parameter controls │
│ • 7 themed pages (Study Area, Water Availability, etc.) │
└──────────────────────────┬──────────────────────────────────────┘
│ HTTP/JSON
↓
┌─────────────────────────────────────────────────────────────────┐
│ Backend API (scripts/api_server.py) │
│ FastAPI + Polars │
│ • GET /variables, /params, /time │
│ • POST /resolve_scenario │
│ • GET /series (variable × scenario → time series) │
└──────────────────────────┬──────────────────────────────────────┘
│ Reads
↓
┌─────────────────────────────────────────────────────────────────┐
│ Data Layer (data_parquet/) │
│ Columnar Parquet files │
│ • time.parquet: [step, time] │
│ • scenarios.parquet: [scenario_name, param1..paramN, SNWTP] │
│ • {variable}.parquet: [scenario_name, step, value, variable] │
└─────────────────────────────────────────────────────────────────┘
Located in data/:
TIME.csv: 4725 base scenarios × 1905 timesteps (years)scenario_combinations3.xlsx: Parameter combinations → 4725 base scenario rows- 12 variable CSVs (e.g.,
Total population.csv,YRB available surface water.csv): each 4725×1905 - Note: SNWTP (South-North Water Transfer Project) parameter is added during preprocessing, doubling scenarios to 9450
Generated by scripts/preprocess_to_parquet.py → data_parquet/:
┌──────┬──────────┐
│ step │ time │
├──────┼──────────┤
│ 0 │ 1981.0 │
│ 1 │ 1982.0 │
│ ... │ ... │
│ 1904 │ 2885.0 │
└──────┴──────────┘
┌───────────────┬─────────────────────┬──────────────────────────────────────────┬────────┬─────┐
│ scenario_name │ Fertility Variation │ water saving irrigation efficiency ratio │ SNWTP │ ... │
├───────────────┼─────────────────────┼──────────────────────────────────────────┼────────┼─────┤
│ sc_0 │ 1.6 │ 0.8 │ 0 │ ... │
│ sc_0_snwtp │ 1.6 │ 0.8 │ 1 │ ... │
│ sc_1 │ 1.6 │ 0.8 │ 0 │ ... │
│ sc_1_snwtp │ 1.6 │ 0.8 │ 1 │ ... │
│ ... │ ... │ ... │ ... │ ... │
│ sc_4724 │ 1.8 │ 1.0 │ 0 │ ... │
│ sc_4724_snwtp │ 1.8 │ 1.0 │ 1 │ ... │
└───────────────┴─────────────────────┴──────────────────────────────────────────┴────────┴─────┘
Total scenarios: 9450 (4725 base × 2 SNWTP options)
┌───────────────┬──────┬────────────┬──────────────────┐
│ scenario_name │ step │ value │ variable │
├───────────────┼──────┼────────────┼──────────────────┤
│ sc_0 │ 0 │ 450000000 │ Total population │
│ sc_0 │ 1 │ 451200000 │ Total population │
│ sc_0_snwtp │ 0 │ 450000000 │ Total population │
│ sc_0_snwtp │ 1 │ 451200000 │ Total population │
│ ... │ ... │ ... │ ... │
└───────────────┴──────┴────────────┴──────────────────┘
Each variable file contains ~18M rows (9450 scenarios × 1905 steps).
http://127.0.0.1:8000 (default)
Returns list of available variables (excludes TIME).
[
"GDP per capita",
"Total population",
"YRB available surface water",
...
]Returns parameter names and their unique values.
{
"Fertility Variation": [1.6, 1.7, 1.8],
"water saving irrigation efficiency ratio": [0.8, 0.9, 1.0],
...
}Resolve scenario name from parameter values.
// Request
{
"values": {
"Fertility Variation": 1.6,
"water saving irrigation efficiency ratio": 0.8,
...
}
}
// Response
{
"scenario_name": "sc_0"
}Returns time vector.
[
{"step": 0, "time": 1981.0},
{"step": 1, "time": 1982.0},
...
]Returns time series for a variable under a scenario.
{
"variable": "Total population",
"scenario": "sc_0",
"series": {
"time": [1981.0, 1982.0, ...],
"value": [450000000, 451200000, ...]
}
}New in v2.0: Returns aggregated time series for multiple scenarios matching filters.
// Request
GET /series/multi?variable=YRB%20available%20surface%20water
&filters={"Climate change scenario switch for water yield": [1,2,3], "SNWTP": 0}
&start_year=2020&end_year=2100&aggregate=true
// Response
{
"variable": "YRB available surface water",
"n_scenarios": 15,
"isSingleScenario": false,
"filter_summary": {
"Climate change scenario switch for water yield": [1, 2, 3],
"SNWTP": 0
},
"series": {
"time": [2020.0, 2021.0, ...],
"mean": [55.2, 56.1, ...],
"min": [48.3, 49.0, ...],
"max": [62.8, 63.5, ...],
"std": [4.2, 4.3, ...],
"p05": [50.1, 50.8, ...],
"p95": [60.3, 61.1, ...]
}
}Features:
- Filters scenarios by parameter constraints
- Supports multi-value filters (e.g., climate scenarios [1,2,3])
- Returns statistical aggregation (mean, min, max, std, p05, p95)
- Automatically detects single vs. multi-scenario mode
- SNWTP parameter supported (0=off, 1=on)
New in v2.0: Returns RCP/SSP climate scenario data (temperature and precipitation).
{
"temperature": {
"ssp126": {
"scenario": "ssp126",
"variable": "Temperature",
"values": [12.3, 12.4, ...]
},
"ssp245": {...},
"ssp585": {...}
},
"precipitation": {
"ssp126": {
"scenario": "ssp126",
"variable": "Precipitation",
"values": [520.5, 522.1, ...]
},
"ssp245": {...},
"ssp585": {...}
},
"years": [1981, 1982, ..., 2100]
}Use case: Climate impact analysis and scenario comparison panels.
viz/
├── src/
│ ├── main.tsx # Entry point
│ ├── App.tsx # Main app with 7-page tabs
│ ├── components/
│ │ ├── charts/
│ │ │ ├── PlotlyChart.tsx # Reusable Plotly wrapper (with dark mode)
│ │ │ └── BasicChart.tsx # Fallback chart component
│ │ ├── pages/
│ │ │ ├── EcologicalWaterPageSlider.tsx # ✅ Page 6 (fully integrated)
│ │ │ ├── WaterAvailabilityPage.tsx # ✅ Page 2 (fully integrated) **NEW**
│ │ │ ├── StudyAreaPage.tsx # Page 1 (static design)
│ │ │ ├── DemographicsPage.tsx # Page 3 (static design)
│ │ │ ├── AgriculturePage.tsx # Page 4 (static design)
│ │ │ ├── WaterStressPage.tsx # Page 5 (static design)
│ │ │ └── WaterQualityPage.tsx # Page 7 (static design)
│ │ ├── DataComparisonPanel.tsx # **NEW**: Climate impact comparison component
│ │ ├── GlobalParameterPanel.tsx # Global scenario parameter controls
│ │ └── ui/ # shadcn/Radix UI components
│ ├── contexts/
│ │ └── ScenarioContext.tsx # Global state management for scenarios
│ ├── services/
│ │ └── api.ts # API client functions
│ ├── hooks/
│ │ ├── useApiData.ts # Data fetching hooks
│ │ └── useClimateComparison.ts # **NEW**: Climate data comparison hook
│ └── styles/
│ └── globals.css # Global styles
├── BACKEND_INTEGRATION_GUIDE.md # 📚 Standard integration guide
├── package.json
├── vite.config.ts
└── index.html
Reusable Plotly component with dark mode support:
<PlotlyChart
id="unique-chart-id"
title="Chart Title"
height="400px"
data={plotlyData} // Plotly.Data[]
layout={plotlyLayout} // Partial<Plotly.Layout> (auto dark mode)
config={{ responsive: true, displaylogo: false }}
/>Complete backend integration with slider controls:
- Real-time parameter adjustment (ecological flow: 0.2, 0.25, 0.3)
- Multi-variable support (surface water, discharge, sediment)
- Dark mode adaptive charts
- Statistical analysis panel
- 2020-2100 time range focus
Climate change scenario integration with advanced features:
- Climate Scenario Selection: RCP2.6, RCP4.5, RCP8.5 pathways
- SNWTP Toggle: Local parameter for South-North Water Transfer Project (only affects surface water)
- Multi-Scenario Aggregation: Displays mean values with uncertainty bands (min/max) when multiple scenarios match
- Single-Scenario Mode: Shows individual scenario line when filters result in one scenario
- Climate Impact Panel (
DataComparisonPanel):- Temperature change (2020 → 2100)
- Precipitation change (2020 → 2100)
- Water availability change (2020 → 2100)
- Visual indicators for positive/negative trends
- Real Data Integration: All charts use actual scenario data from backend
- Responsive Layout: Auto-sizing panels without scrollbars
Technical Features:
- Custom data fetching with SNWTP parameter (independent of global state)
useClimateComparisonhook for climate data analysis- Automatic detection of single vs. multi-scenario display
- Uncertainty visualization using Plotly fill areas
StudyAreaPage.tsx: Geographic overview with map placeholderDemographicsPage.tsx: Population and demographic trendsAgriculturePage.tsx: Agricultural water use and irrigationWaterStressPage.tsx: Water stress indicators and vulnerabilityWaterQualityPage.tsx: Water quality parameters and pollution
Advanced multi-scenario analysis with climate impact visualization
- Climate Pathway Selection: Interactive RCP scenario switching (RCP2.6, RCP4.5, RCP8.5)
- SNWTP Local Parameter: Independent toggle for South-North Water Transfer Project
- Only affects surface water availability
- Works alongside global parameters
- Demonstrates local parameter pattern for page-specific controls
- Intelligent Scenario Display:
- Single Scenario Mode: Shows one line when filters yield unique scenario
- Multi-Scenario Mode: Shows mean ± uncertainty bands (min/max) when multiple scenarios match
- Automatic detection with scenario count display
- Climate Impact Dashboard: Real-time calculation of temperature, precipitation, and water availability changes (2020→2100)
- Responsive Design: Self-adapting layout without scrollbars
- ✅ Multi-Scenario Aggregation:
/series/multiAPI with statistical bands - ✅ Climate Data Integration:
/climate-dataendpoint for RCP/SSP data - ✅ Custom Data Fetching: Local SNWTP parameter bypasses global state
- ✅ Uncertainty Visualization: Plotly fill areas for confidence intervals
- ✅ Comparative Analysis:
useClimateComparisonhook for future projections - ✅ Layout Optimization: Flexbox-based auto-sizing panels
Real-time parameter control with statistical analysis
- Real-time Parameter Control: Slider adjusts ecological flow (0.2, 0.25, 0.3)
- Multi-Variable Analysis: Switch between surface water, discharge, and sediment data
- Dark Mode Support: Charts automatically adapt to theme changes
- Statistical Dashboard: Mean, max, min values with corresponding years
- 2020-2100 Focus: Optimized for climate projection analysis
- ✅ API Integration: Full backend connectivity with error handling
- ✅ State Management: Real-time updates with React hooks
- ✅ Chart Rendering: Plotly.js with responsive design
- ✅ Theme Adaptation: Automatic dark/light mode switching
- ✅ Performance: Optimized data loading and caching
These implementations demonstrate reusable patterns:
- Global Parameters (Page 6): Parameter changes affect all pages
- Local Parameters (Page 2 SNWTP): Page-specific controls for targeted analysis
- Multi-Scenario Analysis (Page 2): Statistical aggregation across matching scenarios
- Climate Data Integration (Page 2): External dataset integration for comparative analysis
Convert raw CSVs to Parquet (one-time setup):
make preprocessThis generates data_parquet/ from data/ + scenario_combinations3.xlsx.
make api
# Runs on http://127.0.0.1:8000
# Visit http://127.0.0.1:8000/docs for interactive API docscd viz
npm install
npm run dev
# Opens http://localhost:3000 (or 3001/3002 if ports busy)- Navigate to Page 6 (Ecological Water) for full backend integration demo
- Features: Slider controls, variable switching, dark mode, real-time updates
- Check browser Console for API logs and Network tab for
/resolve_scenariocalls
# Build frontend first
cd viz && npm run build && cd ..
# Deploy to server
./deploy-v2.sh
⚠️ 部署注意事项: 如果部署后浏览器显示旧版本，请查看 Deployment Best Practices 章节了解如何诊断和修复！
- Modify backend endpoints in
scripts/api_server.py - Update data processing in
scripts/preprocess_to_parquet.py - Keep backend running:
make api
- Works in
viz/src/(all source code tracked in Git) - Adjusts layouts, styles, UI components
- Runs
npm run devlocally to preview changes
# You commit API + data processing changes
git add scripts/ data_parquet/ pyproject.toml makefile
git commit -m "feat: add new variable endpoint"
# Designer commits frontend changes
cd viz
git add src/
git commit -m "style: update page 3 layout"
# Both push/pull to sync
git push origin master
git pull origin masterWe generate and use "safe variable names" for filenames and APIs while preserving original human-readable names:
- Safe name rules: lowercase; non-alphanumeric →
_; collapse multiple_; trim edges. - Parquet filenames use safe names, e.g.
data_parquet/yrb_available_surface_water.parquet. - Mapping file:
data_parquet/variables_map.json(original → safe). - API exposes
GET /variables_mapand accepts either original or safe name in/series.
Examples (original → safe):
hydrologic station discharge[lijin] → hydrologic_station_discharge_lijin
YRB available surface water → yrb_available_surface_water
sediment load[lijin] → sediment_load_lijin
irrigation water demand province sum → irrigation_water_demand_province_sum
production water demand province sum → production_water_demand_province_sum
OA water demand province sum → oa_water_demand_province_sum
domestic water demand province sum → domestic_water_demand_province_sum
GDP per capita → gdp_per_capita
Total population → total_population
YRB WSI → yrb_wsi
water consumption of province in YRB sum → water_consumption_of_province_in_yrb_sum
Frontend/Notebook can display original names while using safe names for stable keys.
Auto-generated as sc_0 to sc_4724 based on parameter combinations.
# pyproject.toml
python = ">=3.11"
polars = "^1.31.0"
fastapi = "^0.115.0"
uvicorn = {extras = ["standard"], version = "^0.30.6"}
pyarrow = "^21.0.0"Install: poetry install
// viz/package.json
"react": "^18.3.1"
"plotly.js-dist-min": "^2.35.2"
"@radix-ui/react-*": "^1.x" (shadcn/ui)
"vite": "6.3.5"Install: cd viz && npm install
- Check backend is running: visit
http://127.0.0.1:8000/variables - Check browser Console for API logs and error messages
- Check Network tab for
/params,/resolve_scenario,/seriesrequests - Verify parameter names match backend API exactly (case-sensitive)
- Hard refresh:
Cmd+Shift+R(macOS) orCtrl+Shift+R(Windows/Linux)
- Charts automatically adapt to theme changes
- If charts don't update: check
isDarkMode()function and theme listeners
- Backend: Change port in
makefileor runpoetry run uvicorn scripts.api_server:app --port 8001 - Frontend: Vite auto-increments (3000→3001→3002...), or set in
vite.config.ts
Backend already allows all origins (allow_origins=["*"]). If still blocked, check browser console for exact error.
重要: 如果你更新了代码但远端网站没有变化，请参考本章节！
如果远端部署后浏览器显示的还是旧版本：
# 1. 运行诊断脚本（自动检查所有可能的问题）
./diagnose-deployment.sh
# 2. 运行修复脚本（自动修复常见问题）
./fix-deployment.sh症状：
- 已经运行了
deploy-v2.sh - 服务器上的文件已更新
- 浏览器（包括无痕模式）显示的还是旧版本
根本原因：
- Dockerfile 配置: 项目使用
Dockerfile.frontend，它会在 Docker 容器内从源代码重新构建前端 - 缓存机制: Docker 可能使用缓存层，导致新代码没有被打包进镜像
- 源代码未更新: 如果服务器上的源代码是旧的，即使重新构建也会生成旧版本
正确的部署流程：
# 方法 1: 使用修复后的部署脚本（推荐）
./deploy-v2.sh # 现在会自动上传源代码
# 如果上面的方法不生效，使用强制修复脚本
./fix-deployment.sh
# 方法 2: 手动强制重建（适用于紧急情况）
ssh ubuntu@43.165.1.18 'cd /home/ubuntu/watm-dt && \
docker compose down && \
docker compose build --no-cache frontend && \
docker compose up -d'检查文件 Hash（推荐）：
# 本地构建的 JS 文件名
grep -o 'src="[^"]*\.js"' viz/build/index.html | grep "/assets/"
# 远端服务器返回的 JS 文件名
curl -s http://43.165.1.18/ | grep -o 'src="[^"]*\.js"' | grep "/assets/"
# 两者的 hash（文件名中的随机字符串）应该一致
# 例如: index-DpJTYsGo.js浏览器验证：
- 访问 http://43.165.1.18
- 按
Cmd+Shift+R(macOS) 或Ctrl+Shift+R(Windows) 强制刷新 - 打开开发者工具 (F12) → Network 标签
- 查看加载的 JS 文件名，确认 hash 值是否是新的
-
本地必须先构建前端：
cd viz npm run build cd ..
-
Docker 使用源代码构建：项目的
Dockerfile.frontend会在容器内重新构建，所以必须上传最新的源代码，而不仅仅是构建产物。 -
使用 --no-cache 标志：如果怀疑 Docker 使用了缓存，使用
--no-cache强制重建：docker compose build --no-cache frontend
-
检查服务器上的源代码时间：
ssh ubuntu@43.165.1.18 'ls -lh /home/ubuntu/watm-dt/viz/src/App.tsx'确认文件修改时间是最新的。
项目提供了完整的诊断和修复工具：
diagnose-deployment.sh: 自动诊断所有可能的问题（不做修改）fix-deployment.sh: 自动修复常见的部署问题TROUBLESHOOTING_DEPLOYMENT.md: 详细的故障排查文档
项目使用的是多阶段构建的 Dockerfile：
# Dockerfile.frontend
FROM node:18-alpine AS builder
COPY viz/ ./ # 复制源代码
RUN npm run build # 在容器内构建
FROM nginx:alpine
COPY --from=builder /app/build /usr/share/nginx/html优点：
- 最终镜像很小（只包含构建产物）
- 构建环境和运行环境分离
- 不需要在服务器上安装 Node.js
缺点：
- 必须上传源代码（不能只上传构建产物）
- Docker 缓存可能导致更新不生效
- 需要使用
--no-cache确保完全重建
- 部署指南: DEPLOY.md
- 详细故障排查: TROUBLESHOOTING_DEPLOYMENT.md
- 完整部署文档: docs/DEPLOYMENT_GUIDE.md
Use scripts/query_scenarios.py to filter and extract data for analysis:
from scripts.query_scenarios import ScenarioQuery, quick_query, compare_params
# Initialize query engine
query = ScenarioQuery("data_parquet")
# List available variables and parameters
query.list_variables()
query.param_cols
# Filter scenarios by parameter constraints
filtered = query.filter_scenarios({
"Fertility Variation": 1.6,
"Climate change scenario switch for water yield": 1
})
print(f"Matching scenarios: {filtered.height}")
# Get time series for variable(s) under constraints
data = query.get_series(
variables=["Total population", "YRB WSI"],
filters={"Fertility Variation": 1.6},
time_range=(2020, 2050)
)
# Compare parameter impact (wide format for plotting)
comparison = compare_params(
variable="YRB WSI",
fixed_params={"Fertility Variation": 1.6, "Diet change scenario switch": 1},
vary_param="water saving irrigation efficiency ratio",
time_range=(2020, 2100)
)
# Result: rows=time, columns=parameter values (0.8, 0.9, 1.0)See docs/example_usage.ipynb for complete examples.
| Method | Purpose | Returns |
|---|---|---|
filter_scenarios(filters) |
Get scenarios matching parameter constraints | DataFrame of scenarios |
get_series(variables, filters, time_range) |
Get time series in long format | DataFrame [scenario, variable, step, time, value, params...] |
get_series_wide(variable, filters, columns_col) |
Get time series pivoted for comparison | DataFrame [time, param_val_1, param_val_2, ...] |
list_variables() |
List all available variables | List of strings |
get_param_summary(filters) |
Summarize parameter distributions | DataFrame of parameter stats |
- Apply Integration Pattern: Use
viz/BACKEND_INTEGRATION_GUIDE.mdto integrate remaining 5 pages - Standard Template: Copy
EcologicalWaterPageSlider.tsxas starting point for each page - Parameter Mapping: Each page focuses on different parameter combinations
- Add batch endpoints:
GET /series/batchfor multiple variables/scenarios - Implement downsampling (LTTB algorithm) for large time series
- Add statistical bands (p10/p90) for scenario ensembles
- ✅ Completed: Ecological Water page with full backend integration
- Next: Apply integration pattern to remaining 5 pages using
BACKEND_INTEGRATION_GUIDE.md - Standard Process: Copy
EcologicalWaterPageSlider.tsx→ modify parameters → update variables - Add multi-variable comparison (overlay multiple series)
- Implement export (PNG/CSV download)
MIT
Maintained by SongshGeo songshgeo@gmail.com
Project Repository: https://github.com/SongshGeoLab/YR-WatM-DT
For issues or questions, please visit the repository or open an issue.
