Automatic Differentiation-Based Full Waveform Inversion

• 👩‍💻 Introduction
• ⚡️ Installation
• 👾 Examples
  • 1. Iso-acoustic Model Tests
  • 2. Iso-elastic & VTI-elastic Model Tests
  • 3. Misfits Tests
  • 4. Optimizer Tests
  • 5. Regularization Methods
  • 6. Deep Image Prior (Earth Model Reparameterization)
  • 7. Uncertainty Estimation Using Deep Neural Networks (DNNs)
• 📝 Features
• ⚖️ LICENSE
• 🗓️ To-Do List
• 🔰 Contact

---

👩‍💻 Introduction

ADFWI is an open-source framework for high-resolution subsurface parameter estimation by minimizing discrepancies between observed and simulated seismic data. Utilizing automatic differentiation (AD), ADFWI simplifies the derivation and implementation of Full Waveform Inversion (FWI), enhancing the design and evaluation of methodologies. It supports wave propagation in various media, including isotropic acoustic, isotropic elastic, and both vertical transverse isotropy (VTI) and tilted transverse isotropy (TTI) models.

In addition, ADFWI provides a comprehensive collection of Objective functions, regularization techniques, optimization algorithms, and deep neural networks. This rich set of tools facilitates researchers in conducting experiments and comparisons, enabling them to explore innovative approaches and refine their methodologies effectively.

---

⚡️ Installation

To install the Automatic Differentiation-Based Full Waveform Inversion (ADFWI) framework, please follow these steps:

1. Ensure Prerequisites
   Before you begin, make sure you have the following software installed on your system:

   • Python 3.8 or higher: Download Python from the official website: Python Downloads.
   • pip (Python package installer).

2. Create a Virtual Environment (Optional but Recommended) It is recommended to create a virtual environment to manage your project dependencies. You can use either venv or conda. For example, using conda:

   conda create --name adfwi-env python=3.8
   conda activate adfwi-env

3. Install Required Packages

• Method 1: Clone the github Repository This method provides the latest version, which may be more suitable for your research:

  git clone https://github.com/liufeng2317/ADFWI.git
  cd ADFWI

  Then, install the necessary packages:

  pip install -r requirements.txt

• Method 2: Install via pip Alternatively, you can directly install ADFWI from PyPI:

    pip install ADFWI-Torch

4. Verify the Installation To ensure that ADFWI is installed correctly, run any examples located in the examples folder.

5. Troubleshooting If you encounter any issues during installation, please check the Issues section of the GitHub repository for potential solutions or to report a new issue.

---

👾 Examples

1. Iso-acoustic Model Tests

Model Test Name	Implemented	Example's Path	Example's Figure
Iso-acoustic Marmousi2	✅	Example-Marmousi2
Iso-acoustic FootHill	✅	Example-FootHill
Iso-acoustic SEAM-I	✅	Example-SEAM-I
Iso-acoustic Overthrust-offshore	✅	Example-Overthrust-offshore
Iso-acoustic Anomaly	✅	Example-Anomaly

2. Iso-elastic & VTI-elastic Model Tests

Model Test Name	Implemented	Example Path	Example Figure
Iso-elastic Marmousi2	✅	Example-Marmousi2
VTI-elastic Anomaly	✅	Example-Anomaly

3. Misfits Tests

It is worth mentioning that here we show the performance of different objective functions under poorer initial models, and under better initial model conditions, each objective function performs better, and the relevant results can be found in Better Initial Model.

Model Test Name	Implemented	Example Path	Example Figure
Ricker-Synthetic-Test	✅	Example-Ricker (some other tests in the same folder)
L1-norm	✅	Example-L1
L2-norm	✅	Example-L2
T-distribution (StudentT)	✅	Example-StudentT
Envelope	✅	Example-Envelope
Global Correlation (GC)	✅	Example-GC
Soft Dynamic Time Warping (soft-DTW)	✅	Example-SoftDTW
Wasserstein Distance with Sinkhorn	✅	Example-Wasserstein
Hybrid Misfit: Envelope & Global Correlation (WECI)	✅	Example-WECI

4. Optimizer Tests

Model Test Name	Implemented	Example Path	Example Figure
Stochastic Gradient Descent (SGD)	✅	Example-SGD
Average Stochastic Gradient Descent (ASGD)	✅	Example-ASGD
Root Mean Square Propagation (RMSProp)	✅	Example-RMSProp
Adaptive Gradient Algorithm (Adagrad)	✅	Example-Adagrad
Adaptive Moment Estimation (Adam)	✅	Example-Adam
Adam with Weight Decay (AdamW)	✅	Example-AdamW
Nesterov-accelerated Adam (NAdam)	✅	Example-NAdam
Rectified Adam (RAdam)	✅	Example-RAdam

5. Regularization Methods

Model Test Name	Implemented	Example Path	Example Figure
no-regularization	✅	no-regular
Tikhonov-1st Order	✅	Example-Tikhonov1
Tikhonov-2nd Order	✅	Example-Tikhonov2
Total Variation-1st Order	✅	Example-TV1
Total Variation-2nd Order	✅	Example-TV2

6. Deep Image Prior (Earth Model Reparameterization)

Model Test Name	Implemented	Example Path	Example Figure
no-regularization	✅	no-regular
2-Layer CNN	✅	Example-2LayerCNN
3-Layer CNN	✅	Example-3LayerCNN
3-Layer Unet	✅	Example-3LayerUNet
4-Layer Unet	✅	Example-4LayerUNet

7. Uncertainty Estimation Using Deep Neural Networks (DNNs)

We employ a 2-layered CNN architecture, derived from the Deep Image Prior (DIP) test described earlier, to perform uncertainty estimation. The code can be found 2-layer-CNNs-Uncertainty-Estimation.The variable p represents the dropout ratio applied during both training and inference to evaluate uncertainty.

---

📝 Features

• Multi-Wave Equation:

  • Iso-Acoustic
  • Iso-Elastic
  • VTI-Elastic
  • TTI-Elastic

• Various Objective Functions

  • L1-norm
  • L2-norm
  • Smooth-L1 norm
  • Envelope
  • Global Correlation
  • T-Distribution (StudentT)
  • Soft Dynamic Time Wrapping (SoftDTW)
  • Wasserstein Distance-based with Sinkhorn (Wassrestein)

• Various Optimization Methods

  • SGD
  • ASGD
  • RMSProp
  • Adagrad
  • Adam
  • AdamW
  • NAdam
  • RAdam

• Deep Neural Network Integration

  • DNNs Reparameterization: DNNs reparameterize the Earth model, introducing learnable regularization to improve the inversion process.
  • Dropout: Applied to assess inversion uncertainty by randomly dropping units during training, providing a measure of model robustness.
  • Multiphysics Joint Inversion (on-going): Neural networks are used to fuse data from different physical fields, enabling joint inversion for a more comprehensive and accurate Earth model.

• Resource Management

  • Mini-batch: multi-shot data can be large due to different source positions and time steps. Splitting it into mini-batches prevents loading the entire dataset into memory at once.
  • Checkpointing: a key memory-saving technique, particularly for backpropagation in FWI. Instead of storing all intermediate results, only a few checkpoints are saved. During backpropagation, missing steps are recomputed, reducing memory usage at the cost of extra computation.
  • boundary saving (on-going): methods are being developed to efficiently reduce memory usage by saving only the wavefield boundaries during forward propagation instead of the entire wavefield, allowing for their later use in backpropagation.

• Acceleration Methods

  • GPU Acceleration: Utilizes the parallel processing power of GPUs to significantly speed up computations, especially for large-scale simulations like FWI.
  • JIT(Just-in-Time): Speeds up code execution by compiling Python code into optimized machine code at runtime, improving performance without modifying the original codebase.
  • Reconstruction Using Lower-Level Language (C++) (on-going): Involves rewriting performance-critical components in C++ to leverage lower-level optimizations, resulting in faster execution times and improved overall efficiency.

• Robustness and Portability

  • Each of the method has proposed a code for testing.

---

⚖️ LICENSE

The Automatic Differentiation-Based Full Waveform Inversion (ADFWI) framework is licensed under the MIT License. This license allows you to:

• Use: You can use the software for personal, academic, or commercial purposes.
• Modify: You can modify the software to suit your needs.
• Distribute: You can distribute the original or modified software to others.
• Private Use: You can use the software privately without any restrictions.

---

🗓️ To-Do List

• Memory Optimization through Boundary and Wavefield Reconstruction

  Objective: Implement a strategy to save boundaries and reconstruct wavefields to reduce memory usage.

  Explanation: This approach focuses on saving only the wavefield boundaries during forward propagation and reconstructing the wavefields as needed. By doing so, it aims to minimize memory consumption while ensuring the accuracy of wave propagation simulations, particularly in large-scale models.

  Related Work:

  • [1] P. Yang, J. Gao, and B. Wang, RTM using Effective Boundary Saving: A Staggered Grid GPU Implementation, Comput. Geosci., vol. 68, pp. 64–72, Jul. 2014. doi: 10.1016/j.cageo.2014.04.004.
  • [2] Wang, S., Jiang, Y., Song, P., Tan, J., Liu, Z., & He, B., 2023. Memory Optimization in RNN-Based Full Waveform Inversion Using Boundary Saving Wavefield Reconstruction, IEEE Trans. Geosci. Remote Sens., 61, 1–12. doi: 10.1109/TGRS.2023.3317529.
• C++ / C-Based Forward Propagator

  Objective: Develop a forward wave propagation algorithm using C++ or C.

  Explanation: Implementing the forward propagator in lower-level languages like C++ or C will significantly enhance computational performance, particularly for large-scale simulations. The aim is to leverage the improved memory management, faster execution, and more efficient parallel computing capabilities of these languages over Python-based implementations.

• Resource Optimization for Memory Efficiency

  Objective: Reduce memory consumption for improved resource utilization.

  Explanation: The current computational framework may encounter memory bottlenecks, especially when processing large datasets. Optimizing memory usage by identifying redundant storage, streamlining data structures, and using efficient algorithms will help in scaling up the computations while maintaining or even enhancing performance. This task is critical for expanding the capacity of the system to handle larger and more complex datasets.

• Custom Input Data Management System

  Objective: Develop a tailored system for managing input data effectively.

  Explanation: A customized data management framework is needed to better organize, preprocess, and handle input data efficiently. This may involve designing workflows for data formatting, conversion, and pre-processing steps, ensuring the consistency and integrity of input data. Such a system will provide flexibility in managing various input types and scales, and it will be crucial for maintaining control over data quality throughout the project lifecycle.

• Enhanced Gradient Processing

  Objective: Implement advanced techniques for gradient handling.

  Explanation: Developing a sophisticated gradient processing strategy will improve the inversion results by ensuring that gradients are effectively utilized and interpreted. This may include techniques such as gradient clipping, adaptive learning rates, and noise reduction methods to enhance the stability and convergence of the optimization process, ultimately leading to more accurate inversion outcomes.

• Multi-Scale Inversion Strategies

  Objective: Introduce multi-scale approaches for improved inversion accuracy.

  Explanation: Multi-scale inversion involves processing data at various scales to capture both large-scale trends and small-scale features effectively. Implementing this strategy will enhance the robustness of the inversion process, allowing for better resolution of subsurface structures. Techniques such as hierarchical modeling and wavelet analysis may be considered to achieve this goal, thus improving the overall quality of the inversion results.

• Real Data Testing

  Objective: Evaluate the performance and robustness of the developed methodologies using real-world datasets. Explanation: Conducting tests with actual data is crucial for validating the effectiveness of the implemented algorithms. This will involve the following steps:

  1. Dataset Selection: Identify relevant real-world datasets that reflect the complexities of the target applications. These datasets should include diverse scenarios and noise characteristics typical in field data.

  2. Preprocessing: Apply necessary preprocessing techniques to ensure data quality and consistency. This may include data normalization, filtering, and handling missing or corrupted values.

  3. Implementation: Utilize the developed algorithms on the selected datasets, monitoring their performance metrics such as accuracy, computational efficiency, and convergence behavior.

  4. Comparison: Compare the results obtained from the implemented methods against established benchmarks or existing methodologies to assess improvements.

  5. Analysis: Analyze the outcomes to identify strengths and weaknesses, and document any discrepancies or unexpected behaviors. This analysis will help refine the algorithms and inform future iterations.

  6. Reporting: Summarize the findings in a comprehensive report, detailing the testing procedures, results, and any implications for future work.

  This actual data testing phase is essential for ensuring that the developed methodologies not only perform well in controlled environments but also translate effectively to real-world applications. It serves as a critical validation step before broader deployment and adoption.

---

🔰 Contact

Developed by Feng Liu at the University of Science and Technology of China (USTC) and Shanghai Jiao Tong University (SJTU).

The related paper Full Waveform Inversion of (An)Isotropic Wave Physics in a Versatile Deep Learning Framework is currently in preparation.

For any inquiries, please contact Liu Feng via email at: liufeng2317@sjtu.edu.cn or liufeng2317@mail.ustc.edu.cn.

@software{LiuFeng,
  author       = {Feng Liu, Haipeng Li, GuangYuan Zou and Junlun Li},
  title        = {ADFWI},
  month        = July,
  year         = 2024,
  version      = {v1.0},
}