Heat Transfer and Diffusion Equations Using Physics-Informed Neural Networks

Author: Abdo Al-atrash

Table of Contents

1. Introduction
2. Heat Transfer and Diffusion Equations
3. Physics-Informed Neural Networks (PINNs)
4. Implementing PINNs for Heat Transfer and Diffusion
   • 1. Setting Up the Environment
   • 2. Defining the PINN Model
   • 3. Preparing the Data
   • 4. Training the Model
   • 5. Evaluating the Model
5. Advantages and Challenges
6. Resources and Further Reading

---

Introduction

Heat transfer and diffusion are fundamental processes in physics and engineering, describing how thermal energy and substances spread through different media. Accurately modeling these processes is crucial for applications ranging from material design to environmental engineering.

Traditional numerical methods, such as the Finite Element Method (FEM) or Finite Difference Method (FDM), are commonly used to solve the governing Partial Differential Equations (PDEs). However, these methods can be computationally intensive, especially for high-dimensional problems or complex geometries. Physics-Informed Neural Networks (PINNs) offer an alternative by integrating neural networks with physical laws, potentially reducing computational costs and improving scalability.

---

Heat Transfer and Diffusion Equations

Heat Transfer

Heat transfer involves the movement of thermal energy from one location to another and can occur via three primary mechanisms:

1. Conduction: Transfer of heat through a material without the material itself moving.
2. Convection: Transfer of heat by the physical movement of fluid.
3. Radiation: Transfer of heat through electromagnetic waves.

The Heat Equation is a fundamental PDE that describes the distribution of heat (temperature) in a given region over time:

$$ \frac{\partial T}{\partial t} = \alpha \nabla^2 T + \frac{Q}{\rho c_p} $$

• $T$: Temperature
• $\alpha$: Thermal diffusivity
• $Q$: Heat source term
• $\rho$: Density
• $c_p$: Specific heat capacity

Diffusion

Diffusion describes the process of a substance spreading out to evenly fill its container or environment. The Diffusion Equation is analogous to the heat equation:

$$ \frac{\partial C}{\partial t} = D \nabla^2 C $$

• $C$: Concentration of the diffusing substance
• $D$: Diffusion coefficient

Both equations are parabolic PDEs and share similar mathematical structures, making PINNs applicable to both.

---

Physics-Informed Neural Networks (PINNs)

PINNs are a class of neural networks that incorporate physical laws, expressed as PDEs, into the loss function during training. This integration ensures that the neural network solutions adhere to the underlying physics.

Key Features

• Data-Driven and Physics-Based: Combines observational data with physical laws.
• Flexible and Mesh-Free: Not restricted by grid-based methods, allowing for easier handling of complex geometries.
• Scalable: Suitable for high-dimensional problems.

Loss Function in PINNs

The loss function typically consists of:

1. Residual Loss: Ensures the PDE is satisfied.
2. Boundary and Initial Condition Losses: Ensures the solution adheres to these conditions.
3. Data Loss: Incorporates any available observational data.

$$ \text{Loss} = \text{MSE}{\text{PDE}} + \text{MSE}{\text{IC}} + \text{MSE}{\text{BC}} + \text{MSE}{\text{Data}} $$

---

Implementing PINNs for Heat Transfer and Diffusion

In this section, we'll implement a PINN to solve the 1D Heat Equation using Python and TensorFlow.

1. Setting Up the Environment

First, ensure you have the necessary libraries installed. You can install them using pip if they are not already available.

# Install necessary libraries
!pip install tensorflow numpy matplotlib

2. Defining the PINN Model

We'll create a HeatTransferPINN class that defines the neural network architecture, computes the PDE residual, and sets up the loss function.

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

  class HeatTransferPINN:

def __init__(self, layers, alpha, lb, ub):
"""
Initialize the PINN model.
Parameters:

- layers: List containing the number of neurons in each layer.
- alpha: Thermal diffusivity.
- lb: Lower bound of the domain (t, x).
- ub: Upper bound of the domain (t, x).

"""

self.alpha = alpha
self.lb = lb
self.ub = ub
self.model = self.build_model(layers)

def build_model(self, layers):
"""
Builds a feedforward neural network.
Parameters:
- layers: List containing the number of neurons in each layer.
Returns:
- model: A TensorFlow Keras Sequential model.
"""

model = tf.keras.Sequential()
for width in layers[:-1]:
model.add(tf.keras.layers.Dense(width, activation='tanh'))
model.add(tf.keras.layers.Dense(layers[-1], activation=None))
return model
def pdb_t(self, t, x):
"""
Computes the PDE residual for the heat equation.
Parameters:
- t: Tensor containing time coordinates.
- x: Tensor containing spatial coordinates.
Returns:
- residual: Tensor representing the residual of the PDE.
"""
with tf.GradientTape(persistent=True) as tape:
tape.watch([t, x])
T = self.model(tf.concat([t, x], axis=1))
T_t = tape.gradient(T, t)
T_x = tape.gradient(T, x)
T_xx = tape.gradient(T_x, x)
del tape
residual = T_t - self.alpha * T_xx
return residual
def loss_function(self, t, x, T_true, t_bc, x_bc, T_bc, t_ini, x_ini, T_ini):
"""
Computes the total loss combining PDE residual, boundary, and initial conditions.
Parameters:
- t, x: Tensors for collocation points in the domain.
- T_true: Tensor for true temperature values at collocation points.
- t_bc, x_bc, T_bc: Boundary condition tensors.
- t_ini, x_ini, T_ini: Initial condition tensors.
Returns:
- total_loss: Scalar tensor representing the total loss.
"""
# PDE residual
f = self.pdb_t(t, x)
mse_pde = tf.reduce_mean(tf.square(f))
# Boundary conditions
T_pred_bc = self.model(tf.concat([t_bc, x_bc], axis=1))
mse_bc = tf.reduce_mean(tf.square(T_pred_bc - T_bc))
# Initial conditions
T_pred_ini = self.model(tf.concat([t_ini, x_ini], axis=1))
mse_ini = tf.reduce_mean(tf.square(T_pred_ini - T_ini))
# Total loss
total_loss = mse_pde + mse_bc + mse_ini
return total_loss
def train(self, optimizer, epochs, t, x, T_true, t_bc, x_bc, T_bc, t_ini, x_ini, T_ini):
"""
Trains the PINN model.
Parameters:
- optimizer: TensorFlow optimizer instance.
- epochs: Number of training epochs.
- t, x, T_true: Collocation points and true temperature.
- t_bc, x_bc, T_bc: Boundary condition tensors.
- t_ini, x_ini, T_ini: Initial condition tensors.
"""
for epoch in range(epochs):
with tf.GradientTape() as tape:
loss = self.loss_function(t, x, T_true, t_bc, x_bc, T_bc, t_ini, x_ini, T_ini)
gradients = tape.gradient(loss, self.model.trainable_variables)
optimizer.apply_gradients(zip(gradients, self.model.trainable_variables))
if epoch % 1000 == 0:
print(f'Epoch {epoch}, Loss: {loss.numpy()}')

3. Preparing the Data

We'll define the domain, initial conditions, and boundary conditions for the 1D Heat Equation.

# Define parameters
alpha = 0.01 # Thermal diffusivity
layers = [2, 20, 20, 20, 1] # Neural network layers
lb = np.array([0, 0]) # Lower bound for t and x
ub = np.array([1, 1]) # Upper bound for t and x

  # Initialize PINN

pinn = HeatTransferPINN(layers, alpha, lb, ub)

  # Generate collocation points
N_f = 10000 # Number of collocation points
t_f = np.random.uniform(lb[0], ub[0], (N_f, 1))
x_f = np.random.uniform(lb[1], ub[1], (N_f, 1))

  # Generate boundary condition points (e.g., x=0 and x=1 for all t)
N_bc = 200
t_bc = np.random.uniform(lb[0], ub[0], (N_bc//2, 1))
x_bc_0 = np.zeros((N_bc//2, 1))
x_bc_1 = np.ones((N_bc//2, 1))
t_bc = np.vstack([t_bc, t_bc])
x_bc = np.vstack([x_bc_0, x_bc_1])
# Define boundary temperature (e.g., T=0)
T_bc = np.zeros((N_bc, 1))
  # Generate initial condition points (t=0 for all x)
N_ini = 200
x_ini = np.random.uniform(lb[1], ub[1], (N_ini,1))
t_ini = np.zeros((N_ini,1))
# Define initial temperature distribution (e.g., T=sin(pi*x))
T_ini = np.sin(np.pi * x_ini)
  # Convert all data to TensorFlow tensors
t_f = tf.convert_to_tensor(t_f, dtype=tf.float32)
x_f = tf.convert_to_tensor(x_f, dtype=tf.float32)
t_bc = tf.convert_to_tensor(t_bc, dtype=tf.float32)
x_bc = tf.convert_to_tensor(x_bc, dtype=tf.float32)
T_bc = tf.convert_to_tensor(T_bc, dtype=tf.float32)
t_ini = tf.convert_to_tensor(t_ini, dtype=tf.float32)
x_ini = tf.convert_to_tensor(x_ini, dtype=tf.float32)
T_ini = tf.convert_to_tensor(T_ini, dtype=tf.float32)

4. Training the Model

We'll train the PINN using the Adam optimizer.

# Training the model
# Define optimizer
optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
# Train the model
epochs = 10000
pinn.train(optimizer, epochs, t_f, x_f, None, t_bc, x_bc, T_bc, t_ini, x_ini, T_ini)

5. Evaluating the Model

After training, we'll evaluate the model's performance by comparing the predicted temperature distribution with the analytical solution.

# Evaluating the model
# Define the analytical solution for comparison
def analytical_solution(t, x, alpha):
return np.exp(- (np.pi**2) * alpha * t) * np.sin(np.pi * x)
# Generate a grid for evaluation
t_star = np.linspace(lb[0], ub[0], 100)
x_star = np.linspace(lb[1], ub[1], 100)
T_pred = np.zeros((len(t_star), len(x_star)))
T_true = np.zeros((len(t_star), len(x_star)))
for i in range(len(t_star)):
for j in range(len(x_star)):
t_input = tf.convert_to_tensor([[t_star[i], x_star[j]]], dtype=tf.float32)
T_p = pinn.model(t_input)
T_pred[i,j] = T_p.numpy()
T_true[i,j] = analytical_solution(t_star[i], x_star[j], alpha)
# Plot the results
X, T = np.meshgrid(x_star, t_star)
plt.figure(figsize=(12, 5))
  # Predicted Temperature
plt.subplot(1, 2, 1)
plt.contourf(X, T, T_pred, 100, cmap='viridis')
plt.colorbar()
plt.title('Predicted Temperature Distribution')
plt.xlabel('x')
plt.ylabel('t')
# Analytical Temperature
plt.subplot(1, 2, 2)
plt.contourf(X, T, T_true, 100, cmap='viridis')
plt.colorbar()
plt.title('Analytical Temperature Distribution')
plt.xlabel('x')
plt.ylabel('t')
plt.tight_layout()
plt.show()
# Compute and plot the error
error = np.abs(T_pred - T_true)
plt.figure(figsize=(6,5))
plt.contourf(X, T, error, 100, cmap='viridis')
plt.colorbar()
plt.title('Absolute Error |Predicted - True|')
plt.xlabel('x')
plt.ylabel('t')
plt.show()

---

Advantages and Challenges

Advantages

• Integration of Physics: Ensures solutions adhere to physical laws, reducing reliance on large datasets.
• Flexibility: Can handle complex geometries and high-dimensional problems without mesh generation.
• Data Efficiency: Requires fewer data points compared to purely data-driven models.

Challenges

• Training Stability: Balancing different loss components can be challenging and may lead to convergence issues.
• Computational Cost: Training deep neural networks can be resource-intensive, especially for large-scale problems.
• Scalability: Extending PINNs to very high-dimensional problems or integrating with other physical phenomena remains an active area of research.

---

Resources and Further Reading

• Original PINNs Paper: Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations by Raissi, M., Perdikaris, P., & Karniadakis, G. E.
• PINN GitHub Repository: PINNs Repository
• Tutorial on PINNs: Physics-Informed Neural Networks with TensorFlow
• Advanced PINNs: DeepXDE: A Deep Learning Library for Solving Differential Equations
• TensorFlow Documentation: TensorFlow Official Docs

---