fixed figures

docs/index.md

@@ -2,7 +2,7 @@
 layout: default
 ---
 ### Authors
-[Maziar Raissi](http://www.dam.brown.edu/people/mraissi/), [Zhicheng Wang](http://meche.mit.edu/people/staff/zhicheng@mit.edu), [Michael Triantafyllou](http://meche.mit.edu/people/faculty/MISTETRI@MIT.EDU), and [George Karniadakis](https://www.brown.edu/research/projects/crunch/george-karniadakis)
+[Maziar Raissi](https://maziarraissi.github.io/), [Zhicheng Wang](http://meche.mit.edu/people/staff/zhicheng@mit.edu), [Michael Triantafyllou](http://meche.mit.edu/people/faculty/MISTETRI@MIT.EDU), and [George Karniadakis](https://www.brown.edu/research/projects/crunch/george-karniadakis)
 
 ### Abstract
 
@@ -33,7 +33,7 @@ f_L := \rho \eta_{tt} + b \eta_t + k \eta.
 \end{array}
 $$
 
-![](http://www.dam.brown.edu/people/mraissi/assets/img/DeepVIV_1.png)
+![](https://maziarraissi.github.io/assets/img/DeepVIV_1.png)
 > _Pedagogical physics-informed neural network:_ A plain vanilla densely connected (physics uninformed) neural network, with 10 hidden layers and 32 neurons per hidden layer per output variable (i.e., 1 x 32 = 32 neurons per hidden layer), takes the input variable t and outputs the displacement. As for the activation functions, we use sin(x). For illustration purposes only, the network depicted in this figure comprises of 2 hidden layers and 4 neurons per hidden layers. We employ automatic differentiation to obtain the required derivatives to compute the residual (physics informed) networks. The total loss function is composed of the regression loss of the displacement on the training data, and the loss imposed by the differential equation. Here, the differential operator is computed using automatic differentiation and can be thought of as an "activation operator". Moreover, the gradients of the loss function are back-propagated through the entire network to train the parameters of the neural network as well as the damping and the stiffness parameters using the Adam optimizer.
 
 It is worth noting that the damping $$b$$ and the stiffness $$k$$ parameters turn into parameters of the resulting physics informed neural network $$f_L$$. We obtain the required derivatives to compute the residual network $$f_L$$ by applying the chain rule for differentiating compositions of functions using [automatic differentiation](https://en.wikipedia.org/wiki/Automatic_differentiation). Automatic differentiation is different from, and in several respects superior to, numerical or symbolic differentiation -- two commonly encountered techniques of computing derivatives. In its most basic description, automatic differentiation relies on the fact that all numerical computations are ultimately compositions of a finite set of elementary operations for which derivatives are known. Combining the derivatives of the constituent operations through the chain rule gives the derivative of the overall composition. This allows accurate evaluation of derivatives at machine precision with ideal asymptotic efficiency and only a small constant factor of overhead. In particular, to compute the required derivatives we rely on [Tensorflow](https://www.tensorflow.org), which is a popular and relatively well documented open source software library for automatic differentiation and deep learning computations.
@@ -62,7 +62,7 @@ Here, $$u(t,x,y)$$ and $$v(t,x,y)$$ are the streamwise and crossflow components
 
 **Problem 1 (VIV-I):** Given scattered and potentially noisy measurements $$\{t^n, x^n, y^n, u^n, v^n\}_{n=1}^N$$ of the velocity field -- Take for example the case of reconstructing a flow field from scattered measurements obtained from [Particle Image Velocimetry](https://en.wikipedia.org/wiki/Particle_image_velocimetry) (PIV) or [Particle Tracking Velocimetry](https://en.wikipedia.org/wiki/Particle_tracking_velocimetry) (PTV) -- in addition to the data $$\{t^n,\eta^n\}_{n=1}^{N}$$ on the displacement and knowing the governing equations of the flow, we are interested in reconstructing the entire velocity field as well as the pressure field in space-time. Such measurements are usually collected only in a small sub-domain, which may not be appropriate for classical CFD computations due to the presence of numerical artifacts. Typically, the data points are scattered in both space and time and are usually of the order of a few thousands or less in space. For a visual representation of the distribution of observation points $$\{t^n, x^n, y^n\}_{n=1}^N$$ scattered in space and time please refer to the following figure.
 
-![](http://www.dam.brown.edu/people/mraissi/assets/img/PointCloud.png)
+![](https://maziarraissi.github.io/assets/img/PointCloud.png)
 > _Point-cloud:_ Depicted on the left panel is a visual representation of the distribution of observation points scattered in space and time. A randomly selected snapshot of the distribution of data points is also shown in the right panel. To capture the boundary layers a finer resolution of points are sampled closer to the cylinder.
 
 To solve the aforementioned problem, we proceed by approximating the latent functions $$u(t,x,y)$$, $$v(t,x,y)$$, $$p(t,x,y)$$, and $$\eta(t)$$ by a single neural network outputting four variables while taking as input $$t, x$$, and $$y$$. This prior assumption along with the incompressible Navier-Stokes equations result in the following Navier-Stokes informed neural networks (see the following figure)
@@ -75,7 +75,7 @@ e_3 := u_x + v_y.
 \end{array}
 $$
 
-![](http://www.dam.brown.edu/people/mraissi/assets/img/DeepVIV_2.png)
+![](https://maziarraissi.github.io/assets/img/DeepVIV_2.png)
 > _Navier-Stokes informed neural networks:_ A plain vanilla densely connected (physics uninformed) neural network, with 10 hidden layers and 32 neurons per hidden layer per output variable (i.e., 4 x 32 = 128 neurons per hidden layer), takes the input variables t, x, y and outputs the dispacement, u, v, and p. As for the activation functions, we use sin(x). For illustration purposes only, the network depicted in this figure comprises of 2 hidden layers and 5 neurons per hidden layers. We employ automatic differentiation to obtain the required derivatives to compute the residual (physics informed) networks. If a term does not appear in the blue boxes, its coefficient is assumed to be zero. It is worth emphasizing that unless the coefficient in front of a term is non-zero, that term is not going to appear in the actual "compiled" computational graph and is not going to contribute to the computational cost of a feed forward evaluation of the resulting network. The total loss function is composed of the regression loss of the velocity fields u, v and the displacement on the training data, and the loss imposed by the differential equations. Here, the differential operators are computed using automatic differentiation and can be thought of as "activation operators". Moreover, the gradients of the loss function are back-propagated through the entire network to train the neural network parameters using the Adam optimizer.
 
 We use automatic differentiation to obtain the required derivatives to compute the residual networks $$e_1$$, $$e_2$$, and $$e_3$$. The shared parameters of the neural networks $$u$$, $$v$$, $$p$$, $$\eta$$, $$e_1$$, $$e_2$$, and $$e_3$$ can be learned by minimizing the sum of squared errors loss function
@@ -121,7 +121,7 @@ e_4 := c_t + u c_x + v c_y - Pe^{-1}(c_{xx} + c_{yy}).
 \end{array}
 $$
 
-![](http://www.dam.brown.edu/people/mraissi/assets/img/DeepVIV_3.png)
+![](https://maziarraissi.github.io/assets/img/DeepVIV_3.png)
 > _Navier-Stokes informed neural networks:_ A plain vanilla densely connected (physics uninformed) neural network, with 10 hidden layers and 32 neurons per hidden layer per output variable (i.e., 5 x 32 = 160 neurons per hidden layer), takes the input variables t, x, y and outputs the displacement, c, u, v, w, and p. As for the activation functions, we use sin(x). For illustration purposes only, the network depicted in this figure comprises of 2 hidden layers and 6 neurons per hidden layers. We employ automatic differentiation to obtain the required derivatives to compute the residual (physics informed) networks. If a term does not appear in the blue boxes, its coefficient is assumed to be zero. It is worth emphasizing that unless the coefficient in front of a term is non-zero, that term is not going to appear in the actual "compiled" computational graph and is not going to contribute to the computational cost of a feed forward evaluation of the resulting network. The total loss function is composed of the regression loss of the passive scalar c and the displacement on the training data, and the loss imposed by the differential equations. Here, the differential operators are computed using automatic differentiation and can be thought of as "activation operators". Moreover, the gradients of the loss function are back-propagated through the entire network to train the neural networks parameters using the Adam optimizer.
 
 The residual networks $$e_1$$, $$e_2$$, and $$e_3$$ are defined as before. We use automatic differentiation to obtain the required derivatives to compute the additional residual network $$e_4$$. The shared parameters of the neural networks $$c$$, $$u$$, $$v$$, $$p$$, $$\eta$$, $$e_1$$, $$e_2$$, $$e_3$$, and $$e_4$$ can be learned by minimizing the sum of squared errors loss function
@@ -145,7 +145,7 @@ To generate a high-resolution dataset for the VIV problem we have performed dire
 
 To illustrate the effectiveness of our approach, let us start with the two time series depicted in the following figure consisting of $$N=111$$ observations of the displacement and the lift force. These data correspond to damping and stiffness parameters with exact values $$b=0.084$$ and $$k=2.2020$$, respectively. Here, the cylinder is assumed to have a mass of $$\rho = 2.0$$. This data-set is then used to train a 10-layer deep neural network with 32 neurons per hidden layers by minimizing the sum of squared errors loss function (the first loss function introduced in the current work) using the [Adam optimizer](https://arxiv.org/abs/1412.6980). Upon training, the network is used to predict the entire solution functions $$\eta(t)$$ and $$f_L(t)$$, as well as the unknown structural parameters $$b$$ and $$k$$. In addition to almost perfect reconstructions of the two time series for displacement and lift force, the proposed framework is capable of identifying the correct values for the structural parameters $$b$$ and $$k$$ with remarkable accuracy. The learned values for the damping and stiffness parameters are $$b = 0.08438281$$ and $$k = 2.2015007$$. This corresponds to around $$0.45\%$$ and $$0.02\%$$ relative errors in the estimated values for $$b$$ and $$k$$, respectively.
 
-![](http://www.dam.brown.edu/people/mraissi/assets/img/displacement_lift.png)
+![](https://maziarraissi.github.io/assets/img/displacement_lift.png)
 > _Vortex Induced Vibrations:_ Observations of the displacement are plotted in the left panel while the data on the lift force are depicted in the right panel. These observations are shown by the red circles. Predictions of the trained neural networks are depicted by blue solid lines.
 
 As for the activation functions, we use $$\sin(x)$$. In general, the choice of a neural network's architecture (e.g., number of layers/neurons and form of activation functions) is crucial and in many cases still remains an art that relies on one's ability to balance the trade off between *expressivity* and *trainability* of the neural network. Our empirical findings so far indicate that deeper and wider networks are usually more expressive (i.e., they can capture a larger class of functions) but are often more costly to train (i.e., a feed-forward evaluation of the neural network takes more time and the optimizer requires more iterations to converge). Moreover, the sinusoid (i.e., $$\sin(x)$$) activation function seems to be numerically more stable than $$\tanh(x)$$, at least while computing the residual neural networks $$f_L$$, and $$e_i$$, $$i=1,\ldots,4$$. In this work, we have tried to choose the neural networks' architectures in a consistent fashion throughout the manuscript by setting the number of layers to 10 and the number of neurons to 32 per output variable. Consequently, there might exist other architectures that improve some of the results reported in the current work.
@@ -154,24 +154,24 @@ As for the activation functions, we use $$\sin(x)$$. In general, the choice of a
 
 Let us now consider the case where we do not have access to direct measurements of the lift force $$f_L$$. In this case, we can use measurements of the velocity field, obtained for instance via [Particle Image Velocimetry](https://en.wikipedia.org/wiki/Particle_image_velocimetry) (PIV) or [Particle Tracking Velocimetry](https://en.wikipedia.org/wiki/Particle_tracking_velocimetry) (PTV), to reconstruct the velocity field, the pressure, and consequently the drag and lift forces. A representative snapshot of the data on the velocity field is depicted in the top left and top middle panels of the following figure. The neural network architectures used here consist of 10 layers with 32 neurons in each hidden layer. A summary of our results is presented in the following figure. The proposed framework is capable of accurately (of the order of $$10^{-3}$$) reconstructing the velocity field; however, a more intriguing result stems from the network's ability to provide an accurate prediction of the entire pressure field $$p(t,x,y)$$ in the absence of any training data on the pressure itself (see the following figure). A visual comparison against the exact pressure is presented in the following figure for a representative snapshot of the pressure. It is worth noticing that the difference in magnitude between the exact and the predicted pressure is justified by the very nature of incompressible Navier-Stokes equations, since the pressure field is only identifiable up to a constant. This result of inferring a continuous quantity of interest from auxiliary measurements by leveraging the underlying physics is a great example of the enhanced  capabilities that our approach has to offer, and highlights its potential in solving high-dimensional data assimilation and inverse problems.
 
-![](http://www.dam.brown.edu/people/mraissi/assets/img/VIV_Case2_data_on_velocities_results.png)
+![](https://maziarraissi.github.io/assets/img/VIV_Case2_data_on_velocities_results.png)
 > _VIV-I (Velocity Measurements):_ A representative snapshot of the data on the velocity field is depicted in the top left and top middle panels of this figure. The algorithm is capable of accurately reconstructing the velocity field and more importantly the pressure without having access to even a single observation on the pressure itself. To compute the difference between the predicted and exact pressure fields we had to subtract the spacial average pressure from both predicted and exact fields because for incompressible fluids the pressure is unique only up to a constant.
 
 The trained neural networks representing the velocity field and the pressure can be used to compute the drag and lift forces by employing equations for drag and lift, respectively. The resulting drag and lift forces are compared to the exact ones in the following figure. In the following, we are going to use the computed lift force to generate the required training data on $$f_L$$ and estimate the structural parameters $$b$$ and $$k$$ by minimizing the the first loss function intoduced in the current work. Upon training, the proposed framework is capable of identifying the correct values for the structural parameters $$b$$ and $$k$$ with remarkable accuracy. The learned values for the damping and stiffness parameters are $$b = 0.0844064$$ and $$k = 2.1938791$$. This corresponds to around $$0.48\%$$ and $$0.37\%$$ relative errors in the estimated values for $$b$$ and $$k$$, respectively.
 
-![](http://www.dam.brown.edu/people/mraissi/assets/img/VIV_Case2_data_on_velocities_lift_drag.png)
+![](https://maziarraissi.github.io/assets/img/VIV_Case2_data_on_velocities_lift_drag.png)
 > _VIV-I (Velocity Measurements):_ In this figure, the resulting lift (left) and drag (right) forces are compared to the exact ones.
 
 **Inferring Lift and Drag Forces from Flow Visualizations**
 
 Let us continue with the case where we do not have access to direct observations of the lift force $$f_L$$. This time rather than using data on the velocity field, we use measurements of the concentration of a passive scalar (e.g., dye or smoke) injected into the system. In the following, we are going to employ such data to reconstruct the velocity field, the pressure, and consequently the drag and lift forces. A representative snapshot of the data on the concentration of the passive scalar is depicted in the top left panel of the following figure. The neural networks' architectures used here consist of 10 layers with 32 neurons per each hidden layer per output variable. A summary of our results is presented in the following figure. The proposed framework is capable of accurately (of the order of $$10^{-3}$$) reconstructing the concentration. However, a truly intriguing result stems from the network's ability to provide accurate predictions of the entire velocity vector field as well as the pressure, in the absence of sufficient training data on the pressure and the velocity field themselves (see the following figure). A visual comparison against the exact quantities is presented in the following figure for a representative snapshot of the velocity field and the pressure. This result of inferring multiple hidden quantities of interest from auxiliary measurements by leveraging the underlying physics is a great example of the enhanced capabilities that physics-informed deep learning has to offer, and highlights its potential in solving high-dimensional data-assimilation and inverse problems.
 
-![](http://www.dam.brown.edu/people/mraissi/assets/img/VIV_case2_concentration_results.png)
+![](https://maziarraissi.github.io/assets/img/VIV_case2_concentration_results.png)
 > _VIV-II (Flow Visualizations Data):_ A representative snapshot of the data on the concentration of the passive scalar is depicted in the top left panel of this figure. The algorithm is capable of accurately reconstructing the concentration of the passive scalar and more importantly the velocity field as well as the pressure without having access to enough observations of these quantities themselves. To compute the difference between the predicted and exact pressure fields we had to subtract the spacial average pressure from both predicted and exact fields because for incompressible fluids the pressure is unique only up to a constant.
 
 Following the same procedure as in the previous example, the trained neural networks representing the velocity field and the pressure can be used to compute the drag and lift forces by employing the equations for drag and lift, respectively. The resulting drag and lift forces are compared to the exact ones in the following figure. In the following, we are going to use the computed lift force to generate the required training data on $$f_L$$ and estimate the structural parameters $$b$$ and $$k$$ by minimizing the first loss function introduced in the current work. Upon training, the proposed framework is capable of identifying the correct values for the structural parameters $$b$$ and $$k$$ with surprising accuracy. The learned values for the damping and stiffness parameters are $$b = 0.08600664$$ and $$k = 2.2395933$$. This corresponds to around $$2.39\%$$ and $$1.71\%$$ relative errors in the estimated values for $$b$$ and $$k$$, respectively.
 
-![](http://www.dam.brown.edu/people/mraissi/assets/img/VIV_case2_concentration_lift_drag.png)
+![](https://maziarraissi.github.io/assets/img/VIV_case2_concentration_lift_drag.png)
 > _VIV-II (Flow Visualizations Data):_ In this figure, the resulting lift (left) and drag (right) forces are compared to the exact ones.
 
 * * * * *
@@ -197,4 +197,3 @@ This work received support by the DARPA EQUiPS grant N66001-15-2-4055 and the AF
 	  journal={arXiv preprint arXiv:1808.08952},
 	  year={2018}
 	}
-