Parallel MODFLOW6 slower than serial #2319

DouweKamper · 2024-09-27T11:02:24Z

DouweKamper
Sep 27, 2024

Hey community,

Parallel MODFLOW6 appears approximately an order of magnitude slower than serial for my model. I am running the script from spyder on my Windows laptop using Flopy version 6.6.0.dev0 as suggested by @jdhughes-usgs. My code and output are shown below. I would really appreciate it if someone can help me speed up my parallel code.

Cheers, Douwe

parallel code

import os
from tempfile import TemporaryDirectory
from pprint import pformat
import numpy as np
import flopy
from flopy.mf6.utils import Mf6Splitter


# %% define model parameters
ZTOP = 0.0  # highest point in z direction
NROW = 25  # number of rows (for discretization in x)
NCOL = 25  # number of rows (for discretization in y)
DELR = 40.0  # length of a single cell in x direction
DELC = 40.0  # length of a single cell in y direction
HB = 10.0  # head at boundary (Dirichlet boundary condition)
WEL_SPECS = [
    (0, int(25 / 2), int(25 / 2), -500.0)
]  # well location and pumping rate
NAME = "model_1"  # name of model
NLAY = 1  # number of layers in model
BOTM = -50  # vertical discretization all layers
HK = 5.0  # horizontal conductivity
VK = 5.0  # vertical conductivity

# %% set up temporary workspace
with TemporaryDirectory() as temp_dir:
    workspace = os.path.join(temp_dir, f"workspace_{NAME}")

    # %% create flopy model objects
    # Create the Flopy simulation object
    sim = flopy.mf6.MFSimulation(
        sim_name=NAME, exe_name="mf6", version="mf6", sim_ws=workspace
    )

    # Create the Flopy temporal discretization object
    tdis = flopy.mf6.modflow.mftdis.ModflowTdis(
        sim,
        pname="tdis",
        time_units="DAYS",
        nper=1,  # one stress period in total
        perioddata=[
            (1.0, 1, 1.0)
        ],  # stress period of 1 day, in steps of 1 day
    )

    # Create the Flopy groundwater flow (gwf) model object
    model_nam_file = f"{NAME}.nam"
    gwf = flopy.mf6.ModflowGwf(
        sim, modelname=NAME, model_nam_file=model_nam_file
    )

    # Create the Flopy iterative model solver (ims)
    ims = flopy.mf6.modflow.mfims.ModflowIms(
        sim, pname="ims", complexity="SIMPLE"
    )

    # %% create packages
    # discretization package
    dis = flopy.mf6.modflow.mfgwfdis.ModflowGwfdis(
        gwf,
        pname="dis",
        nlay=NLAY,
        nrow=NROW,
        ncol=NCOL,
        delr=DELR,
        delc=DELC,
        top=ZTOP,
        botm=BOTM,
    )

    # initial conditions package
    start = HB * np.ones((NLAY, NROW, NCOL))
    ic = flopy.mf6.modflow.mfgwfic.ModflowGwfic(gwf, pname="ic", strt=start)

    # node property flow package
    npf = flopy.mf6.modflow.mfgwfnpf.ModflowGwfnpf(
        gwf, pname="npf", icelltype=1, k=HK, k33=VK, save_flows=True
    )

    # constant head package
    chd_rec = (
        []
    )  # contains all tuples defining location and value of constant heads
    for i in range(NROW):  # 25 equals nrow and ncol (square)
        chd_rec.append(((0, 0, i), HB))
        chd_rec.append(((0, 24, i), HB))
        if i not in (0, NROW - 1):
            chd_rec.append(((0, i, 0), HB))
            chd_rec.append(((0, i, 24), HB))

    chd = flopy.mf6.modflow.mfgwfchd.ModflowGwfchd(
        gwf,
        pname="chd",
        maxbound=len(chd_rec),
        stress_period_data=chd_rec,
        save_flows=True,
    )

    # Create the well package
    wel = flopy.mf6.ModflowGwfwel(gwf, stress_period_data=WEL_SPECS)

    # output control package
    head_filerecord = [f"{NAME}.hds"]
    saverecord = [("HEAD", "LAST")]  # later only save specific locations?
    printrecord = [("HEAD", "LAST")]
    oc = flopy.mf6.ModflowGwfoc(
        gwf,
        saverecord=saverecord,
        head_filerecord=head_filerecord,
        printrecord=printrecord,
    )

    # %% write simulation
    array = np.ones((NROW, NCOL), dtype=int)
    array[0:8, 0:10] = 2
    array[8:17, 0:10] = 3
    array[17:25, 0:10] = 4
    array[0:8, 10:19] = 5
    array[8:17, 10:19] = 6
    array[17:25, 10:19] = 7
    array[0:8, 19:25] = 8
    mfsplit = Mf6Splitter(sim)
    new_sim = mfsplit.split_model(array)  # split the model, rename sim
    new_sim.write_simulation()

    # %% run simulation
    try:
        success, buff = new_sim.run_simulation(silent=False, processors=8)
        assert success, pformat(buff)
    except AssertionError as e:  # excecuted if Not succesfull
        print("MODFLOW Simulation failed")
        print(f"Error: {e}")

    # output
    m1 = new_sim.get_model("model_1_1")
    m2 = new_sim.get_model("model_1_2")
    m3 = new_sim.get_model("model_1_3")
    m4 = new_sim.get_model("model_1_4")
    m5 = new_sim.get_model("model_1_5")
    m6 = new_sim.get_model("model_1_6")
    m7 = new_sim.get_model("model_1_7")
    m8 = new_sim.get_model("model_1_8")
    # heads
    h1 = m1.output.head().get_alldata()[-1]
    h2 = m2.output.head().get_alldata()[-1]
    h3 = m3.output.head().get_alldata()[-1]
    h4 = m4.output.head().get_alldata()[-1]
    h5 = m5.output.head().get_alldata()[-1]
    h6 = m6.output.head().get_alldata()[-1]
    h7 = m7.output.head().get_alldata()[-1]
    h8 = m8.output.head().get_alldata()[-1]
    # combine outputs
    array_dict = {1: h1, 2: h2, 3: h3, 4: h4, 5: h5, 6: h6, 7: h7, 8: h8}
    h = mfsplit.reconstruct_array(array_dict)  # heads of full model

parallel output

runfile('C:/logbook5_q_git/models_d_splitter_answer_github.py', wdir='C:/logbook5_q_git')
writing simulation...
  writing simulation name file...
  writing simulation tdis package...
  writing solution package ims_-1...
  writing package sim_1_6.gwfgwf...
  writing package sim_1_7.gwfgwf...
  writing package sim_1_8.gwfgwf...
  writing package sim_2_3.gwfgwf...
  writing package sim_2_5.gwfgwf...
  writing package sim_3_4.gwfgwf...
  writing package sim_3_6.gwfgwf...
  writing package sim_4_7.gwfgwf...
  writing package sim_5_6.gwfgwf...
  writing package sim_5_8.gwfgwf...
  writing package sim_6_7.gwfgwf...
  writing model model_1_1...
    writing model name file...
    writing package dis...
    writing package ic...
    writing package npf...
    writing package chd...
    writing package oc...
  writing model model_1_2...
    writing model name file...
    writing package dis...
    writing package ic...
    writing package npf...
    writing package chd...
    writing package oc...
  writing model model_1_3...
    writing model name file...
    writing package dis...
    writing package ic...
    writing package npf...
    writing package chd...
    writing package oc...
  writing model model_1_4...
    writing model name file...
    writing package dis...
    writing package ic...
    writing package npf...
    writing package chd...
    writing package oc...
  writing model model_1_5...
    writing model name file...
    writing package dis...
    writing package ic...
    writing package npf...
    writing package chd...
    writing package oc...
  writing model model_1_6...
    writing model name file...
    writing package dis...
    writing package ic...
    writing package npf...
    writing package wel_0...
INFORMATION: maxbound in ('gwf6', 'wel', 'dimensions') changed to 1 based on size of stress_period_data
    writing package oc...
  writing model model_1_7...
    writing model name file...
    writing package dis...
    writing package ic...
    writing package npf...
    writing package chd...
    writing package oc...
  writing model model_1_8...
    writing model name file...
    writing package dis...
    writing package ic...
    writing package npf...
    writing package chd...
    writing package oc...
FloPy is using the following executable to run the model: ..\Users\Douwe\logbook5\Scripts\mf6.exe
FloPy is using C:\Users\Douwe\logbook5\Scripts\mpiexec.exe to run C:\Users\Douwe\logbook5\Scripts\mf6.exe on 8 processors.








                               MODFLOW 6 EXTENDED
                               MODFLOW 6 EXTENDED
                U.S. GEOLOGICAL SURVEY MODULAR HYDROLOGIC MODEL
                  VERSION 6.6.0.dev0 (preliminary) 09/26/2024
                               ***DEVELOP MODE***

   MODFLOW 6 compiled Sep 26 2024 03:53:05 with Intel(R) Fortran Intel(R) 64
  Compiler Classic for applications running on Intel(R) 64, Version 2021.12.0
                U.S. GEOLOGICAL SURVEY MODULAR HYDROLOGIC MODEL
                  VERSION 6.6.0.dev0 (preliminary) 09/26/2024
                               ***DEVELOP MODE***

   MODFLOW 6 compiled Sep 26 2024 03:53:05 with Intel(R) Fortran Intel(R) 64
  Compiler Classic for applications running on Intel(R) 64, Version 2021.12.0
                             Build 20240222_000000

This software is preliminary or provisional and is subject to 
revision. It is being provided to meet the need for timely best 
science. The software has not received final approval by the U.S. 
Geological Survey (USGS). No warranty, expressed or implied, is made 
by the USGS or the U.S. Government as to the functionality of the 
                             Build 20240222_000000

software and related material nor shall the fact of release 
constitute any such warranty. The software is provided on the 
condition that neither the USGS nor the U.S. Government shall be held 
liable for any damages resulting from the authorized or unauthorized 
use of the software.
This software is preliminary or provisional and is subject to 
revision. It is being provided to meet the need for timely best 
science. The software has not received final approval by the U.S. 
Geological Survey (USGS). No warranty, expressed or implied, is made 
by the USGS or the U.S. Government as to the functionality of the 
software and related material nor shall the fact of release 
constitute any such warranty. The software is provided on the 
condition that neither the USGS nor the U.S. Government shall be held 
liable for any damages resulting from the authorized or unauthorized 
use of the software.

 
 MODFLOW runs in PARALLEL mode
 

 
 MODFLOW runs in PARALLEL mode
                               MODFLOW 6 EXTENDED
 
 Run start date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:11
 
                U.S. GEOLOGICAL SURVEY MODULAR HYDROLOGIC MODEL
                               MODFLOW 6 EXTENDED
 Run start date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:11
 
                U.S. GEOLOGICAL SURVEY MODULAR HYDROLOGIC MODEL
                  VERSION 6.6.0.dev0 (preliminary) 09/26/2024
                               ***DEVELOP MODE***
                  VERSION 6.6.0.dev0 (preliminary) 09/26/2024
                               ***DEVELOP MODE***


   MODFLOW 6 compiled Sep 26 2024 03:53:05 with Intel(R) Fortran Intel(R) 64
  Compiler Classic for applications running on Intel(R) 64, Version 2021.12.0
   MODFLOW 6 compiled Sep 26 2024 03:53:05 with Intel(R) Fortran Intel(R) 64
  Compiler Classic for applications running on Intel(R) 64, Version 2021.12.0
                             Build 20240222_000000
                             Build 20240222_000000

This software is preliminary or provisional and is subject to 
revision. It is being provided to meet the need for timely best 
 Writing simulation list file: mfsim.p7.lst
science. The software has not received final approval by the U.S. 
Geological Survey (USGS). No warranty, expressed or implied, is made 
by the USGS or the U.S. Government as to the functionality of the 
software and related material nor shall the fact of release 
constitute any such warranty. The software is provided on the 
condition that neither the USGS nor the U.S. Government shall be held 
liable for any damages resulting from the authorized or unauthorized 
use of the software.


This software is preliminary or provisional and is subject to 
 
 MODFLOW runs in PARALLEL mode
 
revision. It is being provided to meet the need for timely best 
science. The software has not received final approval by the U.S. 
Geological Survey (USGS). No warranty, expressed or implied, is made 
by the USGS or the U.S. Government as to the functionality of the 
 Run start date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:11
 
software and related material nor shall the fact of release 
constitute any such warranty. The software is provided on the 
condition that neither the USGS nor the U.S. Government shall be held 
liable for any damages resulting from the authorized or unauthorized 
                               MODFLOW 6 EXTENDED
use of the software.

 Using Simulation name file: mfsim.nam
 
 
                               MODFLOW 6 EXTENDED
                U.S. GEOLOGICAL SURVEY MODULAR HYDROLOGIC MODEL
                  VERSION 6.6.0.dev0 (preliminary) 09/26/2024
                U.S. GEOLOGICAL SURVEY MODULAR HYDROLOGIC MODEL
                               ***DEVELOP MODE***
 Writing simulation list file: mfsim.p3.lst

   MODFLOW 6 compiled Sep 26 2024 03:53:05 with Intel(R) Fortran Intel(R) 64
  Compiler Classic for applications running on Intel(R) 64, Version 2021.12.0
                             Build 20240222_000000

This software is preliminary or provisional and is subject to 
revision. It is being provided to meet the need for timely best 
science. The software has not received final approval by the U.S. 
Geological Survey (USGS). No warranty, expressed or implied, is made 
by the USGS or the U.S. Government as to the functionality of the 
 MODFLOW runs in PARALLEL mode
software and related material nor shall the fact of release 
constitute any such warranty. The software is provided on the 
                  VERSION 6.6.0.dev0 (preliminary) 09/26/2024
condition that neither the USGS nor the U.S. Government shall be held 
liable for any damages resulting from the authorized or unauthorized 
 Writing simulation list file: mfsim.p2.lst
 
                               ***DEVELOP MODE***
use of the software.


   MODFLOW 6 compiled Sep 26 2024 03:53:05 with Intel(R) Fortran Intel(R) 64
 
  Compiler Classic for applications running on Intel(R) 64, Version 2021.12.0
                             Build 20240222_000000

This software is preliminary or provisional and is subject to 
revision. It is being provided to meet the need for timely best 
science. The software has not received final approval by the U.S. 
 MODFLOW runs in PARALLEL mode
Geological Survey (USGS). No warranty, expressed or implied, is made 
by the USGS or the U.S. Government as to the functionality of the 
 
software and related material nor shall the fact of release 
constitute any such warranty. The software is provided on the 
condition that neither the USGS nor the U.S. Government shall be held 
liable for any damages resulting from the authorized or unauthorized 
use of the software.

 
 MODFLOW runs in PARALLEL mode
 
 Run start date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:11
                               MODFLOW 6 EXTENDED
                U.S. GEOLOGICAL SURVEY MODULAR HYDROLOGIC MODEL
 Run start date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:11
 
 
 Run start date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:11
 
                  VERSION 6.6.0.dev0 (preliminary) 09/26/2024
 Writing simulation list file: mfsim.p5.lst
 Writing simulation list file: mfsim.p6.lst
                               MODFLOW 6 EXTENDED
                U.S. GEOLOGICAL SURVEY MODULAR HYDROLOGIC MODEL
 Using Simulation name file: mfsim.nam
 
                               ***DEVELOP MODE***
 Using Simulation name file: mfsim.nam
 
 Using Simulation name file: mfsim.nam
 

   MODFLOW 6 compiled Sep 26 2024 03:53:05 with Intel(R) Fortran Intel(R) 64
  Compiler Classic for applications running on Intel(R) 64, Version 2021.12.0
                             Build 20240222_000000

This software is preliminary or provisional and is subject to 
 Using Simulation name file: mfsim.nam
 
revision. It is being provided to meet the need for timely best 
science. The software has not received final approval by the U.S. 
Geological Survey (USGS). No warranty, expressed or implied, is made 
                  VERSION 6.6.0.dev0 (preliminary) 09/26/2024
                               ***DEVELOP MODE***
by the USGS or the U.S. Government as to the functionality of the 
software and related material nor shall the fact of release 
constitute any such warranty. The software is provided on the 

   MODFLOW 6 compiled Sep 26 2024 03:53:05 with Intel(R) Fortran Intel(R) 64
  Compiler Classic for applications running on Intel(R) 64, Version 2021.12.0
                             Build 20240222_000000

This software is preliminary or provisional and is subject to 
revision. It is being provided to meet the need for timely best 
science. The software has not received final approval by the U.S. 
Geological Survey (USGS). No warranty, expressed or implied, is made 
by the USGS or the U.S. Government as to the functionality of the 
software and related material nor shall the fact of release 
constitute any such warranty. The software is provided on the 
condition that neither the USGS nor the U.S. Government shall be held 
liable for any damages resulting from the authorized or unauthorized 
use of the software.

 
 MODFLOW runs in PARALLEL mode
 
 Run start date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:11
 
 Writing simulation list file: mfsim.p0.lst
condition that neither the USGS nor the U.S. Government shall be held 
liable for any damages resulting from the authorized or unauthorized 
use of the software.

 
 MODFLOW runs in PARALLEL mode
 
 Writing simulation list file: mfsim.p1.lst
 Run start date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:11
 
 Using Simulation name file: mfsim.nam
 
 Writing simulation list file: mfsim.p4.lst
 Using Simulation name file: mfsim.nam
 
 Using Simulation name file: mfsim.nam
 
    Solving:  Stress period:     1    Time step:     1
    Solving:  Stress period:     1    Time step:     1
    Solving:  Stress period:     1    Time step:     1
    Solving:  Stress period:     1    Time step:     1
    Solving:  Stress period:     1    Time step:     1
    Solving:  Stress period:     1    Time step:     1
    Solving:  Stress period:     1    Time step:     1
    Solving:  Stress period:     1    Time step:     1
 
 Run end date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:17
 Elapsed run time:  5.552 Seconds
 
 Normal termination of simulation.
 
 Run end date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:17
 Elapsed run time:  5.552 Seconds
 
 Normal termination of simulation.
 
 Run end date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:17
 Elapsed run time:  5.615 Seconds
 
 Normal termination of simulation.
 
 Run end date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:17
 Elapsed run time:  5.646 Seconds
 
 Normal termination of simulation.
 
 Run end date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:17
 Elapsed run time:  5.646 Seconds
 
 Normal termination of simulation.
 
 Run end date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:17
 Elapsed run time:  5.662 Seconds
 
 Normal termination of simulation.
 
 Run end date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:17
 Elapsed run time:  5.755 Seconds
 
 Normal termination of simulation.
 
 Run end date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 13:10:17
 Elapsed run time:  5.787 Seconds
 
 Normal termination of simulation.

serial code

import os
from tempfile import TemporaryDirectory
from pprint import pformat
import numpy as np
import flopy
from flopy.mf6.utils import Mf6Splitter


# %% define model parameters
ZTOP = 0.0  # highest point in z direction
NROW = 25  # number of rows (for discretization in x)
NCOL = 25  # number of rows (for discretization in y)
DELR = 40.0  # length of a single cell in x direction
DELC = 40.0  # length of a single cell in y direction
HB = 10.0  # head at boundary (Dirichlet boundary condition)
WEL_SPECS = [
    (0, int(25 / 2), int(25 / 2), -500.0)
]  # well location and pumping rate
NAME = "model_1"  # name of model
NLAY = 1  # number of layers in model
BOTM = -50  # vertical discretization all layers
HK = 5.0  # horizontal conductivity
VK = 5.0  # vertical conductivity

# %% set up temporary workspace
with TemporaryDirectory() as temp_dir:
    workspace = os.path.join(temp_dir, f"workspace_{NAME}")

    # %% create flopy model objects
    # Create the Flopy simulation object
    sim = flopy.mf6.MFSimulation(
        sim_name=NAME, exe_name="mf6", version="mf6", sim_ws=workspace
    )

    # Create the Flopy temporal discretization object
    tdis = flopy.mf6.modflow.mftdis.ModflowTdis(
        sim,
        pname="tdis",
        time_units="DAYS",
        nper=1,  # one stress period in total
        perioddata=[
            (1.0, 1, 1.0)
        ],  # stress period of 1 day, in steps of 1 day
    )

    # Create the Flopy groundwater flow (gwf) model object
    model_nam_file = f"{NAME}.nam"
    gwf = flopy.mf6.ModflowGwf(
        sim, modelname=NAME, model_nam_file=model_nam_file
    )

    # Create the Flopy iterative model solver (ims)
    ims = flopy.mf6.modflow.mfims.ModflowIms(
        sim, pname="ims", complexity="SIMPLE"
    )

    # %% create packages
    # discretization package
    dis = flopy.mf6.modflow.mfgwfdis.ModflowGwfdis(
        gwf,
        pname="dis",
        nlay=NLAY,
        nrow=NROW,
        ncol=NCOL,
        delr=DELR,
        delc=DELC,
        top=ZTOP,
        botm=BOTM,
    )

    # initial conditions package
    start = HB * np.ones((NLAY, NROW, NCOL))
    ic = flopy.mf6.modflow.mfgwfic.ModflowGwfic(gwf, pname="ic", strt=start)

    # node property flow package
    npf = flopy.mf6.modflow.mfgwfnpf.ModflowGwfnpf(
        gwf, pname="npf", icelltype=1, k=HK, k33=VK, save_flows=True
    )

    # constant head package
    chd_rec = (
        []
    )  # contains all tuples defining location and value of constant heads
    for i in range(NROW):  # 25 equals nrow and ncol (square)
        chd_rec.append(((0, 0, i), HB))
        chd_rec.append(((0, 24, i), HB))
        if i not in (0, NROW - 1):
            chd_rec.append(((0, i, 0), HB))
            chd_rec.append(((0, i, 24), HB))

    chd = flopy.mf6.modflow.mfgwfchd.ModflowGwfchd(
        gwf,
        pname="chd",
        maxbound=len(chd_rec),
        stress_period_data=chd_rec,
        save_flows=True,
    )

    # Create the well package
    wel = flopy.mf6.ModflowGwfwel(gwf, stress_period_data=WEL_SPECS)

    # output control package
    head_filerecord = [f"{NAME}.hds"]
    saverecord = [("HEAD", "LAST")]  # later only save specific locations?
    printrecord = [("HEAD", "LAST")]
    oc = flopy.mf6.ModflowGwfoc(
        gwf,
        saverecord=saverecord,
        head_filerecord=head_filerecord,
        printrecord=printrecord,
    )

    # %% write simulation
    sim.write_simulation()

    # %% run simulation
    try:
        success, buff = sim.run_simulation(silent=False)
        assert success, pformat(buff)
    except AssertionError as e:  # excecuted if Not succesfull
        print("MODFLOW Simulation failed")
        print(f"Error: {e}")

    # output
    m = sim.get_model("model_1")
    h = m.output.head().get_alldata()[-1]

serial output

writing simulation...
  writing simulation name file...
  writing simulation tdis package...
  writing solution package ims...
  writing model model_1...
    writing model name file...
    writing package dis...
    writing package ic...
    writing package npf...
    writing package chd...
    writing package wel_0...
INFORMATION: maxbound in ('gwf6', 'wel', 'dimensions') changed to 1 based on size of stress_period_data
    writing package oc...
FloPy is using the following executable to run the model: ..\..\..\..\..\logbook5\Scripts\mf6.exe
                               MODFLOW 6 EXTENDED
                U.S. GEOLOGICAL SURVEY MODULAR HYDROLOGIC MODEL
                  VERSION 6.6.0.dev0 (preliminary) 09/26/2024
                               ***DEVELOP MODE***

   MODFLOW 6 compiled Sep 26 2024 03:53:05 with Intel(R) Fortran Intel(R) 64
  Compiler Classic for applications running on Intel(R) 64, Version 2021.12.0
                             Build 20240222_000000

This software is preliminary or provisional and is subject to 
revision. It is being provided to meet the need for timely best 
science. The software has not received final approval by the U.S. 
Geological Survey (USGS). No warranty, expressed or implied, is made 
by the USGS or the U.S. Government as to the functionality of the 
software and related material nor shall the fact of release 
constitute any such warranty. The software is provided on the 
condition that neither the USGS nor the U.S. Government shall be held 
liable for any damages resulting from the authorized or unauthorized 
use of the software.

 
 MODFLOW runs in SEQUENTIAL mode
 
 Run start date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 12:31:31
 
 Writing simulation list file: mfsim.lst
 Using Simulation name file: mfsim.nam
 
 Solving:  Stress period:     1    Time step:     1
 
 Run end date and time (yyyy/mm/dd hh:mm:ss): 2024/09/26 12:31:31
 Elapsed run time:  0.469 Seconds
 
 Normal termination of simulation.

Answered by langevin-usgs

Sep 27, 2024

@DouweKamper, this is expected behavior. There is a bit more to the story with parallel MODFLOW. When you split a single model into multiple models and run them in parallel, there is overhead due to the communication between processors. For small models, the combined overhead and numerical solution will be greater than simply solving the single model in serial. You will not typically see efficiency gains with parallel MODFLOW until the models have hundred of thousands of cells or more.

View full answer

langevin-usgs · 2024-09-27T11:10:14Z

langevin-usgs
Sep 27, 2024
Maintainer

@DouweKamper, this is expected behavior. There is a bit more to the story with parallel MODFLOW. When you split a single model into multiple models and run them in parallel, there is overhead due to the communication between processors. For small models, the combined overhead and numerical solution will be greater than simply solving the single model in serial. You will not typically see efficiency gains with parallel MODFLOW until the models have hundred of thousands of cells or more.

1 reply

DouweKamper Sep 27, 2024
Author

@langevin-usgs thank you for clearing that up! I have a follow-up question though.

I have multiple small models for which I am comparing calibration methods. This requires upwards of 1 million MODFLOW6 simulations. Therefore, I would like to decrease simulation time of my models, if possible. I noticed that you need to specify the number of processors when running MODFLOW6 in parallel, but not the number of cores. Does this mean that serial MODFLOW6 can use multiple cores? Because, I have access to an HPC system with 64 cores per CPU.

langevin-usgs · 2024-09-27T11:53:09Z

langevin-usgs
Sep 27, 2024
Maintainer

If you have run small models many many times, then you probably should not use parallel MODFLOW. Instead, you should parallelize the number of runs being performed by the calibration routine. Calibration software, like PEST, are capable of running a model many hundreds or thousands of times in pursuit of model calibration and quantification of uncertainty. It sounds to me like your efforts towards parallelization should focus in that direction instead.

1 reply

DouweKamper Sep 27, 2024
Author

You are correct ofcourse. I actually tried parallelizing my calibration software first (emcee), but encountered an OSError (synchronously opening and closing files) that I did not manage to solve. That's when I came up with the idea to parallelize MODFLOW6 instead, as it is the most computationally demanding part of my code by far anyway.

I do appreciate that you cleared up that parallel MODFLOW6 is not useful in my case. At least now I know that I should focus my efforts on parallelizing my calibration software instead.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parallel MODFLOW6 slower than serial #2319

{{title}}

Replies: 2 comments 2 replies

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Parallel MODFLOW6 slower than serial #2319

DouweKamper Sep 27, 2024

Replies: 2 comments · 2 replies

langevin-usgs Sep 27, 2024 Maintainer

DouweKamper Sep 27, 2024 Author

langevin-usgs Sep 27, 2024 Maintainer

DouweKamper Sep 27, 2024 Author

DouweKamper
Sep 27, 2024

Replies: 2 comments 2 replies

langevin-usgs
Sep 27, 2024
Maintainer

DouweKamper Sep 27, 2024
Author

langevin-usgs
Sep 27, 2024
Maintainer

DouweKamper Sep 27, 2024
Author