# Parallel mumps solver and custom solver

**URL:** <https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93>\
**Category:** General\
**Created:** [July 10, 2025, 3:59am UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93 "2025-07-10T03:59:04Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Djoumessi](https://avatars.discourse-cdn.com/v4/letter/d/da6949/32.png) [@Djoumessi](https://getfem.discourse.group/u/Djoumessi)\
**Post date:** [July 10, 2025, 3:59am UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93/1 "2025-07-10T03:59:04Z")

</div>

Hello everyone,

I am working on a large multi-physics problem. My code was not originally written with parallelism in mind, but I would like to speed up the linear solve phase by using MUMPS in parallel.

Is it possible to parallelize only the MUMPS solver (via MPI) without rewriting the entire code to be parallel? In other words, can I benefit from the internal parallelism of MUMPS by simply running my executable with `mpirun`, even if the matrix assembly remains sequential?

Also, I am starting to develop my own solvers for specific physics. I am already using `real_rhs()` and `real_tangent_matrix()` to extract the system, but I would appreciate any examples or resources on how to structure a clean custom solver in the GetFEM workflow.

Thank you in advance for your help and guidance.

Best regards,

René

---

<div class="post-metadata">

**Author:** ![Konstantinos.Poulios](https://avatars.discourse-cdn.com/v4/letter/k/ac91a4/32.png) [@Konstantinos.Poulios](https://getfem.discourse.group/u/Konstantinos.Poulios)\
**Post date:** [July 10, 2025, 5:28am UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93/2 "2025-07-10T05:28:44Z")

</div>

I assume you have read

> **[MPI Parallelization of GetFEM — GetFEM](https://getfem.org/userdoc/parallel.html)**

this documentation requires some updates but it is in general helpful.

If you stick to using GetFEM to compute the rhs and tangent matrix, your code should already be able to run in parallel, both the assembly and the linear solve (with MUMPS, the latter requires the first).

MUMPS has a very nice architecture which makes parallelization so easy. If K\_i is the part of the tangent matrix that the i-th process sees, then the overall tangent matrix should be:

K = K\_1 + K\_2 + …+ K\_N

where K\_1, K\_2 etc. can have overlapping entries as long as the sum of these entries matches the corresponding entry in K.

If you compile GetFEM with GETFEM\_PARA\_LEVEL=2, then all meshes you use will be partitioned with Metis. This will define internally as many mesh regions as the processes, let’s call them Metis\_1, Metis\_2, … Metis\_N.

Then whenever you assemble a term on region let’s say -1 (entire mesh), process “i” will perform the assembly only on region Metis\_i instead.

If you perform an assembly on your own defined region, e.g. RG\_RED, then process “i” will perform the assembly on the intersection of RG\_RED and Metis\_i

At the end in a GetFEM model, vectors are broadcasted to all processes, matrices do not. That’s how GetFEM prepares the data in the format MUMPS expects them. This parallelization is basically totally automated. If you write your entire model with GWFL expressions, and you do not use any strange model bricks, you need to do nothing else than compiling the parallel version of GetFEM.

---

<div class="post-metadata">

**Author:** ![Djoumessi](https://avatars.discourse-cdn.com/v4/letter/d/da6949/32.png) [@Djoumessi](https://getfem.discourse.group/u/Djoumessi)\
**Post date:** [July 10, 2025, 6:56am UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93/3 "2025-07-10T06:56:09Z")

</div>

Hello,

Thank you for your clear explanation.

I would just like to confirm that I am indeed in the case where the assembly of the tangent matrix and the RHS is **automatically handled by GetFEM** , and thus **can be parallelized** with MUMPS without rewriting my whole code.

In my case:

- I use a `getfem::model` object (through the Python interface).
- I don’t use “model bricks”, but instead describe the physics using GWFL expressions like:

```auto
md.add_nonlinear_term(..., expression)

```

- I actually solving with:

```auto
md.solve('noisy',
         'max_iter', 100,
         'max_res', 5e-4,
         'lsolver', 'mumps',
         'lsearch', 'simplest',
         'alpha max ratio', 1.0,
         'alpha mult', 0.8,
         'alpha min', 1e-4)

```

- I also extract the tangent matrix and RHS with:

```auto
K = md.tangent_matrix()
F = md.rhs()
dx = gf.linsolve_mumps(K, F)

```

- And I would like to solve with this

```auto
dx = gf.linsolve_mumps(K, F)

```

- I execute the code using:

```auto
mpirun -np 4 python script.py

```

My understanding is:  
In this setup, GetFEM assembles the tangent matrix and RHS in parallel, using Metis partitions (Metis\_1, Metis\_2, …, Metis\_N) associated with each MPI process.  
MUMPS then solves the linear system in distributed mode, with no additional MPI handling on my side.  
I do not need to change anything in my code logic, as long as I compile GetFEM with `GETFEM_PARA_LEVEL=2` and MUMPS is linked with MPI.

Could you please confirm that this understanding is correct?  
Also, is there anything important I should know if I want to go one step further by using `linsolve_mumps()` offers?

Thank you again for your help,  
Best regards,

---

<div class="post-metadata">

**Author:** ![Konstantinos.Poulios](https://avatars.discourse-cdn.com/v4/letter/k/ac91a4/32.png) [@Konstantinos.Poulios](https://getfem.discourse.group/u/Konstantinos.Poulios)\
**Post date:** [July 10, 2025, 7:37am UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93/4 "2025-07-10T07:37:01Z")

</div>

yes, I can confirm that for `md.solve()`

For `linsolve_mumps`, you just need to be on the latest version 5.4.4 or the current development version, which include this commit:

> **[Enable accessing parallel MUMPS in the scripting API linsolve function -...](https://cgit.git.savannah.gnu.org/cgit/getfem.git/commit/?id=3f88b0cbaf33dc600183b70dd5f69107e383627e)**

For even more advanced cases, where a matrix needs to be factorized once and used in multiple solves, I have recently (post 5.4.4) introduced a new mumps\_context object, that you can learn more about by studying the examples found here:

> **[check\_mumps\_ctx.py « python « tests « interface - getfem.git -](https://cgit.git.savannah.gnu.org/cgit/getfem.git/tree/interface/tests/python/check_mumps_ctx.py)**

---

<div class="post-metadata">

**Author:** ![Djoumessi](https://avatars.discourse-cdn.com/v4/letter/d/da6949/32.png) [@Djoumessi](https://getfem.discourse.group/u/Djoumessi)\
**Post date:** [July 10, 2025, 8:21am UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93/5 "2025-07-10T08:21:24Z")

</div>

Thank you very much for your confirmation.

Indeed, MPI seems to be working on my side: when I run a code with `mpirun`, the same process is executed on the number of processors I choose.

However, when I try the example you provided, I get the following error:

```auto
Traceback (most recent call last):
  File "/home/rene.djoumessi/getFemSim/code_c_real_col/TestMPI.py", line 36, in <module>
    if gf.util_mpi_parallelism_level() >= 2:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'getfem' has no attribute 'util_mpi_parallelism_level'

```

This seems to indicate that my current GetFEM installation was not compiled with `GETFEM_PARA_LEVEL=2`.

Thanks again for your help  
I’ll reach out again if I have further questions.  
Best.

---

<div class="post-metadata">

**Author:** ![Konstantinos.Poulios](https://avatars.discourse-cdn.com/v4/letter/k/ac91a4/32.png) [@Konstantinos.Poulios](https://getfem.discourse.group/u/Konstantinos.Poulios)\
**Post date:** [July 10, 2025, 8:25am UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93/6 "2025-07-10T08:25:12Z")

</div>

this simply means that the `util_mpi_parallelism_level` function was added to GetFEM after the version you are using.

---

<div class="post-metadata">

**Author:** ![Djoumessi](https://avatars.discourse-cdn.com/v4/letter/d/da6949/32.png) [@Djoumessi](https://getfem.discourse.group/u/Djoumessi)\
**Post date:** [July 10, 2025, 5:45pm UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93/7 "2025-07-10T17:45:44Z")

</div>

Thank you again for your help. One more clarification, please.

For now, I’m using version 5.3 of GetFEM, which I installed by cloning the repository some time ago. I’m not sure if I can use this version directly for parallel MUMPS support, or if I need to recompile GetFEM with version 5.4 or the development branch.

Thank you.

---

<div class="post-metadata">

**Author:** ![Konstantinos.Poulios](https://avatars.discourse-cdn.com/v4/letter/k/ac91a4/32.png) [@Konstantinos.Poulios](https://getfem.discourse.group/u/Konstantinos.Poulios)\
**Post date:** [July 10, 2025, 6:50pm UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93/8 "2025-07-10T18:50:28Z")

</div>

version 5.3 is too old

---

<div class="post-metadata">

**Author:** ![Djoumessi](https://avatars.discourse-cdn.com/v4/letter/d/da6949/32.png) [@Djoumessi](https://getfem.discourse.group/u/Djoumessi)\
**Post date:** [July 15, 2025, 1:16pm UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93/9 "2025-07-15T13:16:08Z")

</div>

Dear Konstantinos,  
I’ve try to compile the version 5.4 in our cluster and I’m getting the following error after ```make install``

```auto
 /usr/bin/mkdir -p '/home/rene.djoumessi/share/getfem/lib/python3.12/site-packages/getfem'
 /usr/bin/install -c -m 644 _getfem.cpython-312-x86_64-linux-gnu.so '/home/rene.djoumessi/share/getfem/lib/python3.12/site-packages/getfem'
Traceback (most recent call last):
  File "<string>", line 2, in <module>
ModuleNotFoundError: No module named 'imp'
make[4]: *** [Makefile:475: install-nodist_gfpyexecPYTHON] Error 1
make[4]: Leaving directory '/home/rene.djoumessi/libs/getfem-5.4.4/interface/src/python'
make[3]: *** [Makefile:568: install-am] Error 2
make[3]: Leaving directory '/home/rene.djoumessi/libs/getfem-5.4.4/interface/src/python'
make[2]: *** [Makefile:838: install-recursive] Error 1
make[2]: Leaving directory '/home/rene.djoumessi/libs/getfem-5.4.4/interface/src'
make[1]: *** [Makefile:455: install-recursive] Error 1
make[1]: Leaving directory '/home/rene.djoumessi/libs/getfem-5.4.4/interface'
make: *** [Makefile:598: install-recursive] Error 1
rene.djoumessi@musampro3:~/libs/getfem-5.4.4$ ~/share

```

This is a part of what I have on the terminal. the installation works under python 3.11 but failled under python 3.12, asking the following module `imp`  
Please any idea?  
Thanks in advance  
Rene

---

<div class="post-metadata">

**Author:** ![Konstantinos.Poulios](https://avatars.discourse-cdn.com/v4/letter/k/ac91a4/32.png) [@Konstantinos.Poulios](https://getfem.discourse.group/u/Konstantinos.Poulios)\
**Post date:** [July 15, 2025, 1:46pm UTC](https://getfem.discourse.group/t/parallel-mumps-solver-and-custom-solver/93/10 "2025-07-15T13:46:30Z")

</div>

this looks like some misconfiguration in your cluster, does not look like a GetFEM specific issue.