Elmer vs FEniCS - Part 3
Stefano Tronci
In the Elmer vs FEniCS - Part 2 episode we studied the effect of mesh size on accuracy. We largely found the same results observed in the Mesh Order and Accuracy episode. We also confirmed that the agreement between FEniCS and Elmer is very good. However, we noticed that FEniCS was significantly slower than Elmer. This because we used a MUMPS direct method on FEniCS and a BiCGStabl iterative method on Elmer. In this episode we will explore the iterative solver options for FEniCS and attempt to optimize execution time for our simulation.
Project Files
All the files used for this project are available at the repositories below:
The Problem
In the Elmer Model and Solver Parameters episode we discussed the various Elmer solver parameters. FEniCS is not dissimilar. In FEM we turn PDEs into algebraic equations in vector-matrix form. Since these matrices are big solving the equations by controlling numerical errors is hard. At the same time, these matrices can get so large that even modern day computers can have a hard time solving them directly. For these reasons there are numerous direct as well as iterative algorithms. Iterative algorithms are preferred for large problems as they save execution time and resources.
Computational linear algebra is a mature field. A plethora of methods exist to solve linear algebra problems. These methods are implemented in many libraries, such as PETSC, MUMPS, BLACS, LAPACK and ScaLAPACK just to name a few. Most scientific software depends on these libraries. As a result, most solver parameters are very similar across different PDE solvers. Hence, most of what we discussed about in the Elmer Model and Solver Parameters episode applies to FEniCS as well.
For this episode we explored the FEniCS solver parameters in order to find a better performing set of values. To do so we used the fine mesh (5 mm) from the Elmer vs FEniCS - Part 2 episode. We did so with various iterative solvers until we found something that worked well.
Results
Trial and error is often a good way to search for solver parameters, unless the specifics of our problem already point to an optimal algorithm in some sense. After some trial and error we set on a gmres iterative solver with ilu preconditioning. This because BiCGStabl and ilut, which we used in Elmer, are not available in FEniCS. In terms of convergence tolerances and maximum iterations we matched the parameters that we used with Elmer ($10^{-10}$ and $10000$ respectively). The results are presented below.
Execution Time
The execution times for the simulations are reported below.
Elmer: $204$ seconds.FEniCS: $55$ seconds.
Solution
As done in the Elmer vs FEniCS - Part 2 episode, we take each mesh node and evaluate the solution at the node. We compute the distance of the node from the source centre and plot magnitude and phase of the solutions as functions of this distance. We do this because the Pulsating Sphere field depends only on distance. The solutions are shown below.
Click Here for Larger Version
Error
As done in the Elmer vs FEniCS - Part 2 episode, for each solver we compute the quotient between the solution and the exact field. By taking the decibel value of the magnitude of this quotient we will have the magnitude error in $\text{dB}$. The angle of this quotient is the phase error. The results are shown below.
Error Values
Click Here for Larger Version
Error Statistics
Click Here for Larger Version
Discussion
FEniCS with the gmres solver is quite a lot faster than Elmer with the BiCGStabl solver, about $3.7$ times faster. However, it is not as accurate. The error plots show higher magnitude errors, especially towards the termination of the domain. Phase errors are somewhat comparable and very low. Despite the FEniCS errors being higher than the Elmer errors they are still within $0.1$ $\text{dB}$. This is the same error we expect from state of the art measurements. We can hence consider FEniCS accuracy satisfying. Phase errors are within $200 \cdot 10^{-6}$ $\text{rad}$ ($0.01$ $\text{deg}$), which is also very good.
Whilst the error is not statistical in nature error statistics allow us to quickly eyeball trends in the error that are not obvious. Kernel Density Estimation was performed on the error data as in the Elmer vs FEniCS - Part 2 episode. The error statistics show a lower mean and an upward spread for the FEniCS error. The FEniCS phase error has similar spread to its Elmer counterpart, but it is multi-modal.
We found that FEniCS delivered a good solution with good accuracy much faster then Elmer. However, we shouldn’t be quick to jump to conclusions about the two solver packages. First, we did not compare the exact same solver algorithms. This because FEniCS and Elmer offer a different selection of solvers. It is entirely possible to lower Elmer’s accuracy by selecting another solver-preconditioning pair while lowering execution time as well. Also, there is a more subtle issue. The simulations were performed on the same computer, a NUC11PHKi7C with $64$ $\text{GB}$ of RAM. This computer run Arch Linux. However, while Elmer was installed from the arch4edu repository, FEniCS and its dependencies were installed from AUR. This means that FEniCS and many of its dependencies, such as PETSC, were built from source on the local machine, rather then being a pre-built package that is downloaded. This in turns can make the software more optimized.
Nevertheless, we shown that FEniCS is perfectly capable of solving PDEs with very good compromises between accuracy and speed.
Conclusion
In this episode we solved the pulsating sphere problem over the fine mesh from the Elmer vs FEniCS - Part 2 episode. This time, instead of using a direct solver for FEniCS, we used an iterative gmres solver with ilu preconditioning. FEniCS delivered a less accurate solution, but still very accurate, within $0.1$ $\text{dB}$ for magnitude and $0.01$ $\text{deg}$ for phase. The solution time for FEniCS was under a minute on a high performance mini-pc (NUC11PHKi7C), while the Elmer solution was solved in $3.5$ minutes on the same machine. This shows that FEniCS is capable of providing good accuracy with good computation times.
License Information
This work is licensed under a Creative Commons Attribution 4.0 International License.