Performance is slower and slower on long-time adaptive FEM

I am building codes for an adaptive local mesh refinement scheme in finite element method.

The 4-by-ne element and 2-by-nd node are used to store node connectivities of elements and coordinates of nodes, respectively. ne and nd are unknown and will increase in running.

In each computation iteration step, ne and nd are decided until the refined domain is determined.

The mesh is refined and updated by a user-defined function, [element, node] = RefineMesh(element, node, ...), element and node are then updated within:

element = [element, newElem];
node = [node, newNode];

Note that preallocation is used for newElem and newNode.

Some other variables are also variable-size, such as energy field values on integration points, these are recreated by preallocation in each iteration:

energy = zeros(4, size(element,2));

I also use parfor to assemble global stiffness matrix over elements. The key procedures of single element stiffness matrix are tranformed to MEX files using codegen. The sparse matrix is recreated in each iteration. GPU computing is also used to solve Ku=F iteratively.

My questions are:

1. Why the performance is lower and slower during long-time running athough I follow some approapriate suggestions in Techniques to Improve Performance (such as preallocation and vectorization). To be specific, the assembly time is getting longer.
2. Finally, this may lead to out-of-memory error. However, when I exit and restart MATLAB, the computation can continue from where it was last aborted. Note that clear all and delete(gcp('nocreate')) do not work for addressing the out-of-memory error, only exiting and restarting effective. This is very strange.

Answer 1

[surya venu]

Hi,

Your situation touches on several complex aspects of optimizing MATLAB code, especially for computationally intensive tasks like finite element method (FEM) simulations that involve adaptive mesh refinement.

Here are the explanations:

Performance Degradation Over Time

1. Memory Fragmentation: Repeated allocation and deallocation of arrays can lead to memory fragmentation. Although MATLAB handles memory management internally, fragmentation can still occur, especially with variable-sized data structures that change size frequently. This can slow down memory access and allocation over time.
2. Growing Data Size: As your mesh refines, both "ne" and "nd" increase, leading to larger matrices and arrays. Operations on larger datasets naturally take longer, contributing to the impression of decreasing performance. This is compounded by any loops or operations that scale poorly with data size.
3. Sparse Matrix Operations: If the sparsity pattern of your matrices changes with each refinement (which is likely in adaptive mesh refinement), MATLAB has to recompute the optimal storage each time you recreate the sparse matrix. This can add significant overhead.
4. MEX File Overheads: While MEX files can speed up operations, the overhead of calling a MEX file from MATLAB might become noticeable, especially if these calls are made frequently within a tight loop.
5. Parallel Overheads: The "parfor" loop has overheads, especially in distributing data to workers and gathering results. If the computations within each iteration of the "parfor" loop are not sufficiently large, the overhead can dominate the computation time.

Recommendations

• Profile Your Code: Use MATLAB's Profiler (profile on; profile viewer) to identify bottlenecks. Pay special attention to functions that take longer over time. Check out: https://www.mathworks.com/help/matlab/ref/profile.html
• Investigate Memory Usage: Use MATLAB's memory function to monitor memory usage over time. This can help identify leaks or unexpected memory growth. Check out: https://www.mathworks.com/help/matlab/ref/memory.html
• Parallel Computing: Ensure that the overhead of distributing tasks in "parfor" is justified by sufficiently large computations within each loop iteration. Also, consider using fewer workers if overhead dominates.
• MEX Memory Management: If you're using custom MEX files, ensure they manage memory correctly. Incorrect memory management in C/C++ code can lead to leaks that MATLAB cannot recover.

Hope it helps.