How to ensure that my simulation will pass without saturating my computer memory?
Deploying advanced methods in large industrial models is not always straightforward! This recently happened to one of our customer with whom we hit some memory problems during on-site deployment. With SDT for MATLAB we tend to keep everything in-core (i.e. loaded in RAM) for computation time optimization. With large models or “modest” computers this can quickly get difficult. In our case, we had a 3.3 million DOF model, a 200 modes basis to handle and 64GB of RAM (modest is relative in the computational world). For our expansion procedure, we need
- Assembled mass and stiffness matrices 15GB
- Factorized stiffness operator for mode computation and uplifts 35GB
- The 200 modes basis 5GB
- Enough space to realize matrix projections and sensor bases computation ?>10 GB?
In practice matrix products require a certain level of memory duplication. The result has to be allocated and then matrix operations are performed. It can lead to intense memory peaks that can either terminate the process or even crash the machine. In our application case raw data already weighed 55GB. It left little place to play on with the 5GB basis and associated memory peaks!
RAM footprint optimization strategies
In SDT we have developed RAM footprint optimization strategies with different levels to balance with computation slowdown.
- Level 1 keeps everything in-core but introduces operations by block to smoothen memory peaks. This is the most used case in our applications. Operations are decomposed and run blockwise to avoid large temporary matrices to be generated during the operations. BlasBufSize and MklServBufSize preferences control this behavior. They give target block sizes. Giving a 2GB size thus allows keeping the memory peak under 4GB in general while keeping good computation times. We experience a slowdown on critical operations by less than a factor 3.
- Level 2 applies Out-Of-Core strategies on bases. To gain even more RAM it becomes necessary to offload some data on the disk. Slowdown here gets higher and depend on the disk speed due to read/write operations that will be generated. EigOOC and OutOfCoreBufferSize preferences controls this behavior. They respectively give a basis threshold to offload data with the omat object and drive read/write buffer size. Drive format either uses HDF5 or our customized binary format that can also be parametrized depending on the machine. Slowdown can reach a factor 5 to 10 depending on the buffer sizes.
- Level 3 offloads base matrices. This is a simple thing to write matrices on disk and offload them from the RAM when not used in the current sequence. RAMoptim option controls this behavior for some procedures for which it makes sense. Slowdown is very limited but requires more disk space.
- Level 4 applies Out-Of-Core to the matrix factorization. The elephant in the middle, the matrix factor is usually the largest RAM user. For our current solution (mainly MKL Pardiso) an out-of-core capability exists. MklServOOC preference controls this behavior. Slowdown and residual peak RAM remain however important so that we keep it as the ultimate solution!
What about SWAP?
When the RAM gets full the OS resorts to use virtual memory, or SWAP https://thisvsthat.io/ram-memory-vs-swap-memory. This is comparable to an Out-Of-Core strategy at very low-level run by the system. Modern OSes can do this efficiently, tending to offload currently unused memory blocks like our level 3. Disk I/O intensity and GUI layer offloading, however, tend to slowdown or even make the computer non-responsive. Level 1 and 2 strategies should be preferred if sufficient.
- On Linux, the SWAP must be a dedicated partition. We recommend using a separate flash drive NVMe M.2 larger than the total RAM as priority SWAP. It is fast enough to let the computer run properly even with large swapping.
- On Windows, default behavior is not paramount. The configuration allows however to point virtual memory to another drive so that an equivalent to the Linux solution Is possible. What is critical to exploit swap is at least to prevent Windows from swapping on the partition where the system is installed. It is the default setup not to force users defining partitions, but it makes it dramatically inefficient!