ARM32: Massive /memfd:doublemapper memory leaks causing OOM kills and segfaults during JIT compilation

[ifv-sel]

Description

On ARM32 embedded Linux systems, .NET 8.0.121 exhibits severe memory leaks where /memfd:doublemapper (deleted) entries accumulate indefinitely, leading to virtual memory fragmentation, OOM kills, and segmentation faults. This appears to be the same issue described in #89776.

Process memory mappings show thousands of leaked doublemapper entries:

0x91287000 0x91297000    0x10000  0x1c31000 /memfd:doublemapper (deleted)
0x91297000 0x912a7000    0x10000  0x1c21000 /memfd:doublemapper (deleted)
0x912a7000 0x912b7000    0x10000  0x1c11000 /memfd:doublemapper (deleted)
... [thousands more entries]

Crash Symptoms

1. Segmentation faults during JIT compilation in fragmented memory regions
2. OOM kills when virtual address space is exhausted
3. Stack corruption during exception handling due to memory fragmentation

Reproduction Steps

https://github.com/ifv-sel/Doublemapper

Expected behavior

• JIT code pages should be properly unmapped when no longer needed
• /memfd:doublemapper (deleted) entries should not accumulate
• System should handle large file operations without memory exhaustion

Actual behavior

• Doublemapper entries accumulate indefinitely
• Virtual address space becomes heavily fragmented
• Process crashes or gets OOM killed during normal operations

Regression?

Haven't tested earlier than 8.0.4.07

Known Workarounds

The following environment variables prevent doublemapper accumulation:

export DOTNET_TieredCompilation=0      # Primary fix - prevents recompilation
export DOTNET_ReadyToRun=0             # Forces consistent JIT path  
export DOTNET_gcServer=0               # Reduces memory overhead
export DOTNET_JitMinOpts=1             # Minimal JIT optimizations
export DOTNET_EnableWriteXorExecute=0  # Bypasses doublemapper entirely

Configuration

• Platform: ARM32 (socfpga-cyclone5 - Intel Cyclone V FPGA SoC)
• OS: Embedded Linux
• .NET Version: 8.0.121
• Architecture: ARM32
• Memory: Constrained embedded environment

Other information

Stack Traces

JIT Compilation Crash

#0  0xb6859320 in ClassLoader::LoadConstructedTypeThrowing at clsload.cpp:965
#1  0xb685a3d4 in ClassLoader::LoadGenericInstantiationThrowing at clsload.cpp:1724
...
#29 0xb6a14036 in ThePreStub () at arm/asmhelpers.S:199
#30 0x93254004 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Exception Handling Crash

#0  IsFunctionFragment (baseAddress=2449276928, pFunctionEntry=0x0) at codeman.cpp:831
#1  FindRootEntry (baseAddress=2449276928, pFunctionEntry=<optimized out>) at codeman.cpp:939
...
#21 0x91fd4034 in ?? ()   # JIT-compiled code region

[dotnet-policy-service]

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.