4K at 120 Hz on Windows, 60 Hz on Linux
Asking the amdgpu driver why it pruned a mode, finding a 5% margin on an RX 580, and rebuilding only one kernel module on NixOS.
My desktop has a Dell G3223Q, a 4K monitor that can do up to 144 Hz, connected over DisplayPort to an AMD RX 580. On Windows, the same machine, card and cable gave me 4K at 120 Hz. On Linux, I could only pick 4K at 60 Hz.
I had assumed the drivers were set up properly, because NixOS takes care of that for me. So the difference bugged me more than it probably should have.
The mode was there, then it wasn't
The monitor's EDID advertised 4K at 120 and 144 Hz. The list my compositor (niri) offered stopped at 60 Hz for 4K, while 1440p and 1080p went all the way to 144 Hz. Something between the monitor and the compositor was throwing the faster modes away.
My first suspect was the DisplayPort link. If it came up as HBR2 (DP 1.2) instead of HBR3 (DP 1.4), 4K at 120 Hz simply would not fit. The kernel exposes the negotiated link in debugfs:
sudo cat /sys/kernel/debug/dri/1/DP-2/link_settings
It said HBR3. So it wasn't the cable, and it wasn't a DP 1.2 setting in the monitor's menu.
A wrong first guess
Looking through the amdgpu sources, I found a table for this GPU generation (Polaris) with a pixel clock limit of 600 MHz:
.pixel_clk_khz = 600000,
It matched the mode list suspiciously well. 1440p at 144 Hz needs about 586 MHz and was offered; 4K at 120 Hz needs over 1000 MHz and wasn't.
It was a nice theory. It was also wrong, and I only found that out by asking the driver directly instead of reading more code.
Asking the driver why
The DRM subsystem can log why it rejects a mode. With the KMS debug category enabled, amdgpu prints a line for every mode it prunes, along with the reason. I turned it on, made the driver re-read the monitor, and turned it off again:
echo 0x4 | sudo tee /sys/module/drm/parameters/debug
echo detect | sudo tee /sys/class/drm/card1-DP-2/status
echo 0 | sudo tee /sys/module/drm/parameters/debug
sudo dmesg | grep 'Pruned mode'
The output looked like this (trimmed):
Pruned mode 3840 x 2160 (clk 1188000) RGB 8-bpc -- No DP link bandwidth
Pruned mode 3840 x 2160 (clk 1074730) RGB 10-bpc -- No DP link bandwidth
Pruned mode 3840 x 2160 (clk 1074730) RGB 8-bpc -- Bandwidth validation failure (BW and Watermark)
Pruned mode 3840 x 2160 (clk 1250000) RGB 8-bpc -- No DP link bandwidth
No mention of a 600 MHz limit. Instead, there were two different reasons.
Reading the log
The monitor offers 4K at 120 Hz with two timings. The standard one has a pixel clock of 1188 MHz. At 8 bits per colour (24 bits per pixel), that is about 28.5 Gb/s. An HBR3 link carries 25.92 Gb/s of data, so that mode genuinely does not fit. "No DP link bandwidth" is a physical limit, and no patch can change it.
The second timing uses reduced blanking: 1074.73 MHz. At 8 bits per colour it needs 25.79 Gb/s. That fits into HBR3, barely, with about half a percent to spare.
This mode failed for a different reason: "Bandwidth validation failure". That check isn't about the cable. It's the driver working out whether the card's own display engine can keep up.
The 144 Hz mode (1250 MHz) is out of reach either way. Polaris has no Display Stream Compression, so there is no way to squeeze it through the link.
A 5% margin
The bandwidth validation for this generation lives in drivers/gpu/drm/amd/display/dc/basics/dce_calcs.c. One of its inputs is a throughput factor for the display pipe, set to 105 / 100: the driver assumes the pipe needs 5% more than the raw numbers say.
For this mode, that makes the required display clock (dispclk) about 1134 MHz. The RX 580's display clock tops out at 1108 MHz, so the driver rejects the mode. Without the extra 5%, the same mode needs about 1080 MHz, which is within the card's normal top clock state.
The fix was one line:
dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_int_to_fixed(1);
This doesn't overclock anything. Voltages and clocks stay where they were. It removes a safety margin, which means the card now runs this mode with very little headroom.
Windows evidently accepts the same mode on the same card. I didn't investigate what AMD's Windows driver does differently.
Not rebuilding the whole kernel
On NixOS, the obvious way to apply a kernel patch is boot.kernelPatches. That works, but it changes the kernel derivation, so the binary cache no longer has it. My machine compiled the entire kernel for well over an hour to change one number.
Instead, I rebuilt only the amdgpu module from the same kernel source, as an out-of-tree module:
amdgpu = kernel.stdenv.mkDerivation {
pname = "amdgpu-polaris-4k120";
inherit (kernel) version src;
patches = [ ./patches/amdgpu-polaris-4k120.patch ];
nativeBuildInputs = kernel.moduleBuildDependencies;
buildPhase = ''
make ${lib.escapeShellArgs kernelModuleMakeFlags} -j"$NIX_BUILD_CORES" \
-C ${build} "M=$PWD/drivers/gpu/drm/amd/amdgpu" modules
'';
installPhase = ''
make ${lib.escapeShellArgs kernelModuleMakeFlags} \
-C ${build} "M=$PWD/drivers/gpu/drm/amd/amdgpu" \
"INSTALL_MOD_PATH=$out" INSTALL_MOD_DIR=updates DEPMOD=true modules_install
'';
};
The module is installed into updates/, which takes priority over the copy that ships with the kernel. The kernel itself still comes from the cache, and the module builds in about six minutes.
One build failure was worth the detour. The amdgpu tracepoint header points at its own directory with a relative path that only works inside a full kernel tree. My first fix used an absolute path under /build/linux-6.18.57/..., and the compiler couldn't find it. linux is a predefined macro in GCC, so the preprocessor turned the path into /build/1-6.18.57/.... Pointing the header at . worked, because the amdgpu Makefile already puts its directory on the include path.
Did it work?
Yes. After a reboot, niri listed 3840x2160@119.880, and the monitor has been running at 120 Hz since.
There is one trade-off I only noticed later. A second monitor, an LG at 4K and 60 Hz, hangs off the same card. With the Dell at 4K and 120 Hz, the display engine has no room left for it. Enabling both at once fails the driver's check, so I now switch between them one at a time. For my setup that's fine, but it's a direct consequence of removing the margin.
What I took away
The most useful thing here was the DRM debug log. I spent a while on a plausible theory from reading the source, and one log line made it obsolete. It said exactly which mode was rejected, at which bit depth, and by which check.
The rest was arithmetic: pixel clock times bits per pixel against the link rate, and the required display clock against the card's limit. Once the numbers were on the table, the 5% margin was easy to spot.
I worked through this with Claude, which was good at reading the driver code alongside me. But the part that settled it was asking the driver itself.