OBS Video Encoding Masterclass: Keyframes, B-Frames, AV1 vs H.264 & Bitrate
Why your live stream looks pixelated in fast-motion foliage, the mathematics of Group of Pictures (GOP) temporal compression, why Twitch mandates 2-second keyframes, and how next-gen AV1 beats H.264 by 40% at identical bitrates.
01. Video Compression Mechanics: The Group of Pictures (GOP)
Raw, uncompressed 1080p video at 60 frames per second requires approximately 3,000,000 kbps (3 Gbps) of network throughput. To transmit this video over Twitch, Kick, or YouTube at consumer bandwidths of 6,000 to 8,000 kbps, encoders must compress video data by a factor of 500:1.
This massive compression is achieved through temporal inter-frame redundancy reduction, organized into an alternating sequence known as a Group of Pictures (GOP). An encoder processes three fundamental frame types:
A complete, self-contained standalone image (similar to a JPEG photo). It contains 100% of the pixel information required to render the frame without referencing any previous or future frames. It resets the decoder pipeline and corrects any accumulated transmission errors.
Forward-predicted frames that only encode the motion vectors and pixel changes that moved relative to the preceding I-frame or P-frame. Areas of the screen that did not move (like static UI HUDs) consume virtually zero bitrate.
The most efficient compression units. B-frames calculate motion trajectories by interpolating between both past reference frames AND future reference frames. They consume 60% less bitrate than P-frames while preserving smooth visual continuity.
GOP Size (Frames) = Framerate (FPS) × Keyframe Interval (Seconds)
At 60 FPS: GOP = 60 × 2 = 120 Frames
If your Keyframe Interval is set to 2 seconds, the encoder injects an uncompressed reference I-Frame exactly every 120 frames (once every 2000 milliseconds). This guarantees that any viewer joining your live stream has to wait at most 2 seconds before their web browser decodes a full picture.
02. Codec Showdown: H.264 (AVC) vs HEVC (H.265) vs AV1
For over twenty years, H.264 (Advanced Video Coding) has been the undisputed standard of internet video. However, H.264 relies on fixed 16x16 macroblock partitions that struggle desperately when high-velocity esports motion (grass in Battle Royales, rapid 180° mouse flicks) overwhelms the available bitrate.
| Codec Standard | Block Partition Flexibility | VMAF Quality Score @ 8000 kbps | Bandwidth Savings vs H.264 | Licensing / Royalty |
|---|---|---|---|---|
| H.264 (AVC High Profile) | Fixed 16×16 macroblocks | 78.4 / 100 | Baseline (0%) | MPEG-LA Royalties |
| HEVC (H.265) | Up to 64×64 CTUs | 87.2 / 100 | ~25% - 30% savings | High Patent Licensing Fees |
| AV1 (AOMedia Video 1) | Dynamic 128×128 Superblocks + Recursive Tiling | 94.8 / 100 | ~35% - 42% savings | 100% Open & Royalty-Free |
AV1 introduces 128x128 superblocks with recursive quad-tree sub-partitioning down to 4x4 blocks, directional intra-prediction in 56 angles, and non-linear filter deblocking. When streaming fast games on YouTube (via RTMP with AV1 or HLS) or Kick/Twitch via modern enhanced broadcasting protocols, AV1 renders crystal-clear 1440p60 gameplay at bitrates that would turn H.264 into a blurry, unreadable mush.
03. Hardware Silicon: NVIDIA NVENC vs CPU x264 Architecture
A persistent myth in streaming communities is that CPU software encoding (x264 Medium) looks vastly superior to hardware encoding. In 2026, this belief is completely obsolete:
The NVIDIA NVENC chip (found on GeForce GPUs) is physically separated from the CUDA compute cores. When OBS captures a frame from DirectX, the frame is transferred directly over internal GPU PCIe buffers into NVENC without touching system RAM or CPU caches. 8th and 9th Generation NVENC units feature dual hardware AV1/HEVC encoders operating in parallel, easily compressing 4K 120 FPS video with zero degradation to in-game 0.1% low frametimes.
Encoding 1080p60 via x264 Medium requires massive parallel compute threads. As the encoder competes against the game's physics and DirectX draw-call threads for L3 cache and memory bus bandwidth, CPU thread contention causes sudden thread stalls. This manifests as micro-stutters and input lag spikes in competitive titles (CS2, Valorant, Apex Legends).
04. Rate Control: Constant Bitrate (CBR) vs CQP / CRF
Selecting the wrong rate control mode in OBS Studio will either cause network frame drops or ruin the visual clarity of your video recordings:
CBR forces the encoder to output a strictly uniform amount of data every second (e.g. exactly 8000 kbps), regardless of whether you are standing still staring at a wall or firing a machine gun in a smoke grenade. Ingest servers (Twitch, Kick) require CBR because their edge CDNs cannot handle erratic bandwidth spikes without dropping TCP packets.
CQP does not restrict bitrate. Instead, you specify a target visual fidelity (e.g. CQ Level 18). During simple scenes (menus), bitrate drops to 3,000 kbps to save SSD disk space. During intense explosions and foliage motion, bitrate dynamically surges to 80,000 kbps, guaranteeing zero artifacting for video editing in Premiere or DaVinci Resolve.
05. Optimal OBS Studio Configuration Matrix (2026)
Use these verified engineering presets in OBS Studio (Settings -> Output -> Output Mode: Advanced) based on your hardware and streaming destination:
| Setting Parameter | Twitch / Kick Live Stream | YouTube Live (AV1 / Enhanced) | Local Master Recording |
|---|---|---|---|
| Encoder | NVIDIA NVENC H.264 | NVIDIA NVENC AV1 | NVENC AV1 or HEVC |
| Rate Control | CBR | CBR | CQP (Level 18 - 20) |
| Bitrate | 8000 kbps (Max Safe) | 12,000 - 18,000 kbps (1440p) | Variable (Dynamic) |
| Keyframe Interval | 2 s (Strict Requirement) | 2 s (Strict Requirement) | 0 s (Auto / 2s for editing) |
| Preset | P6: Slower (Better Quality) | P6: Slower (Better Quality) | P7: Slowest (Best Quality) |
| Tuning | High Quality | High Quality | High Quality |
| Multi-Pass Mode | Two Passes (Quarter Res) | Two Passes (Quarter Res) | Two Passes (Full Res) |
| Psycho Visual Tuning | Checked (Enabled) | Checked (Enabled) | Checked (Enabled) |
| Max B-Frames | 2 | 2 | 2 |
โ OBS Video Encoding & Codec FAQs
Everything you need to know about Unicode font generation, game rules, and device compatibility.