Answer First
HeyGen and Google Cloud documented Avatar IV running on a single eight-chip Trillium (v6e) host. After the first TPU version worked, the teams made that same workload 1.86× faster while retaining output-quality gates.
The 1.86× number is not a generic TPU-versus-GPU benchmark. It compares the optimized TPU build with the first working TPU build. The source separately reports performance comparable to HeyGen’s 8×H100 production setup and up to 25% better cost efficiency per generated video minute for this pipeline.
The Documented Workload
Avatar IV is described as a more-than-18-billion-parameter diffusion stack available through HeyGen’s web product and API. It turns one photo and an audio track into a talking, moving person.
Each streamed video chunk passes through:
- a diffusion transformer conditioned on audio;
- a second transformer for super-resolution;
- a VAE decoder that converts latents into pixels.
The output is 720p or 1080p at 25 frames per second. Playback begins while later chunks are still rendering, so a late chunk can stall the stream. End-to-end chunk time—not an isolated kernel score—was the production constraint.
Why the Port Needed Eight Chips
The two transformers total more than 36 GB of bf16 weights, while each Trillium chip has 32 GB of HBM. HeyGen therefore used:
- FSDP weight sharding across eight chips;
- Ulysses sequence parallelism across the same mesh;
- Trillium SparseCore for asynchronous weight gathers;
- torchax to dispatch PyTorch production code to JAX arrays and XLA;
- Pallas kernels specialized for the pipeline’s attention shapes.
This is not evidence that every video model needs eight chips. The topology follows this model’s weights, sequences, stages, deadlines, and target output.
Where the 1.86× Improvement Came From
The official engineering record describes several cumulative changes:
| Bottleneck | Workload-specific response |
|---|---|
| Exposed all-to-all collectives | Split attention heads into groups so transfers could overlap sibling computation |
| Partial blocks in sparse attention | Use frame-aligned block sizes so live blocks were full and mask/padding work could be removed |
| Serial online-softmax dependency | Use a proven norm-based upper bound for eligible heads, with fallback for heads where the bound was too loose |
| Repacking before collectives | Fuse operations into a Pallas kernel that writes the physical layout expected by the collective |
| Scheduler blind spots | Provide explicit scheduler settings and honest cost estimates for custom kernels |
The team says each change had to win end to end. A collective optimization that looked faster alone could lose when the full pipeline saturated the interconnect.
Quality Gates Were Part of the Result
HeyGen used two tiers:
- Byte-identical gate: delivered video hashes had to match frame for frame when a change should preserve reduction order.
- Bounded-difference gate: changes that altered reduction order had to stay inside a separately measured bf16 reassociation band; anything below it went to model owners for blind, frame-by-frame review.
The team also required dual baseline runs to hash identically before trusting a delta. A faster residual-stream candidate was discarded because output fell below the quality band. That rejected result is as important as the reported speedup: performance did not override acceptance criteria.
What “Comparable to 8×H100” Means
The source reports comparable production performance for the documented Avatar IV pipeline and up to 25% better cost efficiency per generated minute. It does not establish:
- universal superiority of TPU over H100;
- identical price, availability, quota, or utilization for another customer;
- equal performance for training or a different inference graph;
- zero porting or compiler-engineering cost;
- automatic preservation of output quality.
Include accelerator, host, networking, storage, compilation, idle capacity, failed runs, observability, engineering, and migration risk in the comparison.
Who Should Evaluate This Pattern
The pattern is relevant when a large streamed video pipeline has repeatable shapes, end-to-end latency pressure, enough scale to amortize compilation and optimization, and a quality test strong enough to compare accelerator builds.
Start with the Avatar IV GPU-to-TPU migration checklist and use the Avatar IV deployment readiness tool to identify missing workload, quality, operations, and cost evidence.
Frequently Asked Questions
What Avatar IV workload did HeyGen run on TPU?
HeyGen and Google documented the more-than-18-billion-parameter Avatar IV talking-head pipeline on one eight-chip Trillium v6e host. It turns a photo and audio into streamed 720p or 1080p video at 25 frames per second using two transformers and a VAE decoder.
Was Avatar IV 1.86x faster than GPU?
That is not what the source says. The 1.86× figure compares the optimized TPU pipeline with HeyGen’s first working TPU version. The post separately says the result was comparable to its 8×H100 production setup for this workload.
Did the TPU port require a full JAX rewrite?
No. HeyGen used torchax, a PyTorch frontend on JAX, so production model code could target JAX arrays and XLA. TPU-specific work still included sharding, custom Pallas kernels, compiler settings, layout, and scheduling contracts.
What cost result did HeyGen report?
The post says the final pipeline was up to 25% more cost efficient per generated video minute than HeyGen’s 8×H100 production setup. This is a workload-specific report, not a universal TPU price guarantee.
How did HeyGen protect output quality?
The team used two quality tiers: byte-identical delivered video where reduction order was preserved, and a measured similarity band plus blind frame-by-frame owner review when numerical order changed. Candidates below the band were rejected.