LongCat-AudioDiT
A non-autoregressive diffusion text-to-speech model operating directly in waveform latent space, with 1B and 3.5B variants achieving SOTA zero-shot voice cloning.
Independent third-party resource. Not affiliated with or endorsed by LongCat or Meituan.
Overview
LongCat-AudioDiT is a non-autoregressive diffusion-based text-to-speech (TTS) model released by Meituan's LongCat team in March 2026. Available in 1-billion and 3.5-billion-parameter variants, it achieves state-of-the-art zero-shot voice cloning performance, surpassing the previous SOTA model Seed-TTS on speaker similarity (Seed-ZH SIM 0.818 vs 0.809; Seed-Hard SIM 0.797 vs 0.776). The model's core innovation is operating directly in waveform latent space via a Wav-VAE encoder, bypassing traditional mel-spectrogram intermediates — a design choice that the publisher argues eliminates the compounding errors introduced by multiple signal representations. With a compression ratio exceeding 2,000x (24kHz audio compressed to an approximately 11.7Hz frame rate), AudioDiT enables efficient training and inference on large-scale speech data. The model was trained on 1 million hours of Chinese and English speech without requiring high-quality human-annotated datasets — a significant cost advantage over traditional TTS approaches that depend on studio-quality recordings.
Architecture
LongCat-AudioDiT is built on a three-component architecture described in the technical report (arXiv:2603.29339). First, a Wav-VAE encoder compresses raw 24kHz waveform audio into a compact latent representation with over 2,000x compression — reducing a second of speech to approximately 12 latent frames. Second, a Diffusion Transformer (DiT) operates in this compressed latent space, performing the non-autoregressive denoising process to generate speech latents conditioned on text input and a reference speaker embedding. Third, a Wav-VAE decoder reconstructs the waveform from the denoised latents. At inference time, the model uses Adaptive Projection Guidance (APG) — a replacement for traditional Classifier-Free Guidance (CFG) that the publisher describes as yielding more stable and higher-quality generation. The speaker embedding is extracted from a single reference audio sample for zero-shot voice cloning, with no additional fine-tuning required.
Training
The model was trained on 1 million hours of Chinese and English speech data, notably without relying on high-quality human-annotated datasets. The publisher positions this large-scale self-supervised or weakly-supervised training approach as a key differentiator: by using readily available speech data at scale rather than expensive studio recordings, the model learns robust speech representations that generalize across speakers and languages. Training jointly covers both Chinese and English without language-specific modules.
Key Benchmarks
Publisher-reported scores. Not independently verified by this site. Descriptions summarize what each benchmark measures.
| Benchmark | Score | What It Measures |
|---|---|---|
| Seed-ZH (SIM) | 0.818 | Chinese speaker similarity benchmark — measures how closely cloned speech matches the reference speaker's voice characteristics |
| Seed-Hard (SIM) | 0.797 | Difficult-case speaker similarity benchmark — tests voice cloning on challenging speakers and speech patterns |
Deployment
Model weights for both 1B and 3.5B variants are available on HuggingFace (meituan-longcat/LongCat-AudioDiT) with inference code. The GitHub repository (meituan-longcat/LongCat-AudioDiT) provides training and inference pipelines. The 1B variant is optimized for scenarios requiring lower latency and reduced VRAM (edge devices, real-time applications), while the 3.5B variant targets maximum quality at the cost of higher compute requirements. Deployment requires a GPU with sufficient VRAM for the DiT backbone.
Capabilities
Described in the cited primary documentation. Not an independent performance evaluation.
- Zero-shot voice cloning: Seed-ZH SIM 0.818, Seed-Hard SIM 0.797.
- Two model sizes: 1B and 3.5B parameters.
- Wav-VAE encoder with >2,000x compression (24kHz to 11.7Hz frame rate).
- Adaptive Projection Guidance (APG) replacing classifier-free guidance.
- Trained on 1M hours of Chinese and English speech.
- Waveform latent space direct generation — no mel-spectrogram intermediates.
Access & License
Open source. Weights and code publicly released.
Sources
- LongCat-AudioDiT (arXiv:2603.29339)
Primary sourcePublished 2026-03-31Accessed 2026-07-27
Wav-VAE + DiT architecture, APG replacing CFG, SOTA zero-shot voice cloning: Seed-ZH SIM 0.818, Seed-Hard SIM 0.797.
- LongCat-AudioDiT on HuggingFace (1B, 3.5B)
Primary sourceAccessed 2026-07-27
Model weights for 1B and 3.5B variants with inference code.
- LongCat-AudioDiT GitHub Repository
Primary sourceAccessed 2026-07-27
Training/inference code and model configuration.
FAQ
- Does LongCat-AudioDiT support voice cloning with a single reference sample?
- Yes. LongCat-AudioDiT achieves zero-shot voice cloning, meaning it can clone a speaker's voice from a single reference audio sample without fine-tuning. The 3.5B variant reaches a speaker similarity of 0.818 on Seed-ZH.
- Is this page an official LongCat or Meituan page?
- No. This page is published by an independent third-party site. It is not affiliated with, endorsed by, or sponsored by LongCat or Meituan.
- How often is this page updated?
- This page was last verified on 2026-07-27. Content is reviewed when new publisher documentation or model releases become available.
Independent third-party disclosure
This page is published by an independent third-party site. It is not affiliated with, endorsed by, sponsored by, or operated by Meituan, LongCat, or any of their affiliates. The content summarizes publicly-available primary documentation and does not represent the views of any referenced organization.
Last reviewed: 2026-07-27