LongCat-Flash-Omni
A 560B-parameter open-source omni-modal model with 27B activated, excelling at real-time audio-visual interaction — text, image, audio, and video in a single end-to-end framework.
Independent third-party resource. Not affiliated with or endorsed by LongCat or Meituan.
Overview
LongCat-Flash-Omni is a 560-billion-parameter omni-modal model released by Meituan's LongCat team in November 2025. Built on the ScMoE architecture from LongCat-Flash-Chat, it extends the model to process text, image, audio, and video within a single end-to-end framework — enabling real-time audio-visual interaction without separate modality-specific models. On average, 27 billion parameters are activated per token through the Zero-Computation Experts mechanism. The model supports a 128K context window and was designed for deployment in the LongCat App (iOS and Android), where it powers real-time voice calls, image understanding, and multimodal chat. The publisher describes a curriculum-inspired progressive training strategy that transitions from simpler modality tasks to complex multi-modal sequence modeling, preserving strong unimodal performance while adding cross-modal capabilities.
Architecture
LongCat-Flash-Omni extends the ScMoE (Shortcut-connected Mixture of Experts) architecture from the Flash model family by adding modality-specific perception encoders and a speech reconstruction module. The key architectural innovation is a chunk-wise feature interleaving scheme that enables low-latency streaming multimodal input and output — the model can begin processing audio while it is still being spoken and generate response text or speech tokens before the input is complete. The publisher reports that this design sustains over 90% of the throughput achieved by text-only training, meaning the addition of vision and speech capabilities does not significantly degrade text performance or inference speed. A Modality-Decoupled Parallelism training scheme separates optimization across modalities during distributed training, allowing each accelerator to focus on its assigned modality's data pipeline without cross-modal interference.
Training
Training followed a curriculum-inspired progressive strategy: the model was first trained on pure text (leveraging the LongCat-Flash base), then gradually introduced to single-modality tasks (image understanding, speech recognition), and finally trained on complex multi-modal sequence modeling tasks that require simultaneous text, vision, and audio reasoning. The end-to-end evaluation included 250 real users and 10 expert reviewers via the LongCat App. The publisher reports strong performance across all modalities with competitive benchmark results spanning omni-modal, visual, video, and audio tasks.
Key Benchmarks
Publisher-reported scores. Not independently verified by this site. Descriptions summarize what each benchmark measures.
| Benchmark | Score | What It Measures |
|---|---|---|
| Omni-Bench | 61.38 (open-source SOTA) | Omni-modal benchmark — evaluates understanding across text, image, audio, and video simultaneously in multi-turn dialogue settings |
| WorldSense | 60.89 | Real-world multimodal understanding benchmark testing commonsense reasoning across modalities |
| MMBench-EN | 87.5 | Multi-modal understanding benchmark in English, covering perception and reasoning across 20 ability dimensions |
| DocVQA | 91.8 | Document visual question answering — measures ability to read and answer questions about scanned documents and charts |
| MVBench | 75.2 | Multi-modal video understanding benchmark covering 20 temporal reasoning tasks across long and short video |
| LibriSpeech (ASR) | CER 1.57 | Automatic speech recognition benchmark — character error rate on English read speech; lower is better |
Deployment
LongCat-Flash-Omni is deployed through the LongCat App (iOS and Android), where users can initiate real-time voice calls, upload images for analysis, and engage in multimodal conversation. Model weights are available on HuggingFace (meituan-longcat/LongCat-Flash-Omni) and GitHub (meituan-longcat/LongCat-Flash-Omni) under the MIT License. The model integrates with LongCat-Audio-Codec (0.43–0.87 kbps, ~100ms latency) for audio tokenization and detokenization. For self-hosted deployment, the 560B MoE model requires significant GPU resources — as with Flash-Chat, SGLang or vLLM with tensor and expert parallelism is the recommended serving approach. The LongCat API platform provides cloud access for developers who do not have sufficient local compute.
Capabilities
Described in the cited primary documentation. Not an independent performance evaluation.
- Real-time audio-visual interaction — text, image, audio, and video within a single framework.
- 560B total parameters (MoE), 27B activated per token on average.
- 128K token context window with multi-turn dialogue support.
- Open-source SOTA on Omni-Bench and WorldSense omni-modal benchmarks.
- Low-latency streaming multi-modal input/output with chunk-wise feature interleaving.
- Strong vision benchmarks: MMBench-EN 87.5, DocVQA 91.8, OCRBench 84.9.
- Audio capabilities: ASR (LibriSpeech CER 1.57), S2TT, TTS, and audio understanding.
- Video understanding: MVBench 75.2, VideoMME (w/ audio) 78.2.
- Integrates with LongCat-Audio-Codec (0.43–0.87 kbps, ~100ms latency).
- Deployed via LongCat App (iOS, Android) for real-time voice and image interaction.
Access & License
MIT License. Open-source release includes model weights and inference code.
Documented context window: 128,000 tokens (per the cited publisher API documentation).
Sources
- LongCat-Flash-Omni Technical Report (arXiv:2511.00279)
Primary sourcePublished 2025-11-03Accessed 2026-07-26
Technical report describing the omni-modal architecture, curriculum-inspired progressive training, Modality-Decoupled Parallelism training scheme sustaining >90% text-only throughput, and comprehensive benchmark results across omni-modal, vision, video, and audio tasks.
- LongCat-Flash-Omni GitHub Repository
Primary sourceAccessed 2026-07-26
Official repository under MIT License. Contains model architecture overview, evaluation results, and deployment instructions.
- LongCat-Flash-Omni on HuggingFace
Primary sourceAccessed 2026-07-26
Model weights and model card with comprehensive benchmark tables for omni-modal, vision, video, and audio evaluations.
- Meituan Tech Post — LongCat-Flash-Omni Announcement
Publisher documentationPublished 2025-11-03Accessed 2026-07-26
Publisher announcement detailing the launch of LongCat App alongside the model release, end-to-end evaluation with 250 users and 10 expert reviewers.
Related comparisons
- LongCat-Flash-Omni and DeepSeek V4-Flash-Vision-Exp: Omni-Modal vs Vision-Exp Compared
A source-based comparison of LongCat-Flash-Omni (560B open-weight omni-modal model, MIT License, text+image+audio+video in one framework) and DeepSeek V4-Flash-Vision-Exp (experimental vision model, text parity with V4-Flash, multimodal agent capability near Opus 4.8, API-only). Covers modality scope, benchmarks, pricing, licensing, and deployment.
FAQ
- Where can the primary source for LongCat-Flash-Omni be found?
- The official GitHub repository (meituan-longcat) and HuggingFace page contain the primary documentation, model weights, and license for LongCat-Flash-Omni.
- Is LongCat-Flash-Omni available through the LongCat API?
- LongCat-Flash-Omni is documented in the meituan-longcat GitHub organization. Refer to the publisher API documentation at longcat.chat for current API availability.
- Is this page an official LongCat or Meituan page?
- No. This page is published by an independent third-party site. It is not affiliated with, endorsed by, or sponsored by LongCat or Meituan.
- How often is this page updated?
- This page was last verified on 2026-07-26. Content is reviewed when new publisher documentation or model releases become available.
Independent third-party disclosure
This page is published by an independent third-party site. It is not affiliated with, endorsed by, sponsored by, or operated by Meituan, LongCat, or any of their affiliates. The content summarizes publicly-available primary documentation and does not represent the views of any referenced organization.
Last reviewed: 2026-07-26