MOSS-TTS Local Transformer v1.5 on SGLang-Omni
A short pointer to the LMSYS article on work I contributed to: serving MOSS-TTS-Local Transformer v1.5 on SGLang-Omni with native streaming at 48 kHz stereo.
Wenyao Gao
My main interest and focus is on machine learning systems and large language model inference — serving models, managing KV cache, squeezing latency out of GPUs. I also contribute to open-source inference engines like SGLang and vLLM. This site is where I write down what I learn, lessons and the mistakes along the journey.
Check my blogs and projects below!
Featured
Ming-Omni on SGLang: Architecture and the Optimizations Behind Fast Omni Serving
How we serve Ming-Omni in SGLang: the unified multimodal architecture, and the optimizations — an encoder kernel fix, tensor parallelism, CFM CUDA-graph capture, and streaming TTS — behind fast omni serving.
Higgs Audio v3 TTS on SGLang-Omni
A short pointer to the LMSYS article on work I contributed to: serving Higgs Audio v3 TTS with SGLang-Omni for real-time, controllable voice agents.