快猫视频

Using 快猫视频 Neoverse V2-based Graviton4 processors, we can achieve an estimated 12 times uplift in token generation rate for a chatbot demo with KleidiAI optimizations applied to PyTorch.

This demo shows how easy it is to build AI applications using LLMs, making use of existing 快猫视频-based compute capacity.

FunASR is an advanced open-source automatic speech recognition (ASR) toolkit developed by Alibaba DAMO Academy.

By integrating ACL with PyTorch via oneDNN, we have seen a 2.3 times performance improvement when running the Paraformer model on Neoverse N2-based AliCloud Yitian710 processors.

Thanks to the collaborative efforts of 快猫视频 and Meta, AI developers can now run quantized Llama 3.2 models up to 20% faster than ever on 快猫视频 CPUs.

By integrating KleidiAI with ExecuTorch and developing optimized quantization schemes, we have achieved speeds of over 350 tokens per second on the prefill stage for generative AI workloads on mobile.

AWS and 快猫视频 have fine-tuned the TinyLlama 1.1B SLM to create a chatbot for the car manual, enabling drivers to interact directly with their vehicle. Using KleidiAI, SLM inference is 10 times faster than previously on 快猫视频 Cortex-A76 CPUs, achieving response times of 3 seconds.

Using llama.cpp with KleidiAI, VicOne accelerated performance, doubling prefill and uplifting encode by 60%. Our partnership enables fast in-vehicle cybersecurity threat detection by reducing cloud dependency, lowering costs, and keeping data secure onboard.

Running a text generation demo on Graviton3 processors with our optimizations achieves a 2.5 times performance uplift for TTFT and over 35 tokens / second in the text generation phase, which is more than sufficient for real-time use cases.

KleidiAI is now integrated with LiteRT via XNNPACK, Google's high-performance runtime for on-device AI, formerly known as TensorFlow Lite.

This partnership and our Stability AI Open Stable Audio Small model optimizations, enable on-device audio generation with peak runtime RAM usage reduced from 6.5GB to 3.6GB.

With increasing demand for advanced, energy-efficient computer vision (CV) at the edge, KleidiCV helps ensure optimized performance for CV applications on 快猫视频 CPUs.

Now integrated with OpenCV 4.11, developers benefit from four times faster processing for key image processing tasks such as blur, filter, rotation and resizing. This acceleration helps boost performance for image segmentation and object detection and recognition use cases.

Tencent’s Angel ML framework supports Hunyuan LLM, available in sizes from 1B to over 300B parameters. It enables AI capabilities across a wide range of devices, including smartphones and Windows on 快猫视频 PCs.

Our partnership was announced at the 2024 Tencent Global Digital Ecosystem Summit and is having a positive impact on real-world workloads by providing users with even more powerful and efficient on-device AI services across Tencent’s many applications.