Handy creator unveils transcribe.cpp: 60+ models, GPU support and a plug-and-play transcription alternative

Local speech transcription has long presented developers with a frustrating choice: juggle whisper.cpp or ONNX, and add MLX for Apple devices, resulting in the burden of maintaining two engine sets and porting models individually. On July 19, sebjones, creator and maintainer of the popular speech-to-text app Handy, transformed these cross-platform distribution challenges into transcribe.cpp, releasing version v0.1.0. Built on ggml, this library supports all current transcription models. Every model from the handy-computer organization on Hugging Face has passed numerical validation and word error rate (WER) testing, guaranteeing consistency with official reference implementations. Acknowledging this is an early release (v0.1.0) with potential rough edges, the author invites community feedback to drive improvements.

transcribe.cpp emerged from the painful reality of distributing cross-platform applications using existing ASR stacks, which often deliver a subpar user experience. While some libraries claim multi-model support, their obscure origins and unclear testing leave critical questions unanswered: Will updates cease? Are there official bindings for desktop or mobile apps? Is this merely demo code? How does it benchmark against ONNX? Seeking a solution for Handy, the author needed a library that allows direct model downloads and inference matching reference outputs, leverages GPU acceleration, integrates seamlessly without heavy dependencies like PyTorch, and supports Mac, Windows, and Linux. After evaluating options, ggml proved the ideal foundation due to its robust community and distribution capabilities.

transcribe.cpp offers a fast, accurate inference engine with broad model coverage, supporting 16 ASR families and over 60 models, with more on the way. Acceleration is achieved via four GPU paths: Vulkan, Metal, CUDA, and TinyBLAS. Treating Vulkan as the baseline for local inference, the author benchmarked each model on a Ryzen 4750U (Fedora) and an M4 Max. Every model underwent numerical validation and comprehensive WER scanning across thousands of audio segments, ensuring outputs match or closely align with reference implementations. These results are publicly accessible in the transcribe.cpp repository and corresponding Hugging Face model pages. Functionally, it supports streaming and batch transcription, serving as a drop-in replacement for whisper.cpp. Since Handy originally used whisper.cpp, the author ensured compatibility with its popular .bin files, allowing transcribe.cpp to run them directly. While some flags remain unaligned, the whisper implementation is reliable for most use cases, with comparable performance.

Recognizing that real-world adoption requires robust language bindings, the library prioritized this from the start. Written in C/C++, transcribe.cpp includes official bindings for Python, JavaScript/TypeScript, Rust, and Objective-C/Swift. The author welcomes community contributions for additional languages, driven by Handy’s success and the need to distribute local speech transcription widely. These bindings address the practical requirements of integrating voice functionality into popular applications.

The ultimate goal of transcribe.cpp is to simplify local ASR usage. With high accuracy achievable on most devices, cloud transmission becomes unnecessary. Highlighting its efficiency, the author notes that even on low-power hardware like the RK3566 chip, transcribe.cpp runs models faster than real-time on CPU, and handles real-time transcription with minimal power consumption using advanced models. As local inference grows due to privacy and latency concerns, distribution remains a key challenge. While transcribe.cpp doesn’t solve every issue, it represents a significant step forward in making local speech processing accessible and efficient.

The author extends gratitude to key contributors: Mozilla AI and its BiR project, along with engineer Davide, for early support; ggml and its community, which formed the project’s foundation; Modal, for providing computing credits for WER testing and CUDA verification; Blacksmith, for handling part of the CI/CD pipeline; and Hugging Face, a pillar of the local AI community that provided private storage space for the handy-computer organization’s models. Regarding AI assistance in development, the author confirms its use but emphasizes that no text was generated by AI; all content was written manually, noting that creating an engine of this scale in months using only ggml would be impossible without such support.

Original link: workshop.cjpais.com/projects/transcribe-cpp