THE ORA BLOG

Blog

Technical deep-dives, benchmarks, and case studies from the Ora Computing team.

TECHNICAL•2026-09-03•~17 min read

Where the Bits Go: Non-Uniform Quantization of Qwen3.5-9B

Not all parts of a model deserve the same precision. We present a quantization stack that spends a fixed bit-width budget across a model, allocating precision where it matters most. Applying it to Qwen3.5-9B, we produce a family of models at different compression levels, all runnable on standard llama.cpp. Our findings show that smart allocation and representation choice significantly improve the reasoning ability of aggressively quantized models.

INFRASTRUCTURE•2026-08-27•5 min read

How We Scale ML Workflows without Losing Our Agility

Getting the newest LLM compression tech to customers fast is hard when you need a whole zoo of GPUs and can't afford to lose track of your work. Here is the small, cheap infrastructure we built to do it — SkyPilot to find and manage GPUs across clouds, MLflow to keep every experiment organized and safe.

TECHNICAL•2026-08-21•13 min read

Quantizing Small Models to 3 and 2 Bits with ORA-QAT

We push Qwen3-4B from 16 bits to 3 and 2 with our novel algorithm for efficient quantization-aware training. At 3 bits, the model occupies only 22% of the BF16 size and keeps almost 97% of full-precision quality, beating the GPTQ baseline by 5.3 points on average; at 2 bits it occupies only 18% and more than doubles GPTQ's score. The training runs on a single GPU and produces a quantized checkpoint in just an hour.