BITCOS and the Case for Going Below 1.58 Bits

Key takeaways

• The 1.58-bit figure is not a universal minimum for storing ternary weights.
• Uneven value frequencies or predictable patterns can allow lower average storage costs.
• Lossless compression preserves existing ternary weights without adding quantization error.
• Faster inference depends on whether reduced data movement outweighs decompression costs.

For an AI model, moving weights into place can matter as much as doing the math. BITCOS starts with a useful question: can you store ternary weights in less than 1.58 bits per weight without changing their values? The answer depends on what that number actually measures.

Where 1.58 bits comes from

Ternary weights typically take one of three values: -1, 0, or +1. A single bit represents only two states, so storing each weight separately in a fixed-width field requires two bits.

Packing weights together changes the arithmetic. Two weights have nine possible combinations. Three have 27. A block of N weights has 3ᴺ.

To distinguish every possible block using a fixed-length binary code, you need:

ceil(N × log₂(3)) bits

As the block gets larger, the storage cost per weight approaches:

log₂(3) ≈ 1.585 bits

That calculation covers every possible combination with the same code length. It makes no allowance for some combinations appearing much more often than others.

The information content of a single ternary value also reaches 1.585 bits when all three outcomes are equally likely. Unequal probabilities lower that entropy. Relationships between successive values can create further opportunities for compression.

So 1.58 bits is a meaningful number. Treating it as an absolute floor for every compression scheme gives it a job it was never qualified to do.

More zeros can mean fewer bits

Compression rewards predictability. If a value appears frequently, assigning it a shorter representation can save more space than longer representations for rare values consume.

That is the principle behind entropy coding.

Consider a hypothetical set of ternary weights with this distribution:

Assuming the weights occur independently with those probabilities, their entropy is about 1.06 bits per weight. An ideal lossless coding scheme can approach that average over long sequences.

There are still three possible values. But guessing the next one has become considerably easier: zero is a strong bet.

This is an illustrative calculation, not a measured BITCOS result. Actual storage requirements also depend on block sizes and the extra information needed to decode them.

Even an equal mix of the three values may contain useful structure. Repeated sequences or predictable relationships between neighboring weights can offer savings to a compressor that recognizes those patterns.

The opportunity comes from the statistics of the data, not just the number of values it can contain.

Preserve the weights, change the packaging

The idea behind BITCOS is to keep the weight values intact while storing them more efficiently.

That distinction matters because quantization and lossless compression do different jobs.

Quantization reduces numerical precision or restricts the available values. Mapping a range of floating-point weights onto -1, 0, and +1 can introduce differences from the original model.

Lossless compression changes how those resulting values are recorded. If the input is -1, 0, 0, +1, decompression must recover exactly that sequence.

Compressing ternary weights this way adds no further quantization error. The reference point is the existing ternary model; any accuracy differences introduced by the earlier conversion from higher precision still remain.

Storage comparisons need equally careful boundaries. A compressed representation may require probability tables or indexes identifying where blocks begin. Those bytes count too.

For total model size, separately stored parameters such as scaling factors also belong in the calculation. A bits-per-weight headline is useful only when its accounting is clear.

Smaller files still have to earn their speed

Eventually, compressed weights have to become usable by the computation. How that happens determines which benefits survive deployment.

Two broad approaches illustrate the tradeoff:

• Decompress before execution. The smaller file helps with storage and transfer, but inference uses memory for the restored weight representation.
• Decompress as computation proceeds. Reading compressed blocks can reduce data movement, provided decoding feeds usable weights to the computation efficiently.

The second approach is especially interesting when performance is limited by how quickly the hardware can fetch weights from memory. Fewer bytes to read can mean less time waiting.

But decoding takes work. If decompression consumes all the time saved on memory access, the speed advantage disappears.

Variable-length representations also complicate access. With fixed-width weights, locating a particular weight is straightforward arithmetic. When encoded lengths vary, jumping to a specific region may require an additional index or block structure, with its own storage and processing costs.

That makes compression ratio only one part of the evaluation. Runtime memory use, token generation speed, and response latency all matter. Comparisons also need matching models, hardware, input lengths, and numbers of concurrent requests.

Going below 1.58 bits is entirely consistent with information theory when the weights contain exploitable statistical structure. The practical test for BITCOS is whether those saved bits become lower memory use and shorter waits while the model serves requests. The storage format has to prove itself at runtime.