Sum of Products vs Sum of Shifts
Two fundamentally different approaches to weighted accumulation. Both compute the same answer — 13 × 11 = 143 — but one uses brute-force long multiplication (sum of partial products) while the other decomposes the weight into powers of two (sum of shifts). The gate count tells the story.
🏭 Sum of Products: Long Multiplication in Silicon
The same pencil-and-paper method you learned in school — implemented in 3,200 logic gates
1101 (13)
× 1011 (11)
──────
1101 ← partial product 0 (×1, shift 0)
1101· ← partial product 1 (×1, shift 1)
0000·· ← partial product 2 (×0, shift 2)
1101··· ← partial product 3 (×1, shift 3)
────────
10001111 ← add all rows + carry propagation
= 143 ✓
-
1
Read each bit of the multiplier to decide which rows to generate
240 gates
-
2
Generate every partial product row — one AND gate per bit pair
700 gates
-
3
Feed all rows into the Wallace Tree to reduce them with carry-save adders — the single largest block
1,000 gates
-
4
Final carry-propagate addition to resolve all remaining carries
300 gates
-
5
Normalize the result, round, handle exceptions (NaN, overflow, underflow)
370 gates
-
6
Store result in pipeline registers for the next clock cycle
300 gates
3,200 gates → 4.6 pJ (Horowitz 45nm)
🧮 Sum of Shifts: The Abacus Principle
Don't multiply — decompose the weight into powers of two, then shift and add
13 = 16 − 4 + 1
= 2⁴ − 2² + 2⁰ ← 3 terms (R=3)
13 × 11
= (2⁴ − 2² + 2⁰) × 11
= (11 << 4) − (11 << 2) + (11 << 0)
= 176 − 44 + 11
= 143 ✓
Three shifts. Two adds. Zero multiplies.
-
1
Barrel-shift the input by the first exponent (e=4) — this is just moving wires, near-zero energy
20 gates
-
2
Check the sign bit — add or subtract? Just an XOR on each bit
16 gates
-
3
Accumulate into a single integer adder — the only real computation
24 gates
-
4
Control & sequencing — SRAM coefficient read, cycle counter FSM, accumulator latch
~20 gates
-
↻
Repeat for remaining terms (shift by e=2, then e=0). Same 80 gates, reused each cycle
0 extra
80 gates fired × 3 cycles → 0.15 pJ
Same answer: 143. The MAC fires 3,200 gates once. The BSA fires 80 gates three times — same hardware, reused.
The MAC exists because we chose to multiply. BSA asks: what if we never multiply at all?
⚡ It Gets Better: Dynamic-R and QAT
The example above used R=3 terms for the weight 13. But real neural network weights aren't random — after
Quantization-Aware Training (QAT), the training process pushes weights toward exact powers of two.
Most weights need only 1 or 2 terms. The average across billions of parameters:
Actual mean (LLMs)
R̄ = 1.6
Now combine Dynamic-R with attention sparsity (most attention values are near-zero and can be skipped).
This creates two-dimensional sparsity — savings multiply:
| Sparsity Dimension |
What It Skips |
Reduction |
| Dynamic-R (weight rows) |
Unnecessary shift-add terms per weight |
5× (1.6 vs 8 cycles) |
| CSC Attention (columns) |
Near-zero attention entries |
10× (at 10% density) |
| Combined (2D sparsity) |
Both — compound savings |
~50× → 2% of dense FP32 |
A BSA PE with Dynamic-R fires its ~80 active gates an average of 1.6 times per weight, not 8.
Combined with attention sparsity, total compute drops to ~2% of dense FP32.
Validated on 8 production LLMs — Mistral, LLaMA, Gemma, Phi-3, Qwen, DeepSeek, Falcon — with perplexity loss < 0.3.
🧮 The Name Was Never an Accident
Sibacus — from abacus. The abacus was humanity's first computing tool,
and it worked on exactly this principle: positional shift and accumulate.
No multiplication. Just slide beads to new positions and count the result.
The MAC was a 50-year detour into brute-force arithmetic.
BSA is a return to the most energy-efficient computational principle ever discovered —
now implemented in silicon instead of wooden beads.