PrimeNeural/pnn-vit-b16-v41-d50-1bit

1-bit PNN 50 % ViT-B/16 — a vision transformer in 8.9 MB

© PreciseStatistics GmbH (Prime Neural Networks). Patent pending. Proprietary weights, published for evaluation. Running them requires the PNN package with 1-bit support, available on request.

A ViT-B/16 in which all 48 linear layers of the 12 transformer blocks are 1-bit PNN layers at 50 % density: every connection is a sign, +1 or −1, and each output reads half of its inputs. 71.19 % ImageNet-1k top-1 in 8.9 MB — 39× smaller than the dense FP32 model.

At a glance

	FP32 dense	1-bit PNN 50 %
Model file	346.3 MB	8.9 MB (39× smaller)
ImageNet-1k top-1	84.51 %	71.19 % after 10 epochs, still rising
Weight bits per connection	32	1, plus FP16 scales: 0.53 bits per weight
Arithmetic per output	multiply-adds over all inputs	additions and subtractions over half the inputs
CPU, one image, 4 cores (measured)	105.5 ms (INT8)	52.7 ms (2.0× faster)
Processing element, images/s per mm²	1× (INT8)	11.7×
Energy per image (2 nm, 1 GHz, modelled)	2.43 mJ (INT8)	0.08 mJ (30× less)

What PNN is

PNN (Prime Neural Network) makes a layer small by keeping it full rank but sparse: every output reads a fixed, structured set of inputs, the same number for every output. The connectivity is defined by construction, so no index is stored — a PNN layer costs only its weights and its scales. The construction is confidential and disclosed to licensees under NDA.

One 1-bit PNN layer

A 1-bit PNN layer computes y = h ⊙ (W±1 · (g ⊙ x)) + b: the inputs are scaled once, every output adds or subtracts the inputs it is connected to, and the sum is scaled once. The layer stores only its signs and its scales.

	Inputs read per output (768 / 3,072 inputs)	Bits per weight	Operations per token, 48 layers
1-bit PNN 50 %	384 / 1,536	0.528	42.5 M additions or subtractions
dense FP32, for reference	768 / 3,072	32	84.9 M multiply-adds

Size and accuracy

8.9 MB for the whole model, of which the 48 transformer layers take 5.6 MB; the patch embedding, layer norms and classification head stay FP16.

The model starts from the published FP32 PNN 50 % model (PrimeNeural/pnn-vit-b16-v41-d50), whose weights are replaced by their signs, and is trained for 10 epochs with distillation from the dense model (28 minutes per epoch on one NVIDIA H100).

Epoch	1	2	3	4	5	6	7	8	9	10
Top-1 (%)	41.39	57.21	59.83	63.04	64.04	65.30	66.38	67.93	69.39	71.19
Signs changed	2.6 %	2.9 %	2.5 %	2.3 %	2.1 %	1.9 %	1.6 %	1.3 %	0.9 %	0.5 %

Accuracy was still gaining 1.8 points in the last epoch while the signs had settled: a longer schedule is the direct lever.

Speed

On a CPU (Intel Core i3-8100, AVX2, measured), one image of 197 tokens through the 48 block layers:

Per image, 48 layers	1 thread	vs dense	4 threads	vs dense
INT8 dense	331–343 ms	1.0×	105.5 ms	1.0×
INT8 PNN 50 %	252.9 ms	1.4×	87.2 ms	1.2×
1-bit PNN 50 %	193.1 ms	1.8×	52.7 ms	2.0×

The 1-bit kernel uses PNN's SLNG — single load, no gather: one ordinary vector load where other sparse kernels need a gather, no index lists, and the same int8 dot-product instruction dense layers use. Every output was checked bit for bit against a reference.

On silicon, the 1-bit PNN processing element, taken through place and route in sky130 in the same flow and at the same 25 ns clock as the 8-bit element, takes 0.17× the area and 0.07× the power.

Against NVIDIA Jetson

ViT-B/16, batch 1	Jetson Orin Nano	Jetson AGX Orin	1-bit PNN 50 % chip, 16 nm (modelled)
Speed	59 images/s	43 images/s	590 images/s (10×)
Energy per image	≤ 425 mJ	≤ 940 mJ	1.0 mJ
Power	25 W mode	15–40 W	0.6 W
Size	70 × 45 mm module	100 × 87 mm module	12 mm² die
ImageNet-1k top-1	84.5 % (dense)	84.5 % (dense)	71.19 %

Jetson: published latencies (Orin Nano 17.0 ms, arXiv 2512.09792; AGX Orin 23.5 ms, arXiv 2512.20120); energy is the power mode × latency, an upper bound. The PNN chip is a model built from the measured area and power ratios of the 1-bit element, not silicon (node data ±40 %).

Files

File	Content
model.safetensors	8.9 MB. Per PNN layer: weight_packed (uint8, signs, 8 per byte), weight_scale_in (FP16, per input), weight_scale_out (FP16, per output), bias (FP16). All other parameters dense FP16.
config.json	Architecture, quantization_config (quant_method: "pnn", format: "binary-packed"), and the PNN layer list with in/out widths and inputs per output.

Sign encoding: bit 1 = +1, bit 0 = −1, little-endian within each byte; a latent value of exactly zero is +1, as in training. No index list, mask or connectivity rule is stored: which inputs each output reads comes from the PNN construction in the licensed package. Every stored sign equals the trained model's; the scales are stored in FP16, which leaves 2 of the 147,456 scales (both below 3 × 10⁻⁶) at zero.

Patent and licensing

The PNN connectivity structure and the 1-bit (binary) PNN layer are the subject of a registered patent in France and Europe and US Patent Application No. 19/685,168 (filed 22 May 2026, patent pending).

The weights are published for evaluation. A licence covers the right to build and sell products using PNN, with the PNN encoder, the CPU kernels including the 1-bit kernel, and the PNN processing elements at 8- and 1-bit weights. Production licences are per design, with a royalty per unit.

Contact: PreciseStatistics GmbH, Karlsruhe, Germany · info@primeneural.net · https://primeneural.net