# Quantaria

> Quantaria is a research company building multimodal generative models whose weights are tensor networks (MPS/MPO) from the first training step. One shared backbone generates both text and images, and because the weights use the same format physicists use for quantum states, individual layers have a direct path from GPU to quantum hardware (QPU).

Status: Hilbert-1 Nano (~100M parameters) and Hilbert-1 Base (~1B parameters) are live and usable on the Quantaria console at https://console.quantaria.xyz. Hilbert-1 Large is planned. All figures under "Research foundations" come from published papers on the methods Quantaria combines, not from Quantaria's own models. Quantaria models train and run on GPUs today; they do not run on quantum computers.

Website: https://quantaria.xyz
Console: https://console.quantaria.xyz
Docs: https://docs.quantaria.xyz
Contact: hello@quantaria.xyz

## Approach

- Tensor-network weights from step zero: every layer is a matrix product state/operator (MPS/MPO), trained natively and never compressed from a dense model afterwards.
- Unified multimodal objective (Transfusion-style): next-token loss on text and diffusion loss on image patches over one set of shared tensor cores. Basis: arXiv:2408.11039
- Tensor Product Attention: queries, keys and values factorized from contextual activations; RoPE-compatible, about 10x smaller KV cache. Basis: arXiv:2501.06425
- MPO mixture-of-experts: experts share one central tensor and keep small expert-specific auxiliary tensors. Basis: arXiv:2203.01104
- Rank-adaptive training: bond dimensions (χ) shrink or grow during training. Basis: CoMERA, arXiv:2405.14377
- Schmidt-rank truncation: each bond keeps its dominant Schmidt coefficients, the operation that bounds entanglement in a quantum state.
- Canonical-form gauging: cores kept as left/right isometries for numerically stable contraction.

## Models

- Quantaria Hilbert (live): flagship unified text and image model.
  - Hilbert-1 Nano (live on the console): ~100M parameters, 2K-token text context, 32×32 image output.
  - Hilbert-1 Base (live on the console): ~1B parameters, 8K-token context, 256×256 images, adds MPO mixture-of-experts.
  - Hilbert-1 Large (planned): scale set after Base results.
- Quantaria Schmidt (planned): compact text-only model for long context on laptops and phones, using low-χ cores and Tensor Product Attention.
- Quantaria Dirac (research): hybrid quantum-classical layers (quantum circuit + MPO + quantum circuit) with Cayley-parameterized unitary adapters; circuit simulators first, then cloud QPUs.
- Quantaria Born (research): Born-machine generative heads that sample images from an MPS with probability |ψ(x)|².
- Quantaria Bond (planned): open-source PyTorch library behind every model (tensor-train and MPO layers, rank-adaptive training, canonical gauging, fused contraction kernels).

## Quantum path

- A bond of dimension χ carries at most log₂χ bits of entanglement entropy across its cut.
- An MPS with bond dimension 2^k can be prepared by a staircase circuit whose gates act on k+1 qubits (Schön et al., arXiv:quant-ph/0501096).
- MERA-style disentanglers can move correlations from an MPO into parameterized quantum circuits; in published work this cut one SmolLM2 attention layer to ~3,000x fewer classical parameters at +0.26% perplexity, with part of inference on an IBM Heron processor (arXiv:2410.17397).
- Quantum layers ship in production models only where real hardware supports them.

## Research foundations (published results, not Quantaria benchmarks)

- 200,000x compression of a single dense layer with tensor-train factorization (arXiv:1509.06569)
- 14,000x fewer parameters in MERA layers at under 1% accuracy loss, on image classifiers (arXiv:1711.03357)
- ~3,000x fewer classical parameters in a hybrid quantum attention layer at +0.26% perplexity (arXiv:2410.17397)
- 93% memory reduction on LLaMA-7B at a 2–3% accuracy cost (arXiv:2401.14109)
- 27x fewer parameters than Switch Transformer with MPO mixture-of-experts (arXiv:2203.01104)
- ~10x smaller KV cache with Tensor Product Attention (arXiv:2501.06425)
- 9x more memory-efficient training than GaLore, with 2–3x faster epochs (arXiv:2405.14377)
- ~40% fewer parameters at near-baseline perplexity for GPT-2 trained in tensor-train form (arXiv:2306.02697)

## Roadmap

- Live: Hilbert-1 Nano and Base on the Quantaria console; benchmarks against same-size dense transformers (loss curves, perplexity, FID, memory) to follow.
- Next: open-source Bond; train Schmidt; scale Hilbert to Large.
- Then: Dirac hybrid layers on simulators, then cloud quantum hardware.
- Research: area-law rank allocation from measured entanglement entropy, DMRG-style sweeps alongside gradient descent, Born-machine heads.