← All proposals

Status

luxfi/p2p is archived. The peer-to-peer transport for the new final

Lux network is QUIC + ZAP + mDNS + Kademlia DHT. One transport, one byte

stream, four layers. There is no other transport. There is no fallback,

no shim, no env flag, no backwards-compat wrapper, no AppRequest

envelope, no Handler/Sender interface.

Activated at the genesis of the new final Lux network:

2025-12-25 16:20 Pacific (unix 1766708400).

The pre-Quasar Edition Lux network (2020–2025) ran on a TCP-framed

luxfi/p2p built on top of luxd's network/peer package. That is a

separate network and is out of scope.

Abstract

luxfi/p2p braided three concerns into one Go package: a transport

(TCP framing on top of luxd's network/peer), an envelope format

(AppRequest/AppResponse/AppGossip), and request/response

correlation by uint32 request IDs that collide under high concurrency.

On top of that braid, the application VMs (platformvm, xvm, example

xsvm) each invented their own Handler and Sender shapes and gossip

semantics. The result was a tower of indirection — five layers of "send

bytes to a peer" — with no content addressing, no stream multiplexing,

and no way to fetch a chunk by hash.

This LP collapses the tower. The transport is QUIC. QUIC carries native

TLS 1.3, multiplexes streams, correlates request/response by stream ID,

and migrates connections across IP changes. The frame format on every

QUIC stream is ZAP — the same zero-copy wire used everywhere else in the

Lux stack (LP-022, LP-182, LP-184, LP-185, LP-186). LAN peer discovery

is mDNS. Content-addressable storage is Kademlia DHT. Application

semantics — consensus votes, mempool txs, state snapshots — are typed

ZAP streams over QUIC, with stream-type byte routing.

Handler and Sender are gone. AppRequest is gone. The uint32

request ID is gone. The luxfi/p2p Go module is moved to

~/work/_archive/p2p-deleted-YYYYMMDD-HHMMSS after the 11 luxd

consumer files cascade.

The four layers are independent and orthogonal:

• Layer A — QUIC + ZAP byte transport
• Layer B — mDNS LAN discovery
• Layer C — Kademlia DHT content addressing
• Layer D — typed application streams (consensus, mempool, state, chunk)

Each layer is independently complete. No layer reaches across to braid

itself into another layer's concerns.

Above Layer A, one further contract lives at this LP: the

Transport.Pick(peer) selector. The selector is what every consumer

calls to obtain a Transport for a given peer. The set of registered

implementations is open: QUIC ships in this LP as the default; LP-206

registers cxl-pool for same-rack peers; LP-207 registers rdma-ib

and rdma-rocev2 for cross-rack peers; future transports register

via Transport.Register(name, factory) without amending LP-201. The

selector is one function in one place — it does not appear in any

implementation LP.

Architecture

Layer A — QUIC + ZAP transport

Every peer-to-peer message in the Lux network is a ZAP buffer on a QUIC

stream. The QUIC connection is the peer session. The QUIC stream is the

message lifetime.

Stream model

There is no long-lived request/response stream. A bidirectional stream

carries exactly one request and exactly one response, then closes. QUIC

handles correlation: the response is on the same stream as the request.

There is no uint32 request ID. There is no correlation table.

Frame format

Every stream payload is a ZAP buffer. The first byte of every stream is

a stream-type byte (0xD0..0xDF, allocated in Wire Schemas below). The

remainder is the ZAP buffer for that type, with offsets defined by the

LP-022 schema. There is no length prefix — QUIC streams are

self-delimiting, and the ZAP buffer's first field is its total length.

TLS 1.3 + PQ-KEM

QUIC carries TLS 1.3 natively. Client certificates are mandatory; the

certificate subject's public key is the node's identity key

(secp256k1 + ML-DSA-65 hybrid, per LP-173). Plaintext is rejected.

Under the strict-PQ profile, the TLS 1.3 key exchange uses the LP-175

SessionKEM hybrid (X25519 + ML-KEM-768). Outside strict-PQ, classical

X25519 is permitted. The profile gate is the single function

contract.RefuseUnderStrictPQ — no second policy point in this layer.

Connection migration

QUIC connections survive IP changes via connection IDs. A validator that

NAT-rebinds, restarts in a new container, or moves between networks

keeps the same logical session. The TCP-framed luxfi/p2p dropped on

every IP change and re-handshaked from zero.

No head-of-line blocking

QUIC multiplexes streams; a slow ConsensusCert push does not block a

fast TxGossip on the same connection. The TCP-framed luxfi/p2p had

one byte stream per peer, so a 10 MiB block snapshot blocked all

mempool gossip behind it.

Implementation

~/work/lux/zap/quic/ already exists: client.go, server.go,

conn.go, factory.go, tls.go, config.go. LP-201 promotes this

package from "optional alongside TCP" to "default and only". The TCP

transport in ~/work/lux/zap/transport/transport.go is retained only

for in-process plugin IPC (rpcchainvm Unix-socket transport) and is not

exposed to peer-to-peer.

The high-performance shared-memory paths in

~/work/lux/zap/transport/ (default.go, dpdk{,_linux,_other}.go,

gpudirect{,_linux,_other}.go, uma{,_darwin,_linux,_other}.go) are

unchanged and continue to serve in-host IPC. They are not P2P

transports — they are buffer-placement and ingress-acceleration

mechanisms that compose under whichever wire transport Transport.Pick

returns (next section).

Transport.Pick() contract

Transport.Pick(peer PeerID) Transport is the single function every

consumer (consensus, mempool, state-sync, DHT) calls to obtain a

Transport for a given peer. It is defined here, in LP-201, and

implementation-only LPs (LP-206 CXL pool, LP-207 RDMA-IB, future

transports) MUST NOT re-specify it. They register an implementation

and otherwise stay in their lane.

Selector function

// Pick returns the highest-capability transport that both self and
// peer support. The set of registered transports is open; registration
// happens at init() time via Transport.Register. Pick never returns
// nil — TCP is the always-available fall-through.
//
// The selection decision is opaque to layers above. A ConsensusVote
// to a same-DC peer uses RDMA-IB; the same ConsensusVote to a WAN
// peer uses QUIC; the consensus engine cannot tell.
Transport.Pick(peer PeerID) Transport

Registration API

// Register makes a transport implementation available to Pick. Called
// from each transport's init() — exactly one Register call per
// transport. Re-registration under the same name panics; this is a
// programming error, not a runtime condition.
//
// name is the low-cardinality identifier ("tcp", "quic", "cxl-pool",
// "rdma-ib", "rdma-rocev2"). factory constructs a Transport for a
// specific peer.
//
// DPDK is intentionally NOT a registered name. DPDK is a kernel-bypass
// NIC-ingestion mechanism that composes UNDER the QUIC implementation
// (it changes how UDP packets reach this process); it does not pick
// peers. Same orthogonality applies to UMA, GPUDirect-RDMA, and other
// buffer-placement / ingress-acceleration capabilities — those live
// on the existing Capabilities{GPUResident, ZeroCopy, ...} struct.
Transport.Register(name string, factory TransportFactory)

type TransportFactory func(self, peer PeerID, cfg Config) (Transport, error)

Capability probe

// TransportProbe asks a peer what transports it supports. Probe runs
// once at peering and the result is cached for the lifetime of the
// session; topology changes (e.g. a NIC link drop) invalidate the
// cached entry and trigger a re-probe on next Pick.
//
// The probe payload is a ZAP frame on a QUIC stream (stream-type byte
// 0xDC PeerInfo, extended with a capability bitmap field). QUIC is
// always available, so probe always reaches.
TransportProbe(peer PeerID) []TransportCapability

type TransportCapability uint8

const (
    CapTCP          TransportCapability = 1
    CapQUIC         TransportCapability = 2
    CapCXLPool      TransportCapability = 3
    CapRDMAIB       TransportCapability = 4
    CapRDMARoCEv2   TransportCapability = 5
    // Future transports allocate the next free value via LP amendment.
)

The capability enum is wire-transport-only — it answers "what

delivery medium does this peer expose." Buffer-placement and

ingress-acceleration capabilities (UMA-direct, GPUDirect-RDMA,

DPDK kernel-bypass) are orthogonal and live on the existing

Capabilities{GPUResident, ZeroCopy, MinLatencyMicros} struct in

~/work/lux/zap/transport/transport.go. A peer may report

CapQUIC AND a GPUDirect-RDMA NIC-to-GPU placement — these

compose; they do not select.

Selection priority

Pick chooses the highest-capability transport that BOTH peers

support. The priority is fixed (lower index wins):

Pick walks the priority list and returns the first capability that

appears in BOTH self.caps and peer.caps. The fall-through to TCP

is unconditional — there is no failure mode in which Pick returns

nil. If a higher-priority transport's factory returns an error at

construction (e.g. NIC link down, IB QP exhausted), Pick records

the error in the per-peer cache and retries the next-lower

capability; the cache is invalidated on a transport-layer

unreachable signal so the failure is transient.

Registration is open-ended

Adding a new transport is one new LP that says "implements LP-201

contract." It defines a new CapXxx constant, a TransportFactory,

and an init() that calls Transport.Register. It does not

re-specify the selector, does not amend the priority list (priority

goes at the bottom unless the new LP argues for a different slot),

and does not modify any consumer call site. The consumer call site

is one place: Transport.Pick(peer).

On-disk implementation

~/work/lux/zap/transport/transport.go::Pick(preferred string) is

the in-host buffer-placement selector that auto-picks across

gpudirect > dpdk > uma > default. That selector follows the same

pattern this section formalises (open registration, capability

priority, structured boot log via logPickOnce), but operates on

the buffer-placement dimension rather than the wire-transport

dimension. The wire-transport Pick(peer) defined in this section

will live alongside it as a sibling function once the per-peer

selector lands — same package, same registration convention. The

two selectors are orthogonal: one picks where bytes land in this

process; the other picks how bytes reach this process.

Layer B — mDNS LAN discovery

LAN discovery uses standard mDNS / DNS-SD. WAN bootstrap continues to

use the genesis-pinned bootstrappers list. mDNS supplements; it does

not replace.

Service record

_lux._udp.local.    PTR   <nodeID>._lux._udp.local.
<nodeID>._lux._udp.local.   SRV   0 0 9651 <nodeID>.local.
<nodeID>._lux._udp.local.   TXT   chain=<chainSet> ver=<luxdVersion> pk=<base64 pubkey> sig=<base64 sig>

The sig field is an ML-DSA-65 signature over `(nodeID, chain, ver,

pk) by the node's identity key. Peers verify sig against pk` and

verify pk corresponds to nodeID (NodeID = sha256(pk)[:20]) before

establishing a QUIC session. mDNS announcements without a valid

signature are dropped at the discovery layer — they never reach Layer A.

Activation matrix

WAN bootstrap is always active and is the source of truth for any

network that crosses an L3 boundary.

Implementation

~/work/lux/network/discovery/mdns/ (new package). Uses the standard

mDNS responder pattern: one goroutine listening on 224.0.0.251:5353

(IPv4) and [ff02::fb]:5353 (IPv6), one goroutine sending periodic

announcements at jittered 60s intervals.

Layer C — Kademlia DHT content storage

Content-addressable storage. 256-bit address space. The address of a

chunk is sha256(content). Lookups are O(log N) hops.

Why DHT, not broadcast

The legacy luxfi/p2p broadcasted 10 MiB block snapshots to N peers,

producing 10·N MiB of bandwidth per snapshot. The DHT advertises a

content hash; peers pull by hash from the closest k=20 peers, producing

~10·k MiB across the network independent of N. For a network of 1000

validators with snapshots every 30s, this is the difference between

333 GiB/s and 6.6 GiB/s aggregate.

Parameters (standard Kademlia)

Content types and TTL

Garbage collection is LRU on bucket overflow. Pinned content is exempt

from LRU eviction; the node trades disk for retention. Disk cap per

content type is configurable (default 100 GiB BlockSnapshot, 10 GiB

StateSnapshot, 1 GiB ChunkPayload, 100 MiB LightClientProof).

Bootstrap

DHT bootstrap nodes inherit from the network's bootstrappers list.

First connection sequence: WAN bootstrap → QUIC handshake → DHT

FindNode(self) → bucket population. A node is DHT-ready when at

least one bucket has ≥ 8 entries.

Eclipse resistance

A node refuses to populate a bucket with more than 4 peers from the

same /24 IPv4 prefix or /48 IPv6 prefix. An attacker that controls one

/24 cannot eclipse a victim's bucket. Sybil resistance at the

validator level is provided by stake (LP-170 finality + LP-171 DKG);

non-validators are rate-limited at the QUIC stream layer (Layer A).

Implementation

~/work/lux/network/dht/ (new package). Reference: BitTorrent BEP 5,

IPFS Kademlia. Wire schemas 0xDD (DHTFindNode), 0xDE (DHTFindValue),

0xDF (DHTStore), all carried on bidirectional QUIC streams.

Layer D — Application streams

Handler and Sender are gone. Each application VM gets typed ZAP

streams. The stream-type byte (the first byte of every QUIC stream)

routes the message to the right handler.

Each application VM (platformvm, xvm, EVM, future xsvm) opens

streams of the types it owns and ignores stream types it does not

understand. There is no central router; the stream-type byte is its

own routing key.

Concurrency model

Each peer connection has a single QUIC connection. The application VM

opens a new QUIC stream for each message. Stream creation is cheap

(~zero round-trips after the QUIC handshake) and is the natural

correlator. There is no shared request table, no uint32 ID, no

in-flight map.

Wire schemas

Stream-type bytes 0xD0..0xDF are reserved by LP-201. Schema bodies are

LP-022 ZAP buffers; field offsets are defined here.

Schema IDs outside 0xD0..0xDF are not part of LP-201. Schema IDs

0xC0..0xCF are reserved for future use by adjacent transport LPs.

Migration ledger

Per-file migration of the 11 luxd production .go files that import

github.com/luxfi/p2p. Verified by `grep -rln 'github.com/luxfi/p2p'

~/work/lux/node --include='*.go'`:

After the cascade:

grep -rln 'github.com/luxfi/p2p' ~/work/lux/node --include='*.go'  # → empty
go mod edit -droprequire=github.com/luxfi/p2p
go mod tidy
mv ~/work/lux/p2p ~/work/_archive/p2p-deleted-$(date +%Y%m%d-%H%M%S)

The two test files that import luxfi/p2p

(network/network_test.go, network/example_test.go) are deleted as

part of the cascade; the new transport's tests live in

~/work/lux/network/dht/, ~/work/lux/network/discovery/mdns/, and

~/work/lux/zap/quic/.

Security

Authenticated peering

TLS 1.3 with mandatory client certificates. Certificate public key is

the node's identity key. NodeID = sha256(pubkey)[:20]. A peer that

cannot prove possession of the private key for its claimed NodeID

cannot establish a QUIC session. There is no plaintext fallback.

Post-quantum migration

Under the strict-PQ profile (LP-175 SessionKEM), the TLS 1.3 KEX uses

X25519 + ML-KEM-768 hybrid. Under the default profile, X25519. The

profile gate is one function (contract.RefuseUnderStrictPQ) called

once at session establishment — verification logic and policy

enforcement stay in their lanes.

mDNS spoof resistance

mDNS TXT records carry an ML-DSA-65 signature over `(nodeID, chain,

ver, pk)`. A peer that announces a NodeID it cannot prove possession of

is rejected at the discovery layer before any QUIC handshake. mDNS is

multicast, so an attacker on the same L2 segment can hear announcements

but cannot forge them.

DHT eclipse resistance

Kademlia buckets enforce ≤ 4 peers per /24 IPv4 prefix and ≤ 4 per /48

IPv6 prefix. An attacker that controls one /24 cannot eclipse a

victim's view of the address space. Bucket diversity is checked on

every insert; over-quota inserts are rejected, not silently logged.

Sybil resistance

At the validator layer, Sybil resistance is by stake (LP-170 + LP-171).

Non-validators are rate-limited at the QUIC stream layer:

Violations close the QUIC connection (not just the stream) and place

the peer on a 5-minute backoff.

DoS resistance

QUIC's amplification limit (3× the incoming bytes until the peer is

address-validated) is enforced by the stack. The Lux server-side

default refuses to send more than 1× until the client's address is

validated via QUIC's retry mechanism, eliminating reflection

amplification entirely for new peers.

Performance characteristics

Versus the legacy TCP-framed luxfi/p2p:

The 50× snapshot dissemination delta assumes a network of 1000

validators producing one 10 MiB snapshot every 30s. Aggregate

bandwidth: legacy 333 GiB/s, LP-201 6.6 GiB/s. Numbers from the

Kademlia k=20 bucket size; α=3 lookup concurrency does not change the

steady-state aggregate.

Activation

activates: 2025-12-25T16:20:00-08:00
activates-unix: 1766708400

This LP activates atomically with LP-022, LP-175, LP-182, LP-184,

LP-185, LP-186 at the genesis of the new final Lux network. There is

no soft activation, no flag day, no migration window. The pre-genesis

network used luxfi/p2p. The post-genesis network uses LP-201. They

are different networks.

Implementation scope

LP-201 defines the selector contract and the QUIC + mDNS + DHT

default implementations. The cascade migration of the 11 luxd consumer

files is implementation work that follows once this LP lands:

1. Promote ~/work/lux/zap/quic/ from optional to default; remove the

TCP peer-to-peer transport from the public API.

2. Land the wire-transport Pick(peer) selector alongside the existing

buffer-placement Pick(preferred) in ~/work/lux/zap/transport/.

QUIC registers via Transport.Register("quic", quicFactory) from

~/work/lux/zap/quic/init.go. TCP registers as the fall-through.

3. Create ~/work/lux/network/discovery/mdns/ (new package).

4. Create ~/work/lux/network/dht/ (new package).

5. Per-file migration of the 11 luxd consumers (see Migration ledger).

6. grep -rln 'github.com/luxfi/p2p' ~/work/lux/node --include='*.go'

returns empty.

7. go mod edit -droprequire=github.com/luxfi/p2p && go mod tidy.

8. mv ~/work/lux/p2p ~/work/_archive/p2p-deleted-YYYYMMDD-HHMMSS.

Each step is independently testable and independently mergeable. The

cascade lands across luxfi/zap, luxfi/node, and the new

luxfi/network subpackages.

Cross-references

• LP-022 — ZAP wire protocol (the byte format on every stream)
• LP-175 — SessionKEM (PQ-KEM session keys for TLS 1.3)
• LP-182 — Consensus wire ZAP (the schemas on ConsensusVote /

ConsensusProposal / ConsensusCert streams)

• LP-184 — X-chain wire (the schemas tx streams carry)
• LP-185 — EVM internal wire (the schemas EVM streams carry)
• LP-186 — Chains VM wire (the schemas chain VMs carry)
• LP-206 — CXL Coherent State Pool — registers cxl-pool

(CapCXLPool) against the §"Transport.Pick() contract" defined here

• LP-207 — RDMA over InfiniBand — registers rdma-ib

(CapRDMAIB) and rdma-rocev2 (CapRDMARoCEv2) against the

§"Transport.Pick() contract" defined here