LP-6442: LuxDA Retrieval Markets

Abstract

This LP defines the retrieval market for LuxDA, enabling economic incentives for long-term storage and bandwidth provision. Store providers earn fees for serving data beyond the guaranteed retention period.

Motivation

Beyond protocol-guaranteed retention, users need:

Extended Storage: Keep data for years, not days
Bandwidth Incentives: Reward providers for serving data
Quality of Service: Premium retrieval performance
Geographic Distribution: Data closer to users

Specification

1. Market Structure

1.1 Roles

Role	Description	Requirements
Client	Pays for storage/retrieval	LUX tokens
Provider	Stores and serves data	Storage, bandwidth, stake
Indexer	Tracks provider inventory	Query service

1.2 Service Types

type ServiceType uint8

const (
    ServiceStorage    ServiceType = 1  // Store data for duration
    ServiceRetrieval  ServiceType = 2  // Serve data on demand
    ServicePinning    ServiceType = 3  // Keep data available
    ServiceReplication ServiceType = 4 // Replicate to locations
)

2. Storage Deals

2.1 Deal Structure

type StorageDeal struct {
    // Identification
    DealID       [32]byte
    Client       Identity
    Provider     Identity

    // Content
    CID          *CID
    Size         uint64
    Replicas     uint8

    // Terms
    StartTime    uint64
    Duration     uint64
    PricePerByte uint64  // Per byte per epoch
    TotalPrice   *big.Int

    // Payment
    PaymentMethod PaymentMethod
    Collateral    *big.Int

    // State
    Status       DealStatus
}

type DealStatus uint8
const (
    DealProposed DealStatus = 0
    DealAccepted DealStatus = 1
    DealActive   DealStatus = 2
    DealExpired  DealStatus = 3
    DealFailed   DealStatus = 4
)

2.2 Deal Lifecycle

Client Proposes Deal
         ↓
Provider Evaluates
         ↓
Provider Accepts / Rejects
         ↓
Client Funds Escrow
         ↓
Deal Active
         ↓
Provider Proves Storage (periodic)
         ↓
Deal Expires → Funds Released
    OR
Provider Fails Proof → Slashed

2.3 Deal Proposal

type DealProposal struct {
    CID           *CID
    Size          uint64
    Duration      uint64
    MaxPrice      uint64
    MinReplicas   uint8
    Locations     []string  // Preferred regions
    StartDeadline uint64    // Must start by this time
    ClientSig     []byte
}

func (c *Client) ProposeDeal(proposal *DealProposal) (*DealID, error) {
    // Find matching providers
    providers := c.Indexer.FindProviders(proposal)

    // Request quotes
    quotes := make([]*Quote, 0)
    for _, provider := range providers {
        quote, err := provider.RequestQuote(proposal)
        if err == nil && quote.Price <= proposal.MaxPrice {
            quotes = append(quotes, quote)
        }
    }

    // Select best quotes
    selected := selectBestQuotes(quotes, proposal.MinReplicas)

    // Create deals
    var dealID [32]byte
    for _, quote := range selected {
        deal := createDeal(proposal, quote)
        dealID = deal.DealID
        c.submitDeal(deal)
    }

    return &dealID, nil
}

3. Retrieval Protocol

3.1 Retrieval Request

type RetrievalRequest struct {
    CID         *CID
    Offset      uint64  // For range requests
    Length      uint64
    MaxPrice    uint64  // Max price per byte
    ClientID    Identity
    Nonce       [32]byte
    Signature   []byte
}

type RetrievalResponse struct {
    RequestID   [32]byte
    Status      RetrievalStatus
    Price       uint64
    Data        []byte
    Proof       []byte  // Payment proof or merkle proof
}

3.2 Payment Channels

For efficient micropayments:

type PaymentChannel struct {
    ChannelID   [32]byte
    Client      Identity
    Provider    Identity
    Capacity    *big.Int
    Spent       *big.Int
    Expiry      uint64
}

type PaymentVoucher struct {
    ChannelID   [32]byte
    Amount      *big.Int  // Cumulative amount
    Signature   []byte
}

func (c *PaymentChannel) CreateVoucher(amount *big.Int) *PaymentVoucher {
    newSpent := new(big.Int).Add(c.Spent, amount)
    return &PaymentVoucher{
        ChannelID: c.ChannelID,
        Amount:    newSpent,
        Signature: c.Client.Sign(c.ChannelID, newSpent),
    }
}

3.3 Retrieval Flow

def retrieve_data(cid, client):
    # 1. Find providers
    providers = indexer.find_providers(cid)

    # 2. Get quotes
    best_quote = min(
        (p.get_quote(cid) for p in providers),
        key=lambda q: q.price
    )

    # 3. Open or reuse payment channel
    channel = client.get_channel(best_quote.provider)
    if not channel or channel.capacity < best_quote.total_price:
        channel = client.open_channel(best_quote.provider, deposit)

    # 4. Stream retrieval with incremental payment
    data = []
    for chunk in best_quote.provider.stream_data(cid):
        # Verify chunk
        if not verify_chunk(chunk, cid):
            raise InvalidChunk()

        # Pay for chunk
        voucher = channel.create_voucher(chunk.size * best_quote.price_per_byte)
        best_quote.provider.receive_voucher(voucher)

        data.append(chunk.data)

    return b''.join(data)

4. Pricing

4.1 Price Discovery

type PriceOracle struct {
    // Historical prices
    StoragePrices   []PricePoint
    RetrievalPrices []PricePoint

    // Current market rates
    CurrentStorage   uint64  // Per GB per month
    CurrentRetrieval uint64  // Per GB served
}

func (po *PriceOracle) GetStoragePrice(size uint64, duration uint64) *big.Int {
    // Base price from market
    basePrice := po.CurrentStorage * size / (1024 * 1024 * 1024)

    // Duration multiplier
    months := duration / (30 * 24 * 3600)
    totalPrice := basePrice * months

    return big.NewInt(int64(totalPrice))
}

4.2 Provider Pricing

type ProviderPricing struct {
    // Storage pricing
    StoragePerGBMonth uint64

    // Retrieval pricing
    RetrievalPerGB    uint64
    BandwidthTiers    []BandwidthTier

    // Minimum deal size
    MinDealSize uint64
    MinDuration uint64
}

type BandwidthTier struct {
    MaxBandwidth uint64  // Bytes per second
    PriceMultiplier float64
}

5. Proofs

5.1 Storage Proofs

Providers must prove they still have data:

type StorageProof struct {
    DealID      [32]byte
    Epoch       uint64
    Challenge   [32]byte  // Random challenge from chain
    Response    []byte    // Proof of data possession
    Signature   []byte
}

func GenerateStorageProof(deal *StorageDeal, challenge [32]byte) (*StorageProof, error) {
    // Select random chunks based on challenge
    chunkIndices := deriveChunkIndices(challenge, deal.CID)

    // Generate merkle proof for chunks
    proofData := make([]byte, 0)
    for _, idx := range chunkIndices {
        chunk := loadChunk(deal.CID, idx)
        proofData = append(proofData, chunk...)
    }

    return &StorageProof{
        DealID:    deal.DealID,
        Epoch:     currentEpoch(),
        Challenge: challenge,
        Response:  sha3.Sum256(proofData),
    }, nil
}

5.2 Delivery Proofs

type DeliveryProof struct {
    RequestID     [32]byte
    CID           *CID
    BytesDelivered uint64
    ClientSig     []byte  // Client confirms receipt
    ProviderSig   []byte
}

6. Slashing

6.1 Slashing Conditions

Condition	Penalty
Failed storage proof	5% collateral
Repeated proof failures	100% collateral + deal termination
Data unavailable	Proportional to remaining deal
Invalid data served	10% collateral

6.2 Dispute Resolution

type Dispute struct {
    DisputeID   [32]byte
    DealID      [32]byte
    Claimant    Identity
    Type        DisputeType
    Evidence    []byte
    Status      DisputeStatus
}

func ResolveDispute(dispute *Dispute) Resolution {
    switch dispute.Type {
    case DisputeDataUnavailable:
        // Request data from provider
        data, err := provider.RequestData(dispute.DealID)
        if err != nil {
            return SlashProvider(dispute.DealID)
        }
        // Verify data matches deal
        if !verifyData(data, dispute.DealID) {
            return SlashProvider(dispute.DealID)
        }
        return DismissDispute()

    case DisputeInvalidData:
        // Verify evidence
        if verifyInvalidDataEvidence(dispute.Evidence) {
            return SlashProvider(dispute.DealID)
        }
        return DismissDispute()
    }
}

7. Provider Registration

7.1 Registration

type ProviderRegistration struct {
    ProviderID  Identity
    Endpoints   []string
    Capacity    ProviderCapacity
    Stake       *big.Int
    Pricing     ProviderPricing
    Regions     []string
    Signature   []byte
}

type ProviderCapacity struct {
    StorageBytes     uint64
    BandwidthBps     uint64
    MaxDeals         uint32
    MinDealSize      uint64
}

func (r *Registry) RegisterProvider(reg *ProviderRegistration) error {
    // Verify stake
    if reg.Stake.Cmp(MinProviderStake) < 0 {
        return ErrInsufficientStake
    }

    // Lock stake
    if err := r.StakeContract.Lock(reg.ProviderID, reg.Stake); err != nil {
        return err
    }

    // Register
    r.Providers[reg.ProviderID] = reg
    return nil
}

7.2 Reputation

type ProviderReputation struct {
    TotalDeals       uint64
    SuccessfulDeals  uint64
    FailedDeals      uint64
    TotalBytesServed uint64
    AverageLatency   float64
    UptimeRatio      float64
}

func (r *Registry) GetReputation(providerID Identity) *ProviderReputation {
    // Calculate from on-chain history
    return calculateReputation(providerID)
}

8. Indexer Service

8.1 Provider Index

type Indexer struct {
    // CID -> Providers mapping
    ContentIndex map[*CID][]Identity

    // Provider -> Capabilities
    ProviderIndex map[Identity]*ProviderRegistration
}

func (i *Indexer) FindProviders(cid *CID) []Identity {
    return i.ContentIndex[cid]
}

func (i *Indexer) AnnounceContent(provider Identity, cids []*CID) error {
    for _, cid := range cids {
        i.ContentIndex[cid] = append(i.ContentIndex[cid], provider)
    }
    return nil
}

9. Metrics

Metric	Description
`market_deals_active`	Active storage deals
`market_deals_value`	Total value locked
`market_retrieval_volume`	Bytes retrieved
`market_provider_count`	Registered providers
`market_average_price`	Average price per GB

Rationale

Why Payment Channels?

Micropayments impractical on-chain
Low latency for streaming retrieval
Reduce transaction costs

Why Storage Proofs?

Verify providers actually store data
Enable automated dispute resolution
Create accountability

Why Provider Reputation?

Help clients choose reliable providers
Incentivize good behavior
Market-based quality assurance

Backwards Compatibility

This specification introduces new functionality and does not modify existing protocols. It is fully backwards compatible with existing implementations.

Security Considerations

Collusion

Providers and clients could collude:

Mitigated by random challenges
Slashing makes collusion costly

Sybil Attacks

Fake providers to manipulate market:

Stake requirement raises cost
Reputation tracks real performance

Data Ransom

Provider refuses to serve unless paid more:

Deal terms are binding
Slashing for non-delivery
Replication provides alternatives

Test Plan

Unit Tests

Deal Lifecycle: Propose → Accept → Complete
Payment Channels: Open, use, close
Storage Proofs: Generate and verify

Integration Tests

Full Retrieval: End-to-end payment + data
Dispute Resolution: Challenge and response
Market Dynamics: Multiple providers competing

Economic Tests

Price Discovery: Market equilibrium
Incentive Alignment: Provider behavior
Attack Resistance: Sybil, collusion

References

LP-6442 v1.0.0 - 2026-01-02

LuxDA Retrieval Markets