China Internet Network Information Center (CNNIC)

yuhaisheng@cnnic.cn

Internet Network Working Group Agent Registry Synchronization Distributed Systems IPv6 Consistency This document analyzes the data synchronization problems between multiple distributed Agent registry centers in IPv6 networks. When Agent networks span multiple organizational domains, geographic regions, or autonomous systems, each region's Agent registry center needs to synchronize Agent connection information and capability descriptions with others. This document presents a network-layer perspective on the main problems, challenges, and design considerations, providing a foundation for the development of subsequent solutions.

Introduction

Problem Background In IPv6-supported global Agent networks, each organization, region, or autonomous system may maintain an independent Agent Registry Center that records information about agents in that domain (such as connection addresses, available capabilities, and operational status). When these registry centers need to interconnect, they face the following problems:

1. Information Silos: Each registry center's data cannot be mutually accessed
   • Agent A is registered in Beijing registry center, Agent B in Shanghai registry center
   • When A needs to call B's capabilities, B's existence and address cannot be discovered
   • Each cross-domain access requires manual configuration or out-of-band communication
2. Redundant Registration: The same information is registered multiple times in different centers
   • Cross-domain Agents need to be registered in multiple registry centers
   • Information updates require synchronization across multiple locations
   • This easily leads to information inconsistency
3. Real-time Issues: Synchronization delays for Agent status changes
   • When an Agent goes online/offline, other centers cannot learn promptly
   • Cross-domain calls may access unavailable Agents
   • This affects the overall reliability of the system
4. Cross-domain Permission Problems: Access control between different domains
   • How to ensure only authorized Agents can access
   • How to prevent information leakage (e.g., not exposing sensitive capabilities to competitors)
   • How to implement cross-domain access auditing
5. Consistency Challenges: Data consistency in distributed scenarios
   • Information about the same Agent may be inconsistent across multiple centers
   • Network interruptions between registry centers create synchronization dilemmas
   • How to handle malicious modifications or conflicts
6. Network Complexity: Multi-level characteristics of IPv6 networks
   • Differences between border domains, regional domains, and global domains
   • Huge differences in synchronization delays between different levels
   • Impact of NAT, firewalls, QoS, and other network features

Scope and Limitations This document does not define a specific protocol, but rather analyzes the above problems and discusses design considerations.

Explicit Goals

• Explain the nature and difficulties of problems
• Analyze different design trade-offs
• Propose architectural considerations
• Provide foundation for subsequent RFCs or standards

Explicit Non-Goals

• Do NOT design a new DNS system
• Do NOT invent new authentication mechanisms (use existing DIDs, etc.)
• Do NOT define complete protocol formats (discuss framework only)
• Do NOT implement reference code

Environmental Assumptions The document assumes:

• Each registry center operates independently
• IPv6 network connectivity (possibly through multiple hops)
• Peer-to-peer (P2P) or hierarchical architecture
• No central authority
• Trust relationships exist between registry centers but they operate independently

Key Problem Analysis

Problem 1: Synchronization Scope and Granularity Problem Statement: Which information should be synchronized between registry centers? How should granularity be divided?

Types of Information to Synchronize Candidate Options:

Option A: Minimal Set (Registration Only)

Advantages: Simple, low bandwidth. Disadvantages: Limited functionality, requires multiple queries.

Option B: Complete Set (Full Synchronization)

Advantages: Full functionality, fast queries. Disadvantages: Complex, privacy risks, redundant data.

Option C: Classified Synchronization (On-Demand)

Advantages: Flexible, customizable. Disadvantages: Complex, difficult to manage, easy to become inconsistent.

Information Granularity Issues Should Agent capabilities be sent together or separately? When an Agent has multiple capabilities (e.g., a translator with multiple language pairs), should all capabilities be sent to all centers, or only those permitted and needed? Full transmission risks privacy leakage and bandwidth waste. Customized transmission by requester requires tracking permissions for each requester, increasing complexity. Layered transmission (public + authorized layer) requires pre-defined classification schemes.

Problem 2: Synchronization Topology Architecture Problem Statement: How should multiple registry centers interconnect? What topology structure should be adopted?

Topology Options Three main architectural patterns exist:

Option 1: Peer-to-Peer (P2P)

Advantage: Fully decentralized, no single point of failure. Disadvantage: O(n²) connections, network complex, difficult to manage. Suitable for: Fewer than 10 centers.

Option 2: Hierarchical/Star

Advantage: Clear hierarchy, simple management, scalable. Disadvantage: Single point of failure risk, high cross-layer query latency. Suitable for: All scales.

Option 3: Hybrid (Multi-center + Backup Links)

Advantage: High reliability, complete redundancy. Disadvantage: Complex, high cost. Suitable for: Critical applications.

Network Constraints Topology selection must consider:

• Geographic distribution determines natural grouping
• Autonomous System (AS) boundaries affect routing stability
• Latency characteristics vary: local <5ms, national <50ms, intercontinental >100ms
• ISP link failure rates and multi-link redundancy cost-benefit analysis
• Regulatory constraints on cross-border data flow

Problem 3: Consistency Model Problem Statement: How should the system operate when information between registry centers becomes inconsistent?

Consistency Options

Option 1: Strong Consistency

All centers have identical information at any time. Advantages: Good user experience, unambiguous. Disadvantages: System unavailable during network partitions, requires complex 2PC algorithms, long synchronization delays, low throughput, nearly impossible to implement cross-domain.

Option 2: Eventual Consistency

All centers eventually synchronize to the same state but may be temporarily inconsistent. Advantages: High availability, low latency, high throughput, easy to implement and scale. Disadvantages: Temporary data inconsistency, complex conflict resolution.

Option 3: Weak Consistency

Best-effort synchronization, no guarantees. Advantages: Simplest implementation, best performance. Disadvantages: Information may be permanently inconsistent, unpredictable, difficult to debug.

Conflict Resolution Challenges When the same Agent information is modified simultaneously in two centers, determining which version is "correct" becomes non-trivial. Different conflict resolution strategies (Last-Write-Wins, Vector Clocks, CRDTs, Manual Intervention, Abort) have different trade-offs in accuracy, complexity, and cost.

Problem 4: Synchronization Triggering Mechanisms Problem Statement: When should registry centers synchronize information? Periodic, event-driven, on-demand, or hybrid?

Triggering Method Comparison Each method has different latency, bandwidth predictability, and complexity characteristics.

Periodic Synchronization (Heartbeat)

Latency: High (seconds). Bandwidth: Predictable/fixed. Complexity: Low. Use: State information.

Event-Driven

Latency: Low (milliseconds). Bandwidth: Bursty/unpredictable. Complexity: Medium. Use: Change events.

On-Demand Query (Pull)

Latency: Variable. Bandwidth: Sparse/low. Complexity: Medium. Use: Specific queries.

Hybrid (Periodic + Event + On-Demand)

Latency: Low/optimized. Bandwidth: Optimized/balanced. Complexity: High. Use: All scenarios.

Failure Recovery Problem If using periodic heartbeats (e.g., 30-second interval with 3-attempt timeout), detecting that an offline Agent needs up to 90 seconds plus timeout margin. Some applications cannot tolerate 120-second detection delays. However, reducing detection latency increases heartbeat traffic, creating a fundamental trade-off.

Problem 5: Security Considerations Problem Statement: How can independent registry centers trust each other? How to prevent information leakage and tampering?

Authentication Problem Verifying that a registry center is genuinely the "Shanghai Center" is non-trivial. IP-based verification is insufficient due to potential hijacking. Multiple approaches exist (DNS DNSSEC, PKI/Certificates, DID Blockchain, Preconfigured Whitelists) each with different trust models and operational costs.

Privacy Leakage Problem A registry center may not want to expose all Agent capabilities, particularly proprietary or competitive capabilities. Yet full synchronization naturally exposes all capabilities. Selective hiding requires complex access control mechanisms, creating tension between functional completeness and privacy protection.

Access Control Problem Who should be able to access whose registry data? Options range from complete openness (trusting all) to complete privacy (trusting none), with fine-grained ACL-based control in between. The access control matrix grows as O(n²) with the number of centers, making management increasingly difficult.

Problem 6: IPv6 Network-Specific Issues Problem Statement: How do IPv6 network characteristics affect synchronization design?

IPv6-Specific Challenges

• Address Translation and NAT: IPv6 addresses may change (ISP dynamic prefix assignment), and enterprise Agents may lack direct public addresses. Discovery mechanisms must handle address reachability.
• Multi-path and Multi-homing: Agents may have multiple IPv6 addresses. Synchronization must determine whether to send all addresses or just preferred ones, and how clients select which address to use.
• Link-Local Addresses: fe80::/10 addresses are only valid on-link and cannot be used for cross-domain synchronization, yet some scenarios (campus networks) may only have these addresses.
• Packet Size: IPv6 MTU is typically 1280 bytes (considering extension headers), yet capability information often exceeds this, requiring fragmentation or compression.
• Unicast vs Multicast: While IPv6 has better multicast support, cross-domain multicast routing is difficult, reliability is poor (UDP-based), and some ISPs don't support it cross-domain.

Problem 7: Scalability and Performance Problem Statement: How can the system support millions of Agents? Where are the performance bottlenecks?

Scale Analysis With 1 million Agents distributed across 1,000 registry centers, assuming 500-byte messages and 30-second heartbeat intervals, the required bandwidth is ~16.6 MB/second globally. However, hot-spot problems emerge:

• Popular Agents receive 100x concurrent queries, saturating links
• Single center failure redirects all load to backups, potentially causing cascading failure
• Uneven distribution means some centers need 10x average capacity

Consistency vs Performance Trade-off The CAP Theorem states that distributed systems can achieve at most two of: Consistency, Availability, and Partition tolerance. For inter-registry synchronization spanning multiple administrative domains and potential network partitions, prioritizing Availability and Partition tolerance (i.e., Eventual Consistency) is the practical choice over Strong Consistency.

Problem 8: Management and Operations Problem Statement: How to manage multiple independent registry centers? How to control operational costs and complexity?

Monitoring and Diagnostics Operations teams need to answer questions like:

• How many registry centers currently exist?
• What is the synchronization state between centers?
• Why is Agent information inconsistent across centers?
• Why has latency suddenly increased?
• How to diagnose cross-center query failures?

Each question requires non-trivial tooling and infrastructure.

Upgrades and Evolution Managing software version upgrades across independent centers requires:

• Backward compatibility between old and new versions
• Continuous service availability during upgrades
• Rollback mechanisms if new versions have issues
• Version-specific protocol handling

Problem 9: Standards and Interoperability Problem Statement: Can registry center implementations from different vendors interoperate? What standards are needed?

Interoperability Challenges Different vendor implementations may have different understandings of:

• What is an "Agent capability"?
• What does "synchronization" mean?
• What are the consistency guarantees?
• How are conflicts resolved?

Standards are needed to define:

1. Information model (what is an Agent? what information must be synchronized?)
2. Synchronization protocol (message format, interaction patterns)
3. Version management (version negotiation mechanisms)
4. Extension mechanisms (how to add new fields?)
5. Compliance testing (how to verify correct implementation?)

Relationship with Existing Standards Existing potentially relevant standards have limitations:

• DNS: Mature and widely deployed, but not designed for Agent discovery, lacks state and capability description, high query latency
• DNSSEC: Provides security verification, but complex and deployment difficult
• mDNS: Excellent for local network discovery, but unsuitable for cross-domain, multicast-based, unreliable
• DID: Distributed identity identification, but designed for identity not service discovery
• RDAP: Mature query language, but primarily designed for domain names and AS numbers

Conclusion: No existing standard completely fits; a new standard or extension may be needed.

Design Considerations and Trade-offs

Architecture Trade-off Matrix Based on the preceding problems, key architectural decisions and trade-offs: A summarized set of trade-offs is presented here in prose:

• Synchronization Content: Minimal set is fast and lightweight; full synchronization provides complete information but increases bandwidth and privacy risk; classified synchronization balances functionality and resource use.
• Topology: Peer-to-peer is decentralized but complex; hierarchical/star is easier to manage but introduces central points of failure; hybrid offers redundancy at the cost of complexity.
• Consistency: Strong consistency is accurate but unavailable during partitions; eventual consistency is practical for cross-domain systems; weak consistency is simple but unreliable.
• Triggering: Periodic synchronization is predictable; event-driven updates are responsive; on-demand queries conserve bandwidth; hybrid methods aim to optimize latency and cost.

Key Design Principles

Distributed-First Principle: Minimize central nodes. Implications: Avoid single points of failure, reduce central node operational costs, enable autonomous management of registry centers, allow partially-connected network topologies.

Eventual Consistency First Principle: Prioritize availability and fault tolerance; accept temporary inconsistency. Implications: Support asynchronous synchronization, system remains available during network partitions, clear conflict resolution strategy, periodic full synchronization ensures eventual consistency.

Minimal Information Principle Principle: Consider synchronizing only minimally necessary information first, then expand incrementally. Implications: First version synchronizes only basic connection information; capability descriptions retrieved via other mechanisms or cached; state information maintained via heartbeats; privacy-sensitive information protected by access control.

No-Assumptions Principle Principle: Do not assume ideal network environments or operational capabilities. Implications: No assumption of clock synchronization (use logical clocks or version numbers); handle unreliable links (support packet loss and retransmission); handle insufficient bandwidth (support compression and incremental updates); assume imperfect operations tools (design simple diagnostics).

Information Model Design Considerations

Minimal Information Set The "minimum necessary information" for an Agent should include:

MUST Have (Mandatory Fields):

Agent ID/DID (unique identification), IPv6 address (network communication), Port/Service endpoint (connection specification)

SHOULD Have (Recommended):

Online status (avoid accessing unavailable Agents), Timestamp (support consistency detection), Version number (detect updates), Registry center ID (track data origin)

MAY Have (Optional):

Capability list, Performance metrics, Access policies

Cost analysis shows mandatory + recommended fields (~500 bytes) are suitable for periodic synchronization; optional fields should be on-demand or separately cached.

Capability Information Model Three approaches exist:

1. No synchronization (only identity): Minimize message size (500B), maximize privacy, but require additional queries (50-200ms latency per query).
2. Full synchronization: Enable complete information in one query (10-50ms), support cross-domain capability matching, but large messages (3-5KB), frequent updates, privacy risks.
3. Layered synchronization (basic + detailed): Balance functionality and size (~800B), support basic capability matching, detailed info separately cached.

Version Control Strategy

Version Tracking Methods

Option A: Global Timestamps

Intuitive but depends on accurate clock synchronization; clock skew causes errors; cannot express causality.

Option B: Logical Clocks (Lamport)

No clock synchronization required; supports total ordering; cannot determine physical time order; cannot detect "very old" updates.

Option C: Vector Clocks

Supports causality detection; can judge concurrency; high complexity O(n); increased message size.

Recommendation: Hybrid approach using both timestamp (for readability and audit) and logical version number (for consistency checking), decoupling their purposes.

Conflict Resolution Algorithms When the same Agent information is modified simultaneously in two centers:

Layer 1: For simple state (online/offline)

Use Last-Write-Wins (LWW) with timestamps

Layer 2: For versioned data (capability lists)

Use logical version numbers

Layer 3: For complex conflicts

Use human intervention or CRDTs

In most scenarios, Layer 1 is sufficient.

Open Questions and Future Discussion

Critical Open Questions

When Should an Agent be Deleted from a Center? After an Agent goes offline (stopped sending heartbeats), when should its record be deleted? Immediate deletion loses recovery capability; delayed deletion wastes storage. Different applications may need different retention periods.

Cross-domain Permission Conflicts If Organization A's Agent is registered in Organization B's center, but later A and B have disputes, can B delete A's records? If B deletes the records, should other centers also delete them? If A keeps pushing updates, how should B handle them? This requires clear "data ownership" definitions.

Multiple Centers Having Different Understandings of the Same Agent Agent-1's connection address differs between Beijing and Shanghai centers. This could be legitimate (Agent has multiple addresses), a data staleness issue, or malicious modification. How to determine which is correct and merge conflicting records?

Extreme Latency Differences Within the same network, local centers may have <10ms latency while remote centers have >150ms. Should the protocol prioritize local center queries? If local data is incomplete, what's the fallback? Can the protocol be "geography-aware"?

Duplicate Agent Detection Due to synchronization delays and errors, the same Agent might be registered under different identifiers in the same center. How to automatically detect and merge duplicates without cascading failures?

Implementation Challenges

Cache Consistency Different centers may cache Agent information with different TTLs. This creates scenarios where the same Agent has inconsistent information across centers even after synchronization. Solutions include unified TTLs (reduces optimization), cache validation timestamps (increases complexity), accepting cache inconsistency (relies on eventual consistency), or avoiding caches entirely (increases latency).

Cascading Failures When one center fails, query traffic redirects to other centers, potentially multiplying their load 5-10 times. Without sufficient redundancy, the backup centers may also fail, causing system-wide collapse. Requires careful capacity planning, active traffic distribution, and rapid failure detection.

Large-Scale Synchronization Costs At scale (1 million Agents, 1000 centers), even though average bandwidth seems acceptable, non-uniform distribution, hot-spot queries, network routing inefficiencies, and burst traffic during recovery create real bottlenecks. Design must consider flow prediction, priority-based dropping, and bandwidth limit configurations.

Recommended Problem Resolution Directions

Short-term (First Protocol Version) High priority:

• Define minimal information set for synchronization
• Establish cross-domain authentication (DID or PKI-based)
• Specify consistency guarantees and conflict resolution
• Implement privacy protection via ACL and access control

Medium priority:

• Performance optimization (incremental updates, compression)
• Cache management (TTL and refresh mechanisms)
• Network adaptivity (support multiple addresses, failover)
• Monitoring and diagnostics (logging and metrics export)

Medium-term (Subsequent Versions)

• Automatic failure recovery and self-healing networks
• Intelligent caching policies (ML prediction + dynamic TTL)
• Cross-domain access management (XACL/attribute-based authorization)
• Geography-aware synchronization (BGP + geolocation encoding)

Areas Requiring Additional Research

• Scalability limits: Performance of million-scale Agents across 1000 centers
• Security analysis: Formal proofs of protocol security
• Implementation best practices: Key techniques for high-performance implementations
• Deployment patterns: Evolution from small to large scale
• Cost-benefit analysis: Actual deployment costs vs benefits vs centralized alternatives

Conclusions This document analyzes data synchronization problems for distributed Agent registry centers in IPv6 networks. Key findings include:

Core Challenges

1. Consistency vs Availability Trade-off: Strong consistency leads to unavailability; eventual consistency accepts temporary inconsistency.
2. Privacy vs Functionality Conflict: Complete information synchronization exposes privacy; minimal information limits functionality.
3. Latency vs Scalability Contradiction: Low latency requires dense communication; scaling to millions of Agents requires reducing communication.
4. Fundamental Distributed System Difficulties: No central authority, unreliable networks, difficult failure detection.

Design Recommendations

1. Adopt eventual consistency model to prioritize system availability.
2. Minimize synchronization content: start with connection information, expand incrementally.
3. Use layered architecture: exploit geographic locality at border, regional, and global levels.
4. Implement clear version management to support conflict detection and resolution.
5. Reserve extension space for future optimizations and customizations.

Future Work This document provides a foundation for problem analysis. Subsequent RFCs should:

1. Based on problem analysis, formulate concrete synchronization protocols.
2. Define minimized information models and interaction patterns.
3. Provide interoperability testing frameworks.
4. Collect lessons from practical deployments.

Normative References Informative References World Wide Web Consortium

Appendix A: Problem Checklist Maintainers of this IETF draft should periodically review:

• Is the synchronization scope clearly defined?
• Are consistency guarantees and conflict resolution explicit?
• Have all major security threats been considered?
• Can different implementations interoperate?
• Can the system scale to millions of Agents?
• Does performance meet critical application requirements?
• Is operational cost manageable?
• Is space reserved for future improvements?
• Are deployment guidelines clear?
• Are compliance tests defined?

Appendix B: Glossary

Agent Registry (AR)

A centralized service that maintains Agent information

Distributed Registry

A federation of multiple Agent Registries

Registry Synchronization

Information synchronization between multiple registries

Eventual Consistency

Consistency model allowing temporary inconsistency

Conflict-free Replicated Data Type (CRDT)

Data structure that automatically supports merging

Last-Write-Wins (LWW)

Conflict resolution strategy using latest update

Vector Clock (VC)

Time mechanism tracking causality

Access Control List (ACL)

List-based access control

Decentralized Identifier (DID)

Distributed identity identification

Autonomous System (AS)

A network administration domain

Quality of Service (QoS)

Service quality metrics

Service Level Agreement (SLA)

Agreement specifying service levels

Mean Time To Recovery (MTTR)

Average time to recover from failure