concord

Concord is a resilient implementation of the Chord protocol developed in Go, designed to provide a reliable and consistent hashing mechanism that can effectively handle node failures. This makes it a robust solution for building scalable distributed systems.

Overview

Concord operates on the foundational principles established by Pamela Zave, ensuring a well-architected implementation. The primary features of Concord include:

• Failure Resilience: The system is configured to support resilience against node failures, maintaining operational consistency for the distributed network.
• Consistent Hashing: It provides fundamental primitives necessary for consistent hashing in distributed environments.
• Range Change Callbacks: With the OnRangeChange callback feature, applications are notified when a node takes on responsibility for new key ranges, enhancing the functionality of distributed hash tables (DHTs).
• Customizable Hashing: Users can implement their own hash functions and configure the hash bit-width, accommodating up to 64-bit keys.
• Structured Logging: Concord utilizes Go's log/slog for structured and customizable logging, facilitating better monitoring.
• gRPC Communication: It leverages gRPC for efficient internal node-to-node communication.

Usage Examples

Below are usage examples demonstrating how to create and manage nodes within a Concord cluster:

Creating a New Cluster

package main

import (
    "log"
    "github.com/ollelogdahl/concord"
)

func main() {
    config := concord.Config{
        Name:     "node1",
        BindAddr: "0.0.0.0:7946",
        AdvAddr:  "node1.example.com:7946",
        OnRangeChange: func(r concord.Range) {
            log.Printf("Range changed: %d-%d", r.Start, r.End)
        },
    }

    node := concord.New(config)
    if err := node.Start(); err != nil {
        log.Fatal(err)
    }
    defer node.Stop()

    if err := node.Create(); err != nil {
        log.Fatal(err)
    }

    log.Printf("Node %s (ID: %d) created cluster", node.Name(), node.Id())
}

Joining an Existing Cluster

package main

import (
    "context"
    "log"
    "github.com/ollelogdahl/concord"
)

func main() {
    config := concord.Config{
        Name:     "node2",
        BindAddr: "0.0.0.0:7947",
        AdvAddr:  "node2.example.com:7947",
    }

    node := concord.New(config)
    if err := node.Start(); err != nil {
        log.Fatal(err)
    }
    defer node.Stop()

    ctx := context.Background()
    if err := node.Join(ctx, "node1.example.com:7946"); err != nil {
        log.Fatal(err)
    }

    log.Printf("Node %s joined cluster", node.Name())
}

Key Lookup

key := []byte("my-key")
server, err := node.Lookup(key)
if err != nil {
    log.Fatal(err)
}

log.Printf("Key is managed by node %s (ID: %d) at %s",
    server.Name, server.Id, server.Address)

Development Guidelines

Concord is built using Go 1.24.0 or higher and requires the Protocol Buffers compiler along with Go generators for gRPC support. The development process includes building, testing, and fuzzing functionalities that help ensure the robustness of the implementation. This project also provides example nodes to facilitate new users in getting started quickly.

Acknowledgements

Concord is rooted in the original research on the Chord protocol. References for additional reading include the paper "Chord: A Scalable Peer-to-Peer Lookup Service for Internet" and the formal verification of Chord in "Reasoning about Identifier Spaces: How to Make Chord Correct."

This project aims to make consistent and scalable distribution systems accessible through a well-formed library that emphasizes resilience and adaptability.