pgwire-replication

pgwire-replication

This low-level, high-performance PostgreSQL logical replication client is built directly on top of the PostgreSQL wire protocol (pgwire). It is specifically designed for Change Data Capture (CDC), change streaming, and WAL replay systems, providing developers with explicit control over replication state, deterministic restart behavior, and minimal runtime overhead.

Key Features

• Direct pgwire Implementation: Leverages PostgreSQL's logical replication capabilities without the overhead of higher-level clients like libpq or tokio-postgres.
• Customizable LSN Control: Offers user-controlled Log Sequence Number (LSN) starting and stopping semantics, essential for resuming replication after interruptions.
• Predictable Behavior: Ensures consistent feedback and backpressure mechanisms, allowing for clean integration into asynchronous systems and process coordinators.
• Support for SCRAM-SHA-256 and MD5 Authentication: Securely interact with PostgreSQL using modern authentication methods.
• TLS/mTLS Support: Enable Transport Layer Security for secure replication connections, ensuring the safety and confidentiality of the data being transmitted.

Basic Usage Example

Here is a simple example of how to integrate pgwire-replication into an application:

use pgwire_replication::{ReplicationClient, ReplicationEvent};

let mut repl = ReplicationClient::connect(config).await?;

while let Some(event) = repl.recv().await? {
    match event {
        ReplicationEvent::XLogData { wal_end, data, .. } => {
            process(data);
            repl.update_applied_lsn(wal_end);
        }
        ReplicationEvent::KeepAlive { .. } => {}
        ReplicationEvent::StoppedAt { reached } => break,
    }
}
// Clean end-of-stream

Bounded Replay and Progress Tracking

The pgwire-replication crate allows for controlled replay of WAL using explicit LSNs, providing a way to pause and resume replication based on specified positions. This is particularly useful for ensuring data consistency during recovery or backfill operations.

Conclusion

As a focused library for PostgreSQL logical replication, pgwire-replication stands out by providing specialized functions necessary for performance-sensitive applications. Its direct interaction with the PostgreSQL wire protocol makes it an optimal choice for systems that rely on efficient change data capture with minimal latency and overhead.