Samchika

Samchika is an innovative, high-performance file processing library designed for the Java programming language. With a focus on multithreading, Samchika caters to handling CPU-intensive tasks efficiently, allowing users to process large files rapidly and in parallel.

Key Features

• Fully Multithreaded: Optimized for parallel file processing, ensuring maximum performance.
• Simple API: Effortlessly integrate by supplying just the file path and processing logic.
• Optional Runtime Statistics: Provides insights into time taken, memory usage, and thread performance.
• Ideal for Large Text Files: Perfect for use in scenarios such as log processing and dataset analysis.
• Open-source Friendly: Contributions are encouraged, fostering community collaboration.

Use Cases

Samchika is particularly advantageous in scenarios such as:

• Log Analysis & Processing
• ETL (Extract, Transform, Load) Operations
• Processing Large Text Corpuses
• Generating Batch Reports
• Building Data Transformation Pipelines
• Conducting Real-time Data Processing

Explore the examples directory for implementations that demonstrate these applications.

Quick Example

To transform a large CSV file with optimal performance, implement the following code:

SmartFileProcessor.builder()
    .inputPath("large_dataset.csv")
    .outputPath("transformed_dataset.csv")
    .batchSize(10000)
    .lineProcessor(line -> line.toUpperCase())
    .displayStats(true)
    .build()
    .execute();

Performance

Samchika has been benchmarked against traditional BufferedReader-based methods, showcasing significant improvements in processing times, especially on multi-core systems. Users can observe over a 70% performance gain on larger files (e.g., 16 GB) while maintaining manageable memory usage (approximately 800 MB) during operations.

This library is a response to the prevalent challenges in processing large text files, designed to provide an elegant API combined with robust multithreading capabilities.