dir2txt

dir2txt is a powerful command-line interface (CLI) tool designed to export a directory's structure and contents into neatly formatted .txt or .json files. This utility is optimized for speed and precision, transforming entire project directories into readable outputs effortlessly.

Key Features

• Beautiful Directory Tree View: Generate an intuitive visualization of your directory structure.
• File Content Dumping: Optionally includes actual file contents while stripping comments, making the output cleaner.
• Respect for Ignored Files: Adheres to .gitignore, .dockerignore, and other custom ignore files to manage file selections effectively.
• Smart Binary Detection: Automatically identifies and skips binary files to focus on relevant content.
• Text and JSON Outputs: Outputs in either plain text or structured JSON format, ideal for integration with AI applications and data processing pipelines.

Optimized for AI and LLM Workflows

dir2txt serves as a critical first step in various AI workflows, including those utilizing GPT-based agents and Retrieval-Augmented Generation (RAG). It prepares codebases for context ingestion, allowing for smoother interactions with language models.

Use Cases Include:

• Preparing prompt-friendly project dumps for AI models like ChatGPT or Claude.
• Feeding structured trees and contents into vector databases for efficient searches and queries.
• Preprocessing source code for embedding and retrieval.
• Supporting local AI agents that need to understand code structures for tasks like auditing or debugging.

Typical Pipeline Example

The following pipeline illustrates how dir2txt fits into a broader AI processing workflow:

[ Source Code Directory ]
           │
           ▼
      📂 dir2txt
    (Tree + Content Dump)
           │
           ▼
  🔗 Chunking + Embedding
   (e.g., LangChain, llama-index)
           │
           ▼
     🗂️ Vector DB / Index
    (ChromaDB, Weaviate, FAISS)
           │
           ▼
     🤖 LLM / AI Agent
 (e.g., GPT, Claude, Mistral, LLaMA)

Example Usage

To use dir2txt, execute the following command for optimal results:

dir2txt ./my-project \
  --ignore node_modules,build \
  --strip-comments \
  --include-dotfiles \
  --json

dir2txt offers a seamless solution for transforming messy source directories into structured inputs suitable for AI processing, enhancing the efficiency and clarity of information retrieval in modern programming environments.