WayURLS

Wayurls is an innovative Command-Line Interface (CLI) tool designed to seamlessly fetch URLs from popular web archiving sources such as the Wayback Machine, Common Crawl, and VirusTotal.

Key Features

• Multisource Fetching: Effortlessly retrieve URLs from multiple reputable sources, giving you comprehensive access to archived content.
• Flexible Options: Customize your URL fetching process with various options:
  • -d: Display fetch dates in the results.
  • -t <domain|file>: Specify a target domain or use a file containing a list of domains.
  • -n: Exclude subdomains for a cleaner result set.
  • -o <file>: Output your results to a file, defaulting to standard output.
  • -v: List all crawled versions of the URLs, or get details from VirusTotal with the -vt <key> option using your API key.

Usage Examples

Unlock the power of Wayurls to access archived URLs with straightforward commands:

# Fetch URLs for a single domain
wayurls example.com

# Fetch URLs from a list of domains
wayurls -t domains.txt -o results.txt

# Fetch URLs with dates, excluding subdomains
wayurls -d -n -t example.com

# List crawled versions of URLs
wayurls -v example.com

Wayurls invites contributions and improvements from the community, fostering collaborative development for enhanced URL management and archive retrieval.

Harness the potential of historical web data effortlessly with Wayurls!