turtlesim_agent

turtlesim_agent is an innovative AI agent that transforms the classic ROS turtlesim simulator into a dynamic canvas powered by natural language processing. Leveraging the capabilities of LangChain, this agent interprets text-based instructions to create visual drawings, effectively turning the turtlesim turtle into a digital artist. This project demonstrates how large language models can engage with external environments, showcasing creative behavior through user-defined prompts.

Features

Interactive Drawing Capabilities

Users can specify shapes or drawing intentions using plain English, which the AI agent then interprets and executes using turtlesim's motion commands. For instance, users can generate basic shapes or complex imagery, such as a rainbow:

Example Prompts

📐 Drawing Basic Shapes

🌈 Drawing a Rainbow

Prompt:
"I want you to draw a rainbow composed of 7 semi-circular arcs, each with a different color and a radius ranging from 2.0 cm to 2.7 cm..."

Note: The outcomes may vary due to the non-deterministic nature of language models; results can differ based on prompt phrasing, timing, or model adjustments.

Getting Started

This project requires:

• ROS 2 Humble Hawksbill
• Python 3.10+
• Other packages specified in requirements.txt

Running the Agent

turtlesim_agent provides two interaction modes:

• CLI Interface: Ideal for debugging and understanding the agent's reasoning.
• GUI Chat Interface: Offers a more intuitive user experience.

# For CLI
$ ros2 run turtlesim turtlesim_node
$ ros2 run turtlesim_agent turtlesim_agent_node

# For GUI
$ ros2 launch turtlesim_agent turtlesim_agent.launch.xml

Extend Agent Functionality

The tools/ directory contains various functions that the agent can call during its reasoning processes. Users are encouraged to augment the agent's capabilities by creating custom tools in this directory. For adding new functions:

1. Create a new *_tools.py file.
2. Ensure compatibility with LangChain signatures.
3. Import these in all_tools.py for accessibility.

Contribution and Collaboration

Contributions to enhance turtlesim_agent are welcome. Suggestions for new tools, smarter models, or novel use cases can significantly enrich the project. Users are invited to contribute ideas through issues or pull requests, and any successful drawing experiments can be documented on the project's wiki.