turtlesim agent — Transforming natural language into…

Pitch

Description

Comments

About this project

turtlesim_agent is an innovative AI agent that elevates the classic ROS turtlesim simulator into a creative platform. By interpreting text-based prompts, this project allows users to guide a digital turtle in producing intricate drawings, demonstrating the powerful intersection of language models and visual art.

turtlesim_agent is an innovative AI-driven solution that transforms the classic ROS turtlesim simulator into an interactive canvas. This project harnesses the capabilities of large language models to interpret natural language instructions, enabling users to create intricate drawings through simple text prompts. With turtlesim_agent, the simulated turtle becomes a digital artist, executing user-defined visual scenarios based on descriptions provided in plain English.

Key Features

Interactive Drawing: Users can instruct the AI to draw shapes and designs, such as basic geometric figures or detailed objects like rainbows, simply by typing their requests.
AI-Powered Creativity: Leveraging the power of LangChain, the agent reasons through commands to realize artistic intentions, showcasing the interaction between AI and external environments.

Examples of Drawings

Basic Shapes: Visualize fundamental shapes with ease.
Complex Designs: Create elaborate structures, such as a rainbow consisting of 7 semi-circular arcs in diverse colors, adhering to specifications regarding size, color sequence, and pen width.

Sample Prompt for Rainbow Drawing:

I want you to draw a rainbow composed of 7 semi-circular arcs, each with a different color and a radius ranging from 2.0 cm to 2.7 cm. The colors should follow the traditional rainbow order: violet, indigo, blue, green, yellow, orange, red with the pen's width of 5. Please offset the starting position of each semi-circle by 0.1 cm to avoid overlap.

Architecture and Tools

The project is organized in a structure that supports various functionalities necessary for drawing:

Motion Control: Directs the turtle's movements.
Pen Management: Controls pen states like color and width.
Mathematical Tools: Provides basic arithmetic and geometric calculations.
Simulation Management: Resets the simulation and manages turtle instances.

Extending Functionality

To enhance the agent's creativity further, new tools can easily be added:

Create a new *_tools.py file in the tools/ directory.
Define your functions compatible with LangChain.
Ensure they are imported in all_tools.py for accessibility by the agent.

Interactive Interface

This agent offers two modes of engagement:

CLI Interface: Ideal for debugging and exploring the inner workings of the AI agent's reasoning process.
GUI Chat Interface: Provides a user-friendly environment for interaction, suitable for casual users.

Contribution

Innovations for enhancing the turtlesim_agent are encouraged. Contributions can range from adding new drawing tools to experimenting with smarter models and prompts. Check the wiki to document successful experiments and outcomes.

Explore the possibilities of AI-driven artistry today with turtlesim_agent.