app-use

App Use provides a straightforward solution for enabling AI agents to interact with mobile applications effectively. By offering a simple yet powerful interface, it facilitates straightforward automation of tasks across various mobile platforms.

Features

• Cross-Platform Compatibility: Supports both iOS and Android applications.
• Customizable Agents: Easily define the mobile device and application settings for the agent to operate efficiently.
• AI Integration: Utilize advanced AI models like OpenAI's GPT-4 for intelligent task handling.

Quick Start

To initiate the app, you need Python version 3.11 or later. Installation can be performed using pip:

pip install app-use

For those requiring memory functionality (requires Python < 3.13):

pip install "app-use[memory]"

Example Usage

Create an instance of the app targeting the desired mobile environment:

app = App(
    platform_name="ios",
    device_name='Your Device Name',
    bundle_id="com.apple.calculator",
    appium_server_url='http://localhost:4723',
    {"udid": 'device-id'},
)

Then, spin up your AI agent:

import asyncio
from dotenv import load_dotenv
load_dotenv()
from app_use import Agent
from langchain_openai import ChatOpenAI

async def main():
    agent = Agent(
        task="What is 2+2?",
        llm=ChatOpenAI(model="gpt-4o"),
        app=app
    )
    await agent.run()

asyncio.run(main())

Make sure to set up your API keys in the .env file to connect to the required services:

OPENAI_API_KEY=
ANTHROPIC_API_KEY=
AZURE_OPENAI_ENDPOINT=
AZURE_OPENAI_KEY=
GOOGLE_API_KEY=
DEEPSEEK_API_KEY=
GROK_API_KEY=
NOVITA_API_KEY=

Demos

Experience the capabilities of App Use through various use-case demonstrations, such as ordering items from popular apps like Lululemon and DoorDash:

Lululemon App	DoorDash

Video

https://x.com/itsericktorres/status/1932996729458110482?s=46

Community & Support

Contributions are encouraged, and the development environment for App Use is actively maintained to ensure reliability and ease of use for all users. Contributions are welcomed through pull requests.