reasoning_core

Reasoning Core is an advanced text-based Reinforcement Learning Virtual Environment (RLVR) specifically designed for training Large Language Models (LLMs) in symbolic reasoning tasks. It emphasizes a variety of expressive symbolic challenges, encompassing full-fledged First-Order Logic (FOL), formal mathematics utilizing TPTP, novel domains in formal planning, and intricate syntax tasks.

Key Features

• Expressive Symbolic Tasks: Engage with a diverse range of tasks that enhance LLM reasoning capabilities across multiple domains.
• Integration with OpenAI: Utilize the OpenAI client for seamless evaluations and integration.
• Task Generation: Efficiently generate tasks and datasets for comprehensive LLM training using multi-threaded processes.

Quick Start Examples

To employ the Reasoning Core environment, install the necessary tools and begin using its functionalities:

Prime Environment Setup

#!pip install uv 
!uv tool install prime --with openai  -q
!uv tool run prime -- env install sileod/reasoning-core-env

from verifiers import load_environment
import os; from openai import OpenAI

env = load_environment("reasoning-core-env")

client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key=os.getenv("OPENROUTER_API_KEY"))
results = env.evaluate(client=client, model="gpt-4.1-mini", num_examples=20, rollouts_per_example=1)
df = env.make_dataset(results).to_pandas()

Standalone Use

For independent task usage, the library can be installed directly:

pip install reasoning_core

from reasoning_core import list_tasks, get_task, score_answer

T = get_task('arithmetics')()
x = T.generate_example()
assert score_answer(x.answer, x) == 1

Dataset Generation

To produce datasets for use in HuggingFace, execute:

bash run_generate.sh

Reasoning Gym Compatibility

Reasoning Core features a custom interface that can seamlessly integrate with Reasoning Gym (RG), allowing the inclusion of distinct tasks. Here’s an example:

import reasoning_gym

from reasoning_core import register_to_reasoning_gym
register_to_reasoning_gym()

specs = [
    DatasetSpec(name='leg_counting', weight=2, config={}), 
    DatasetSpec(name='arithmetics', weight=2, config={}), 
]
D = reasoning_gym.create_dataset('composite', size=10, seed=42, datasets=specs)

For further details, access the full documentation and explore the capabilities of Reasoning Core to enhance model performance in symbolic reasoning tasks.