Skyulf

Skyulf is an innovative self-hosted Machine Learning Operations (MLOps) Hub that enables users to build, train, and manage ML pipelines entirely on their own infrastructure, ensuring full data sovereignty without the concern of vendor lock-in. Designed as an intuitive solution to streamline data science workflows, Skyulf eliminates the hassle of convoluted code integrations.

Key Features

• Visual Feature Canvas: Utilize a node-based editor to visually clean, transform, and engineer features, freeing users from the complexities of writing extensive code. The canvas boasts over 25 built-in nodes for diverse data manipulation tasks.
• Modern Technology Stack: Leveraging FastAPI for high-performance backend capabilities, React for a dynamic frontend, Celery for seamless asynchronous task management, and Redis for efficient data handling.
• Asynchronous Processing: Heavy model training tasks run in the background using Celery, ensuring that the user interface remains responsive and functional.
• Flexible Data Ingestion: Supports various data formats including CSV, Excel, JSON, Parquet, and SQL, starting with SQLite for easy setup and scalability to PostgreSQL as needs grow.
• Comprehensive Model Training: Built-in support for Scikit-Learn models with features for hyperparameter optimization including Grid, Random, and Halving search, alongside optional Optuna integration.
• Model Registry & Deployment: Track models through version control, monitor performance metrics, and deploy to a live inference API with ease.
• Experiment Tracking: Compare different model training runs side-by-side visually with interactive charts, confusion matrices, and ROC curves for informed decision-making.

Future Development

Skyulf aims to evolve into a complete App Hub for AI with a phased approach:

• Phase 1: Focus on stability and refinement.
• Phase 2: Enhance data science capabilities with advanced exploratory data analysis and fairness checks.
• Phase 3: Develop a plugin system and integration of Generative AI/LLM builders.
• Phase 4: Expand features to include real-time collaboration and support for IoT exports.

For an in-depth look at the development roadmap, refer to the full roadmap.

Overview of Workflow

Skyulf provides a streamlined pathway from dataset ingestion to model training, encapsulated in a user-friendly interface. Following the flow from data source to training validation and testing, users can leverage Celery for efficient background processing.

Skyulf is actively developed and welcomes contributions from the community. Engage with the project, contribute, or simply try out the capabilities of this feature-rich MLOps hub.