Netra Kernel offers cutting-edge ahead-of-time GPU kernels optimized for production inference on AMD hardware. This project compiles model operations into fixed-contract raw-assembly kernels and allocates them as loadable Netra Engines, providing a streamlined experience similar to TensorRT's workflow for profiles, tactics, layout planning, static memory management, and engine construction.

Key Features of Netra Kernel

• Unmatched Native Performance: Netra ensures that kernels remain as schedule-specific raw assembly, allowing reusable mechanisms across various model operations such as Qwen, Gemma, and Llama.
• Predictable Serving: All attributes including shapes, data types, layouts, quantization, and launch parameters are finalized ahead of time, ensuring consistency.
• Optimized Engine Path: Minimal overhead during engine initialization allows for quick loading of modules, symbol resolution, memory binding, and optional HIP graph capture, while enabling the reuse of cached handles and caller-owned streams during serving.
• Safe Specialization: With precise profile guards, Netra prevents approximate dispatch, keeping the process reliable and supported contracts returning an explicit fallback if unsupported.
• Evidence-based Optimization: Tactics maintain maturity and correctness backed by evidence, guaranteeing high-performance outputs.

Proven Technology

The initial backend supports gfx950/wave64, focusing on high-throughput FP8 inference specifically designed for large language models like Qwen3.6-35B. Extensive validation has demonstrated significant performance achievements:

The Netra Platform

The architecture of the Netra platform consists of multiple components:

Model or canonical graph
          │
          ▼
  Netra Compiler
  contracts · profiles · layouts · tactics · memory plan
          │
          ▼
  Specialized raw assembly + Netra Engine
  fixed symbols · fixed launches · graph recipe · fallbacks
          │
          ▼
  Netra Runtime
  cached HIP handles · stable bindings · caller-owned stream

Integration and Repository Layout

The project is structured to keep the public integration surface distinct from target-specific kernels and inference-engine adapters:

netra-kernel/
├── compiler/netra_compiler/   # AOT compiler, IR, planners, frontends, backends
│   ├── backends/gfx950/       # Target-specific compiler implementation
│   └── library_data/          # Packaged kernels, manifests, and schemas
├── python/netra_kernel/       # Bundle, runtime, CLI, and adapter APIs
│   ├── runtime/               # Framework-neutral Python FFI
│   ├── sglang/                # SGLang adapter and startup plugin only
│   └── assets/                # Deployment profiles and packaged overlays
├── include/netra/engine.h     # Stable framework-neutral C ABI
├── runtime/                   # Native runtime implementations
├── examples/reference_engine/ # Minimal C consumer with no framework dependency
├── kernels/                   # Architecture-specific kernel sources
├── manifests/                 # Model, deployment, and tactic contracts
├── schemas/                   # Public JSON schemas
├── docker/sglang/             # Thin-plugin and accepted-source images
├── tools/                     # Build, compiler, benchmark, and CI tooling
├── tests/                     # Compiler, package, and GPU validation
└── docs/                      # Integration guides and acceptance evidence

Supported Capabilities

• GPU Target: AMD gfx950, wave64
• Quantization Focus: Primarily utilizes FP8 E4M3 with tailored block-scale and layout semantics
• Model Input Types: Accepts canonical JSON graphs along with specific model manifests.

For developers interested in leveraging the capabilities of AMD GPUs for inference tasks, Netra Kernel represents a powerful and efficient strategy, providing solid foundations for both performance and flexibility in deployment.