NeuralSVG: An Implicit Representation for Text-to-Vector Generation

NeuralSVG is an innovative framework designed to generate vector graphics from textual prompts, introducing ordered and editable shapes that facilitate dynamic conditioning, such as background colors. This advanced model supports the production of diverse color palettes for each learned representation, enhancing the user experience for artists and designers.

Overview

Vector graphics are critical in design as they offer an adaptable means of creating highly editable, resolution-independent visuals. With recent advancements in vision-language and diffusion models, there is a notable surge in interest towards text-to-vector graphics generation. Existing solutions, however, often yield over-parameterized outputs and typically neglect the layered structure, which is fundamental to vector graphics.

NeuralSVG addresses these challenges by providing an implicit neural representation for generating vector graphics directly from text prompts. Drawing inspiration from Neural Radiance Fields (NeRFs), NeuralSVG encodes entire scenes into the weights of a small multi-layer perceptron (MLP) network, optimized through Score Distillation Sampling (SDS). To reinforce the layered structure of the generated SVGs, a dropout-based regularization technique is introduced, ensuring each shape maintains its standalone contextual meaning. Moreover, the neural representation enables inference-time control, allowing for dynamic adaptations based on user inputs, all from a single learned model.

Through comprehensive qualitative and quantitative analyses, NeuralSVG has demonstrated a superior capacity in producing structured and flexible SVG graphics compared to existing approaches.

Features

• Dynamic Conditioning: Generate graphics with varying background colors and effects based on user-input prompts.
• Layered Structure: Each output shape retains distinct meaning, which is essential for effective design functionalities.
• Flexible Usage: Modify the generated SVGs easily according to specific requirements, maximizing artistic versatility.
• Support for Multiple Color Palettes: Experiment with various colors while preserving the integrity of each graphic representation.

Examples

NeuralSVG is capable of generating a wide range of vector graphics, shown in multiple examples such as:

• Minimalist vector art of a sunflower
• Baby penguin illustration
• A stack of pancakes with a bunny
• Minimal line drawings of various subjects, including flora and fauna.

Usage

The framework allows users to experiment with different prompts, colors, and aspect ratios using simple command-line instructions.

For example, to generate a sunflower with specific background colors, run:

python scripts/train.py --config_path config_files/run_shaping.yaml --data.text_prompt="minimalist vector art of a sunflower" --model.toggle_color="true" --model.toggle_color_bg_colors="['light-red', 'light-green', 'light-blue', 'gold', 'gray']" --model.lora_weights="./lora_weights/lora_weights_sd21b_bg_color.safetensors" --log.exp_name='neuralsvg_sunflower'  

Conclusion

NeuralSVG represents a significant advancement in the field of text-to-vector graphics, empowering artists with a powerful tool to transform their ideas into structured visual representations. For those seeking to expand their creative capabilities, exploring NeuralSVG's potential offers a promising addition to vector graphic design workflows.