libu8ident - Unicode Security Guidelines for Identifiers

The libu8ident library provides robust security checks for Unicode identifiers within various systems such as compilers, interpreters, filesystems, and login mechanisms, all without requiring the full Unicode database. It aims to help developers prevent vulnerabilities associated with Unicode identifiers, such as bidirectional (bidi) attacks, homoglyph attacks, mixed scripts, and confusables, while ensuring proper normalization and storage of identifiers.

Key Features:

• Implements essential Unicode identifier security checks without the overhead of maintaining an extensive database.
• Protects against a wide range of vulnerabilities including mixed scripts and confusables, which are often overlooked in existing systems, making approximately 98% of systems vulnerable.
• Normalization support: Ensures all UTF-8 identifiers are stored in their normalized NFC forms, minimizing potential security risks associated with identifier spoofing.
• Configurable tokenizer options: Customize the tokenizer for identifier validation, allowing for various security profiles and Unicode properties, such as ID_Start and ID_Continue.

Motivation for Development:

• The need for rigorous checks on Unicode identifiers has become increasingly critical due to the rising instances of security vulnerabilities exploiting Unicode's complexity. Notable references include resources from the Unicode Consortium and issues raised in various development communities.

Usage Example:

const [ ENV_PROD, ENV_DEV ] = [ 'PRODUCTION', 'DEVELOPMENT'];
const environment = 'PRODUCTION';
function isUserAdmin(user) {
    if (environmentǃ=ENV_PROD) {
        // Bypass authorization checks in DEV
        return true;
    }
    return false;
}

In the code snippet above, the identifier environmentǃ is misleading due to the use of a similar-looking character, demonstrating how easily confusion can arise without proper checks.

Valid Character Checks: Each identifier must follow the specifications outlined in the Unicode Technical Reports (e.g., TR31), ensuring that identifiers start and continue with allowed characters, while also adhering to user-defined security profiles.

Security Profiles: libu8ident supports several predefined security profiles to categorize the string based on the allowed character sets:

• Level 1: ASCII-Only
• Level 2: Single Script
• Level 3: Highly Restrictive
• Level 4: Moderately Restrictive (Recommended)
• Level 5: Minimally Restrictive
• Level 6: Unrestricted

API Integration: The library offers a straightforward API including functionality for initializing the library, checking identifier validity, and managing contexts. Below is a simple call to check an identifier:

int result = u8ident_check("identifier");
if (result == U8ID_ERR_SCRIPTS) {
   // Handle error accordingly.
}

By integrating libu8ident into applications, developers can significantly enhance the security of Unicode identifiers, ensuring they conform to modern security best practices.