Skip to content

Llama 3.2 1B/3B fail to load in candle backend (hardcoded tie_word_embeddings=false) #646

Description

@ohdearquant

Bug

The candle backend (crates/ruvllm/src/backends/candle_backend.rs, ModelArchitecture::Llama load path) hardcodes tie_word_embeddings: false. Llama 3.2 1B/3B tie the output projection to the input embeddings, so their checkpoints ship no separate lm_head.weight tensor. With the flag forced to false, candle looks for lm_head.weight and the load fails.

Expected

Llama 3.2 1B/3B checkpoints load and generate correctly.

Fix

Detect tied embeddings from tensor presence: tie_word_embeddings = !vb.contains_tensor("lm_head.weight"). Models shipping an explicit lm_head.weight are unaffected.

Fixed by #645.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions