Gemma4 Architecture Error

Running without SSL init: using 13 threads for HTTP server start: binding port with default address family main: loading model srv load_model: loading model 'gemma4.gguf' common_init_result: fitting params to device memory, for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4' llama_model_load_from_file_impl: failed to load model llama_params_fit: encountered an error while trying to fit params to free device memory: failed to load model llama_params_fit: fitting params to free memory took 0.14 seconds llama_model_load_from_file_impl: using device MTL0 (Apple M4 Pro) (unknown id) - 38338 MiB free llama_model_loader: loaded meta data with 48 key-value pairs and 720 tensors from gemma4.gguf (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = gemma4 llama_model_loader: - kv 1: general.type str = model llama_model_loader: - kv 2: general.sampling.top_k i32 = 64 llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000 llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000 llama_model_loader: - kv 5: general.name str = Gemma-4-E4B-Uncensored-HauhauCS-Aggre... llama_model_loader: - kv 6: general.finetune str = 5Ref llama_model_loader: - kv 7: general.basename str = KL0.0378 llama_model_loader: - kv 8: general.size_label str = 7.5B llama_model_loader: - kv 9: gemma4.block_count u32 = 42 llama_model_loader: - kv 10: gemma4.context_length u32 = 131072 llama_model_loader: - kv 11: gemma4.embedding_length u32 = 2560 llama_model_loader: - kv 12: gemma4.feed_forward_length u32 = 10240 llama_model_loader: - kv 13: gemma4.attention.head_count u32 = 8 llama_model_loader: - kv 14: gemma4.attention.head_count_kv u32 = 2 llama_model_loader: - kv 15: gemma4.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 16: gemma4.rope.freq_base_swa f32 = 10000.000000 llama_model_loader: - kv 17: gemma4.attention.layer_norm_rms_epsilon f32 = 0.000001 llama_model_loader: - kv 18: gemma4.attention.key_length u32 = 512 llama_model_loader: - kv 19: gemma4.attention.value_length u32 = 512 llama_model_loader: - kv 20: gemma4.final_logit_softcapping f32 = 30.000000 llama_model_loader: - kv 21: gemma4.attention.sliding_window u32 = 512 llama_model_loader: - kv 22: gemma4.attention.shared_kv_layers u32 = 18 llama_model_loader: - kv 23: gemma4.embedding_length_per_layer_input u32 = 256 llama_model_loader: - kv 24: gemma4.attention.sliding_window_pattern arr[bool,42] = [true, true, true, true, true, false,... llama_model_loader: - kv 25: gemma4.attention.key_length_swa u32 = 256 llama_model_loader: - kv 26: gemma4.attention.value_length_swa u32 = 256 llama_model_loader: - kv 27: gemma4.rope.dimension_count u32 = 512 llama_model_loader: - kv 28: gemma4.rope.dimension_count_swa u32 = 256 llama_model_loader: - kv 29: tokenizer.ggml.model str = gemma4 llama_model_loader: - kv 30: tokenizer.ggml.tokens arr[str,262144] = ["<pad>", "<eos>", "<bos>", "<unk>", ... llama_model_loader: - kv 31: tokenizer.ggml.scores arr[f32,262144] = [-1000.000000, -1000.000000, -1000.00... llama_model_loader: - kv 32: tokenizer.ggml.token_type arr[i32,262144] = [3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 33: tokenizer.ggml.merges arr[str,514906] = ["\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n", ... llama_model_loader: - kv 34: tokenizer.ggml.bos_token_id u32 = 2 llama_model_loader: - kv 35: tokenizer.ggml.eos_token_id u32 = 1 llama_model_loader: - kv 36: tokenizer.ggml.unknown_token_id u32 = 3 llama_model_loader: - kv 37: tokenizer.ggml.padding_token_id u32 = 0 llama_model_loader: - kv 38: tokenizer.ggml.mask_token_id u32 = 4 llama_model_loader: - kv 39: tokenizer.chat_template str = {%- macro format_parameters(propertie... llama_model_loader: - kv 40: tokenizer.ggml.add_space_prefix bool = false llama_model_loader: - kv 41: tokenizer.ggml.add_bos_token bool = false llama_model_loader: - kv 42: general.quantization_version u32 = 2 llama_model_loader: - kv 43: general.file_type u32 = 18 llama_model_loader: - kv 44: quantize.imatrix.file str = /media/hauhaucs/New Volume/LM Models/... llama_model_loader: - kv 45: quantize.imatrix.dataset str = /home/hauhaucs/llama.cpp/groups_merge... llama_model_loader: - kv 46: quantize.imatrix.entries_count u32 = 342 llama_model_loader: - kv 47: quantize.imatrix.chunks_count u32 = 93 llama_model_loader: - type f32: 339 tensors llama_model_loader: - type q8_0: 85 tensors llama_model_loader: - type q6_K: 295 tensors llama_model_loader: - type bf16: 1 tensors print_info: file format = GGUF V3 (latest) print_info: file type = Q6_K print_info: file size = 5.81 GiB (6.63 BPW) llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4' llama_model_load_from_file_impl: failed to load model common_init_from_params: failed to load model 'gemma4.gguf' srv load_model: failed to load model, 'gemma4.gguf' srv operator(): operator(): cleaning up before exit... main: exiting due to model loading error danielkliewer@Daniels-MacBook-Pro-3 chopnscrwbot %

This isn’t a random runtime bug — your stack is telling you something very specific: > **`unknown model architecture: 'gemma4'`** ### What’s actually happening Your build of entity["software","llama.cpp","C++ LLM inference engine"] **does not support the Gemma 4 architecture**. Even though: - The `.gguf` file is valid ✅ - Metadata is readable ✅ - Quantization is fine (Q6_K) ✅ …it fails at the architecture layer because your binary literally doesn’t know what `gemma4` is. --- ## Why this happens entity["organization","Google DeepMind","AI research organization"] introduced newer Gemma architectures (Gemma 2, 3, 4, etc.), but: - Support in entity["software","llama.cpp","C++ LLM inference engine"] **lags behind new releases** - Or requires **very recent commits / forks** - Or sometimes **isn't merged at all yet** Your log confirms: ``` general.architecture = gemma4 ``` …but your binary only supports older architectures (like `llama`, `mistral`, `gemma`, etc.). --- ## How to fix it (choose one) ### Option 1 — Update llama.cpp (most likely fix) You probably just need the latest build. ```bash cd llama.cpp git pull make clean make -j ``` If you're on Metal (you are, M4 Pro), rebuild with: ```bash LLAMA_METAL=1 make -j ``` Then try again. --- ### Option 2 — You’re ahead of upstream (very possible) If this is a **bleeding-edge / custom model** (and it looks like it is: *“Gemma-4-E4B-Uncensored…”*): 👉 There may be **no official support yet** In that case: - Check where you got the model - Look for: - a **custom fork of llama.cpp** - or instructions like: > "Requires special branch" --- ### Option 3 — Use a supported model (quick sanity check) Try a known working one: - Gemma 2 - LLaMA 3 - Mistral If those load → your environment is fine, and the issue is 100% model compatibility. --- ### Option 4 — Convert / downgrade (harder) If you’re determined: - You’d need tooling that converts `gemma4` → a supported arch - This is **non-trivial and often impossible cleanly** --- ## What I would do (practical path) Given your setup and goals (local agents, orchestration, etc.): 1. **Verify your llama.cpp commit** 2. If still broken: - Drop this model for now - Use: - Gemma 2 (stable) - or a strong 7B–13B LLaMA variant 3. Come back to Gemma 4 when tooling catches up --- ## One subtle clue in your logs This line: ``` quantize.imatrix.file = /media/hauhaucs/... ``` This model was likely: - Quantized in a **custom environment** - Possibly using a **non-standard pipeline** That increases the chance it requires a matching runtime. --- ## If you want, I can go deeper I can: - Check if Gemma 4 support exists in a specific branch - Find you a working model that matches your hardware perfectly - Or help you wire this into your agent framework cleanly Just tell me 👍