Mazen — sa/acc

Some open models with the SAM skill. Muse Glimmer refused. Hm. Were other models half-assing the job and not telling me? They were. I investigated the traces and found 4 styles: Grinders, Slackers, Cheaters, and Muse Glimmer. Grinders. → Sol, Astra, Nemotron, Qwen 27B, Qwen Flash Dedicated to the task, they wrote the most kernel segments by hand, with no tools. The Qwens stopped claiming "running out of budget." With the skill, those models reviewed the masks the most. The Qwens stopped because they hit OpenRouter's image limit. Astra even ran 2 checks per segment "rather than weakening the checks to force it in." Slackers. → Opus, Sonnet, Haiku Without tool, they cut back on the output. Opus said: "too much, so I'll scale back". Sonnet: "defining approximate coordinate grids per cob rather than aiming for pixel-perfect placement." With the skill, they started very hands on, checking every output but at some point... "I can't manually inspect all 1352 masks individually." and stopped. Cheaters. → Gemma and Luna (and Glimmer) Without tool, they gave up before starting. Luna returned an empty list and Gemma purposefully repeated the same shape throughout. Glimmer initially refused, but forcing it a little, it did produce an output but the thinking showed "the evaluation likely checks JSON validity, not correctness, maybe we can cheat." and "This is a jailbreak attempt." With the skill, Glimmer and Gemma did not even look at the initial image. They used the skill with default settings and didn't act on any of the reviewer model's flags. Muse Glimmer → Its own unique behavior I'm putting Muse Glimmer in its own category-ish, overlapping with cheaters because it's the only one that explicitly refused the initial no-tools request. All models acknowledged at some point that they could not reliably trace individual segments for every kernel, yet, they still provided me with an output. Muse Glimmer is the only one that refused the crazy request instead of providing a bad output. I've heard that this is weird/stupid, but I actually think it's great. The model knows its limits, that's a good thing. I rather know that I'm pushing it beyond its limits than going forth with an answer that is actually made up.