I am currently looking for a model that can run on my phone, it could be <8b or even <4b. It should have a reduced positivity/yes-man bias. I am at a point in my language learning journey where it’s more effective to learn a language through trying to actually construct a sentence (which is often through interaction) instead of just reading. Since there are times I am offline, a local LLM that is competent at multiple languages and decent at simulating characters texting would be a great help.

  • SuspiciousCarrot78@aussie.zoneEnglish
    1·
    8 hours ago

    I actually (just last night) abliterated a Qwen3.5-2B for this sort of purpose (well, more specifically, to fit neatly into a socket for a project). It’s fast and light, cooked for edge devices, and should have inherited all of base Qwen’s tricks (~200 languages, vision etc) polaris-heretic-Q4_K_M-GGUF

    Try it and see if it works? I inadvertently made it really fucking love dotpoints (GPT-OSS 20B disease) so am trying to unfuck it right now.

    Else - I can recommend something like Granite-4H or the old Qwen3-4B 2507 instruct

    granite-4.1-3b-heretic.i1-Q4_K_M

    Qwen3-4B 2507 instruct