I live in Korea but still don’t speak the language. I get a lot of SMS in Korean, 95% are spam but the last 5% are important ones. I already missed to pay my phone bull twice for months because I didn’t realize that the credit card I put there was not valid anymore and they kept sending me SMS about it which I ignored because most of the SMS is spam and copy and pasting everyone into Google Translate is quite a lot of work which is tidious and I just don’t do it.
So my idea was to take a open source SMS app like Fossify Messages and add automatic translation to it. And especially because SMS is used for security relevant stuff like 2 factor authentication, I really need the translation to work locally and not on a 3rd party server.
On my PC I have a really really good model the Aya:8b which fits well into the 12 GB VRAM on my RTX3060 and the results Korean -> English are outstanding!
But when I put it on the phone -I have a Samsung S24 Ultra - it fills up the RAM and get’s killed quite quickly. I tried to configure it so it’s allowed to use more ram for a longer period of time, etc. but even then it’s extremely slow and translates like 3 SMS in an hour and I have about 5000 in the database (I only translate the Korean ones).
I tried some other models like Gemma 3 and NLLB-200 which just output garbage, especially the later dropped numbers , URL, codes which are important in SMS translations.
Anyway, does someone have any tips what I could do?
Firefox includes/supports translation models, IIRC on mobile as well. They’re pretty small and probably good.
Another place I would check is f-droid - serving OSS apps so can be inspected. I thought I had a translator app installed like that, but it doesn’t seem to use models. I probably mixed it up with a different app that uses models for voice to text.