I live in Korea but still don’t speak the language. I get a lot of SMS in Korean, 95% are spam but the last 5% are important ones. I already missed to pay my phone bull twice for months because I didn’t realize that the credit card I put there was not valid anymore and they kept sending me SMS about it which I ignored because most of the SMS is spam and copy and pasting everyone into Google Translate is quite a lot of work which is tidious and I just don’t do it.
So my idea was to take a open source SMS app like Fossify Messages and add automatic translation to it. And especially because SMS is used for security relevant stuff like 2 factor authentication, I really need the translation to work locally and not on a 3rd party server.
On my PC I have a really really good model the Aya:8b which fits well into the 12 GB VRAM on my RTX3060 and the results Korean -> English are outstanding!
But when I put it on the phone -I have a Samsung S24 Ultra - it fills up the RAM and get’s killed quite quickly. I tried to configure it so it’s allowed to use more ram for a longer period of time, etc. but even then it’s extremely slow and translates like 3 SMS in an hour and I have about 5000 in the database (I only translate the Korean ones).
I tried some other models like Gemma 3 and NLLB-200 which just output garbage, especially the later dropped numbers , URL, codes which are important in SMS translations.
Anyway, does someone have any tips what I could do?
Ah cool, perhaps I can figure out what model the’re using because it’s open source, thanks for sharing!
God damn this app is amazing! I’ve never seen such speed in translation like the SMS text I paste in is translated within less than half a second! And the translation quality is extremely good!
On top of all those good things it even exposes a android service so other apps can send in a string and it translates it for them and sends it back for them to use asynchronously, holly shit!
With this speed I can remove all the background translation code and all the storage and database, etc and can just translate the SMS when it becomes visible!
I will try to integrate it. Perhaps it is so little code that it’d be worth to upstream it to Fossify if it works well.