I live in Korea but still don’t speak the language. I get a lot of SMS in Korean, 95% are spam but the last 5% are important ones. I already missed to pay my phone bull twice for months because I didn’t realize that the credit card I put there was not valid anymore and they kept sending me SMS about it which I ignored because most of the SMS is spam and copy and pasting everyone into Google Translate is quite a lot of work which is tidious and I just don’t do it.
So my idea was to take a open source SMS app like Fossify Messages and add automatic translation to it. And especially because SMS is used for security relevant stuff like 2 factor authentication, I really need the translation to work locally and not on a 3rd party server.
On my PC I have a really really good model the Aya:8b which fits well into the 12 GB VRAM on my RTX3060 and the results Korean -> English are outstanding!
But when I put it on the phone -I have a Samsung S24 Ultra - it fills up the RAM and get’s killed quite quickly. I tried to configure it so it’s allowed to use more ram for a longer period of time, etc. but even then it’s extremely slow and translates like 3 SMS in an hour and I have about 5000 in the database (I only translate the Korean ones).
I tried some other models like Gemma 3 and NLLB-200 which just output garbage, especially the later dropped numbers , URL, codes which are important in SMS translations.
Anyway, does someone have any tips what I could do?
God damn this app is amazing! I’ve never seen such speed in translation like the SMS text I paste in is translated within less than half a second! And the translation quality is extremely good!
On top of all those good things it even exposes a android service so other apps can send in a string and it translates it for them and sends it back for them to use asynchronously, holly shit!
With this speed I can remove all the background translation code and all the storage and database, etc and can just translate the SMS when it becomes visible!
I will try to integrate it. Perhaps it is so little code that it’d be worth to upstream it to Fossify if it works well.