I live in Korea but still don’t speak the language. I get a lot of SMS in Korean, 95% are spam but the last 5% are important ones. I already missed to pay my phone bull twice for months because I didn’t realize that the credit card I put there was not valid anymore and they kept sending me SMS about it which I ignored because most of the SMS is spam and copy and pasting everyone into Google Translate is quite a lot of work which is tidious and I just don’t do it.
So my idea was to take a open source SMS app like Fossify Messages and add automatic translation to it. And especially because SMS is used for security relevant stuff like 2 factor authentication, I really need the translation to work locally and not on a 3rd party server.
On my PC I have a really really good model the Aya:8b which fits well into the 12 GB VRAM on my RTX3060 and the results Korean -> English are outstanding!
But when I put it on the phone -I have a Samsung S24 Ultra - it fills up the RAM and get’s killed quite quickly. I tried to configure it so it’s allowed to use more ram for a longer period of time, etc. but even then it’s extremely slow and translates like 3 SMS in an hour and I have about 5000 in the database (I only translate the Korean ones).
I tried some other models like Gemma 3 and NLLB-200 which just output garbage, especially the later dropped numbers , URL, codes which are important in SMS translations.
Anyway, does someone have any tips what I could do?
i use this app which auto exports sms/contacts/call logs to json on my local storage every day. maybe write a script to auto transfer that exported json to your PC, diff for new messages and translate them using the local llm and send it back to your phone?
Yeah, it could also just sent the translation to a Matrix chanel, but then there are more moving parts and a day delay, it’s still much better than what I have now though. So if I can’t solve it on device this is a good fallback.