Seems to make sense for maximum privacy. Put together a large enough model to answer health queries, have OCR and image recognition to read exams, give it web access to search for medication details and, of course, gather raw data from any devices you use to measure weight, heart rate, etc.

But that’s just in theory and we all know things are hard to put together. In practice, have you had any experience getting anything like this working locally?

  • hendrik@palaver.p3x.deEnglish
    3·
    5 hours ago

    I feel you can’t just dump in the CSV values from your Xiaomi Scale and Garmin watch… And hope AI will figure out the correct math on your body… And then also come up with good recommendations.

    As far as I know, there are a few local, selfhosted health trackers available. It’s a bit tricky to own the correct gadgets that connect to it… But I don’t think there’s anything with AI.

    I mean to give proper recommendations, you’d need a very elaborate setup. It needs all the sensor values. Then correlate it with what you’re doing all day long… What you eat and how much you drink… The AI (or traditional algorithms) can’t see. So maybe it can calculate your BMI in a thinking step. But it’s a whole lot of math to then figure out if your too fat, or have muscle mass… And then find out what that means for your diet. AI won’t figure that out along the way. So you’re probably looking at a few thousands of lines of code, after reading a few textbooks on biology.

    I mean you can try to vibe-code some agent. But I think your best bet is to look for some open-source software cloning Google Health, or something like that. (And then maybe you can write some MCP server for that. And an agent to interpret the aggregated results.)

    • rkd@sh.itjust.worksOPEnglish
      1·
      14 minutes ago

      For sure, context rot is a problem, but that’s also the easiest thing to control for in this case. If sensor data is relevant to you, having some code to process and reduce it to a dashboard you can read is always a good idea, independently of getting an LLM into the loop.

      This becomes more complicated with data you can’t really understand like results from blood tests, for example. But maybe you just don’t summarize any of that.