I keep a lot of notes in markdown files, and I’d like an LLM to assist.
I regularly use Open WebUI with with inference routed through huggingface. Open WebUI kind of has this functionality like you can upload a markdown file and prompt it to improve it in whatever way, but of course that’s a fairly clunky workflow.
I really want something built into the editor, that can use RAG to consider other files in context.
I also don’t want to be locked in to a specific LLM or provider, I’d like to be able to link it to OpenRouter or similar.
This sounds really cool. I hadn’t heard of a vector embedder / vector store before. Definitely need to look into those.
Do you have a big GPU to run local ollama ?
So I do inference over api to open routers I wish I had the GPU or an apple to run a decent LLM locally. Embedding is very cheap comparatively, I use a clip embedding model so I can have images and text in the same vectorspace.