Off-and-on trying out an account over at @tal@oleo.cafe due to scraping bots bogging down lemmy.today to the point of near-unusability.

  • 3 Posts
  • 1.01K Comments
Joined 3 years ago
cake
Cake day: October 4th, 2023

help-circle
  • Yes this works when I’m having issues.

    Okay, gotcha. In that case, my guess is that the most-likely issue is with the DNS daemon that OpenWRT is using, which is most-likely dnsmasq.

    You could confirm that your DNS configuration on the LAN hosts is actually using OpenWRT as the DNS server if you wanted — I could walk through that, if you’re not sure. But from a test-when-it’s-happening situation, I’d probably:

    • Install dig on the Mint box, if it’s not already installed. That’s a DNS diagnostic program that doesn’t use your system resolving mechanism, talks directly to DNS, so it cuts out any other factors. It’ll be in the bind9-dnsutils package.

        $ sudo apt install bind9-dnsutils
      
    • Install dig on the OpenWRT box.

        $ opkg update
        $ opkg install bind-dig
      
    • Confirm that you can successfully run it on both the OpenWrt box and your laptop when there are no problems (your router’s LAN address probably being something like 192.168.1.1):

        $ dig @your-router-LAN-address www.google.com
      
    • Confirm that you can successfully run it on both the OpenWrt box and your laptop when there are no problems using whatever outside DNS server you have the OpenWrt box set to use (assuming 1.1.1.1 here):

        $ dig @1.1.1.1 www.google.com
      
    • Next time you have problems, repeat the above. If both the dig on your OpenWrt box and on your laptop using the OpenWrt DNS server now fail, then it’s probably pretty safe to say that it’s the DNS server on the OpenWrt box at fault. If just the one on your laptop fails, then it’s going to be a communication issue between your LAN box and the OpenWrt DNS server.

    Assuming that the problem is the OpenWrt DNS server, and assuming that it’s dnsmasq, to troubleshoot further, I might try looking at its logs, or installing tcpdump on the OpenWrt box and running that, something like $ tcpdump -nvvv -i any udp port domain. That should let you see both the incoming DNS queries going to the OpenWrt DNS server as well as the queries that it is sending upstream (the -n will prevent tcpdump from itself sending DNS queries to look up IP addresses, which would generate more noise, the -vvv to show all possible information about the packets, and the -i any to show all packets on all interfaces). That’ll let you see whether it’s getting flooded with DNS queries and to see what queries it’s sending to the upstream DNS server out on the Internet.

    That won’t alone solve your problem, but it’d be the next step I’d try to narrow down what’s going on.


  • I am still having a gaming rig here with rx6900xt as well but way too big to get it wife approved into the living room and have no man cave to run it 24/7.

    It’s pretty trivial to make use of an LLM compute box remotely; in fact, most of the software out there is designed around doing this, since lots of people use cloud-based LLM compute machines. I use the Framework Desktop in this fashion — I leave it headless, just as an LLM compute node for whatever machine is running software that needs number-crunching done. So if your gaming machine is fine for you in terms of compute capability, you might want to just use it remotely from the living room with another machine being in the living room.

    Another benefit of sticking the compute box elsewhere is that while my Framework Desktop is very quiet (single large fan, about 120W TDP, and is notable for being rather quieter than other AI Max-based systems), keeping my 7900 XTX loaded will spin up the fans. You may not want to have a heavy-duty number-crunching machine in the living room from a noise standpoint.



  • I use a 128GB Framework Desktop. Back when I got it, it was $2,500 with 8TB of SSD storage, but the RAM shortage has driven prices up to substantially more. That system’s interesting in that you can tell Linux to use essentially all of the memory as video memory; it has an APU with unified memory, so the GPU can access all that memory.

    That’ll get you 70B models like llama 3-based stuff at Q6_K with 128K of context window, which is the model max. Speeds with that level of model are okay for chatbot-like operation, but you won’t want to run code generation with that.

    For some tasks, you may be better-off using a higher-bandwidth-but-less-memory video card and an MoE model; this doesn’t keep all of the model active and in video memory, only loading relevant expert models. I can’t suggest much there, as I’ve spent less time with that.

    If you don’t care about speed — you probably do — you can run just about anything with llama.cpp using the CPU and main memory, as long as you have enough memory. That might be useful if you just want to evaluate the quality of a given model’s output, if you want to get a feel for what you can get out of a given model before buying hardware.

    You might want to ask on !localllama@sh.itjust.works, as there’ll be more people familiar there (though I’m not on there myself).

    EDIT: I also have a 24GB Radeon 7900 XTX, but for LLM stuff like llama.cpp, I found the lack of memory to be too constraining. It does have higher memory bandwidth, so for models that fit, it’s faster than the Framework Desktop. In my experience, discrete GPUs were more interesting for image diffusion models like Stable Diffusion — most open-weight image diffusion models are less-memory hungry – than LLM stuff. Though if you want to do Flux v2, I wasn’t able to fit it on that card. I could run it on the Framework Desktop, but at the resolutions I wanted to run it at, the poor ol’ Framework took about 6 or 7 minutes to generate an image.

    EDIT2: I use all AMD hardware, though I agree with @anamethatisnt@sopuli.xyz that Nvidia hardware is going to be easier to get working; a lot of the AMD software is much more bleeding edge, as Nvidia got on all this earlier. That being said, Nvidia also charges a premium because of that. I understand that a DGX Spark is something of an Nvidia analog to the Framework Desktop and similar AI Max-based systems, has unified memory, but you’ll pay for it, something like $4k.


  • Okay. It’s going to be a little harder to diagnose it since the problem isn’t immediately visible, but you’ve got all the Linux toolset there, so that’s helpful.

    Is the DNS server you’re trying to use from the LAN machines running on the OpenWrt machine, or off somewhere on the Internet?

    EDIT: Or on the LAN, I guess.

    EDIT2: Oh, you answered that elsewhere.

    I am using my routers DNS, and it’s reachable from my laptop.

    Have you tried doing a DNS lookup from the router (pinging a host by name, say) when you were having the problems?

    If so and it didn’t work, that’d suggest that the problem is the upstream DNS server. If that’s the problem, as IsoKiero suggests, you might set the OpenWrt box to use a different DNS server.

    If so, and it worked, that’d suggest that the issue is the OpenWrt host’s DNS server serving names. It sounds like OpenWrt uses dnsmasq by default.

    If not…that’d probably be what I’d try next time the issue comes up.




  • Okay. So, I don’t know exactly what you’re doing to test that, but I’m going to assume, say, trying to go somewhere in a Web browser.

    First off, I have occasionally seen problems myself where consumer broadband routers that have been on for a long time wind up briefly becoming unresponsive. Probably some sort of memory leak or something. So if you haven’t rebooted the thing and seen whether all your problems magically stop showing up, I’d probably try that. Quick and easy.

    Okay. Say that doesn’t do it.

    When you confirm that the router can reach the Internet during this period of outage, how are you doing that? Going to a management Web UI from a wired-LAN device and trying to ping some host on the Internet?



  • Shortage Likely

    https://en.wikipedia.org/wiki/OpenWrt

    OpenWrt (from open wireless router) is an open-source project for embedded operating systems based on Linux, primarily used on embedded devices to route network traffic.

    OpenWrt can run on various types of devices, including CPE routers, residential gateways, smartphones, pocket computers (e.g., Ben NanoNote). It is also possible to run OpenWrt on personal computers and laptops.

    OpenWrt also recommends choosing a device with a minimum of 16 MB of flash and 128 MB of RAM, preferably higher amounts.[77]

    If you can install OpenWrt, you can probably get ahold of hardware that can run OpenWrt.

    OpenWrt provides regular bug fixes and security updates even for devices that are no longer supported by their manufacturers.

    You’ll probably also have a longer device lifetime.



  • You can probably do it, but I’m not sure how many users you’d get, as I think that it’d be a less-usable interface.

    • You’d need some way to handle voting; that doesn’t intrinsically show up. Maybe you could do it via expecting users to send specially-structured emails.

    • If by “fediverse” you specifically are talking about the Threadiverse — Lemmy, Piefed, and Mbin — then you’re going to have to also deal with a lack of a way to handle responding to a given comment (unless you intend to forward all comments to all posts that a user has subscribed to to an email address, and then just only let them respond to those).

    • Email isn’t natively encrypted, so if that’s a concern and you want to deal with that, you’d need something like a PGP key that users could register, I guess.

    • Email clients don’t, as far as I know — I haven’t gone looking — natively have Markdown support, so either you need to throw out formatting or have some sort of mapping to and from Markdown to HTML. I don’t know if something like pandoc would be sufficient for that.

    • No native “report” functionality. Maybe you could do it via expecting users to send specially-structured emails.

    If what you want is to take advantage of existing native clients, my suggestion is that you’d probably get more mileage out of doing a bidirectional Usenet-to-Threadiverse gateway than an email-to-Threadiverse gateway. That has a much closer mapping in terms of functionality than email. You could do that a lot more efficiently in terms of bandwidth. Your “Usenet group list” would be a set of community@instance name entries, and you map posts to top level messages, and comments to responses to those.

    The major downside there is that I don’t think that any Usenet clients have native Markdown support and you still don’t have voting or native reporting functionality.

    The only obvious benefit I can think of from either Usenet or email is that there are clients for both that support offline functionality, and I don’t know of any Threadiverse-native clients that do. I think the major point I’d raise would be “you could probably do it, but…what do you gain that outweighs the drawbacks?” Like, I think that you’d probably get more good out of just picking your favorite native Threadiverse client and adding code to that (or starting a new one, if you absolutely can’t stand any of the existing ones).







  • There are some memory latency benefits to putting memory on a single chip, but to date, that’s largely been handled by adding cache memory to the CPU, and later adding multiple tiers of it, rather than eliminating discrete memory.

    The first personal computer I used had 4kB of main memory.

    My current desktop has a CPU with 1MB of L1 cache, 16MB of L2 cache, 128MB of L3 cache, and then the system as a whole has 128GB of discrete main memory.

    Most of the time, the cache just does the right thing, and for software that is highly performance-sensitive, one might go use something like Valgrind’s cachegrind or something like that to profile and optimize the critical bits of software to minimize cache misses.

    I could believe that maybe, say, one could provide on-core memory that the OS could be more-aware of, say, let it have more control over the tiered storage. Maybe restructure the present system. But I’m more dubious that we’ll say “there’s no reason to have a tier of expandable, volatile storage off-CPU at all on desktops”.

    EDIT: That argument is mostly a technical one, but another, this one from a business standpoint. I expect PC builders have a pretty substantial business reason to not want to move to SoCs. Right now, PC builders can, to some degree, use price discrimination to convert consumer surplus to producer surplus. A consumer will typically pay disproportionately more for a computer with more memory, for example, when they purchase from a given vendor. If the system is instead sized at the CPU vendor, then the CPU vendor is going to do the same thing, probably more effectively, as there’s less competition in the CPU market, and it’ll be the PC builder seeing money head over to the CPU vendor — they’ll pay a premium for high-end SoCs.

    In Apple’s case, that’s not a factor, because Apple has vertically-integrated production. They make their own CPUs. Apple’s PC builder guys aren’t concerned about Apple’s CPU guys extracting money from them. But Dell or HP or suchlike don’t manufacture their own CPUs, and thus have a business incentive to maintain a modular system. Unless one thinks that the PC market as a whole is going to transition to a small number of vertically-integrated businesses that look like Apple, I guess, where you have one or two giant PC makers who basically own their supply chain, but I haven’t heard about anything like that happening.



  • Aside from them, discrete graphics cards are history, just as disk controllers were a few decades earlier. DIMM slots are going too. The primary storage will be built in. (The industry missed a great deal there.)

    Discrete disk controllers are still around.

    My last desktop had a PCI SATA card that I added after I exhausted all of the on-motherboard SATA slots.

    My current one has a JBOD SATA USB Mass Storage enclosure.