• 2 Posts
  • 9 Comments
Joined 1 year ago
cake
Cake day: June 29th, 2023

help-circle
  • Piracy. I’d buy albums if I had money, though. I’ll slowly phase into getting them once I get some more cash.

    I can find most stuff I listen to, and I rarely grow my music library. I mostly listen to 20-30 albums, with some more mainstream music peppered in.

    My music library currently sits at 90 gigabytes (mostly flacs), so quite small compared to others I’ve seen around here. Still, I have plenty of variation to keep me entertained :D

    If you have Tidal, aren’t there some apps to rip the lossless audio from there? You could get most of the stuff that you need, and then cancel the subscription. If you feel bad, maybe order some merch from the band, haha.


  • You’re right. I read past the “I want to learn ML” and went straight to “do something useful with the data”.

    If the goal is to understand how modern LLMs work, it’s also good to read up on RNNs and LSTMs. For this, 3Blue1Brown does an amazing job, and even posted an in-depth video about transformers. I’d watch that next, followed by implementing a simple transformer in PyTorch (perhaps using the existing blocks).

    You could argue that it’s important to design everything from scratch first, but it’s easier to first go high level, see how the network behaves, and then attempt to implement it yourself based on the paper. It is up to OP how comfortable he is with the topic though 😁


  • Depending on how much compute you have available, you can look into finetuning models from HuggingFace (e.g. Llama 3, or a smaller Phi model). Look into LoRA, and try to learn how the model you choose calculates the loss.

    There are various ways to train, and usually involves masking the input by replacing random input tokens with the mask token. I won’t go into too much detail with this, because it’s a lot to explain, and I suggest you read an article on this (link1 or link2)



  • The Framework 13 inch model should be plenty, especially if you want to dev on the go. Much more lightweight and smaller, and you can connect it to external monitors if the screen size is not big enough. Also, you shouldn’t have issues running Linux on either laptops.

    Instead of going for the 16 version, I would use the extra 900-1000 euros (that’s the amount I saw I could save between the two almost maxed-out models) to make a dedicated server or mini-cluster to run your workloads. Deploy Kubernetes or Proxmox on it, and you’ll also get some more practice on it outside work if you want to run stuff for your home lab. That is only if you don’t want to game on your laptop, but I’d still put that money aside to make a desktop.