• 1 Post
  • 70 Comments
Joined 3 years ago
cake
Cake day: June 13th, 2023

help-circle




  • Surely you jest because it’s so clearly not if you understand how LLMs work (at the core it’s a statistic model - and therefore all approximation to a varying degree).

    But great can come out of this case if it gets far enough.

    Imagine the ilk of OpenAI, Google, Anthropic, XAI, etc. being forced to admit that an LLM can’t actually do anything but generate approximations of language. That these models (again LLMs in particular) produce approximations of language that are so good they’re often indistinguishable from the versions our brains approximate.

    But at the core they cannot produce facts because the way they are made includes artificially injected randomness layered on-top of mathematically encoded values that merely get expressed as tiny pieces of language (tokens) - ones that happen to be close to each other in a massively multidimensional vector space.

    TLDR - they’d be forced to admit the emperor has no clothes and that’s a win for everyone (except maybe this one guy).

    Also it’s worth noting I use LLMs for work almost daily and have studied them quite a bit. I’m not a hater on the tech. Only the capitalists trying to force it down everyone’s throat in such a way that we blindly adopt it for everything.









  • It depends on many factors including:

    • visits of individual sites
    • requirements of each site (memory, I/O, persistent storage, ephemeral storage, caching, databases, etc.)

    So you’re right that you make an initial guess and go from there.

    Many tools/sites/projects will have minimum system requirements and you can get an idea of minimums using those stats. Some frameworks might even have guidelines available. The one I use most often for example has a configurable memory footprint. So that’s a datapoint I personally use.

    If they’re all the same type of site (example Ghost blogs) using the same setups then it’s often less intense since you can pool resources like DBs and caching layers and go below minimum system requirements (which for many sites include a DB as part of the requirements).

    Some sites might be higher traffic but use fewer resources, others might be the inverse.

    Then there’s also availability. Are these sites for you? Is this for business? What kind of uptime guarantee do you need? How do you want to monitor that uptime and react to needs as they arrive/occur?

    The best way to handle this is in a modern context also depends on how much and what style of ops you want to engage in.

    Auto-scaling on an orchestration platform (something like K8S) or cloud-provider auto-scaling of VMs or something else? Do you want deployments managed as-code via version control? Or will this be more “click Ops”. No judgement here just a thing that will determine which options are best for you. I do strongly recommend on some kind of codified, automated ops workflow - especially if it’s 25 sites, but even with just a handful. The initial investment will pay for itself very quickly when you need to make changes and are relived to have a blueprint of where you are.

    If you want to set it and forget it there are many options but all require some significant initial configuration.

    If you’re ok with maintenance, then start with a small instance and some monitoring and go from there.

    During setup and staging/testing the worst that can happen is your server runs out of resources and you increase its available resources through whatever method your provider offers. This is where as-code workflows really shine - you can rebuild the whole thing with a few edits and push to version control. The inverse is also true - you can start a bit big and scale down.

    Again, finding what works for you is worth some investment (and by works I don’t just mean what runs, but what keeps you sane when things go wrong or need changing).

    Even load testing, which you mentioned, is hard to get right and can be challenging to instrument and implement in a way that matches real-world traffic. It’s worth doing for sites that are struggling under load, but it’s not something I’d necessarily suggest starting with. I could be wrong here but I’ve worked for some software firms with huge user bases and you’d be surprised how little load testing is done out there.

    Either way it sounds like a fun challenge with lots of opportunities for learning new tricks if you’re up for it.

    One thing I recommend avoiding is solutions that induce vendor lock-in - for example use OpenTofu in lieu of something like CloudFormation. If you decide to use something like that in a SaaS platform - try not to rely on the pieces of the puzzle that make it hard (sticky) to switch. Pay for tools that bring you value and save time for sure, but balance that with your ability to change course reasonably quickly if you need to.



  • Never used your project but don’t let this thread get you down.

    Clearly OP loves it - don’t let those who don’t know it or don’t like it be the voices that ring loudest in your ears even if they hurt the most.

    I worked professionally in open source at a company with lots of funding. The tools I worked on were used by millions and millions.

    Every negative comment hurt so much. Every angry user I wanted to talk to. Most of them wanted to TALK AT me. It all hurt. And I was being paid. The engineers on my teams were burnt by the community time and time again.

    If you love what you’re doing and you have a growing or happy audience - stay the course. Listen to criticism, decide if you agree (and maybe take some time when it hurts because the criticism might be valid), make a decision and move on.

    Also, and this is going to be tough, maybe think about expanding or modifying what you mean when you say making Lemmy accessible for everyone.

    Do you mean making a UI that will become the majority default or making a UI that brings some features (or perspective) for users who see value in those features? Trying to make something for everyone in a pond as small as the fediverse, where there are already a plethora of options is a big lift.

    Above all, do you. And that includes this comment which I encourage you to promptly ignore. ;)


  • Apple’s MacBook Pro includes HDMI and a third usb/Thunderbolt port alongside an SDXC and headphone jack (the latter of which is on all their laptops albeit on the other side). This seems like the perfect balance for most users.

    It’s nonsense they don’t include HDMI on the Air, but then “it’s kinda thin and kinda light”.

    I was not sad to see FireWire and mini-DisplayPort replaced with usb-c/thunderbolt.

    Current port line up on “pro” machines: