• 0 Posts
  • 23 Comments
Joined 10 months ago
cake
Cake day: November 10th, 2025

help-circle





  • The “not A but Z” thing and variations thereof was pretty common before LLMs. The noticeable thing about their usage of it is they’re trying to use language meant to take the reader from something they might genuinely be confused about to a surprising conclusion, and using it in a way that’s entirely banal. It relies on distance between A and Z, and genuine possibly of either. Humans tend to have way better intuition about what is surprising to other humans, and don’t make insane mistakes like LLMs do.

    Rather than have “Z” be self-evidently interesting, the LLM need to tell us that it’s not “A”. Except no one thought anything was “A” in the first place, and the “Z” is barely a “B” let alone a “Z”.

    This also goes for couplings of three short descriptors (“Simple. Intuitive. Seamless.”) and summations (“the important part to realise is:”), bullet point lists, etc. All techniques to say: here’s the important part, here’s the bit you should listen to.

    This all being said: the tweet smells like ai to me. Wtf does he mean by sword



  • This is literally one of the most famous essays in AI (it has it’s own Wikipedia article) and I mentioned it by name but sure here you go: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf

    As for more recent stuff, people are doing experiments on it all the time, here’s Nvidia very recently trying to figure out the best mix of training data (how much task-specific training, how much general-knowledge training for optimal results) https://arxiv.org/html/2606.24747

    Google’s 2022 “A generalist agent” - 1091 citations https://arxiv.org/abs/2205.06175

    The entire field is built on statistics. It has many flaws, but this is like, the entire thing people are working on actively. Their goals may not be compatible with the flourishing of humanity, but finding the best way to automate various tasks is their one goal.

    Even from gpt-1 people have been trying to make fine-tuned models for specific tasks and they keep failing compared to general models.

    Elements of this idea do live on though, in MoE architectures, where they take a base model with knowledge of everything, then fine tune various versions of it for different things, and route your request to one of the models fine tuned for your task. This is mainly a workaround for the fact a large model with all parameters doesn’t fit in memory so easily even in the massive Nvidia datacenter gpus, if it did, we can be pretty sure it would beat the smaller “experts” in most of the tasks

    Also like, china isn’t doing different to this? Deepseek (China) and glm5.2 (China) and mistral (France) and various other models are doing the same thing, because that’s the thing that works (for the narrow definition of ai success that tech companies and politicians believe in)


  • Rugnjr@lemmy.blahaj.zonetoFuck AI@lemmy.worldWhat are we doing to stop AI?
    link
    fedilink
    arrow-up
    1
    arrow-down
    6
    ·
    edit-2
    3 months ago

    Have you ever heard of the bitter lesson? Your suggestion has repeatedly been shown not to work. People want it to work so bad but the data doesn’t bear it out: generalist systems beat specialists time and time again.

    This doesn’t make ai good for society, but specialist systems isn’t the right path if what you want is things to be able to do hard tasks




  • Idk, pretty much all the deaths I hear about are open water divers who decided to go into a cave, ignoring the part of their training where someone shouts at them for an hour to never even think about going in a cave even a teeny little bit, no not even just the entrance, no not even just to look.

    This includes people going in to recover bodies. You’ll notice the people dying doing that are often military divers who’ve never been trained for cave diving (true of the recent Maldives incident and also the diver in the Thai cave rescue). I don’t say this to be like oh training is some magical panacea but rather that you specifically really should not go in a cave even if you really want to without getting cave trained first. It’s not even remotely the same thing as open water diving.

    Oh and yeah it’s definitely cave diving that’s the insane one. Never done it myself and never will but it’s an interest of mine because of all the procedure, backups of backups, redundancy etc needed to make it safe. It’s more like spacewalking in that regard.


  • Rugnjr@lemmy.blahaj.zonetome_irl@lemmy.worldme_irl
    link
    fedilink
    arrow-up
    1
    ·
    edit-2
    3 months ago

    Not liking it isn’t the same thing as not doing it, or even not knowing how to do it well. It’s a necessary evil, one of many things in life like having to do laundry, or waking up earlier than you’d like to. Sure there are some people who don’t do those things either (and fall out of society), but there are few who deeply enjoy them. So too with small talk, it’s for colleagues and acquaintances. The way someone becomes my friend is immediately when I realise that they are open to deeper talk