Just made slop like one of us?

Not really, no. Just trying to add some egg carton to the echo chamber.

Any sufficiently disparate opinion is indistinguishable from rage bait

- Abraham Lincoln

If you say so.

I don’t see that at all, but the next few months will tell, I think

He is arguing that if we weren’t in the middle of it, we’d have to agree that the current trajectory looks like we would have expected the initial period of the singularity to look. The functional limitations of LLMs have been touted by those other people for years now, but the capabilites have kept accelerating.

Personally, I hope against hope that the current mood in SF can put the brakes on for long enough to see where we’re going. Even better would be a real capabilities plateau, but I’m not holding my breath.

If we assume he’s talking to his peers, it is a brilliant argument, since this is a thing people in CS have been discussing since, well, Turing. It is very hard to believe that someone who spent any time at all thinking about this 20 years ago would disagree.

After postdoctorates at the Institute for Advanced Study and the University of Waterloo, he took a faculty position at MIT in 2007.[4] His primary area of research is quantum computing and computational complexity theory more generally.

In the summer of 2016 he moved from MIT to the University of Texas at Austin as David J. Bruton Jr. Centennial Professor of Computer Sciences #2 and as the founding director of UT Austin’s new Quantum Information Center.[7] In summer 2022 he announced he would be working for a year at OpenAI on theoretical foundations of AI safety.[8][9] He worked at the company for two years.[10]

You should shoot him an email and let him know that he’s not qualified to write about this

sTocHastIC ParRoT

“If oil rigs were profitable, the shipyard wouldn’t sell you one”

I despise Musk as much as the next Group 14-based lifeform, but there is such a thing as specialisation.

The marketing dept, obviously

Kaiser is one of the coauthors of Attention is all you need, the paper that introduced the Transformer architecture, the basis for all major LLMs.

3b1b has a writeup: https://www.3blue1brown.com/lessons/attention/

For example, imagine that the text we input was most of an entire mystery novel, all the way up to a point near the end, which reads:

Therefore the murderer was…

If the model is going to accurately predict the next word, that final vector in the sequence which began its life simply embedding the word was will have to have been updated by all of the attention blocks to represent much more than the individual word.

It will have to have somehow encoded all of the information from the full context window that’s relevant to predicting the next word

Attention is the mechanism that lets an LLM use the (correct parts of the) entire context to predict the next word.

While superficially correct, your description of agentic work overemphasises the error bit of the trial and error. The newest models have been trained on this loop and the tool use involved, the failure rate of tool calls in a given run is low, and the corrections precise. The final outcome is valid and useful in the vast majority of cases.

This article outlines the co-development of harnesses and models, if you’re at all interested: https://www.latent.space/p/attention-interface

Good points as well. I guess my own view is coloured by having access to models that I find actually useful in my work. If my experience was only grating Claude prose and soulless AI “art” I’m not sure the tech itself would appeal all that much.

Probably that also blinds me a bit to what you argue, but I agree that the reasoning is sound from a point of view where all AI is useless. I’m just not sure that other areas won’t have the same OMG moment that coding had earlier this year.

A year ago, that was my experience coding with AI as well. Sometime this spring that changed, especially when using coding agents, and lately (as I’ve stated elsewhere) the quality is on average pretty good. And contrary to what you’re implying, I’m not that easy to impress…

If there’s anything I hope you take from this exchange, it’s that the capabilities of AI shouldn’t be a part of your arguments against the current SV mania. The concentration of power, the disregard for communities and the environment, the stated goals of replacing human labour, all of that (and a lot more!) is enough, but it is what surrounds the technology itself. That technology is advancing, maybe feeding on itself, so an attack based on what it can do now can become outdated (and I’d argue that some of yours already are).

So vaporware to support other vaporware? Metavapor? Plasmaware?

That might very well be, and would probably be the best outcome we can hope for. I guess we’ll see the shape of the curve in the next year-ish - if recursive self-improvement hasn’t come into play for real by then, I can’t see how the current investment level is defensible (but I’m not sure I see that now, so who knows)

Agree, in that case the development would probably be more incremental, focusing on what can be improved without hundreds of thousands of GPUs available, and moving inference maybe to a local-first setting. There would be no promise of 1000x profits from that, so maybe we’d get a more managable pace.

Thank you too. I have the same feeling, just the other way around - lemmy seems to have little patience for even slight positivity towards AI and LLMs in particular. Having an actual discussion is refreshing.

Regarding the profit motive, I concur, at least for the US side. I’m less certain about China, but I’m not very knowledgable there, so maybe the same mechanisms are in effect.

I don’t think “computer follows instruction” is the right angle to look at this from. The instructions that the literal computer followed were a ton of matrix multiplication operations. The consequences of that arithmetic is easier to analyze as the emergent behaviour of the “gestalt” that produces the words that calls the tools etc. (This is also the reason that dismissing the entire field as “stochastic parrots” and “spicy autocomplete” misses the mark - if you want to predict the next word all the way through a counterexample to the Jacobian Conjecture, it’s hard to see how that can be done without a - for lack of a better word - mental model of the problem)

If you do any coding at all, I encourage you to look at what the latest models output. The average quality of work from a frontier model is amazing. Yes, there are bugs, but with adversarial auto-review it’s absolutely on par with a journeyman human programmer. The problem is of course that if you don’t hire junior programmers and let them do that work, you’ll never get new experts, and that’s a clear worry.

My point with the national security angle was that if you extrapolate just a little bit from current capabilities, you get to a point where an “AI gap” is a problem, regardless of the techbro claims. Keeping a close eye on that is firmly within the responsibility of a national government. Personally, I don’t see any good outcomes from an AI race like that, unless we actually hit a hard ceiling on further expansion. Fingers crossed.

I’m not at all confident that they’re hitting a ceiling yet, and I suspect one’s outlook on that depends on how the information bubble you’re in is shaped. I concede that mine is influenced by my interest in the underlying technology.

I do believe that even if the bubble popped right now, and the current models are the best we’ll get for the next ten years, that would be enough to have dramatic consequences (aside from the econuclear fallout from the crash, that is).