Repple (she/her)

Honestly the 2008 real estate/subprime bubble felt the same way. People were talking about the bubble popping for what seemed like forever until it actually did.

Well, if you consider programming an art, it’s fucking destroying art.

I tried switching to ddg several times over the last decade or so and always gave up because of issues like this, especially with technical or obscure topics. Recently started paying for Kagi and do not have the same issues at all there. The results are good and nicely presented without ads. Of course, on the flip side it’s pretty expensive.

Maybe it’s time to open up a hipster bespoke ram shop. You may only get 1kb but it will be the most beautiful hand-woven core memory you’ve ever seen.

To be more specific (for anyone interested), the next word predictors are usually a type of model called an LSTM (at least I think that’s the most common). This model type has been used for a long time for dealing with sequential data. In 2014 there was a famous paper introducing an attention mechanism. This was a rather brilliant, though relatively minor extension to how LSTMs work. Essentially between each step of an LSTM it generates some data representing the model’s knowledge of the sequence to that point. The attention mechanism looks back at these intermediate values and determines how relevant each state is to the current point in the sequence and pulls in the most relevant bits. This vastly improved the memory of the LSTM over longer sequences.

In 2017 there was another famous paper “attention is all you need” which said something to the effect of “the attention mechanism is doing all the work, we don’t need the rest of the LSTM we can replace it by running attention between all point combinations in the sequence.” It’s actually significantly slower to run as the model grows, but much much faster to train because it’s not intrinsically sequential. This is the transformer model that’s the basis of all our LLMs.

Obviously some massive simplifications here but as despite being fairly anti AI, I do love the engineering behind it. So yeah, pretty literally a fancy text predictor, but it turns out when you throw all the compute you can muster at a fancy word predictor is makes the world go crazy

Language needs to change because it’s completely undecipherable to lay people. I use all sorts of models to great effect in my work. Random forests, LSTMs, SVMs, etc all have tons of great uses. I am pretty anti “AI” as lay people understand it (though the technology is super cool on a technical level)

On the plus side, I can much more easily convince people to use any sort of machine learning models than I used to be able to by calling it AI.

Yeah, the price kept me away for a long time. I do now find it worth it but completely get that it wouldn’t be for many. So nice to be the consumer rather than the product, though.

It’s so bad it drove me to finally pay for Kagi, which I must say is incredibly better than ddg or ecosia

The only codebase an agentic system could refactor in 15 minutes would be almost trivially small. I still couldn’t do it in 15 mins, but give me a couple hours and I’ll make much more meaningful improvements

🤢

Fuck, tears of the rich sounds like icing on the $6000/hr cake. Though they would probably expect comfort from me rather than laughter and mockery…

I’m a woman engineer and for that kind of money might be able to stomach this (not that I’m anywhere near attractive enough for them to hire me), but I’m pretty sure if I were attractive enough and got hired they would probably only last a few minutes before I completely emasculated them by actually knowing what the fuck I’m talking about.

Sounds pretty fun, actually, now that I think about it… as long as I didn’t need to fuck them

Yeah I tried switching to ddg and ecosia several times and had the same experience, especially for technical searches. I’m trying Kagi right now and it seems better, but haven’t done too many technical ones yet, so we’ll see on that front

This is what happens when increased velocity is demanded. It takes at least as long to verify that the code works well as it would have for me to write and verify. I can pump out a ton of code with an LLM, but if you want quality it’ll take me just as long as 2 years ago, probably longer because of my decreased practice lately

Yeah it really feels like an LLM should work better than a phone tree for that, but every time I actually encounter one it’s so so much worse.

If it’s concentrating solar, those things are literally an eyesore. Shit’s bright.

4.6 Opus was a huge jump from earlier models and the first that was actually useful for things like this from my experience (and 4.7 is significantly worse for some reason).

I have made many anti-LLM posts here and I remain pretty negative on them, but they have absolutely become useful. Part of the problem is the truth is really somewhere between the insane promises and the dismissals.

My problems are many fold though, from being propped up by insane subsidies, the massive power usage to the thing I most care about: taking more power from the masses. The more useful they get, the more power gets concentrated to those able to afford the data centers.

Computers used to be at least somewhat democratizing, sure there were some things like weather modeling that an ordinary person couldn’t do, but a random person on thier computer could put something together to change the world.

What happens when the breakthroughs are available only for the wealthiest? Regular folks can buy tokens at a reasonable price today, but running cutting edge models on consumer hardware isn’t really feasible. We’ve ceded too much control.