• 2 Posts
  • 426 Comments
Joined 2 years ago
cake
Cake day: February 5th, 2025

help-circle


  • If AI is so smart,

    First off, the label AI is bad - it’s Artificial, but Intelligent - or smart - it is not.

    LLM - Large Language Model + scaffolding. They match patterns. They build an unimaginably complex “context window” of 200,000 tokens and from that they synthesize their responses to prompts of usually a few dozen tokens based on trillions of model weights trained on hundreds of trillions of tokens. Optical character recognition has been doing this to automatically “read” the routing numbers printed on the bottom of checks since the early 1960s, that old OCR just has much much lower dimensions of patterns to match.

    The “Mechanical Turk, one step removed” is helping to tune those trillions of model weights by reinforcing when it “gets things right” and flagging when it “gets things wrong.” The LLM has to find this out somehow, that’s what the employees are doing - training it, teaching it right from wrong.

    AlphaZero taught itself to play chess, Go, and other relatively simple games nearly 9 years ago now - it mastered those with “best next move to win” pattern analysis / generation all by itself because the rules of those games are crisp, well defined, simple. What makes “a good response” in a conversation is a hugely different animal with orders of magnitude more dimensions and levels of nuance.

    After all this training, the next generation model should, statistically, generate “correct” conversational responses more often, according to what the inputs of the employees consider and tell the model is correct and incorrect for a given situation.



  • I worked in industry for 12 years at market rates after getting my MS degree. In years 10-12 we hired post-docs to do some fancy lab work and publish findings - I could do that work but I was focused on other concerns. Welp, thanks to 9-11 those other concerns lost their funding, but the lab work continued. I inquired about possibly serving as the next lab-tech… those post docs were being paid less than 1/3 what I was making at industry rates. They were still getting around mostly on their bicycles, catching rides with other people when needed, and they had pretty much as many years of study and lab experience after getting their MS as I did industry experience, they just chose to continue on the PhD and Post-doctoral track for some reason… maybe they really like bicycles?


  • In the 80s it was “market rate.” In engineering my TA position paid $14k a year + free tuition. The money was a little less than half of what I could earn in industry at the time, but free tuition (and an actual job offer instead of an open-ended job hunt) made it attractive enough to keep me around for a Master’s degree. They offered me a PhD, with a raise to $18k/yr for the TA-ship, again right at 1/2 of what my MS was worth on the open market - I had had enough of academia, no thanks. Only took 6 weeks to find a “real” job.

    Over in the business school, they had a lot more students competing for the TA positions, over there they paid $4k per year + free tuition. Seems that more business students come from families where their expenses continue to be paid by family through 6+ years of college.

    I co-taught a lab with one other TA vs about 15 students, and we helped grade papers for the lecture class… could we have been replaced with AI? Maybe, but would AI stand over your shoulder and ask you “what happens when I pull this wire out of your breadboard?” then talk you through the reasons why your quick and dirty design may work in the lab, but it doesn’t meet design specifications and will likely fail in real-world use where it’s used millions of times a minute for years instead of 3 or 4 times for a bench test - pulling that wire is the same as your design being completely incorrect, and yet: your bench tests are all still passing… think about that.





  • I was going more for the absurdity of a single person emitting more CO2 than a township of 150,000 people, plus the underutilization / hoarding of this imaginary construct: money by the people who have more than they need - as you allude to.

    I also agree: the price of things does frequently correlate with CO2 emitted. Even when you hire someone for a service that emits virtually no CO2, what do they do with that money? Turn around and spend it on things that do emit CO2. Much of the cost of goods has to do with the cost of the energy required to produce and deliver the good (again cost correlating with CO2 emissions).

    The only “industry” I can think of that moves significant money around without generating CO2 directly is finance, though when the money quits running through finance’s internal spin-cycle and finally hits the real world, it’s again purchasing the relase of CO2.






  • The US has been cutting per capita emissions for 20 years, while China is still increasing.

    But they haven’t caught US yet!

    United States: The average American emits roughly 14 to 15 metric tons of carbon dioxide per year.

    China: The average Chinese citizen emits about 8.7 to 9.2 metric tons of carbon dioxide per year.

    Next question: what’s the average CO2 emission per capita of Billionaires?

    A billionaire’s total average carbon footprint is roughly 1.9 million to 3 million metric tons of (CO_{2}) equivalent ((CO_{2}e)) per year.

    So, Billionaire’s income ratio is around 500,000:1 of an average US citizen, but their CO2 emissions are only about 170,000x - that makes Billionaires almost 3x greener with their greenbacks than an SUV commuting wage slave.



  • Yeah, improving on 2024 performance was a pretty low bar to clear, by mid-2025 I saw the continuing improvement and decided that even if it was marginal at the time, learning how to use it was probably worthwhile given the improvements that seemed to be coming. Those improvements definitely did come from my perspective. Much of it was in the harnesses - many things I used to have to tell models explicitly, repeatedly in fall of 2025 they started doing without explicit prompting by spring of 2026. I also think I learned what they could be expected to do well and what was a waste of time trying which made me more productive with them as well.

    I recently heard that Kimi v3 has made significant progress in code quality - with some people calling it “on par” with Claude. I primarily use Claude Opus - when I tried Fable during their free preview it “felt” even better, but not enough better to shell out a lot of extra cash for personal playtime projects. Kimi isn’t as accessible under the fixed monthly price model so I’m unlikely to try it anytime soon.