• 0 Posts
  • 114 Comments
Joined 3 years ago
cake
Cake day: June 12th, 2023

help-circle

  • In what world is a production codebase 75k lines. A single project at my last company was 1.4 million lines of code. Every single one of our microservices had more lines of code than that. And just my team was responsible for 50+ microservices. And thirteen tokens per line is hilariously low.

    Holding a week of chat in your head is nothing when 90% of it is stuff that is incorrect or discussed and minds changed. This is exactly why my boss keeps saying conflicting things, the LLM doesn’t know what is true or not. It just regurgitates the random thing from context.











  • They’re definitely not. Hard drives use a fraction of the energy a single LLM prompt does. A study calculated that every 100 words uses an entire water bottle’s worth of water. That’s a lot of cooling for 100 words. It’s very likely that these companies use several gpus for every single prompt, meaning huge servers for just a few users.

    Meanwhile the internet archive has archived a significant portion of the internet with nary a data center in sight.

    You can compress video. You can’t compress the energy needed to generate output.