Hacker Newsnew | past | comments | ask | show | jobs | submit | threetonesun's commentslogin

If we kill all the platforms where content for training LLMs comes from, what do LLMs train on?

This. I'm really bothered by the almost cruel glee with which a lot of people respond to SO's downfall. Yeah, the moderation was needlessly aggressive. But it was successful at creating a huge repository of text-based knowledge which benefited LLMs greatly. If SO is gone, where will this come from for future programming languages, libraries, and tools?

This always feels to me like, an elephant in the room.

I’d love to read a knowledgeable roundup of current thought on this. I have a hard time understanding how, with the web becoming a morass of SEO and AI slop - with really no effort being put into to keeping it accurate - we’ll be able to train LLMs to the level we do today in the future.


Newspapers, scientific papers and soon, real-world interactions.

News is the main feed of new data and that can be an infinite incremental source of new information


You talk about news here like it's some irrefutable ether LLMs can tap into. Also I'd think newspapers and scientific papers cover extremely little of what the average person uses an LLM to search for.

Currently after 3 years the price of the GPU if you decide to sell it might be a wash, much like it was after the crypto boom. Granted you have to pay for electricity to run it, but you also have full control over what it runs.

An F-150 Lightning and Cybertruck weigh somewhere between 6000 and 7000 pounds, so I personally think of them the same way as if you replaced your horse with a hippo.

It's not hard to convince people to move to electric, just make it such a better economic proposition that it would be silly not to.


No pickup is light. A Lightning weighs about the same as a gasser F250, and less than a diesel F250.

I don't need a pickup truck, but if I ever did, I'd get whatever my landscaper has. Unlike most people with Rivians, Lightnings, Cybertrucks, Ridgelines, and Raptors, he totally relies on that truck for work.

So far it's Tacoma. Maybe some day he'll have an EV instead.


I knew someone who had a Tacoma for construction work; he got it because everyone had one.

Later he had to take it in to the shop and they gave him a loaner cargo van, and from then on he regretted not getting a cargo van instead of a truck.

The vast majority of what he did with the truck was carry tools, which are easier to access in a van; the few times he carried materials he would have to unload the whole truck or get the trailer anyway.


Calorie wise you'd need to 2-3 apples to equal most Little Debbie snacks, but in terms of satiation the opposite is true.

I don't disagree. I was responding to the specific claim in the post I replied to:

> I can buy a bag of apples for less than what a pack of Little Debbie snacks cost.

Satiety was not part of the post I replied to.


Maybe! But the fact that Apple also makes the operating system means the hardware/software integration can’t be beat. I’ve never used a non-Apple laptop where the trackpad worked a tenth as good as any MacBook.

Do you believe that ChatGPT is doing the the research? I'm all in favor of better access and tools to research but at least in the US all of the research is being defunded, we're actively kicking researchers out of the country, and a bunch of white billionaires are proposing this as an alternative, based on training data they won't share.

This is a product feature that invalidates WebMD and the like. It does not solve any health problems.


You could go back in time and say this about jQuery. Tailwind's future was always questionable because CSS is growing in new and amazing ways, and wrapping the complexity of new CSS features into helper classes isn't really a sustainable model.

That said if someone wants a business model, figure out a way to get paid to get AI to make UIs using newer CSS features, because right now it's quite terrible at it.


The difference is that jQuery was replaced by other libraries, while Tailwind grows in popularity, but due to AI its creator doesn’t benefit from this popularity as much as before

jQuery was essentially replaced by JavaScript (and browser compatibility) getting better, but it continued to exist and grow because it was the de facto way to DOM manipulation, especially if you had to copy and paste off of Stack Overflow, or roll out a framework based UI.

Tailwind being the default choice for AI UIs is not that different, it can continue to grow in usage but the fundamental need for Tailwind has passed.


The difference is jquery went away because better things replaced it (in javascript). If the fundamental need for tailwind has passed why is it's usage growing? It's more that the problem solved by the paid portion of tailwind is now solved by AI.

I'm not sure megayachts have no economic downside. They create some jobs, certainly, but if the golden goose is catering to the whims of eccentric billionaires, then collectively we're losing smart and talented people who could be building things like better public transportation, or more reliable and sustainable infrastructure.

You're imagining a nonexistent problem.

Luxury yachts and public transit make approximately 0 talent impact on the world.

Building each is an extremely tiny industry.

These are also very average talent required jobs.


And I would wonder if overpriced luxury items in general are detrimental to the economy. It feels to simple me that transacting items with subjective added values above costs shrinks economy a bit.

To me it feels that overpriced items cause less trickle downs if they cost less to make: the delta would go into an oblivion. Trickle downs don't appear to be real, but less than already immeasurable sounds bad.


I disagree with this. Some of the more extravagant engineering prowess has divested from insane eccentric billionaires

Like the other commenter said the larger issue is the monopolization (see groq x nvidia for a glaring example) and other forms of rent seeking that clamp down on market fairness and efficiency


Mega yachts are not airplanes, computers, software or some other cutting edge or basic research technology. It's a boat that other then adding starlink hasn't meaningfully changed for decades.

Mega yachts are designed using CAD software, the prototypes are milled or even sometimes now 3d printed out of huge foam panel. Advanced hulls are constructed from a number of materials and methods. All of these things required bespoke skilled labor that cannot be replaced with robots or AI.

I think mega yachts are stupid, that doesn't mean they shouldn't exist and to act like they are the same as even 20 years ago is willfully ignorant


It feels like the quality of our eccentric billionaires has gone downhill.

We've all seen the comic where the villian tells Spider-Man "I don't want to cure cancer, I want to turn people into dinosaurs." I would respect that a hell of a lot more than whatever Bezos/Musk/Zuckerberg/etc are doing with their money. Hell, I'd settle for Howard Hughes levels of madness-mixed-with-genius.

Everyone just buys the same Billionaire Starter Pack these days consisting of ugly yachts and mansions, and the obligatory rocket company. The change is just spent on aggressive hoard-management and political machinations. Is their heart even in any of those ventures? When they wake up, so they say "I spent the first 40 years of my life selling my soul so I can do this, and it was so worth it?"

I know if you dropped twelve figures in my lap, I'd be saying things like "who can I hire to help me engineer an army of superintelligent dragon creatures" or "let's level Peoria, Arizona and rebuild it as a car-free zone with the density of midtown Mahnattan", not "let's buy Yahoo, because all the other depressing media outlets are spoken for already."


I don't think the lack of talent is preventing us from having good public transportation. Diverting smart folks from yacht building will not eliminate the problems with unions, zoning, or crime to name a few of the issues. These are difficult problems because people and their different desires are involved.

One city has a millionaire who builds a yacht for 100M dollars in a local shipyard and uses it for holidays. The neighbor city has a millionaire who spends 100M dollars to build 10 ferries he gifts to the city. The general population is clearly better off in the second case, even if it does not matter for the workers in the shipyard.

Cute gambit to talk about ferries, the one mode of transport that doesn't have insane land-use debates. (Although they do need waterfront property on both ends.) I'll give you a point for that one.

But let's be serious. Ferries have a very limited use in only a few cities. Even then, the appeal is limited because they're relatively slow.

I submit that the most common result of replacing one yacht with $100m of public transit spending is that the unions and the bureaucracy will eat up the $100m in a few minutes.

The theory here is that diverting more smart people into "good" careers like urban planning will be great. But if we look at the last 100 years in the United States, the rise of careers like urban planning have been correlated with an explosion in construction costs.

Yet back in the bad old days when there weren't urban planning degrees and only a few effete twits went to college, private capitalism was able to build two big urban transit systems in NYC. No book smart people. Just sandhogs and profit motive. How much did it cost to add just a few stops to the NYC subway over the last two decades?


Yes, when you don't care about the environment or safety regulations or displacing people poorer than you, infrastructure is a very easy problem to solve. Fortunately for you, we seem to be heading back towards that way of thinking.

Also, I live near a city that had one of those ridiculous, way over budget projects where sure, some money was funneled to unions and bureaucracy or whatever evil monsters you've concocted here. No amount of billionaire pet projects could match the amount of good it did across the number of people it affected. Sometimes the inefficiencies of human cooperation are greased with money, and that's perfectly fine.


Is 75% of the country zoned SFH because of safety regulations or environmental reasons?

That seems very lucky of the second city. Let's hope we get lucky with some generous billionaires soon!

I don't claim that regulations are simple, and incentives couldn't be created to result in infrastructure investment by the wealthy... but I won't hold my breath.


A lot of Windows 8 I liked, but Windows perpetually suffers from needing to support older versions of Windowing systems, or some corporate usecase from the early 90s that carries too much money to ever say no to implementing.

Windows 11 is, I think, worse than MacOS these days, half for still dragging the past along with it, and half for introducing a second start menu just for ads.


I think Windows greatest strength is their greatest weakness, which is backwards compatibility. MacOS greatest weakness is their UX, which has slowly been going downhill for the past few years and on this release took a nose dive. It is a wild reversal from the mid 2000s when Apple's UX was so far superior to anything else that it felt revelatory to switch from Windows XP to OSX.

It's funny back in the Google Reader days monetizing via RSS was quite common. You'd publish the truncated version to RSS and force someone to visit the site for the whole version, usually just in exchange for ad views. Honestly while it wasn't the greatest use of RSS it was better than most paid blogs today being ad-wall pop-up pay-gate nightmares of UX.

Even the short snippets are better if one wants to aggregate interesting topics and then read what seems interesting. Not just endlessly scroll each site individually.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: