> My 30k ft view is that the stock will inevitably slide as AI datacenter spending goes down. Right now Nvidia is flying high because datacenters are breaking ground everywhere but eventually that will come to an end as the supply of compute goes up.
Exactly, it is currently priced as though infinite GPUs are required indefinitely. Eventually most of the data centres and the gamers will have their GPUs, and demand will certainly decrease.
Before that, though, the data centres will likely fail to be built in full. Investors will eventually figure out that LLMs are still not profitable, no matter how many data centres you produce. People are interested in the product derivatives at a lower price than it costs to run them. The math ain't mathin'.
The longer it takes to get them all built, the more exposed they all are. Even if it turns out to be profitable, taking three years to build a data centre rather than one year is significant, as profit for these high-tech components falls off over time. And how many AI data centres do we really need?
I would go further and say that these long and complex supply chains are quite brittle. In 2019, a 13 minute power cut caused a loss of 10 weeks of memory stock [1]. Normally, the shops and warehouses act as a capacitor and can absorb small supply chain ripples. But now these components are being piped straight to data centres, they are far more sensitive to blips. What about a small issue in the silicon that means you damage large amounts of your stock trying to run it at full power through something like electromigration [2]. Or a random war...?
> The counterargument to this is that the "economic lifespan" of an Nvidia GPU is 1-3 years depending on where it's used so there's a case to be made that Nvidia will always have customers coming back for the latest and greatest chips. The problem I have with this argument is that it's simply unsustainable to be spending that much every 2-3 years and we're already seeing this as Google and others are extending their depreciation of GPU's to something like 5-7 years.
Yep. Nothing about this adds up. Existing data centres with proper infrastructure are being forced to extend use for previously uneconomical hardware because new data centres currently building infrastructure have run the price up so high. If Google really thought this new hardware was going to be so profitable, they would have bought it all up.
> This approach still works, why do something else?
One issue is that the time provided to mark each piece of work continues to decrease. Sometimes you are only getting 15 minutes for 20 pages, and management believe that you can mark back-to-back from 9-5 with a half hour lunch. The only thing keeping people sane is the students that fail to submit, or submit something obviously sub-par. So where possible, even for designing exams, you try to limit text altogether. Multiple choice, drawing lines, a basic diagram, a calculation, etc.
Some students have terrible handwriting. I wouldn't be against the use of a dumb terminal in an exam room/hall. Maybe in the background it could be syncing the text and backing it up.
> Unless you're specifically testing a student's ability to Google, they don't need access to it.
I've been the person testing students, and I don't always remember everything. Sometimes it is good enough for the students to demonstrate that they understand the topic enough to know where to find the correct information based on a good intuition.
Your blue book is being graded by a stressed out and very underpaid grad student with many better things to do. They're looking for keywords to count up, that's it. The PI gave them the list of keywords, the rubric. Any flourishes, turns of phrase, novel takes, those don't matter to your grader at 11 pm after the 20th blue book that night.
Yeah sure, that's not your school, but that is the reality of ~50% of US undergrads.
Very effective multiple choice tests can be given, that require work to be done before selecting an answer, so it can be machine graded. Not ideal in every case but a very quality test can be made multiple choice for hard science subjects
But again, the test creator matters a lot here too. To make such an exam is quite the labor. Especially as many/most PIs have other better things to do. Their incentives are grant money, then papers, then in a distant 3rd their grad students, and finally undergrad teaching.any departments are explicit on this. To spend the limited time on a good undergrad multiple choice exam is not in the PIs best interest.
Which is why, in this case of a good Scantron exam, they're likely to just farm it out to Claude. Cheap, easy, fast, good enough. A winner in all dimensions.
Also, as an aside to the above, an AI with OCR for your blue book would likely be the best realistic grader too. Needs less coffee after all
For large classes or test questions used over multiple years, you need to take care that the answers are not shared. It means having large question banks which will be slowly collected. A good question can take a while to design, and it can be leaked very easily.
Pros and cons. Multiple choice can be frustrating for students because it's all or nothing. Spend 10 minutes+ on question, make a small calculation error and end up with a zero. It's not a great format for a lot of questions.
They're also susceptible to old-school cheating - sharing answers. When I was in college, multiple choice exams were almost extinct because students would form groups and collect/share answers over the years.
You can solve that but it's a combinatorial explosion.
This is what my differential equations exams were like almost 20 years ago. Honestly, as a student I considered them brutal (10 questions, no partial credit available at all) even though I'd always been good at math. I scraped by but I think something like 30% of students had to retake the class.
Now that I haven't been a student in a long time and (maybe crucially?) that I am friends with professors and in a relationship with one, I get it. I don't think it would be appropriate for a higher level course, but for a weed-out class where there's one Prof and maybe 2 TAs for every 80-100 students it makes sense.
Stanford started doing 15 minute exams with ~12 questions to combat LLM use. OTOH I got a final project feedback from them that was clearly done by an LLM :shrug:
> I got a final project feedback from them that was clearly done by an LLM
I've heard of this and have been offered "pre-prepared written feedback banks" for questions, but I write all of my feedback from scratch every time. I don't think students should have their work marked by an LLM or feedback given via an LLM.
An LLM could have a place in modern marking, though. A student submits a piece of work and you may have some high level questions:
1. Is this the work of an LLM?
2. Is this work replicated elsewhere?
3. Is there evidence of poor writing in this work?
4. Are there examples where the project is inconsistent or nonsensical?
And then the LLM could point to areas of interest for the marker to check. This wouldn't be to replace a full read, but would be the equivalent of passing a report to a colleague and saying "is there anything you think I missed here?".
I wrote something similar years ago, which would instead convert an image into a mesh of polygons. The idea was to have a vector low-size SVG that could be used as an image placeholder or background for web pages.
I think I lost the code, but it was initially a genetic algorithm that randomly placed overlapping polygons, but the later improved method had connected polygons that shared points - which was far more computationally cheaper.
Another method I explored was to compose a representative image via a two-colour binarised bitmap, which provided a pixelated version of the image as a placeholder.
The core idea is that you drop the image as a small Data URI straight into the page, and then fetch the high-detail version later. From the user's perspective, they are getting a very usable web page early on, even on poor connections.
> How much time could you conceivably use this for?
I also thought the same. It's a nice feature to have, but typing out significant code on that keyboard is a burden compared to a full-sized keyboard. Plus, it's not small enough to carry on the daily.
I personally believe the correct form factor for such devices is a smart watch, where code is written off-device and deployed to it, and the results of the code can be enjoyed throughout the day.
For example, I've developed(/ing) a Micropython based smart watch [1] where code is deployed onto the watch. The idea is to be able to deploy apps and interact with them via a simple interface. Being able to interact with my code daily helps keep the device relevant and to be able to make continual progress.
$1 USD is ~90 Indian Rupees, 1450 Argentinian Peso or over 1 million Iranian Rial [1]. In some places, $1 USD could be a week's work. On the collection side, you could be seriously over-charging people. On the distribution side, you could be seriously overpaying people for their work - and encourage scams, etc.
> GitHub should charge every org $1 more per user per month and direct it into an Open Source fund, held in escrow.
Sure. It'll be some charity, then somebody gets paid $200k+ per year to distribute what remains after they've taken the majority, all whilst avoiding most taxes. To receive the money the person has to ID themselves, financial background checks need to be done, a minimum amount needs to be reached before a payment is made, and then after passing through multiple wanting hands, they end up with a fraction.
> Those funds would then be distributed by usage - every mention in a package.json or requirements.txt gets you a piece of the pie.
What even is "usage"? How many times it appears in a number of repos? How many users there are of the project? Is the usefulness and value of a project limited to the number of people that directly use it?
> Or don’t! Let’s not do anything! People’s code and efforts - fueling incredibly critical bits of infrastructure all around the world - should just be up for grabs. Haha! Suckers!
> Anyway, you all smarter than me people can figure it out. I just cannot accept that what we have is “GOOD”. xx
It's entirely possible you can make things worse by avoiding doing nothing. Sometimes in life you have to pick the lesser of evils.
Quite a good implementation, got to 62 and was playing a "space filling curve" strategy to bleed off some of the length. Looking at the leader-board I suspect some people are just sending off their own custom submissions [1].
There is a maximum theoretical length and the sphere actually allows you to wrap around it, Rod of Asclepius style, until you get there - picking up an apple on each cycle. I suspect it's not more than a few hundred segments though so those ~600 submissions are probably someone gaming the submission with a forged score.
Exactly, it is currently priced as though infinite GPUs are required indefinitely. Eventually most of the data centres and the gamers will have their GPUs, and demand will certainly decrease.
Before that, though, the data centres will likely fail to be built in full. Investors will eventually figure out that LLMs are still not profitable, no matter how many data centres you produce. People are interested in the product derivatives at a lower price than it costs to run them. The math ain't mathin'.
The longer it takes to get them all built, the more exposed they all are. Even if it turns out to be profitable, taking three years to build a data centre rather than one year is significant, as profit for these high-tech components falls off over time. And how many AI data centres do we really need?
I would go further and say that these long and complex supply chains are quite brittle. In 2019, a 13 minute power cut caused a loss of 10 weeks of memory stock [1]. Normally, the shops and warehouses act as a capacitor and can absorb small supply chain ripples. But now these components are being piped straight to data centres, they are far more sensitive to blips. What about a small issue in the silicon that means you damage large amounts of your stock trying to run it at full power through something like electromigration [2]. Or a random war...?
> The counterargument to this is that the "economic lifespan" of an Nvidia GPU is 1-3 years depending on where it's used so there's a case to be made that Nvidia will always have customers coming back for the latest and greatest chips. The problem I have with this argument is that it's simply unsustainable to be spending that much every 2-3 years and we're already seeing this as Google and others are extending their depreciation of GPU's to something like 5-7 years.
Yep. Nothing about this adds up. Existing data centres with proper infrastructure are being forced to extend use for previously uneconomical hardware because new data centres currently building infrastructure have run the price up so high. If Google really thought this new hardware was going to be so profitable, they would have bought it all up.
[1] https://blocksandfiles.com/2019/06/28/power-cut-flash-chip-p...
[2] https://www.pcworld.com/article/2415697/intels-crashing-13th...
reply