I think what PG meant by "do things that don't scale" is earnest effort in service of building a real product: talking to users, manually onboarding, hand-holding early customers so you can learn fast and iterate toward something that eventually does scale.
What this startup did isn't that, AFAICT. It wasn't manual work in service of learning...it was just fraud as a business model, no? Like, they were pretending the technology existed before it actually did. There's a bright line between unscalable hustle and misleading customers about what your product actually is.
Doing unscalable things is about being scrappy and close to the problem. Pretending humans are AI is just straight up deceiving people.
Totally agree with this point. There are several advice that pg and similar roles give which are not universally true. I reiterate your point that "doing things that don't scale" is meant specifically for searching for 1-1 user experience advice.
A similar exmaple is "Make something people want". This is generally true advice in focusing your efforts on solving customer's problems. Yet, this is disastrous advice if taking literally to the fullest extent (you can only imagine).
What this startup did isn't that, AFAICT. It wasn't manual work in service of learning...it was just fraud as a business model, no? Like, they were pretending the technology existed before it actually did. There's a bright line between unscalable hustle and misleading customers about what your product actually is.
Doing unscalable things is about being scrappy and close to the problem. Pretending humans are AI is just straight up deceiving people.