I'm afraid all you're going to get from me is anecdata, but I find a lot of it very compelling.

I talk to extremely experienced programmers whose opinions I have valued for many years before the current LLM boom who are now flying with LLMs - I trust their aggregate judgement.

Meanwhile my own https://tools.simonwillison.net/colophon collection has grown to over 120 in just a year and a half, most of which I wouldn't have built at all - and that's a relatively small portion of what I've been getting done with LLMs elsewhere.

Hard to measure productivity on a "wouldn't exist" to "does exist" scale.

Every time you post about this stuff you get at least as much pushback as you get affirmation, and yet when you discuss anything related to peer responses, you never seem to mention or include any of that negative feedback, only the positive...

I don't get it, what are you asking me to do here?

You want me to say "this stuff is really useful, here's why I think that. But lots of people on the internet have disagreed with me, here's links to their comments"?

you said

    I talk to extremely experienced programmers whose opinions I have valued for many years before the current LLM boom who are now flying with LLMs - I trust their aggregate judgement.

but every time i've seen you comment on this website or other similar websites on the topic of using LLMs for coding, at least half of the responses you get express precisely the opposite perspective -- that they are not flying with LLMs at all

so i think it is disingenuous to make claims like that without at least acknowledging the differences in experience which are pretty clearly demonstrated

this wouldn't be particularly worth mentioning, if it weren't for the fact that you comment extensively, prolifically, on agentic coding topics, on this website and many others, and your comments are generally effusively and uncritically positive, no matter what responses you get, over time, from anyone

Cool right? It's my playground for vibe coded apps, except I started it nearly a year before the term "vibe coding" was introduced.

I wrote more about it here: https://simonwillison.net/2024/Oct/21/claude-artifacts/ - and a lot of them have explanations in posts under my tools tag: https://simonwillison.net/tags/tools/

It might also be the largest collection of published chat transcripts for this kind of usage from a single person - though that's not hard since most people don't publish their prompts.

Building little things like this is really effective way of gaining experience using prompts to get useful code results out of LLMs.

what on earth?

these are absolutely trivial, toy example programs. they've got nothing to do with anything that anyone is meaningfully talking about when they talk about using LLMs for coding stuff.

is this kind of stuff what you're referring to when you comment on using LLMs for programming?

clone, i dunno, https://github.com/minio/minio, and ask the LLM to implement a non-trivial feature -- this is what everyone else is talking about! not "implement a YAML to JSON converter in the browser"

> Cool right?

100s of single commit AI generated trash in the likes of "make the css background blue".

On display.

Like it's something.

You can't be serious.

The whole debate about LLMs and productivity consistently brings the "don't confuse movement with progress" warning to my mind.

But it was already a warning before LLMs because, as you wrote, people are bad at measuring productivity (among many things).