If you want to make a claim that we won't even see change coming, maybe the large mammals->humanoids transition isn't the greatest example. That took several hundreds of thousands of years.
They did, to some extent, see it coming (in observing a threat becoming more prevalent and dangerous). But they did not have a way to stop humans. Meanwhile, I'm just staring at the power switch on my computer, wondering how we'd stop AI.
(This is not an argument for unrestrained AI - we should safeguard, absolutely. It is a counterpoint to your "we won't see it coming" argument)
We see climate change coming (and here) and yet seeing it coming isn't doing much to stop it.
This is we talk about Moloch commonly on HN. We see systems that are going to have obvious bad outcomes for most individuals, and be unaligned with what most humans want, and yet the system persists and even grows worse. This is how I believe it will be with AI too. It will make the rich insanely rich and powerful even at the risk of destroying us all.
They did, to some extent, see it coming (in observing a threat becoming more prevalent and dangerous). But they did not have a way to stop humans. Meanwhile, I'm just staring at the power switch on my computer, wondering how we'd stop AI.
(This is not an argument for unrestrained AI - we should safeguard, absolutely. It is a counterpoint to your "we won't see it coming" argument)