They've certainly thought about it a lot more than me. I remain in my not too sure state about whether we could in fact muddle through somehow. But while I have absorbed some of the pessimism, I think it's probably harmful overall.
"We'll have to work faster" was once a condolence in the face of death. The world death clock still stands at 1.8 per second after many years of global progress. This is unacceptable, and reducing this value to 0 should be a fiery motivator, especially for young people with a lot of talent.
AGI is one path to get there. There are other potential ones. But of course AGI is the "big one" in that it would solve everything else as a special case. Assuming it doesn't destroy us.
Such motivation used to exist in important people (like those at MIRI). It still may exist elsewhere. But I don't think there will be much rallying around dying with a bit more dignity.
There was/is another source of motivation, more selfish, and that's the one of "all this" seemingly being possible within our lifetimes. (Us being anyone with still at least ~30 years of life expectancy left, and a few more older people who think well of cryonics.) "All this" being huge technological changes, typically transhumanist in style. It's a common historical bias, a famous example being AI researchers in the 50s thinking that they could solve the problem over a summer. But half a century later, by the mid to late 90s, we could more justifiably predict something soon just from the rate of progress. It's either got to stop, or keep going and lead to some strange new possibilities. Perhaps it might stop somewhere strange for a while. But perhaps it won't "stop" and we'll truly hit a Singularity moment. John Wright's Golden Age at minimum.
It's still optimistic in vision, but creates a bigger drive for passion (that can be sustaining over a lifetime, look at Goertzel) when you can clearly taste the possibility of seeing it for yourself. If you were an adult in the 60s, you might have been able to see the long term potential future, but you could realistically only hope to build some stepping stones along the way to it.
Perhaps it is the same today, and even those just born today are no more likely to see humanity's glorious potential future than those born a hundred years ago. And not necessarily due to some existential risk, but simply because progress takes longer than we think, and too many are blinded by a taste that they can live to see it. Thiel is 55, I'd bet he's been thinking about his mortality over the next 30 years more than he did 30 years ago, and where humanity will be.
On the side of the dignity guys, I suppose it may be better that humanity lasts a little longer before being replaced. And perhaps if it lasts a bit longer, that may be just long enough so that we can in fact muddle through. Proposals of global governance and technological control and so on have been flung around, maybe they'd work, maybe not. How much of the reaction against them is really worrying about the downsides of tyranny, which humanity has lived with for much of its history in some form or another (if nothing else the tyranny of nature that has been with us for 300 thousand years or so)? If such draconian measures give the species an extra 50 years, an extra 100 years, an extra 5000 years as we plunge back into some dark age... if we come out at the end of that and manage to muddle through to a successful Singularity, is that not better than giving it our all now when we're most likely to fail and doom the planet? But if we do need another long period of time to do this, it's again unlikely anyone alive will live to see it (or be em'd to see it), and instead they can't even look forward to their current comfortable existences but imagine ending their days in a much worse state than they remember from their youth. The 90s was the best decade after all.
I don't know. Just rambling thoughts and wondering where people get their motivations from.
Posted on 2022-11-23 by Jach