It seems very obvious to me that humans should attempt to make the future of humanity good.
There should be probably some much larger amount of money applied to AI safety in multiple ways.
The greatest benefits from AI will probably be in eliminating drudgery. So, in terms of tasks that are mentally boring, not interesting, there's arguably breakthroughs in areas that are currently beyond human intelligence..
It's best to prepare for or to try to prevent a negative circumstance from occurring than to wait for it to occur and then be reactive, and this is a case where the potential range of negative outcomes are quite - some of them are quite severe. So, it's not clear whether we'd be able to recover from some of these negative outcomes. "In fact, certainly you can construct scenarios where recovery of human civilization does not occur." When the risk is that severe, it seems like you should be proactive and not reactive.