If it fails and we have to rollback, I literally have a step in my runbook labeled 'weep'.
What will happen is a series of small nightmares that will eventually bring things back to a semi-working order. The ways that it could fail are a bit too dynamic to fully consider before-hand. Worst case I'd probably need to fail back to the old primary and rebuild all of the replicas.
from overview for alienth /r/announcements/comments/7xozuw/when_this_post_is_1_hour_old_reddit_will_go_down/dua1jgl/
via IFTTT
