MySQL replication - master failed - how to recover
I recently experienced a problem with my huge MYSQL database. The SSD on the master failed - basically it refused to be recognised so was useless as a drive. My Databases are pretty big - terrabytes. I have several slaves which are replicas of the master so this failure was an irritation rather than a disaster. Here's how I recovered it and also recovered from some "edge cases" I discovered with the rather pathetic MySQL replication process. A key lesson I've learnt is I probably need to set up RAID on the master. I've had hardware RAID in the past but unfortunately I discovered a common failure mode that two drives on the same controller can have a common mode failure which can kill both drives - hence at the time I decided not to have RAID. I need to revisit this decision with soft RAID as it would same time in recovery. My databases are big so the first problem I encountered is that there isnt enough spare disk space on the slaves to dump a copy o