Page 2 of 2 FirstFirst 12
Results 11 to 19 of 19

Thread: MySQL down on cpanel78

  1. #11
    Administrator AndrewT's Avatar
    Join Date
    Mar 2004
    Location
    Tulsa, OK
    Posts
    3,640

    Default

    First and foremost, I have to apologize for the lack of updates. While the forums certainly won't take priority in an issue like this, many tickets did go unanswered for quite some time. To say it has been hectic here this morning would be a huge understatement. Rest assured that we are continuing to work on this and get your server back online as soon as possible.

  2. #12
    Junior Member
    Join Date
    Mar 2004
    Posts
    28

    Default

    Thanks for keeping us updated. I hope it can be restored with no data loss. The problem is bigger for us now because at this moment is not only MySQL but the entire server wich is down ;(
    Anyway, I'm pretty sure that you and your team are doing the best to fix it as soon as possible. Despite this problem, I'm still "in love" with Dathorn

  3. #13
    Senior Member
    Join Date
    Mar 2004
    Posts
    111

    Default

    cpanel 72 is back up...thank you..!

  4. #14
    Junior Member
    Join Date
    Mar 2004
    Posts
    28

    Default

    Andrew, is everything OK or you are still working on it?

  5. #15
    Junior Member
    Join Date
    Mar 2004
    Posts
    28

    Default

    From Andrew: An update will be coming shortly.

  6. #16
    Administrator AndrewT's Avatar
    Join Date
    Mar 2004
    Location
    Tulsa, OK
    Posts
    3,640

    Default

    Hello,

    First and foremost, I want to apologize once again for the downtime that has occurred today. As you know very well, this is not normal for us and we strive very hard to avoid situations like this.

    This morning's extended downtime was the result of a few very unfortunate failures. At approximately 3:00 AM CDT logs indicate that cpanel78 began experiencing file system errors. At this point we don't know the exact cause of this but it appears to be due to a failed SSD not having been dropped by the RAID controller properly. Because of this, the corruption was able to spread to the RAID 1 mirror. Eventually the RAID controller did drop the failed SSD and rebuild the RAID 1 array with one of the hot spares present in the server but it was too late and the corruption had already occurred.

    Our initial plan was to attempt to repair the corrupted files but these turned out to be far more widespread than we expected. This unfortunately left us with only one option; restoring from backup. We proceeded to restore this server's root partition from yesterday's backup. As a result of this, MySQL databases and any other data not residing in a user's home directory will have been reverted to what it was on May 31st at approximately 8 AM CDT. The home partition was not affected (this includes each account's email, public_html data, etc.). We will be following up with a much more thorough investigation of what transpired and will be working with our hardware vendors to hopefully eliminate this from ever occurring again.

    If you experience any further problems please don't hesitate to submit a ticket so that we may investigate them. We greatly appreciate your patience and understanding.

    Best Regards,
    Andrew Thornton
    President - Dathorn, Inc.

  7. #17
    Senior Member
    Join Date
    Mar 2004
    Posts
    111

    Default ahhh... why was cpanel 72 down then? Thanks.

    It was down for at least about 4 hours or so...I didn't notice it until late: about 920 am EST.

    Quote Originally Posted by AndrewT View Post
    Hello,

    First and foremost, I want to apologize once again for the downtime that has occurred today. As you know very well, this is not normal for us and we strive very hard to avoid situations like this.

    This morning's extended downtime was the result of a few very unfortunate failures. At approximately 3:00 AM CDT logs indicate that cpanel78 began experiencing file system errors. At this point we don't know the exact cause of this but it appears to be due to a failed SSD not having been dropped by the RAID controller properly. Because of this, the corruption was able to spread to the RAID 1 mirror. Eventually the RAID controller did drop the failed SSD and rebuild the RAID 1 array with one of the hot spares present in the server but it was too late and the corruption had already occurred.

    Our initial plan was to attempt to repair the corrupted files but these turned out to be far more widespread than we expected. This unfortunately left us with only one option; restoring from backup. We proceeded to restore this server's root partition from yesterday's backup. As a result of this, MySQL databases and any other data not residing in a user's home directory will have been reverted to what it was on May 31st at approximately 8 AM CDT. The home partition was not affected (this includes each account's email, public_html data, etc.). We will be following up with a much more thorough investigation of what transpired and will be working with our hardware vendors to hopefully eliminate this from ever occurring again.

    If you experience any further problems please don't hesitate to submit a ticket so that we may investigate them. We greatly appreciate your patience and understanding.

    Best Regards,
    Andrew Thornton
    President - Dathorn, Inc.

  8. #18
    Senior Member
    Join Date
    Mar 2004
    Posts
    111

    Default

    Sorry... never mind about cpanel 72.... hee, hee... I'm on 78. Thanks.

  9. #19
    Administrator AndrewT's Avatar
    Join Date
    Mar 2004
    Location
    Tulsa, OK
    Posts
    3,640

    Default

    There is no cpanel72 and hasn't been for many months. cpanel72 was migrated to cpanel78 back in November. If you login to our client portal you'll also see that you're located on cpanel78.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •