1/10/06 - The Planet Network Problems

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • AndrewT
    Administrator
    • Mar 2004
    • 3655

    #1

    1/10/06 - The Planet Network Problems

    The Planet is currently experiencing problems with at least one of their main routers which is why many of you are unable to access your websites at this time, although this does not affect everyone. I will update this thread as more information becomes available.
  • AndrewT
    Administrator
    • Mar 2004
    • 3655

    #2
    Everything should be reachable once again. Please submit a ticket if you continue to have problems. I should have more of an official explanation as to what happened later today.

    Comment

    • AndrewT
      Administrator
      • Mar 2004
      • 3655

      #3
      Just a quick update until an RFO is available..

      The problem should be resolved now. We'll issue an official RFO, but it appears that the CEF process crashed on one of our distibution switches at DLLSTX3, causing packets to be switched sub-optimally through that equipment. We had to hard reload it to get it to start functioning normally. Needless to say, we'll be contacting Cisco about this bug. We apologize for the inconvenience.

      William Charnock
      Sr. VP Engineering/Operations
      ThePlanet Internet Services

      Comment

      • AndrewT
        Administrator
        • Mar 2004
        • 3655

        #4
        Here is the official RFO:

        RFO 1/10/06 DSR02.DLLSTX3 performance issues

        Summary:

        On Jan 10th 2006 at approximately 7:19am CST DSR02.DLLSTX3 began experiencing low traffic levels. The Planet engineers were alerted and proceeded to investigate the issue. After eliminating external causes the decision was made to reboot the router. During this time some customers may have experienced packet loss or latency due to the reload and reconvergence. By 8:03pm CST the router was back to normal operation.

        Future Mitigation:

        Planet engineers are working with the vendor to ensure that the router hardware and software are upgraded and patched to prevent any future outages.

        Comment

        Working...