9/2/12 Denial of Service Attack Updates: Resolved and All Services Restored
- September 2, 2013
- 39 Comments
Our server is currently under a denial of service attack. This means that an attacker is flooding some of our IP addresses with garbage requests and preventing us from serving cart pages for legitimate requests.
The primary IPs affected include the *.dpdcart.com domain and service to the admin panel is not currently affected. However, because the dpdcart.com domain is affected this does affect adding products to the cart and checkout.
We are working with our datacenter to mitigate the attack, including installing network appliances designed to stop these types of attacks, and we hope to have service restored for everyone shortly.
We will be posting updates to Twitter and here as they happen.
10:40AM Update: Services are being restored now. Change should propagate over all affected IP addresses over the next few minutes.
12:30PM Update: A Cisco threat mitigation appliance has been installed and networks where the attack are originating from are being blocked. Some users who share these networks may be blocked also. This is unavoidable but blocking these networks allows the vast majority to get through.
4:30PM Update: Network traffic is returning to normal, but there are still a few networks and ISP that are being blocked due to the ongoing attack. We’ll continue to monitor and report our status, and a large majority of traffic is currently getting through.
7:30PM Update: Datacenter is reporting all traffic is back to normal and DDoS event is over (hopefully). Everyone should be able to access their cart and checkout.
7:32PM Update: DDoS’d again, twice as hard as before. We’re working on it.
10:30PM (EST) Update: Once again, traffic is starting to come through to the DPD Server. We’ve placed the server behind a threat management appliance so some networks where the distributed attack are originating will continue to be blocked, but for many locations carts will be accessible again.
We’re working with a 3rd party cloud provider to set up continuous DDoS protection by blocking the traffic before it reaches our server. We’ll be up all night getting this set up so we can get this behind us and return to full service for everyone.
We know this has been hard on everyone and we’re taking every step possible to restore service to everyone as soon as possible.
10:30AM Update: DPD is still being protected by the DDoS mitigation appliance and is serving requests to most traffic. The mitigation appliance is still blocking some networks where the ongoing attack is originating from, but most requests are coming through.
There was another attack spike this morning which was mitigated.
We’re still working with the cloud based DDoS protection provider to get our servers protected so we can get off the appliance, but its a very slow and complicated process. We’re working with their tech team to get it in action as soon as possible.
We’re working in rotating shifts to keep people on the job 24/7 during the attack.
6PM Update: New threat mitigation system in place, services are returning to normal for the cart domain. For full details please check your email or read online here: eepurl.com/EFUlX