Outage - post incident review

On 29th July 2025, the DataTables.net site had a major outage as a result of an attack. This took out the main site with its examples and documentation, the support forum, and the sub-domains, including, most importantly, the DataTables CDN.

Digging into such an event, when something has gone so wrong, is not a fun thing to do, but it is important for me to do so I can learn what I can do better in future, for you so you can understand what happened and the challenges faced, and for the community so we can try to stop this sort of thing happening in future.

Domain hijacking

Any website has a large surface which needs to be protected against hacking attempts - the application software, the HTTP server, the actual server the site resides on, DNS, and of course, the domain registration itself. The attack yesterday came on the domain name, whereby the attacker managed to move the domain out of the account I have with the domain registrar I use, into another controlled by the attacker, at which point they were able to change the name servers, pointing the domain at a different server.

To be clear, any account information you have on DataTables.net is safe - the server was never breached. The content on the DataTables server is safe. The DataTables source code is safe. Services are now all normal, but this was a major disruption for the site and anyone using its services, with some caching issues persisting at the time of writing.

How?!

So how did the attacker manage to execute this domain takeover? Around mid-June, I started receiving an extraordinary amount of email to one of my old email addresses. It was sign-ups for just about every list imaginable - three per minute, which went on continuously. I created a filter and assigned everything from that email address to a folder.

The attacker's next step was to create an account with the same registrar datatables.net is registered with and request a transfer of the domain to their account. They used an email address intentionally crafted to look like it could be mine and submitted a fake driver's license and utility bill with information that could only have been from leaked WHOIS data. The registrar accepted this as proof of identity and started the transfer process. That included sending an email to me to confirm the transfer, an email which I never saw due to the flood of emails (which it is now easy to say was the start of the attack).

At the end of a five day period with no reply from me about the transfer, the registrar defaulted to accepting the transfer and the domain was moved to a different account. At no point were any of my accounts or email compromised - the attacker managed to socially engineer the registrar into making the transfer.

Events on the day

Once the domain was under the attacker's control, events unfolded as follows on 29th July:

• 02:57 - The name servers for datatables.net were changed to point at different CloudFlare name servers. CloudFlare blocked all traffic to the new target with a 1000 error. Some users are still able to load CDN resources due to local cache.
• 07:10 - I read monitoring emails saying DataTables.net when I woke. Dashed to a computer.
• 07:21 - I contacted the registrar, having realised that the domain was no longer under my account.
• 08:45 - While replying to emails and messages saying the site was offline, I realised I could create a mirror domain of the CDN.
• 09:42 - datatables-cdn.com was deployed as a mirror, allowing those affected to change just the domain - the script paths would still work.
• 11:05 - Submitted evidence that I was the rightful owner to the registrar.
• 12:10 - Submitted further evidence to the registrar (pro-tip, make sure your WHOIS information is up-to-date).
• 13:11 - Domain transferred back to my account - name servers reassigned to what they should be, and services start recovering.

Unfortunately, while the datatables.net server was fully ready at this point, DNS propagation is not instant, and many routers will cache DNS lookups. The DataTables TTL for DNS is all set to 5 minutes ("Auto" in CloudFlare's DNS management), which should allow for reasonably fast propagation, however, many routers appear to incorrectly hold their cache much longer and as a result many users continued to have problems loading resources from the CDN (in some cases this is still the case at the time of writing).

CDN

Normally if a site like DataTables were to go down, it would be irritating, but we can all survive without documentation for a software library for a day. However, the impact in this case was significant due to the DataTables CDN.

DataTables and its extensions (minus Editor, due to its commercial license) are MIT open source software, and to make it easily accessible for developers, it can be obtained in a number of ways - NPM, download for self hosting, or via the DataTables CDN server. Many choose to use the CDN - I provide it for free and it is fast and convenient. I use CloudFlare as a caching service, which makes it super fast and reliable (as long as it has a domain pointing at it!).

The image above shows the gap in the requests to DataTables during the attack. The CDN has around 55TB of data transferred per month. At approximately 3'440 million requests, that is a lot of Javascript and CSS! It will also give you an idea of the impact that a disruption to the CDN will have around the web, and what makes DataTables a valuable target for an attack of this type.

Credit where it is due

I'd like to extend a big thank you to my domain registrar for resolving this issue. It could have been difficult, but they made the process clear, straightforward and explained actions to me along the way.

CloudFlare, they blocked requests during the outage with a 1000 error, which could otherwise have been served maliciously.

Finally, to you, the DataTables community. I fully recognise how taxing it will have been for many of you trying to work out why the CDN wasn't working yesterday and the impact on your own businesses. Not once did I have people virtually yelling at me once I communicated the issue, I was almost universally wished well while I got on with trying to fix it.

And contempt where it belongs

The fact that someone would attack an open source product such as DataTables sickens me. I release by far the majority of my work as free open source software, host a free to use CDN, and support the software. I'm not just trying to make the world that little bit better; I really enjoy what I do as well. This attack was an attack on all of the work I've done, and given how it was executed, the attacker doesn't appear to have got out of it other than proof that they could, and to damage my reputation.

Learning process

In online security, it is not a question of if one is going to be attacked; it is a question of when. For me, it was over the last month, culminating in the events of yesterday. It was an attack vector that I had not expected - no compromised accounts, but rather social engineering against the registrar.

Immediate actions I've taken:

• All email from my domain registrar is now marked as high priority and will never skip the inbox.
• Put forward my opinion that having the default timeout action of a domain transfer request to accept it is wrong.
• Added DNS monitoring for DataTables.net (HTTP and ping were already monitored along with various other automated security scanners).

Further actions:

• I will make the CDN server available for anyone to spin a new one up with checkout from GitHub.
• Continue to improve my security monitoring of all domains I manage.

Actions you can take:

• Use sub-resource integrity for all external Javascript and CSS you load. Since February this year, the DataTables download builder will automatically calculate the SRI for the resources selected. Use it! It makes sure that what you want to load is what actually gets loaded.
• Consider if you want to download and host the files yourself, rather than loading them from an external CDN.
• If you ever get a flood of sign up emails - consider that it is to hide something!

Moving on

My plan for yesterday had been to continue work on Editor 2.5 and ColumnControl 1.1. Once the dust has fully settled, I will be back on the case - expect to hear more about them in August.

Stay safe out there everyone.