Hacked! And I didn't like it - URLScan is Step Zero
My blog was down a few days ago. I've had downtime in the minutes over the last few years, but as far as I recall, it's never been down for any significant time. Keyvan noticed that a bunch of us were attacked. Phil Haack was also, ahem, haacked.
UPDATE: To be clear, I wasn't really hacked. I was "DoS'ed" or brought down for a little bit by a distributed denial of service attack that spiked my CPU. I'm advocating that you constrain the URLs that input that get to your application by either black-listing, or white-listing allowed content.
I host at ORCSWeb and have forever. We're in the process of making a lot of chances to my blog. I'm on an x64 machine (I've blogged about DasBlog on x64 before), but running in 32-bit AppPool We're moving my blog to a dedicated server, switching to x64, and we also were upgrading to UrlScan 3.0 which just had a Beta Release in June.
Anyway, in this crazy process, there was window of time where I didn't have UrlScan enabled on the machine. I mistakenly thought that the Ninjas wouldn't be able to catch me if I was on fire. In fact, not so.
Speaking of Ninjas, Wade Hilmo is a ninja at Microsoft who writes UrlScan.
There's a great IIS7 Request Filter for protecting against nasty attacks, but UrlScan Beta 3.0 still has the edge on the filter for the time being. Version 3.0 of UrlScan adds:
- Support for query string scanning, including an option to scan an unescaped version of the query string.
- Change notification for configuration (no more restarts for most settings.)
- UrlScan can be installed as a site filter. Different sites can have their own copy, with their own configuration.
- Escape sequences can be used in the configuration file to express CRLF, a semicolon (normally a comment delimiter) or unprintable characters in rules.
- Custom rules can be created to scan the URL, query string, a particular header, all headers or combination of these. The rules can be applied based on the type of file requested.
- Support for 64 bit IIS worker processes.
This release of UrlScan is a beta, but it's config file is backward compatible and there's a GoLive license. It's working great for me. However, to quote Wade:
"While they are effective against the current wave of automated attacks, they cannot protect against more directed attacks against a specific server."
This was a SQL Injection attack with URLs that looked like this (and some variations):
[08-11-2008 - 17:29:31] Client at 201.67.x.x: Query string length exceeded maximum allowed. Request will be rejected. Site Instance='13', QueryString= 'guid=0b93befc-3543-4bfc-ba8e-6cd340b6d9d3;DECLARE%%20@S%%20VARCHAR(4000);SET%%20@S=CAST(0x4445434C4152452...(incrediblyLONGQueryString)...220%%20AS%%20VARCHAR(4000));EXEC(@S);--', Raw URL='/blog/CommentView.aspx'
In this example, it's hitting CommentView.aspx and trying to add a bunch of T-SQL at the end, with the most evil part encoded inside a CAST() statement. It's a distributed attack with a bunch of (likely innocent) drones reaching out to be mean. In a few hour period, there were thousands of attacks for over 250 different IP addresses.
Fortunately DasBlog doesn't use a database at all, rather a bunch of XML files for storage. Unfortunately, the application was still trying to map these query strings to blog posts, and the result took my blog down.
There's really two main things to think about when dealing with user input, remembering that the URL is an input point for your application!
- Trust no input from the user.
- Constrain the input that reaches your application code as much as you can. Deny as early as possible (hardware, load-balancer, appliance, module, etc).
We need to tighten up DasBlog to more quickly reject URLs that are surely not requests for blog posts, but a tool like UrlScan allows me to easily reject obvious attacks in an way that is more efficient than letting my application code do it.
I would encourage you to take a moment and do a threat analysis on your own websites, and make sure that you ARE constraining input appropriately.
One thing to note, you can and will likely break things for a while with UrlScan, as it does constrain input and you might have valid URLs you aren't aware of. For example, UrlScan broke OpenID authentication for me as the query strings included dots, which UrlScan was denying and also the presence of the word "open" in the querystring. Other denials can happen because of keywords in the URL or length of the querystring. Be sure to test appropriately and watch the UrlScan logs of denials. You can set very blanket rules, or constrain by extension.
We always installed UrlScan on staging and production machines when I was in banking, and made them part of the testing and deployment process. In these times, having a filter installed like UrlScan is Step Zero. I will remember that in the future! Thanks to ORCSweb for answering my 2am emails and helping me fix it in near-real-time!
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
Douglas Karr
Thanks.
The other gotcha with URLScan was that it blocks requests of a certain Content-Length. Our site has video uploads and then were hitting 404s upon completion because URLScan defaults to a 30MB Content-Length.
Scott
I'm running dasBlog installed in a shared hosting enviroment (as well as a lot of other dasBlog users I guess). Do you have any tips that we could use to prevent this sort of attacks?
I'm guessing that since I'm in a shared hosting enviroment I can't install UrlScan (I guess this is something my hosting provider has to do), but correct me if I'm wrong.
off topic but SP1 was officially released today! Where's your post?
:)
http://www.techcrunch.com/2008/06/11/my-blog-was-hacked-is-yours-next-huge-wordpress-security-issues/
The type of attack we're talking about here could be attempted on any blog engine and take that engine down if the proper denial systems are not in place as Scott points out. Kudos to Scott for sharing the experience so we can all get busy checking our own vulnerabilities.
Glass houses and all that...
;-{>
-Tyler
IIS7 includes virtually all the features of UrlScan 2.5; however, neither UrlScan 2.5 nor plain IIS7 provide facilities for filtering URI query strings. This is a new feature in UrlScan 3.0.
(Also, if the application running on top of IIS7 is insecure, as is the case in SQL injection attacks, there is only so much that IIS can do for it. UrlScan 3.0 provides a way of filtering URI query string based attacks but that'll only last until an attacker figures out a way to beat your filter. As Scott indicated, the ultimate solution is to fix the app's code to keep this from happening.)
Re constraining input, I don't know how UrlScan works (it's not clear from your post what you are advocating), but the way to go is white listing, not black listing.
Maybe it's the hack that's got you annoyed.
Still it's out there
http://www.microsoft.com/downloads/details.aspx?FamilyID=fbee1648-7106-44a7-9649-6d9f6d58056e&DisplayLang=en
and If there is any, Discard the request.
there is only one Problem with this Approach, that you will never be able to Submit a value, which Consists one of the Above Keywords.
PLZ Comment
To avoid some issues with open, etc follow some tips discussed in the comments here
http://blogs.iis.net/nazim/archive/2008/06/30/using-the-new-rules-configuration-in-urlscan-v3-0-beta-part-2.aspx
about the rules for IIS.
I'll get a half decent urlscan.ini posted online at some point. Still waiting for Wade to fix urlscan 3.0 in iis 6 though as I cannot easily test in active production environments atm.
Alberto,
URLScan scans the URL for certain keywords (as the name suggests) so you need to have blacklists not whitelists.
So it was a DDOS attack that actually took your site down for awhile?
I've been seeing these same errors in my own logs but my site is not vulnerable to SQL injection so it hasn't affected me much at least so far. My question is was the DDOS effective because it was raising errors or just because of the sheer volume of requests? I guess they haven't hit my sites as hard as yours since my sites have not been affected other than logging errors, but it raises a question that I've wondered about for a long time. I know that errors/sec is a performance metric we want to be low, but can causing these errors make it take substantially less requests to cause a denial of service than it would if no error was occurring?
I've seen so many variations of things that are apparently designed to cause ASP.NET errors, lots of viewstate errors for example, but one of the easiest ways to cause an error is apparently to make an http request for a protected file like anything with .config extension. I've seen a number of these type errors in my logs over time. I don't have url scan myself, but I guess you could block those type of requests too and keep it from causing errors.
Best Regards,
Joe Audette
ps really enjoyed your recent accessibility podcast!
My wife's (dotnetnuke) site wasn't taken down, but it got a lot slower due to filling up the log. I cleared the log and set it to only retain the past few days and it picked back again.
You are wrong you can stop this attack:
QueryString= 'guid=0b93befc-3543-4bfc-ba8e-6cd340b6d9d3;DECLARE%%20@S%%20VARCHAR(4000);SET%%20@S=CAST(0x4445434C4152452...(incrediblyLONGQueryString)...220%%20AS%%20VARCHAR(4000));EXEC(@S);--', Raw URL='/blog/CommentView.aspx'
with urlscan 3.0
See the sql injection rules for URLscan here:
http://blogs.iis.net/nazim/archive/2008/06/30/using-the-new-rules-configuration-in-urlscan-v3-0-beta-part-2.aspx
Where the keyword CAST is used (which converts the hexstring to ascii) it is blocked (among others). In fact this one attack was stopped (in this example) by a limit to to the length of the query string that is processed before the sql injection rules.
Also you can alter the list to a more detailed on to stop 'false positives' for this.
Thanks
Ruchi
One thing that I'm doing though is logging the IP's but as you mentioned they are probably inoccent drones. However there's an interesting pattern. Two South American countries and the rest from Eastern Europe and Asia.
http://www.west-wind.com/weblog/posts/447503.aspx
No, it wasn't an OrcsWeb attack as I'm hosted on my own VPS and Phil Haack is also hosted on CrystalTech.
Do you have any details on why these request were taking down dasDlog?
Were there just that many of them or is there something that dasBlog is doing with the query that was causing the problem?
Secure-by-default coding and configuration is the necessary approach. It can be hard to think like a bad guy though. Often people ask for a list of how they should code to avoid vulnerabilities, when perhaps a better approach would be to think about how to code to break your own system. As a rule (in my opinion) a combination of nothing-allowed-by-default input validation (with a whitelist of allowed content and types, where it's practical) and parameterized queries specifically for SQL-driven app implementations is good stuff. Neil Carpenter wrote fairly brilliantly and succinctly about parameterized queries recently - check it out. Says Neil: "To use an analogy, using input validation to stop SQL injection is like using anti-virus software to stop malware. It might work, it might not, but you’d be far better off if you actually resolved the vulnerability instead of just trying to mitigate it."
Once you install UrlScan 3.0 as an ISAPI filter you must ensure you have a properly configured UrlScan.ini file (found in the system/inetsrv/urlscan folder).
Ensure the UseAllowVerbs option (found at the top of the ini file) is =1 and add "DEBUG" in the [AllowVerbs] section.
You may refer to this MS Support article but be warned it improperly states adding "debug" instead of "DEBUG" to the AllowVerbs section.
(example)
UseAllowVerbs=1 ; If 1, use [AllowVerbs] section, else use the
[AllowVerbs]
;
; The verbs (aka HTTP methods) listed here are those commonly
; processed by a typical IIS server.
;
; Note that these entries are effective if "UseAllowVerbs=1"
; is set in the [Options] section above.
;
GET
HEAD
POST
DEBUG
Comments are closed.