The police announcement about closing the Climategate investigation turned my mind back to hack/spam protection. A few of you will have noticed I’ve been fiddling with the hack protection. I apologize to any who were presented the “scary page”, especially the person using a proxy.
Just in case my new rules are bad I like to let you know about the recent batch:
- I tweaked so ZB Block now checks with a list of TOR exit nodes when people comment. This should bounce people who are commenting using TOR exit nodes. If you don’t know what these are, you are almost certainly not using one. (TOR is popular with virulent trolls.)
- For roughly 30 minutes, I tried blocking everything with the word ‘proxy’ in the host unless it also contained ‘.googlebot.com’ in the host. I quickly learned why this is a very, very, very bad idea.
Those wondering why I would even consider such a move can learn my motivation by scanning can visit this page which shows quite often, hackers, bots etc. hide behind hosts with the word “proxy” in the title. Everything on that page got caught for violating some rule other than containing the word “proxy” in the title, so I thought I’d give blocking ‘proxy’ a shot.
I knew this was iffy. So I added the rule and checked the kill_log.txt file every 5 minutes. Unfortunately, I quickly also saw that hosts like “proxy.a.perfectly.valid.business.com” in the killed logs. I’m sure that was a human. Not only human, but a human who was smart enough not to reload the page 3 times– so they didn’t get themselves banned! If you saw that… sorry. I won’t be blocking you and your coworkers for using a “proxy.a.perfectly.valid.business.com” in future and you should be able to come back.
- I added a rule that notices when someone makes a direct call to a ‘theme’ that does not exist at my site and bans that IP. (The theme is what makes the blog look as it does.) All blog visitors call the theme but no blog visitor should ever even think of trying to call a theme directly and they certainly shouldn’t call a theme I don’t even use. But bots rather frequently call non-existent themes associated with WordPress vulnerabilities especially the ones associated with the “timthumb.php” or the “uploadify.php” exploit. I’m now catching and banning these very early on.
- I added a rule that notices when someone tries to call a non-existent ‘plugin’ directly and bans them. Plugins add functionality to WordPress- for example, one plugin lets you edit comments; another fishes out the recent comments for the sidebar and so on. But people shouldn’t be calling plugins I don’t use at this blog.
This rule makes me a little nervous so I’m watching to see if it affects any humans. So far, it hasn’t. If you see the “scary” page, I apologize in advance. Send me an email and I can fix it. (I think you won’t see the page by accidentally violating a badly written rule but sometimes I only recognize the flaws in the implementation after I implement a rule.)
- Sometime last week, I got sick and tired of my killed_log.txt file filling with Brazilian’s pretending to be googlebot and banned Brazil at Cloudflare. This means people in Brazil will be required to fill out a Captcha. People in China are also required to fill out a Captcha. (To see how constant the Brazilian Googlebot spoofing was, visit googlebot spoofers; the letters “BR” stand for Brazil. If you are planning a trip to Brazil, let me know and I can probably whitelist your hotel’s IP.
- I created pages to permit me to search my spam logs for IPs, hosts, and various search terms. I’ll be displaying some of these from time to time in blog posts at the most boring blog ever, Ban Nasties.
- I’ve been adding large blocks of IPs from troublesome sites to the range of IPs blocked at Cloudflare. The trouble spots are generally associated with hosting services and cloud networks. Most people surf the web using ISPs; so blocking the more troublesome servers should affect a small fraction of potential visitors. But if you use a VPN you might have a difficulty. If you do, let me know. I some cases, I might whitelist your IP even if the IP you present is associated by a company from which lots of hackage emanates. In somecases, I might suggest you switch IPs (as some VPN hosts permit.)
I should be back to climate blogging tomorrow or Thursday.
Good luck protecting your site. I have found valuable information here and hope to continue mostly reading but not commenting here.
Thanks Rob.
Some of the protection is against things that are just voracious. SEO bots seem to want to index every page on the planet numerous times. Some companies operate by letting other companies instruct their crawler– so the same crawler can be visiting a site for “company A” looking for one thing and for “company B” looking for something else all during the same time period. This just hogs server resources.
But other things are more nefarious. I have no idea what is up with the Googlespoofers. I just know they aren’t Google and there are tons of them. So… bye, bye, Brazil!
I also told Cloudflare about the googlespoofing issue. I got a reply and I think their engineers may start screening for google spoofs. A lot of data passes through Cloudflare and they should be able to inhibit some of those googlespoofers who operate out of a few specific ISPs.
test.
You passed! 🙂
This from twitter:
@ScotClimate: Scottish Government found to have lied on key figure. Is the Scottish Climate Bill dead?. Will the minister resign? http://bit.ly/OwkVl1
The Scottish government lied to politicians about key financial data which was central to the argument for the bill when they passed the Scottish Climate Change Bill. The government citing Stern said that the economic cost of a 2-3°C rise would be “between 5-20% of GDPâ€. In fact Stern suggests there may not be any net economic harm quoting figures of 0-3%
The figures are so key to justifying the bill, that it really is difficult to see how this bill could withstand a legal challenge.
… but the scandal gets worse. The Scottish paper (The Courier) which broke this story seems to have been lent on to remove the story. Presumably by someone in government.
This is about as bad as we can get. It appears the world’s most enthusiastic government for climate change is now embroiled in lies & cover-up.
I should probably delete ‘test’ comments.
You just reminded me of the massive headache I’m going to have soon. I’m building a couple small websites for people, and it looks like they’ll want me to set up (and possibly manage) their servers as well. Since I recommended web hosting, I’ll be in about the same boat as you.
Stupid script kiddies make the world a much worse place.
Brandon–
You said it!
It’s pretty easy to see that most of what I am blocking is harmless– but bandwidth/cpu sucking. Also, the fact that they are there in the logs can complicate noticing the actual crack attempts. That’s why I got sick of the Brazilian googlespoofers. The spoofing is 100% detectable but it was just so much! I’m hoping Cloudflare deals with that. I suspect they will. It’s just the sort of screen they can set up easily and that 100% of their customers would want. Of course, the Brazilians might just switch to spoofing something else– but at least what claims to be google will be google. That makes things easier.
I’m curious. You are going to great lengths to protect your WordPress blog. But if a hacker penetrates WordPress via someone elses blog, doesn’t that potentially allow the hacker into yours? (May be a dumb question, but whatever …)
jim2, that’s not a dumb question. The key here is lucia hosts the blog herself. This means it isn’t on the WordPress servers. If it were, she wouldn’t have to deal with as much of this stuff as she does, but she would be exposed to the risk you bring up (though one would hope WordPress has good security on its servers).
lucia uses the WordPress software, not the WordPress servers.
jim2
Not a dumb question. But the answer is no because I’m self hosting.
In contrast, Anthony, Jeff Id, SteveMc host at wordpress.com. If someone hacks into wordpress.com, potentially all three of them have an issue, but I don’t. On the other hand, I have to watch my server logs.
I like having my own site for a variety of reasons– some historic. I still have a knitting site. I can host things like “sockulators” in the blog and do the uah betting because I self host. I couldn’t do that hosted at wordpress.com.
BTW: It’s kind of fun to watch the sever logs today. The ‘most boring blog ever’ has no traffic. So… after after dinner I went to see the server logs. After I published this post here, Goggle found my links to my other pages over on that site and raced over there. The it hit all the ‘follow’ links on that page. Wow!
Good thing I coded that to ‘nofollow’ most the links or I would have sent Google into a frenzy!
Brandon and Lucia, you must be mistaken. Those can’t simply be script kiddies. It’s obviously a “sophisticated and carefully orchestrated attack”
John, I doubt you have the highly skilled training that the Norfolk Constabulary possesses to make such a technical comment. 😉
[Just as I rely on bobbies trained to cite you for jaywalking for my analysis of hacks and computer security breaches, I also use the guy who changes the oil in my car as a paid consultant to write critical 3-d fluid mechanical software for me.]
John Vetterling, unfortunately, script kiddies are capable of performing a “sophisticated and carefully orchestrated attack.” A primary aspect of script kiddies is the reuse of code. They take code other people have written, perhaps make some tweaks, and run it. If the code they’re copying from can launch a sophisticated attack, they can launch one too.
As for carefully orchestrated, I’m not clear on just what that means, but botnets are a common tool, and I’d think they’d qualify.
What a mess, coffee all over the place!
Best Always!
tweaked to check at stop forum spam. test
Lucia I’m getting a mostly blank page that says:
502 Bad Gateway
cloudflare-nginx
I’ve been able to access various pages but I think this happens when I use the back button.
I’m coming in from Berlin/DE via a company proxy with a Firefox 14.0.1 browser.
I also noted something strange with the posting order since Steve passed the magic 100K mark. The 1xxxxx posts come first and then the older posts. This is using the CA assistant which does reordering and I’m guessing that the problem is there but I thought I’d mention it in case others see the same thing.
bob
Interesting is also that this is the first time I’ve been moderated, perhaps that’s a side effect?
bob
bob–
Moderation can be weird. I don’t know why that happened to you– but it has nothing to do with my changes to ZB Block. The moderation is by Akismet and WordPress core.
I don’t know what ‘502 Bad Gateway
cloudflare-nginx’ means but it seems to have to do with cloudflare. I’m not sure it has anything to do with my spam software– but let me know if that persists.
I don’t use CA assistant myself so I don’t know whether it has any oddities with numbering. It might have. I’m unique in showing raw comment numbers. Most people use dynamics numbering so you see comment numbers like 1-400 and so on. Pete (author of CA assistant) might have assumed comments would never hit 100,000. If so that’s something that would need to be fixed by Pete. (Anything I might try would make matters worse rather than better.)
I use static numbering because occasionally comments get deleted and some confusion can ensue when people say something about what they read in comment ’20’. When 19 was deleted, 20 become 19 and confusion ensues.
By the way, I have discovered that nearly everything that leaves “bad cookies” and shows a 2nd IP in the X_FORWARDED_FOR headers turns out to be an anonymous proxy.
Re: lucia (Jul 26 12:12),
I’m not sure what the problem with the blank page is, perhaps it is local (I’m not seeing it from home) I’ll check again tomorrow.
My moderation was because I used my real name instead of my alias, silly me!
I’ve reported the problem with the comment ordering at CA and another user has seen it too. I expect the problem is a bug in the extension as you suggested.
Thanks,
bob
lucia (Comment #100017)
I use CA Assistant with Firefox, and have always gotten the static comment numbers. Which is good.
schnoerkelman
On days when I’m fiddling with customsig.txt you never know….. There could be some weird interaction with my having changed things or you hitting the page just as I’m “saving” a php file you need to load. I just don’t know.
Re: AMac (Jul 26 14:19),
If you click on my comment (or yours or Lucia’s) is it displayed at the top of the page or the bottom?
Currently I see (from the top): Posting, 100005, 100006, 100017, 100020, 100022, 100023 then 99930, 99931, …
Re: lucia (Jul 26 14:25),
The strange thing this morning (my time) was that I could click a comment and see it then if I used the back button I got the error message. I got the error message on the top level as well (…/musings/). This was with the firefox browser I always use. I sent the first two comments with IE and that was working fine. It could be something on the internal firewall at my end but it was strange since it mentioned cloudflare-nginx. As I said earlier, I’ll check tomorrow morning and see if it’s “fixed” by magic over night.
schnoerkelman (Comment #100024)
Today, my viewing experience is similarly strange. At the moment I see: 100005, 100006, 100017, 100020, 100022, 100023, 100024, 100025, then 99930, 99931, …
The way you break down an alibi or cover story, is by going over all the details. When I’m told climategate was the work of a “highly sophisticated†and “highly competent†hacker, but then look at some of the details, like the footprints left all over the job, I’m unconvinced, to say the least.
http://thepointman.wordpress.com/2012/07/27/so-was-climategate-a-hack-after-all/
Pointman