This is an open thread. But I want to tell people what’s been up on the back end of the blog.
Have some of you seen internal error pages? Been presented the “scary page” after commenting? That was my fault. Thanks for emailing me.
A variety of temporary glitches were introduced as I edited both the ‘.htaccess’ and the ‘zbblock’ files that I use to deal with bots. I did something of a ‘re-organize’.
Why?I’ve been dealing with what appears to have been a fairly low-tech, slow, d-semi-dos; that is: it seemed to be ‘distributed’ and but seemed to only semi-deny service. So “distributed-semi-denial-of-service. The main features were/are
- A specific old post was requested roughly every two minutes. (The request rate is now reduced.)
- The overwhelming majority of requests come from thousands of different IPs associated with server farms or some sort. Of the remaining, most requests come from countries with reputations of hosting quite a bit of hacking/spamming. A small amount does come from connection providing ISPs in ‘mostly clean’ countries.
- Some of the same IPs would return and ask for the post after a period of an hour or so. These ‘slow’ returns any individual IP makes it difficult for an upstream CDN like Cloudflare to detect its undesired traffic.
- There were/are features that make this seem like it might be ‘personal’ (in some sense) rather than just “garden variety script-kiddy bots who just look for low hanging fruit”. That said, it’s really hard to say.
- During one period, at least made several requests that revealed x-forward headers which included a second IP: 198.50.228.116. This IP is at AS16276 OVH and whois tells me it’s a “Private Customer” in “Sofia, BG”. So, I’ve nicknamed the person (or group of person) hitting the blog “Bozer and his Bulgarians”. (This is not to say that I actually suspect this originates in Bulgaria, but who knows.)
Although this behavior was not going to take down the server, it was pesky.
But I also decided the pesky behavior could be turned into an episode of using lemons to make lemonade: it gave me a chance to identify all the new ‘pesky servers’ and update my list of ‘currently active bad servers’. I hadn’t done that in… oh…. a year or two. So, it needed doing.
Because if the features of these connections, I also changed strategy for dealing with these IPs. Previously I dealt with all the proxyIPs in ZBblock and ultimately banned some at Cloudflare. I now detect a sizable number of pesky IP ranges and do
RewriteRule ^(.*) http://%{REMOTE_ADDR}/ [L]
in htaccess.
This rewrite rule sends the request back to its originating IP. The current rules are likely over inclusive, and I’ll be backing some off over time. If you or someone you know runs across it, you (or they) will be told you are having trouble connecting to the site and your IP will be displayed to you.
A few countries are also currently blocked at Cloudflare. Some of these countries will be unblocked by the end of the week, others will not. (China will never be unblocked. Sorry.)
Of course, somethings still only get blocked in ZBblock. (Last night, people who commented were blocked. Sorry!)
If you see any ‘block’ page, email me. Or tweet me. (Of course, I’m aware if that’s happening to you, you won’t read this post. So, it’s a bit of a catch-22. But if someone else tells you they are encountering the problem, have them email me. )
If someone does ask me to let them through, I will ask them their IPs to help me fix the problem for them– either by opening a wide range or opening up a specific one for individual static IP. If they refuse to provide IP — as some people do, I will be unable to fix the problem. If– for some mysterious reason — they insist on connecting through Tor, a vpn or some server farm, and lecture me on how I should permit them — and everyone else on Tor, vpns, or serverfarms– to do so I will tell them to pound sand. I know perfectly well that they have a non-Tor/VPN/server-farm IP they use to connect to Tor/vpn/serverfarm. If they are too stubborn to use that to read my blog, I’m too stubborn to let them use Tor/vpn/serverfarm.
Likewise: I am probably blocking most rss feeds. I can– over time– open up some of these as I identify which need to be unblocked. However, from my point of view, preventing the D-S-DOS is a higher priority than unblocking feeds. If you the rss feed you prefer is currently blocked and you want it unblocked quickly, you will need to email me, ask me, and– possibly– provide information to help me identify the IP ranges/user agents etc. that particular feed uses. Not to sound too snotty: but if you aren’t willing to do some digging to supply me with information regarding the feed you prefer, it’s not going to get unblocked quickly. There are tons of things hitting the feed, not all are feed readers.
Obviously, as I am currently blogging lightly, I don’t expect to be overwhelmed with request for connection clearing feed etc. But if your university, country, feed etc. has been blocked let me know. If it’s easy to fix, I’ll fix it. If it requires info to fix, I’ll assign you the task of getting the info, and then fix it.
Anyway, for now, the d-semi-dos seems to have slowed down. The effects of its main strategy seem to be neutralized. If it is a person and it is personal, my posting may cause it to change strategies. If it’s just a script-kiddie, it’s taken care of.
Either way, open thread.
Gosh Lucia.
Do I hit the Blackboard that much? I was rereading some old material, and then after that the old page was cached as a convenient link, so I’d hit it a bunch instead of the ‘front door’.
Sorry about that.
Do I actually touch my face 2000 times a day as well I wonder, as claimed in the movie ‘Contagion’?
I don’t know about the rest of that stuff, but I might have had something to do with the old post part.
I’m not Bulgarian as far as I’m aware anyways. But how on earth did you know about my nickname? :p
You got me scratching my head over this part:
Maybe my behavior was just coincidental. I don’t do TOR or anything strange; no reason I can think of this should be the case due to me.
Mark–
I assure you it’s not you hitting the post in question! Or at least if it is, you are saving one ‘good’ IP to read and post comments. If you are hitting with IPs from “quadranet”, “colocrossing” and so on, you are a very bad boy.
Some people tend to be unwilling to believe there really are bots out there. But there are!
Thanks Lucia, good to hear it’s not me gumming up the works. 🙂
Mark,
The only real negative is that right now I may be blocking some people who surf from work. But that’s always been the case. Some people connect through their company IP address and those might be coming out through a server IP. Not all though– many smaller companies use things like comcast ATT too.
But the thing that was hitting that particular post… it wasn’t ‘real people’. It was definitely bots.
Ohh… It’s either Bozer or one of his Bulgarians again! And the x-forward stuff works in .htaccess!! Yippee!!
pre #: OvhSystemsCanadaForwarded_AS16276 @: 02/23/2015 02:02:24 pm – Running:
Host: atlantic480.us.unmetered.com [XX] CFRay [1bd6b51a046515b3FRA]
IP: 188.138.9.49
Culprit: OvhSystemsCanadaForwarded_AS16276
Referrer: |httpknittinghaiku.com| 73 s
:
Via: 1.1 vps26422 (squid/3.3.8)
X_FORWARDED_FOR: 198.50.228.116,188.138.9.49:
Note they presented “188.138.9.49” but got 302’s because they were one of the Bulgarians!
May be unrelated, but this morning I tried to access the blog on my Android, and got a Verizon sign on screen. My carrier isn’t Verizon.
What do you mean by a “Verizon sign”. That is: did you get a long winded very apologetic text message with at least 2 paragraphs that happened to have the word “Verizon” in there? Or did you get some sort of graphic showing Verizon?
If the latter that has nothing to do with my redirects or scripts. If the former, that shouldn’t happen because “Verizon” is not blocked or redirected. But also, if the former, I can’t imagine you’d call the very long winded apologetic text a “sign”. A “message” maybe, but not a “sign”.
Anyway: which? Because right now, it sounds like “nothing to do with me”, so I’m not going on a bug hunt! But if it was “a long winded message”, I need to go on a bug hunt.
🙂 I love it when things work too.
Gratz Lucia.
It was a sign on screen. Asking for a login ID and password, and that’s all.
Now that I think about it, I was on my home WIFI, which is Verizon. But it’s the only time I’ve ever seen that screen, and I was doing a lot of other surfing with no problem.
John M —
I had the same thing happen a week or so ago, also using home Wi-fi which is Verizon. The URL “rankexploits.com/musings/2015/topic-title” had been morphed into “IPv4-address/2015/topic-title” (or something like that, e.g “1.2.3.4/2015/topic-title”), where the IPv4 address was my router’s external IP address. Hence the sign-on screen: the URL referred to a page held by the router.
Lucia’s explanation above is that her scripts re-direct back to the originating IP, which is my (or your) router’s address. So that quirk — which came and went rather unpredictably — is now explained.
Since this is an open thread: Mike Mann has a new post at RealClimate discussing his paper in Science and claiming that (shocking surprise!) there is not (and never was) a pause, and further, rapid global warming is sure to return very soon. The phrase “double-down on his bet” is too weak a description. He is going to look pretty silly if the recent relatively slow rate of warming continues for another decade, as seems likely.
Good luck dealing with this. I have a feeling this is not going to end well.
SteveF, the odd part is I don’t see very much daylight between what Mann did and what Mann so strongly berated Curry for doing.
Mann calls it a “fake pause” which is wrong of course.
Either way it’s a real pause, hiatus, slowdown. If you can measure it, it’s a real effect, not a “fake” one.
SteveF (#135515),
I haven’t yet read the paper. But I notice that Mann has an interesting way of spinning. He writes, “[T]here is not currently a “pause” in global warming.” The abstract is a little less spun, referring to a “‘false pause’ in warming,” and allowing that the AMO/PDO have “produce[d] a slowdown … in warming of the past decade.” So, it’s a war of framing: “slowdown” is OK, “pause” isn’t (unless qualified by “false”), we’re left unsure about “hiatus”. Whether the article is any improvement on e.g. Yao et al. remains to be seen.
According to the RC post, the authors computed natural variability by using the CMIP5 models to compute the forced response, and subtracting that from actuals. The result, Mann writes, is that during “recent decades” — presumably 80s/90s — the temperature rise was not enhanced by natural variability, but “internal climate variability instead partially offset global warming.” [Emphasis in original.] Sounds circular to me…if one assumes that models’ TCR is correct, then one finds that the TCR was not enhanced by tuning to follow the 80s/90s rise.
We’ll see. Any analysis which starts by assuming that the models’ sensitivity is correct, seems of dubious value when the key question is whether the models’ sensitivity is correct.
I believe that’s called ‘begging the question‘.
SteveF,
Yeah. I haven’t read the paper yet.
Doug McNeil and co-authors wrote a paper looking at statistics of weather -in-models. Assuming model weather is ‘right’, they conclude there is a 16% chance (I think) of hiatus continuuing 5 years. (This is a conditional probability– basically, given there already was a hiatus, what’s the chance it will continue.)
The McNeil paper doesn’t seem to diagnose whether the current ‘lull’ has the signature of being ‘weather noise’.
That’s a bit different from Mann’s approach which is to try to diagnose that we are in a particular type of lull based on earth temperature data– and so Mann is saying the current lull is “weather noise” masking.
In both cases, you get an “if models are right, then after a hiatus, warming tends to resume, and at a faster rate.”
Of course it is true that if models are right, then we are “due” for warming.
Anyway, obviously, the hiatus whether “true” or “false” or whatever word mann is applying is obviously giving modelers the heebee jeebies. We are at the point where either it needs to end or people are going to have to admit the models are off. By the same token, if it does end, people will have to admit that it was strong natural variability masking things.
I’m betting: we will see a warming trend over 30 years. It going to be less than 0.2K/dec. Tom Fuller is going to win Joe Romm’s money. (The latter will have surprised me. I advised him not to bet.)
Pause is real ( and evident ).
But, the (insignificant) negative temperature trend since 2001 has gone (insignificantly) positive for a number of data sets, with the passage of 2014.
Fifteen years is still brief – got another fifteen to wait?
To those wondering
The quirk may blink on and off for you. I’ve neutralized the Bulgarians, but I’m reorganizing the way some things are done. I test an idea, if the idea traps bots with few false positives, I do some fiddling to make it not take to many resources.
While doing so, I make some mistakes (both typo and conceptual.) Even if I make no mystakes, there is a 2-3s period when .htaccess is being uploaded during which you will get see “internal error”. I see it a lot because I reload after uploading (to test for typo mistakes). But you may see it too.
If you look at the AMO index, it’s barely peaked and hasn’t gone negative yet. We could see ten to twenty years more of slower than predicted increase before the greenhouse warming overwhelms it.
The greenhouse warming has not overwhelmed anything for 15 years so there is a problem in feedback accounting, not natural variability,which most people seem to be in denial of.
Mann does not want it to exist. How funny to admit it exists when denying it exists as in “there is not a pause, and further, rapid global warming is sure to return very soon “.
Not sure now what Lucia’s definition of a pause was but it seemed to fit Mann’s “the pause you have when you are not having a pause”
The greenhouse warming has never “overwhelmed” anything. It is an integral part of the climate system from the beginning. It is the alarmist paradigm that is overwhelmed by reality.
The alarmists should not be allowed to frame the discussion of climate any longer.
It seems clear that the contorted framing (‘false’ pause, ‘temporary’ hiatus, etc.) is the result of heebee jeebies, and not just among modelers… just as much, or more, among the most alarmed of the alarmed. There is a fundamental difference between people who see that long term warming of 0.1C per decade instead of 0.25C is important for public policy decisions, and people who think the public policy (stop fossil fuel use!) should be the same independent of the rate of warming…. indeed even if there is no significant warming. It comes down to values, priorities, goals, and beliefs; the disagreement is virtually divorced from technical questions like “how much warming will there be?”
DeWitt,
Yes, long tern pseudo-cyclical behaviors have probably masked some of the increase in GHG forcing. But I think we need to remember that those same behaviors almost certainly added substantially to the warming between the mid 1970’s and the early 2000’s. That models mostly match the warming during that earlier period, but now diverge is a good indication of how far the models are ‘off’ in diagnosed sensitivity. Papers like Mann’s in Science (and many other similar papers) show just how important the meme of ‘extreme future warming’ is to those anxious to implement draconian public energy policies.
If the true long term rate of warming is half what the CMIP5 ensemble says it is, then draconian public policies are just not going to to happen. Hence, gnashing of teeth and tearing of hair…. and a blizzard of papers offering ‘explanations’ for slower warming, none of which ever entertain the most obvious: the models are much too sensitive to GHG forcing.
I agree with hunter. “Overwhelms” is non-science and even worse poetry. Only a devout Warmer would even imagine such a thing.
Andrew
Just for another perspective
I was in Caracal, Romania (100 miles outside Bucharest) for the last two weeks in February. I would access this page probably every other day. Every time before loading the home page, i would get a gateway page (for lack of a better word) asking for the number on the graphic. It always worked for me.
Chris in GA,
That’s what’s supposed to happen when a country is “banned” at Cloudflare. Glad to hear it works.
The ban is slowing down Romanian bots. It’s regretable people need to fill out the captcha, but….what are ya gonna do?
Captcha! That’s the word! A minor inconvenience for security – sorta like the TSA.
Andrew_KY,
Thanks for the comment. The framing of the climate obsession is largely non-rational: Non-quantified risks that must be stopped no matter the cost; Frequent rebranding of the obsession’s name; Hatred of those who largely agree but differ on relatively minor details; inability to actually debate their position in open forums; reliance on demeaning and dehumanizing those who disagree; calls to silence all who disagree.
Dr Liljegren:
Are the attacks that hit your server unique, or are they widespread?
Is there a way to identify motivation?
I feel really naive on what you are confronting.
Oh, the urge to spread commas randomly was suppressed.
Thank you
Generally speaking, attacks of various sorts are rampant all over the web. I don’t know about this particular one which seems to have somewhat odd feature– but possibly if I knew other web admins, it would turn out this, too, is common.
Generally, the way to identify motivation is to first trace and identify the person or persons attacking and then interrogate them. This is not practical. The other way is to allow them to hack you and see what they end up doing after getting ‘in’.
Lucia,
Or let them hack some part of your system like a honeypot and see what they get up to in there.
Mark
I have honey pots all over the place. But I don’t know you you can diagnose motive with honey pots.
Well…. I guess unless you make a honeypot go to a section that looks like store with data base with fake credit card number blah, blah and see them hack in and “steal” them. But that’s more than a honeypot, and it’s awfully complicated.
Besides that, catching one entity doing A doesn’t tell you anything about the entity doing “B”. I still see vulnerability scanning. That’s separate from this current thing. It’s quite likely the scanning is done by a separate person/group and that person/group could have different motives.
I see.
Hmm… what I said about it being more than a honey pot is wrong. I looked it up and that is a honey pot.
I have simpler things– hidden from human links in the page html. Some crawlers try to load all of them– then I ban them. I do identify quite a few scrapers that way. They tend to be on servers.
That said: I am guessing those are “scrapers”– as in their motive is to scrape. I don’t really know that. But all in all, it would cost me in computer resources to let them continue to crawl until they “found” something they “wanted” and started to try to steal that precise things. Beyond that, even if I discovered what they wanted, I might not catch them, so to some extent…. what’s the point?
Some companies find it worthwhile to set up complicated things, but it’s not very useful here.
Absolutely; the expense in time and resources to figure out what intruders are up to needs to be justified.