Random

Brief Update: I’ve been a little distracted.

20 Comments

I know various people are curious about the non-posting. Some are sending me suggestions for ideas. To help those sending ideas understand why I am not setting aside time to give the idea the full consideration it might merit, I thought I’d give a brief update:

  1. Jim Sr’s location: In hospital. Number of emergency room admits in last week? Three. I think.
  2. Rosemary L. (aka, my mother-in-law): Visiting Jim Sr. at hospital — I think. My brother’s-in-law were supposed to pick her up and take here there; I’ve not heard of any change in plans. So, I think that went as planned.
  3. Jim-hubby’s location: At hospital– I think.
  4. Rosemary and Jim’s furnishings: On way to the Devonshire assisted living. Jim hired movers yesterday so we could bump the schedule for the move up from Friday.
  5. Where am I. At home. 🙂 I’m on call to run errands or fill in if things to wrong today. (So far, nothing has but we all thought having a person whose job is to gopher if necessary was important.)
  6. Where will I sleep tonight? Probably at the Devonshire to keep Rosemary company.
  7. Bot banning script: The method is in place and being tweaked. There are several path’s for identifying ‘bots and banning. Path 1 intercepts requests things that don’t exist. This looks very, very effective at saving resources; cpu, memory and bandwidth used to deliver pages to bots trying to crack in and wreck havoc are way down. (This is not just due to reduced traffic related to non-posting. The ‘bot crack-in attempts are way down.) Path 2 looks at requests for existing posts after they are delivered is working pretty good. It’s main purpose is to block the better scrapers, spoofers and referrer spam. (Uhmm. In the process of doing this, I’ve looked at some referrer spam. Those of you who know, know what this means…).

    I’m trying to decide if I even need to do work on what I’d imaged was required for full implementation of “path 3” which relates to image scraping. I did what I considered a first round of diversion and scraping is already way down. (But there are issues when images are hotlinked; I might need a tweak. Thank to all who reported whether they could see blue, lavender, yellow, pink or whatever colored triangles. That really helped me get data on how the diversions were playing with Cloudflare. I couldn’t do that without visitor feedback. Oh…. if you notice images hotlinked from my blog not showing other sites, that’s the thing I need to tweak. I’m going to create a whitelist so images at the Whiteboard and a few other places don’t seem to mysteriously appear or disappear ‘depending’. )

  8. Since the method of reducing ‘bot resource gluttony is effective, it can now be “plugin-i-fied to share with wordpress users. I need to think through the best way to do that. There are tradeoffs between “most secure and effective” and “easy for the computer illiterate to load and use”. Some thought is required.
  9. Some of you experienced more widespread outages recently. Why? Dreamhost was down periodically. Oh. And their ftp passwords were (or might have been) compromised. See FTP passwords, emergency maintenance and Major network outage (RESOLVED). (I actually like Dreamhost. Long ago I had a different host that never communicated. )
  10. Apologies for banning scores of you for roughly 2 hours yesterday. If you were banned, your “sin” was to try to look at “YellowImage.jpg” — the image that was never there. For about two hours a bug I had fixed but re-introduced deemed trying to load images that were not there not-a-good-thing and told Cloudflare you were evil. Luckily, Carrick let me know, I fixed the bug and manually tried to get those affected de-banned from Cloudflare. (Debanning at Cloudflare is a bit of a PITA; they promise to improve this feature. If someone complains elsewhere, tell them to email me and provide their IP which the Cloudflare ban page should tell them.)
  11. The two cats including the fat diabetic one are fine!

Not much more to say. Glad to see you are all chatting away. Open thread.

20 thoughts on “Brief Update: I’ve been a little distracted.”

  1. Lucia –
    I vote you give yourself a ‘hosts’ extension and put a couple of Blackboard quatloos on the annual anomaly. Or just mention your estimate for the other punters edification…

    Best wishes in all the too-ing and fro-ing.

  3. Lucia, you have your priorities correct. While people’s situations are never the same I have been through similar circumstances and I know that some members of the families end up shouldering a lot of responsibilities.

    If people are suggesting ideas for a post why not have an open thread on a technical subject. I would suggest some here but I tend to be a thread killer.

  4. Would all of this tweaking explain the required Captcha log in yesterday? The message said something to the effect that I was being allowed temporary access and that my behavior was being monitored…?

  5. Duke C–
    Do you mean the banned page at cloudflare? I think the answer is yes. What happened was:
    0) I intercept all requests for uri’s that don’t exist. (That is: if you try to load an address that corresponds to no post, image, pdf, css or anything, I intercept that. After intercepting, I analyze to see:

    1) If people try to view blog posts using protocol http 1.0 headers I ban them. In practice, this never affects people (who use “normal” browsers) but does catch spamforum bots. (I catch about… oh…. 10 spamforum bots a day this way. Each wants to race through and load a bazillion pages.)

    2) If people try to download images using protocol http 1.0 headers, that’s ok. People download images that way all the time.

    3) I noticed that 1 count them 1 person got banned for downloading a missing pdf using protocol http 1.0, and decided that should be ok.

    4) I went in, added 2 lines of code, cutting and pasting. I made a cut and past error, did not proofread adequately, did not post test adequately, and went along my merry way.

    5) About oh… an hour later…. Carrick emailed me. He’d been banned. I looked at why he was banned, said “D’oh”, fixed the bug, and then started to manually unban people.

    6) While unbanning, my emails was going “bing-bing-bing-bing” as a dozen or more people let me know they were banned too. One emailed Jeff Id to find out what happened. The content was

    Is it just me or are a lot of people having trouble accessing the blackboard. I went to read a complete story and all I get is you are banned. This is hardly appropriate when I haven’t done anything. Any ideas?

    I had to agree. It was hardly appropriate. I don’t remember what exactly I replied to Jeff– but basically, I’d screwed up.

    7) I tried to finish manually, answered the emails and told people to see if they were unbanned. (Unbanning lots of people quickly at cloudflare is a PITA and error prone.) A few more people told me they were still banned– and when they provided IPS, I unbanned them.

    BTW: I don’t quite know what sort of “monitoring” cloudflare does. I think they just try to see if you start hammering away at a site. The try to ban IPs that do that.

  9. Duke C. Cloudflare doesn’t seem to customize their pages. It’s true that sometimes the banned page appears because they detect that you sent some sort of gawd-awful request that indicates your pc might be infected. But yesterday lots of people were banned from The Blackboard because they tried to look at “YellowImage.jpg”. This had nothing to do with a virus.

    Mind you, it’s possible that your pc has a virus and everyone else was only banned for trying to look at “YellowImage.jpg”. But if you are visiting today, it’s probably just the whole “YellowImage.jpg” thing.

  10. Lucia,
    Best wishes to you and your family through these tough times.
    We are in a similar healthcare pattern at this time, minus the assisted living/nursing home dimension. As to the tech challenges, if anyone is up to it, it is you.

  11. Lucia,
    One more question:
    Do you still have a store offering those nice mugs and other mementos?

  12. Chuckles,
    Somehow, using a Playstation as a browser seems impressive. Maybe a clearer perception of the activity than our’s?

  13. hunter– right now there are 4 mugs, all are “by Josh”. I need to make a year end temperature trend mug. But… not today….

  14. j ferguson–
    The person using a playstation came from an image search service and only looked at an image. I suspect it’s not a good browser, but it can browse.

  15. I bequeath you ten of my quatloos for keeping your priorities straight and keeping the Blackboard train running on time, as well as your service to others in bot slaying, Lucia.

  16. The email to JeffID was from me. I like to read the blogs and keep informed. I won’t comment on the technical stuff as I have no credentials to do so. Thanks for explaining what happened. I enjoy this and other blogs, there are 2 I don’t read, real climate and the other one where the blog host changes comments of readers.

  17. genealogymaster –
    Nice comment.
    I like the reference to the other one. It reminds me of he who shall not be named from Harry Potter.

    For my sins I sometimes read real climate – it is good for my soul and it keeps me from becoming too partisan. I find I can’t learn anything there unless I keep something of an open mind…

    Like you I have no credentials to comment on the technical stuff…….but it doesn’t stop me 🙂

  18. Re open thread –

    I know it has been discussed before about absolute surface temperatures in models being off and so comparisons have focused on the changes in surface temperature. However, I’ve been doing some fairly straight-forward global averaging with respect to the net TOA flux, and I’ve been a bit surprised to find that these values are all over the place in the CMIP3 models I’ve looked at:

    http://troyca.wordpress.com/2012/01/27/radiative-imbalance-in-cmip3-models/

    I was wondering if there’d been any discussion about the absolute net radiation balance being off in models? In 8.3.1.1.2, “The balance of radiation at the top of the atmosphere” of AR4 there are places where they mention how the zonal averages are off (with LW compensating for SW), but there’s no focus on the global energy imbalance at TOA, and I’m not sure how a physically realistic model could show a century of increasing temperatures with a negative net imbalance, as the GISS model runs seem to show? That Hansen (2005) paper seems to show the same results as mine, except that the net flux is zeroed at at the start of the run, so is Hansen simply treating the net TOA flux as something that can be re-baselined? That would seem at first glance to be more problematic than simply re-baselining surface temps, as the *absolute* radiative imbalance is what determines the rate of temperature increase…is there a constant flux adjustment made throughout the time period (the previous section in AR4 seems to suggest they have done away with the flux adjustments)? Am I misinterpreting the results from these model runs?

    Thanks to anyone with more info on this.

  20. Les–
    Looks like we passed the test! Now ZBblock only checks the stop-forum-spam list if someone tries to log in, register or do something like that. This should also reduce the wait when people comment.

Comments are closed.