I’ve been working on a form that will permit people to de-ban themselves at Cloudflare. The form is currently skanky revealing, but I think functional. I’d like to request people who tolerate skanky looking forms to visit the unban request form, fill out the boxes above the “submit” box and then click submit. (Do not change anything in the boxes below the submit box.)
If you fill the form out out correctly, you should get message sort of like this
Three IPs 99.150.207.90 =? 99.150.207.90; unban this: 99.150.207.90
You entered the correct captcha challenge; but lots of ‘bots can be programmed to memorize. Now I’ll check whether you can add.
You can add! You look human so far.
Now I’m going to check to see if you took between 5 seconds and 125 seconds to submit a correct form.I’m going to send an email message to xxx@wherever.com
It won’t really send email.
What I want you to do is tell me whether the form said you’d filled it out correct when you had. Also, let me know if any of the captcha’s seem screwy. (Note: Their contents should match the “hints” right to the left of each box.)
If you want to help on a different project, before testing the form, visit picscout, download, install and activate their add on and fiddle with my form while the add on is activated. While doing this, wait for the little whirlie thing in the right sidebar to finish whirling before clicking submit. After you are done, turn the add-on off and uninstall.
It said I failed the captch, plus it gave the answer to the addition.
All looks good:
Three IPs 82.24.254.230 =? 82.24.254.230; unban this: 82.24.254.230
You entered the correct captcha challenge; but lots of ‘bots can be programmed to memorize. Now I’ll check whether you can add.
You can add! You look human so far.
Now I’m going to check to see if you took between 5 seconds and 125 seconds to submit a correct form.
I’m going to send an email message to
etc.
Mike N–
Right now, it’s giving people “tips”. Those are so I can just sit here running it over an over again seeing if I get bounced and if I do I can check whether I actually typed wrong.
So…. did you enter the wrong answer? Or did it say you gave the wrong answer when you’d given the right one?
Lucia,
I’d suggest that an ‘unban-request’ form needs a ‘Donate’ button, that needs to be primed with a suitably plump donation before the form becomes active?
Wasn’t it R. Nixon esq who said, ‘when you’ve got them by the *****, their hearts and minds will follow?’
On the Captcha side, I’ve noticed with a number of Captcha systems that they seem to hand out the same Captcha to all arrivals untill someone completes it. After which, all the others get an ‘invalid capture’ message when they submit.
He usually comes out on his own if you just wait a bit. 😛
Andrew
Quite happy with the Captcha and my mathematical skills, but the IP address is a bridge too far perhaps? It didn’t like my actual non-routable 192.168 IP, it was happier with my router DHCP IP from my service provider (which I could change with a router rebot), and I was too lazy to go look up the BT Ealing 10gig Backbone concentrator/gateway IP.
It also claimed to be sending me a message, but she doesn’t write, she doesn’t call.etc 🙂
It gave me the answers, told me I was not a robot but said my IP address (which I left blank) was not real so it flunked me.
I thought bugs was too sharp to get lost in a deban request form.
Chuckles–
This captcha doesn’t work that way. The page runs a random number generator and pulls characters out of a string of possible characters. So, it might create “xRt2X”. Then I:
1) Encrypt and url encodes the xRt2X.
2) Call an image creation routine using the encrypted version of xRt2X. If you look at the source, you’d see calls like “>img src=’CaptchaSecurityImages.php?decrypt=1&code=6vdR698eJX6BLSENAcawhseLv6R9Z97o2F7dfH5nqyI%3D< ” That string is the coded version of the characters in the image.
Then the CaptchaSecurityImages.php de-urlencodes and decrypts the code so it knows that I want it to make an image with xRt2X in it. It makes the images and provides it to the page.
Everyone who visits gets a fresh image.
Currently, the “hints” are there because I was having trouble encrypting and decrypting properly last night. About 1 in 4 images didn’t match the text they were supposed to match. (This is one of the reasons I want people to give it a try. If the captcha thingie isn’t working, you’ll know. It’s unmistakable.)
I think the problem was in not using the key properly….. Or something. But now that I think I have things working, I’m just reloading the page over and over to make sure the “hints” match the images. Right now, I think they always do. But Mike N’s answer worries me. Of course, if he really did not enter the right value, that’s just what it’s supposed to do.
Hunter: Good! That’s what its supposed to do if you leave the IP blank. 🙂
Chuckles:
Yes. Right now, that’s a lie.
I’ll change the filter. My choices were
$valid= filter_var($ip, FILTER_VALIDATE_IP, FILTER_FLAG_NO_PRIV_RANGE | FILTER_FLAG_NO_RES_RANGE) ;
or
$valid= filter_var($ip, FILTER_VALIDATE_IP) ;
I thought… uh…. whatever? I’ll just pick the stricter sounding one. But since you are blocked, maybe it’s the private range issue. Let me know if you can use it now.
Can’t find him. Try making your forms sadder! 🙂
The submit time test didn’t work…I left the page and came back >5 minutes later, but the script appears to have accepted my request. Other tests seem to work – captcha, addition, syntactically invalid IP.
HaroldW– Yep. I commented out the time part while I was swearing at the computer trying to figure out why things weren’t encrypting/decrypting!
I’ll go put those back in!
Invalid IP but the other tests worked. Lurking about a year, Enjoy the blog.
Lucia – I passed all the tests. Luckily the form supplied all the answers in advance. That’s dumbed-down modern education for you.. 🙂
On giving the wrong answers, it complained correctly.
I’m still looking for a “thumbs up” emoticon…
Captcha was happy with my input, resulting page was similar to your example. Skankiness aside, it seemed to work as planned.
Cui–
Yep. I’ve been cutting and pasting the answers while extending the script. I discovered the form was not robust to extra whitespace. So, “xvVHz” is different from “xvVHz ” Luckily, php has a nice trim() function to remove trailing white space.
(I wonder if anyone will accidentally do ” xvVHz”? Hmmm… now that I thought of it, I need to fix that potential error too.
Lucia –
Everything worked out fine, and I was glad of the hints for the adding questions 😉
It also correctly said I hadn’t been banned for at least a week, which was nice to be reminded of.
Happy days from a UK IP! 🙂
Failed the first time due to timeout due to reading the instructions and getting my IP address. Second try passed with flying colors … scratch that … no colors … just black print on white background. Do I get a gold star?
Clearly I’m going to need gold stars to let people know they passed.
The reason for the time out is that in the long run a bot could decide to just sit on the page and try a dictionary attack. People who don’t know the IP they want to unban will have to go hunt for it, come back and refresh the page.
I do want to thank everyone because forms like this can suffer from pesky errors. Having lots of people fill it out can help find them.
There are numerous places to get your IP address. A couple of examples:
http://www.whatismyip.com/
http://www.speedtest.net/
Bots defeat captcha systems using OCR. You might find this interesting:
http://www.devshed.com/c/a/PHP/Improve-PHP-Captcha-with-Optical-Character-Recognition-Tests/
I think I am deserving of 2 gold stars.
Greg– My full captcha module has noise. I commented the bits that insert noise out for initial testing: I don’t want to be squinting to difficult captchas over and over and over while testing repeatedly. So for now, I want them to be easy to see while I check other issues.
Even though bots use OCR to read captchas, I’m pretty sure if the image call for an image that read “XYZ” was
<img src=’CaptchaSecurityImages.php?decrypt=XYZ > even stooopider bots could answer the captcha. In fact, I know that the bots can read variables hidden in forms because in the past bots bet on UAH. My first easy captchas for UAH had the answer passed in a hidden field in the form. But one month suddenly a bunch of bots passed it!
GregF–
This is cool:
http://www.lagom.nl/linux/hkcaptcha/sourcecode.html
It has code to make the captcha distorted and add the wobbly lines.
Im figuring Eli failed the math part and is too ashamed to comment
Lucia,
your captchas as of my try this morning look susceptible to OCR. It might also be good to be wary of color combinations which might deflect the color-blind.
What is wanted is a Turing test. You are working up to it. So far, you are relying on bot inefficiencies and weaknesses such as inability to “read” the unexpected – the addition questions, weak (but improving) ability to read distorted text, and so forth.
In the late ’70s cp/m dial-up bulletin boards required that you knew that DDT was the cp/m debugging utility. If you didn’t know, you were denied access. But this was more of an idiot-filter, which doesn’t seem needed here – dare I assume I would make the cut were there to be one.
[this went into moderation – I’d love to know why. “Idiot” maybe ?]
Progress:
I’m now logging the unban requests. I’m writing the bits to actually process the requests. The part that sends email will be written last. Then everything gets cleaned up.
In the meantime I can see that someone from Brazil has requested getting unbanned! This is not too surprising because the Brazilians are being banned like crazy because so many hits are referrer spoofing. I wonder if it’s a bot or a person? I better add noise to that captcha now. The question: how much noise? I guess that will be the next bleg.
I got the addition right. My point is it’s impossible to get it wrong, since the sum is there BEFORE you hit submit.
j ferguson
Yep! They are currently very susceptible to OCR. Very. They will be less so by the weekend. That and cleaning up to make the forms less skanky will be done at the tail end.
That said: I think for my purposes, the captcha + email is probably more than enough security to prevent bots from getting in easily or potentially hammering cloudflare with requests. (I’m only permitted 150 /hour. I don’t use anywhere near that many but it could happen if a bot were allowed to create and run valid special links over and over and over…)
The reason is that what you call bot “inefficiencies” aren’t necessarily inefficiencies from the bot owners point of view. You have to remember that for the most part, these bots aren’t targetting any one blog or site. They are just going around looking for things. It’s unlikely many (or even any) will:
1) Read the error at Cloudflare and notice which IP was banned.
2) Ask around for the address of my unban request page.
3) Make a request using an OCR reading adding enabled script that knows exactly where to put the appropriate values to “pass”.
4) Give me a valid email.
5) Wait around to get the email.
6) Click the link all to
7) Be able to do something like try to download 100/minute or leave spam. (And possibly get banned again!)
Someone might write a script that could do all that. But generally, they’d try to crack into Paypal, guessing user logins. That way the ultimate benefit is money, not just scraping a site.
It’s true that would be an inefficiency if the bot owner really had a personal issue with my blog and was writing code to specially attack it. But, honestly, that person is going to win. I’m just not going to create Fort Knox security for a blog. I just need enough to make the cost/benefit ratio too high for most bots.
Yep!
My impression is that blue and yellow are good. So is blue and white. Also, one of the reasons for asking people is if blue/yellow turns out not to be good someone might speak up.
I don’t know. I’d have to look at the moderated words but that might be one of them. When logged in as author you can get away with anything. I nearly always am so I forget which words are “bad”.
MikeN
Well…. not impossible. I’ve been known to forget to change the number after refreshing the screen! 😉
But I did find a problem– the form is sensitive to white space. So if you enter “10 ” with a space, that’s not the same as “10” without a space. After you had the issue, I added “trim($input)” to get rid of trailing white space. But I realized later I need to take out all white space. Someone might enter ” 10″ which is also not “10”!
Your comment helped me figure out this issue. Thanks.
Lucia,
A while back, E.M. Smith hosted a discussion on the astonishing ease with which paragraphs composed of words missing most of their letters could be read by most of us. My memory is that first letter, last and maybe one other was usually sufficient in context. This might pose a challenge to those for whom English is not first language, but such truncated words could be impossible for a bot to guess. The clue would have to be internal to the truncated txt.
If all the text in the de-ban filter was graphic, a bot not going after your site to the exclusion of others might just give up and go elsewhere. After all, the text in the message doesn’t need to be text. Could be graphical handwriting.
so there would be four or five graphics,not text, with one graphic the captcha. it’s true that this would take more bytes, but???
The OCR utility which came with Acrobat 4 was very good at typewritten pages, better, to my astonishment, than the then (1995) available non-professional OCR software. The product was text interspersed with graphics of the words it couldn’t read. There was an option to go through the results and type in all the words in graphics which would then be replaced.
Amazingly, the search function knew what most of them were as we discovered when search would produce words in graphics.
Unlike the Kurtzweil Xerox software of the time, there was no bravery variable on Acrobat. It drove me nuts that Acrobat knew what most of the words were but wouldn’t do the text conversion.
Another thing that could be varied is the size of a captcha. What if it was REALLY BIG?
J Ferg, Solutions are at hand 🙂
http://www.mrc-cbu.cam.ac.uk/people/matt.davis/Cmabrigde/
‘Aoccdrnig to a rscheearch at an Elingsh uinervtisy, it deosn’t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer is at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe. ceehiro.’
Hah, Cleuckhs, scuh slikl, do not go itno bot bnisuses, or bidness as tehy say in Altatna. Randieg may be esay, but witrnig is a baer.
ragders…
My earlier post is clearly in error. The letters have to be there but not the order between start and finish. it would be a great experiment as a bot foiler.
Something i worry about is that the folks i encounter here and there on the web are so much sharper than most of the ones i meet in person. to wit…
Lucia
Does this apply only to “bugs” or to “rabbits” too? 🙂
Chuckles:
As that source goes on to say, that meme isn’t true. The mind can correct certain changes in letter order without any real trouble, but not all. For example, “crceort” is not something the mind can easily read. It’s highly probable the original meme was intentionally designed to be easy to read.
By the way, I find it funny that source has a typo in it’s headline. It says “iprmoetnt” which should obviously have an “a” instead of an “e” in it. (There may be more. I didn’t look closely.)
Brandon,
I doubt that “Cleuckhs” could ever have been recognized as “Chuckles.” The difficulty of doing my sentence made me think it was a better job for a computer. But then I had the idea that the entire page could be several graphics with only text being the bot catching field. I thought this would be more effective, less tricky.
Lucia,
Have you considered trying pictures instead of words. A human would have no trouble typing the word “cow” or “horse” when shown the image of such an animal. A bot would have no idea what the image is, and OCR wouldn’t be of much help to it.
Skeptical– No. For now, other than adding noise, I think my captchas are more than sufficient for the task at hand. I don’t think I have a captcha problem to be solved. (In fact, I think even without noise, my captchas are more than strong enough for the application I am using. )
I do think the captcha discussion is interesting on a general level. While humans would probably do very well picking images, and coding the captcha based on images would be easy, the task of finding images would involve a huge amount of work for me. If people are worried about bots cracking my current captchas, then they have to also be worried about bots downloading all my images, identifying the animals, and storing those things. This is entirely do-able. To make the use of images more challenging for bots than the current method, I suspect I would need at least 10^6 suitable public domain images. (I guess alternatively, I could find an entity willing to grant a license for free use). Doing the research to be certain the all 10^6 images are public domain (so as to avoid potential liability for copyright suits) is outside the scope of the task I have assigned myself.
So, while as a technical matter, the animal method might work, as a practical matter, it is too labor intensive for me to implement.
Lucia,
Considering that the bots aren’t targetting you specifically, I wouldn’t have thought that they would go to all the trouble of downloading and identifying your images. Much easier for them to just find another site to hit.
But as you said, it would be a lot of work on your part finding large volumes of royalty-free images… I was naively thinking you could get away with using a couple of dozen different images.
Skeptical–
If they aren’t targeting me specifically, the current system has no captcha problem. So there is no need to fix that which is not broken.
But as I said: the captcha issue is interesting in a generic sense. So– addressed in a generic sense– , as an engineering task to create the perfect captcha system, the image system has the difficulty of potentially involving a large amount of human work. That task is solvable, but someone would need to be assigned the task. In addition to assigning that task, someone needs to figure out the code to implement the task. After all– in addition to making sure a ‘bot can’t just memorize all in the database images and run an image comparator, you also have to avoid making it even easier for even stooopider bots to identify the images. Here’s a thing one might overlook:
Suppose you give the first animal image file in your database the name “/image1.jpg”, the second animal image is “/image2.jpg” and so on. The names of the image files are static. You now have a problem. Because if your captcha calls the images by their static names, then the ‘bot only has to memorize that “image1.jpg” is a cat, while “image2.jpg” is a dog and so on. It doesn’t need to carry the burden of image recognition!
So to use this method you must code things to mask the names of the static files from the bots. That’s needs to be thought out and coded. This needs to happen even if you only have a dozen of so images.
So: The task of coding the animal-image based captcha is not trivial. In 30 seconds, I can anticipate at least 1 blunder that would make the method as cheaper and easier for a bot to defeat as the image of 5 characters or any addition problem.
To get that method to work requires some thought and coding. It could probably be made to work– but I’m not going to spend my time on thinking it through. I suspect other people may be doing it– funded and for tasks that need higher security than my “unban from cloudflare” script.
Lucia,
I was waiting for you to come up with the perfect script, then I was going to offer to market it for you… we’d both get rich… LOL. How does “Lucia’s Bot Buster” sound?
Seriously, I think it’s going to be a never ending war. Even if you came up with a script to constantly change the image filenames, you’d need an endless supply of new images to stop image comparisons.
The only other thing I can think of is to have images of real world things with random characters inserted into the image. Real people would identify what the image is of, while the bots would use OCR to pick out the random characters. Anyone who is silly enough to type in the random characters instead of naming the image deserves to stay banned. 😛
Oddly my IP just got banned by cloudflare.
Seemed to work pretty well over all, though I had to do some fishing to work out where the URL is located for unbanning yourself.
(Seems like it should be listed on the page that says “You’re banned sucka!”)
The idea of using your own script or a tweak on a given script is an appealing one. Basically the bots get tuned to bypass standard bot busters. If you tweak your bot buster enough, they will have to special case you, which probably means 99.9% of the bots won’t ever work with your private version.
Carrick
I agree. But Cloudflare doesn’t let me customize that. So…. I’m going to put it on the sidebar and regulars are going to either
a) Have to remember to read the google cache or
b) Send out a tweet to ask others the address.
I haven’t thought of a better way given my options.
Well now that I know this I have it in my bookmarks!
I worked out the url by logging into my office server then running “lynx”.
Having it as a link on your homepage would work for those of us who know how to use google cache.
(For those who don’t, type cache:http://rankexploits.com/musings in a google search window.)
Skeptical–
The animal captcha I’d read about involved getting access to the human societies dog/cat images show several and having people check “dog” or “cat” below each image. I’m sure the humane society could provide many fresh images a month for a small fee.
But it’s not practical for me.
Brandon
Right now it is, but I’d thought of this problem. Right now, key is the same for all captchas. I use the garden variety function “mcrypt_encrypt in php– so nothing fancy. I have a “settings” file where I enter a key value– so $key=”whatever”. Right now that key is used for everything. Someone clever, dedicated and with programming skillzzzz could come over, load and reload the page and figure out what the key is.
But I had planned to change to a system where I created the key I used by taking the key in the settings file (example “whatever”) and then using something like $key=”whatever”.date(“dhis”,$time) with whatever being the code I entered into the appropriate place in my settings file.
Of course, then I have to store the time in a file so that the script knows what to use to decrypt. But I already do store the time for the request because I use it to decide that a request is stale and also to delete from the request files. So this method means that there are over 26 million keys a year. The visitor would probably have a difficult time figuring out what the key is.
I had a couple other ideas.
But this is sufficient to ensure that someone who loads and reloads can’t just read the encrypted value of whatever numbers and alphanumeric code they get know they all have the same key and reverse engineer the key.
If you know a better easier way, I’d be happy to learn it. (I know there is always the “store the key in a file” method. 🙂 )
BTW: while cooking corned beef I have been fiddleing with a new captcha thingie:
http://theknittingfiend.com/unban/CaptchaDev.php I think it’s easier to read but:
1) Spacing varies and sometimes overlaps. (Maybe not often enough?)
2) Letters can be inclined +/- 20 degrees.
3) Elevation of letters varies.
4) Tweaked the line drawing so lines tend to be longer and cut across letters.
Other things that would be easy:
1) I could make each letter a different color. Mind you, I need contrast between the background and letter so I don’t know if it would help much. But tweaking around the main text color would be trivial to do.
2) Obviously, if I do (1) I need to make the noise different colors too. (I tend to think this different color idea end up easier for an OCR and harder for people. But if I’m wrong, it’s doable..)
3) I could make the lines thicker by always drawing two parrallel lines spaced by 1 pixel.
4) I could stick with a two color captcha but switch randomly. So some people would get yellow background/ blue letters and others would get blue background yellow letters. I tend to think this won’t help much but once again: do able.
Carrick
That’s precisely right in practice.
Provide the prize for breaking in isn’t very attractive, and your tweaks include a security feature that is not common to the more popular bot repulsion methods, the method doesn’t have to be hard to break in theory. What happens is that in practice no one ever spends the two minutes to write the bot that overcomes your method.
Merely asking people to add has been working betting script has stopped bots for a long time now provided the form doesn’t include the answer. Heck, it took the bots a long time to become adding enabled when I passed the correct answer in in a hidden box in the betting form.
The month a bot figured out how to bet, several bet. I seriously doubt that bot was written specifically to bet on UAH. I suspect there was different adding-captcha comment spam protection form somewhere on the web and someone wrote a form to solve that. Then those bots just roved around.
Mind you, if your captcha is protecting a resource a programmer wants to access very badly, it had better be very good. But my captcha is a bit more like one of those really cheap bike locks. It’s just enough to keep an opportunist from seeing a totally unlocked bike and riding away with it.
A little like being chased by a bear. If two of you are running, you don’t need to be faster than the bear, just faster than the other guy.
j ferguson:
Or faster than your kid, wife, etc.
Joking.
j ferguson–
Well… also assuming you aren’t much more delicious to bears than the other guy running.
it must be the quatloos.
Lucia,
I just tried your new captcha… it’s giving me an array_push warning on line 70.
Warning: array_push() “[“function.array-push“]”: First argument should be an array in ….theknittingfiend.com/unban/CaptchaDev.php on line 70
I just tried again… it’s working now.
Skeptical… yeah… I was fiddling…. The “Dev” is what I added for the one I’m testing.
I’m adding the link to the unban page now. I think it really works, the captchas are … mostly readable….I could tweak that if they are still too tough– mostly I implemented Brandon’s suggestion of varying the spacing, but while I was at it I made some letters be inclined forward or backwards. I’m going to solve a few every day and tweak until I’m never ridiculously puzzled.
The hidden hashes don’t give away the solution. (I’d thought of Brandon’s criticism about the hashes giving away the solution, and planned to implement the fix. His mentioning it motivated me to do it sooner rather than later.)
Now I’m going to put the link in the sidebar.
Lucia,
Sounds like you’re almost there with it. I’m glad you didn’t go with the idea of changing colors of each letter. I’m sure that wouldn’t have been good for colorblind people.
I’ll view the page a few more times over the coming days and I’ll let you know if I can’t read any of them.
Skeptikal– The link to the UNBAN page is now in the sidebar.
Have you given any attention to the accessibility issues? I just visited the unban form, using Lynx, and found it totally un-usable. What considerations have you made for those using non-graphical browsers, or those with vision impairments?
I rarely use Lynx any more, because too many sites do not make any allowance for persons using anything other than the Big 4 browsers; everything is visual. I can’t imagine how frustrating it is for the sight impaired, even with modern screen readers.
A plain text reverse Turing test should be reasonably accessible, where the Captcha is definitely not.
cheers,
gary
Gary–
Sorry you found it unusable. The unban page includes this text If the script does not work for you contact Mar_20_12_PM_107.2.lucialiljegren@spamgourmet.com for help.
Did you find the email address unusuable? I and a few others have tested it and it’s worked so far. But if you used it and it doesn’t work I’d like to know that.
If a visitor needs help for any reason, they can send email.
I suspect most those who are not vision impaired but use no-graphical browsers out of preference will generally fire up a readily available graphical browser they keep just for these purposes. So I consider the issues of those who just prefer non-graphical browsers a non-problem.
The vision impaired who read the information supplied in will likely email me.
You mean other than providing the contact email address? I thought giving them the email address would be sufficient.
Describe the sort of test you envision.