IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

IWETHEY Home / IWETHEY Board / Internet Forum / might have to do with the fact that most engines
Post #279,493
by boxley
3/25/07 12:27:02 PM
Reply
New might have to do with the fact that most engines
ignore the robots.txt file. Eaasiest way might be to present the old static pages until they quit.
thanx,
bill
Any opinions expressed by me are mine alone, posted from my home computer, on my own time as a free american and do not reflect the opinions of any person or company that I have had professional relations with in the past 51 years. meep

reach me at [link|mailto:bill.oxley@cox.net|mailto:bill.oxley@cox.net]
Post #279,497
by crazy
3/25/07 12:56:52 PM
Reply
New But then they'd never quit.
Hmmm.

You could go the pain in the ass track down and hassle people mode, which may or may not work, or you could setup some type of mechanism to black-hole their IP address after 'X' number of failures. Preferably on a device BEFORE your web server gets hit.
     Not sure if this belongs here or in "Networking"... - (cwbrenn) - (15) - March 25, 2007, 12:03:22 PM EDT
         might have to do with the fact that most engines - (boxley) - (1) - March 25, 2007, 12:27:02 PM EDT
             But then they'd never quit. - (crazy) - March 25, 2007, 12:56:52 PM EDT
         Welcome back. - (Another Scott) - March 25, 2007, 01:07:25 PM EDT
         Probably not bots - more likely fans - (tuberculosis) - (11) - March 25, 2007, 01:44:27 PM EDT
             Re: Probably not bots - more likely fans - (cwbrenn) - (10) - March 25, 2007, 03:48:27 PM EDT
                 This looks like a good article on it. - (crazy) - March 25, 2007, 04:32:44 PM EDT
                 Here's the short version - (drewk) - (8) - March 25, 2007, 07:38:34 PM EDT
                     Oh, OK... - (cwbrenn) - (7) - March 25, 2007, 07:56:13 PM EDT
                         124K - eek! - (cwbrenn) - (6) - March 25, 2007, 11:04:19 PM EDT
                             I wouldn't sweat it. - (static) - (1) - March 26, 2007, 12:27:16 AM EDT
                                 oh, that's right... - (cwbrenn) - March 26, 2007, 12:54:39 AM EDT
                             RedirectMatch? - (cwbrenn) - (3) - March 26, 2007, 12:58:16 PM EDT
                                 you might not even need the... - (folkert) - (2) - March 26, 2007, 01:00:56 PM EDT
                                     But are the wildcards used correctly? - (cwbrenn) - (1) - March 26, 2007, 01:05:00 PM EDT
                                         Looks like it. According to current Apache docs: - (folkert) - March 26, 2007, 01:16:02 PM EDT

Blessed are those who expect nothing, for never shall they be disappointed.
45 ms