Home
« Video: John Resig: “The DOM Is a Mess”

Calling out bad crawlers: the Kintiskton nuisance

Posted Feb 17th, 2009 by David Calhoun in Uncategorized

I have never been involved in creating a web crawler, but as a website owner I’m well aware of the behavior of good crawlers versus bad crawlers.  For instance, a good crawler must not only follow the rules set by robots.txt, but it must also not impose an undue load on the server being indexed.

Famously, Cuil exhibited this bad behavior for at least several months before they claimed to have fixed it.  In any case, I had to ban their IP range because they were just hitting my site too hard (compared to all of the other major crawlers out there).

Today I’m looking at my traffic stats for my WWI flight sim site and I see that yesterday I got over 200% new visitors.  Strange thing was there was no major referring site, only direct hits!  What on earth!  So I check the logs and find that most of the IPs are from the range 65.208.151.112-65.208.151.119, which resolves to kintiskton-gw.customer.alter.net [63.114.61.170] before the tracert dies.

Apparently this IP block is owned by Kintiskton LLC, whatever that is.  When I do a Google search, I can’t find the actual company, only complaints about its crawler abusing people’s websites going back to December 2008 (several months).

The IP block is hosted by Verizon Business, so I shot over an email to abuse@verizon.net.  After several months of this Kintiskton doing their excessive crawling, hopefully Verizon will eventually step up and look into it.  Apparently they haven’t yet…

In the meantime, it’s good old Apache to the rescue.

I’ll be adding this to my .htaccess file:

Deny from 65.208.151.112
Deny from 65.208.151.113
Deny from 65.208.151.114
Deny from 65.208.151.115
Deny from 65.208.151.116
Deny from 65.208.151.117
Deny from 65.208.151.118
Deny from 65.208.151.119

  • LGR on 27 Feb 2009 at 12:11 pm

    I have the exact same problem. I just ban those IP’s by default now on every site I work on. I guess that is all we can do to stop them. Thanks Verizon Business for being a nuisance.

  • mrdeus on 28 Feb 2009 at 2:12 pm

    I blocked this bot today. It’s annoying because it loads images and everything. And does it all at once, like you said.

    It’s shorter to use the CIDR notation for the .htaccess file:

    Deny from 65.208.151.112/29

  • Kintiskton « Mark Turner Dot Net on 23 Apr 2009 at 9:45 am

    [...] that made choose not to share my web content with Kintiskton. I first added the subnet to my .htaccess rules but decided to add it to my firewall rules. That took care of [...]

  • Julian Perry on 15 May 2009 at 12:10 am

    I am getting very similar activity from cuil.com.
    Direct hits from 216.129.119.16.
    Google Adsense shows the hits as page impressions for one day then they’re gone the next.

  • canuck on 17 Jun 2009 at 7:25 am

    Hi. They came on to my site and I received over 600 hits from them in approx half an hour. I was able to exempt my statcounter.com from logging their IP address, but that doesn’t mean they can no longer access my site.

    I use blogger. Is there a place in my Template where I can insert info that would deny them even loading my page? I am computer not-too-smart and so if you can help in kid language, that would be ideal :)

    Cheers & thanks.

  • bob3160 on 12 Jun 2010 at 7:52 am

    Just blocked them. Wondered what was making my server so busy.

  • Tony on 13 Jul 2010 at 6:25 am

    They were just crawling my site pretty heavily but I was watching in real time my site traffic, so they done a bit of crawling until I made an htaccess file with:

    deny from 65.208.151.

    And it stopped them within seconds :)

    I chose to deny the the whole range (1-255) because no doubt they will use new ip’s soon enough but they should only change their server ip’s (meaning only that last number will change).

    If anyone wants to stop them crawling, simply make a text file, and write in it:

    deny from 65.208.151.

    and save it. Then upload the txt file to your web hosting and rename it .htaccess and its done :)

    You can’t rename your text file to .htaccess in windows you have to do it in your ftp when the file hits your web server because windows FORCES you to put something before a . in a file name. Linux/UNix web servers do not.

  • Brinley Ang on 12 Aug 2010 at 12:38 am

    It even parses urls in JavaScript code without correctly executing the code itself and ended up hitting my site with 404s.

  • Freya on 06 Nov 2010 at 6:34 am

    It is still very active so reporting the abuse to Verizon was useless. I just blocked it.

  • 65.208.151.118 on 11 Nov 2010 at 1:48 am

    you can find a good opinion about that ip address on here http://www.technonsense.com/2010/11/65-208-151-118/

  • Rick on 17 Aug 2011 at 7:34 am

    I’ve compiled a list of the IP addresses that I have found to come from these clowns. It sounds like someone who is trying to find people to sue under the guise of “The World Internet Authority”. LMAO, they even have a facebook fan page. Just go to kintiskton.com and see for yourself.

    Anyway, here’s the IP’s that I have banned because they all come back as registered to Kintiskton, LLC.

    Hope it helps…
    63.114.61.170
    65.200.47.0-65.200.47.29
    65.200.90.160-65.200.90.175
    63.110.158.29-63.110.158.48
    65.208.151.29-65.208.151.112
    65.208.151.112-65.208.151.119
    63.110.148.48-63.110.158.55
    65.200.47.0-65.200.47.7

  • Uknowit on 13 Oct 2011 at 4:22 pm

    Over 2 years and Kintiskton’s business IP’s / servers are still allowed to run amok and do damage (abuse team at their NOC / ISP does nothing about reports of malicious activity).

    They’re running a script that will try to DDOS any site that hosts photo galleries and/or PHP scripts that offer “demo” accounts. VERY malicious attacks occured on my photo gallery demo site for so long that I finally stopped “temp” blocking them and implemented the real solution.

    To block the malicious illegal activity from Kintiskton and their IP’s such as 65.208.151.112 I simply did this:

    Went into my hosting cPanel and set their ENTIRE IP RANGE to Deny:

    65.208.151.112/29

    Just set 65.208.151.112/29 as blocked in your cPanel (or .htaccess , or ask your host to do it) and the problem is solved.

    Still can’t believe a major ISP will ignore so many abuse reports about 65.208.151.112 and Kinikston and allow them to continue to operate / perpetrate illegal activity!

    But ever since blocking 65.208.151.112/29 in my .htaccess and cPanel , no problems anymore. If everyone starts blocked the entire range 65.208.151.112/29 then eventually the ISP will have no choice but to take action. Until then, us “little guys” can easily just brush them off now…

Trackback URI | Comments RSS

Leave a Reply

Categories

RSS Feed Subscribe to RSS
Powered by WordPress.