Results 1 to 2 of 2

Thread: Search engine bots causing massive bandwidth usage

  1. #1
    Member Nitron's Avatar
    Join Date
    Apr 2004
    Location
    Minnesota
    Posts
    37

    Default Search engine bots causing massive bandwidth usage

    Various search engine bots, mainly from Google (googlebot) and MSN, are causing massive bandwidth usage on my vBulletin 3, I'm already at 22GB usage this month, when usually I rarely go past 8GB.

    To make a long question short, I have my vB3 in a subdirectory, how would I construct a robots.txt file to deny all search engine bots just for that subdirectory and everything in it?

  2. #2
    Member
    Join Date
    Mar 2004
    Location
    India
    Posts
    78

    Default

    Create a robots.txt in your root directory with the following contents:
    Code:
    User-agent: *
    Disallow: /<forum-folder-name>/

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •