It depends upon the forum software, but the one that I use (PHPBB3) has a flag to permit web crawlers or not.  That forum also has rules that you can configure about who can read posts, make posts, etc in each forum.    I have the OVLR forum set up so that one must be a member and logged in before you can see any messages.  In order to be a member, a moderator has to approve the user and our rules are that you have to convince us that an applicant is a real person and has an interest in Land Rovers before approval is given.  Hence 90%+ of user requests are rejected.  What this means is that someone has to apply and then wait up to a few days to get approved before they can see if the forum is useful to them or not.  This means that membership is smaller and there is less traffic.

 

Ben