Bacardi

Well I read that most search robots don't use keywords anymore (because they have become useless), that Google needs its own metatags, etc etc...
Who knows more, and what do you do?

I'd also love to know the tags to stop Google (and others) from displaying images on their sites if I didn't give them an authorisation to do so.

And I'd also would love to know how one can know whether someone is stealing bandwidth as I have a lot of used bandwidth compared to the hits on various pages.

More later.

lilwicked805

Meta tags are important. Make sure your site is optimized for such tags.

As for stopping access to images use a robots.txt file and tell the search engines where to go. Bad thing is some don't pay attention to the txt file hence my php file that blocks them from my site until I remove their IP from the database.

Example of txt file:
User-agent: *
Disallow: /erik/images/
Disallow: /erik/forms/

The user agent * tells the engines they are allowed but follow the disallow rules below. Don't block them from the images that make up your site if you want a cache file on the engines. If you don't want a cache copy of your site out there then block the images file for the main site. This file goes in your root directory. Also an htaccess file is much needed. You can block a lot from this if you don't use a php script to lock up the bad boys.

Example of htaccess:

SetEnvIfNoCase User-Agent "Indy Library" bad_bot
SetEnvIfNoCase User-Agent "Internet Explore 5.x" bad_bot
#SetEnvIf Remote_Addr "216\.88\.158\.141" bad_bot
#Order Allow,Deny
#Allow from all
#Deny from env=bad_bot
# looksmart.com
SetEnvIf Remote_Addr "216\.88\.158\.((12[8-9])$" bad_bot
SetEnvIf Remote_Addr "209\.249\.66\.((10)|(28)|(34)|(46)|(47))$" bad_bot
Order Allow,Deny
Allow from all
Deny from env=bad_bot

As for bandwidth issues any pic on Google will have a link to your site and image, hence bandwidth issues. They don't cache it, they just link to your site and image. If you do a search on Google and see your images there pay attention to the link they have and you will understand what I mean.

As for others linking to your images check your server logs. I had someone link to one of mine and I sent 3 emails demanding removal or face legal action and they did not take it down so I made a special gif for that file name and needless to say they immediately took it down then!

Hope this helps!

lilwicked805

Another thing Erik, the robots.txt file goes in the root directory as well as the htaccess file. It is loaded as "htaccess" no file extension. If you search on Google you can find tons of tuts on both files. Your htaccess file does not need to be as extensive as the one I have listed here in order to protect your site.

Bacardi

Thanks a lot for this.
A bit heavy, but I'll give it a study.
I'm shure it's worth it.

Bacardi

Perhaps I should study php one day...

why do they make it all so difficult?

Bacardi

no delete option?

Bacardi

EEEKKKK!!!! [saywhat]

I just discovered that I have a bunch of options on my site that I didn't even know they existed...

Blinky2004

Nice options Erik. You should be able to everything from there. [righton]

Linda... what are those IPs in your htaccess file for? Have you found some specific bots that dn't follow the rules?

lilwicked805

Oh yes Mark several! Those are just a few of the many I have listed for the down right mean bots that want to access forms and private sections of the site when the robots.txt file tells them not to. The file I have above is for a client who didn't mind it being posted. I also have a dbase that when they access anything they shouldn't or someone tries to rip the site their ip is thrown into the dbase and they are immediately blocked from the site until I remove their ip from the dbase. You can of course add ip's as needed to the dbase.

Bacardi

Yup. That hotlink protection should stop people from linking directly to images or whatever content I specify. But I must say that I don't even know half of all these things. This will mean another study. It's very clear to me that the last couple of years, website-building has changed from simple html tags to high end professional and very specialized knowledge that needs updating daily if one wants to stay with the pack of the other designerd. My congrats to you who can master all this!

But first I got some beta-testing to do, a website to get online, paintings to make, an article to write, ...

to sleep, perchance to dream,...