Do Spiders & bots read Javascript?

Hi guys,

just pondering over here about whether or not search engine spiders ignore Javascript..

I was thinking that a site with frames could be fully indexed, with redirections to the index page if the referrer isn't part of the site in question - ie to ensure that framesets are always present.

comments?

later
Christo

I don't believe they compile and run javascript, if that's what you mean.

I'm a little confused by what you're trying to accomplish.

Redirecting users based on user_agent strings is a task better left to apache or mod_perl. . . assuming you're doing it for the express purpose of serving custom content to web spiders.

well if a site (actually a friend's site) consistes of about 12 pages, all displayed in a frameset.. so there is a navbar on the left, which when clicked on, causes the respective pages to be loaded into the main frame..

Now he wants all his pages to be indexed by google, but if a user just clicks on a google result, he/she will be taken to that page, which will be displayed without the surrounding frameset - cos it's just an html file and the frameset is elsewhere (actually the index page).

still with me?

So he is planning to use js redirects on every page OTHER than the index page, to test that a hit has been referred from elsewhere IN the site. If the hit has come from OUTside, then it's fair to assume that the user has arrived mid-site and not via the framed index page, meaning that none of the banners and navbars are visible.

I'm terrible at explaining stuff.

Anyway, his concern is that spiders will see redirects on all these pages and immediately stop slurping the page contents, so these pages won't get rated/indexed..

How do people deal with this issue in sites with framesets? I guess a PHP direct would be more appropriate if a redirect really is required!

christo !

spiders don´t parse javascript.
make sure all your pages work also without javascript (at least link them without js). so this is wrong:
<a href="#" onclick="parent.frames['mainframe'].location='1.html'">Menu1</a>

and this is right:
<a target="mainframe" href="1.html" onclick="parent.frames['mainframe'].location='1.html';return false">Menu1</a>

some spiders don´t parse even frames. so make on every page a "<noframes>" tag and make only links there:
<noframes>
<a href="1.html">Menu1</a>
...

put a javascript on the framed pages (here: 1.html)
<script>
if (self==top) location.href='frameset.html';
</script>
then, if someone accesses your pages directly, they´ll reload the frameset. even better, use a dynamic frameset:
<script>
if (self==top) location.href='frameset.php?page=1.html';
</script>


don´t use this feature in your "normal", spider-compatible linking since most spiders don´t index pages that contain a "?" in their URL!

Manuel,

thanks for the reply to this post - It didn't go unnoticed, I have just been without connectivity for a while..!


Just what I was looking for!

thanks
christo

As a follow up,

If spiders and bots ignore javascript, then I wonder what they would make of a page which contained a <div> which is laden with keywords, but is accompanied by a piece of javascript that sets it's (css selector) visibility to 'hidden' or display to 'none' ? Surely they are smart enough to detect this sort of cheekiness ?

christo