Search Engine Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsWeb DesignSearch Engine Optimization

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old August 19th, 2008, 12:51 PM
Perceptes's Avatar
Perceptes Perceptes is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2004
Location: San Diego, CA
Posts: 47 Perceptes User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 6 h 58 m 55 sec
Reputation Power: 5
Send a message via AIM to Perceptes
Question Certain subpages not showing up in Google

My company's site is having an odd issue with subpage listings. It's a business school and we want a search for a faculty member's name to give their personal page as linked on http://rady.ucsd.edu/faculty/directory/ from within our custom google search engine. The strange thing is that some of these work and some don't, yet they're all clearly linked from that same directory page.

For example, try searching for "David Schkade" in the search box at the top right. This returns his page first, as expected. Now try searching for "Vish Krishnan". His page does not show up in the results at all, let alone as the top entry. I've submitted an XML sitemap to Google in the hopes that may help. Any other ideas on why not all of these pages are indexed? Thanks in advance.

Reply With Quote
  #2  
Old August 19th, 2008, 01:03 PM
groundscape's Avatar
groundscape groundscape is offline
Permanently Banned
Dev Shed Intermediate (1500 - 1999 posts)
 
Join Date: Jun 2001
Location: Right now? On my chair.
Posts: 1,795 groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)  Folding Points: 700 Folding Title: Novice Folder
Time spent in forums: 1 Week 4 Days 18 h 43 m 31 sec
Warnings Level: 10
Number of bans: 1
Reputation Power: 0
Do you have a robots.txt file restricting it?

Reply With Quote
  #3  
Old August 19th, 2008, 01:08 PM
Perceptes's Avatar
Perceptes Perceptes is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2004
Location: San Diego, CA
Posts: 47 Perceptes User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 6 h 58 m 55 sec
Reputation Power: 5
Send a message via AIM to Perceptes
Quote:
Originally Posted by groundscape
Do you have a robots.txt file restricting it?


We do use a robots.txt file and I noticed it had several outdated entries. However, none of them included the faculty directory. In any case, the file is now cleaned up.

Reply With Quote
  #4  
Old August 19th, 2008, 01:09 PM
groundscape's Avatar
groundscape groundscape is offline
Permanently Banned
Dev Shed Intermediate (1500 - 1999 posts)
 
Join Date: Jun 2001
Location: Right now? On my chair.
Posts: 1,795 groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)groundscape User rank is Captain (20000 - 30000 Reputation Level)  Folding Points: 700 Folding Title: Novice Folder
Time spent in forums: 1 Week 4 Days 18 h 43 m 31 sec
Warnings Level: 10
Number of bans: 1
Reputation Power: 0
Well now wait a few days for Google to index your page and see if any changes are made .

Reply With Quote
  #5  
Old August 20th, 2008, 06:12 AM
mattaseymour mattaseymour is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2008
Posts: 144 mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)  Folding Points: 336 Folding Title: Novice Folder
Time spent in forums: 1 Day 7 h 52 m 49 sec
Reputation Power: 128
Send a message via MSN to mattaseymour Send a message via Google Talk to mattaseymour
Quote:
Originally Posted by groundscape
Well now wait a few days for Google to index your page and see if any changes are made .


Create a XML sitemap and submit it to google. This will then tell google that the page exsits and that it needs to be indexed.

This will be the quickest method of getting the page indexed.

XML sitemaps can be submitted through google.com/webmaster

Reply With Quote
  #6  
Old August 20th, 2008, 08:25 PM
jazajay jazajay is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2007
Posts: 336 jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 6 h 16 sec
Reputation Power: 9
Hi Perceptes.
An xml file is going to do you jack TBH.
The file is easy indexable off the home page by a page to the faculty directory then a link straight to the page for each teacher, if Google cant find that with out help then somethings up and a xml file is not going to cure it, or the underlying problem.

The faculty directory page is not indexed which is a reason why it's not found as it is directly linked off that page and also the page you gave that is indexed, but that is weird as that page is a sub link off your global navigation and should be 1 of the first pages to be indexed.

Your robots.txt file is fine, but you knew that, all the links that I have seen don't use the nofollow attribute and the page doesn't have a meta noindex.

Conculsion if the page is easy to find by a crawler and nothing is blocking it by access, I'm going on the fact that you haven't used a server side search engine block as that I cant tell, is that you are being penalized for something.

I would suggest using base as all your urls are relative and with no base it may cause an issue with the actually file path, TBH thats a long shot as I have never heard of that being an issue before but it is something you can try nonetheless.

How many pages do you have on the site?
Google has 253 pages indexed for -
http://rady.ucsd.edu/

If you have 300+ you know you have a major issue.

If the XML file does solve it and does get it indexed it still doesn't explain the problem, all you have done is circumvented it.

Also how many pages link to the Vish Krishnan page?
Is it just a link off the directory page?
If it is then the problem may be that the page is called directory, again a logical assumption you can try.

Google has stopped indexing pages with the url as links, links page etc.... due to the fact that they are just crap recip pages that have nothing of value and are there to manipuilate the SERPs the problem with Directory and resources are again they are common names for such links pages and as a result Google may not be indexing the page and thus pages off it, thus creating you a problem.

If you have around 258 pages on the whole site and it's only the pages off this page that are having an issue with indexing change the file name to faculty site map or something similar and see if that makes any differance. But it should be fine TBH it should be one of the first pages to be indexed regardless of where the search engine hits your site.

The fact that it isn't is a problem.

Jaza

Reply With Quote
  #7  
Old August 21st, 2008, 10:47 AM
bryan_m's Avatar
bryan_m bryan_m is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jan 2007
Posts: 111 bryan_m User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 21 h 1 m 19 sec
Reputation Power: 2
May be indexed at a lower position, did you checked that? I think you should also update content of specific page and use sitemap as suggested above.
__________________
Micfo International LLC
VPS Web Hosting || cPanel Reseller Hosting || Window Hosting Phone : 1-866-686-8611

Reply With Quote
  #8  
Old August 21st, 2008, 04:22 PM
mattaseymour mattaseymour is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2008
Posts: 144 mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)mattaseymour User rank is First Lieutenant (10000 - 20000 Reputation Level)  Folding Points: 336 Folding Title: Novice Folder
Time spent in forums: 1 Day 7 h 52 m 49 sec
Reputation Power: 128
Send a message via MSN to mattaseymour Send a message via Google Talk to mattaseymour
It does say in googles terms and conditions that not all pages are likely to be indexed....?

Reply With Quote
Reply

Viewing: Dev Shed ForumsWeb DesignSearch Engine Optimization > Certain subpages not showing up in Google


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump

 Free IT White Papers!
 
How to Present Effectively Online
This white paper offers practical and actionable advice on the key steps that any presenter should consider as they plan and execute a Webinar or online meeting.

 
Open Source Security Myths
Open Source Software (OSS) is computer software whose source code is available to the general public with relaxed or non-existent intellectual property restrictions (or arrangement such as the public domain), and is usually developed with the input of many contributors.

 
Power and Cooling Capacity Management for Data Centers
This paper describes the principles for achieving power and cooling capacity management.

 
Scalable, Fault-Tolerant NAS for Oracle - The Next Generation
For several years NAS has been evolving as a storage alternative for Oracle databases, and for good reason: NAS is quite often the simplest, most cost-effective storage approach for Oracle. Learn about the benefits that HP's approach to scalable NAS brings to Oracle environments in this comprehensive white paper.

 
Understanding Web Application Security Challenges
This white paper discusses many common threats and preventive measures for Web application security, and explains what you can do to help protect your organization.

 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 5 hosted by Hostway
Stay green...Green IT