|
Search Engine Articles
Submitting to the Spidering
Search Engines by Jill Whalen (The Web Whiz)
Written April 2001 - Updated March 2002
I'm often asked how often one should submit or resubmit their site to the
spidering search engines. Many people believe they should submit their site on a
periodic basis to keep their site in the search engines' "faces." Submitting
URLs that are already in a search engine's database, which hasn't had any
significant content changes, is not a good idea.
A general rule of thumb for submitting and resubmitting pages is to first check
each engine to see if the new page has been indexed. There is no reason to
resubmit a page to an engine if the updated page is already in the database.
Submitting to Google
Google is the best engine for spidering through sites. It's only necessary to
add your main URL and Google will eventually spider the rest of your site. You
can check if Google has indexed your site by typing your URL into their search
box. They currently take about one-to-two months to index and reindex pages.
Representatives from Google have warned that they prefer to find sites and pages
on their own, rather than you submitting them through their form. However, they
also state that there is no penalty for submitting to them through their form.
Google has no immediate plans to start a pay-for-inclusion program. Google's Add
URL form can be found here.
Submitting to Hotbot/Inktomi
At HotBot, you can type www.yoursite.com into their search box and see if it
shows up. Since HotBot uses the Inktomi database for its search results, you can
simply wait for the Inktomi spider to find your site. If you've been listed in
the LookSmart directory, the Inktomi spider should easily find you in no time.
You can also submit to them through any Inktomi partner site. As with most of
the search engines, HotBot/Inktomi does not guarantee that they will list all
sites that are submitted to them for free. For guaranteed inclusion in their
database, you might want to use the Inktomi Paid Inclusion Program. Basically,
Inktomi has partnered with some submission companies that, for a fee, guarantee
the Inktomi spider will crawl your submitted pages every 48 hours. This will
also keep your URL in their database for a year. You won't get any boost in
rankings, but you'll have peace of mind knowing your site is listed.
Submitting to AltaVista
AltaVista used to be one the quickest to index new pages. However, those days
are long gone, unless you use their paid inclusion program. In order to keep the
amount of spam submittals in check, AV has instituted an interesting submittal
procedure. When you submit a URL to AV's free "Add-URL form," you'll see a
graphical image of some strange- looking letters. You are instructed to type
those letters into a box in order to continue the submission. Once you do that,
you can then submit up to five URLs. I think it's a neat idea, to thwart
automatic submission programs. Since I hand submit anyway, it's not a problem
for me. As long as it means that my submissions get added in a relatively short
time period, I'm all for it!
To see if your pages are in AltaVista, type in "URL:www.yoursite.com" into their
search box (substituting your own domain name for yoursite.com). Your indexed
pages will show up. Compare the title and description AV lists, with the Title
tag and Meta description tag on your actual pages. If AV shows your old title
and/or description, then they haven't yet indexed your new page. If they show
your new stuff, then you're all set! Links to AltaVista's various submission
programs can be viewed here.
Submitting to Lycos/FAST
Sites have been getting added fairly quickly and flawlessly as of late to Lycos.
You can usually type your URL into their search box to see if it's in their
database. Lycos has partnered with FAST, whose spider crawls through pages on a
regular basis. Very often FAST will find your inner pages without you needing to
submit them at all. Lycos/FAST also recently instituted a new pay-for-inclusion
program called, InSite Select which will guarantee your pages regular 48 hour
spidering. Lycos' add URL programs can be reached here.
Excite, Webcrawler and GO are Dead
Excite and its sister engine, Webcrawler, along with Disney's Go (formerly
Infoseek) are all basically dead. They currently use Overture pay-per-click ad
links, and have no database of their own.
Patience is a Virtue
Once you've submitted your URL to the engines, the next thing for you to do is
practice the fine art of P-A-T-I-E-N-C-E!!! Unless you've paid for inclusion,
your best bet is to submit and then forget about it for awhile. Wait for three
months, then do a reality check of all the engines as noted above. I know that
three months is probably too long for most of you to wait, but if you don't like
to be driven crazy, it's a good strategy. I admit that I do check at least once
a month after submission, but for the average Web site owner, waiting three
months is a smart idea. After three months, you'll probably be pleasantly
surprised to find that your site has been indexed by all of the search engines.
And better yet…if you optimized them correctly they should be bringing in tons
of highly targeted traffic!
Contact Jill Whalen by e-mail at jill@highrankings.com, or by phone at
508-309-3037.
=======================================================
Jill Whalen of High Rankings is an internationally recognized
search engine optimization consultant
and editor of the free weekly High Rankings Advisor
search engine marketing
newsletter
She specializes in search engine optimization, SEO consultations and seminars.
Jill's handbook, "The
Nitty-gritty of Writing for the Search Engines" teaches business owners how
and where to place relevant keyword phrases on their Web sites so that they make
sense to users and gain high rankings in the major search engines.
=======================================================
|
|
