Space 1999 Eagle Transporter Forum


Go Back   Space 1999 Eagle Transporter Forum > Administration Complex > Commander's Office

Notices

SPONSORED BY
&

Reply Bookmark and Share
 
Thread Tools Display Modes
Old 18-02-2008, 09:55 PM   #1
cricket
Medical Officer
 
cricket's Avatar
 
Join Date: Aug 2006
Location: Midwest, USA
Posts: 4,560
Default arachnid invasion

S'okay... on top of the frequent google spiders, the almost as frequent yahoo slurp spiders (obviously in need of a review in dining manners! ), now we have an MSNBot spider. These are good things, right? The entire computer spider-thing is a bit fuzzy in my understanding.
cricket is off duty   Bookmark and Share Reply With Quote
Old 19-02-2008, 01:09 AM   #2
Tim Smith
Communications Officer
 
Tim Smith's Avatar
 
Join Date: May 2006
Location: Birmingham, AL
Posts: 1,180
Default

Crickets should probably avoid them!
Tim Smith is off duty   Bookmark and Share Reply With Quote
Old 19-02-2008, 01:29 AM   #3
cricket
Medical Officer
 
cricket's Avatar
 
Join Date: Aug 2006
Location: Midwest, USA
Posts: 4,560
Default

groan.... I set myself up for that, didn't I?
cricket is off duty   Bookmark and Share Reply With Quote
Old 19-02-2008, 01:41 AM   #4
Bishop
Chief Science Officer
 
Bishop's Avatar
 
Join Date: Mar 2007
Location: South Australia
Posts: 11,796
Default

I'm interested in what they do too. Are they there searching for info that people are requesting on various subjects or what?
Bishop is off duty   Bookmark and Share Reply With Quote
Old 19-02-2008, 05:26 AM   #5
CR
Science Officer
 
CR's Avatar
 
Join Date: Dec 2005
Location: wandering the universe
Posts: 5,472
Default

I'm interested in what the heck you two are talking about. (And cricket thinks she's the computer-illiterate one! )
CR is off duty   Bookmark and Share Reply With Quote

Old 19-02-2008, 05:47 AM   #6
Bishop
Chief Science Officer
 
Bishop's Avatar
 
Join Date: Mar 2007
Location: South Australia
Posts: 11,796
Default

All the spiders who visit and are listed in the "whose on duty" bit at the bottom of the page.
Bishop is off duty   Bookmark and Share Reply With Quote
Old 19-02-2008, 11:02 AM   #7
tx4
Medical Technician
 
tx4's Avatar
 
Join Date: Sep 2007
Location: Midwest, U.S.A.
Posts: 178
Default

My assumption has been that they're constantly roving every link they can find and scanning the contents of each page so as to keep their extremely massive search engine databases as up-to-date as possible. That way, when someone goes to a search engine and types in a word or phrase --say "Moonbase Alpha" -- that the engine can output results extremely quickly, just by looking it up in their own database.

The spiders are thus supporting user searches, but as a separate action, not part of an actual user search. That is, I think the spiders are not here looking for "Moonbase Alpha" that someone just typed in (the search would take too long), but just wandering around and absorbing every word they can find to add to their databases for if/when someone does search for "Moonbase Alpha" later on.

What surprises me, though, is that it there are so many spiders from any one company at any one moment. I guess it is a sign of how up-to-date the engine wants to stay? Efficiency or inefficiency of their download and processing speed? Raw power of their systems? Not really sure about the quantity; but if nothing else, I suppose it helps the Catacombs of the Moon section here stay "current" to the rest of the Internet; i.e. that its listings start to become visible quickly?

Just my unofficial 1.999 cents.
tx4 is off duty   Bookmark and Share Reply With Quote
Old 19-02-2008, 11:57 AM   #8
cricket
Medical Officer
 
cricket's Avatar
 
Join Date: Aug 2006
Location: Midwest, USA
Posts: 4,560
Default

Ah. That makes sense, and is in line with what I've heard/read before, but not really understood. So that is how a google search finds the data it uses to kick out its results.

We had a band-width problem with them before, IIRC. It almost did the site in. (Or am I thinking of something else??)
cricket is off duty   Bookmark and Share Reply With Quote
Old 19-02-2008, 12:09 PM   #9
tx4
Medical Technician
 
tx4's Avatar
 
Join Date: Sep 2007
Location: Midwest, U.S.A.
Posts: 178
Default

Quote:
Originally Posted by cricket View Post
We had a band-width problem with them before, IIRC. It almost did the site in. (Or am I thinking of something else??)
Might that explain the following, which I've been wondering about (from the front page down by the Who's On Duty bit)? "Most users ever online was 200, 8th September 2007 at 05:40." A few weeks ago, this forum had 440 members (membership has climbed a lot recently -- welcome everyone! ), and five months ago presumably had less than 440, only for 200 of them to get online at once? Either that was one big "posting party," or a ton of spiders or something else tried swarming in?

I have no idea; I've just wondered about that 200 number, and am curious whether it is connected to your recollection....
tx4 is off duty   Bookmark and Share Reply With Quote
Old 19-02-2008, 12:14 PM   #10
cricket
Medical Officer
 
cricket's Avatar
 
Join Date: Aug 2006
Location: Midwest, USA
Posts: 4,560
Default

Yes, I seem to remember that it all tied in. Those 200 'members' were something like 15 members and 185 arachnids.

Eagle would certainly remember the specifics.
cricket is off duty   Bookmark and Share Reply With Quote

Old 19-02-2008, 12:28 PM   #11
Bishop
Chief Science Officer
 
Bishop's Avatar
 
Join Date: Mar 2007
Location: South Australia
Posts: 11,796
Default

A number of us were having difficult logging on or getting room even though logged on with the old forum software for a little while last year. That was put down (I think) to the spiders. Still happens to me at various points.
Bishop is off duty   Bookmark and Share Reply With Quote
Old 19-02-2008, 03:10 PM   #12
captainjamie
Communications Officer
 
captainjamie's Avatar
 
Join Date: May 2007
Location: Somewhere in Kent
Posts: 918
Default

Quote:
Originally Posted by cricket View Post
Ah. That makes sense, and is in line with what I've heard/read before, but not really understood. So that is how a google search finds the data it uses to kick out its results.

We had a band-width problem with them before, IIRC. It almost did the site in. (Or am I thinking of something else??)
There's a file called robots.txt which search engines are supposed to obey for indexing websites, telling them which folders to look in, etc. Need to gen up on that as I'm doing some website work here...
captainjamie is off duty   Bookmark and Share Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT. The time now is 11:27 PM.


Powered by vBulletin® Version 3.7.2
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
This site and contents are copyright Bernard Walsh 2010 all rights reserved, no reproduction of material in any form without written permission email: Bernie.walsh@mac.com