Posted: 12/02/2002 3:05:00 PM
Hi, I think its very hard work add links manually. So I think.. ¿Its very dificult make a spider to add all the webs taken the information of the title or the metatags? Like Google ?
Best Regards.
Home
/ Forums
/ In-Portal CMS
/ New Features & Improvement Suggestions
/ In-Portal maybe Spider pages?
/ Topic Posts
In-Portal maybe Spider pages? (4)
Hi, I think its very hard work add links manually. So I think.. ¿Its very dificult make a spider to add all the webs taken the information of the title or the metatags? Like Google ?
Best Regards.
Posted: 12/21/2002 3:24:00 PM
It can get very difficult depending on the features of such spider. As you may know, Google is a very, very complex system with a system of rankings, etc. We would not consider writing something of that magnitude.
However, a simple spider that would populate the database with URLs, names and descriptions is definitely possible. In fact, it is already on our list of proposed modules. I cannot promise anything concrete at this point, but there is a good chance of us releasing this module sometime in the middle of next year.
Posted: 12/22/2002 12:59:00 AM
GREAT ! In two years I insert manually 3000 links only.
If you can developed this tool, I may select the country of the links?
Links of Spain only for example?
That be wonderfull !!
MrDisaster
Member
Joined: 10-Sep-2004
Location: Italy
Posts: 61
Posted: 02/02/2003 9:22:00 AM
I think that adding a good filter to that spider should be one of the most critical part of it. I think it wouldn't be of any use to have the spider indexing the whole world wide web (apart for the ones that want compete with google or any other search engine).
Filtering with keywords, language, date, type of document and any other info available would be a first step. And what about the server's resources consumption?
If the product is well done I would gladly buy even this module... :smile:
Regards,
Mr Disaster
