Users Developers
Home / Forums / In-Portal CMS / New Features & Improvement Suggestions / In-Portal maybe Spider pages? / Topic Posts

In-Portal Forum

This is a place for users of In-Portal to ask questions, discuss various topics, and interact with other members of the In-Portal Community. Please report bugs through the Bugs Team (not the Forum). If you are interested in contributing or joining one of the many Teams for In-Portal, please check out the Contribute section of the website.

In-Portal Forum

In-Portal maybe Spider pages? (4)

Posted: 12/02/2002 3:05:00 PM

Reply Quoted  

Hi, I think its very hard work add links manually. So I think.. ¿Its very dificult make a spider to add all the webs taken the information of the title or the metatags? Like Google ?

Best Regards.

Posted: 12/21/2002 3:24:00 PM

Reply Quoted  

It can get very difficult depending on the features of such spider. As you may know, Google is a very, very complex system with a system of rankings, etc. We would not consider writing something of that magnitude.

However, a simple spider that would populate the database with URLs, names and descriptions is definitely possible. In fact, it is already on our list of proposed modules. I cannot promise anything concrete at this point, but there is a good chance of us releasing this module sometime in the middle of next year.

Posted: 12/22/2002 12:59:00 AM

Reply Quoted  

GREAT ! In two years I insert manually 3000 links only.

If you can developed this tool, I may select the country of the links?

Links of Spain only for example?

That be wonderfull !!

Posted: 02/02/2003 9:22:00 AM

Reply Quoted  

I think that adding a good filter to that spider should be one of the most critical part of it. I think it wouldn't be of any use to have the spider indexing the whole world wide web (apart for the ones that want compete with google or any other search engine).

Filtering with keywords, language, date, type of document and any other info available would be a first step. And what about the server's resources consumption?

If the product is well done I would gladly buy even this module... :smile:

Mr Disaster