 |
simonlilly
User
Jun 12, 2002, 8:21 AM
Post #1 of 3
(3299 views)
Shortcut
|
Is there a maximum number of html pages that the search engine can cope with? or it more a case of looking at my webservers specifications?
Our search (http://www.meridiantv.com/search/) brings out brilliant results, but can take a while to produce them.
Thanks,
Simon
---
Simon Lilly
Web Bloke
www.meridiantv.com
---
|
|
| |
|
Dave
Staff
/ Moderator
Jun 12, 2002, 10:37 AM
Post #2 of 3
(3292 views)
Shortcut
|
Hi Simon,
There's no set limit. It really does depend on your hardware, the size of the pages, etc, but realistically we recommend between 300 - 500 pages for relatively speedy results. We've seen some sites that search a whole lot more pages than that without problems, but we want to make sure Search Engine works well on everybody's servers so we recommend 300 - 500..
The best thing to do if you have more pages than that is to split them up into different categories. How Search Engine works is that it keeps a real time cache (copy) of all the text from the files it searches. If you remove or update a file it will automatically detect this and update its cache. However, the first time you run Search Engine on a new batch of files it will take longer to response, because it will be created text cache files for all your files.
We did it this way, using dynamically updated text cache files, because it's easiest for most users to maintain. They don't have to run an indexing process once a night, everything will always be up to date, etc. Also, it's much faster to search text only cache files then parsing through HTML in realtime for hundreds of files.
So the everytime you add more pages to the Search Engine, the first search you do will be slower because it's automatically caching the text from all those pages, but subsequent searches should be faster. The speed is very relative to your server, but it is doing a real time search on those cached text files. And the best thing to do if you have lots of files is to split them up into more search categories. Less files to search means faster results.
Dave Edis - Senior Developer
interactivetools.com
(This post was edited by Fraser on Jun 13, 2002, 10:15 AM)
|
|
| |
|
Dave
Staff
/ Moderator
Jun 12, 2002, 10:40 AM
Post #3 of 3
(3291 views)
Shortcut
|
Also, As I recall Search Engine caches search results as well. So if you search for "online holidays", it will look up the results and cache them. So paging through the results for that search (after the first page has loaded) or any subsequent searches for that keyword should be very quick (until it expires from the results cache).
Dave Edis - Senior Developer
interactivetools.com
|
|
| |
|
