Skip to content
Search engines 5511dd3

Recommendations for a site search engine?

O

The author's views are entirely their own (excluding the unlikely event of hypnosis) and may not always reflect the views of Moz.

Table of Contents

O

Recommendations for a site search engine?

The author's views are entirely their own (excluding the unlikely event of hypnosis) and may not always reflect the views of Moz.

I need recommendations for a site search engine that does the following:
  1. Spiders your site's content to X levels deep
  2. Build a database of page titles, urls, and content
  3. Provide a searchable front-end that is easily customizable to match an existing site
I could write my own. It'd just be a simple spider program that spiders through every URL on a site (via wget, curl, or whatever), drops the url and content into a database, slap a fulltext index on the title/content, and matches against that data in a search query. Time is of the essence here, however, and if there's a way to avoid re-inventing the wheel I'd like to take that route.

I don't want to use the google API. It takes too long to update it's index and I'd like the site index to be updated hourly. I've already tried phpdig, tsep, and php-crawler. Phpdig was the best of all of them, but their templating system (and php code in general) is horrendous and I'm about ready to give up on it. I've also heard mention of Lucene being a great alternative to using mysql fulltext indexes, but I think it's overkill for what I need.

I'm looking for something that uses php and mysql.

Any suggestions?
Back to Top

With Moz Pro, you have the tools you need to get SEO right — all in one place.

Read Next

The Mozbot Mashup: Roger Explores the World of Generative AI Imagery

The Mozbot Mashup: Roger Explores the World of Generative AI Imagery

Mar 07, 2023
One Secret to Improve SEO in 2021: Guestographics

One Secret to Improve SEO in 2021: Guestographics

Jul 21, 2021
Pop-Ups, Overlays, Modals, Interstitials, and How They Interact with SEO

Pop-Ups, Overlays, Modals, Interstitials, and How They Interact with SEO

Apr 28, 2017

Comments

Please keep your comments TAGFEE by following the community etiquette

Comments are closed. Got a burning question? Head to our Q&A section to start a new conversation.