openwebspider Open Source Web Spider and Search Engine

Full-text search: MySQL vs MongoDB vs Sphinx

You have indexed all your favorite music sites, now you has an index with hundreds of thousands of pages and you want to allow your users to make searches in the best and fastest...

OpenWebSpider and Sphinx Search Engine

Do you need a really powerful search engine? MySQL full-text index is quite good and supports interesting features to make a reasonably fast search engine but, if you are looking for something more, well,...

OpenWebSpider mirrorjs UI

OpenWebSpider XMas edition

I am very pleased to announce a new phantasmagorical version of openwebspider!!! Bazinga! To be honest this is just the announcement that I’m working on it and that things are going quite well. I...

OpenWebSpider# v0.1.4

Released OpenWebSpider# v0.1.4 now with MP3s and PDFs support! New tables has been added please refer to this page to learn more: Database Structure This is the complete CHANGELOG: Mysql/NET Connector upgraded to 5.2.5.0...

Mono 2.4 has been released

Mono 2.4 has been released! The Mono Project aims to make developers productive and happy: Mono 2.4 is our gift to the world. Sponsored by Novell, the Mono open source project has an active...

OpenWebSpider# v0.1.3

Released OpenWebSpider v0.1.3 CHANGELOG: New feature: CRAWLER NAME and CRAWLER VERSION used in the User-Agent string in HTTP Requests New feature: New configuration file field: sql_hostlist_where New feature: new command-line argument: –keep-dup BUG: fixed...