Category: OpenWebSpider

OpenWebSpider(js) v0.3.0 with support to PostgreSQL

Released OpenWebSpider(js) v0.3.0 Support for PostgreSQL Update Mode Support for HTTP Headers: Last-Modified and ETag Indexer Web Service Example tools using Indexer Web Service (github page)     Getting started Download Screenshots Changelog GitHub...

Full-text search: MySQL vs MongoDB vs Sphinx

You have indexed all your favorite music sites, now you has an index with hundreds of thousands of pages and you want to allow your users to make searches in the best and fastest...

OpenWebSpider and Sphinx Search Engine

Do you need a really powerful search engine? MySQL full-text index is quite good and supports interesting features to make a reasonably fast search engine but, if you are looking for something more, well,...

OpenWebSpider mirrorjs UI

OpenWebSpider XMas edition

I am very pleased to announce a new phantasmagorical version of openwebspider!!! Bazinga! To be honest this is just the announcement that I’m working on it and that things are going quite well. I...

OpenWebSpider# v0.1.4

Released OpenWebSpider# v0.1.4 now with MP3s and PDFs support! New tables has been added please refer to this page to learn more: Database Structure This is the complete CHANGELOG: Mysql/NET Connector upgraded to 5.2.5.0...

OpenWebSpider# v0.1.3

Released OpenWebSpider v0.1.3 CHANGELOG: New feature: CRAWLER NAME and CRAWLER VERSION used in the User-Agent string in HTTP Requests New feature: New configuration file field: sql_hostlist_where New feature: new command-line argument: –keep-dup BUG: fixed...