Tagged: openwebspider

OpenWebSpider(js) v0.3.0 with support to PostgreSQL

Released OpenWebSpider(js) v0.3.0 Support for PostgreSQL Update Mode Support for HTTP Headers: Last-Modified and ETag Indexer Web Service Example tools using Indexer Web Service (github page)     Getting started Download Screenshots Changelog GitHub...

OpenWebSpider and Sphinx Search Engine

Do you need a really powerful search engine? MySQL full-text index is quite good and supports interesting features to make a reasonably fast search engine but, if you are looking for something more, well,...

OpenWebSpider mirrorjs UI

OpenWebSpider XMas edition

I am very pleased to announce a new phantasmagorical version of openwebspider!!! Bazinga! To be honest this is just the announcement that I’m working on it and that things are going quite well. I...

OpenWebSpider# v0.1.3

Released OpenWebSpider v0.1.3 CHANGELOG: New feature: CRAWLER NAME and CRAWLER VERSION used in the User-Agent string in HTTP Requests New feature: New configuration file field: sql_hostlist_where New feature: new command-line argument: –keep-dup BUG: fixed...

OpenWebSpider# v0.1.1

Released OpenWebSpider v0.1.1 CHANGELOG: New feature: new command-line argument: −−req−timeout New feature: new command-line argument: −−stress−test BUG: fixed a bug in http.cs::getURL() [New features here: OpenWebSpider# v0.1 Command Line Arguments/Usage] [ Read more about:...