openwebspider Open Source Web Spider and Search Engine

OpenWebSpider mirrorjs UI

OpenWebSpider XMas edition

I am very pleased to announce a new phantasmagorical version of openwebspider!!! Bazinga! To be honest this is just the announcement that I’m working on it and that things are going quite well. I...

openwebspider# Usage Example

OpenWebSpider# v0.1.4

Released OpenWebSpider# v0.1.4 now with MP3s and PDFs support! New tables has been added please refer to this page to learn more: Database Structure This is the complete CHANGELOG: Mysql/NET Connector upgraded to 5.2.5.0...

mono_24

Mono 2.4 has been released

Mono 2.4 has been released! The Mono Project aims to make developers productive and happy: Mono 2.4 is our gift to the world. Sponsored by Novell, the Mono open source project has an active...

OpenWebSpider# v0.1.3

Released OpenWebSpider v0.1.3 CHANGELOG: New feature: CRAWLER NAME and CRAWLER VERSION used in the User-Agent string in HTTP Requests New feature: New configuration file field: sql_hostlist_where New feature: new command-line argument: –keep-dup BUG: fixed...

Mono 2.0 has been released

The Mono Project aims to make developers productive and happy: Mono 2.0 is our gift to the world. Sponsored by Novell (http://www.novell.com), the Mono open source project has an active and enthusiastic contributing community...

OpenWebSpider# v0.1.2

Released OpenWebSpider v0.1.2 CHANGELOG: BUG: fixed the regex used to extract URLs from (I)FRAME New feature: OpenWebSpider# can index images (new table: images) New feature: new command-line argument: −−images Improved Stress-test facility: now OpenWebSpider#...

OpenWebSpider# v0.1.1

Released OpenWebSpider v0.1.1 CHANGELOG: New feature: new command-line argument: −−req−timeout New feature: new command-line argument: −−stress−test BUG: fixed a bug in http.cs::getURL() [New features here: OpenWebSpider# v0.1 Command Line Arguments/Usage] [ Read more about:...