Released OpenWebSpider# v0.1.4 now with MP3s and PDFs support! New tables has been added please refer to this page to learn more: Database Structure
This is the complete CHANGELOG:
- Mysql/NET Connector upgraded to 5.2.5.0
- Enhanced encodings support
- New feature: Support to META: “robots” (NOINDEX, NOFOLLOW)
- New feature: New configuration file field: crawler_id
- New field “crawler_id” in table “hostlist”
- New table: crawler_act
- New feature: Remote actions over running crawlers [Status, Play, Pause, Kill]
- New file support: PDFs [Using PDFBox and IKVM]
- New table: pdf
- New file support: MP3s [Using UltraID3Lib]
- New table: mp3
- New feature: new command-line argument: −−pdf
- New feature: new command-line argument: −−mp3
Go to the DOWNLOAD page
OpenWebSpider# explained with 4 video: Compile, Configure and RUN!
alexhemp responded on 07 May 2009 at 5:47 pm #
0.14 – in some random moment reported
– Error [Wait Threads]: List has changed.
+ Killing threads…
Shen139 responded on 08 May 2009 at 10:57 am #
Thanks, In certain conditions OWS could give that error.
I fixed the bug! You can download new compiled executable here: http://www.openwebspider.org/public/source-code/ows/OpenWebSpiderCS_v0.1/bin/OpenWebSpiderCS.exe.zip
I’ll release fixed code as soon as possible.
Shen139 responded on 08 May 2009 at 11:18 am #
You can download here the complete package (source code + libraries + bin):
http://downloads.sourceforge.net/openwebspider/OpenWebSpiderCS_v0.1.4.1.zip
alexhemp responded on 18 May 2009 at 10:19 pm #
Sorry, but error still exists
– Error [Wait Threads]: List has changed.
+ Killing threads…
Occurs randomly, after about 1500-3000 URL’s indexed on local server.
FreeBSD 7.0/Mono 2.4
Shen139 responded on 19 May 2009 at 9:59 am #
Hi,
I don’t think you are using latest version of OWS!
Yes, I know I released a bug fixing release without changing the version number (I’ll release v0.1.5 asap);
anyway: latest version of OWS uses different error messages for that error:
” – Error 1 [Wait Threads]”
” – Error 2 [Wait Threads]”
The error message: “- Error [Wait Threads]” (without error number) is the message used in the previous release.
Please download v0.1.4 from sourceforge. (maybe when you downloaded it not all servers were aligned)