OpenWebSpider# v0.1: Getting Started

Requirements:

  1. Latest stable version of Mono or the Microsoft .NET Framework 2.0 or above (under Windows)
  2. One of the operating system supported by Mono (if you use Mono): Linux, Solaris, Mac OS X, Windows, and Unix [Read more about Mono and .NET Framework]
  3. MySQL Connector/NET (It’s provided within the OpenWebSpider# package but you can also download the latest version from the official website: MySql Connector/NET)

OpenWebSpider# doesn’t need to be compiled on your machine! You can use the binaries in the bin folder.
If you want you can compile it with MonoDevelop or Microsoft Visual C# (the Express Edition is freely downloadable from the Microsoft Website)



Create Database and Tables:
Watch this video: Create Database And Tables For Openwebspider v0.7
Video: Create Database And Tables for OpenWebSpider v0.7

This video still apply to OpenWebSpider# v0.1!

  1. Select the text in the file SQL/script.sql
  2. open a mysql terminal (you can also use SQLyog MySQL GUI – Community Edition)
  3. paste the text of the script there (the script will create default databases and tables)

Configure OpenWebSpider#
OpenWebSpider# has a simple configuration file: “openwebspider.conf” placed in the same directory of the binary (you can change name and folder of the file using the command line argument:
−−conf−file <filename>
Read more about OpenWebSpider# v0.1 Command Line Arguments/Usage

−−−[

# Sample openwebspider.conf

# server1 is the server with the DB ‘DB1’
# db1 is the database of the hosts
db1 = <database (Default: ows_hosts)>
mysqlserver1 = <host/IP (Default: localhost)>
port1 = <MySQL Server port (Default: 3306)>
userdb1 = <username>
passdb1 = <password>

#server2 is the server with the DB ‘DB2’
#db2 is the database of the indexed pages
db2 = <database (Default: ows_index)>
mysqlserver2 = <host/IP (Default: localhost)>
port2 = <MySQL Server port (Default: 3306)>
userdb2 = <username>
passdb2 = <password>

#EOF#

]−−−

mysqlserver1and mysqlserver2 are usually the same MySQL Server but you could use two servers: one per database!

Execute OpenWebSpider#:

  • Under Windows:
    1. you should open a Console [Start -> Execute Command -> cmd.exe]
    2. Change Directory and set that of OpenWebSpider’s bin folder
      (Example: cd c:\openwebspider\bin)
    3. Run it with: OpenWebSpiderCS.exe −−help
  • Under Linux:

    1. Open a Terminal (Console)
    2. Change Directory and set that of OpenWebSpider’s bin folder
      (Example: cd /home/user/projects/openwebspider/bin)
    3. Run it with: mono OpenWebSpiderCS.exe −−help

That’s all!!!

Leave a Reply