Changelog

0.3.0

  • Support for PostgreSQL
  • Update Mode
  • Support for HTTP Headers: Last-Modified and ETag
  • Indexer Web Service
  • Example tools using Indexer Web Service



0.2.3

  • Support for MongoDB (2.6+)
  • Support to UTF-8 pages
  • Fixed a bug in pdf.js
  • Changed license to MIT

 

0.2.1

  • Delete duplicate pages (option)
  • Support for <META> ROBOTS tag (<META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>)
  • Support for <BASE> URL (<base href=”http://www.w3schools.com/images/” target=”_blank”>)
  • New plugin: regex
  • Saving last URLs (worker + pages map) + last search query
  • Minor fixes