Undev/redmine_xapian

h1. Redmine Xapian search plugin

!https://travis-ci.org/Undev/redmine_xapian.png!:https://travis-ci.org/Undev/redmine_xapian
"!https://codeclimate.com/github/Undev/redmine_xapian.png!":https://codeclimate.com/github/Undev/redmine_xapian

With this plugin you will be able to do searches by file name and by strings inside your documents through the xapian search engine.
This plugin replaces search controller, its view and search and acts_as_searchable methods.


h1. 1.  Installation and Setup

A copy of the plugin can be downloaded from  Github at https://github.com/xelkano/redmine_xapian/downloads

h2. 1.1. Required packages

Branch redmine-1 (version 1.3.X) only compatible with Redmine 1.4.X
Branch master (version 1.4.x) only compatible with Redmine 2.0

To use "xapian":http://xapian.org search engine you must install libxapian-ruby and xapian-omega packages (on ubuntu). To index some files with omega you may have to install some other packages like xpdf, antiword, ...

From "Omega documentation":http://xapian.org/docs/omega/overview.html:

    * PDF (.pdf) if pdftotext is available (comes with xpdf)
    * PostScript (.ps, .eps, .ai) if ps2pdf (from ghostscript) and pdftotext (comes with xpdf) are available
    * OpenOffice/StarOffice documents (.sxc, .stc, .sxd, .std, .sxi, .sti, .sxm, .sxw, .sxg, .stw) if unzip is available
    * OpenDocument format documents (.odt, .ods, .odp, .odg, .odc, .odf, .odb, .odi, .odm, .ott, .ots, .otp, .otg, .otc, .otf, .oti, .oth) if unzip is available
    * MS Word documents (.doc, .dot) if antiword is available
    * MS Excel documents (.xls, .xlb, .xlt) if xls2csv is available (comes with catdoc)
    * MS Powerpoint documents (.ppt, .pps) if catppt is available (comes with catdoc)
    * MS Office 2007 documents (.docx, .dotx, .xlsx, .xlst, .pptx, .potx, .ppsx) if unzip is available
    * Wordperfect documents (.wpd) if wpd2text is available (comes with libwpd)
    * MS Works documents (.wps, .wpt) if wps2text is available (comes with libwps)
    * AbiWord documents (.abw)
    * Compressed AbiWord documents (.zabw) if gzip is available
    * Rich Text Format documents (.rtf) if unrtf is available
    * Perl POD documentation (.pl, .pm, .pod) if pod2text is available
    * TeX DVI files (.dvi) if catdvi is available
    * DjVu files (.djv, .djvu) if djvutxt is available
    * XPS files (.xps) if unzip is available

On debian you can use this command to install required indexing tools:

<pre>
# apt-get install unzip antiword ghostscript catdoc xpdf libwpd-tools libwps-tools gzip unrtf
</pre>

Since Redmine 1.4, Redmine uses Bundler and required xapian gem can be installed running bundle command in redmine root directory. To install required xapian gem you might need to install uuid-dev package.

On debian run:

<pre>
# apt-get install uuid-dev
</pre>

Now, go to redmine root directory and run bundle command to install required gem:

<pre>
# cd /var/www/redmine
# bundle
</pre>

h2. 1.2. Plugin installation

Install redmine_xapian into plugins directory with:

* cd into [redmine_root]
* run command "bundle install --without development test"
* cd into [redmine_root]/plugins/
* run command "git clone http://github.com/xelkano/redmine_xapian.git" or download and extract tar or zip file into redmine plugins directory..
* rake db:migrate_plugins RAILS_ENV="production"
* Restart web server

Now, you can see a new check box “files” on search screen, that allows you to search attachments by file name and its contents.
Xapian plugin checks for ruby bindings before its startup, if plugin can not find them,it is not activated and this message is going to appear in redmine log “REDMAIN_XAPIAN ERROR:No Ruby bindings for Xapian installed !!. PLEASE install Xapian search engine interface for Ruby" If you see this message,please make sure that xapian library are correctly installed for your ruby environment.

h2. 1.3. Setup

To use stemmed languages on searches, xapian needs to generate stemmed forms for each language and store them on each database.
Before you can use the plugin, you have to populate the omega databases for each stemmed language you want to use:

<pre>
omindex -s english --db /var/www/xapian-index/english /var/www/redmine/files/
</pre>

<pre>
omindex -s german --db /var/www/xapian-index/german /var/www/redmine/files/
</pre>

use *omindex -h* for help.

You can launch this command from crontab, one or two times a day.
The database directories have to be named with language name you want to use, english directory for english stemmed forms, german directory for german stemmed forms,…

Example of cron job (every 30 minutes):

<pre>
*/30 * * * * root /usr/bin/omindex -s english -U / --db /var/www/xapian-index/english /var/www/redmine/files/
</pre>


After file indexing and plugin setup on settings you can search attachment by name and content.

To perform attachment searches view_document permission is needed, be sure it is checked.


h1. 3. KNOWN BUGS

- When search is performed across redmine database and xapian database, results count is approximate and may not be accurate due to the required effort in database operations to determine this value.

- If you are upgrading plugin version, please, first configure plugin and save settings, otherwise may appear errors in search screen.