Calibration Software
The benchmarking suite Retrieval Analyzer includes a collection of three mapping methods described in the paper. The inputs to Retrieval Analyzer are a vector of computer scores (your image retrieval scores for the image pairs we have provided in our ground truth data) and corresponding human scores (our ground truth data). The outputs consist of a correlation score and an estimated precision-recall curve. We provide an option for chosing from any of the three mapping methods, but we recommend using the method that maximizes the correlation score.Download
We provide various options for download:- You could download the entire suite. [ Click here]
- Or You could browse the directory and download the individual files [ Click here]
Support
This suite has been tested on Linux (Redhat and Fedora distributions). I have not tested it on any other unix/linux platforms. This program has not been tested on Microsoft Windows. However, if you are succesfull in porting this program to either Windows or a different linux/unix distribution, then I encourage you to share it with others. Send me an email and I will be happy to setup a link to a ported version.Terms of Use
The use of this software is restricted to educational and research purposes. Modifying the code is encouraged, as long as appropriate credit is provided, and authorship is not misrepresented. If you would like to make commercial use of any of the code being distributed, then please contact Nikhil. If you make use of this software for your research, I would appreciate it if you cite or acknowlege the web site and/or the "Evaluating Image Retrieval".Installation
Once you have downloaded, unzipped, and untarred, you will have a number of matlab files listed below, executable files and a script file. We provide only executables for the Bayesian fitting tool. For the complete source code please contact Nicholas Heard . Please be notified that this method has not been recommended by Nicholas Heard to be used in the mapping scenario. However, experiments using this method by us have shown that this is a reasonable method to be used in the current scenario.The code.tar.gz decompresses into a directory (retrieval_analyzer) containing a number of matlab files listed below, executable files and a script file. It is best to have the data be present in the current directory (directory to which retrieval_analyzer.tar.gz was unzipped and untarred). Outputs are written into a new directory "results" within the existing directory. Have a look at the directory organization to get a feel for the installation.