{
    "info": {
        "author": "Manuel Guenther",
        "author_email": "manuel.guenther@idiap.ch",
        "bugtrack_url": null,
        "classifiers": [
            "Framework :: Bob"
        ],
        "description": "=====================================\r\nScore Calibration in Face Recognition\r\n=====================================\r\n\r\nThis package provides the source code to run the experiments published in the paper `Score Calibration in Face Recognition <http://publications.idiap.ch/index.php/publications/show/2822>`_.\r\nIt relies on the FaceRecLib_ to execute the face recognition experiments, and on Bob_ to compute the calibration experiments.\r\n\r\n.. note::\r\n  Currently, this package only works in Unix-like environments and under MacOS.\r\n  Due to limitations of the Bob_ library, MS Windows operating systems are not supported.\r\n  We are working on a port of Bob_ for MS Windows, but it might take a while.\r\n  In the meanwhile you could use our VirtualBox_ images that can be downloaded `here <http://www.idiap.ch/software/bob/images>`_.\r\n\r\n\r\nInstallation\r\n============\r\nThe installation of this package relies on the `BuildOut <http://www.buildout.org>`_ system. By default, the command line sequence::\r\n\r\n  $ ./python bootstrap.py\r\n  $ ./bin/buildout\r\n\r\nshould download and install all requirements, including the FaceRecLib_, the database interfaces `xbob.db.scface <http://pypi.python.org/pypi/xbob.db.scface>`_, `xbob.db.mobio <http://pypi.python.org/pypi/xbob.db.mobio>`_ and all their required packages.\r\nThere are a few exceptions, which are not automatically downloaded:\r\n\r\n\r\nBob\r\n---\r\nThe face recognition experiments rely on the open source signal-processing and machine learning toolbox Bob_.\r\nTo install Bob_, please visit http://www.idiap.ch/software/bob and follow the installation instructions.\r\nPlease verify that you have at least version 1.2.0 of Bob installed.\r\nIf you have installed Bob in a non-standard directory, please open the buildout.cfg file from the base directory and set the ``prefixes`` directory accordingly.\r\n\r\n.. note::\r\n  The experiments that we report in the Paper_ were generated with Bob version 1.2.1 and FaceRecLib_ version 1.2.1.\r\n  If you use different versions of either of these packages, the results might differ slightly.\r\n  For example, we are aware that, due to some initialization differences, the results using Bob 1.2.0 and 1.2.1 are not identical, but similar.\r\n\r\n\r\nImage Databases\r\n---------------\r\nThe experiments are run on external image databases.\r\nWe do not provide the images from the databases themselves.\r\nHence, please contact the database owners to obtain a copy of the images.\r\nThe two databases used in our experiments can be downloaded here:\r\n\r\n- SCface [``scface``]: http://www.scface.org\r\n- MOBIO [``mobio``]: http://www.idiap.ch/dataset/mobio\r\n\r\n.. note::\r\n  For the MOBIO database, you need to sign the EULA to get access to the data -- the process is explained under the above MOBIO link.\r\n  After signing the EULA, please download the image files **IMAGES_PNG.tar.gz** and the annotation files **IMAGE_ANNOTATIONS.tar.gz** and extract the archives into directories of your choice.\r\n\r\nImportant!\r\n''''''''''\r\nAfter downloading the databases, you will need to tell our software, where it can find them by changing the **configuration files**.\r\nIn particular, please update the ``scface_directory`` in **xfacereclib/paper/IET2014/database_scface.py**, as well as ``mobio_image_directory`` and ``mobio_annotation_directory`` in **xfacereclib/paper/IET2014/database_mobio.py**.\r\nPlease let all other configuration parameters unchanged as this might influence the face recognition experiments and, hence, the reproducibility of the results.\r\n\r\nGetting help\r\n------------\r\nIn case anything goes wrong, please feel free to open a new ticket in our GitLab_ page, or send an email to manuel.guenther@idiap.ch.\r\n\r\n\r\nRecreating the results of the Paper_\r\n====================================\r\n\r\nAfter successfully setting up the databases, you are now able to run the face recognition and calibration experiments as explained in the Paper_.\r\n\r\nThe experiment configuration\r\n----------------------------\r\nThe face recognition experiment are run using the FaceRecLib_, but for convenience there exists a wrapper script that set up the right parametrization for the call to the FaceRecLib_.\r\nThe configuration files that are used by the FaceRecLib_, which contain all the parameters of the experiments, can be found in the **xfacereclib/paper/IET2014/** directory.\r\nParticularly, the **xfacereclib/paper/IET2014/dct_mobio.py** and **xfacereclib/paper/IET2014/isv_mobio.py** files contain the configuration for the DCT block features and the ISV algorithm as described in the Paper_.\r\nAccordingly,\r\n\r\nRunning the experiments\r\n-----------------------\r\nThis script can be found in ``bin/iet2014_face_recog.py``.\r\nIt requires some command line options, which you can list using ``./bin/iet2014_face_recog.py --help``.\r\nUsually, the command line options have a long version (starting with ``--``) and a shortcut (starting with a single ``-``), here we use only the long versions:\r\n\r\n- ``--temp-directory``: Specify a directory where temporary files will be stored (default: ``temp``). This directory can be deleted after all experiments ran successfully.\r\n- ``--result-directory``: Specify a directory where final result files will be stored (default: ``results``). This directory is required to evaluate the experiments.\r\n- ``--databases``: Specify a list of databases that you want your experiments to run on. Possible values are ``scface`` and ``mobio``. By default, experiments on both databases are executed.\r\n- ``--protocols``: Specify a list of protocols that you want to run. Possible values are ``combined``, ``close``, ``medium`` and ``far`` for database ``scface``, and ``male`` and ``female`` for ``mobio``. By default, all protocols are used.\r\n- ``--combined-zt-norm``: Execute the face recognition experiments on the SCface database with combined ZT-norm cohort.\r\n- ``--verbose``: Print out additional information or debug information during the execution of the experiments. The ``--verbose`` option can be used several times, increasing the level to Warning (1), Info (2) and Debug (3). By default, only Error (0) messages are printed.\r\n- ``--dry-run``: Use this option to print the calls to the FaceRecLib_ without executing them.\r\n\r\nAdditionally, you can pass options directly to the FaceRecLib_, but you should do that with care.\r\nSimply use ``--`` to separate options to the ``bin/iet2014_face_recog.py`` from options to the FaceRecLib_.\r\nFor example, the ``--force`` option might be of interest.\r\nSee ``./bin/faceverify.py --help`` for a complete list of options.\r\n\r\nIt is advisable to use the ``--dry-run`` option before actually running the experiments, just to see that everything is correct.\r\nAlso, the Info (2) verbosity level prints useful information, e.g., by adding the ``--verbose --verbose`` (or shortly ``-vv``) on the command line.\r\nA commonly used command line sequence to execute the face recognition algorithm on both databases could be:\r\n\r\n1. Run the experiments on the MOBIO database::\r\n\r\n    $ ./bin/iet2014_face_recog.py -vv --databases mobio\r\n\r\n2. Run the experiments on the SCface database, using protocol-specific files for the ZT-norm::\r\n\r\n    $ ./bin/iet2014_face_recog.py -vv --databases scface\r\n\r\n3. Run the experiments on the SCface database, using files from all distance conditions for the ZT-norm::\r\n\r\n    $ ./bin/iet2014_face_recog.py -vv --databases scface --combined-zt-norm --protocols close medium far\r\n\r\n.. note::\r\n  All output directories of the scripts will be automatically generated if they do not exist yet.\r\n\r\n.. warning::\r\n  The execution of the script may take a long time and require large amounts of memory -- especially on the MOBIO database.\r\n  Nevertheless, the scripts are set up such that they re-use all parts of the experiments as far as this is possible.\r\n\r\n\r\n\r\nEvaluating the experiments\r\n--------------------------\r\nAfter all experiments have finished successfully, the resulting score files can be evaluated.\r\nFor this, the ``bin/iec2014_evaluate.py`` script can be used to create the Tables 3, 4, 5 and 6 of the Paper_, simply by writing LaTeX-compatible files that can later be interpreted to generate the tables.\r\n\r\nGenerating output files\r\n'''''''''''''''''''''''\r\n\r\nAlso, all information are written to console (when using the ``-vvv`` option to enable debug information), including:\r\n\r\n1. The :math:`C^{\\mathrm{min}}_{\\mathrm{ver}}` of the development set, the :math:`C^{\\mathrm{min}}_{\\mathrm{ver}}` of the evaluation set and the :math:`C_{\\mathrm{ver}}` of the evaluation set based on the optimal threshold on the development set.\r\n\r\n2. The :math:`C_{\\mathrm{frr}}` on both development and evaluation set, using the threshold defined at ``FAR=1%`` of the development set.\r\n\r\n3. The :math:`C_{\\mathrm{ver}}` on the development and evaluation set, when applying threshold :math:`\\theta_0=0` (mainly useful for calibrated scores).\r\n\r\n4. The :math:`C_{\\mathrm{cllr}}` performance on the development and the evaluation set.\r\n\r\n5. The :math:`C^{\\mathrm{min}}_{\\mathrm{cllr}}` performance on the development and the evaluation set.\r\n\r\nAll these numbers are computed with and without ZT score normalization, and before and after score calibration.\r\n\r\nTo run the script, some command line parameters can be specified, see ``./bin/iec2014_evaluate.py --help``:\r\n\r\n- ``--result-directory``: Specify the directory where final result files are stored (default: ``results``). This should be the same directory as passed to the `bin/iec2014_execute.py`` script.\r\n- ``--databases``: Specify a list of databases that you want evaluate. Possible values are ``scface`` and ``mobio``. By default, both databases are evaluated.\r\n- ``--protocols``: Specify a list of protocols that you want to evaluate. Possible values are ``combined``, ``close``, ``medium`` and ``far`` for database ``scface``, and ``male`` and ``female`` for ``mobio``. By default, all protocols are used.\r\n- ``--combined-zt-norm``: Evaluate the face recognition experiments on the SCface database with combined ZT-norm cohort.\r\n- ``--combined-threshold``: Evaluate the face recognition experiments on the SCface database by computing the threshold on the combined development set.\r\n- ``--latex-directory``: The directory, where the final score files will be placed into, by default this directory is ``latex``.\r\n\r\nAgain, the most usual way to compute the resulting tables could be:\r\n\r\n1. Evaluate experiments on MOBIO::\r\n\r\n    $ bin/iet2014_evaluate.py -vvv --database mobio\r\n\r\n2. Evaluate experiments on SCface with distance-dependent ZT-norm::\r\n\r\n    $ bin/iet2014_evaluate.py -vvv --database scface\r\n\r\n3. Evaluate experiments on SCface with distance-independent ZT-norm::\r\n\r\n    $ bin/iet2014_evaluate.py -vvv --database scface --combined-zt-norm --protocols close medium far\r\n\r\n4. Evaluate experiments on SCface with distance-independent threshold (will mainly change the :math:`C_{\\mathrm{ver}}` of the evaluation set)::\r\n\r\n    $ bin/iet2014_evaluate.py -vvv --database scface --combined-threshold --protocols close medium far\r\n\r\n5. The experiments to compare linear calibration with categorical calibration as given in Table 7 of the Paper_ are run using the ``bin/iet2014_categorical.py`` script::\r\n\r\n    $ bin/iet2014_categorical.py -vvv\r\n\r\n\r\nGenerate the LaTeX tables\r\n'''''''''''''''''''''''''\r\n\r\nFinally, the LaTeX tables can be regenerated by defining the accordant ``\\Result`` and ``\\ResultAtZero`` LaTeX macros and include the resulting files.\r\nE.g., to create Table 3 of the Paper_, define::\r\n\r\n  \\newcommand\\ResultIII[2]{\\\\}\r\n  \\newcommand\\ResultII[9]{#1\\,\\% \\ResultIII}\r\n  \\newcommand\\Result[9]{#1\\,\\% & #4\\,\\% & #2\\,\\% & #3\\,\\% & #5\\,\\% & #6\\,\\% & #9\\,\\% & #7\\,\\% & #8\\,\\% &\\ResultII}\r\n  \\newcommand\\ResultAtZero[8]{}\r\n\r\nset up your ``tabular`` environment with 10 columns and input at according places::\r\n\r\n  \\input{latex/mobio_male}\r\n  \\input{latex/mobio_female}\r\n  \\input{latex/scface_close}\r\n  \\input{latex/scface_medium}\r\n  \\input{latex/scface_far}\r\n  \\input{latex/scface_combined}\r\n\r\nAccordingly, the other tables can be generated from files:\r\n\r\n- Table 4a):  ``latex/scface_close-zt.tex``, ``latex/scface_medium-zt.tex`` and ``latex/scface_far-zt.tex``\r\n- Table 4b):  ``latex/scface_close-thres.tex``, ``latex/scface_medium-thres.tex`` and ``latex/scface_far-thres.tex``\r\n- Tables 5 and 6: ``latex/mobio_male.tex``, ``latex/mobio_female.tex``, ``latex/scface_close-zt.tex``, ``latex/scface_medium-zt.tex``, ``latex/scface_far-zt.tex`` and ``latex/scface_combined.tex`` .\r\n- Table 7: ``latex/calibration-none.tex``, ``latex/calibration-linear.tex`` and ``latex/calibration-categorical.tex``\r\n\r\n\r\nGenerate the score distribution plots\r\n'''''''''''''''''''''''''''''''''''''\r\n\r\nAt the end, also the score distribution plots that are shown in Figures 3 and 4 of the Paper_ can be regenerated.\r\nThese plots require the face recognition experiments to have finished, and also the categorical calibration to have run.\r\nAfterwards, the script ``bin/iet2014_plot.py`` can be executed.\r\nAgain, the script has a list of command line options:\r\n\r\n- ``--result-directory``: Specify the directory where final result files are stored (default: ``results``). This should be the same directory as passed to the `bin/iec2014_execute.py`` script.\r\n- ``--figure``: Specify, which figure you want to create. Possible values are 3 and 4.\r\n- ``--output-file``: Specify the file, where the plots should be written to. By default, this is ``Figure_3.pdf`` or ``Figure_4.pdf`` for ``--figure 3`` or  ``--figure 4``, respectively.\r\n\r\nHence, running::\r\n\r\n  $ ./bin/iet2014_plot.py -vv --figure 3\r\n  $ ./bin/iet2014_plot.py -vv --figure 4\r\n\r\nshould be sufficient to generate the plots.\r\n\r\n\r\n.. _paper: http://publications.idiap.ch/index.php/publications/show/2822\r\n.. _idiap: http://www.idiap.ch\r\n.. _bob: http://www.idiap.ch/software/bob\r\n.. _facereclib: http://pypi.python.org/pypi/facereclib\r\n.. _gitlab: http://gitlab.idiap.ch/manuel.guenther/xfacereclib-paper-iet2014\r\n.. _virtualbox: http://www.virtualbox.org",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://gitlab.idiap.ch/manuel.guenther/xfacereclib-paper-iet2014",
        "keywords": "",
        "license": "LICENSE.txt",
        "maintainer": "",
        "maintainer_email": "",
        "name": "xfacereclib.paper.IET2014",
        "package_url": "https://pypi.org/project/xfacereclib.paper.IET2014/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/xfacereclib.paper.IET2014/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "http://gitlab.idiap.ch/manuel.guenther/xfacereclib-paper-iet2014"
        },
        "release_url": "https://pypi.org/project/xfacereclib.paper.IET2014/1.0.0/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Running the experiments as given in paper:",
        "version": "1.0.0"
    },
    "last_serial": 2355369,
    "releases": {
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6bd2567f15c3564ee08b4eea08c2ed4b",
                    "sha256": "3c9146d089e4aa957b26b51d4899de91cfe444bfb8ee36ca75ddbcf62c4384fc"
                },
                "downloads": -1,
                "filename": "xfacereclib.paper.IET2014-1.0.0.zip",
                "has_sig": false,
                "md5_digest": "6bd2567f15c3564ee08b4eea08c2ed4b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 53830,
                "upload_time": "2014-01-16T11:20:42",
                "url": "https://files.pythonhosted.org/packages/54/da/a320dd20d874d88175ed4d83622192cac708b9cb159092b2c9b8d2acdce0/xfacereclib.paper.IET2014-1.0.0.zip"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "6bd2567f15c3564ee08b4eea08c2ed4b",
                "sha256": "3c9146d089e4aa957b26b51d4899de91cfe444bfb8ee36ca75ddbcf62c4384fc"
            },
            "downloads": -1,
            "filename": "xfacereclib.paper.IET2014-1.0.0.zip",
            "has_sig": false,
            "md5_digest": "6bd2567f15c3564ee08b4eea08c2ed4b",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 53830,
            "upload_time": "2014-01-16T11:20:42",
            "url": "https://files.pythonhosted.org/packages/54/da/a320dd20d874d88175ed4d83622192cac708b9cb159092b2c9b8d2acdce0/xfacereclib.paper.IET2014-1.0.0.zip"
        }
    ]
}