{ "info": { "author": "Zhi Li", "author_email": "zli@netflix.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS", "Operating System :: Unix", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3" ], "description": "SUREAL - Subjective Recovery Analysis\n=====================================\n\n.. image:: https://img.shields.io/pypi/v/sureal.svg\n :target: https://pypi.org/project/sureal/\n :alt: Version on pypi\n\n.. image:: https://travis-ci.org/Netflix/sureal.svg?branch=master\n :target: https://travis-ci.org/Netflix/sureal\n :alt: Build Status\n\nSUREAL is a toolbox developed by Netflix for recovering quality scores from noisy measurements obtained by subjective tests.\nRead `this `_ paper for some background. SUREAL is being imported by the VMAF_ package.\n\nCurrently, SUREAL supports Python 2.7 and 3.7.\n\n.. _VMAF: https://github.com/Netflix/vmaf\n\n\nInstallation\n============\nSUREAL can be either installed through ``pip`` (available via PyPI_), or locally.\n\nInstallation through ``pip``\n----------------------------\n\nTo install SUREAL via ``pip``, run::\n\n pip install sureal\n\nLocal installation\n------------------\n\nTo install locally, first, download the source. Under the root directory, (perferrably in a virtualenv_), install the requirements::\n\n pip install -r requirements.txt\n\nUnder Ubuntu, you may also need to install the ``python-tk`` (Python 2) or ``python3-tk`` (Python 3) packages via ``apt``.\n\nTo test the source code before installing, run::\n\n python -m unittest discover -s test -p '*_test.py'\n\nThe code thus far has been tested on Ubuntu 16.04 LTS and macOS 10.13.\n\nLastly, install SUREAL by::\n\n pip install .\n\nIf you want to edit the source, use ``pip install --editable .`` or ``pip install -e .`` instead. Having ``--editable`` allows the changes made in the source to be picked up immediately without re-running ``pip install .``\n\n.. _PyPI: https://pypi.org/project/sureal/\n.. _virtualenv: https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/\n\n\nUsage in command line\n=====================\n\nRun::\n\n sureal\n\nThis will print usage information::\n\n usage: subjective_model dataset_filepath [--output-dir output_dir] [--print]\n\nIf ``--output-dir`` is given, plots will be written to the output directory.\n\nIf ``--print`` is enabled, output statistics will be printed on the command-line and / or the output directory.\n\nBelow are two example usages::\n\n sureal MLE resource/dataset/NFLX_dataset_public_raw_last4outliers.py\n sureal MLE_CO resource/dataset/VQEGHD3_dataset_raw.py\n\n\nHere ``subjective_model`` are the available subjective models offered in the package, including:\n\n - MOS - Standard mean opinion score\n\n - MLE - Full maximum likelihood estimation (MLE) model that takes into account both subjects and contents\n\n - MLE_CO - MLE model that takes into account only subjects (\"Content-Oblivious\")\n\n - DMOS - Differential MOS, as defined in [ITU-T P.910](https://www.itu.int/rec/T-REC-P.910)\n\n - DMOS_MLE - apply MLE on DMOS\n\n - DMOS_MLE_CO - apply MLE_CO on DMOS\n\n - SR_MOS - Apply subject rejection (SR), as defined in [ITU-R BT.500](https://www.itu.int/rec/R-REC-BT.500), before calculating MOS\n\n - ZS_SR_MOS - Apply z-score transformation, followed by SR, before calculating MOS\n\n - SR_DMOS - Apply SR, before calculating DMOS\n\n - ZS_SR_DMOS - Apply z-score transformation, followed by SR, before calculating DMOS\n\n\nDataset files\n-------------\n\n``dataset_filepath`` is the path to a dataset file.\nDataset files may be ``.py`` or ``.json`` files.\nThe following examples use ``.py`` files, but JSON-formatted files can be constructed in a similar fashion.\n\nThere are two ways to construct a dataset file.\nThe first way is only useful when the subjective test is full sampling,\ni.e. every subject views every distorted video. For example::\n\n ref_videos = [\n {\n 'content_id': 0, 'content_name': 'checkerboard',\n 'path': 'checkerboard_1920_1080_10_3_0_0.yuv'\n },\n {\n 'content_id': 1, 'content_name': 'flat',\n 'path': 'flat_1920_1080_0.yuv'\n },\n ]\n dis_videos = [\n {\n 'content_id': 0, 'asset_id': 0,\n 'os': [100, 100, 100, 100, 100],\n 'path': 'checkerboard_1920_1080_10_3_0_0.yuv'\n },\n {\n 'content_id': 0, 'asset_id': 1,\n 'os': [40, 45, 50, 55, 60],\n 'path': 'checkerboard_1920_1080_10_3_1_0.yuv'\n },\n {\n 'content_id': 1, 'asset_id': 2,\n 'os': [90, 90, 90, 90, 90],\n 'path': 'flat_1920_1080_0.yuv'\n },\n {\n 'content_id': 1, 'asset_id': 3,\n 'os': [70, 75, 80, 85, 90],\n 'path': 'flat_1920_1080_10.yuv'\n },\n ]\n ref_score = 100\n\n\nIn this example, ``ref_videos`` is a list of reference videos.\nEach entry is a dictionary, and must have keys ``content_id``, ``content_name`` and ``path`` (the path to the reference video file).\n``dis_videos`` is a list of distorted videos.\nEach entry is a dictionary, and must have keys ``content_id`` (the same content ID as the distorted video's corresponding reference video),\n``asset_id``, ``os`` (stands for \"opinion score\"), and ``path`` (the path to the distorted video file).\nThe value of ``os`` is a list of scores, reach voted by a subject, and must have the same length for all distorted videos\n(since it is full sampling).\n``ref_score`` is the score assigned to a reference video, and is required when differential score is calculated,\nfor example, in DMOS.\n\nThe second way is more general, and can be used when the test is full sampling or partial sampling\n(i.e. not every subject views every distorted video).\nThe only difference from the first way is that, the value of ``os`` is now a dictionary, with the key being a subject ID,\nand the value being his/her voted score for particular distorted video. For example::\n\n 'os': {'Alice': 40, 'Bob': 45, 'Charlie': 50, 'David': 55, 'Elvis': 60}\n\n\nSince partial sampling is allowed, it is not required that every subject ID is present in every ``os`` dictionary.\n\n\nUsage in Python code\n====================\n\nSee `here `_ for an example script to use SUREAL in Google Collab notebook.\n\n\nFor developers\n==============\n\nSUREAL uses tox_ to manage automatic testing and continuous integration with `Travis CI`_ on Github, and setupmeta_ for new version release, packaging and publishing. Refer to `DEVELOPER.md `_ for more details.\n\n.. _tox: https://tox.readthedocs.io/en/latest/\n.. _Travis CI: https://travis-ci.org/Netflix/sureal\n.. _setupmeta: https://github.com/zsimic/setupmeta", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/Netflix/sureal", "keywords": "", "license": "Apache 2.0", "maintainer": "", "maintainer_email": "", "name": "sureal", "package_url": "https://pypi.org/project/sureal/", "platform": "", "project_url": "https://pypi.org/project/sureal/", "project_urls": { "Homepage": "https://github.com/Netflix/sureal" }, "release_url": "https://pypi.org/project/sureal/0.4.1/", "requires_dist": null, "requires_python": "", "summary": "Subjective Recovery Analysis", "version": "0.4.1" }, "last_serial": 5347852, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "3c7a10182c7f3a2565776365965495ad", "sha256": "8758b5a9e55d58fe1e250efdc734e2a882d8981e62debd0f38ad387dd450699c" }, "downloads": -1, "filename": "sureal-0.1.0-py2-none-any.whl", "has_sig": false, "md5_digest": "3c7a10182c7f3a2565776365965495ad", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 3352, "upload_time": "2018-09-12T04:42:05", "url": "https://files.pythonhosted.org/packages/73/c0/2b6d4f57bce8d5122662f9028f6f22e033653cfe0ff6781320d066e1c7dc/sureal-0.1.0-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c7fea734bb8ff8279af20b52a1769fc2", "sha256": "14421415693fe8af7ce6732e99f50b3abf094fec7a0616eda995d4f3f1dae994" }, "downloads": -1, "filename": "sureal-0.1.0.tar.gz", "has_sig": false, "md5_digest": "c7fea734bb8ff8279af20b52a1769fc2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3425, "upload_time": "2018-09-12T04:42:06", "url": "https://files.pythonhosted.org/packages/09/03/e5b7aa68abded0629dbfa67a98af3c4cec21d4d9893fc3f23a4d7be3b404/sureal-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "e5b415f25a496ebbcfd4e0b157d820bc", "sha256": "53dd7c14d4a2dbec2a5131381684f10c5c135ba697e58d58f34b09a41cfa6e44" }, "downloads": -1, "filename": "sureal-0.1.1-py2-none-any.whl", "has_sig": false, "md5_digest": "e5b415f25a496ebbcfd4e0b157d820bc", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 20214, "upload_time": "2018-09-12T23:04:49", "url": "https://files.pythonhosted.org/packages/34/ce/0fe137e5f3aa0ef7494a86f3e178476aa91db7da31f305ef4f582f60c913/sureal-0.1.1-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "cb36b8e5cfca589d83aeda9a26da3372", "sha256": "b6c51e7dc4fa59c9dbcdc21a6bfdaa3ec0de9c417d2937b4e5cb17c537222f49" }, "downloads": -1, "filename": "sureal-0.1.1.tar.gz", "has_sig": false, "md5_digest": "cb36b8e5cfca589d83aeda9a26da3372", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19369, "upload_time": "2018-09-12T23:04:50", "url": "https://files.pythonhosted.org/packages/02/4d/be56600d9a5a68f89e8ba7bfa85111428264046d6cf99ff3b0c39b56fda9/sureal-0.1.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "d39d6f64c4aec7f3a5d58b67345b5c9a", "sha256": "c4224b18644d322830000a2df0fac033decab6e15a7c670ec2d11aacfe9b21c1" }, "downloads": -1, "filename": "sureal-0.2-py2-none-any.whl", "has_sig": false, "md5_digest": "d39d6f64c4aec7f3a5d58b67345b5c9a", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 22483, "upload_time": "2018-12-01T00:24:43", "url": "https://files.pythonhosted.org/packages/0b/e8/55dbaa959fb0cdb59f7b0ddc7ee489d83d318ebc5c968ff276fdccff26d4/sureal-0.2-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a6541114d213e9e0a76c39b70662638d", "sha256": "dd00c1a9f13559546023a3a8d15b35703b1b9152d9cded17d9a19bdce5401bce" }, "downloads": -1, "filename": "sureal-0.2.tar.gz", "has_sig": false, "md5_digest": "a6541114d213e9e0a76c39b70662638d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 21837, "upload_time": "2018-12-01T00:24:44", "url": "https://files.pythonhosted.org/packages/60/9a/9157d26ee0262d932fde45a0bc206ee6c83a526405aea729997128eb440d/sureal-0.2.tar.gz" } ], "0.3": [ { "comment_text": "", "digests": { "md5": "fe0d855af6c8094947a8ac891666e7ac", "sha256": "0b47dfa5fd6e32c43e91602f4c61d9792e36f326ae0f4dd2612fb7701a9e240d" }, "downloads": -1, "filename": "sureal-0.3-py2-none-any.whl", "has_sig": false, "md5_digest": "fe0d855af6c8094947a8ac891666e7ac", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 36772, "upload_time": "2019-03-22T00:22:43", "url": "https://files.pythonhosted.org/packages/c9/39/e57f22e77446a3e91e13f94401cab70090a3c7dca8437ba56301ef872ca5/sureal-0.3-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c5b83abcb95436aac9f48fc309ea0aa7", "sha256": "9665fc8b3e82a0577cd5d7800fb9bac3377d23ee5a5e0f38a3a96bc22bb16a78" }, "downloads": -1, "filename": "sureal-0.3.tar.gz", "has_sig": false, "md5_digest": "c5b83abcb95436aac9f48fc309ea0aa7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 31606, "upload_time": "2019-03-22T00:22:45", "url": "https://files.pythonhosted.org/packages/78/97/a0c14e91a318903315f99cd9d32571c177582e620912111922057a2b76ca/sureal-0.3.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "18d42a30dbea03c012077bdff0b3d16e", "sha256": "900f8d87beb15779524d7745a5f652ed3121697dbbecdaefeb0818f11b41dd94" }, "downloads": -1, "filename": "sureal-0.3.1-py2-none-any.whl", "has_sig": false, "md5_digest": "18d42a30dbea03c012077bdff0b3d16e", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 38498, "upload_time": "2019-04-03T00:03:20", "url": "https://files.pythonhosted.org/packages/d1/32/c125fc3ea56ec74c1f5b3b584a73b08f27658a7090e369bf02b534186c8d/sureal-0.3.1-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6317c5cb3c0fb84a11875a77e1973031", "sha256": "569df7f4c79ff3b35f3d0dcec36a811675b81f039b75783808acd961b129ed29" }, "downloads": -1, "filename": "sureal-0.3.1.tar.gz", "has_sig": false, "md5_digest": "6317c5cb3c0fb84a11875a77e1973031", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32040, "upload_time": "2019-04-03T00:03:21", "url": "https://files.pythonhosted.org/packages/4e/38/2125f0b16c4e4b00506a77036769c7a312aa3a583dd1359b89ae295241f7/sureal-0.3.1.tar.gz" } ], "0.3.2": [ { "comment_text": "", "digests": { "md5": "a77f558a1a670f95910f8e59140b785d", "sha256": "ef7a0106a221481ecd6badb59aceb4f6d222e01dcdcf393beadb0de09115531f" }, "downloads": -1, "filename": "sureal-0.3.2-py2-none-any.whl", "has_sig": false, "md5_digest": "a77f558a1a670f95910f8e59140b785d", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 38493, "upload_time": "2019-04-03T00:12:33", "url": "https://files.pythonhosted.org/packages/f0/38/df11331e5646913451e77c958e03b1e3729558faabd6f19616f5fcda1ebd/sureal-0.3.2-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "135c3283c1202b24baf69c411d98d1ff", "sha256": "a718ba8d79055d77dfd4bfae766af6c2b7777cff262fd52713ad2394beca6559" }, "downloads": -1, "filename": "sureal-0.3.2.tar.gz", "has_sig": false, "md5_digest": "135c3283c1202b24baf69c411d98d1ff", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32038, "upload_time": "2019-04-03T00:12:35", "url": "https://files.pythonhosted.org/packages/b3/ff/26cfdc01451e46bb67fbe7796780208b7e475fabcbeadfbd10ea74532c1e/sureal-0.3.2.tar.gz" } ], "0.3.3": [ { "comment_text": "", "digests": { "md5": "824afe769e9cd4a220e0f8b6067c194a", "sha256": "ddebfc039a27513f1b3ef13c91430a42724e9ba1cb1c528b08a91135d7dd8526" }, "downloads": -1, "filename": "sureal-0.3.3-py2-none-any.whl", "has_sig": false, "md5_digest": "824afe769e9cd4a220e0f8b6067c194a", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 38488, "upload_time": "2019-04-03T00:20:05", "url": "https://files.pythonhosted.org/packages/80/6e/d0e8d1ab6ff449ac69d4d0dc8438e6d73a0e539157db0509c0ea316143b8/sureal-0.3.3-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "484a7f6e701f871ced599a8805d0bdbf", "sha256": "92286c04973b6a219c6dfb308863abfd2d911a43fdaa4d6c77c76281f0de584d" }, "downloads": -1, "filename": "sureal-0.3.3.tar.gz", "has_sig": false, "md5_digest": "484a7f6e701f871ced599a8805d0bdbf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32044, "upload_time": "2019-04-03T00:20:07", "url": "https://files.pythonhosted.org/packages/a2/75/1c75cea6afb825a7a1058346f73570b019c38deeae6a52c623621719eae6/sureal-0.3.3.tar.gz" } ], "0.3.4": [ { "comment_text": "", "digests": { "md5": "7389ccb7b892e58d82a15336919e3e8a", "sha256": "6d923d696e0429f9391ce36292fdc43a322ae1cd8851681fc6f2e998d9767852" }, "downloads": -1, "filename": "sureal-0.3.4-py2-none-any.whl", "has_sig": false, "md5_digest": "7389ccb7b892e58d82a15336919e3e8a", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 38491, "upload_time": "2019-04-03T00:32:01", "url": "https://files.pythonhosted.org/packages/ea/90/5046eb1dcb82ddc8f9af02e64c8a6bbfb1f5e4935923e352a85d52a30c97/sureal-0.3.4-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "5be3114af3d243c1956a5cef809ebe1b", "sha256": "c8cade9cb190fc9bbff251bf01e1b4c1a432ea6b408e67c20073e00eefba7589" }, "downloads": -1, "filename": "sureal-0.3.4.tar.gz", "has_sig": false, "md5_digest": "5be3114af3d243c1956a5cef809ebe1b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32048, "upload_time": "2019-04-03T00:32:05", "url": "https://files.pythonhosted.org/packages/59/19/467adb820e958037c4be858761f627eeae638a9ff436e73de7672c727776/sureal-0.3.4.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "c66e68543873984b1160f3b81df52a15", "sha256": "440a16162fe3ed9115c8a5cf909d25458150cb2937ebe41f5896f74a00f4b380" }, "downloads": -1, "filename": "sureal-0.4.0-py3-none-any.whl", "has_sig": false, "md5_digest": "c66e68543873984b1160f3b81df52a15", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 39785, "upload_time": "2019-06-02T03:53:58", "url": "https://files.pythonhosted.org/packages/a2/04/63e09c044787b82592d8e0145c3272364cefcbee66dda93419e72ef269cb/sureal-0.4.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9f8eaa9ae039484161f20a19493f4200", "sha256": "84510ca2613a2ceecbfc00cea9ec44074022dfb96dfa83f860e717d3c1a4f8b5" }, "downloads": -1, "filename": "sureal-0.4.0.tar.gz", "has_sig": false, "md5_digest": "9f8eaa9ae039484161f20a19493f4200", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33558, "upload_time": "2019-06-02T03:54:00", "url": "https://files.pythonhosted.org/packages/53/24/2bf09bb442b47cb3d60bbf0a33cefc7f934b19787615b0e93d5074a7c525/sureal-0.4.0.tar.gz" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "2c3696433e230fbe4d7176b9e9e75049", "sha256": "8ab206ab60de811337334a087bb43863ef446ac05424e551efc81d312c5ed227" }, "downloads": -1, "filename": "sureal-0.4.1-py3-none-any.whl", "has_sig": false, "md5_digest": "2c3696433e230fbe4d7176b9e9e75049", "packagetype": "bdist_wheel", "python_version": "3.7", "requires_python": null, "size": 39785, "upload_time": "2019-06-02T04:39:29", "url": "https://files.pythonhosted.org/packages/8c/b5/29b901f12b076273a3dd70953178ab04a9c4250ddafb27df1a30f79bda1b/sureal-0.4.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c71ad3046228de4bde935b94118baf28", "sha256": "51c77af34366fc51aec22c3ab278560ee6d9c3c819b594ddb23999f9996608f1" }, "downloads": -1, "filename": "sureal-0.4.1.tar.gz", "has_sig": false, "md5_digest": "c71ad3046228de4bde935b94118baf28", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33555, "upload_time": "2019-06-02T04:39:27", "url": "https://files.pythonhosted.org/packages/ab/0e/1869808e15e972dd78a29b838b054011b0fd07264b5a291b3d801099e8ce/sureal-0.4.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "2c3696433e230fbe4d7176b9e9e75049", "sha256": "8ab206ab60de811337334a087bb43863ef446ac05424e551efc81d312c5ed227" }, "downloads": -1, "filename": "sureal-0.4.1-py3-none-any.whl", "has_sig": false, "md5_digest": "2c3696433e230fbe4d7176b9e9e75049", "packagetype": "bdist_wheel", "python_version": "3.7", "requires_python": null, "size": 39785, "upload_time": "2019-06-02T04:39:29", "url": "https://files.pythonhosted.org/packages/8c/b5/29b901f12b076273a3dd70953178ab04a9c4250ddafb27df1a30f79bda1b/sureal-0.4.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c71ad3046228de4bde935b94118baf28", "sha256": "51c77af34366fc51aec22c3ab278560ee6d9c3c819b594ddb23999f9996608f1" }, "downloads": -1, "filename": "sureal-0.4.1.tar.gz", "has_sig": false, "md5_digest": "c71ad3046228de4bde935b94118baf28", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33555, "upload_time": "2019-06-02T04:39:27", "url": "https://files.pythonhosted.org/packages/ab/0e/1869808e15e972dd78a29b838b054011b0fd07264b5a291b3d801099e8ce/sureal-0.4.1.tar.gz" } ] }