{ "info": { "author": "Fritz Obermeyer", "author_email": "fritz.obermeyer@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4" ], "description": "|Build Status| |Code Quality| |Latest Version|\n\nGoftests\n========\n\nGeneric goodness of fit tests for random plain old data.\n\nGoftests is intended for unit testing random samplers that generate\narbitrary plain-old-data, and focuses on robustness rather than\nstatistical efficiency. In contrast to\n`scipy.stats `__\nand\n`statsmodels `__,\ngoftests does not make assumptions on the distribution being tested, and\nrequires only a simple (sample, prob) interface provided by MCMC\nsamplers.\n\nInstalling\n----------\n\n::\n\n pip install goftests\n\nUsing goodness of fit tests\n---------------------------\n\nGoftests implements generic statistical tests for Monte Carlo samplers\nthat generate (sample, probability) pairs. The four basic generic tests\nare\n\n- ``discrete_goodness_of_fit(samples, probs_dict, ...)`` - for\n discrete-valued data with most of the mass on a few values.\n\n- ``density_goodness_of_fit(samples, probs, ...)`` - for continuously\n distributed univariate data with no discrete component.\n\n- ``vector_density_goodness_of_fit(samples, probs, ...)`` - for\n continuously distributed multivariate data with no discrete\n component.\n\n- ``mixed_density_goodness_of_fit(samples, probs, ...)`` - for\n discretely-indexed continuously distributed data, subsuming the all\n the other tests.\n\nEach test outputs a goodness of fit number, say ``gof``, which should be\nuniformly distributed in the interval [0,1], and which tends to fail\nwith numbers close to zero. Thus to test a sampler we usually write\n\n::\n\n TEST_FAILURE_RATE = 1e-3\n\n def test_my_sampler(count=100):\n seed_all(0)\n samples = [my_sampler.rvs() for _ in xrange(count)]\n probs = [my_sampler.pdf(x) for x in samples]\n gof = mixed_density_goodness_of_fit(samples, probs)\n assert gof > TEST_FAILURE_RATE\n\nBy specifying a global ``TEST_FAILURE_RATE`` for a suite of tests, we\ncan tune the number of tests expected to fail. In a large suite of\ntests, some may fail at the specified seeds despite the distribution\nbeing correct, typically with gof just below the threshold. In this case\nwe usually increment the seed on those tests. In a suite of 1000 tests\nwith ``TEST_FAILURE_RATE = 1e-3``, we should expect to have to increment\nthe seed on 1 test, on average.\n\nDiagnosing common errors\n~~~~~~~~~~~~~~~~~~~~~~~~\n\nToo few samples\n^^^^^^^^^^^^^^^\n\nThe discrete tests only look at the most-likely few values, and requires\nenough samples in the top few values to be able to use Pearson's \u03c72\ntest. This is typically at least 100 samples.\n\nThe univariate tests require around 100 samples, and the multivariate\nrequire at least 1000 per dimension.\n\nToo many samples\n^^^^^^^^^^^^^^^^\n\nWhen testing for millions of samples or more, tests can fail due to\nnumeric imprecision rather than poor distribution. We have had success\ntesting with 102-105 samples.\n\nPoorly mixing Markov chains\n^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nWhen testing samples generated by MCMC, the samples are often correlated\nso that testing the entire chain fails. In this case, try running the\nchain for longer and subsampling only every kth step in the chain.\n\nSticky Markov chains\n^^^^^^^^^^^^^^^^^^^^\n\nThe density tests are sensitive to Sticky Markov chains. As with poorly\nmixing chains, try running for longer.\n\nSpuriously failing tests\n^^^^^^^^^^^^^^^^^^^^^^^^\n\nBe sure to set a deterministic seed via ``seed_all(...)`` before\ngenerating each dataset. In particular, some test runners run tests in a\nnondeterministic order, so setting the seed before generating each\nsample ensures deterministic behavior.\n\nImplementing new tests\n----------------------\n\nThe goodness of fit tests are mostly implemented by reduction to other\ntests, eventually reducing to the multinomial goodness of fit test which\nuses Pearson's \u03c72 test on each of the multinomial's bins.\n\n.. figure:: /doc/reductions.png\n :alt: Reductions\n\n Reductions\n\nTo implement a new test, you can implement from scratch, reduce to\nanother test in goftests, or reduce to standard tests in another package\nlike\n`scipy.stats `__\nor\n`statsmodels `__.\n\nReferences\n----------\n\n- [1] Peter J Bickel, Leo Breiman (1983) \"Sums of functions of nearest\n neighbor distancess, moment bounds, limit theorems and a goodness of\n fit test\"\n `pdf `__\n\n- [2] Mike Williams (2010) \"How good are your fits? Unbinned\n multivariate goodness-of-fit tests in high energy physics\"\n `pdf `__\n\nLicense\n-------\n\nCopyright (c) 2014 Salesforce.com, Inc. All rights reserved. Copyright\n(c) 2015 Gamelan Labs, Inc. Copyright (c) 2016 Google, Inc. Licensed\nunder the Revised BSD License. See `LICENSE.txt `__ for\ndetails.\n\n.. |Build Status| image:: https://travis-ci.org/posterior/goftests.svg?branch=master\n :target: https://travis-ci.org/posterior/goftests\n.. |Code Quality| image:: http://img.shields.io/scrutinizer/g/posterior/goftests.svg\n :target: https://scrutinizer-ci.com/g/posterior/goftests/code-structure/master/hot-spots\n.. |Latest Version| image:: https://badge.fury.io/py/goftests.svg\n :target: https://pypi.python.org/pypi/goftests\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/posterior/goftests", "keywords": "", "license": "Revised BSD", "maintainer": "", "maintainer_email": "", "name": "goftests", "package_url": "https://pypi.org/project/goftests/", "platform": "", "project_url": "https://pypi.org/project/goftests/", "project_urls": { "Homepage": "https://github.com/posterior/goftests" }, "release_url": "https://pypi.org/project/goftests/0.2.7/", "requires_dist": null, "requires_python": "", "summary": "Goodness of fit tests for general datatypes", "version": "0.2.7" }, "last_serial": 2977838, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "e0eaacef621ed75b38d8fbf10c8bc612", "sha256": "7a7adb55d13eedd0aa658b3cd5b9c5512fe780ec3b35e8b2a1be17606782541c" }, "downloads": -1, "filename": "goftests-0.1.0.tar.gz", "has_sig": false, "md5_digest": "e0eaacef621ed75b38d8fbf10c8bc612", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2082, "upload_time": "2015-02-17T21:23:29", "url": "https://files.pythonhosted.org/packages/3f/b4/06e4ee4bb89ef47d9ee452e5745099a762936cf3a35ed7d163002c7e8462/goftests-0.1.0.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "1d8031deefac4151137646f771c980d2", "sha256": "48ec2c8afb628c49f3f3bb0d47bce17650c51d089877b9354007302053c02a77" }, "downloads": -1, "filename": "goftests-0.1.3.tar.gz", "has_sig": false, "md5_digest": "1d8031deefac4151137646f771c980d2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2089, "upload_time": "2015-02-17T22:22:46", "url": "https://files.pythonhosted.org/packages/0c/3a/221a8b71d483f3186dca7f46b2b0d44951ac9d8639c4ed1689012e544bdb/goftests-0.1.3.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "d90c1c3ba25ca9455c501dd652c0ddc4", "sha256": "cc23987d6df9195cbea619f359ff8e7afcf18beebf127372d3724724b4b8ec19" }, "downloads": -1, "filename": "goftests-0.1.5.tar.gz", "has_sig": false, "md5_digest": "d90c1c3ba25ca9455c501dd652c0ddc4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4863, "upload_time": "2015-02-17T22:36:23", "url": "https://files.pythonhosted.org/packages/e8/a1/2a1e5370a71057080d1a1a53f76456e4b09c5cfd29689266b543652f5429/goftests-0.1.5.tar.gz" } ], "0.1.6": [ { "comment_text": "", "digests": { "md5": "849598ec9195f83bdd1577f07a3ed7d6", "sha256": "3cc0846e0f23f604684582bfdf1d8fc48294604c3bc376b2c5b953f8e3934175" }, "downloads": -1, "filename": "goftests-0.1.6.tar.gz", "has_sig": false, "md5_digest": "849598ec9195f83bdd1577f07a3ed7d6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4855, "upload_time": "2015-02-17T22:43:31", "url": "https://files.pythonhosted.org/packages/43/59/8ab094c0e085d1d95859fbd2416065ab53610cb45e3a4e059fc61a89fdfc/goftests-0.1.6.tar.gz" } ], "0.1.7": [ { "comment_text": "", "digests": { "md5": "164aa8dc2e909b48683bd56ac0a0317e", "sha256": "9e34d994696fca474e80dc203bf9ff9028fa8a207d0b986598f4541bb3797056" }, "downloads": -1, "filename": "goftests-0.1.7.tar.gz", "has_sig": false, "md5_digest": "164aa8dc2e909b48683bd56ac0a0317e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5473, "upload_time": "2015-02-17T23:02:28", "url": "https://files.pythonhosted.org/packages/10/37/9ae5f0699bc1aeed27978b7835047c0e23a0aa2e11abc4c5774d686c3486/goftests-0.1.7.tar.gz" } ], "0.1.8": [ { "comment_text": "", "digests": { "md5": "a990dc68189fd8d8e5ef0644c4b6f147", "sha256": "49f4a70e92391357f65786cdf4237d46b447e7a7ec109bf8b8dc4c3bcc1bf780" }, "downloads": -1, "filename": "goftests-0.1.8.tar.gz", "has_sig": false, "md5_digest": "a990dc68189fd8d8e5ef0644c4b6f147", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7823, "upload_time": "2015-02-18T05:10:30", "url": "https://files.pythonhosted.org/packages/05/8a/6d8ce4d249f6253f52bebc051db902fae7c846ef416e6d2700d7c2a40166/goftests-0.1.8.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "8c1bccd8028cb01a062e0502b1422e6f", "sha256": "3de2f4a9ce8eac6ca729e89a7292df418dc5b0e78eacdafa20e57bc46f03aadd" }, "downloads": -1, "filename": "goftests-0.2.0.tar.gz", "has_sig": false, "md5_digest": "8c1bccd8028cb01a062e0502b1422e6f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8127, "upload_time": "2015-02-18T23:23:14", "url": "https://files.pythonhosted.org/packages/6e/4e/836c6e92fe3ee0cc63c5d5f1a0127a7034e6cc051d576207cc5b05030317/goftests-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "ed5a12b0620795843c5f62bd6a6b187d", "sha256": "99459767f7abc338f30008440ba593221f39d7c843d71f190532ee34fb5ba60c" }, "downloads": -1, "filename": "goftests-0.2.1.tar.gz", "has_sig": false, "md5_digest": "ed5a12b0620795843c5f62bd6a6b187d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9843, "upload_time": "2016-06-08T03:22:29", "url": "https://files.pythonhosted.org/packages/f8/82/dfb09dbb617d814f8d343203a6ba31579178d422d738ceb2139f6db38251/goftests-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "566b3cb47ff4abdf7d96c171c8b96bea", "sha256": "c6a2ba17258ffccb776f894915a5f5d482d5b3da498d1902e8a6e77944ea855c" }, "downloads": -1, "filename": "goftests-0.2.2.tar.gz", "has_sig": false, "md5_digest": "566b3cb47ff4abdf7d96c171c8b96bea", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8323, "upload_time": "2016-06-08T03:29:36", "url": "https://files.pythonhosted.org/packages/25/6c/7a334306f524fba2b3b5bd77b58a29d476c2c2c776ea17f911f8d237dbe5/goftests-0.2.2.tar.gz" } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "9d21be396392744e13e4c392550f484a", "sha256": "786e6877ac327576a07a50da47a28df28340cac8028cfd992660234ab34f5910" }, "downloads": -1, "filename": "goftests-0.2.3.tar.gz", "has_sig": false, "md5_digest": "9d21be396392744e13e4c392550f484a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8875, "upload_time": "2016-06-26T00:46:10", "url": "https://files.pythonhosted.org/packages/ce/d4/ed689341b27de2486b241a369a279d8eaac9fd0dc659deeaede3dcd26619/goftests-0.2.3.tar.gz" } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "556969f0e02feccd1cd3e3e81ffabbe6", "sha256": "a450bed49d375b5d41678182682d04555037e5e2db5b93031b7cc05aee0a383a" }, "downloads": -1, "filename": "goftests-0.2.4.tar.gz", "has_sig": false, "md5_digest": "556969f0e02feccd1cd3e3e81ffabbe6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9015, "upload_time": "2017-03-16T06:31:59", "url": "https://files.pythonhosted.org/packages/2d/d6/f83c867e7817ccda3a2b426b7e80c45297314f2b536116af589539c79a06/goftests-0.2.4.tar.gz" } ], "0.2.5": [ { "comment_text": "", "digests": { "md5": "af4629786f869b2ed4327a024d430ae3", "sha256": "90212ba69fca85c5ec8d280a45b475f787fb914a57b2a836fc545c661b182a13" }, "downloads": -1, "filename": "goftests-0.2.5.tar.gz", "has_sig": false, "md5_digest": "af4629786f869b2ed4327a024d430ae3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9318, "upload_time": "2017-03-17T03:15:54", "url": "https://files.pythonhosted.org/packages/15/3f/08dec679d2b8c34ba1620677e675f0c842e8046e6cc5fb52d051cc185871/goftests-0.2.5.tar.gz" } ], "0.2.6": [ { "comment_text": "", "digests": { "md5": "4308f2d908f05626803eb253a18adaaf", "sha256": "c61ca2bfd35c72d9cac9aac467f5aa25a892857dfc305e4102d6f2448cc0998f" }, "downloads": -1, "filename": "goftests-0.2.6.tar.gz", "has_sig": false, "md5_digest": "4308f2d908f05626803eb253a18adaaf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12016, "upload_time": "2017-06-25T18:32:35", "url": "https://files.pythonhosted.org/packages/4c/9b/0654c7e16435c0f89b9a1944f45e89104a0510145362e0aead6879e760e9/goftests-0.2.6.tar.gz" } ], "0.2.7": [ { "comment_text": "", "digests": { "md5": "a2383f5d038cbee4acf17dad900083d3", "sha256": "e6cd0dba04aeb364ee6482277ac08d24d02dc441e764b4088e7d29c468302ffa" }, "downloads": -1, "filename": "goftests-0.2.7.tar.gz", "has_sig": false, "md5_digest": "a2383f5d038cbee4acf17dad900083d3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12010, "upload_time": "2017-06-25T18:33:55", "url": "https://files.pythonhosted.org/packages/2d/a6/ff51317b377d8e1564c972e9889995c73a8465665d9fe9b57b6b9d276dee/goftests-0.2.7.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "a2383f5d038cbee4acf17dad900083d3", "sha256": "e6cd0dba04aeb364ee6482277ac08d24d02dc441e764b4088e7d29c468302ffa" }, "downloads": -1, "filename": "goftests-0.2.7.tar.gz", "has_sig": false, "md5_digest": "a2383f5d038cbee4acf17dad900083d3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12010, "upload_time": "2017-06-25T18:33:55", "url": "https://files.pythonhosted.org/packages/2d/a6/ff51317b377d8e1564c972e9889995c73a8465665d9fe9b57b6b9d276dee/goftests-0.2.7.tar.gz" } ] }