{ "info": { "author": "Curtis G. Northcutt", "author_email": "cgn@mit.edu", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "Intended Audience :: Education", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Operating System :: MacOS", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Operating System :: Unix", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Scientific/Engineering", "Topic :: Scientific/Engineering :: Artificial Intelligence", "Topic :: Scientific/Engineering :: Mathematics", "Topic :: Software Development", "Topic :: Software Development :: Libraries", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": ".. figure:: https://raw.githubusercontent.com/cgnorthcutt/cleanlab/master/img/cleanlab_logo.png\n :target: https://github.com/cgnorthcutt/cleanlab/\n :align: center\n :alt: cleanlab \n\n| \n\n``cleanlab`` is a machine learning python package for **learning with noisy labels** and **finding label errors in datasets**. ``cleanlab`` CLEANs LABels. It is is powered by the theory of **confident learning**.\n\n|pypi| |py_versions| |build_status| |coverage|\n\n.. |pypi| image:: https://img.shields.io/pypi/v/cleanlab.svg\n :target: https://pypi.org/pypi/cleanlab/\n.. |py_versions| image:: https://img.shields.io/pypi/pyversions/cleanlab.svg\n :target: https://pypi.org/pypi/cleanlab/\n.. |build_status| image:: https://travis-ci.com/cgnorthcutt/cleanlab.svg?branch=master\n :target: https://travis-ci.com/cgnorthcutt/cleanlab\n.. |coverage| image:: https://codecov.io/gh/cgnorthcutt/cleanlab/branch/master/graph/badge.svg\n :target: https://codecov.io/gh/cgnorthcutt/cleanlab\n\n``cleanlab`` finds and cleans label errors in any dataset using state-of-the-art algorithms for learning with noisy labels by characterizing label noise. ``cleanlab`` is fast: its built on optimized algorithms and parallelized across CPU threads automatically. ``cleanlab`` implements the family of theory and algorithms called **confident learning** with provable guarantees of exact noise estimation and label error finding (even when model output probabilities are noisy/imperfect). \n\nHow does **confident learning** work? Find out here: `TUTORIAL: confident learning with just numpy and for-loops `__.\n\n``cleanlab`` supports multi-label, multiclass, sparse matrices, and more. \n\n\nIts called ``cleanlab`` because it CLEANs LABels.\n================================================= \n\n``cleanlab`` is:\n\n1. fast - Single-shot, non-iterative, parallelized algorithms (e.g. < 1 second to find label errors in ImageNet)\n2. robust - Provable generalization and risk minimimzation guarantees, including imperfect probability estimation.\n3. general - Works with any probablistic classifier: PyTorch, Tensorflow, MxNet, Caffe2, scikit-learn, etc.\n4. unique - The only package for multiclass learning with noisy labels or finding label errors for any dataset / classifier.\n\n\nFind label errors with PyTorch, Tensorflow, MXNet, etc. in 1 line of code!\n==========================================================================\n\n.. code:: python\n\n # Compute psx (n x m matrix of predicted probabilities) on your own, with any classifier.\n # Be sure you compute probs in a holdout/out-of-sample manner (e.g. cross-validation)\n # Now getting label errors is trivial with cleanlab... its one line of code.\n # Label errors are ordered by likelihood of being an error. First index is most likely error.\n from cleanlab.pruning import get_noise_indices\n\n ordered_label_errors = get_noise_indices(\n s = numpy_array_of_noisy_labels,\n psx = numpy_array_of_predicted_probabilities,\n sorted_index_method='normalized_margin', # Orders label errors\n )\n\n\nLearning with noisy labels in 3 lines of code!\n==============================================\n\n.. code:: python\n\n from cleanlab.classification import LearningWithNoisyLabels\n from sklearn.linear_model import LogisticRegression\n\n # Wrap around any classifier. Yup, you can use sklearn/pyTorch/Tensorflow/FastText/etc.\n lnl = LearningWithNoisyLabels(clf=LogisticRegression()) \n lnl.fit(X = X_train_data, s = train_noisy_labels) \n # Estimate the predictions you would have gotten by training with *no* label errors.\n predicted_test_labels = lnl.predict(X_test)\n\n\nCheck out these `examples `__ and `tests `__ (includes how to use pyTorch, FastText, etc.).\n\n\n\nInstallation\n============\n\nPython 2.7, 3.4, 3.5, and 3.6 are supported.\n\nStable release:\n\n.. code-block:: bash\n\n $ pip install cleanlab\n\nDeveloper (unstable) release:\n\n.. code-block:: bash\n\n $ pip install git+https://github.com/cgnorthcutt/cleanlab.git\n\nTo install the codebase (enabling you to make modifications):\n\n.. code-block:: bash\n\n $ conda update pip # if you use conda\n $ git clone https://github.com/cgnorthcutt/cleanlab.git\n $ cd cleanlab\n $ pip install -e .\n\n\nCitations and Related Publications\n==================================\n\nIf you use this package in your work, please cite the following:\n\n::\n\n @misc{northcutt2019cleanlab,\n author = {Curtis Northcutt},\n title = {Clean Lab},\n year = {2019},\n howpublished = {\\url{https://github.com/cgnorthcutt/cleanlab}},\n note = {commit xxxxxxx, version xxxx}\n }\n\nIf you compare with, build on, or use confident learning (the theory and methods behind cleanlab), please cite the following. We will release a paper strictly on confident learning later this year, in addition to other related publications.\n\n::\n\n @inproceedings{northcutt2017rankpruning,\n author={Northcutt, Curtis G. and Wu, Tailin and Chuang, Isaac L.},\n title={Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels},\n booktitle = {Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence},\n series = {UAI'17},\n year = {2017},\n location = {Sydney, Australia},\n numpages = {10},\n url = {http://auai.org/uai2017/proceedings/papers/35.pdf},\n publisher = {AUAI Press},\n } \n\nCollaboration\n-------------\n\nMost of the algorithms, theory, and results of ``cleanlab`` remain unpublished. If you'd like to work together, please reach out. \n\n``cleanlab`` on MNIST\n---------------------\n\nWe use ``cleanlab`` to automatically identify ~50 label errors in the MNIST dataset. \n\n.. figure:: https://raw.githubusercontent.com/cgnorthcutt/cleanlab/master/img/mnist_training_label_errors24_prune_by_noise_rate.png\n :align: center\n :alt: Image depicting label errors in MNIST train set \n\nLabel errors of the original MNIST **train** dataset identified algorithmically using the rankpruning algorithm. Depicts the 24 least confident labels, ordered left-right, top-down by increasing self-confidence (probability of belonging to the given label), denoted conf in teal. The label with the largest predicted probability is in green. Overt errors are in red.\n\n\n``cleanlab`` Generality: View performance across 4 distributions and 9 classifiers.\n-----------------------------------------------------------------------------------\n\nWe use ``cleanlab`` to automatically learn with noisy labels regardless of dataset distribution or classifier. \n\n.. figure:: https://raw.githubusercontent.com/cgnorthcutt/cleanlab/master/img/demo_cleanlab_across_datasets_and_classifiers.png\n :align: center\n :alt: Image depicting generality of cleanlab across datasets and classifiers \n\nEach figure depicts the decision boundary learned using ``cleanlab.classification.LearningWithNoisyLabels`` in the presence of extreme (~35%) label errors. Label errors are circled in green. Label noise is class-conditional (not simply uniformly random). Columns are organized by the classifier used, except the left-most column which depicts the ground-truth dataset distribution. Rows are organized by dataset used. A matrix characterizing the label noise for the first row is shown below. \n\nEach figure depicts accuracy scores on a test set as decimal values: \n\n1. LEFT (in black): The classifier test accuracy trained with perfect labels (no label errors). \n2. MIDDLE (in blue): The classifier test accuracy trained with noisy labels using ``cleanlab``. \n3. RIGHT (in white): The baseline classifier test accuracy trained with noisy labels.\n\nAs an example, this is the noise matrix (noisy channel) *P(s \\| y)* characterizing the label noise for the first dataset row in the figure. *s* represents the observed noisy labels and *y* represents the latent, true labels. The trace of this matrix is 2.6. A trace of 4 implies no label noise. A cell in this matrix is read like, \"A random 38% of '3' labels were flipped to '2' labels.\"\n\n====== ==== ==== ==== ==== \np(s|y) y=0 y=1 y=2 y=3\n====== ==== ==== ==== ==== \ns=0 0.55 0.01 0.07 0.06\ns=1 0.22 0.87 0.24 0.02\ns=2 0.12 0.04 0.64 0.38\ns=3 0.11 0.08 0.05 0.54\n====== ==== ==== ==== ====\n\nThe code to reproduce this figure is available `here `__.\n\n\nGet started with easy, quick examples.\n======================================\n\nNew to **cleanlab**? Start with:\n\n1. `Visualizing confident\n learning `__\n2. `A simple example of learning with noisy labels on the multiclass\n Iris dataset `__.\n\nThese examples show how easy it is to characterize label noise in\ndatasets, learn with noisy labels, identify label errors, estimate\nlatent priors and noisy channels, and more.\n\n.. ..\n\n \n\nUse ``cleanlab`` with any model (Tensorflow, caffe2, PyTorch, etc.)\n-------------------------------------------------------------------\n\nAll of the features of the ``cleanlab`` package work with **any model**.\nYes, any model. Feel free to use PyTorch, Tensorflow, caffe2,\nscikit-learn, mxnet, etc. If you use a scikit-learn classifier, all\n``cleanlab`` methods will work out-of-the-box. It\u2019s also easy to use\nyour favorite model from a non-scikit-learn package, just wrap your\nmodel into a Python class that inherits the\n``sklearn.base.BaseEstimator``:\n\n.. code:: python\n\n from sklearn.base import BaseEstimator\n class YourFavoriteModel(BaseEstimator): # Inherits sklearn base classifier\n def __init__(self, ):\n pass\n def fit(self, X, y, sample_weight = None):\n pass\n def predict(self, X):\n pass\n def predict_proba(self, X):\n pass\n def score(self, X, y, sample_weight = None):\n pass\n\n # Now you can use your model with `cleanlab`. Here's one example:\n from cleanlab.classification import LearningWithNoisyLabels\n lnl = LearningWithNoisyLabels(clf=YourFavoriteModel())\n lnl.fit(train_data, train_labels_with_errors)\n\nWant to see a working example? `Here\u2019s a compliant PyTorch MNIST CNN class `__\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nAs you can see\n`here `__,\ntechnically you don\u2019t actually need to inherit from\n``sklearn.base.BaseEstimator``, as you can just create a class that\ndefines .fit(), .predict(), and .predict_proba(), but inheriting makes\ndownstream scikit-learn applications like hyper-parameter optimization\nwork seamlessly. For example, the `LearningWithNoisyLabels()\nmodel `__\nis fully compliant.\n\nNote, some libraries exists to do this for you. For pyTorch, check out\nthe ``skorch`` Python library which will wrap your ``pytorch`` model\ninto a ``scikit-learn`` compliant model.\n\n\nDocumentation by Example\n========================\n\n``cleanlab`` Core Package Components\n------------------------------------\n\n1. **cleanlab/classification.py** - The LearningWithNoisyLabels() class for learning with noisy labels.\n2. **cleanlab/latent_algebra.py** -\tEqualities when noise information is known.\n3. **cleanlab/latent_estimation.py** -\tEstimates and fully characterizes all variants of label noise.\n4. **cleanlab/noise_generation.py** - Generate mathematically valid synthetic noise matrices.\n5. **cleanlab/polyplex.py** -\tCharacterizes joint distribution of label noise EXACTLY from noisy channel.\n6. **cleanlab/pruning.py** - Finds the indices of the examples with label errors in a dataset.\n\nMany of these methods have default parameters that won\u2019t be covered\nhere. Check out the method docstrings for full documentation.\n\nMulticlass learning with noisy labels (in **3** lines of code):\n---------------------------------------------------------------\n\n**rankpruning** is a fast, general, robust algorithm for multiclass\nlearning with noisy labels. It adds minimal overhead, needing only\n*O(nm2)* time for n training examples and m classes, works with any\nclassifier, and is easy to use. Here is the example from above, with\nadded commments for clarity.\n\n.. code:: python\n\n # LearningWithNoisyLabels implements a faster,\n # cross-platform and more-compatible version of the RankPruning\n # algorithm for learning with noisy labels. Unlike the original\n # algorithm which only worked for binary classification,\n # LearningWithNoisyLabels generalizes the theory and algorithms\n # of RankPruning for any number of classes.\n from cleanlab.classification import LearningWithNoisyLabels\n # LearningWithNoisyLabels uses logreg by default, so this is unnecessary. \n # We include it here for clarity, but this step is omitted below.\n from sklearn.linear_model import LogisticRegression as logreg\n\n # 1.\n # Wrap around any classifier. Yup, neural networks work, too.\n lnl = LearningWithNoisyLabels(clf=logreg()) \n\n # 2.\n # X_train is numpy matrix of training examples (integers for large data)\n # train_labels_with_errors is a numpy array of labels of length n (# of examples), usually denoted 's'.\n lnl.fit(X_train, train_labels_with_errors) \n\n # 3.\n # Estimate the predictions you would have gotten by training with *no* label errors.\n predicted_test_labels = lnl.predict(X_test)\n\nEstimate the confident joint, the latent noisy channel matrix, *P(s \\| y)* and inverse, *P(y \\| s)*, the latent prior of the unobserved, actual true labels, *p(y)*, and the predicted probabilities.\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n*s* denotes a random variable that represents the observed, noisy\nlabel and *y* denotes a random variable representing the hidden, actual\nlabels. Both *s* and *y* take any of the m classes as values. The\n``cleanlab`` package supports different levels of granularity for\ncomputation depending on the needs of the user. Because of this, we\nsupport multiple alternatives, all no more than a few lines, to estimate\nthese latent distribution arrays, enabling the user to reduce\ncomputation time by only computing what they need to compute, as seen in\nthe examples below.\n\nThroughout these examples, you\u2019ll see a variable called\n*confident_joint*. The confident joint is an m x m matrix (m is the\nnumber of classes) that counts, for every observed, noisy class, the\nnumber of examples that confidently belong to every latent, hidden\nclass. It counts the number of examples that we are confident are\nlabeled correctly or incorrectly for every pair of obseved and\nunobserved classes. The confident joint is an unnormalized estimate of\nthe complete-information latent joint distribution, *Ps,y*. Most of the\nmethods in the **cleanlab** package start by first estimating the\n*confident_joint*.\n\nOption 1: Compute the confident joint and predicted probs first. Stop if that\u2019s all you need.\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: python\n\n from cleanlab.latent_estimation import estimate_latent\n from cleanlab.latent_estimation import estimate_confident_joint_and_cv_pred_proba\n\n # Compute the confident joint and the n x m predicted probabilities matrix (psx),\n # for n examples, m classes. Stop here if all you need is the confident joint.\n confident_joint, psx = estimate_confident_joint_and_cv_pred_proba(\n X=X_train, \n s=train_labels_with_errors,\n clf = logreg(), # default, you can use any classifier\n )\n\n # Estimate latent distributions: p(y) as est_py, P(s|y) as est_nm, and P(y|s) as est_inv\n est_py, est_nm, est_inv = estimate_latent(confident_joint, s=train_labels_with_errors)\n\nOption 2: Estimate the latent distribution matrices in a single line of code.\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: python\n\n from cleanlab.latent_estimation import estimate_py_noise_matrices_and_cv_pred_proba\n est_py, est_nm, est_inv, confident_joint, psx = estimate_py_noise_matrices_and_cv_pred_proba(\n X=X_train,\n s=train_labels_with_errors,\n )\n\nOption 3: Skip computing the predicted probabilities if you already have them.\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: python\n\n # Already have psx? (n x m matrix of predicted probabilities)\n # For example, you might get them from a pre-trained model (like resnet on ImageNet)\n # With the cleanlab package, you estimate directly with psx.\n from cleanlab.latent_estimation import estimate_py_and_noise_matrices_from_probabilities\n est_py, est_nm, est_inv, confident_joint = estimate_py_and_noise_matrices_from_probabilities(\n s=train_labels_with_errors, \n psx=psx,\n )\n\nEstimate label errors in a dataset:\n-----------------------------------\n\nWith the ``cleanlab`` package, we can instantly fetch the indices of all\nestimated label errors, with nothing provided by the user except a\nclassifier, examples, and their noisy labels. Like the previous example,\nthere are various levels of granularity.\n\n.. code:: python\n\n from cleanlab.pruning import get_noise_indices\n # We computed psx, est_inv, confident_joint in the previous example.\n label_errors = get_noise_indices(\n s=train_labels_with_errors, # required\n psx=psx, # required\n inverse_noise_matrix=est_inv, # not required, include to avoid recomputing\n confident_joint=confident_joint, # not required, include to avoid recomputing\n )\n\nEstimate the latent joint probability distribution matrix of the noisy and true labels, *P(s,y)*:\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nTo compute *P(s,y)*, the complete-information\ndistribution matrix that captures the number of pairwise label flip\nerrors when multipled by the total number of examples as *n* P(s,y)*.\nUsing `cleanlab.latent_estimation.calibrate_confident_joint`, \nthis method guarantees the rows of *P(s,y)* correctly sum to *p(s)*, \nand np.sum(confident_joint) == n (the number of labels).\n\nThis method occurs when hyperparameter prune_count_method =\n\u2018inverse_nm_dot_s\u2019 in LearningWithNoisyLabels.fit() and get_noise_indices().\n\n.. code:: python\n\n from cleanlab.latent_estimation import compute_confident_joint\n joint = compute_confident_joint(s=noisy_labels, psx=probabilities)\n\nIf you've already computed the confident joint, then you can\nestimate the complete joint distribution of label noise by:\n\n.. code:: python\n\n from cleanlab.latent_estimation import estimate_joint\n joint = estimate_joint(confident_joint=cj, s=noisy_labels)\n\nGenerate valid, class-conditional, unformly random noisy channel matrices:\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: python\n\n # Generate a valid (necessary conditions for learnability are met) noise matrix for any trace > 1\n from cleanlab.noise_generation import generate_noise_matrix_from_trace\n noise_matrix = generate_noise_matrix_from_trace(\n K = number_of_classes, \n trace = float_value_greater_than_1_and_leq_K,\n py = prior_of_y_actual_labels_which_is_just_an_array_of_length_K,\n frac_zero_noise_rates = float_from_0_to_1_controlling_sparsity,\n )\n\n # Check if a noise matrix is valid (necessary conditions for learnability are met)\n from cleanlab.noise_generation import noise_matrix_is_valid\n is_valid = noise_matrix_is_valid(noise_matrix, prior_of_y_which_is_just_an_array_of_length_K)\n\nSupport for numerous *weak supervision* and *learning with noisy labels* functionalities:\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: python\n\n # Generate noisy labels using the noise_marix. Guarantees exact amount of noise in labels.\n from cleanlab.noise_generation import generate_noisy_labels\n s_noisy_labels = generate_noisy_labels(y_hidden_actual_labels, noise_matrix)\n\n # This package is a full of other useful methods for learning with noisy labels.\n # The tutorial stops here, but you don't have to. Inspect method docstrings for full docs.\n\n\nThe Polyplex\n------------\n\nThe key to learning in the presence of label errors is estimating the joint distribution between the actual, hidden labels \u2018*y*\u2019 and the observed, noisy labels \u2018*s*\u2019. Using ``cleanlab`` and the theory of confident learning, we can completely characterize the trace of the latent joint distribution, *trace(P(s,y))*, given *p(y)*, for any fraction of label errors, i.e.\u00a0for any trace of the noisy channel, *trace(P(s|y))*.\n\nYou can check out how to do this yourself here: 1. `Drawing\nPolyplices `__ 2. `Computing\nPolyplices `__\n\nLicense\n-------\n\nCopyright (c) 2017-2019 Curtis Northcutt. Released under the MIT License. See `LICENSE `__ for details.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/cgnorthcutt/cleanlab", "keywords": "machine_learning denoising classification weak_supervision learning_with_noisy_labels unsupervised_learning", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "cleanlab", "package_url": "https://pypi.org/project/cleanlab/", "platform": "", "project_url": "https://pypi.org/project/cleanlab/", "project_urls": { "Homepage": "https://github.com/cgnorthcutt/cleanlab" }, "release_url": "https://pypi.org/project/cleanlab/0.0.14/", "requires_dist": [ "numpy (>=1.11.3)", "scikit-learn (>=0.18)", "scipy (>=1.1.0)" ], "requires_python": "", "summary": "The Python package for cleaning and learning with noisy labels. Works for all noisy label distributions, datasets, and models.", "version": "0.0.14" }, "last_serial": 5899660, "releases": { "0.0.10": [ { "comment_text": "", "digests": { "md5": "5a2b7f83c5213e8da2af9ae40492d61e", "sha256": "9a73b8206c4413c52532cf92ecfbb1d0106e4b78d0d2bc8c53c29c27c66dca6b" }, "downloads": -1, "filename": "cleanlab-0.0.10-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "5a2b7f83c5213e8da2af9ae40492d61e", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 57242, "upload_time": "2019-07-28T14:55:35", "url": "https://files.pythonhosted.org/packages/9a/e8/648f6c08ae7c09e65137c473ee0efee5f4fb17a8d929aa8d34c76b30157b/cleanlab-0.0.10-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e50cc2924f37ebcad13a0d4f264b1732", "sha256": "6b7f2e3610cc9a96019f07bb3e402c246703d5a637ca5079156d149715b8ab4d" }, "downloads": -1, "filename": "cleanlab-0.0.10.tar.gz", "has_sig": false, "md5_digest": "e50cc2924f37ebcad13a0d4f264b1732", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 55946, "upload_time": "2019-07-28T14:55:37", "url": "https://files.pythonhosted.org/packages/00/2a/2122a0b8e7b46efe7b7643c4540d8cb28ce0aabf708e6e31c3561ba5e595/cleanlab-0.0.10.tar.gz" } ], "0.0.11": [ { "comment_text": "", "digests": { "md5": "d87fd5b97930764357b9a1befc67e565", "sha256": "b275cc82bd4177291544e4632f02eb5cb61885771748eacfbf1a1e402a29c1da" }, "downloads": -1, "filename": "cleanlab-0.0.11-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d87fd5b97930764357b9a1befc67e565", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 50155, "upload_time": "2019-07-30T01:18:02", "url": "https://files.pythonhosted.org/packages/6b/ee/ffc47a5dcb250a6e4817e1dd410cee78b24b8337e4e0b580e7f902347dfd/cleanlab-0.0.11-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8f8a963b26d88a7f7a920c19c8125475", "sha256": "f8160af85ac5a5e25c870a8308348f897f34afdb17917014c403fb9629adf427" }, "downloads": -1, "filename": "cleanlab-0.0.11.tar.gz", "has_sig": false, "md5_digest": "8f8a963b26d88a7f7a920c19c8125475", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 50636, "upload_time": "2019-07-30T01:18:05", "url": "https://files.pythonhosted.org/packages/e8/de/40b5b64995a0d71f2fdacf177100d86927355b7ed8926331effa6c30c7d1/cleanlab-0.0.11.tar.gz" } ], "0.0.12": [ { "comment_text": "", "digests": { "md5": "b4bd9bf7a8b4bfd43e8d2df15af80ac9", "sha256": "a24f2c9701b41ab31b8a7ab4131f5f69c497635a313836e2f915260fd9f4ec97" }, "downloads": -1, "filename": "cleanlab-0.0.12-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "b4bd9bf7a8b4bfd43e8d2df15af80ac9", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 50158, "upload_time": "2019-08-19T22:26:11", "url": "https://files.pythonhosted.org/packages/28/ce/2f9905c75b6852cb6085130a2df53fb2ef4c9892c647378cfdc28bf8085d/cleanlab-0.0.12-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1f96a4b420e0eda7b84f09f85c11fef5", "sha256": "9cb8003d2fb706cd9d04fdac66fcbca3067720929622ddc7b017cb5e6493dd89" }, "downloads": -1, "filename": "cleanlab-0.0.12.tar.gz", "has_sig": false, "md5_digest": "1f96a4b420e0eda7b84f09f85c11fef5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 50650, "upload_time": "2019-08-19T22:26:12", "url": "https://files.pythonhosted.org/packages/31/1f/fcbde46ddd7314d18cddd2187cafb1e859f52fcd266cbdadb606e23ad30e/cleanlab-0.0.12.tar.gz" } ], "0.0.13": [ { "comment_text": "", "digests": { "md5": "6182830879292ad0fde873cc9a0b0eb6", "sha256": "559e50e41ff2dd6c60e66e85fd0f72d7bc7d0876eba028091dcbfff52919ece2" }, "downloads": -1, "filename": "cleanlab-0.0.13-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "6182830879292ad0fde873cc9a0b0eb6", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 51853, "upload_time": "2019-09-27T15:50:54", "url": "https://files.pythonhosted.org/packages/aa/73/69661f5d41601efa6168ed1d8912716648e5742ecd112ec9ded58704dee1/cleanlab-0.0.13-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "5ea4adef18b23b823aed2f545f3749b2", "sha256": "d5f924245b1a614a674ddf65b23351160e0af0929d04a708a6ac8287f41fe7f0" }, "downloads": -1, "filename": "cleanlab-0.0.13.tar.gz", "has_sig": false, "md5_digest": "5ea4adef18b23b823aed2f545f3749b2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 52049, "upload_time": "2019-09-27T15:50:56", "url": "https://files.pythonhosted.org/packages/13/47/6035694b598971fa871a0f0f300c29a8f7f01fc7e4dc1b953bd0f62a1f3c/cleanlab-0.0.13.tar.gz" } ], "0.0.14": [ { "comment_text": "", "digests": { "md5": "fc27cc6919520e5bae0daaf61c82ff4f", "sha256": "6ce360bef72db2ef435cf2de43c2f68de5326401f8f169b102e4e9fe401b8668" }, "downloads": -1, "filename": "cleanlab-0.0.14-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "fc27cc6919520e5bae0daaf61c82ff4f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 51874, "upload_time": "2019-09-28T13:21:57", "url": "https://files.pythonhosted.org/packages/6f/8b/88d8e5cd8119b07c5cdd9efa1619b0f6a111fe8aacd50a0ce4c6942486b9/cleanlab-0.0.14-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "303e7dbde7d858074f56584293568108", "sha256": "28ea1167c76fa9e44463b4ae5d2e582afe4e20047112aae6a1794bf4e0f00ee8" }, "downloads": -1, "filename": "cleanlab-0.0.14.tar.gz", "has_sig": false, "md5_digest": "303e7dbde7d858074f56584293568108", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 45549, "upload_time": "2019-09-28T13:21:58", "url": "https://files.pythonhosted.org/packages/63/f0/bbeee2ce7484980728096b9e071faa4e278b4b11220cb8c779fbf49aacf5/cleanlab-0.0.14.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "ee77d32de781616175e78ac1be37c4b9", "sha256": "4bd0dfa7a634335d16d5589b5cc40f29c166fd0cac6cdde4dc1087365c9f2a83" }, "downloads": -1, "filename": "cleanlab-0.0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ee77d32de781616175e78ac1be37c4b9", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 47742, "upload_time": "2018-11-22T21:40:36", "url": "https://files.pythonhosted.org/packages/15/be/60f6e25fb7098cfd49b8ba727df365f655a58a5aa24671fd7ddadc98e99c/cleanlab-0.0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ab78dec5b7820324df446074609316a9", "sha256": "0e86af32372e883f783615fbd336804454b91b49a3578545c917663cbda81ad2" }, "downloads": -1, "filename": "cleanlab-0.0.2.tar.gz", "has_sig": false, "md5_digest": "ab78dec5b7820324df446074609316a9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42880, "upload_time": "2018-11-22T21:40:39", "url": "https://files.pythonhosted.org/packages/8f/07/4d0d6860eef88b44a2eeaddeb26364d3808796fe555d1108f06fea5f116b/cleanlab-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "c3a2972830b632063a3deb1c5bc0f9fe", "sha256": "22a9ab854068243cf60bfa7fa88fbd25571e4b096cde34e1a759d3494e9992c2" }, "downloads": -1, "filename": "cleanlab-0.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "c3a2972830b632063a3deb1c5bc0f9fe", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 47845, "upload_time": "2018-11-22T21:46:14", "url": "https://files.pythonhosted.org/packages/bc/38/27e6e40ddd7abf9b2e646252f24851adc96f8d1a9f5118ed9520705cb50f/cleanlab-0.0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2b4e517ed2252fba4c2a695549451866", "sha256": "3451233db618aa1e26cba972916fcef8df3469c05ff50d90ded6a73b8a10a050" }, "downloads": -1, "filename": "cleanlab-0.0.3.tar.gz", "has_sig": false, "md5_digest": "2b4e517ed2252fba4c2a695549451866", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42948, "upload_time": "2018-11-22T21:46:16", "url": "https://files.pythonhosted.org/packages/ef/22/10e9ef5c44c37be1995421f2a4d66e03db8287bcd4909e94b314236e1c96/cleanlab-0.0.3.tar.gz" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "1c54217eba65cbc1386b65dc4bb29ad7", "sha256": "c54211a1532b41f20f6b6eebad86052683f1b31d42da0c34bf7f5edb101d78e8" }, "downloads": -1, "filename": "cleanlab-0.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "1c54217eba65cbc1386b65dc4bb29ad7", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 47640, "upload_time": "2018-11-26T03:50:01", "url": "https://files.pythonhosted.org/packages/d5/2f/6d66353a643f40c608b0e32fadd05be086ef973016c1f7fd22b52358f5ac/cleanlab-0.0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "19d3f32c1b8ab3614f5cb876511db8d5", "sha256": "276a4075e07ac9e37b7d431f618a6b18af64cc69b90d2d6ae7a724599659fd87" }, "downloads": -1, "filename": "cleanlab-0.0.4.tar.gz", "has_sig": false, "md5_digest": "19d3f32c1b8ab3614f5cb876511db8d5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42845, "upload_time": "2018-11-26T03:50:03", "url": "https://files.pythonhosted.org/packages/36/e9/0f703f9171fb21939fd5b977135024fba259d5ae3aad1300a25f2a0b53f1/cleanlab-0.0.4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "e50f49f545a2ddcd67d04766818d0cf4", "sha256": "2a410ad54317b3f4cb570ecf431154fbe04febd0cd17655c5738a2bf5dbe97d0" }, "downloads": -1, "filename": "cleanlab-0.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e50f49f545a2ddcd67d04766818d0cf4", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 48204, "upload_time": "2018-12-08T08:53:30", "url": "https://files.pythonhosted.org/packages/29/e6/54bd3e5793a738102ae2b30ce5b089938fff4533d6142df15d55093c6ae3/cleanlab-0.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "626a00d3c1723ef1ee8f6d955940a3f1", "sha256": "02b9c426e133b8b765d74e4d94a508cdd53e87c6204a7f957d9378d6b9b8268f" }, "downloads": -1, "filename": "cleanlab-0.0.5.tar.gz", "has_sig": false, "md5_digest": "626a00d3c1723ef1ee8f6d955940a3f1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 45584, "upload_time": "2018-12-08T08:53:31", "url": "https://files.pythonhosted.org/packages/0f/d1/f45593dee75a45809b21563283d5968eb9f6f717e1ab1cd19b671f255ec0/cleanlab-0.0.5.tar.gz" } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "178bed746ca0a7f83b0d5506aa8ebea6", "sha256": "5138e993d5ba3dc891ad87bb5957f61fdc27826a3d8e2eb7c0757becdad58849" }, "downloads": -1, "filename": "cleanlab-0.0.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "178bed746ca0a7f83b0d5506aa8ebea6", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 41812, "upload_time": "2018-12-12T06:39:59", "url": "https://files.pythonhosted.org/packages/f2/eb/4e3164613bf570162a09f46b91524ff5933ed0506151f6b3d34233bdcc9c/cleanlab-0.0.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "05f4dfd98014813e08ccfc781b14ff78", "sha256": "647bd605007fb5228dd1b0262f36728834edc2ebef1e84ef8b36fff87b8fe9e7" }, "downloads": -1, "filename": "cleanlab-0.0.6.tar.gz", "has_sig": false, "md5_digest": "05f4dfd98014813e08ccfc781b14ff78", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42915, "upload_time": "2018-12-12T06:40:01", "url": "https://files.pythonhosted.org/packages/c3/9f/0cdd22d2548c84fc5a8e5d83b8240aa4c666c8a0ecfd06ca68a7a2f25d1f/cleanlab-0.0.6.tar.gz" } ], "0.0.7": [ { "comment_text": "", "digests": { "md5": "84c8fac46a9c88766670f16d0d5a35e7", "sha256": "5dcedd6bf076c421672914edfd687a06e89efb17b2e0ff666791893a0a586f63" }, "downloads": -1, "filename": "cleanlab-0.0.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "84c8fac46a9c88766670f16d0d5a35e7", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 44974, "upload_time": "2019-01-06T06:08:01", "url": "https://files.pythonhosted.org/packages/1f/2a/a198ca939d0398e66571957321408c50244392761f473b846297e91fea71/cleanlab-0.0.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9bb0860730914059ca269823c9ddeec1", "sha256": "7138bc13e685e4b9b0218acb207bf7f9cfda8f329ddc6cdc6c158593835324b3" }, "downloads": -1, "filename": "cleanlab-0.0.7.tar.gz", "has_sig": false, "md5_digest": "9bb0860730914059ca269823c9ddeec1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 46166, "upload_time": "2019-01-06T06:08:03", "url": "https://files.pythonhosted.org/packages/2d/4a/6c578dd1118361075f2eba69299c03f6dfc30a2ec67a8cd4a26356a07d84/cleanlab-0.0.7.tar.gz" } ], "0.0.8": [ { "comment_text": "", "digests": { "md5": "38167ebbfe6170b4908b74f1c3100026", "sha256": "443eacf4783b4abcc86e3dc4296df378c5325abc473b1797c2fdb669270bed0c" }, "downloads": -1, "filename": "cleanlab-0.0.8-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "38167ebbfe6170b4908b74f1c3100026", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 46644, "upload_time": "2019-02-18T01:28:43", "url": "https://files.pythonhosted.org/packages/fb/e6/19ee3809f6b252b5f36004df32a43a2db12cd73f20e14adcbd3cace8d5a6/cleanlab-0.0.8-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4a5599c03d622cff628323b6c0f10bd7", "sha256": "9e5bc92df7a47db3f9d4a9402c467c5a8874bcba01f079638de5a34e0597c5c4" }, "downloads": -1, "filename": "cleanlab-0.0.8.tar.gz", "has_sig": false, "md5_digest": "4a5599c03d622cff628323b6c0f10bd7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 47897, "upload_time": "2019-02-18T01:28:45", "url": "https://files.pythonhosted.org/packages/fe/3f/e45e1e72d886180f9d7dedd125de20768e8aa5372a564edcacded79640ab/cleanlab-0.0.8.tar.gz" } ], "0.0.9": [ { "comment_text": "", "digests": { "md5": "84d84e75d7a7f2bead0c21acf30d0991", "sha256": "e4bb6265937d35fed9b737b00e88ffdc1741929ee069cfbeb3839cf84f99ee6d" }, "downloads": -1, "filename": "cleanlab-0.0.9-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "84d84e75d7a7f2bead0c21acf30d0991", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 57190, "upload_time": "2019-07-28T14:45:53", "url": "https://files.pythonhosted.org/packages/b4/74/59d0cf550faac0242591d2362a3fac1e743b218ca3344e22e5ce3d10e7ba/cleanlab-0.0.9-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "53154f39440d4e1a8f62062dbfdac0b7", "sha256": "d2b5db17face78d6dceebea158ca52225c2a18519efca84ad3d4e84a45c74df2" }, "downloads": -1, "filename": "cleanlab-0.0.9.tar.gz", "has_sig": false, "md5_digest": "53154f39440d4e1a8f62062dbfdac0b7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 55917, "upload_time": "2019-07-28T14:45:55", "url": "https://files.pythonhosted.org/packages/63/db/cac94e81d3a78ef37695074912961d4f8691e9b1c1c60984d39c6c5b7e58/cleanlab-0.0.9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "fc27cc6919520e5bae0daaf61c82ff4f", "sha256": "6ce360bef72db2ef435cf2de43c2f68de5326401f8f169b102e4e9fe401b8668" }, "downloads": -1, "filename": "cleanlab-0.0.14-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "fc27cc6919520e5bae0daaf61c82ff4f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 51874, "upload_time": "2019-09-28T13:21:57", "url": "https://files.pythonhosted.org/packages/6f/8b/88d8e5cd8119b07c5cdd9efa1619b0f6a111fe8aacd50a0ce4c6942486b9/cleanlab-0.0.14-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "303e7dbde7d858074f56584293568108", "sha256": "28ea1167c76fa9e44463b4ae5d2e582afe4e20047112aae6a1794bf4e0f00ee8" }, "downloads": -1, "filename": "cleanlab-0.0.14.tar.gz", "has_sig": false, "md5_digest": "303e7dbde7d858074f56584293568108", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 45549, "upload_time": "2019-09-28T13:21:58", "url": "https://files.pythonhosted.org/packages/63/f0/bbeee2ce7484980728096b9e071faa4e278b4b11220cb8c779fbf49aacf5/cleanlab-0.0.14.tar.gz" } ] }