{
    "info": {
        "author": "Ian Ozsvald",
        "author_email": "ian@ianozsvald.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Environment :: Console",
            "Intended Audience :: Developers",
            "Intended Audience :: Science/Research",
            "License :: Free To Use But Restricted",
            "Natural Language :: English",
            "Operating System :: OS Independent",
            "Programming Language :: Python",
            "Topic :: Software Development :: Libraries :: Python Modules",
            "Topic :: Software Development :: Testing"
        ],
        "description": "# discover_feature_relationships\n\nAttempt to discover 1D relationships between all columns in a DataFrame using scikit-learn (RandomForests) and standard correlation tests (Pearson, Spearman and Kendall via Pandas). \n\nThe goal is to see if we can better understand the data in a DataFrame by learning which features (1 column at a time) predict each other column. This code attempts to learn a predictive relationship between the Cartesian product (all pairs) of all columns.\n\nRather than just learning which column(s) predict a target column, we might want to know what other relationships exist (e.g. during Exploratory Data Analysis) and whether some predictive features are driven by other less-predictive features (to help us find new & better features or data sources). We might also sense-check out data by checking that certain relationships exist.\n\nBy default it assumes every target column is a regression challenge. You can provide a list of columns to treat as classification challenges. For regression we cap negative scores at 0 (r^2 can be arbitrarily negative, we cap at 0 to make this a little easier to interpret). \n\nText-encoded columns are automatically LabelEncoded (this is a sensible default but may not reveal information in your case, you might need to provide your own smarter encoding). This adds to the correlation plots in YellowBrick and Pandas Profiling where the text columns are not auto-encoded.\n\nWe might want to use this tool alongside:\n\n* https://github.com/ResidentMario/missingno\n* https://github.com/pandas-profiling/pandas-profiling\n\nThe project (and the examples) live on GitHub:\n\n* https://github.com/ianozsvald/discover_feature_relationships\n\n## Titanic example\n\n[Titanic Notebook](./examples/example_titanic_discover_feature_relationships.ipynb)\n\n* Embarked (classification) is predicted well by Fare, also by Age\n* Pclass (regression) is predicted by Fare but Fare (regression) is poorly predicted by Pclass\n* Sex (classification) is predicted well by Survived\n* Survived (classification) is predicted well by Sex, Fare, Pclass, SibSpParch\n  * Predicting this feature at circa 0.62 is equivalent to \"no information\" as 0.62 is the mean of Survived\n* SibSpParch is predicted by both SibSp and Parch (SibSpParch is the sum of both - it is an engineered additional feature) - it is also predicted by Fare\n* SibSp and Parch are also predicted by Fare (but less well so than by SibSpParch)\n\n![alt text](examples/example_titanic_output.png)\n\nThis is generated using:\n```\ndf = pd.read_csv(\"titanic_train.csv\")\n...\n\nimport discover\ndf_results = discover.discover(cols, classifier_overrides, df)\n\ndf_results.pivot(index='target', columns='feature', values='score').fillna(1) \\\n.style.background_gradient(cmap=\"viridis\", low=0.3, high=0.0, axis=1) \\\n.set_precision(2)\n```\n\n## Boston example\n\n[Boston Notebook](./examples/example_boston_discover_feature_relationships.ipynb)\n\n* NOX predicts AGE and DIS (but not the other way around)\n* target predicts LSTAT, LSTAT weakly predicts target, LSTAT weakly predicts RM\n* DIS predicts AGE, AGE weakly predicts DIS\n* INDUS predicts CRIM and somewhat AGE, B\n* target weakly predicts RM, RM weakly predicts target\n\n# Requirements\n\n* python 3.6+\n* scikit-learn (0.19+)\n* pandas\n* jupyter notebook\n* matplotlib\n* seaborn\n\n```\nconda install scikit-learn pandas jupyter pytest seaborn\nconda install -c conda-forge watermark\n```\n\n# Setup\n\n## Install from PyPI\n\n`pip install discover_feature_relationships`\n\nhttps://pypi.org/project/discover-feature-relationships/\n\n## Install from source\n\nFirst check-out from GitHub, then install with ```python setup.py install```, then `cd` into the `examples` folder and run the Notebooks.\n\n\n# Tests\n\n* Run `discover.py` for a simple test that the code is working \n* Run `pytest` to run `test_discover.py` for a single unit test (use `pytest -s` to see `print` outputs)\n\n# Note to Ian for Development\n\nEnvironment: `. ~/anaconda3/bin/activate discover_feature_relationships`\n\n## Installer\n\nTo push to PyPI I need to follow https://docs.python.org/3/distributing/index.html#distributing-index - specifically `python se\ntup.py sdist bdist_wheel` and `twine upload dist/*`. This uses https://pypi.org/project/twine/ .\n\n## TODO\n\n* consider version info: https://github.com/pallets/flask/blob/master/setup.py\n* consider python_requires line in setup\n* consider adding dependencies\n* add dependencies to setup\n* clean up the list of links\n* bump version nbr\n* add conda installer\n\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/ianozsvald/discover_feature_relationships",
        "keywords": "",
        "license": "MIT",
        "maintainer": "Ian Ozsvald",
        "maintainer_email": "ian@ianozsvald.com",
        "name": "discover-feature-relationships",
        "package_url": "https://pypi.org/project/discover-feature-relationships/",
        "platform": "Any.",
        "project_url": "https://pypi.org/project/discover-feature-relationships/",
        "project_urls": {
            "Homepage": "https://github.com/ianozsvald/discover_feature_relationships"
        },
        "release_url": "https://pypi.org/project/discover-feature-relationships/1.0.3/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "discover_feature_relationships tries to find column-to-column relationships in your dataframe using machine learning",
        "version": "1.0.3"
    },
    "last_serial": 4682493,
    "releases": {
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e9eec9b37aa948be3e117002424d70d6",
                    "sha256": "20a0ae279c7ccffd99532d02473f4947037fe1fa0422873091e631d37229e200"
                },
                "downloads": -1,
                "filename": "discover_feature_relationships-1.0.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "e9eec9b37aa948be3e117002424d70d6",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 4145,
                "upload_time": "2019-01-10T19:22:35",
                "url": "https://files.pythonhosted.org/packages/68/43/418368c62720de45df7445bfe8fd4dde784e68bd1c2c7fec17a6f78c366f/discover_feature_relationships-1.0.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "a987fcd8819b4d8cb7012395c4e721ea",
                    "sha256": "acd230012906b2382cb0bf475c201345e0c36c6d0d957bb5452977946cefd9e8"
                },
                "downloads": -1,
                "filename": "discover_feature_relationships-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "a987fcd8819b4d8cb7012395c4e721ea",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4384,
                "upload_time": "2019-01-10T19:22:37",
                "url": "https://files.pythonhosted.org/packages/0e/42/f880bd42916ff0cbad3dbe75521943c40b1a0195ddeee80e3b1b0008ab2b/discover_feature_relationships-1.0.0.tar.gz"
            }
        ],
        "1.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fcef36a9fb897c582a29a857e1b6e9f4",
                    "sha256": "d7531478d2960c2f4d5eb3290e554b98b264710b434500854413ad5578e7fe28"
                },
                "downloads": -1,
                "filename": "discover_feature_relationships-1.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "fcef36a9fb897c582a29a857e1b6e9f4",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 5994,
                "upload_time": "2019-01-10T19:46:19",
                "url": "https://files.pythonhosted.org/packages/89/48/619f33382b95640f138c57d23f210a2c1dd45cfa3a2bb59530e2cc59571a/discover_feature_relationships-1.0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3283e35d18e8165e1f8aba58b5f1ce6d",
                    "sha256": "b01b609b2fcbfdd5b85809eb6ac9182ed2625b0b285cca42a581988ac20f2e67"
                },
                "downloads": -1,
                "filename": "discover_feature_relationships-1.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "3283e35d18e8165e1f8aba58b5f1ce6d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5038,
                "upload_time": "2019-01-10T19:46:21",
                "url": "https://files.pythonhosted.org/packages/18/2f/f6c7d7b2c29dd314dfe76e31765b92c955d3d00321ca75bec9ac0ac28d97/discover_feature_relationships-1.0.1.tar.gz"
            }
        ],
        "1.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5ba6e7f329096c9416e8edce8e18abbc",
                    "sha256": "921a868709b3e1b36aecfa28111b65a2cabfe0250a002ccdad6c14d7e03dc5ca"
                },
                "downloads": -1,
                "filename": "discover_feature_relationships-1.0.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "5ba6e7f329096c9416e8edce8e18abbc",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 6023,
                "upload_time": "2019-01-10T19:50:39",
                "url": "https://files.pythonhosted.org/packages/af/dc/7ce190ececac1b90b5963b0effcaa96ea3c29645e0e6e06903d6a12dc92d/discover_feature_relationships-1.0.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8bef4b59c6292248488ef6609a73dbc4",
                    "sha256": "0a7413cf814b9d14e4e420d416773201779777b78a08baf129e5afea37289710"
                },
                "downloads": -1,
                "filename": "discover_feature_relationships-1.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "8bef4b59c6292248488ef6609a73dbc4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5065,
                "upload_time": "2019-01-10T19:50:41",
                "url": "https://files.pythonhosted.org/packages/13/36/037a29970ff3d541db6c4701ea719b19a79804ae2bb31473ba76d6f1f369/discover_feature_relationships-1.0.2.tar.gz"
            }
        ],
        "1.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b05cd5b33cd7e316c1aa92d9e0d228b1",
                    "sha256": "e50d92e27fc5cb09bb08db73667edb670e9bba6354e5e4fc5662d84cf08780a1"
                },
                "downloads": -1,
                "filename": "discover_feature_relationships-1.0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b05cd5b33cd7e316c1aa92d9e0d228b1",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 6022,
                "upload_time": "2019-01-10T20:01:47",
                "url": "https://files.pythonhosted.org/packages/8b/fe/22019933e2950e8d6224850afdc243ad770dcdf0a92bf4b525581ef84dc1/discover_feature_relationships-1.0.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "ff7cbd216b17b62b950f021a29cfa57e",
                    "sha256": "faa12280b93c87cdfe69c4b1b1524ebcd701282e21f0feca5467a9cc31cdd4ea"
                },
                "downloads": -1,
                "filename": "discover_feature_relationships-1.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "ff7cbd216b17b62b950f021a29cfa57e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5071,
                "upload_time": "2019-01-10T20:01:50",
                "url": "https://files.pythonhosted.org/packages/c9/a3/8087f6856b837f219e6291b7188a01c0af584e18749016ad22a3872f9170/discover_feature_relationships-1.0.3.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "b05cd5b33cd7e316c1aa92d9e0d228b1",
                "sha256": "e50d92e27fc5cb09bb08db73667edb670e9bba6354e5e4fc5662d84cf08780a1"
            },
            "downloads": -1,
            "filename": "discover_feature_relationships-1.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "b05cd5b33cd7e316c1aa92d9e0d228b1",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 6022,
            "upload_time": "2019-01-10T20:01:47",
            "url": "https://files.pythonhosted.org/packages/8b/fe/22019933e2950e8d6224850afdc243ad770dcdf0a92bf4b525581ef84dc1/discover_feature_relationships-1.0.3-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "ff7cbd216b17b62b950f021a29cfa57e",
                "sha256": "faa12280b93c87cdfe69c4b1b1524ebcd701282e21f0feca5467a9cc31cdd4ea"
            },
            "downloads": -1,
            "filename": "discover_feature_relationships-1.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "ff7cbd216b17b62b950f021a29cfa57e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 5071,
            "upload_time": "2019-01-10T20:01:50",
            "url": "https://files.pythonhosted.org/packages/c9/a3/8087f6856b837f219e6291b7188a01c0af584e18749016ad22a3872f9170/discover_feature_relationships-1.0.3.tar.gz"
        }
    ]
}