{
    "info": {
        "author": "Daniel Kaslovsky",
        "author_email": "dkaslovsky@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Environment :: Console",
            "License :: OSI Approved :: MIT License",
            "Natural Language :: English",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7"
        ],
        "description": "# Coupled-Biased-Random-Walks\nOutlier detection for categorical data\n\n[![Build Status](https://travis-ci.org/dkaslovsky/Coupled-Biased-Random-Walks.svg?branch=master)](https://travis-ci.org/dkaslovsky/Coupled-Biased-Random-Walks)\n[![Coverage Status](https://coveralls.io/repos/github/dkaslovsky/Coupled-Biased-Random-Walks/badge.svg?branch=master)](https://coveralls.io/github/dkaslovsky/Coupled-Biased-Random-Walks?branch=master)\n\n### Overview\nPython [2.7, 3.4, 3.5, 3.6, 3.7] implementation of the Coupled Biased Random Walks (CBRW) outlier detection algorithm described by Pang, Cao, and Chen in https://www.ijcai.org/Proceedings/16/Papers/272.pdf.\n\nThis implementation operates on Python dicts rather than Pandas DataFrames.  This has the advantage of allowing the model to be updated with new observations in a trivial manner and is more efficient in certain aspects.  However, these advantages come at the cost of iterating a (potentially large) dict of observed values more times than might otherwise be necessary using an underlying DataFrame implementation.\n\nIf one is working with data previously loaded into a DataFrame, simply use the result of `pandas.DataFrame.to_dict(orient='records')` instead of the DataFrame itself to add observations to the model.  Note that because it is common for a DataFrame to fill missing values with `nan`, the detector will ignore features with value `nan` in any observation record.  Therefore, there is no need to further preprocess the DataFrame before using its `to_dict` method to create records.\n\n### Installation\nThis package is hosted on PyPI and can be installed via `pip`:\n```\npip install coupled_biased_random_walks\n```\nTo instead install from source:\n```\n$ git clone git@github.com:dkaslovsky/Coupled-Biased-Random-Walks.git\n$ cd Coupled-Biased-Random-Walks\n$ python setup.py install\n```\n\n### Example\nLet's run the CBRW detection algorithm on the authors' example data set from the paper:\n\n<img src=\"./example_table.png\" width=\"400\">\n\nThis data is saved as a [.CSV file](./data/CBRW_paper_example.csv) in this repository and is loaded into memory as a list of dicts by [example.py](./example.py).  Note that we drop the `Cheat?` column when loading the data, as this is essentially the target variable indicating the anomalous activity to be detected.  The detector is instantiated and observations are added as follows:\n```\n>>> detector = CBRW()\n>>> detector.add_observations(observations)\n```\nwhere `observations` is an iterable of dicts such as the one loaded from the example .CSV file.  Once all of the observations are loaded, the detector can be finalized for scoring by calling `fit()` and observations can then be scored.\n```\n>>> detector.fit()\n>>> scores = detector.score(observations)\n```\nEven after fitting and scoring, more observations can be added via `add_observations` and the detector can again be fit to be used for scoring.  The advantage of this implementation is this ability to incrementally update with new observations.\n\nThe results of scoring the example data are shown below.  Note that the only row where fraud was present (`Cheat? = yes`) received the largest anomaly score.\n```\nScore: 0.1055 | Data: {'Gender': 'male', 'Education': 'master', 'Marriage': 'divorced', 'Income': 'low'}\nScore: 0.0797 | Data: {'Gender': 'female', 'Education': 'master', 'Marriage': 'married', 'Income': 'medium'}\nScore: 0.0741 | Data: {'Gender': 'male', 'Education': 'master', 'Marriage': 'single', 'Income': 'high'}\nScore: 0.0805 | Data: {'Gender': 'male', 'Education': 'bachelor', 'Marriage': 'married', 'Income': 'medium'}\nScore: 0.0992 | Data: {'Gender': 'female', 'Education': 'master', 'Marriage': 'divorced', 'Income': 'high'}\nScore: 0.0752 | Data: {'Gender': 'male', 'Education': 'PhD', 'Marriage': 'married', 'Income': 'high'}\nScore: 0.0741 | Data: {'Gender': 'male', 'Education': 'master', 'Marriage': 'single', 'Income': 'high'}\nScore: 0.0815 | Data: {'Gender': 'female', 'Education': 'PhD', 'Marriage': 'single', 'Income': 'medium'}\nScore: 0.0728 | Data: {'Gender': 'male', 'Education': 'PhD', 'Marriage': 'married', 'Income': 'medium'}\nScore: 0.0979 | Data: {'Gender': 'male', 'Education': 'bachelor', 'Marriage': 'single', 'Income': 'low'}\nScore: 0.0812 | Data: {'Gender': 'female', 'Education': 'PhD', 'Marriage': 'married', 'Income': 'medium'}\nScore: 0.0887 | Data: {'Gender': 'male', 'Education': 'master', 'Marriage': 'single', 'Income': 'low'}\n```\n\nThe entire example can be reproduced by running:\n```\n$ python example.py\n```\n\nThe CBRW algorithm can also be used to calculate feature weights.  These weights are calculated when the detector is fit and are used during scoring, but can also be used by any other outlier detection algorithm.  Thus, the CBRW algorithm can be used simply to calculate feature weights and need not score observations.  Feature weights are stored as a property of the detector after the detector's `fit` method has been called:\n```\n>>> detector = CBRW()\n>>> detector.add_observations(observations)\n>>> detector.fit()\n>>> detector.feature_weights\n```\nFor the example data, the computed feature weights are\n```\n{'Education': 0.26272841835358907,\n 'Gender': 0.16078750024987953,\n 'Income': 0.2938981973816106,\n 'Marriage': 0.2825858840149206}\n```\n\n### Implementation Notes\n- For efficiency, the detector state is only (re)computed upon calling `.fit()`.  Therefore adding new observations (`.add_observations()`) will not affect scoring until `.fit()` is called.  Refitting overwrites previous state but includes contribution from all added observations.\n- The `.add_observations()` and `.fit()` methods can be chained together if one-line training is desired: `detector.add_observations(observations).fit()`.\n- An observation containing a feature name or feature value that has not been previously fit will be scored as `nan`.  To instead ignore any such \"new\" features and score an observation based on known features only, initialize the detector with `ignore_unknown=True`.\n\n### Tests\nTo run unit tests:\n```\n$ python -m unittest discover -v\n```\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/dkaslovsky/Coupled-Biased-Random-Walks",
        "keywords": "anomaly detection, outlier detection,categorical data,random walk",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "coupled-biased-random-walks",
        "package_url": "https://pypi.org/project/coupled-biased-random-walks/",
        "platform": "",
        "project_url": "https://pypi.org/project/coupled-biased-random-walks/",
        "project_urls": {
            "Homepage": "https://github.com/dkaslovsky/Coupled-Biased-Random-Walks"
        },
        "release_url": "https://pypi.org/project/coupled-biased-random-walks/1.0.0/",
        "requires_dist": [
            "numpy (==1.14.5)",
            "scipy (==1.1.0)",
            "six (==1.11.0)"
        ],
        "requires_python": "",
        "summary": "Outlier detection for categorical data",
        "version": "1.0.0"
    },
    "last_serial": 4379776,
    "releases": {
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f20f4585a46d1b173ca4fb5fcce8ed35",
                    "sha256": "753dd16394ac5edbe737de75b99717b621021d541f7178253f3817938dc6aa7a"
                },
                "downloads": -1,
                "filename": "coupled_biased_random_walks-1.0.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f20f4585a46d1b173ca4fb5fcce8ed35",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 11721,
                "upload_time": "2018-10-16T02:54:04",
                "url": "https://files.pythonhosted.org/packages/2b/a4/b6539da6f65446f356d87221f7c2d50abe60d751b4171be1b161a0ea4497/coupled_biased_random_walks-1.0.0-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "925b050630b509f3a8858054827968f7",
                    "sha256": "2476d23050c57b3dc850809512f65d220ca3dab0edb09ca60762e7cfb8af8f31"
                },
                "downloads": -1,
                "filename": "coupled_biased_random_walks-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "925b050630b509f3a8858054827968f7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8208,
                "upload_time": "2018-10-16T02:54:05",
                "url": "https://files.pythonhosted.org/packages/02/5c/b65192bc1a987f54cfff794528ad258ce7161ce8698d306b6df99b2cf559/coupled_biased_random_walks-1.0.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "f20f4585a46d1b173ca4fb5fcce8ed35",
                "sha256": "753dd16394ac5edbe737de75b99717b621021d541f7178253f3817938dc6aa7a"
            },
            "downloads": -1,
            "filename": "coupled_biased_random_walks-1.0.0-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "f20f4585a46d1b173ca4fb5fcce8ed35",
            "packagetype": "bdist_wheel",
            "python_version": "py2.py3",
            "requires_python": null,
            "size": 11721,
            "upload_time": "2018-10-16T02:54:04",
            "url": "https://files.pythonhosted.org/packages/2b/a4/b6539da6f65446f356d87221f7c2d50abe60d751b4171be1b161a0ea4497/coupled_biased_random_walks-1.0.0-py2.py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "925b050630b509f3a8858054827968f7",
                "sha256": "2476d23050c57b3dc850809512f65d220ca3dab0edb09ca60762e7cfb8af8f31"
            },
            "downloads": -1,
            "filename": "coupled_biased_random_walks-1.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "925b050630b509f3a8858054827968f7",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 8208,
            "upload_time": "2018-10-16T02:54:05",
            "url": "https://files.pythonhosted.org/packages/02/5c/b65192bc1a987f54cfff794528ad258ce7161ce8698d306b6df99b2cf559/coupled_biased_random_walks-1.0.0.tar.gz"
        }
    ]
}