{
    "info": {
        "author": "Alexandru Mara",
        "author_email": "alexandru.mara@ugent.be",
        "bugtrack_url": null,
        "classifiers": [
            "Intended Audience :: Science/Research",
            "License :: OSI Approved :: MIT License",
            "Natural Language :: English",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 3",
            "Topic :: Scientific/Engineering :: Information Analysis"
        ],
        "description": "# EvalNE: A Python library for evaluating Network Embedding methods on Link Prediction #\n\n[![Documentation Status](https://readthedocs.org/projects/evalne/badge/?version=latest)](https://evalne.readthedocs.io/en/latest/?badge=latest)\n[![contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://github.com/Dru-Mara/EvalNE/issues)\n[![MIT license](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/Dru-Mara/EvalNE/blob/master/LICENSE)\n[![made-with-python](https://img.shields.io/badge/Made%20with-Python-1f425f.svg)](https://www.python.org/)\n[![made-with-sphinx-doc](https://img.shields.io/badge/Made%20with-Sphinx-1f425f.svg)](https://www.sphinx-doc.org/)\n\nThis repository provides the source code for EvalNE, an open-source Python\nlibrary designed for assessing and comparing the performance of Network\nEmbedding (NE) methods on Link Prediction (LP) tasks. The library intends to\nsimplify this complex and time consuming evaluation process by providing\nautomation and abstraction of tasks such as hyper-parameter tuning, selection of\ntrain and test edges, negative sampling, selection of the scoring function, etc.\n\nThe library can be used both as a command line tool and an API. In its current \nversion, EvalNE can evaluate unweighted directed and undirected simple networks.\n\nThe library is maintained by Alexandru Mara (alexandru.mara(at)ugent.be). The full\ndocumentation of EvalNE is hosted by *Read the Docs* and can be found \n[here](https://evalne.readthedocs.io/en/latest/).\n\n#### For Methodologists ####\nA command line interface in combination with a configuration file allows the user\nto evaluate any publicly available implementation of a NE method without the need\nto write additional code. These implementations can be obtained from libraries \nsuch as \n[OpenNE](https://github.com/thunlp/OpenNE) or\n[GEM](https://github.com/palash1992/GEM) \nas well as directly from the web pages of the authors e.g. \n[Deepwalk](https://github.com/phanein/deepwalk),\n[Node2vec](https://github.com/aditya-grover/node2vec),\n[LINE](https://github.com/tangjianpku/LINE),\n[PRUNE](https://github.com/ntumslab/PRUNE),\n[Metapath2vec](https://ericdongyx.github.io/metapath2vec/m2v.html),\n[CNE](https://bitbucket.org/ghentdatascience/cne/).\n\nEvalNE also includes the following LP heuristics for both directed and\nundirected networks (in and out node neighbourhoods), which can be used as\nbaselines:\n\n* Random Prediction\n* Common Neighbours\n* Jaccard Coefficient\n* Adamic Adar Index\n* Preferential Attachment\n* Resource Allocation Index\n\n#### For practitioners ####\nWhen used as an API, EvalNE provides functions to:\n\n* Load and preprocess graphs\n* Obtain general graph statistics\n* Compute train/test/validation splits\n* Generate false edges\n* Evaluate link prediction from: \n    * Node Embeddings\n    * Edge Embeddings\n    * Similarity scores (e.g. the ones given by LP heuristics)\n* Provides functions that compute edge embeddings from node feature vectors\n    * Average\n    * Hadamard\n    * Weighted L1\n    * Weighted L2\n* Any sklearn binary classifier can be used as a LP algorithm\n* Implements several accuracy metrics.\n* Includes parameter tuning subroutines\n\n\n## Instalation ##\n\nThe library has been tested on Python 2.7 and Python 3.6.\n\nEvalNE depends on the following packages:\n* Numpy\n* Scipy\n* Sklearn\n* Matplotlib\n* Networkx 2.2\n\nBefore installing EvalNE make sure that `pip` and `python-tk` packages are installed \non your system, this can be done \nby running:\n```bash\n# Python 2\nsudo apt-get install python-pip\nsudo apt-get install python-tk\n\n# Python 3\nsudo apt-get install python3-pip\nsudo apt-get install python3-tk\n```\n\n**Option 1:** Install the library using pip:\n```bash\n# Python 2\npip install evalne\n\n# Python 3\npip3 install evalne\n```\n\n**Option 2:** Cloning the code and installing:\n\n- Clone the EvalNE repository:\n    ```bash\n    git clone https://github.com/Dru-Mara/EvalNE.git\n    cd EvalNE\n    ```\n\n- Download strict library dependencies and install:\n    ```bash\n    # Python 2\n    pip install -r requirements.txt\n    sudo python setup.py install\n\n    # Python 3\n    pip3 install -r requirements.txt\n    sudo python3 setup.py install\n    ```\n\nCheck the installation by running `simple_example.py` or `functions_example.py` e.g.:\n```bash\n# Python 2\ncd examples/\npython simple_example.py\n\n# Python 3\ncd examples/\npython3 simple_example.py\n```\n\n**NOTE:** In order to run the `evaluator_example.py` script, the \nOpenNE library, PRUNE and Metapath2Vec are required. The instructions for installing \nthem are available \n[here](https://github.com/thunlp/OpenNE), [here](https://github.com/ntumslab/PRUNE), \nand [here](https://www.dropbox.com/s/w3wmo2ru9kpk39n/code_metapath2vec.zip?dl=0), \nrespectively. The instructions on how to run evaluations using *.ini* files are \nprovided in the next section. \n\n\n## Usage ##\n\n### As a command line tool ###\n\nThe library takes as input an *.ini* configuration file. This file allows the user \nto specify the evaluation settings, from the methods and baselines to be evaluated\nto the edge embedding methods, parameters to tune or scores to report.\n\nAn example `conf.ini` file is provided describing the available options\nfor each parameter. This file can be either modified to simulate different\nevaluation settings or used as a template to generate other *.ini* files.\n\nAdditional configuration (*.ini*) files are provided replicating the experimental \nsections of different papers in the NE literature. These can be found in different\nfolders under `examples/`. One such configuration file is \n`examples/node2vec/conf_node2vec.ini`. This file simulates the link prediction \nexperiments of the paper \"Scalable Feature Learning for Networks\" by A. Grover \nand J. Leskovec.\n\nOnce the configuration is set, the evaluation can be run as indicated in the next\nsubsection.\n\n#### Running the conf examples ####\n\nIn order to run the evaluations using the provided `conf.ini` or any other *.ini*\nfile, the following steps are necessary: \n\n1. Download/Install the methods you want to test:\n    * For `conf.ini`:\n        * Install [OpenNE](https://github.com/thunlp/OpenNE) \n        * Install [PRUNE](https://github.com/ntumslab/PRUNE)\n    * For other *.ini* files you may need:\n        *   [Deepwalk](https://github.com/phanein/deepwalk),\n            [Node2vec](https://github.com/aditya-grover/node2vec),\n            [LINE](https://github.com/tangjianpku/LINE),\n            [Metapath2vec](https://ericdongyx.github.io/metapath2vec/m2v.html), and/or\n            [CNE](https://bitbucket.org/ghentdatascience/cne/).\n\n2. Download the datasets used in the examples:\n   * For `conf.ini`:\n      * [StudentDB](http://adrem.ua.ac.be/smurfig)\n      * [ArXiv GR-QC](https://snap.stanford.edu/data/ca-GrQc.html)\n   * For other *.ini* files you may need:\n      * [Facebook](https://snap.stanford.edu/data/egonets-Facebook.html) (combined network)\n      * [Facebook-wallpost](http://socialnetworks.mpi-sws.org/data-wosn2009.html)\n      * [ArXiv Astro-Ph](http://snap.stanford.edu/data/ca-AstroPh.html)\n      * [ArXiv Hep-Ph](https://snap.stanford.edu/data/cit-HepPh.html)\n      * [BlogCatalog](http://socialcomputing.asu.edu/datasets/BlogCatalog3)\n      * [Wikipedia](http://snap.stanford.edu/node2vec)\n      * [PPI](http://snap.stanford.edu/node2vec/Homo_sapiens.mat)\n\n\n\n3. Set the correct dataset paths in the INPATHS option of the corresponding *.ini* \nfile. And the correct method paths under METHODS_OPNE and/or METHODS_OTHER options. \n\n4. Run the evaluation:\n    ```bash\n    # For conf.ini run:\n    python evalne ./examples/conf.ini\n\n    # For conf_node2vec.ini run:\n    python evalne ./examples/node2vec/conf_node2vec.ini\n    ```\n\n**Note**: The input networks for EvalNE are required to be in edgelist format.\n\n### As an API ###\n\nThe library can be imported and used like any other Python module. Next we\npresent a very basic example, for more complete ones we refer the user to the\n`examples/` folder.\n\n```python\nfrom evalne.evaluation import evaluator\nfrom evalne.preprocessing import preprocess as pp\n\n# Load and preprocess the network\nG = pp.load_graph('../evalne/tests/data/network.edgelist')\nG, _ = pp.prep_graph(G)\n\n# Create an evaluator and generate train/test edge split\nnee = evaluator.Evaluator()\n_ = nee.traintest_split.compute_splits(G)\n\n# Set the baselines\nmethods = ['random_prediction', 'common_neighbours', 'jaccard_coefficient']\n\n# Evaluate baselines\nnee.evaluate_baseline(methods=methods)\n\ntry:\n    # Check if OpenNE is installed\n    import openne\n\n    # Set embedding methods from OpenNE\n    methods = ['node2vec', 'deepwalk', 'GraRep']\n    commands = [\n        'python -m openne --method node2vec --graph-format edgelist --p 1 --q 1',\n        'python -m openne --method deepWalk --graph-format edgelist --number-walks 40',\n        'python -m openne --method grarep --graph-format edgelist --epochs 10']\n    edge_emb = ['average', 'hadamard']\n\n    # Evaluate embedding methods\n    for i in range(len(methods)):\n        command = commands[i] + \" --input {} --output {} --representation-size {}\"\n        nee.evaluate_cmd(method_name=methods[i], method_type='ne', command=command,\n                         edge_embedding_methods=edge_emb, input_delim=' ', output_delim=' ')\n\nexcept ImportError:\n    print(\"The OpenNE library is not installed. Reporting results only for the baselines...\")\n    pass\n\n# Get output\nresults = nee.get_results()\nfor result in results:\n    result.pretty_print()\n\n``` \n\n### Output ###\n\nThe library can provide two types of outputs, depending on the value of the SCORES option\nof the configuration file. If the keyword *all* is specified, the library will generate a \nfile named `eval_output.txt` containing for each method and network analysed all the \nmetrics available (auroc, precision, f-score, etc.). If more than one experiment repeat \nis requested the values reported will be the average over all the repeats. The output \nfile will be located in the same path from which the evaluation was run.\n\nSetting the SCORES option to `%(maximize)` will generate a similar output file as before.\nThe content of this file, however, will be a table (Alg. x Networks) containing exclusively \nthe score specified in the MAXIMIZE option for each combination of method and network\naveraged over all experiment repeats. \n\nAdditionally, if the option TRAINTEST_PATH contains a valid filename, EvalNE will create\na file with that name under each of the OUTPATHS provided. In each of these paths the\nlibrary will store the true and false train and test sets of edge. \n\n**NOTE**: The tabular output is not available for mixes of directed and undirected networks.\n\n\n## Citation ##\n\nIf you have found EvaNE useful in your research, please cite our \n[arXiv paper](https://arxiv.org/abs/1901.09691):\n\n```\n    @misc{Mara2019,\n      author = {Alexandru Mara and Jefrey Lijffijt and Tijl De Bie},\n      title = {EvalNE: A Framework for Evaluating Network Embeddings on Link Prediction},\n      year = {2019},\n      archivePrefix = {arXiv},\n      eprint = {1901.09691}\n    }\n```\n\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/Dru-Mara/EvalNE",
        "keywords": "evaluation embedding link-prediction networks graphs",
        "license": "MIT License",
        "maintainer": "",
        "maintainer_email": "",
        "name": "evalne",
        "package_url": "https://pypi.org/project/evalne/",
        "platform": "",
        "project_url": "https://pypi.org/project/evalne/",
        "project_urls": {
            "Homepage": "https://github.com/Dru-Mara/EvalNE"
        },
        "release_url": "https://pypi.org/project/evalne/0.2.2/",
        "requires_dist": [
            "numpy (==1.15.1)",
            "scikit-learn (>=0.19.0)",
            "networkx (>=2.2)",
            "scipy",
            "matplotlib"
        ],
        "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, <4",
        "summary": "Open Source Network Embedding Evaluation toolkit",
        "version": "0.2.2"
    },
    "last_serial": 4938815,
    "releases": {
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e787ea266aaabe388ec9ca74c4647656",
                    "sha256": "6be0e4126d98869f787cb626c57300618d19cc64e6516f686286d73ce8b4cdf3"
                },
                "downloads": -1,
                "filename": "evalne-0.2.2-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "e787ea266aaabe388ec9ca74c4647656",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, <4",
                "size": 43687,
                "upload_time": "2019-03-14T10:13:05",
                "url": "https://files.pythonhosted.org/packages/ec/c0/e8d543f156d067cb893f3e57067304aa44dc26bf32e27ceb39a8667be8df/evalne-0.2.2-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "4f126e4a4992e402b04ac43682458946",
                    "sha256": "26e14679bcaa1f01b72a0d31bdc99467b8433c691c87ee8d4d4c15ae416d52f8"
                },
                "downloads": -1,
                "filename": "evalne-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "4f126e4a4992e402b04ac43682458946",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, <4",
                "size": 36382,
                "upload_time": "2019-03-14T10:13:06",
                "url": "https://files.pythonhosted.org/packages/30/6e/5c88a22a56984579e14e8d824f0636dad0e45841d38a4dd336ba379e14e1/evalne-0.2.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "e787ea266aaabe388ec9ca74c4647656",
                "sha256": "6be0e4126d98869f787cb626c57300618d19cc64e6516f686286d73ce8b4cdf3"
            },
            "downloads": -1,
            "filename": "evalne-0.2.2-py2-none-any.whl",
            "has_sig": false,
            "md5_digest": "e787ea266aaabe388ec9ca74c4647656",
            "packagetype": "bdist_wheel",
            "python_version": "py2",
            "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, <4",
            "size": 43687,
            "upload_time": "2019-03-14T10:13:05",
            "url": "https://files.pythonhosted.org/packages/ec/c0/e8d543f156d067cb893f3e57067304aa44dc26bf32e27ceb39a8667be8df/evalne-0.2.2-py2-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "4f126e4a4992e402b04ac43682458946",
                "sha256": "26e14679bcaa1f01b72a0d31bdc99467b8433c691c87ee8d4d4c15ae416d52f8"
            },
            "downloads": -1,
            "filename": "evalne-0.2.2.tar.gz",
            "has_sig": false,
            "md5_digest": "4f126e4a4992e402b04ac43682458946",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, <4",
            "size": 36382,
            "upload_time": "2019-03-14T10:13:06",
            "url": "https://files.pythonhosted.org/packages/30/6e/5c88a22a56984579e14e8d824f0636dad0e45841d38a4dd336ba379e14e1/evalne-0.2.2.tar.gz"
        }
    ]
}