{ "info": { "author": "Fabian Bruckschen", "author_email": "fabian@knuper.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "======\ngnuper\n======\n\n**gnuper** is an open source python package for the preprocessing of mobile phone metadata whilst keeping privacy in mind. It is being developed by the spin-off project `knuper `_ of the Freie Universit\u00e4t Berlin.\nThe package creates features on antenna level off of mobile phone metadata like simple incoming to outgoing ratios of calls or sms as well as entropy and isolation indicators of antennas. Additionally one can include antenna averages of the bandicoot individual indicators developed by the MIT. Further information on this toolkit can be found `here `_.\n\nPrerequisites\n-------------\n- Python3\n- Apache Spark\n\nStructural Overview\n-------------------\nThe following tree diagram shows the entire process from raw mobile phone meta data to final features on antenna level. In order to keep the process fast and the intermediate tables as small as possible, several mid-aggregations are taking place. We call these intermediate steps levels.\nEach level represents another aggregation or transformation step. The color coding signifies the privacy risk. Orange is on user level, blue on antenna (single tower GPS coordinate) level.\n\n.. figure:: docs/Raw_Data_Levels.png\n :alt: Data Levels\n :scale: 60 %\n\n- **Level 0**\nThis level simply unifies and preprocesses (kick out unnecessary information) the raw format. Also obvious machines are being filtered as well as the files restructured into chunks of users.\n\n- **Level 1**\nFor each user a home antenna is being estimated and then interactions aggregated on user level per day and hour. Also bandicoot variables might be calculated if the flag is set.\n\n- **Level 2**\nAntenna level. Users are being allocated to their home antenna and then aggregated in three different categories. First, for each week between weekend or holidays and working days. Second, for every hour of the day irrespective of the date. And third, for all interactions between each pair of antennas.\n\n- **Level 3**\nFeatures are being created for several categories:\n*alltime* (ratios & scaled) - over the whole time period\n*variance* (variances over weeks) - to catch regional variation\n*daily* (outgoing vs. incoming, week parts) - differences between working days and weekends/holidays\n*hourly* (workday, peaks) - differences between the usual working day and several peaks\n*interactions* (distance, isolation, entropy) - network and geolocation related\n*active users* (per home antenna) - number of users allocated to an antenna as their home antenna\nFor the exact formulas, have a look at the specific queries.\n\n- **Level 4**\nUnite the feature categories with the bandicoot features (aggregated on antenna level) to one final dataset, ready for analysis.\n\nSample Data\n-----------\nIn order to create a small synthetic CDR sample data set (<20MB) run the ipython\nscript `cdr_mockup.ipynb `_ e.g. with Jupyter.\n\nSpark\n-----\nThe package has been built using SPARK version 2.2.1.\n\nInstructions on how to install Spark in Ubuntu (tested with 18.04):\n\n1. install java environment ``sudo apt-get install default-jre``\n2. install scala ``sudo apt-get install scala``\n3. download spark\n ``wget http://archive.apache.org/dist/spark/spark-2.2.1/spark-2.2.1-bin-hadoop2.7.tgz``\n4. unzip ``sudo tar -zxvf spark-2.2.1-bin-hadoop2.7.tgz``\n5. and remove ``rm spark-2.2.1-bin-hadoop2.7.tgz``\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://gitlab.com/knuper/gnuper", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "gnuper", "package_url": "https://pypi.org/project/gnuper/", "platform": "", "project_url": "https://pypi.org/project/gnuper/", "project_urls": { "Homepage": "https://gitlab.com/knuper/gnuper" }, "release_url": "https://pypi.org/project/gnuper/0.0.3/", "requires_dist": [ "bandicoot (==0.5.3)", "glob2 (==0.5)", "numpy", "numpydoc", "pandas (==0.23.3)", "py4j (==0.10.7)", "pyarrow", "pyspark (==2.3.1)", "tqdm" ], "requires_python": "", "summary": "Open Source Package for Mobile Phone Metadata Preprocessing", "version": "0.0.3" }, "last_serial": 4628450, "releases": { "0.0.3": [ { "comment_text": "", "digests": { "md5": "f8ebb7a19c4553e219291f1108955571", "sha256": "1f0e68c770cd9e80d38cdafad4513e596b85af0e36d036eb76141710a9841ac7" }, "downloads": -1, "filename": "gnuper-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "f8ebb7a19c4553e219291f1108955571", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 46737, "upload_time": "2018-12-22T22:07:03", "url": "https://files.pythonhosted.org/packages/c4/1c/c7bd32f0ac56b24482a652dd7ef12424293a204e517d10e7bbe799acb7cf/gnuper-0.0.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8f0b803426ec6dc18551e40a43ea1eae", "sha256": "03762142cb120cb0a9a82bf012983a15101db312f5fc76ce7d7fb06b65102ad4" }, "downloads": -1, "filename": "gnuper-0.0.3.tar.gz", "has_sig": false, "md5_digest": "8f0b803426ec6dc18551e40a43ea1eae", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22192, "upload_time": "2018-12-22T22:07:06", "url": "https://files.pythonhosted.org/packages/d9/68/996c067687d25b3f5f09111a7823cebb014477e429055d2ce458ff1a9b9e/gnuper-0.0.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "f8ebb7a19c4553e219291f1108955571", "sha256": "1f0e68c770cd9e80d38cdafad4513e596b85af0e36d036eb76141710a9841ac7" }, "downloads": -1, "filename": "gnuper-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "f8ebb7a19c4553e219291f1108955571", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 46737, "upload_time": "2018-12-22T22:07:03", "url": "https://files.pythonhosted.org/packages/c4/1c/c7bd32f0ac56b24482a652dd7ef12424293a204e517d10e7bbe799acb7cf/gnuper-0.0.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8f0b803426ec6dc18551e40a43ea1eae", "sha256": "03762142cb120cb0a9a82bf012983a15101db312f5fc76ce7d7fb06b65102ad4" }, "downloads": -1, "filename": "gnuper-0.0.3.tar.gz", "has_sig": false, "md5_digest": "8f0b803426ec6dc18551e40a43ea1eae", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22192, "upload_time": "2018-12-22T22:07:06", "url": "https://files.pythonhosted.org/packages/d9/68/996c067687d25b3f5f09111a7823cebb014477e429055d2ce458ff1a9b9e/gnuper-0.0.3.tar.gz" } ] }