{ "info": { "author": "AstroLab Software", "author_email": "peloton@lal.in2p3.fr", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Intended Audience :: Developers", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6" ], "description": "[![Build Status](https://travis-ci.org/astrolabsoftware/spark3D.svg?branch=master)](https://travis-ci.org/astrolabsoftware/spark3D)\n[![codecov](https://codecov.io/gh/astrolabsoftware/spark3D/branch/master/graph/badge.svg)](https://codecov.io/gh/astrolabsoftware/spark3D)\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.github.astrolabsoftware/spark3d_2.11/badge.svg?style=flat)](https://maven-badges.herokuapp.com/maven-central/com.github.astrolabsoftware/spark3d_2.11)\n\n**The package is under an active development!**\n\n## Latest News\n\n- [05/2018] **GSoC 2018**: spark3D has been selected to the Google Summer of Code (GSoC) 2018. Congratulation to [@mayurdb](https://github.com/mayurdb) who will work on the project this year!\n- [06/2018] **Release**: version 0.1.0, 0.1.1\n- [07/2018] **New location**: spark3D is an official project of [AstroLab Software](https://astrolabsoftware.github.io/)!\n- [07/2018] **Release**: version 0.1.3, 0.1.4, 0.1.5\n- [08/2018] **Release**: version 0.2.0, 0.2.1 (pyspark3d)\n- [09/2018] **Release**: version 0.2.2\n- [11/2018] **Release**: version 0.3.0, 0.3.1 (new DataFrame API)\n\n

\n

\n\n## Rationale\n\nspark3D should be viewed as an extension of the Apache Spark framework, and more specifically the Spark SQL module, focusing on the manipulation of three*-dimensional data sets.\n\nWhy would you use spark3D? If you often need to repartition large spatial 3D data sets, or perform spatial queries (neighbour search, window queries, cross-match, clustering, ...), spark3D is for you. It contains optimised classes and methods to do so, and it spares you the implementation time! In addition, a big advantage of all those extensions is to efficiently perform visualisation of large data sets by quickly building a representation of your data set (see more [here](https://astrolabsoftware.github.io/spark3D/)).\n\nspark3D exposes two API: Scala (spark3D) and Python (pyspark3d). The core developments are done in Scala, and interfaced with Python using the great [py4j](https://www.py4j.org/) package. This means pyspark3d might not contain all the features present in spark3D.\nIn addition, due to difference between Scala and Python, there might be subtle differences in the two APIs.\n\nWhile we try to stick to the latest Apache Spark developments, spark3D started with the RDD API and slowly migrated to use the DataFrame API. This process left a huge imprint on the code structure, and low-level layers in spark3D often still use RDD to manipulate the data. Do not be surprised if things are moving, the package is under an active development but we try to keep the user interface as stable as possible!\n\nLast but not least: spark3D is by no means complete, and you are welcome to suggest changes, report bugs or inconsistent implementations, and contribute directly to the package!\n\nCheers,\nJulien\n\n*Why 3? Because there are already plenty of very good packages dealing with 2D data sets (e.g. [geospark](http://geospark.datasyslab.org/), [geomesa](https://www.geomesa.org/), [magellan](https://magellan.ghost.io/), [GeoTrellis](https://github.com/locationtech/geotrellis), and others), but that was not suitable for many applications such as in astronomy!*\n\n## Installation and tutorials\n\n### Scala\n\nYou can link spark3D to your project (either spark-shell or spark-submit) by specifying the coordinates:\n\n```\nspark-submit --packages \"com.github.astrolabsoftware:spark3d_2.11:0.3.0\"\n```\n\n### Python\n\nJust run\n\n```bash\npip install pyspark3d\n```\n\nNote that we release the assembly JAR with it.\n\n### More information\n\nSee our [website](https://astrolabsoftware.github.io/spark3D/)!\n\n## Contributors\n\n* Julien Peloton (peloton at lal.in2p3.fr)\n* Christian Arnault (arnault at lal.in2p3.fr)\n* Mayur Bhosale (mayurdb31 at gmail.com) -- GSoC 2018.\n\nContributing to spark3D: see [CONTRIBUTING](https://github.com/astrolabsoftware/spark3D/blob/master/CONTRIBUTING.md).\n\n## Support\n\n

\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/astrolabsoftware/spark3d", "keywords": "spark,spark3d,scala,python,py4j", "license": "Apache License, Version 2.0", "maintainer": "", "maintainer_email": "", "name": "pyspark3d", "package_url": "https://pypi.org/project/pyspark3d/", "platform": "", "project_url": "https://pypi.org/project/pyspark3d/", "project_urls": { "Homepage": "https://github.com/astrolabsoftware/spark3d" }, "release_url": "https://pypi.org/project/pyspark3d/0.3.1/", "requires_dist": [ "numpy (>=1.14)", "pyspark", "scipy" ], "requires_python": "", "summary": "Spark extension for processing large-scale 3D data sets", "version": "0.3.1" }, "last_serial": 4555687, "releases": { "0.2.1": [ { "comment_text": "", "digests": { "md5": "a95381e784675c9db47b9d28faf94e28", "sha256": "5d5ac3ca94cfd155a5bcef11077d5bf9fa17f0e3accd66dbc647e8fa0088abeb" }, "downloads": -1, "filename": "pyspark3d-0.2.1-py3-none-any.whl", "has_sig": false, "md5_digest": "a95381e784675c9db47b9d28faf94e28", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 411842, "upload_time": "2018-09-20T13:04:26", "url": "https://files.pythonhosted.org/packages/e0/bb/f8c81627776fc1ece783323caf6b9f23d8056bafabeb0ef5f1cc30781026/pyspark3d-0.2.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4f58cc687bf777292110b82c46291fc1", "sha256": "5f7951930f14b5934aad9e3775df17f5fd93c120cc013f85762b5fa11afaeeeb" }, "downloads": -1, "filename": "pyspark3d-0.2.1.tar.gz", "has_sig": false, "md5_digest": "4f58cc687bf777292110b82c46291fc1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 407384, "upload_time": "2018-09-20T13:04:28", "url": "https://files.pythonhosted.org/packages/8b/e8/071adab26ae67feed11220b92eb09ed8bdd18d2ea0055cfa87ceef8e6a16/pyspark3d-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "793270e5e14d0be74d99ad5343b6c309", "sha256": "4000abe8db246a2105da021982ba7b988ca25451e41e3af3848a1ff595c05444" }, "downloads": -1, "filename": "pyspark3d-0.2.2-py3.7.egg", "has_sig": false, "md5_digest": "793270e5e14d0be74d99ad5343b6c309", "packagetype": "bdist_egg", "python_version": "3.7", "requires_python": null, "size": 824511, "upload_time": "2018-11-23T15:13:19", "url": "https://files.pythonhosted.org/packages/2f/80/da7d0dd18e3f0bf30d1ec5e9e2ab237822cca72f77e968a739e4a57da7ab/pyspark3d-0.2.2-py3.7.egg" }, { "comment_text": "", "digests": { "md5": "b71ed5ccfbb5fe3b63d558b296263b1e", "sha256": "b13f6b79060c0ccbd4efded9fa1d6c493b5de228b83bd54a4013d9057c451a8d" }, "downloads": -1, "filename": "pyspark3d-0.2.2-py3-none-any.whl", "has_sig": false, "md5_digest": "b71ed5ccfbb5fe3b63d558b296263b1e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 804244, "upload_time": "2018-09-21T10:04:12", "url": "https://files.pythonhosted.org/packages/dd/80/3e489ec68dd798575e062e26249fc4f77203a7a093d25dbbb7d83c1bf885/pyspark3d-0.2.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a58b6116bfa4ade229738f18f7310d1d", "sha256": "c4daa45a6fb8447fbaffc61357f4286b1dc276ce8b6d63cf0efeeb487c9b963d" }, "downloads": -1, "filename": "pyspark3d-0.2.2.tar.gz", "has_sig": false, "md5_digest": "a58b6116bfa4ade229738f18f7310d1d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 409451, "upload_time": "2018-09-21T10:04:13", "url": "https://files.pythonhosted.org/packages/e8/fa/9538d9581fae4391bc485c9f08b08e32d7d7b84a852cab4bba524c01c406/pyspark3d-0.2.2.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "7490b6e0c7051a5af7a32d60fe9a8ef1", "sha256": "bb8fb6a91bdb5a8de250e111aadcaaf86a46eeb0e78dbb841bc878fd5e0dd83c" }, "downloads": -1, "filename": "pyspark3d-0.3.0-py3-none-any.whl", "has_sig": false, "md5_digest": "7490b6e0c7051a5af7a32d60fe9a8ef1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 810846, "upload_time": "2018-11-23T15:13:17", "url": "https://files.pythonhosted.org/packages/07/da/09e9de1a9579b11cadb7d1be2ee2911be4404deca88131bb3d8616001907/pyspark3d-0.3.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8742325781621387c16504ad703e349c", "sha256": "dd74a598d217368650bed9b47b5d778f73e3977741263f6a8134c28115e310b0" }, "downloads": -1, "filename": "pyspark3d-0.3.0.tar.gz", "has_sig": false, "md5_digest": "8742325781621387c16504ad703e349c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15019, "upload_time": "2018-11-23T15:13:21", "url": "https://files.pythonhosted.org/packages/22/8a/8ca06add6c482f68f904ccba1b69b1fba3ed1b0dab54fa3c538c4e14ca47/pyspark3d-0.3.0.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "e78af121c2965b59183c454769393c56", "sha256": "9e970dc31ed09715595675ea43c194807039bfd864b238a928fbd89495a9f011" }, "downloads": -1, "filename": "pyspark3d-0.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "e78af121c2965b59183c454769393c56", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 28141, "upload_time": "2018-12-03T13:10:39", "url": "https://files.pythonhosted.org/packages/87/9d/33da571a507ee99a3f2518f7f5bb96a1b10ed2abe5d2db8969ca84b42637/pyspark3d-0.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "d0d44532f92b784302658e2c1b071717", "sha256": "e5e1bd6a99451348a9116f3d93ca65269532795112706588586ceb0f2a08094a" }, "downloads": -1, "filename": "pyspark3d-0.3.1.tar.gz", "has_sig": false, "md5_digest": "d0d44532f92b784302658e2c1b071717", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16533, "upload_time": "2018-12-03T13:10:41", "url": "https://files.pythonhosted.org/packages/79/54/4c3c0ade75aedf54d981e012b70e70b325cc799cf50df2b04758ea32542b/pyspark3d-0.3.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "e78af121c2965b59183c454769393c56", "sha256": "9e970dc31ed09715595675ea43c194807039bfd864b238a928fbd89495a9f011" }, "downloads": -1, "filename": "pyspark3d-0.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "e78af121c2965b59183c454769393c56", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 28141, "upload_time": "2018-12-03T13:10:39", "url": "https://files.pythonhosted.org/packages/87/9d/33da571a507ee99a3f2518f7f5bb96a1b10ed2abe5d2db8969ca84b42637/pyspark3d-0.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "d0d44532f92b784302658e2c1b071717", "sha256": "e5e1bd6a99451348a9116f3d93ca65269532795112706588586ceb0f2a08094a" }, "downloads": -1, "filename": "pyspark3d-0.3.1.tar.gz", "has_sig": false, "md5_digest": "d0d44532f92b784302658e2c1b071717", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16533, "upload_time": "2018-12-03T13:10:41", "url": "https://files.pythonhosted.org/packages/79/54/4c3c0ade75aedf54d981e012b70e70b325cc799cf50df2b04758ea32542b/pyspark3d-0.3.1.tar.gz" } ] }