{ "info": { "author": "Matt Hagy", "author_email": "matt@liveramp.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "# mockrdd\nA Python3 module for testing PySpark code.\n\nThe MockRDD class offers similar behavior to [pyspark.RDD](http://spark.apache.org/docs/2.1.0/api/python/pyspark.html#pyspark.RDD) with the following\nextra benefits.\n* Extensive sanity checks to identify invalid inputs\n* More meaningful error messages for debugging issues\n* Straightforward to running within pdb\n* Removes Spark dependencies from development and testing environments\n* No Spark overhead when running through a large test suite\n\nSimple example of using MockRDD in a test.\n```python\nfrom mockrdd import MockRDD\n\ndef job(rdd):\n return rdd.map(lambda x: x*2).filter(lambda x: x>3)\n\nassert job(MockRDD.empty()).collect() == [] \nassert job(MockRDD.of(1)).collect() == [] \nassert job(MockRDD.of(2)).collect() == [4] \n```\n\nConventionally, you'd include a main method to hook the RDD up to product sources and sinks.\nFurther, the testing would be included in a separate file and use the module\n[unittest](https://docs.python.org/3/library/unittest.html) for defining test cases.\n\nSee the docstring of [mockrdd.MockRDD](https://github.com/LiveRamp/mockrdd/blob/master/mockrdd/__init__.py#L166) for more information.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/LiveRamp/mockrdd", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "mockrdd", "package_url": "https://pypi.org/project/mockrdd/", "platform": "", "project_url": "https://pypi.org/project/mockrdd/", "project_urls": { "Homepage": "https://github.com/LiveRamp/mockrdd" }, "release_url": "https://pypi.org/project/mockrdd/0.0.2/", "requires_dist": null, "requires_python": "", "summary": "Utility for testing PySpark code", "version": "0.0.2" }, "last_serial": 4353740, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "6b30b34a729a94d38c4643752b91ab16", "sha256": "012d0e2f173ebecdfd3aa3858896d201670fb58086366e31f2abd1f55c3dc9ad" }, "downloads": -1, "filename": "mockrdd-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "6b30b34a729a94d38c4643752b91ab16", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7280, "upload_time": "2018-10-08T22:10:56", "url": "https://files.pythonhosted.org/packages/6a/13/1c0e4f82e0abf3ce6ed83f74b3bb1ce5fa26cfa87b6c7b60fe7b45ffa49e/mockrdd-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1202a3e7da9805d22b9353d2d626f5f5", "sha256": "26a0802fbf299e075d325dfb04b33fb54bc28cddc4101e258b6d8bef5f4072fc" }, "downloads": -1, "filename": "mockrdd-0.0.1.tar.gz", "has_sig": false, "md5_digest": "1202a3e7da9805d22b9353d2d626f5f5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7087, "upload_time": "2018-10-08T22:10:57", "url": "https://files.pythonhosted.org/packages/69/46/971ac95861b6136c6a309bff781668e504a44486c70151d190894c382348/mockrdd-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "2ef5c945192087eb9c18fce2eb5815a9", "sha256": "c32f7c365c6bbd5125747ae5da27a7ae2fecab377b1f803230799640e71b0a04" }, "downloads": -1, "filename": "mockrdd-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "2ef5c945192087eb9c18fce2eb5815a9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7256, "upload_time": "2018-10-08T22:40:07", "url": "https://files.pythonhosted.org/packages/5e/49/cc8809691187660f6562352df11b3b35d30e825059fa4f78e21257533cd2/mockrdd-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "10e13bd2e8e57a0550dedcf3c99af76b", "sha256": "4947716fae504620f6c5b4c7ba0db8ad20230992a063cdacb257d80705da7cb5" }, "downloads": -1, "filename": "mockrdd-0.0.2.tar.gz", "has_sig": false, "md5_digest": "10e13bd2e8e57a0550dedcf3c99af76b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7056, "upload_time": "2018-10-08T22:40:08", "url": "https://files.pythonhosted.org/packages/32/1d/54e0aaea920a213408ab41ec8bf92eae2e155da2c4d848fae542c7f5dc7e/mockrdd-0.0.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "2ef5c945192087eb9c18fce2eb5815a9", "sha256": "c32f7c365c6bbd5125747ae5da27a7ae2fecab377b1f803230799640e71b0a04" }, "downloads": -1, "filename": "mockrdd-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "2ef5c945192087eb9c18fce2eb5815a9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7256, "upload_time": "2018-10-08T22:40:07", "url": "https://files.pythonhosted.org/packages/5e/49/cc8809691187660f6562352df11b3b35d30e825059fa4f78e21257533cd2/mockrdd-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "10e13bd2e8e57a0550dedcf3c99af76b", "sha256": "4947716fae504620f6c5b4c7ba0db8ad20230992a063cdacb257d80705da7cb5" }, "downloads": -1, "filename": "mockrdd-0.0.2.tar.gz", "has_sig": false, "md5_digest": "10e13bd2e8e57a0550dedcf3c99af76b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7056, "upload_time": "2018-10-08T22:40:08", "url": "https://files.pythonhosted.org/packages/32/1d/54e0aaea920a213408ab41ec8bf92eae2e155da2c4d848fae542c7f5dc7e/mockrdd-0.0.2.tar.gz" } ] }