{ "info": { "author": "Almer Mendoza", "author_email": "amendoza@stratpoint.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7" ], "description": "# Gencon Miner\n\nA general content miner that leverages on Beautiful Soup and Requests to handle extraction. The main goal is to always imagine in terms of targetting parent elements in an HTML form then getting group of tags given that parent.\n\n```python\nfrom gencon_miner import GenconMiner\n```\n\n## From URL\n\n```python\nurl_miner = GenconMiner(url=\"http://google.com\")\ntxt = url_miner.extract('title')\nprint(txt[0].text) # Google\n```\n\n## From text\n\n```python\ntext_miner = GenconMiner(text=\"

Hello

\")\ntxt = text_miner.extract('.myclass')\nprint(txt[0].text) # Hello\n```\n\n## Convert all tag content to string\n\nNote that contents in a tag will be delimited using newline.\n\n```python\nmeaning_of_life = \"\"\"\n

\n Hello\n darkness my old friend\n

\n And another one\n\"\"\"\nbulk_miner = GenconMiner(text=meaning_of_life)\nprint(bulk_miner.to_text()) # Hello\\ndarkness my old friend\\nAnd another one\n```\n\n## Parent to target\n\nUse-case on walking document and extracting the targets.\n\n```python\nsong_of_the_day = \"\"\"\n \n \n \n \n \n \n \n
Mamma MiaHere I go againMy myHow can I resist you
\n\"\"\"\nwalk_miner = GenconMiner(text=song_of_the_day)\nprint(walk_miner.extract('#mother', '.target-1')[0].text) # Mamma Mia\nprint(walk_miner.extract('#mother', '.target-3')[0].text) # My my\nprint(walk_miner.extract('#mother', 'td'))\n# [\n# Mamma Mia,\n# Here I go again,\n# My my,\n# How can I resist you\n# ]\n```\n\n## Author\n\nAlmer Mendoza\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/mamerisawesome/GenconMiner", "keywords": "", "license": "MIT", "maintainer": "Almer Mendoza", "maintainer_email": "amendoza@stratpoint.com", "name": "genconminer", "package_url": "https://pypi.org/project/genconminer/", "platform": "", "project_url": "https://pypi.org/project/genconminer/", "project_urls": { "Homepage": "https://github.com/mamerisawesome/GenconMiner", "Repository": "https://github.com/mamerisawesome/GenconMiner" }, "release_url": "https://pypi.org/project/genconminer/0.1.7/", "requires_dist": [ "beautifulsoup4 (>=4.7,<5.0)", "requests (>=2,<3)" ], "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*", "summary": "Wraps Pip packages (beautifulsoup and requests) to a more concise content extractor", "version": "0.1.7" }, "last_serial": 5335341, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "560f55befca43ed9862f735b0503d514", "sha256": "9a381c5d9ddb106363da66f2b9a91d43e5d333194bfcc287c9dbfe183b06f903" }, "downloads": -1, "filename": "GenconMiner-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "560f55befca43ed9862f735b0503d514", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 6116, "upload_time": "2019-05-06T14:14:07", "url": "https://files.pythonhosted.org/packages/2c/16/f2461e128cf97e85c3933ea635a95018e170844875aef72aab9f287f7da7/GenconMiner-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "382a2501c3610fca429ca507ba5ba2db", "sha256": "8f9443eeb3635c0941f36d2d970a7da695d66ebc3308f688a2f0892fceaf0ddb" }, "downloads": -1, "filename": "GenconMiner-0.1.0.tar.gz", "has_sig": false, "md5_digest": "382a2501c3610fca429ca507ba5ba2db", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 3239, "upload_time": "2019-05-06T14:14:09", "url": "https://files.pythonhosted.org/packages/2c/94/64bc6d3f64cb79f212aa97016986a9a5eb4a15d338a08602336cf23898f4/GenconMiner-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "f75c85b31610afb1242315572e96c50a", "sha256": "aca70453f334c969cf1996d41071bf0f50a9de554a49efa4bd2b4d56891a93d1" }, "downloads": -1, "filename": "GenconMiner-0.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "f75c85b31610afb1242315572e96c50a", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 6335, "upload_time": "2019-05-06T14:28:36", "url": "https://files.pythonhosted.org/packages/10/49/fdb6e52caadcbb0b07ccbbe1d953b6812aa92213e03612a8ca89bafff7d0/GenconMiner-0.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "00f6ac06f1d2efd60ba5e9826ce1cfce", "sha256": "ecd7cb522080b1b622c35856c88bd7b2a931e3dd3fa0e6cfbfd8a91ea286824a" }, "downloads": -1, "filename": "GenconMiner-0.1.1.tar.gz", "has_sig": false, "md5_digest": "00f6ac06f1d2efd60ba5e9826ce1cfce", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 3582, "upload_time": "2019-05-06T14:28:38", "url": "https://files.pythonhosted.org/packages/3a/ce/3dfcdeb4b677fcb2e5ea20014386e63bdee117e564953f9382c8fca69772/GenconMiner-0.1.1.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "234e3d04ddc3a48a44b5aba7b05edb40", "sha256": "9bb116a71f0746c625532b583031016eed5089c0e718ee19d80733c35cddaddc" }, "downloads": -1, "filename": "GenconMiner-0.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "234e3d04ddc3a48a44b5aba7b05edb40", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 6477, "upload_time": "2019-05-07T06:04:06", "url": "https://files.pythonhosted.org/packages/90/22/a1ee30b80b783ea5208e1848c1cca6398934c5c32f7ea792b814489fcaa4/GenconMiner-0.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a5f2bc828758454e5eca068ef204fedb", "sha256": "e77dee7f65cad0e03d9067dae128f9fed0a0f9de662f3bd0ee08b8de225df675" }, "downloads": -1, "filename": "GenconMiner-0.1.3.tar.gz", "has_sig": false, "md5_digest": "a5f2bc828758454e5eca068ef204fedb", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 3600, "upload_time": "2019-05-07T06:04:07", "url": "https://files.pythonhosted.org/packages/38/d0/a127ed34776bde80e32931ef5dea0e041e7e457a10c2fe3e1411b51502cd/GenconMiner-0.1.3.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "4d41af145f22828927966100d06367e8", "sha256": "7dfa8315c8e397c8ffe3727149b9518ff2aebc57d3cddd68b0609343b76e611a" }, "downloads": -1, "filename": "GenconMiner-0.1.5-py3-none-any.whl", "has_sig": false, "md5_digest": "4d41af145f22828927966100d06367e8", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 6552, "upload_time": "2019-05-07T10:06:03", "url": "https://files.pythonhosted.org/packages/c9/6d/0cb07b9873f4623919dce278873fe65e8907eaef84b7e6803cfa8ea32d89/GenconMiner-0.1.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1f18a05e50c04e0df38aa9b6e1c6767c", "sha256": "f1daf25aa3c3e5868be99d4b5862d6982bb20120c29840c4e9c33fa27aaf4d76" }, "downloads": -1, "filename": "GenconMiner-0.1.5.tar.gz", "has_sig": false, "md5_digest": "1f18a05e50c04e0df38aa9b6e1c6767c", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 3624, "upload_time": "2019-05-07T10:06:05", "url": "https://files.pythonhosted.org/packages/79/e9/930aebec14062136588d6019bfd994db19b6bd901143dbee0e1e7a0c4b72/GenconMiner-0.1.5.tar.gz" } ], "0.1.6": [ { "comment_text": "", "digests": { "md5": "689e74fad96eae1b60c371f3a33edc13", "sha256": "9c293da3fefd057bf2ae15b59524f1d382e4db8894d1bad422f7e19ff4d71aa2" }, "downloads": -1, "filename": "GenconMiner-0.1.6-py3-none-any.whl", "has_sig": false, "md5_digest": "689e74fad96eae1b60c371f3a33edc13", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 6618, "upload_time": "2019-05-07T10:10:53", "url": "https://files.pythonhosted.org/packages/46/3e/2eca577e4ba93490210ad09b2c6eb7dad363521ae5c7dd8fbc3d8ddf5e34/GenconMiner-0.1.6-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "12cce690d3d589632fb7801becffe0b5", "sha256": "0b126876f9725b8ae02c35bf3e41435cba5fa28faec0671fbf98ac3f3189f6d9" }, "downloads": -1, "filename": "GenconMiner-0.1.6.tar.gz", "has_sig": false, "md5_digest": "12cce690d3d589632fb7801becffe0b5", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 3623, "upload_time": "2019-05-07T10:10:54", "url": "https://files.pythonhosted.org/packages/c5/39/989ba79b2672dbe51afe490cd593a9b33c08f118dee5e0545d51f6cae42f/GenconMiner-0.1.6.tar.gz" } ], "0.1.7": [ { "comment_text": "", "digests": { "md5": "f31c26bd3b12f40d05c9145846b71ccf", "sha256": "2c0f6677acb51ae9906415437a47ff7c689815263531c79c1c86c102d5b47c0a" }, "downloads": -1, "filename": "GenconMiner-0.1.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f31c26bd3b12f40d05c9145846b71ccf", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*", "size": 4209, "upload_time": "2019-05-30T02:49:22", "url": "https://files.pythonhosted.org/packages/6e/64/aab7dd924a06d4c5f3f6d7da7884afcdce0df60e672bfc08d3f720f9e7b0/GenconMiner-0.1.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f63019474cb78f75f95ad66837b0bed5", "sha256": "e1576f32f6af443ca444c3cc8ed2c94a1072908ac98bb944d873b07c997cb220" }, "downloads": -1, "filename": "GenconMiner-0.1.7.tar.gz", "has_sig": false, "md5_digest": "f63019474cb78f75f95ad66837b0bed5", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*", "size": 3676, "upload_time": "2019-05-30T02:49:23", "url": "https://files.pythonhosted.org/packages/95/29/474d36ebe4e47fa9fd4bd125b72090ae431bd54e2a114a5634d821df39b3/GenconMiner-0.1.7.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "f31c26bd3b12f40d05c9145846b71ccf", "sha256": "2c0f6677acb51ae9906415437a47ff7c689815263531c79c1c86c102d5b47c0a" }, "downloads": -1, "filename": "GenconMiner-0.1.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f31c26bd3b12f40d05c9145846b71ccf", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*", "size": 4209, "upload_time": "2019-05-30T02:49:22", "url": "https://files.pythonhosted.org/packages/6e/64/aab7dd924a06d4c5f3f6d7da7884afcdce0df60e672bfc08d3f720f9e7b0/GenconMiner-0.1.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f63019474cb78f75f95ad66837b0bed5", "sha256": "e1576f32f6af443ca444c3cc8ed2c94a1072908ac98bb944d873b07c997cb220" }, "downloads": -1, "filename": "GenconMiner-0.1.7.tar.gz", "has_sig": false, "md5_digest": "f63019474cb78f75f95ad66837b0bed5", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*", "size": 3676, "upload_time": "2019-05-30T02:49:23", "url": "https://files.pythonhosted.org/packages/95/29/474d36ebe4e47fa9fd4bd125b72090ae431bd54e2a114a5634d821df39b3/GenconMiner-0.1.7.tar.gz" } ] }