{ "info": { "author": "wolfhong", "author_email": "hongxucai1991@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX :: Linux", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", "Topic :: Software Development :: Libraries", "Topic :: Utilities" ], "description": "[![PyPI](https://img.shields.io/pypi/v/python-pisces.svg)](https://pypi.python.org/pypi/python-pisces)\n\n## Why use pisces?\n\nThis project origins from the needs of image dataset for algorithm training.\n\nImage search engines, such as Google, are quite powerful now. They are enough if we could use them.\nHowever, they didn't provide convenient API for us.\n\nPisces uses [selenuim](https://www.seleniumhq.org/), which can work with chromedriver to download the images we need.\nPisces supports these search engines: google, yahoo, bing, baidu(china) and more mainstream engines in the future.\n\n## Why use pisces?(Chinese)\n\n\u672c\u9879\u76ee\u8d77\u6e90\u4e8e\u7b97\u6cd5\u8bad\u7ec3\u9700\u8981\u56fe\u50cf\u6570\u636e\u96c6\u7684\u9700\u6c42.\n\n\u73b0\u6709\u7684\u56fe\u7247\u641c\u7d22\u5f15\u64ce,\u6bd4\u5982Google,\u76f8\u5f53\u5f3a\u5927\u4e86,\u80fd\u591f\u5229\u7528\u8d77\u6765,\u5df2\u7ecf\u53ef\u4ee5\u6ee1\u8db3\u6211\u4eec\u7684\u9700\u6c42. \u7136\u800c,\u8fd9\u4e9b\u7f51\u7ad9\u5e76\u6ca1\u6709\u63d0\u4f9b\u65b9\u4fbf\u7684API\u8ba9\u6211\u4eec\u5f97\u5230\u56fe\u7247\u94fe\u63a5.\n\nPisces\u4f7f\u7528\u4e86[selenuim](https://www.seleniumhq.org/), \u53ef\u8c03\u7528\u4e3b\u6d41\u7684\u6d4f\u89c8\u5668\u4e0b\u8f7d\u641c\u7d22\u5230\u7684\u56fe\u7247. Pisces\u8fd8\u6574\u5408\u4e86\u5176\u4ed6\u7684\u641c\u7d22\u5f15\u64ce\u7684\u641c\u7d22\u7ed3\u679c:google/yahoo/bing/\u767e\u5ea6.\n\n## Installation\n\nFirst, make sure you have [chrome](https://www.google.com/chrome/) browser installed on your computer.\nIf not, selenium gives \"selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary\".\n\nIf you're working on a server without interface (for example, CentOS), you can also follow the steps below to install it:\n\n wget https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm\n yum -y localinstall google-chrome-stable_current_x86_64.rpm\n\nUse pip:\n\n pip install python-pisces\n\nFrom source code:\n\n git clone https://github.com/wolfhong/python-pisces.git\n cd python-pisces\n\n pip install -r requirements.txt\n export PYTHONPATH=./\n python scripts/update_chromedriver.py # download newest chromedriver according to platform\n python setup.py install\n\nIn addition to chrome, you can also use [firefox](https://www.mozilla.org/). However, you need to do extra settings for firefox. You have to download [geckodriver](https://github.com/mozilla/geckodriver/releases) and put it into your `$PATH`, which helps selenium interacting with firefox browser.\n\nPisces uses chrome as the default browser. You needn't download [chromedriver](http://chromedriver.storage.googleapis.com/index.html) youself because Pisces has built-in ones.\n\nPisces first finds chromedriver (for Chrome) or geckodriver (for Firefox) in `pisces/tools/` in its package. If not, then finds it in `$PATH`.\n\n## Console Command\n\nOnce you have installed Pisces, you can easily use it to search for and download images by keywords:\n\n $ pisces -e google -w 8 -n 500 fire \"kitchen fire\" -o ./output\n\nThe above command will start up chromedriver and then google \"fire\" and \"kitchen fire\" with its image search engine,\ndownload images with 8 threads parallelly and then restore these images in \"./output\" directory.\n\nIf you're in China, you're recommended to use `-e baidu` instand of `-e google`(default), because of some network problems. (\u5728\u4e2d\u56fd\u7531\u4e8e\u4e00\u4e9b\u7f51\u7edc\u539f\u56e0\uff0c\u63a8\u8350\u4f7f\u7528\u53c2\u6570`-e baidu`\uff0c\u800c\u4e0d\u662f\u9ed8\u8ba4\u7684`-e google`)\n\n![image](https://raw.githubusercontent.com/wolfhong/python-pisces/develop/overview.png)\n\nUse `pisces -h` to show the usage:\n\n``` console\n usage: command.py [-h] [-q] [--display] [-e ENGINE] [-w WORKERS] [-n NUMBER]\n [-o OUTPUT_DIR] [-v]\n [keywords [keywords ...]]\n\n Use keywords to search for and download images.\n\n positional arguments:\n keywords keywords to search for images\n\n optional arguments:\n -h, --help show this help message and exit\n -q, --quiet quiet (no output)\n --display start up browser with a graphical display, default to no\n -e ENGINE, --engine ENGINE\n the image search engine you want to use, default to\n google. select within [google, bing, yahoo, baidu]\n -b BROWSER, --browser BROWSER\n the browser you have installed, default to chrome.\n select within [chrome, firefox]\n -w WORKERS, --workers WORKERS\n the number of threads when downloading images, default\n to cpu core count\n -n NUMBER, --number NUMBER\n the max number of images you want to download, default\n to 100\n -o OUTPUT_DIR, --output_dir OUTPUT_DIR\n destination to store downloaded images, default to\n ./output\n -v, --version print the version and exit\n```\n\nPisces can work on these platforms: windows, linux, mac, and it had beed tested.\nIf not, you're welcome to [file an issue](https://github.com/wolfhong/python-pisces/issues).\n\nMore examples:\n\n``` console\n $ pisces \"kitchen fire\" \"forest fire\"\n $ pisces -n500 \"\u53a8\u623f\u706b\u707e\" \"\u68ee\u6797\u706b\u707e\"\n $ pisces -e baidu -n500 -o path-to-output \u706b\u707e \u6c34\u707e\n```\n\n## Code Example\n\nPisces can also be included in your projects:\n\n``` python\n from pisces import Pisces\n\n # recommended to use `with`:\n with Pisces(quiet=False, headless=False) as client:\n output_dir = './output_water/'\n client.download_by_word('water', output_dir, engine='google', image_count=20)\n\n # or call `close()` handly:\n client = Pisces(quiet=False, headless=True, workers=4, browser='chrome')\n output_dir = './output_fire/'\n client.download_by_word('fire', output_dir, engine='baidu', image_count=20)\n client.close()\n```\n\n## Tips\n\n- The code uses selenuim. More info to see [ReadTheDocs](http://selenium-python.readthedocs.io/installation.html)\n- If you're in China, you're recommended to use `-e baidu` instand of `-e google`(default), because of some network problems.\n\n## About\n\nPisces is just a tools to search for and download images by keywords, using image search engine such as google, bing, baidu, etc.\nI hope it could help you in somewhere.\n\n* [Issue tracker](https://github.com/wolfhong/python-pisces/issues?status=new&status=open)\n* [Source Code](https://github.com/wolfhong/python-pisces)\n* [PyPI](https://pypi.python.org/pypi/python-pisces)\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/wolfhong/python-pisces", "keywords": "download-images,search-images,algorithm-training,image-dataset,google,baidu", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "python-pisces", "package_url": "https://pypi.org/project/python-pisces/", "platform": "any", "project_url": "https://pypi.org/project/python-pisces/", "project_urls": { "Homepage": "https://github.com/wolfhong/python-pisces" }, "release_url": "https://pypi.org/project/python-pisces/0.0.5/", "requires_dist": [ "selenium (>=3.8.0)", "requests" ], "requires_python": ">=2.7, <4", "summary": "Use keywords to search for and download images.", "version": "0.0.5" }, "last_serial": 3950332, "releases": { "0.0.4": [ { "comment_text": "", "digests": { "md5": "086182cfbfabcd38eefe1881a0fe4c34", "sha256": "0061d8e3b41f6c50672ccbb9a737b45b322d7c30bfd2ed2fb88d7ce7b550c584" }, "downloads": -1, "filename": "python_pisces-0.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "086182cfbfabcd38eefe1881a0fe4c34", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, <4", "size": 12937792, "upload_time": "2018-06-06T12:55:13", "url": "https://files.pythonhosted.org/packages/c7/78/0011d2ee2a9c3fb1a61b6038bd0477a6a316579493d93641886e35ecbba3/python_pisces-0.0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0828fffe5a4c97135f26f2b892ac4ed4", "sha256": "744b9f634e7b6c26117c11d9793fdba26334aa4310c25e26188ea711457f8de0" }, "downloads": -1, "filename": "python-pisces-0.0.4.tar.gz", "has_sig": false, "md5_digest": "0828fffe5a4c97135f26f2b892ac4ed4", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, <4", "size": 12904149, "upload_time": "2018-06-06T12:56:18", "url": "https://files.pythonhosted.org/packages/56/d0/506965a0be0ad786500d0ef5da6229cce5d093bf444dccbae835d5025e31/python-pisces-0.0.4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "97672f0d7691a643b816dfc74ed6eb25", "sha256": "e329eb1fa8328401dac9a7ee650494c19a4339f6823a18940e0a08f5665c309e" }, "downloads": -1, "filename": "python_pisces-0.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "97672f0d7691a643b816dfc74ed6eb25", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, <4", "size": 12992646, "upload_time": "2018-06-11T13:49:37", "url": "https://files.pythonhosted.org/packages/e2/c5/ea9c959dc623817b01a0ffc15a180494e6a72ab5652b8889eeed26be61b2/python_pisces-0.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "63978f8b2ac8817013872d76d153cc77", "sha256": "5010562c6eb994420b3a59161c9ffea776766a78919776689afff17c81e5b13c" }, "downloads": -1, "filename": "python-pisces-0.0.5.tar.gz", "has_sig": false, "md5_digest": "63978f8b2ac8817013872d76d153cc77", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, <4", "size": 12959746, "upload_time": "2018-06-11T13:50:51", "url": "https://files.pythonhosted.org/packages/51/0d/e0fe6e5ef8dd037a0a899a702257b734da0c3d994c0903aa3bd8be26aba0/python-pisces-0.0.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "97672f0d7691a643b816dfc74ed6eb25", "sha256": "e329eb1fa8328401dac9a7ee650494c19a4339f6823a18940e0a08f5665c309e" }, "downloads": -1, "filename": "python_pisces-0.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "97672f0d7691a643b816dfc74ed6eb25", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, <4", "size": 12992646, "upload_time": "2018-06-11T13:49:37", "url": "https://files.pythonhosted.org/packages/e2/c5/ea9c959dc623817b01a0ffc15a180494e6a72ab5652b8889eeed26be61b2/python_pisces-0.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "63978f8b2ac8817013872d76d153cc77", "sha256": "5010562c6eb994420b3a59161c9ffea776766a78919776689afff17c81e5b13c" }, "downloads": -1, "filename": "python-pisces-0.0.5.tar.gz", "has_sig": false, "md5_digest": "63978f8b2ac8817013872d76d153cc77", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, <4", "size": 12959746, "upload_time": "2018-06-11T13:50:51", "url": "https://files.pythonhosted.org/packages/51/0d/e0fe6e5ef8dd037a0a899a702257b734da0c3d994c0903aa3bd8be26aba0/python-pisces-0.0.5.tar.gz" } ] }