{
    "info": {
        "author": "ken",
        "author_email": "kenbliky@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3"
        ],
        "description": "# kcrawler\n\n[![Build Status](https://travis-ci.org/kenblikylee/kcrawler.svg?branch=master)](https://travis-ci.org/kenblikylee/kcrawler)\n[![license](https://img.shields.io/github/license/kenblikylee/kcrawler)](https://github.com/kenblikylee/kcrawler/blob/master/LICENSE)\n\nA python crawler authored by Ken.\n\n\n## 1. \u5b89\u88c5\n\n### 1.1 \u73af\u5883\u8981\u6c42\n\n- python>=3.0\n- pip>=19.0\n\n``` sh\npython -V\npip install --upgrade pip\npip -V\n```\n\n### 1.2 \u67e5\u770b\u6700\u65b0\u7248\u672c\n\n``` sh\npip search kcrawler\n```\n\n### 1.3 \u521d\u6b21\u5b89\u88c5\n\n``` sh\npip install kcrawler\n# or\npip install --index-url https://pypi.org/simple kcrawler\n```\n\n### 1.4 \u66f4\u65b0\u5df2\u6709\u5b89\u88c5\n\n``` sh\npip install --upgrade kcrawler\n# or\npip install --upgrade --index-url https://pypi.org/simple kcrawler\n```\n\n### 1.5 \u5378\u8f7d\n\n``` sh\npip uninstall -y kcrawler\n```\n\n## 2. \u547d\u4ee4\u884c\u8c03\u7528\n\n### 2.1 \u4f7f\u7528\u65b9\u5f0f\n\n\u4f7f\u7528 pip \u5b89\u88c5\u6210\u529f\u540e\uff0c\u4f1a\u81ea\u52a8\u5728\u7cfb\u7edf\u641c\u7d22\u8def\u5f84\u521b\u5efa\u53ef\u6267\u884c\u7a0b\u5e8f\uff1a`kcrawler`, `kcanjuke`, `kcjuejin`\u3002\n\n> \u901a\u5e38\u662f `python` \u6216 `conda` \u5b89\u88c5\u76ee\u5f55\u4e0b\u7684 `bin` \u5b50\u76ee\u5f55\u4e0b\uff0c\u4f8b\u5982\uff1a`/anaconda3/bin/kcrawler`\u3002windows \u5e73\u53f0\u4f1a\u521b\u5efa `.exe` \u6587\u4ef6\u3002\n\n`kcrawler` \u662f\u722c\u53d6\u6240\u6709\u7f51\u7ad9\u5e94\u7528\u7684\u5165\u53e3\uff0c\u547d\u4ee4\u6267\u884c\u683c\u5f0f\u5982\u4e0b\uff1a\n\n``` sh\nkcrawler <webapp> [webapp-data] [--options]\n```\n\n\u7b49\u6548\u4e8e\uff1a\n\n``` sh\nkc<webapp> [webapp-data] [--options]\n```\n\n\u4f8b\u5982\uff1a\n\n```\nkcrawler juejin books --url \"https://...\"\nkcjuejin books --url \"https://...\"\n```\n\n### 2.2 \u4f7f\u7528\u793a\u4f8b\n\n\u4ee5 `kcrawler <webapp> [webapp-data] [--options]` \u65b9\u5f0f\u8fd0\u884c\u4e3a\u4f8b\u3002\n\n#### 2.2.1 \u722c\u53d6\u6398\u91d1\u5c0f\u518c\u6570\u636e\n\n\u6267\u884c\u5982\u4e0b\u547d\u4ee4\uff1a\n\n``` sh\nkcrawler juejin book\n```\n\n\u547d\u4ee4\u6267\u884c\u6210\u529f\uff0c\u663e\u793a\u5982\u4e0b\u7edf\u8ba1\u56fe\u8868\uff1a\n\n![](http://cdn.kenblog.top/juejin_books_927.png)\n\n\u5e76\u5c06\u660e\u7ec6\u6570\u636e\u4fdd\u5b58\u5728\u5f53\u524d\u76ee\u5f55\u4e0b\uff0c\u540c\u65f6\u4fdd\u5b58 `.csv` \u548c `.xls` \u6587\u4ef6\uff0c\u6587\u4ef6\u540d\u683c\u5f0f\u5982\u4e0b\uff1a\n\n`juejin_books_YYYY-MM-DD.csv` `juejin_books_YYYY-MM-DD.xls`\n\n#### 2.2.2 \u722c\u53d6\u6398\u91d1\u4e13\u680f\u9605\u8bfb\u91cf\n\n\u683c\u5f0f\uff1a\n\n``` sh\nkcrawler juejin post --name <username> --limit 100 --url '<user_post_url>'\n```\n\n- name: \u76ee\u6807\u722c\u53d6\u7528\u6237\u7684\u540d\u79f0\uff0c\u53ef\u4ee5\u81ea\u5b9a\u4e49\uff0c\u4ec5\u4ec5\u7528\u4e8e\u533a\u5206\u4e0d\u540c\u7528\u6237\uff0c\u540c\u65f6\u4f5c\u4e3a\u722c\u53d6\u6570\u636e\u4fdd\u5b58\u7684\u6587\u4ef6\u5939\u540d\u79f0\n- limit: \u9650\u5236\u722c\u53d6\u6700\u65b0\u4e13\u680f\u6570\n- url: \u76ee\u6807\u722c\u53d6\u7528\u6237\u7684\u63a5\u53e3\u5730\u5740\uff0c\u8fd9\u4e2a\u53c2\u6570\u771f\u6b63\u51b3\u5b9a\u4e86\u8981\u722c\u53d6\u8c01\u7684\u4e13\u680f\n\nurl \u83b7\u53d6\u65b9\u5f0f\u5982\u4e0b\uff1a\n\n![](http://cdn.kenblog.top/juejin_post_url.png)\n\n\u4e3a\u4e86\u5feb\u901f\u4f53\u9a8c\u722c\u53d6\u6548\u679c\uff0c\u4e5f\u63d0\u4f9b\u4e86 url \u7f3a\u7701\u60c5\u51b5\u4e0b\u7684\u652f\u6301\uff0c\u722c\u53d6\u7528\u6237 [ken](https://juejin.im/user/5bd2b8b25188252a784d19d7/posts) \u7684\u4e13\u680f\uff1a\n\n``` sh\nkcrawler juejin post --name ken --limit 100\n```\n\n\u722c\u53d6\u660e\u7ec6\u6570\u636e\uff0c\u4f1a\u5728 `ken` \u76ee\u5f55\u4e0b\uff0c\u4ee5\u722c\u53d6\u65e5\u671f\u548c\u65f6\u95f4\u547d\u540d\uff0c\u540c\u65f6\u4fdd\u5b58 `.csv` \u6587\u4ef6\u548c `.xls` \u6587\u4ef6\u3002\n\n#### 2.2.3 \u6307\u5b9a\u57ce\u5e02\u722c\u53d6\u5b89\u5c45\u5ba2\u5c0f\u533a\u623f\u4ef7\n\n\u9996\u5148\u9700\u8981\u83b7\u53d6[\u7f51\u7ad9](https://shenzhen.anjuke.com/community/p50/)\u7684 `cookie` \u3002\u83b7\u53d6\u65b9\u5f0f\u53c2\u8003[\u300apython \u81ea\u52a8\u6293\u53d6\u5206\u6790\u623f\u4ef7\u6570\u636e\u2014\u2014\u5b89\u5c45\u5ba2\u7248 \u300b2.4 \u5c0f\u8282](https://juejin.im/post/5d7f021bf265da03cf7abed2#heading-9)\u3002\n\n\n\u5c06 `<anjuke_cookie>` \u66ff\u6362\u6210\u81ea\u5df1 `cookie`\uff0c\u8fd0\u884c\u5982\u4e0b\u547d\u4ee4\uff1a\n\n``` sh\nkcrawler anjuke --city shenzhen --limit 50 --cookie \"<anjuke_cookie>\"\n```\n\n\u4e5f\u53ef\u4ee5\u5c06 `cookie` \u4fdd\u5b58\u5728\u5f53\u524d\u76ee\u5f55\u4e0b\u7684 `anjuke_cookie` (\u65e0\u540e\u7f00)\u6587\u4ef6\u4e2d\uff0c\u8fd0\u884c\u5982\u4e0b\u547d\u4ee4\uff1a\n\n``` sh\nkcrawler anjuke --city shenzhen --limit 50\n```\n\n![](http://cdn.kenblog.top/kcrawler_anjuke_shenzhen.gif)\n\n![](http://cdn.kenblog.top/sz_com_927.png)\n\n\u547d\u4ee4\u6210\u529f\u8fd0\u884c\u6210\u529f\u540e\uff0c\u4f1a\u663e\u793a\u623f\u4ef7\u5e73\u5747\u503c\uff0c\u6700\u5927\u503c\uff0c\u6700\u5c0f\u503c\uff0c\u5e76\u7ed8\u5236\u623f\u4ef7\u5206\u5e03\u76f4\u65b9\u56fe\uff0c\u5173\u95ed\u76f4\u65b9\u56fe\u540e\uff0c\u660e\u7ec6\u6570\u636e\u5c06\u4fdd\u5b58\u5728\u5f53\u524d\u76ee\u5f55\u4e0b\uff0c\u5f62\u5982\uff1a`anjuke_shenzhen_community_price_20xx-xx-xx.csv`\u3002\n\n> \u83b7\u53d6\u5176\u4ed6\u57ce\u5e02\u7684\u623f\u4ef7\uff0c\u53ea\u9700\u5c06 `city` \u53c2\u6570\u6539\u6210\u5b89\u5c45\u5ba2\u7f51\u7ad9\u8986\u76d6\u7684\u57ce\u5e02\u62fc\u97f3\u3002\u53ef\u6253\u5f00\u9875\u9762 [https://www.anjuke.com/sy-city.html](https://www.anjuke.com/sy-city.html) \uff0c\u70b9\u51fb\u9700\u8981\u83b7\u53d6\u7684\u57ce\u5e02\uff0c\u590d\u5236\u6d4f\u89c8\u5668\u5730\u5740\u680f\u4e2d\u57ce\u5e02\u5bf9\u5e94\u7684\u4e8c\u7ea7\u57df\u540d\uff0c\u5982 beijing.anjuke.com \u53ea\u53d6 beijing \u4f5c\u4e3a city \u53c2\u6570\u3002\n\n## 3. \u5bfc\u5165 python \u6a21\u5757\n\n### 3.1 Boss \u63a5\u53e3\n\n``` python\nfrom kcrawler import Boss\nboss  = Boss()\n\nboss_positions = boss.position()\nboss_cities = boss.city()\nboss_hotcities = boss.hotcity()\nboss_industries = boss.industry()\nboss_user_city = boss.userCity()\nboss_expects = boss.expect()\n\njobs = boss.job(0, 1)\ntencent_jobs = boss.queryjob(query='\u817e\u8baf', city=101280600, industry=None, position=101301)\ntencent_jobs = boss.queryjobpage(query='\u817e\u8baf', city=101280600, industry=None, position=101301, page=2)\n\njobcard = boss.jobcard('3c2016bbf8413f3b1XR63t-1FVI~', '505ee74b-504b-4aea-921c-a3dc2016be80.f1:common-155-GroupA--157-GroupA.15')\n```\n\n## Release history\n\n[https://pypi.org/project/kcrawler/#history](https://pypi.org/project/kcrawler/#history)\n\n## License\n\n[MIT](http://opensource.org/licenses/MIT)\n\nCopyright (c) 2019 kenblikylee\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/kenblikylee/kcrawler",
        "keywords": "",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "kcrawler",
        "package_url": "https://pypi.org/project/kcrawler/",
        "platform": "",
        "project_url": "https://pypi.org/project/kcrawler/",
        "project_urls": {
            "Homepage": "https://github.com/kenblikylee/kcrawler"
        },
        "release_url": "https://pypi.org/project/kcrawler/1.0.1/",
        "requires_dist": [
            "requests (>=2.21.0)",
            "beautifulsoup4 (>=4.7.1)",
            "pandas (>=0.25.1)",
            "matplotlib (>=3.0.3)",
            "xlwt (>=1.3.0)",
            "Pillow (>=6.1.0)"
        ],
        "requires_python": ">=3.6",
        "summary": "A python crawler authored by Ken.",
        "version": "1.0.1"
    },
    "last_serial": 5988579,
    "releases": {
        "0.0.18": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3e11881a20152bf633f65424e3bb73fb",
                    "sha256": "900d5fadb25950cc701b5807c63c2b5b8a91abda7e693726e44a29162c559348"
                },
                "downloads": -1,
                "filename": "kcrawler-0.0.18-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "3e11881a20152bf633f65424e3bb73fb",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6",
                "size": 12083,
                "upload_time": "2019-09-26T12:28:27",
                "url": "https://files.pythonhosted.org/packages/dc/ab/209002fd165c2f9efe12ea4e0ee041e6fd29fb6a062f132066106e1743c9/kcrawler-0.0.18-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "c1ea57a6a9e05f39e43eb6a7160cd8c5",
                    "sha256": "7f8278f0d9f9fe56cbf95457af72673f3fdec0aed01c29b3fcf30940ab10ce4f"
                },
                "downloads": -1,
                "filename": "kcrawler-0.0.18.tar.gz",
                "has_sig": false,
                "md5_digest": "c1ea57a6a9e05f39e43eb6a7160cd8c5",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6",
                "size": 7907,
                "upload_time": "2019-09-26T12:28:30",
                "url": "https://files.pythonhosted.org/packages/b3/94/5a06fb71a66e66cb889c6982837a3cf0d86bc19447c0ae56bc3dbd599109/kcrawler-0.0.18.tar.gz"
            }
        ],
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "00ae73fa33a8d1218a5f36b67641714c",
                    "sha256": "51bba9a7fc32f71e43532ef96356fb19602c27bc314f372607687298b0d8e0e9"
                },
                "downloads": -1,
                "filename": "kcrawler-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "00ae73fa33a8d1218a5f36b67641714c",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6",
                "size": 12414,
                "upload_time": "2019-09-27T02:18:39",
                "url": "https://files.pythonhosted.org/packages/3a/78/c1660d892caee48798400c86484f67d7a3553702f1fe09e4ece18009ef83/kcrawler-0.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "0fcc6243b4d6df7d1eed2af781062cba",
                    "sha256": "3377c5e4506bf7d6ef3e8aca60ed6ba4abffcf447f3f90de498ee613c8d4909e"
                },
                "downloads": -1,
                "filename": "kcrawler-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "0fcc6243b4d6df7d1eed2af781062cba",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6",
                "size": 8208,
                "upload_time": "2019-09-27T02:18:41",
                "url": "https://files.pythonhosted.org/packages/48/be/88b8f364a2e8dd8a8d90c3103bf1802046d0030d9b9d1ecc1d6a17637803/kcrawler-0.1.0.tar.gz"
            }
        ],
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "794f7cc7bc345e6a8a63fdee94fda798",
                    "sha256": "7d2aa13caf61cd168d8bb05007a055add22783740cb24cc21287b4173d42f05a"
                },
                "downloads": -1,
                "filename": "kcrawler-1.0.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "794f7cc7bc345e6a8a63fdee94fda798",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6",
                "size": 17389,
                "upload_time": "2019-10-15T16:46:37",
                "url": "https://files.pythonhosted.org/packages/8a/7f/ef2b98c6b951e729715903a95383b9cd4db3bc95ca079e1d8928c5e8d8ef/kcrawler-1.0.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "1a25603f71d037e3efe16fe8ec84ecc3",
                    "sha256": "638a3f62d5ca74158bfcfb0a6b3ee4da9c6f685ebe092f775eeb99bda8ec2169"
                },
                "downloads": -1,
                "filename": "kcrawler-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "1a25603f71d037e3efe16fe8ec84ecc3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6",
                "size": 12899,
                "upload_time": "2019-10-15T16:46:39",
                "url": "https://files.pythonhosted.org/packages/4b/c1/a8484dd734c5e11ad56279dd5421c5f7746a7ee6fce71178d52b2d79731e/kcrawler-1.0.0.tar.gz"
            }
        ],
        "1.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "28035da418ee5654a567b45118dcd9f0",
                    "sha256": "ecce519c57617d7d627f774b98cc2d551fc4860709f314c20188dcc552f1cc2d"
                },
                "downloads": -1,
                "filename": "kcrawler-1.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "28035da418ee5654a567b45118dcd9f0",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6",
                "size": 17398,
                "upload_time": "2019-10-17T09:15:01",
                "url": "https://files.pythonhosted.org/packages/f4/ab/17fcf175afaa9bf72d9a7fec20fc80862264ea34be0c2b9596438998be8f/kcrawler-1.0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "91c68b19e3200c531f7b5da6d7c5a468",
                    "sha256": "b615cf500a7d0844441532a8d12fb6f8af6602a133f3b81174fbafe35853003e"
                },
                "downloads": -1,
                "filename": "kcrawler-1.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "91c68b19e3200c531f7b5da6d7c5a468",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6",
                "size": 15270,
                "upload_time": "2019-10-17T09:15:03",
                "url": "https://files.pythonhosted.org/packages/47/8d/2024688a20cc387001cf48cd528d31872c9cca5fca263afbea2a66ebb25f/kcrawler-1.0.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "28035da418ee5654a567b45118dcd9f0",
                "sha256": "ecce519c57617d7d627f774b98cc2d551fc4860709f314c20188dcc552f1cc2d"
            },
            "downloads": -1,
            "filename": "kcrawler-1.0.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "28035da418ee5654a567b45118dcd9f0",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.6",
            "size": 17398,
            "upload_time": "2019-10-17T09:15:01",
            "url": "https://files.pythonhosted.org/packages/f4/ab/17fcf175afaa9bf72d9a7fec20fc80862264ea34be0c2b9596438998be8f/kcrawler-1.0.1-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "91c68b19e3200c531f7b5da6d7c5a468",
                "sha256": "b615cf500a7d0844441532a8d12fb6f8af6602a133f3b81174fbafe35853003e"
            },
            "downloads": -1,
            "filename": "kcrawler-1.0.1.tar.gz",
            "has_sig": false,
            "md5_digest": "91c68b19e3200c531f7b5da6d7c5a468",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.6",
            "size": 15270,
            "upload_time": "2019-10-17T09:15:03",
            "url": "https://files.pythonhosted.org/packages/47/8d/2024688a20cc387001cf48cd528d31872c9cca5fca263afbea2a66ebb25f/kcrawler-1.0.1.tar.gz"
        }
    ]
}