{
    "info": {
        "author": "Will Bennett",
        "author_email": "william.11bennett@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Environment :: Console",
            "Intended Audience :: Developers",
            "Intended Audience :: Science/Research",
            "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
            "Natural Language :: English",
            "Operating System :: MacOS",
            "Operating System :: Microsoft :: Windows",
            "Operating System :: OS Independent",
            "Operating System :: POSIX :: Linux",
            "Programming Language :: Python",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Topic :: Internet",
            "Topic :: Internet :: WWW/HTTP",
            "Topic :: Internet :: WWW/HTTP :: Indexing/Search"
        ],
        "description": "spidy Web Crawler\n=================\n\nSpidy (/sp\u02c8\u026adi/) is the simple, easy to use command line web crawler.\nGiven a list of web links, it uses Python\n```requests`` <http://docs.python-requests.org>`__ to query the\nwebpages, and ```lxml`` <http://lxml.de/index.html>`__ to extract all\nlinks from the page. Pretty simple!\n\n|spidy Logo|\n\n|Version: 1.6.2| |Release: 1.4.0| |License: GPL v3| |Python 3.3+| |All\nPlatforms!| |Open Source Love| |Lines of Code: 1553| |Lines of Docs:\n605| |Last Commit| |Travis CI Status| |PyPI Wheel| |PyPI Status|\n|Contributors| |Forks| |Stars|\n\nCreated by `rivermont <https://github.com/rivermont>`__ (/r\u026av\u025c\u02d0rm\u0252nt/)\nand `FalconWarriorr <https://github.com/Casillas->`__ (/f\u00e6lc\u028cnra\u026aj\u0254\u02d0r/),\nand developed with help from `these awesome\npeople <https://github.com/rivermont/spidy#contributors>`__. Looking for\ntechnical documentation? Check out\n```DOCS.md`` <https://github.com/rivermont/spidy/blob/master/spidy/docs/DOCS.md>`__\\ \nLooking to contribute to this project? Have a look at\n```CONTRIBUTING.md`` <https://github.com/rivermont/spidy/blob/master/spidy/docs/CONTRIBUTING.md>`__,\nthen check out the docs.\n\n--------------\n\n\ud83c\udf89 New Features!\n===============\n\nMultithreading\n~~~~~~~~~~~~~~\n\nCrawl all the things! Run separate threads to work on multiple pages at\nthe same time. Such fast. Very wow.\n\nPyPI\n~~~~\n\nInstall spidy with one line: ``pip install spidy-web-crawler``!\n\nAutomatic Testing with Travis CI\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nRelease v1.4.0 - #\\ `31663d3 <https://github.com/rivermont/spidy/commit/31663d34ceeba66ea9de9819b6da555492ed6a80>`__\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n`spidy Web Crawler Release\n1.4 <https://github.com/rivermont/spidy/releases/tag/1.4.0>`__\n\nContents\n========\n\n-  `spidy Web\n   Crawler <https://github.com/rivermont/spidy#spidy-web-crawler>`__\n-  `New Features! <https://github.com/rivermont/spidy#-new-features>`__\n-  `Contents <https://github.com/rivermont/spidy#contents>`__\n-  `How it Works <https://github.com/rivermont/spidy#how-it-works>`__\n-  `Why It's\n   Different <https://github.com/rivermont/spidy#why-its-different>`__\n-  `Features <https://github.com/rivermont/spidy#features>`__\n-  `Tutorial <https://github.com/rivermont/spidy#tutorial>`__\n\n   -  `Using with\n      Docker <https://github.com/rivermont/spidy#using-with-docker>`__\n   -  `Installing from\n      PyPI <https://github.com/rivermont/spidy#installing-from-pypi>`__\n   -  `Installing from Source\n      Code <https://github.com/rivermont/spidy#installing-from-source-code>`__\n   -  `Python\n      Installation <https://github.com/rivermont/spidy#python-installation>`__\n\n      -  `Windows and\n         Mac <https://github.com/rivermont/spidy#windows-and-mac>`__\n      -  `Anaconda <https://github.com/rivermont/spidy#anaconda>`__\n      -  `Python\n         Base <https://github.com/rivermont/spidy#python-base>`__\n      -  `Linux <https://github.com/rivermont/spidy#linux>`__\n\n   -  `Crawler\n      Installation <https://github.com/rivermont/spidy#crawler-installation>`__\n   -  `Launching <https://github.com/rivermont/spidy#launching>`__\n   -  `Running <https://github.com/rivermont/spidy#running>`__\n\n      -  `Config <https://github.com/rivermont/spidy#config>`__\n      -  `Start <https://github.com/rivermont/spidy#start>`__\n      -  `Autosave <https://github.com/rivermont/spidy#autosave>`__\n      -  `Force Quit <https://github.com/rivermont/spidy#force-quit>`__\n\n-  `How Can I Support\n   This? <https://github.com/rivermont/spidy#how-can-i-support-this>`__\n-  `Contributors <https://github.com/rivermont/spidy#contributors>`__\n-  `License <https://github.com/rivermont/spidy#license>`__\n\nHow it Works\n============\n\nSpidy has two working lists, ``TODO`` and ``DONE``. 'TODO' is the list\nof URLs it hasn't yet visited. 'DONE' is the list of URLs it has already\nbeen to. The crawler visits each page in TODO, scrapes the DOM of the\npage for links, and adds those back into TODO. It can also save each\npage, because datahoarding \ud83d\ude1c.\n\nWhy It's Different\n==================\n\nWhat sets spidy apart from other web crawling solutions written in\nPython?\n\nMost of the other options out there are not web crawlers themselves,\nsimply frameworks and libraries through which one can create and deploy\na web spider for example Scrapy and BeautifulSoup. Scrapy is a Web\ncrawling framework, written in Python, specifically created for\ndownloading, cleaning and saving data from the web whereas BeautifulSoup\nis a parsing library that allows a programmer to get specific elements\nout of a webpage but BeautifulSoup alone is not enough because you have\nto actually get the webpage in the first place.\n\nBut with Spidy, everything runs right out of the box. Spidy is a Web\nCrawler which is easy to use and is run from the command line. You have\nto give it a URL link of the webpage and it starts crawling away! A very\nsimple and effective way of fetching stuff off of the web.\n\nFeatures\n========\n\nWe built a lot of the functionality in spidy by watching the console\nscroll by and going, \"Hey, we should add that!\" Here are some features\nwe figure are worth noting.\n\n-  Error Handling: We have tried to recognize all of the errors spidy\n   runs into and create custom error messages and logging for each.\n   There is a set cap so that after accumulating too many errors the\n   crawler will stop itself.\n-  Cross-Platform compatibility: spidy will work on all three major\n   operating systems, Windows, Mac OS/X, and Linux!\n-  Frequent Timestamp Logging: Spidy logs almost every action it takes\n   to both the console and one of two log files.\n-  Browser Spoofing: Make requests using User Agents from 4 popular web\n   browsers, use a custom spidy bot one, or create your own!\n-  Portability: Move spidy's folder and its contents somewhere else and\n   it will run right where it left off. *Note*: This only works if you\n   run it from source code.\n-  User-Friendly Logs: Both the console and log file messages are simple\n   and easy to interpret, but packed with information.\n-  Webpage saving: Spidy downloads each page that it runs into,\n   regardless of file type. The crawler uses the HTTP ``Content-Type``\n   header returned with most files to determine the file type.\n-  File Zipping: When autosaving, spidy can archive the contents of the\n   ``saved/`` directory to a ``.zip`` file, and then clear ``saved/``.\n\nTutorial\n========\n\nUsing with Docker\n-----------------\n\nSpidy can be easily run in a Docker container.\n\n-  First, build the ```Dockerfile`` <dockerfile>`__:\n   ``docker build -t spidy .``\n-  Verify that the Docker image has been created: ``docker images``\n-  Then, run it: ``docker run --rm -it -v $PWD:/data spidy``\n-  ``--rm`` tells Docker to clean up after itself by removing stopped\n   containers.\n-  ``-it`` tells Docker to run the container interactively and allocate\n   a pseudo-TTY.\n-  ``-v $PWD:/data`` tells Docker to mount the current working directory\n   as ``/data`` directory inside the container. This is needed if you\n   want Spidy's files (e.g. ``crawler_done.txt``, ``crawler_words.txt``,\n   ``crawler_todo.txt``) written back to your host filesystem.\n\nSpidy Docker Demo\n~~~~~~~~~~~~~~~~~\n\n.. figure:: media/spidy_docker_demo.gif\n   :alt: Spidy Docker Demo\n\n   Spidy Docker Demo\n\nInstalling from PyPI\n--------------------\n\nSpidy can be found on the Python Package Index as ``spidy-web-crawler``.\nYou can install it from your package manager of choice and simple run\nthe ``spidy`` command. The working files will be found in your home\ndirectory.\n\nInstalling from Source Code\n---------------------------\n\nAlternatively, you can download the source code and run it.\n\nPython Installation\n~~~~~~~~~~~~~~~~~~~\n\nThe way that you will run spidy depends on the way you have Python\ninstalled.\n\nWindows and Mac\n^^^^^^^^^^^^^^^\n\nThere are many different versions of\n`Python <https://www.python.org/about/>`__, and hundreds of different\ninstallations for each them. Spidy is developed for Python v3.5.2, but\nshould run without errors in other versions of Python 3.\n\nAnaconda\n''''''''\n\nWe recommend the `Anaconda\ndistribution <https://www.continuum.io/downloads>`__. It comes\npre-packaged with lots of goodies, including ``lxml``, which is required\nfor spidy to run and not including in the standard Python package.\n\nPython Base\n'''''''''''\n\nYou can also just install `default\nPython <https://www.python.org/downloads/>`__, and install the external\nlibraries separately. This can be done with ``pip``:\n\n::\n\n    pip install -r requirements.txt\n\nLinux\n^^^^^\n\nPython 3 should come preinstalled with most flavors of Linux, but if\nnot, simply run\n\n::\n\n    sudo apt update\n    sudo apt install python3 python3-lxml python3-requests\n\nThen ``cd`` into the crawler's directory and run ``python3 crawler.py``.\n\nCrawler Installation\n~~~~~~~~~~~~~~~~~~~~\n\nIf you have git or GitHub Desktop installed, you can clone the\nrepository `from here <https://github.com/rivermont/spidy.git>`__. If\nnot, download `the latest source\ncode <https://github.com/rivermont/spidy/archive/master.zip>`__ or grab\nthe `latest release <https://github.com/rivermont/spidy/releases>`__.\n\nLaunching\n~~~~~~~~~\n\nUse ``cd`` to navigate to the directory that spidy is located in, then\nrun:\n\n::\n\n    python crawler.py\n\n.. figure:: https://raw.githubusercontent.com/rivermont/spidy/master/media/run.gif\n   :alt: \n\nRunning\n~~~~~~~\n\nSpidy logs a lot of information to the command line throughout its life.\nOnce started, a bunch of ``[INIT]`` lines will print. These announce\nwhere spidy is in its initialization process.\n\nConfig\n^^^^^^\n\nOn running, spidy asks for input regarding certain parameters it will\nrun off of. However, you can also use one of the configuration files, or\neven create your own.\n\nTo use spidy with a configuration file, input the name of the file when\nthe crawler asks\n\nThe config files included with spidy are:\n\n-  *``blank.txt``*: Template for creating your own configurations.\n-  ``default.cfg``: The default version.\n-  ``heavy.cfg``: Run spidy with all of its features enabled.\n-  ``infinite.cfg``: The default config, but it never stops itself.\n-  ``light.cfg``: Disable most features; only crawls pages for links.\n-  ``rivermont.cfg``: My personal favorite settings.\n-  ``rivermont-infinite.cfg``: My favorite, never-ending configuration.\n\nStart\n^^^^^\n\nSample start log.\n\n.. figure:: https://raw.githubusercontent.com/rivermont/spidy/master/media/start.png\n   :alt: \n\nAutosave\n^^^^^^^^\n\nSample log after hitting the autosave cap.\n\n.. figure:: https://raw.githubusercontent.com/rivermont/spidy/master/media/log.png\n   :alt: \n\nForce Quit\n^^^^^^^^^^\n\nSample log after performing a ``^C`` (CONTROL + C) to force quit the\ncrawler.\n\n.. figure:: https://raw.githubusercontent.com/rivermont/spidy/master/media/keyboardinterrupt.png\n   :alt: \n\nHow Can I Support This?\n=======================\n\nThe easiest thing you can do is Star spidy if you think it's cool, or\nWatch it if you would like to get updates. If you have a suggestion,\n`create an Issue <https://github.com/rivermont/spidy/issues/new>`__ or\nFork the ``master`` branch and open a Pull Request.\n\nContributors\n============\n\nSee the\n```CONTRIBUTING.md`` <https://github.com/rivermont/spidy/blob/master/spidy/docs/CONTRIBUTING.md>`__\n\n-  The logo was designed by `Cutwell <https://github.com/Cutwell>`__\n\n-  `3onyc <https://github.com/3onyc>`__ - PEP8 Compliance.\n-  `DeKaN <https://github.com/DeKaN>`__ - Getting PyPI packaging to\n   work.\n-  `esouthren <https://github.com/esouthren>`__ - Unit testing.\n-  `Hrily <https://github.com/Hrily>`__ - Multithreading.\n-  `j-setiawan <https://github.com/j-setiawan>`__ - Paths that work on\n   all OS's.\n-  `michellemorales <https://github.com/michellemorales>`__ - Confirmed\n   OS/X support.\n-  `petermbenjamin <https://github.com/petermbenjamin>`__ - Docker\n   support.\n-  `quatroka <https://github.com/quatroka>`__ - Fixed testing bugs.\n-  `stevelle <https://github.com/stevelle>`__ - Respect robots.txt.\n-  `thatguywiththatname <https://github.com/thatguywiththatname>`__ -\n   README link corrections.\n\nLicense\n=======\n\nWe used the `Gnu General Public\nLicense <https://www.gnu.org/licenses/gpl-3.0.en.html>`__ (see\n```LICENSE`` <https://github.com/rivermont/spidy/blob/master/LICENSE>`__)\nas it was the license that best suited our needs. Honestly, if you link\nto this repo and credit ``rivermont`` and ``FalconWarriorr``, and you\naren't selling spidy in any way, then we would love for you to\ndistribute it. Thanks!\n\n--------------\n\n.. |spidy Logo| image:: https://raw.githubusercontent.com/rivermont/spidy/master/media/spidy_logo.png\n   :target: https://github.com/rivermont/spidy#contributors\n.. |Version: 1.6.2| image:: https://img.shields.io/badge/version-1.6.2-brightgreen.svg\n.. |Release: 1.4.0| image:: https://img.shields.io/github/release/rivermont/spidy.svg\n   :target: https://github.com/rivermont/spidy/releases\n.. |License: GPL v3| image:: https://img.shields.io/badge/license-GPLv3.0-blue.svg\n   :target: http://www.gnu.org/licenses/gpl-3.0\n.. |Python 3.3+| image:: https://img.shields.io/badge/python-3.3+-brightgreen.svg\n   :target: https://docs.python.org/3/\n.. |All Platforms!| image:: https://img.shields.io/badge/Windows,%20OS/X,%20Linux-%20%20-brightgreen.svg\n.. |Open Source Love| image:: https://badges.frapsoft.com/os/v1/open-source.png?v=103\n.. |Lines of Code: 1553| image:: https://img.shields.io/badge/lines%20of%20code-1553-brightgreen.svg\n.. |Lines of Docs: 605| image:: https://img.shields.io/badge/lines%20of%20docs-605-orange.svg\n.. |Last Commit| image:: https://img.shields.io/github/last-commit/rivermont/spidy.svg\n   :target: https://github.com/rivermont/spidy/graphs/punch-card\n.. |Travis CI Status| image:: https://img.shields.io/travis/rivermont/spidy/master.svg\n   :target: https://travis-ci.org/rivermont/spidy\n.. |PyPI Wheel| image:: https://img.shields.io/pypi/wheel/spidy-web-crawler.svg\n   :target: https://pypi.org/project/spidy-web-crawler/\n.. |PyPI Status| image:: https://img.shields.io/pypi/status/spidy-web-crawler.svg\n   :target: https://pypi.org/project/spidy-web-crawler/\n.. |Contributors| image:: https://img.shields.io/github/contributors/rivermont/spidy.svg\n   :target: https://github.com/rivermont/spidy/graphs/contributors\n.. |Forks| image:: https://img.shields.io/github/forks/rivermont/spidy.svg?style=social&label=Forks\n   :target: https://github.com/rivermont/spidy/network\n.. |Stars| image:: https://img.shields.io/github/stars/rivermont/spidy.svg?style=social&label=Stars\n   :target: https://github.com/rivermont/spidy/stargazers\n\n\n",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/rivermont/spidy",
        "keywords": "crawler,web crawler,spider,web-spider",
        "license": "GPLv3",
        "maintainer": "",
        "maintainer_email": "",
        "name": "spidy-web-crawler",
        "package_url": "https://pypi.org/project/spidy-web-crawler/",
        "platform": "",
        "project_url": "https://pypi.org/project/spidy-web-crawler/",
        "project_urls": {
            "Homepage": "https://github.com/rivermont/spidy"
        },
        "release_url": "https://pypi.org/project/spidy-web-crawler/1.6.5/",
        "requires_dist": [
            "requests",
            "lxml",
            "flake8",
            "reppy"
        ],
        "requires_python": ">=3.3",
        "summary": "Spidy is the simple, easy to use command line web crawler.",
        "version": "1.6.5"
    },
    "last_serial": 3519020,
    "releases": {
        "1.5.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a76e33130a4b6d5432611725aeed4bc2",
                    "sha256": "aa569bc292c347e7c914ea88176e74721108199cf320ef47702d1df341b1d699"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.5.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a76e33130a4b6d5432611725aeed4bc2",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 27380,
                "upload_time": "2017-10-07T01:48:03",
                "url": "https://files.pythonhosted.org/packages/6f/f2/a9447ab5ef111f432be73e7b6842a9f4632d5445ff6570bcb7e92c0546ff/spidy_web_crawler-1.5.0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "9d8e0025ccbe7fa337e087fc8b681305",
                    "sha256": "bc5b11e62f6bc5f85bb07eef39ccfd41c5981031513338241f882a794e84cce9"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.5.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "9d8e0025ccbe7fa337e087fc8b681305",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 22646,
                "upload_time": "2017-10-07T01:48:06",
                "url": "https://files.pythonhosted.org/packages/73/c2/4b90852a8483bf2534d0a76810a3263d89249fc8d5a36c6e283a5201bda4/spidy-web-crawler-1.5.0.1.tar.gz"
            }
        ],
        "1.5.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "72d9a1a78365b595fa52bb60dbc5cc31",
                    "sha256": "360d44bffacc20bf072466e60c2686a305c6b11d3a3eadff25f94b143d9d48fb"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.5.0.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "72d9a1a78365b595fa52bb60dbc5cc31",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 27575,
                "upload_time": "2017-10-07T15:10:42",
                "url": "https://files.pythonhosted.org/packages/b6/ff/0e9924d70e7e4957137bf58643499f30f24d2b339fc47f81407ad46e637a/spidy_web_crawler-1.5.0.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "a50b1c7971647deb596ff2f50dfea5ef",
                    "sha256": "55eddf2faef16cd3bc29859f6870df03f5055ba735d6090cdaa654549ecf4998"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.5.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "a50b1c7971647deb596ff2f50dfea5ef",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 22884,
                "upload_time": "2017-10-07T15:10:46",
                "url": "https://files.pythonhosted.org/packages/b4/e6/e6c8ff7943180df9bd9d9aacb1274f322e95d52b46cee3b7e2cb628395b0/spidy-web-crawler-1.5.0.2.tar.gz"
            }
        ],
        "1.5.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "07427cfb5d22160c691a88a1393aee55",
                    "sha256": "42d44125060951a3ffba5ae872c701fde5a6bbce9eb220739ff07af22bbea2bc"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.5.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "07427cfb5d22160c691a88a1393aee55",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.3",
                "size": 27796,
                "upload_time": "2017-10-08T20:04:35",
                "url": "https://files.pythonhosted.org/packages/bd/5a/0c4dcd56b74776a5518a43f4e722abb37c81a84bae667ce36d013a6912eb/spidy_web_crawler-1.5.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "bbe0e2abdde8251aa66048128661aafc",
                    "sha256": "fa458e825d3a68e403c6b3cd065835884444f26ab36d85da37ed2dfb88bbc3fb"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.5.2.tar.gz",
                "has_sig": false,
                "md5_digest": "bbe0e2abdde8251aa66048128661aafc",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.3",
                "size": 23998,
                "upload_time": "2017-10-08T20:04:36",
                "url": "https://files.pythonhosted.org/packages/21/d3/fcc738787e0ff397fe2084e1f6dd642621a75751881ff2e88f9f2718caef/spidy-web-crawler-1.5.2.tar.gz"
            }
        ],
        "1.5.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f7e8317d065b9c1787fcb68e90b1c0ab",
                    "sha256": "93a2e5893ffc4c89e4d43c94c7eb85c6885f93c60995027b916ca9a7175beb08"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.5.2.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f7e8317d065b9c1787fcb68e90b1c0ab",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.3",
                "size": 34718,
                "upload_time": "2017-10-10T14:16:04",
                "url": "https://files.pythonhosted.org/packages/ed/81/a006be22381aa75174da768ca94f6a2cd84d3bbc354d18934f4549b9ee77/spidy_web_crawler-1.5.2.1-py3-none-any.whl"
            }
        ],
        "1.5.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d42b55751569e3d6e69ad0afc0f0060a",
                    "sha256": "070c03b12f7ec2fe311d2cb02c535353e8f362626ff631cb05292d37cfb86285"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.5.2.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "d42b55751569e3d6e69ad0afc0f0060a",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.3",
                "size": 34715,
                "upload_time": "2017-10-10T14:16:30",
                "url": "https://files.pythonhosted.org/packages/9c/1b/cc2c419bca1d33a284b283ca7be3c2f4904bf23f456941619cb349894d83/spidy_web_crawler-1.5.2.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "30de6b8c6f06eb40b4fc7c14ee39ef3b",
                    "sha256": "4eb9cfe11e3ec3f6e530725f660a8771695f19141c9ce36b938a67c8d2c4c185"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.5.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "30de6b8c6f06eb40b4fc7c14ee39ef3b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.3",
                "size": 28805,
                "upload_time": "2017-10-10T14:16:34",
                "url": "https://files.pythonhosted.org/packages/ee/7d/adedb8c3b0533c6d652b027d616adf3391b66e4080e2f1396e19d1f4148f/spidy-web-crawler-1.5.2.2.tar.gz"
            }
        ],
        "1.5.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e79b04be82c7b1916fe14dd8f09af93d",
                    "sha256": "f05abafee4bb1efa7b4a1d236d1ec8b9c49f9294aa419a16197ed519d80b27d7"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.5.2.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "e79b04be82c7b1916fe14dd8f09af93d",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.3",
                "size": 34711,
                "upload_time": "2017-10-10T14:20:08",
                "url": "https://files.pythonhosted.org/packages/5e/46/4ac7651e9240f4dd24109b53c5bcca77238a499d8abe73ac582df17ed011/spidy_web_crawler-1.5.2.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "9286a8335005cb3323a791a97ce81b42",
                    "sha256": "ce1473f7e345ed94229e881274ebc92e8e5d77efcf3727b9abde0ba58546e501"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.5.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "9286a8335005cb3323a791a97ce81b42",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.3",
                "size": 28799,
                "upload_time": "2017-10-10T14:20:09",
                "url": "https://files.pythonhosted.org/packages/c8/87/e9471c7a1b6b86310ed80b80ec6f1c9c93cb41b4158bae9ca0b7d301777c/spidy-web-crawler-1.5.2.3.tar.gz"
            }
        ],
        "1.5.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c83755d4901b9c5d6ea69c9dcddb5d23",
                    "sha256": "83d50e2a16d70e9d877dfebab8cad06bdda9db22a9f16ca776c71d0953e709b5"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.5.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "c83755d4901b9c5d6ea69c9dcddb5d23",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.3",
                "size": 33953,
                "upload_time": "2017-10-10T14:36:56",
                "url": "https://files.pythonhosted.org/packages/76/c9/0b32e082cc24554ed3bb397a5784f746dc4c595ae9b418e443edc204b169/spidy_web_crawler-1.5.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "f6e3c4127847de3d6ceebce1b3645d9d",
                    "sha256": "9ba572b79197ef5f27ceadd4349d3fb52c67dfe6c6302631281b681383fc43e5"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.5.3.tar.gz",
                "has_sig": false,
                "md5_digest": "f6e3c4127847de3d6ceebce1b3645d9d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.3",
                "size": 28176,
                "upload_time": "2017-10-10T14:37:02",
                "url": "https://files.pythonhosted.org/packages/9b/e0/78fd05eb1e045f1d519c0ddc4039550d74d7b94f2676d4dc5d45d4892991/spidy-web-crawler-1.5.3.tar.gz"
            }
        ],
        "1.5.3.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9db248b1cf7ae9c8a0e220793057cc02",
                    "sha256": "9b8ee8a3f39953c2e1157b14cdafeb0763eb5fb1f700bf8b348528e7fa8be8e0"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.5.3.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9db248b1cf7ae9c8a0e220793057cc02",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.3",
                "size": 34011,
                "upload_time": "2017-10-10T14:39:46",
                "url": "https://files.pythonhosted.org/packages/01/dd/fb71db4ec0204ecce7b3b3f669dac4d1e7385a310f9a083a6b4ad23ae4f0/spidy_web_crawler-1.5.3.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "380e835f4909f6b46f671ac2f755ca90",
                    "sha256": "21cc77521951d396f1dcd2c7960bdbed8edff2f8702ab14d25533b47b4f54dee"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.5.3.1.tar.gz",
                "has_sig": false,
                "md5_digest": "380e835f4909f6b46f671ac2f755ca90",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.3",
                "size": 28198,
                "upload_time": "2017-10-10T14:39:49",
                "url": "https://files.pythonhosted.org/packages/de/dc/a06a14a883c79c7431f3f460bee30ac5fb400d6bf963c568456423b7a32f/spidy-web-crawler-1.5.3.1.tar.gz"
            }
        ],
        "1.6.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "63da087a8abcbd1aeb36001197744eca",
                    "sha256": "ecead3aad8b4d4e2afdd767a9ff4732f31e1f6547e316c633c27d3117b3e4e87"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.6.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "63da087a8abcbd1aeb36001197744eca",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.3",
                "size": 36022,
                "upload_time": "2017-10-13T02:06:06",
                "url": "https://files.pythonhosted.org/packages/02/8b/547758899524950d1c81e2e74cccde17470ed0c520fd7ee8b89d7903779a/spidy_web_crawler-1.6.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3f0ce28a01e8e61e61933ce8ac77678d",
                    "sha256": "f1aec343b9cab64ba6f47a4793ef4c2ab8c6870cecdff3e31abe1b208d11abf1"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.6.0.tar.gz",
                "has_sig": false,
                "md5_digest": "3f0ce28a01e8e61e61933ce8ac77678d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.3",
                "size": 30278,
                "upload_time": "2017-10-13T02:06:08",
                "url": "https://files.pythonhosted.org/packages/59/f2/95cebef445410d9580429625dbaf9ae10b415a0ead9716fd565e35937495/spidy-web-crawler-1.6.0.tar.gz"
            }
        ],
        "1.6.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "554c82ceb8df4bdb28a322efbd53428a",
                    "sha256": "fe2861bdcd8b36cfea21db6162ad6fb0d8b26c218a21670e1aebcf8e88dd198d"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.6.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "554c82ceb8df4bdb28a322efbd53428a",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.3",
                "size": 37083,
                "upload_time": "2017-10-13T17:42:56",
                "url": "https://files.pythonhosted.org/packages/76/e3/6e22c223c05b9e5f4ebc452345b1d02f1959b304dcfda42dceee8517d764/spidy_web_crawler-1.6.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "6ddf7c546ab2b7e420838ffcc20eb3b1",
                    "sha256": "b32e7ce2ee9dc454baac9d78f408dd5bdf7863eefe426c9fef9d9021e22ce83d"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.6.2.tar.gz",
                "has_sig": false,
                "md5_digest": "6ddf7c546ab2b7e420838ffcc20eb3b1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.3",
                "size": 30955,
                "upload_time": "2017-10-13T17:42:57",
                "url": "https://files.pythonhosted.org/packages/0b/cd/d520ccfa91e5fd0ab27c14ad95ce965ad8842c2043d1a2887f9b42171058/spidy-web-crawler-1.6.2.tar.gz"
            }
        ],
        "1.6.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d6be71491366f9c4dece359a8047197c",
                    "sha256": "3b17445bf568ed8831940c622a8fb79526c9c3ee13a9da79b52eed15cca2d9a6"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.6.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "d6be71491366f9c4dece359a8047197c",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.3",
                "size": 38033,
                "upload_time": "2017-11-04T03:08:39",
                "url": "https://files.pythonhosted.org/packages/ea/17/ee3160103f901bcbbed7a0edcb53dbb7dd33d10ccb6f8bdfe760e7c07d92/spidy_web_crawler-1.6.4-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "f922c44f352067cde1ecbcc934a2df44",
                    "sha256": "4478fd127ed393e87f546ff47967ef0bfb9fdfdf123e650797590e49359a6b6e"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.6.4.tar.gz",
                "has_sig": false,
                "md5_digest": "f922c44f352067cde1ecbcc934a2df44",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.3",
                "size": 31994,
                "upload_time": "2017-11-04T03:08:40",
                "url": "https://files.pythonhosted.org/packages/c3/5c/00f7bd846afe8e8805890d58108fd0171ad478b17c64893215ddb7f4abd9/spidy-web-crawler-1.6.4.tar.gz"
            }
        ],
        "1.6.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "bb3afb06561bfa7001fd59554327597d",
                    "sha256": "16d29dbfbdcff34898166446a1d7e8c85cc732da701078a129fe1de80935137b"
                },
                "downloads": -1,
                "filename": "spidy_web_crawler-1.6.5-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "bb3afb06561bfa7001fd59554327597d",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.3",
                "size": 38035,
                "upload_time": "2018-01-25T01:01:26",
                "url": "https://files.pythonhosted.org/packages/01/ed/d34fb9006f587f6d2e6a5e8108228c870329b5a87b5cf1435c635adeece3/spidy_web_crawler-1.6.5-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "c92f78629cb2f9463db7b3ce090bc035",
                    "sha256": "08b98984b39581a52142385787661eeb3aa43319a54e83ca728b4ad5fad2e958"
                },
                "downloads": -1,
                "filename": "spidy-web-crawler-1.6.5.tar.gz",
                "has_sig": false,
                "md5_digest": "c92f78629cb2f9463db7b3ce090bc035",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.3",
                "size": 32007,
                "upload_time": "2018-01-25T01:01:27",
                "url": "https://files.pythonhosted.org/packages/a1/a6/4b940249d6e857c15afd759013540295130c45ce06d4fd9fda5867f05142/spidy-web-crawler-1.6.5.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "bb3afb06561bfa7001fd59554327597d",
                "sha256": "16d29dbfbdcff34898166446a1d7e8c85cc732da701078a129fe1de80935137b"
            },
            "downloads": -1,
            "filename": "spidy_web_crawler-1.6.5-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "bb3afb06561bfa7001fd59554327597d",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.3",
            "size": 38035,
            "upload_time": "2018-01-25T01:01:26",
            "url": "https://files.pythonhosted.org/packages/01/ed/d34fb9006f587f6d2e6a5e8108228c870329b5a87b5cf1435c635adeece3/spidy_web_crawler-1.6.5-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "c92f78629cb2f9463db7b3ce090bc035",
                "sha256": "08b98984b39581a52142385787661eeb3aa43319a54e83ca728b4ad5fad2e958"
            },
            "downloads": -1,
            "filename": "spidy-web-crawler-1.6.5.tar.gz",
            "has_sig": false,
            "md5_digest": "c92f78629cb2f9463db7b3ce090bc035",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.3",
            "size": 32007,
            "upload_time": "2018-01-25T01:01:27",
            "url": "https://files.pythonhosted.org/packages/a1/a6/4b940249d6e857c15afd759013540295130c45ce06d4fd9fda5867f05142/spidy-web-crawler-1.6.5.tar.gz"
        }
    ]
}