{
    "info": {
        "author": "Ha Pham",
        "author_email": "me@zniper.net",
        "bugtrack_url": null,
        "classifiers": [],
        "description": ".. image:: https://travis-ci.org/zniper/django-scraper.svg?branch=master\n          :target: https://travis-ci.org/zniper/django-scraper\n\n.. image:: https://coveralls.io/repos/zniper/django-scraper/badge.svg?branch=master \n          :target: https://coveralls.io/r/zniper/django-scraper?branch=master\n\n**django-scraper** is a Django application for collecting online content following user-defined instructions\n\nFeatures\n========\n\n* Extract content of given online website/pages and stored under JSON data\n* Crawl then extract content in multiple pages, with given depth.\n* Can download media files present in page\n* Have option for storing data under ZIP file\n* Support standard file system and AWS S3 storage\n* Customisable crawling requests for different scenarios\n* Process can be started from Django management command (~cron job) or with Python code\n* Support extracting multiple content (text, html, images, binary files) in the same page\n* Have content refinement (replacement) rules and black words filtering\n* Support custom proxy servers, and user-agents\n\n*Support Django 1.6, 1.7, and 1.8*\n\nSamples\n=======\n\nBelow is sample result from scraping https://news.ycombinator.com/ask\n\n* `Result ZIP file <https://dl.dropboxusercontent.com/u/44239448/94f8f6d9-news.ycombinator.com.zip>`_\n* JSON result via a renderer:\n\n.. image:: https://dl.dropboxusercontent.com/u/44239448/scraper-screen.jpg\n             :target: https://dl.dropboxusercontent.com/u/44239448/scraper-screen.jpg\n\nInstallation\n============\nThis application requires some other tools installed first::\n    \n    lxml\n    requests\n\n\n**django-scraper** installation can be made using `pip`::\n\n    pip install django-scraper\n\nFor more and latest information about configuration or usage, please visit the repository in github: https://github.com/zniper/django-scraper\n\nSupport\n=======\nIf you have any questions about this application, please email to: me@zniper.net",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/zniper/django-scraper",
        "keywords": "crawl scraper spider web pages data extract collect",
        "license": "The MIT License (MIT)",
        "maintainer": null,
        "maintainer_email": null,
        "name": "django-scraper",
        "package_url": "https://pypi.org/project/django-scraper/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/django-scraper/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "https://github.com/zniper/django-scraper"
        },
        "release_url": "https://pypi.org/project/django-scraper/0.3.8/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Django application for collecting online content following user-defined instructions",
        "version": "0.3.8"
    },
    "last_serial": 1562409,
    "releases": {
        "0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "da44f6016fa35058f7e3952aa740f5ee",
                    "sha256": "ebbda6d1e9044ee194dbd094daf1f308265da3775319fede304fe93b6b7e05c3"
                },
                "downloads": -1,
                "filename": "django-scraper-0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "da44f6016fa35058f7e3952aa740f5ee",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6927,
                "upload_time": "2014-07-04T06:57:32",
                "url": "https://files.pythonhosted.org/packages/03/1e/8c7531d6aa72b26d6b4c05a51d061503776a0af7f01220b8b0354179009e/django-scraper-0.1.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "cdfda13eb824d1ac8d994bbacae07077",
                    "sha256": "b4f166a6e1f81ebaed6b9d7f49f94d9b0931ef72ff700fc9b8f7ae951157bb51"
                },
                "downloads": -1,
                "filename": "django-scraper-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "cdfda13eb824d1ac8d994bbacae07077",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9399,
                "upload_time": "2014-07-11T16:29:29",
                "url": "https://files.pythonhosted.org/packages/b0/27/80bfe7b5545efbfceee49e85a238e57e4aabc8f28166de1bfca14d4ef71f/django-scraper-0.2.0.tar.gz"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0a739b0ca06bc295c2df0092234a32fd",
                    "sha256": "a0351ed4d943e52d3a68922be0f084ec83c7fca6e94c9f6e623948a0b44259eb"
                },
                "downloads": -1,
                "filename": "django-scraper-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "0a739b0ca06bc295c2df0092234a32fd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9850,
                "upload_time": "2014-10-10T09:51:23",
                "url": "https://files.pythonhosted.org/packages/56/5e/c234f08fb4a2c66054709078e5acbb9e6323f4e7737db57842e7891c1b40/django-scraper-0.2.2.tar.gz"
            }
        ],
        "0.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5a2a32f9593c261cd2828d31a695b9bc",
                    "sha256": "e89063474cb0f057b51c40e03d8eb0660baaf7df7b3179342fd5f2971d0f9ceb"
                },
                "downloads": -1,
                "filename": "django-scraper-0.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "5a2a32f9593c261cd2828d31a695b9bc",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 74038,
                "upload_time": "2015-03-12T09:09:20",
                "url": "https://files.pythonhosted.org/packages/f9/45/a115c8ad3b6405345cce9568fe9055e73745009a94ba3c1e9500a39bf82b/django-scraper-0.2.3.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "bf48e1fe42d82f2ecd594212d72b826f",
                    "sha256": "10e84cd6e3720ec604885e4aef3d3700e3b63eef88b03ccfffaa43eb374bd257"
                },
                "downloads": -1,
                "filename": "django-scraper-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "bf48e1fe42d82f2ecd594212d72b826f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 78659,
                "upload_time": "2015-05-07T08:57:51",
                "url": "https://files.pythonhosted.org/packages/4d/24/412defe2048267462fd01f4249001686e8d7f864a596c30acddcac9c1819/django-scraper-0.3.0.tar.gz"
            }
        ],
        "0.3.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "cf7a41e58d474a93b86acf35d08726bd",
                    "sha256": "dcac254b2493e73f491875dcec1444f6048559d5e0b4d3132626eee3ca10e208"
                },
                "downloads": -1,
                "filename": "django-scraper-0.3.8.tar.gz",
                "has_sig": false,
                "md5_digest": "cf7a41e58d474a93b86acf35d08726bd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 62848,
                "upload_time": "2015-05-26T06:22:47",
                "url": "https://files.pythonhosted.org/packages/e0/56/69acca1b5acefee25ef7d96237bb5ccc2dc43fec845577abf3bc131c03f5/django-scraper-0.3.8.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "cf7a41e58d474a93b86acf35d08726bd",
                "sha256": "dcac254b2493e73f491875dcec1444f6048559d5e0b4d3132626eee3ca10e208"
            },
            "downloads": -1,
            "filename": "django-scraper-0.3.8.tar.gz",
            "has_sig": false,
            "md5_digest": "cf7a41e58d474a93b86acf35d08726bd",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 62848,
            "upload_time": "2015-05-26T06:22:47",
            "url": "https://files.pythonhosted.org/packages/e0/56/69acca1b5acefee25ef7d96237bb5ccc2dc43fec845577abf3bc131c03f5/django-scraper-0.3.8.tar.gz"
        }
    ]
}