{
    "info": {
        "author": "eugtsa,azraev",
        "author_email": "eugtsa@gmail.com,azraev@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Natural Language :: English",
            "Operating System :: MacOS :: MacOS X",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.6",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Topic :: Software Development :: Build Tools"
        ],
        "description": "This is the documentation for the Shaman. Multiprocessing application to combine different singular handlers against one message.\n\nThe initial purpose was to create a tool, that:\n    - would make possible to download and analyze a content of an html pages.\n    - simple enough to add a new functionality in it.\n    - hast to be scalable (multiprocessing).\nActual usage can be different from it. There are some spontaneous ideas:\n    - scanning a mongo collection and parsing documents in parallel\n    - parsing a lot of lines from multiple huge files, saving the results to any database (depending on the results)\n\nThere are three parts in the shaman library::\n\n    * stages (actual processors, which do represent some functionality)\n    * consumer (worker, that run them all in a particular order)\n    * daemon (run as many as needed workers. Also used as a CLI unstrument.)\n    All stages are run in a particular order and use the same message object (inside one worker).\n\nINSTALLATION:\n----------------------\nRun the command::\n\n    pip install shaman\n\nIf everything is ok, you should be able to run::\n\n\n    shaman --help\n\nIt has to display::\n\n    usage: shaman [-h] [-i | -d] -c CONFIGURATION [--drop_first DROP_FIRST]\n                  [-p PRINT_FIELDS [PRINT_FIELDS ...]]\n                  [-r REMOVE_FIELDS [REMOVE_FIELDS ...]]\n                  [--ignore_after IGNORE_AFTER]\n                  [{stop,start,restart,} [{stop,start,restart,} ...]]\n\n    Main shaman module. Use it to start|stop|restart daemon or start non-daemon\n    modes of shaman\n\n    positional arguments:\n     {stop,start,restart,}\n                             Command to daemon (default: )\n\n    optional arguments:\n     -h, --help            show this help message and exit\n     -i                    Use stdin input as main input (default: False)\n     -d                    Daemonize main process (default: False)\n     -c CONFIGURATION      Path to configuration file (default: None)\n     --drop_first DROP_FIRST\n                           drop first lines (default: 0)\n     -p PRINT_FIELDS [PRINT_FIELDS ...], --print_fields PRINT_FIELDS [PRINT_FIELDS ...]\n     -r REMOVE_FIELDS [REMOVE_FIELDS ...], --remove_fields REMOVE_FIELDS [REMOVE_FIELDS ...]\n     --ignore_after IGNORE_AFTER\n\nCONFIGURATION:\n---------------------------\n\nYou may find an example configuration file in <path_to_python_lib>/site-packages/shaman/etc/crawler.config\nIt includes 4 stages::\n\n    reading from stdin\n    downloading page\n    detecting charset\n    print url, charset\n\nBy default, all stages reside in <path_to_python_lib>/site-packages/shaman/src/analyzers/ folder.\nYou may create your custom stage and put it into the custom folder.\nThere is a parameter in a configuration file::\n\n    custom_stage_dir = <custom_folder>\n\nIf you put some stages into this folder, shaman will also \"see\" them.\n\nTo check if anything is working, please, run::\n\n    echo \"http://google.ru\" | shaman -c <path_to_config> -i\n\nMore information about the package::\n\n    http://shaman.readthedocs.io/en/latest/\nGithub::\n\n    https://github.com/Landish145/shaman",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/Landish145/shaman",
        "keywords": "crawlers analyze development",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "shaman",
        "package_url": "https://pypi.org/project/shaman/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/shaman/",
        "project_urls": {
            "Homepage": "https://github.com/Landish145/shaman"
        },
        "release_url": "https://pypi.org/project/shaman/0.1.0.dev1/",
        "requires_dist": [
            "argparse",
            "bson",
            "configparser",
            "grab",
            "kafka-python",
            "pyformance",
            "check-manifest; extra == 'dev'",
            "coverage; extra == 'test'"
        ],
        "requires_python": "",
        "summary": "Multiprocessing application to download and analyze a content of an html pages.",
        "version": "0.1.0.dev1"
    },
    "last_serial": 2674824,
    "releases": {
        "0.0.2a1": [],
        "0.0.4.dev1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "55567652aa1c28aac0c26d5e9dc67c92",
                    "sha256": "0d82ac1eaec31fa8e00acf8a0a06c87826981b620cba106c585b1a3a28d94426"
                },
                "downloads": -1,
                "filename": "shaman-0.0.4.dev1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "55567652aa1c28aac0c26d5e9dc67c92",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 128747,
                "upload_time": "2017-02-10T19:40:53",
                "url": "https://files.pythonhosted.org/packages/43/bb/550d0ac0863e8838c3d156965e7b94140d47384ec5d232002b4b498fbf11/shaman-0.0.4.dev1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "79da5e61052f339642cc6aa3b9607159",
                    "sha256": "148cb6fea73eda91b423739fbb0d11d7677a60d92c09e2b0f2a302fe8bce3114"
                },
                "downloads": -1,
                "filename": "shaman-0.0.4.dev1.zip",
                "has_sig": false,
                "md5_digest": "79da5e61052f339642cc6aa3b9607159",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 131489,
                "upload_time": "2017-02-10T19:40:57",
                "url": "https://files.pythonhosted.org/packages/2f/d1/fd42b1a443b2f93f174cb590a4936ca4c28614d49059cd452743aaf41c16/shaman-0.0.4.dev1.zip"
            }
        ],
        "0.0.5.dev1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "536cf801426930c6a8f6b92f3b1caac9",
                    "sha256": "e50a3ba0e6b632082c91f4ab8d96629e83b5e4ff68d76823c0541c4be99df44b"
                },
                "downloads": -1,
                "filename": "shaman-0.0.5.dev1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "536cf801426930c6a8f6b92f3b1caac9",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 130339,
                "upload_time": "2017-02-10T20:12:05",
                "url": "https://files.pythonhosted.org/packages/b4/d6/748299f80e5f841711cbbbe61c08700cb7997a931bfdb1a8fd52e396bd25/shaman-0.0.5.dev1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "1cc5686b5ecc7caa32889e89844c6e56",
                    "sha256": "56e26b8f2ee5a4743e498ad5947d2c1abd761678274ced9eaa0fbb7f553994e2"
                },
                "downloads": -1,
                "filename": "shaman-0.0.5.dev1.zip",
                "has_sig": false,
                "md5_digest": "1cc5686b5ecc7caa32889e89844c6e56",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 133940,
                "upload_time": "2017-02-10T20:12:07",
                "url": "https://files.pythonhosted.org/packages/04/3c/5a3cb25f866ae69f7ab7a5e01d3440b5c57296630bcdbca630b983a71bce/shaman-0.0.5.dev1.zip"
            }
        ],
        "0.0.6.dev1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f869bedf23d681da783a8d536d7128d8",
                    "sha256": "9bd0eb723bb453cff6dae64d5175b442244233e2b29d6dc2648f9c29d8de7ba9"
                },
                "downloads": -1,
                "filename": "shaman-0.0.6.dev1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f869bedf23d681da783a8d536d7128d8",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 130354,
                "upload_time": "2017-02-10T20:21:21",
                "url": "https://files.pythonhosted.org/packages/0c/67/4f0ed5f8ab5c7d9cf4e64bfaa48c7c71e9394342aabe309ab9716b65cfd1/shaman-0.0.6.dev1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "bd9035e7eb3d1425cd5e9418e6a2e5b9",
                    "sha256": "1954e4b5550671e25be440b212e61e9c153cf37643fdf742f64f9a9425383a6b"
                },
                "downloads": -1,
                "filename": "shaman-0.0.6.dev1.zip",
                "has_sig": false,
                "md5_digest": "bd9035e7eb3d1425cd5e9418e6a2e5b9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 133971,
                "upload_time": "2017-02-10T20:21:26",
                "url": "https://files.pythonhosted.org/packages/16/46/11d95970c6e989e41622cba8d87d773b169bb71998f37a304cb2fd24f3ac/shaman-0.0.6.dev1.zip"
            }
        ],
        "0.0.7.dev1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "32b3813fb81be7a2873243269770f7a6",
                    "sha256": "dd093aeb49cb4d5a0ac2040f8ced873b581f950fc9b4281367c694d8ab0c72ce"
                },
                "downloads": -1,
                "filename": "shaman-0.0.7.dev1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "32b3813fb81be7a2873243269770f7a6",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 130403,
                "upload_time": "2017-02-10T20:34:10",
                "url": "https://files.pythonhosted.org/packages/dc/56/9ae8f7d3badd938cbf92c396627ec7018b75e6a8cdf5a713424a70785b1e/shaman-0.0.7.dev1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "d37aa125d344293eb385f4a5728f6b66",
                    "sha256": "fa1528c46d94a34448a2f57663d5e10c64bfccb55889477ed34854678bb4dc64"
                },
                "downloads": -1,
                "filename": "shaman-0.0.7.dev1.zip",
                "has_sig": false,
                "md5_digest": "d37aa125d344293eb385f4a5728f6b66",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 134049,
                "upload_time": "2017-02-10T20:34:13",
                "url": "https://files.pythonhosted.org/packages/7a/18/52fc5f7a55565784c4dd5fcd3d2e7d312b945eae77b85378cf93f7c6982d/shaman-0.0.7.dev1.zip"
            }
        ],
        "0.0.8.dev1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "538724775337b177cbda9036a83adbb9",
                    "sha256": "82ad08d9e7d9e4947da0fca884be7dc6810aba6ae8be1b58f39bd2afdd6dbcec"
                },
                "downloads": -1,
                "filename": "shaman-0.0.8.dev1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "538724775337b177cbda9036a83adbb9",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 130379,
                "upload_time": "2017-02-10T20:37:59",
                "url": "https://files.pythonhosted.org/packages/08/65/f5fd38817f79150117298461707ee30cdc434560a5dc9347eb64e34a46ad/shaman-0.0.8.dev1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "510a06c4ad00862063c20c9903943c41",
                    "sha256": "0849554468a85252719d5c3b97d4c2089cd8012be2ce40f3b81df63382e38a61"
                },
                "downloads": -1,
                "filename": "shaman-0.0.8.dev1.zip",
                "has_sig": false,
                "md5_digest": "510a06c4ad00862063c20c9903943c41",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 134015,
                "upload_time": "2017-02-10T20:38:02",
                "url": "https://files.pythonhosted.org/packages/68/33/cc7064f56689269a238e7b56f8d80acc74b428fb600f9edde6487c57dc27/shaman-0.0.8.dev1.zip"
            }
        ],
        "0.1.0.dev1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "623a65370cf4b546e38477fa6e73ac52",
                    "sha256": "655e89643066e71682e2502017c4653645d4fd3fea44c104f524b176f5155a0a"
                },
                "downloads": -1,
                "filename": "shaman-0.1.0.dev1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "623a65370cf4b546e38477fa6e73ac52",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 130417,
                "upload_time": "2017-03-01T07:51:44",
                "url": "https://files.pythonhosted.org/packages/d0/13/b05fa14de0a16b67720db549456595fd8c02bde156d08de2cf14bb5f34bb/shaman-0.1.0.dev1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "c79a210c0a3373acb44e1f91e6041959",
                    "sha256": "ec43ad608518a36dea9143bf3c2453571437d33f145f55e03677acb8752d941a"
                },
                "downloads": -1,
                "filename": "shaman-0.1.0.dev1.zip",
                "has_sig": false,
                "md5_digest": "c79a210c0a3373acb44e1f91e6041959",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 134052,
                "upload_time": "2017-03-01T07:51:47",
                "url": "https://files.pythonhosted.org/packages/1f/0a/1397db7e59a6e7f738e1fc3d1ce6dab31332c7996f8ab06a3b0340661699/shaman-0.1.0.dev1.zip"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "623a65370cf4b546e38477fa6e73ac52",
                "sha256": "655e89643066e71682e2502017c4653645d4fd3fea44c104f524b176f5155a0a"
            },
            "downloads": -1,
            "filename": "shaman-0.1.0.dev1-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "623a65370cf4b546e38477fa6e73ac52",
            "packagetype": "bdist_wheel",
            "python_version": "py2.py3",
            "requires_python": null,
            "size": 130417,
            "upload_time": "2017-03-01T07:51:44",
            "url": "https://files.pythonhosted.org/packages/d0/13/b05fa14de0a16b67720db549456595fd8c02bde156d08de2cf14bb5f34bb/shaman-0.1.0.dev1-py2.py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "c79a210c0a3373acb44e1f91e6041959",
                "sha256": "ec43ad608518a36dea9143bf3c2453571437d33f145f55e03677acb8752d941a"
            },
            "downloads": -1,
            "filename": "shaman-0.1.0.dev1.zip",
            "has_sig": false,
            "md5_digest": "c79a210c0a3373acb44e1f91e6041959",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 134052,
            "upload_time": "2017-03-01T07:51:47",
            "url": "https://files.pythonhosted.org/packages/1f/0a/1397db7e59a6e7f738e1fc3d1ce6dab31332c7996f8ab06a3b0340661699/shaman-0.1.0.dev1.zip"
        }
    ]
}