{
    "info": {
        "author": "Matthew Story",
        "author_email": "matt.story@axial.net",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "===\r\nFSQ\r\n===\r\n\r\nfsq is a standard for implementing queueing structures in POSIX file-systems.\r\nfsq provides a standard for both directory layouts and work-item naming, which\r\nallow for idempotent work-item construction, atomic enqueueing, and atomic\r\ncompletion.\r\n\r\nThe fsq python library (``import fsq``) provides a programattic way to\r\nenqueue, scan, introspect and manipulate queues from Python.\r\n\r\nThe fsq program provides mechanisms for enqueueing, scanning, introspecting,\r\nauditing and repairing queues from the command line.\r\n\r\nFor more on the fsq standard see ``man 7 fsq``, for more on the fsq program\r\nsee ``man 1 fsq``.\r\n\r\nInstalling\r\n==========\r\n\r\nThe fsq project lives on github_, and is available via pip_.\r\n\r\n.. _github: https://github.com/axialmarket/fsq\r\n.. _pip: https://pypi.python.org/pypi?:action=display&name=fsq\r\n\r\nInstalling v0.2.5 From Pip\r\n--------------------------\r\n\r\n::\r\n\r\n    sudo pip install fsq==0.2.5\r\n\r\nInstalling v0.2.5 From Source\r\n-----------------------------\r\n\r\n::\r\n\r\n    curl https://github.com/axialmarket/fsq/archive/version_0.2.5.tar.gz | tar vzxf -\r\n    cd fsq\r\n    sudo python setup.py install\r\n\r\nQuick Overview\r\n==============\r\n\r\nInstalling Queues\r\n-----------------\r\n\r\nTo install a queue, simply run::\r\n\r\n    $ fsq install a_queue\r\n\r\nOr via the Python API::\r\n\r\n    >>> import fsq\r\n    >>> fsq.install(a_queue)\r\n\r\nInstalling a queue will create a directory in ``FSQ_ROOT`` (``/var/fsq/``)::\r\n\r\n    /var/fsq/a_queue\r\n    \u251c\u2500\u2500 done\r\n    \u251c\u2500\u2500 fail\r\n    \u251c\u2500\u2500 queue\r\n    \u2514\u2500\u2500 tmp\r\n\r\nEnqueueing Work\r\n---------------\r\n\r\nTo enqueue work to the ``a_queue`` queue, simply run::\r\n\r\n    $ echo \"data\" | fsq enqueue a_queue args to enqueue\r\n\r\nOr from the Python API::\r\n\r\n    >>> import fsq\r\n    >>> # enqueue a string\r\n    >>> fsq.senqueue('a_queue', 'data', 'args', 'to', 'enqueue')\r\n    >>> # ... or a file\r\n    >>> fsq.senqueue('a_queue', '/path/to/data.file', 'args', 'to', 'enqueue')\r\n\r\nEnqueueing adds a file to the ``queue`` directory of ``a_queue``::\r\n\r\n    /var/fsq/a_queue\r\n    \u251c\u2500\u2500 done\r\n    \u251c\u2500\u2500 fail\r\n    \u251c\u2500\u2500 queue\r\n    |   \u2514\u2500\u2500 _20131005205643_0_25577_mss_0_args_to_enqueue\r\n    \u2514\u2500\u2500 tmp\r\n\r\nProcessing Work\r\n---------------\r\n\r\nTo process jobs, use the ``fsq scan`` program::\r\n\r\n    $ # echo gets \"args\", \"to\", \"enqueue\" as $1..$3 and \"data\" on stdin\r\n    $ fsq scan a_queue echo\r\n    args to enqueue\r\n\r\nOr from the Python API::\r\n\r\n    >>> import fsq\r\n    >>> for work in fsq.scan('a_queue'):\r\n    ...     print \" \".join(work.arguments)\r\n    ...     fsq.done('a_queue')\r\n    args to enqueue\r\n\r\nWork that is successfully completed moves to the done directory::\r\n\r\n    /var/fsq/a_queue\r\n    \u251c\u2500\u2500 done\r\n    |   \u2514\u2500\u2500 _20131005205643_0_25577_mss_0_args_to_enqueue\r\n    \u251c\u2500\u2500 fail\r\n    \u251c\u2500\u2500 queue\r\n    \u2514\u2500\u2500 tmp\r\n\r\nAs fsq scans each work item, it obtains an exclusive lock on the work item\r\nfile, so it is safe to run multiple scan processes (or threads) in parallel on\r\nthe same queue with no fear of duplicating effort.\r\n\r\nFailures in Processing Work\r\n---------------------------\r\n\r\nShould work fail during processing::\r\n\r\n    $ fsq scan a_queue sh -c 'exit 100'\r\n\r\nOr from the Python API::\r\n\r\n    >>> import fsq\r\n    >>> for work in fsq.scan('a_queue'):\r\n    ...     fsq.fail('a_queue')\r\n\r\nThe failed work will be moved to the fail directory::\r\n\r\n    /var/fsq/a_queue\r\n    \u251c\u2500\u2500 done\r\n    \u251c\u2500\u2500 fail\r\n    |   \u2514\u2500\u2500 _20131005205643_0_25577_mss_0_args_to_enqueue\r\n    \u251c\u2500\u2500 queue\r\n    \u2514\u2500\u2500 tmp\r\n\r\nWork can also fail temporarily, which will cause the work to remain in the\r\n``queue`` directory until it is older than ``FSQ_TTL`` seconds old, or until\r\nit has been tried more than ``FSQ_MAX_TRIES`` times unsuccessfully::\r\n\r\n    $ # exit code 111 indicates temporary failure\r\n    $ FSQ_MAX_TRIES=2 fsq scan a_queue sh -c 'exit 100'\r\n\r\nOr from the Python API::\r\n\r\n    >>> import fsq\r\n    >>> fsq.set_const('FSQ_MAX_TRIES', 2)\r\n    >>> for work in fsq.scan('a_queue'):\r\n    ...     fsq.fail_tmp(work)\r\n\r\nThe name of the work item will change to indicate that the item has failed\r\nonce::\r\n\r\n    /var/fsq/a_queue\r\n    \u251c\u2500\u2500 done\r\n    \u251c\u2500\u2500 fail\r\n    \u251c\u2500\u2500 queue\r\n    |   \u2514\u2500\u2500 _20131005205643_0_25577_mss_1_args_to_enqueue\r\n    \u2514\u2500\u2500 tmp\r\n\r\n\r\nTaking Queues Down\r\n------------------\r\n\r\nTo temporaily stop all scanning of any queue, you simply use the ``fsq down``\r\nprogram::\r\n\r\n    $ fsq down a_queue\r\n\r\nOr from the Python API::\r\n\r\n    >>> import fsq\r\n    >>> fsq.down('a_queue')\r\n\r\nWhich creates a regular file named ``down`` in the ``a_queue`` directory\r\npreventing scan from working on the queue::\r\n\r\n    /var/fsq/a_queue\r\n    \u251c\u2500\u2500 done\r\n    \u251c\u2500\u2500 down\r\n    \u251c\u2500\u2500 fail\r\n    \u251c\u2500\u2500 queue\r\n    |   \u2514\u2500\u2500 _20131005205643_0_25577_mss_0_args_to_enqueue\r\n    \u2514\u2500\u2500 tmp\r\n\r\nTo bring a queue back up again, you simply use the ``fsq up`` program::\r\n\r\n    $ fsq up a_queue\r\n\r\nOr from the Python API::\r\n\r\n    >>> import fsq\r\n    >>> fsq.up('a_queue')\r\n\r\nWhich removes the ``down`` file, and allows the queue to be scanned properly\r\nagain.\r\n\r\nThe tmp Directory\r\n-----------------\r\n\r\nThe tmp directory within a_queue is used by fsq under the hood to ensure that\r\nall items are enqueued atomically.\r\n\r\nThe fsq File Name\r\n-----------------\r\n\r\n::\r\n\r\n    _20120710213904_0_13044_mss_0_args_to_enqueue\r\n    _20120710213904_1_13044_mss_0_args_to_enqueue\r\n    +|-----+------| + |-+-| |+| + |------+------|\r\n    |      |        |   |    |  |        |\r\n    |      |        |   |    |  |        +-> FSQ_DELIMITER seperated\r\n    |      |        |   |    |  |            arguments\r\n    |      |        |   |    |  +-> tries: number of failed attempts\r\n    |      |        |   |    |      to process\r\n    |      |        |   |    +-> hostname: the name of the host on\r\n    |      |        |   |        which the work-item was enqueued.\r\n    |      |        |   +-> pid of the process which enqueued the\r\n    |      |        |       work-item\r\n    |      |        +-> entropy: should a work-item be generated\r\n    |      |            with the same arguments, pid, hostname\r\n    |      |            and timestamp, entropy is incremented to\r\n    |      |            generate uniqueness.\r\n    |      +-> timestamp in FSQ_TIMEFMT format\r\n    +-> FSQ_DELIMITER used at enqueue time\r\n\r\nENVIRONMENT\r\n===========\r\n\r\nThe fsq suite and python library makes use of a number of environment\r\nvariables (each prefixed by ``FSQ_``), which modify its behavior.  Each\r\nenvironment variable is also available as a package-level constant.\r\n\r\nPlease refer to ``man 7 fsq`` for a complete list.\r\n\r\nAUTHORS\r\n=======\r\n\r\n| Matthew Story <matt.story@axial.net>\r\n| Isaac (.ike) Levy <ike@blackskyresearch.net>\r\n| Will O'Meara <will.omeara@axial.net>\r\n| Jeff Rand <jeff.rand@axial.net>\r\n\r\nWith Additional Contributions From:\r\n-----------------------------------\r\n\r\n| Will Martino\r\n| Will Slippey\r\n| Jacob Yuan\r\n\r\nAnd Thanks To:\r\n--------------\r\n\r\n| William Baxter (For trigger, and for inspiring fsq)\r\n| Bruce Guenter (For nullmailer, featuring a simpler file-system queue)\r\n| Daniel J Bernstein (For QMail, inspiring trigger and nullmailer)",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/axialmarket/fsq",
        "keywords": "",
        "license": "3-BSD",
        "maintainer": "",
        "maintainer_email": "",
        "name": "fsq",
        "package_url": "https://pypi.org/project/fsq/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/fsq/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "https://github.com/axialmarket/fsq"
        },
        "release_url": "https://pypi.org/project/fsq/0.2.5/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "File System Queue",
        "version": "0.2.5"
    },
    "last_serial": 1054158,
    "releases": {
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3b6842cfd842cd8d6bfa7e0b93c2081e",
                    "sha256": "3ca7f48e5933079460b73d966bd5c3ec59e96e2c39e7960801cdbf388f1506d0"
                },
                "downloads": -1,
                "filename": "fsq-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "3b6842cfd842cd8d6bfa7e0b93c2081e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 45419,
                "upload_time": "2013-04-03T13:02:35",
                "url": "https://files.pythonhosted.org/packages/3c/72/358a776dde6f37b23bd8aac1de6a1924098796255e39ce4bb548ccabb241/fsq-0.2.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ac64907f55871c35bc2711c93d77d648",
                    "sha256": "d992b1a7df25b61ba69361e3d525715af5c8ef1811ea88a73333cded22727165"
                },
                "downloads": -1,
                "filename": "fsq-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "ac64907f55871c35bc2711c93d77d648",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 45427,
                "upload_time": "2013-04-03T18:03:53",
                "url": "https://files.pythonhosted.org/packages/06/f1/e04e218eff7277d4d5edee121b02b36516fd8afe7e6045deac1bac7b3a5f/fsq-0.2.1.tar.gz"
            }
        ],
        "0.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c04b7bb836fe17e2ce26988a4cb46117",
                    "sha256": "5e043383bb90eedf5d570906f9805dd955dcd6e269a7c61a6ef43cae9ad91edb"
                },
                "downloads": -1,
                "filename": "fsq-0.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "c04b7bb836fe17e2ce26988a4cb46117",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 85805,
                "upload_time": "2013-10-06T01:30:07",
                "url": "https://files.pythonhosted.org/packages/51/da/56da2ba20c04881d72919a61818307637f022e9f58af66edc9072d0e68eb/fsq-0.2.3.tar.gz"
            }
        ],
        "0.2.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6422127c064e5cbbd9278cf331afb0c0",
                    "sha256": "a17cd60e088d282678e8e740153e434f2a059e49b02cd4a918cf17f68fdab5da"
                },
                "downloads": -1,
                "filename": "fsq-0.2.5-py27-none-any.whl",
                "has_sig": false,
                "md5_digest": "6422127c064e5cbbd9278cf331afb0c0",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 96756,
                "upload_time": "2014-04-07T20:01:56",
                "url": "https://files.pythonhosted.org/packages/6a/da/543285e1977edbf1954eada9cda63b2645e8cb88137c53b212919e661e38/fsq-0.2.5-py27-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "4155733d089354d2afc87d9dd6f65f85",
                    "sha256": "bb6b17ba94b3ad429eb8e94c52939204f361ed73da6b7617d317400d0dde9dc9"
                },
                "downloads": -1,
                "filename": "fsq-0.2.5.tar.gz",
                "has_sig": false,
                "md5_digest": "4155733d089354d2afc87d9dd6f65f85",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 73056,
                "upload_time": "2014-04-07T20:01:53",
                "url": "https://files.pythonhosted.org/packages/a2/8d/883268754f95e966fb9d686f0adf6b432e751d1a14750afeb5f648d50b9d/fsq-0.2.5.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "6422127c064e5cbbd9278cf331afb0c0",
                "sha256": "a17cd60e088d282678e8e740153e434f2a059e49b02cd4a918cf17f68fdab5da"
            },
            "downloads": -1,
            "filename": "fsq-0.2.5-py27-none-any.whl",
            "has_sig": false,
            "md5_digest": "6422127c064e5cbbd9278cf331afb0c0",
            "packagetype": "bdist_wheel",
            "python_version": "2.7",
            "requires_python": null,
            "size": 96756,
            "upload_time": "2014-04-07T20:01:56",
            "url": "https://files.pythonhosted.org/packages/6a/da/543285e1977edbf1954eada9cda63b2645e8cb88137c53b212919e661e38/fsq-0.2.5-py27-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "4155733d089354d2afc87d9dd6f65f85",
                "sha256": "bb6b17ba94b3ad429eb8e94c52939204f361ed73da6b7617d317400d0dde9dc9"
            },
            "downloads": -1,
            "filename": "fsq-0.2.5.tar.gz",
            "has_sig": false,
            "md5_digest": "4155733d089354d2afc87d9dd6f65f85",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 73056,
            "upload_time": "2014-04-07T20:01:53",
            "url": "https://files.pythonhosted.org/packages/a2/8d/883268754f95e966fb9d686f0adf6b432e751d1a14750afeb5f648d50b9d/fsq-0.2.5.tar.gz"
        }
    ]
}