{
    "info": {
        "author": "sganis",
        "author_email": "sganis@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python :: 2.7"
        ],
        "description": "**********\nStatwalker\n**********\n\nWalk file systems and collect stats.\n\n.. contents:: \n\nSummary\n=======\n\nStatwalker is a command-line program that scan files recursively (normally called as \"walk\") and collects stats, basically file names and metadata (inode information in Linux systems). It runs in parallel in a single machine, and the output is a comma-separated file (csv), one line per file. These results can be analysed using other tools (see below).\n\nOutput\n======\n\nThe csv file will look like this:\n\n.. code::\n\t\n\tINODE,ATIME,MTIME,UID,GID,MODE,SIZE,DISK,PATH\n\nColum description:\n\n\t1. INODE: device identifier and inode (Linux)\n\t2. ATIME: last access time in unix format (seconds since epoc)\n\t3. MTIME: last modified time in unix fromat\n\t4. UID: user ID\n\t5. GID: group ID\n\t6. MODE: mode, which is file type and permissions\n\t7. SIZE: real size in bytes, same value reported with command du -b\n\t8. DISK: disk usage, which is number of blocks times 512\n\t9. PATH: full path\n\nHow it works\n============\n\nCollecting stats is as simple as this one-liner in bash:\n\n.. code:: bash\n\n\t$ TODO\n\nThere are many tools doing the same thing, the problem is performance. After trying some tools in a file system with many terabytes of data and millions of files, the problem became untractable. I run statwalker in a storage with 100+ millions of files, with a reading rate over NFS folders of 3000 files/second on average, and much faster if disks are local.\n\n\nInstallation\n============\n\nUse pip:\n\n.. code:: bash\n\n    $ pip install statwalker\n\n\nUsage\n=====\n\n.. code::\n\t\n\t# run it from the command line to see available parameters:\n\t$ statwalker -h\n\n\t# run it with options\n\t$ statwalker -o output.csv /home\n\n\nContribute\n==========\n\nClone the github repository:\n\n.. code:: bash\n\n    $ git clone https://github.com/sganis/statwalker.git\n\n\nTODO\n====\n\n- Add documentation with analysis tools, resolution, agregation, benchmark with c++ and mpi versions.\n- Add notes for windows users",
        "description_content_type": null,
        "docs_url": null,
        "download_url": null,
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/sganis/statwalker",
        "keywords": "statwalker walk filesytem stats recursively",
        "license": "MIT",
        "maintainer": null,
        "maintainer_email": null,
        "name": "statwalker",
        "package_url": "https://pypi.org/project/statwalker/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/statwalker/",
        "project_urls": {
            "Homepage": "https://github.com/sganis/statwalker"
        },
        "release_url": "https://pypi.org/project/statwalker/1.1.0/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Walk file systems and collect stats",
        "version": "1.1.0"
    },
    "last_serial": 1542160,
    "releases": {
        "1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "61c17f98830a18993bd380b74c2f317f",
                    "sha256": "ee52b6431a4eb468419362a583ed0f58696ed182d90ecbde79c269f23b87f1e0"
                },
                "downloads": -1,
                "filename": "statwalker-1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "61c17f98830a18993bd380b74c2f317f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7400,
                "upload_time": "2015-05-09T08:43:56",
                "url": "https://files.pythonhosted.org/packages/71/6c/d0a037401dc7221a0b26719656635dddb33313db541ce11ed3eea673ad89/statwalker-1.0.tar.gz"
            }
        ],
        "1.0.0": [],
        "1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "739b63bbb7987a1999b00685e30c58c2",
                    "sha256": "35b4bea6b00adb5b3eee7990fbe184c7b1f5241aa554d99c1a9820e4c041df0f"
                },
                "downloads": -1,
                "filename": "statwalker-1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "739b63bbb7987a1999b00685e30c58c2",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8353,
                "upload_time": "2015-05-11T15:41:16",
                "url": "https://files.pythonhosted.org/packages/80/e2/47b04225bf365e3781ed42af54bf17a591dfc08370b304d77671e3571192/statwalker-1.1.tar.gz"
            }
        ],
        "1.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3ab1516c02b2c8813a79724717512e59",
                    "sha256": "3ecf4184492cc10d5f3df40b0fcf52fffb64d4dd2084c7a32fa4be78bbc0f102"
                },
                "downloads": -1,
                "filename": "statwalker-1.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "3ab1516c02b2c8813a79724717512e59",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8401,
                "upload_time": "2015-05-11T15:45:36",
                "url": "https://files.pythonhosted.org/packages/4a/0b/9a4af086be8f47e9650ab5adcf2a75e6576484de7a7acd729d6fca43c5e6/statwalker-1.1.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "3ab1516c02b2c8813a79724717512e59",
                "sha256": "3ecf4184492cc10d5f3df40b0fcf52fffb64d4dd2084c7a32fa4be78bbc0f102"
            },
            "downloads": -1,
            "filename": "statwalker-1.1.0.tar.gz",
            "has_sig": false,
            "md5_digest": "3ab1516c02b2c8813a79724717512e59",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 8401,
            "upload_time": "2015-05-11T15:45:36",
            "url": "https://files.pythonhosted.org/packages/4a/0b/9a4af086be8f47e9650ab5adcf2a75e6576484de7a7acd729d6fca43c5e6/statwalker-1.1.0.tar.gz"
        }
    ]
}