{ "info": { "author": "Chip Turner", "author_email": "chip@fb.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Environment :: Console", "Framework :: Setuptools Plugin", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.6" ], "description": "# XAR\n\n

\n\"CircleCI\n\"Code\n\"Downloads\"\n

\n\nXAR lets you package many files into a single self-contained executable file.\nThis makes it easy to distribute and install.\n\nA `.xar` file is a read-only file system image which, when mounted, looks like\na regular directory to user-space programs. This requires a one-time\ninstallation of a driver for this file system\n([SquashFS](https://en.wikipedia.org/wiki/SquashFS)).\n\nXAR is pronounced like \"czar\" (/t\u0361\u0282ar/). The 'X' in XAR is meant to be\na placeholder for all other letters as at Facebook this format was originally\ndesigned to replace ZIP-based PAR (Python archives), JSAR (JavaScript archives),\nLAR (Lua archives), and so on.\n\n\n## Use Cases\n\nThere are two primary use cases for XAR files. The first is simply collecting\na number of files for automatic, atomic mounting somewhere on the filesystem.\nUsing a XAR file vastly shrinks the on-disk size of the data it holds.\nCompressing to below 20% of the original size is not unheard of. This can save\nmultiple gigabytes per machine and reduce random disk IO. This is especially\nimportant on machines with flash storage.\n\nThe second use case is an extension of the first -- by making the XAR file\nexecutable and using the `xarexec` helper, a XAR becomes a self-contained\npackage of executable code and its data. A popular example is Python\napplication archives that include all Python source code files, as well as\nnative shared libraries, configuration files, other data.\n\nThis can replace virtualenvs and PEX files with a system that is faster, has\nless overhead, is more compatible, and achieves better compression.\nThe downside is that it requires a setuid helper to perform the mounting.\n\n\n## Advantages of XAR for Python usage\n\n* SquashFS looks like regular files on disk to Python. This lets it use regular\n imports which are better supported by CPython.\n\n* SquashFS looks like regular files to your application, too. You don't need to\n use `pkg_resources` or other tricks to access data files in your package.\n\n* SquashFS with Zstandard compression saves disk space, also compared to a ZIP\n file.\n\n* SquashFS doesn't require unpacking of `.so` files to a temporary location like\n ZIP files do.\n\n* SquashFS is faster to start up than unpacking a ZIP file. You only need to\n mount the file system once. Subsequent calls to your application will reuse\n the existing mount.\n\n* SquashFS only decompresses the pages that are used by the application, and\n decompressed pages are cached in the page cache.\n\n* SquashFS is read-only so the integrity of your application is guaranteed\n compared to using virtualenvs or unpacking to a temporary directory.\n\n## Benchmarks\n\nOptimizing performance (both space and execution time) was a key design goal for\nXARs. We ran benchmark tests with open source tools to compare PEX, XAR, and\nnative installs on the following metrics:\n\n* **Size:** file size, in bytes, of the executable\n* **Cold start time:** time taken when we have nothing mounted or extracted\n* **Hot start time:** time taken when we have extracted cache or mounted XAR squashfs\n\nThe PEXs are built with `python3 setup.py bdist_pex --bdist-all`, and the XARs\nare built with `python3 setup.py bdist_xar --xar-compression-algorithm=zstd`.\n\n| Console script | Size | Cold start time | Hot start time |\n|-----------------------|--------------------|-----------------|----------------|\n| django-admin (native) | 22851072 B | - | 0.220 s |\n| django-admin.pex | 8529089 B | 1.705 s | 0.772 s |\n| django-admin.xar | 5464064 B (-36%) | 0.141 s (-92%) | 0.122 s (-84%) |\n| black (native) | 1020928 B | - | 0.245 s |\n| black.pex | 677550 B | 0.737 s | 0.619 s |\n| black.xar | 307200 B (-55%) | 0.245 s (-67%) | 0.219 s (-65%) |\n| jupyter (native) | 64197120 B | - | 0.399 s |\n| jupyter.pex | 17315669 B | 2.152 s | 1.046 s |\n| jupyter.xar | 17530880 B (+1%) | 0.213 s (-90%) | 0.181 s (-83%) |\n\nThe results show that both file size (with [zstd compression]) and start times\nimprove with XARs. This is an improvement when shipping to large number of\nservers, especially with short-running executables, such as small data\ncollection scripts on web servers or interactive command line tools.\n\n[zstd compression]: https://code.fb.com/core-data/smaller-and-faster-data-compression-with-zstandard/\n\n## Requirements\nXAR requires:\n* Linux or macOS\n* Python >= **2.7.11** & >= **3.5**\n* [squashfs-tools](https://github.com/plougher/squashfs-tools) to build XARs\n* [squashfuse](https://github.com/vasi/squashfuse) >= 0.1.102 **with**\n `squashfuse_ll` to run XARs\n\n\n## Components of XAR\n\n### bdist_xar\n\nThis is a setuptools plugin that lets you package your Python application\nas a .xar file. It requires `squashfs-tools`. Install it from PyPI to get\nthe stable version:\n\n```\npip install xar\n```\n\n*or* you can install it from this repository:\n\n```\npython setup.py install\n```\n\nAfter installation go to your favorite Python project with a console script and\nrun:\n\n```\npython setup.py bdist_xar\n```\n\nThe setuptools extension `bdist_xar` has options to configure the XAR, most\nimportantly `--interpreter` sets the Python interpreter used. Run\n`python setup.py bdist_xar --help` for a full list of options.\n\n### xarexec_fuse\n\nThis is a binary written in C++ used to mount a SquashFS image.\nIt requires `squashfuse` installed. Note that the current `squashfuse` package\non Ubuntu doesn't include `squashfuse_ll`, so you will have to install from\n[source](https://github.com/vasi/squashfuse/releases).\n\nYou can build this part of the code with:\n\n```\nmkdir build && cd build && cmake .. && make && [sudo] make install\n```\n\n## Examples\n\n### bdist_xar\n\nSimply run:\n\n```\npython /path/to/black/setup.py bdist_xar [--xar-compression-algorithm=zstd]\n/path/to/black/dist/black.xar --help\n```\n\n### make_xar\n\nXAR provides a simple CLI to create XARs from Python executables or directories.\nWe can create a XAR from an existing Python executable zip file, like a PEX.\n\n```\nmake_xar --python black.pex --output black.xar\n```\n\nYou can also create a XAR from a directory, and tell XAR which executable to\nrun once it starts.\n\n```\n> mkdir myxar\n> echo -n \"#\\!/bin/sh\\nshift\\necho \\$@\" > myxar/echo\n> chmod +x myxar/echo\n> make_xar --raw myxar --raw-executable echo --output echo\n> ./echo hello world\nhello world\n```\n\n`xarexec_fuse` will execute the executable it is given using the XAR path as the\nfirst argument, and will forward the XARs arguments after.\n\n## Running the Circle CI tests locally\n\nFirst you need to install docker (and possible docker-machine), as it is how it\nruns the the code. Then you need to\n[install](https://circleci.com/docs/2.0/local-cli/) the `circleci` cli, and run\n\n circleci build\n\nIf you change `.circleci/config.yml` you should validate it before committing\n\n circleci config validate\n\n\n## Contributing\nSee the CONTRIBUTING file for how to help out.\n\n\n## License\nXAR is BSD-licensed.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/facebookincubator/xar", "keywords": "", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "xar", "package_url": "https://pypi.org/project/xar/", "platform": "", "project_url": "https://pypi.org/project/xar/", "project_urls": { "Homepage": "https://github.com/facebookincubator/xar" }, "release_url": "https://pypi.org/project/xar/19.4.22/", "requires_dist": [ "pip (>=10.0.1)", "setuptools (>=34.1)" ], "requires_python": "", "summary": "The XAR packaging toolchain.", "version": "19.4.22" }, "last_serial": 5344774, "releases": { "18.7.12": [ { "comment_text": "", "digests": { "md5": "bab7c5afcd2a269042357d28204149fb", "sha256": "2d34e8a2f572e2a89e3c837df38f45a3e2192bea9591c4a4e7073dcf39b7092f" }, "downloads": -1, "filename": "xar-18.7.12-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "bab7c5afcd2a269042357d28204149fb", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 39598, "upload_time": "2018-07-12T21:19:52", "url": "https://files.pythonhosted.org/packages/f0/6f/81b27ad41bb4c73ffde800edb2e3d99567ea72024a72874de6a8f64467f0/xar-18.7.12-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9f749a82cbe0fb797874842cd4348cf9", "sha256": "5e1d962429f017ccba765aefce59657712fe9f245ff3ff9a345dffe3b75de06b" }, "downloads": -1, "filename": "xar-18.7.12.tar.gz", "has_sig": false, "md5_digest": "9f749a82cbe0fb797874842cd4348cf9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 37057, "upload_time": "2018-07-12T21:20:00", "url": "https://files.pythonhosted.org/packages/8e/2e/ecdc71e21f3a572210201a9d69455e6f8610f4d9621140e06a44257400d9/xar-18.7.12.tar.gz" } ], "19.4.22": [ { "comment_text": "", "digests": { "md5": "c4ec8c3766c67ecd51ad2d2451b0cf57", "sha256": "8e103e5b7dafb82d45d7a9c5d5022306d7c96951fb90a4d8691ca447fbf28564" }, "downloads": -1, "filename": "xar-19.4.22-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "c4ec8c3766c67ecd51ad2d2451b0cf57", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 74252, "upload_time": "2019-04-22T18:04:57", "url": "https://files.pythonhosted.org/packages/bf/15/07113c26d46f8a529d2477fd6644145aca7859b9c1475b17f9528d6b46bf/xar-19.4.22-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4a7dadf4ad3cd642226b5972d238c69c", "sha256": "0aa99d7226cfb32e97bd64b1d694796c43f5597bf07ba6ad19f04c1c2ae41895" }, "downloads": -1, "filename": "xar-19.4.22.tar.gz", "has_sig": false, "md5_digest": "4a7dadf4ad3cd642226b5972d238c69c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 63743, "upload_time": "2019-04-22T18:05:06", "url": "https://files.pythonhosted.org/packages/88/7d/3dc3474922889922c8f7eb399e6750e627d26b05316201633ffd505e4a70/xar-19.4.22.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "c4ec8c3766c67ecd51ad2d2451b0cf57", "sha256": "8e103e5b7dafb82d45d7a9c5d5022306d7c96951fb90a4d8691ca447fbf28564" }, "downloads": -1, "filename": "xar-19.4.22-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "c4ec8c3766c67ecd51ad2d2451b0cf57", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 74252, "upload_time": "2019-04-22T18:04:57", "url": "https://files.pythonhosted.org/packages/bf/15/07113c26d46f8a529d2477fd6644145aca7859b9c1475b17f9528d6b46bf/xar-19.4.22-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4a7dadf4ad3cd642226b5972d238c69c", "sha256": "0aa99d7226cfb32e97bd64b1d694796c43f5597bf07ba6ad19f04c1c2ae41895" }, "downloads": -1, "filename": "xar-19.4.22.tar.gz", "has_sig": false, "md5_digest": "4a7dadf4ad3cd642226b5972d238c69c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 63743, "upload_time": "2019-04-22T18:05:06", "url": "https://files.pythonhosted.org/packages/88/7d/3dc3474922889922c8f7eb399e6750e627d26b05316201633ffd505e4a70/xar-19.4.22.tar.gz" } ] }