{ "info": { "author": "Giampaolo Rodola", "author_email": "g.rodola@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: MacOS :: MacOS X", "Operating System :: POSIX :: AIX", "Operating System :: POSIX :: BSD", "Operating System :: POSIX :: BSD :: FreeBSD", "Operating System :: POSIX :: Linux", "Operating System :: POSIX :: SunOS/Solaris", "Programming Language :: C", "Programming Language :: Python :: 2.5", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.0", "Programming Language :: Python :: 3.1", "Programming Language :: Python :: 3.2", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Topic :: Internet :: File Transfer Protocol (FTP)", "Topic :: Internet :: WWW/HTTP", "Topic :: System :: Networking", "Topic :: System :: Operating System" ], "description": ".. image:: https://pypip.in/d/pysendfile/badge.png\n :target: https://crate.io/packages/pysendfile/\n :alt: Download this month\n\n.. image:: https://pypip.in/v/pysendfile/badge.png\n :target: https://pypi.python.org/pypi/pysendfile/\n :alt: Latest version\n\n.. image:: https://pypip.in/license/pysendfile/badge.png\n :target: https://pypi.python.org/pypi/pysendfile/\n :alt: License\n\n.. image:: https://api.travis-ci.org/giampaolo/pysendfile.png?branch=master\n :target: https://travis-ci.org/giampaolo/pysendfile\n :alt: Travis\n\n===========\nQuick links\n===========\n\n- `Home page `_\n- `Mailing list `_\n- `Blog `_\n- `What's new `_\n\n=====\nAbout\n=====\n\n`sendfile(2) `__ is a system call which\nprovides a \"zero-copy\" way of copying data from one file descriptor to another\n(a socket). The phrase \"zero-copy\" refers to the fact that all of the copying\nof data between the two descriptors is done entirely by the kernel, with no\ncopying of data into userspace buffers. This is particularly useful when\nsending a file over a socket (e.g. FTP).\nThe normal way of sending a file over a socket involves reading data from the\nfile into a userspace buffer, then write that buffer to the socket via\n`send() `__ or\n`sendall() `__:\n\n.. code-block:: python\n\n # how a file is tipically sent\n\n import socket\n\n file = open(\"somefile\", \"rb\")\n sock = socket.socket()\n sock.connect((\"127.0.0.1\", 8021))\n\n while True:\n chunk = file.read(65536)\n if not chunk:\n break # EOF\n sock.sendall(chunk)\n\nThis copying of the data twice (once into the userland buffer, and once out\nfrom that userland buffer) imposes some performance and resource penalties.\n`sendfile(2) `__ syscall avoids these\npenalties by avoiding any use of userland buffers; it also results in a single\nsystem call (and thus only one context switch), rather than the series of\n`read(2) `__ /\n`write(2) `__ system calls (each system call\nrequiring a context switch) used internally for the data copying.\n\n.. code-block:: python\n\n import socket\n from sendfile import sendfile\n\n file = open(\"somefile\", \"rb\")\n blocksize = os.path.getsize(\"somefile\")\n sock = socket.socket()\n sock.connect((\"127.0.0.1\", 8021))\n offset = 0\n\n while True:\n sent = sendfile(sock.fileno(), file.fileno(), offset, blocksize)\n if sent == 0:\n break # EOF\n offset += sent\n\n==================\nA simple benchmark\n==================\n\nThis `benchmark script `__\nimplements the two examples above and compares plain socket.send() and\nsendfile() performances in terms of CPU time spent and bytes transmitted per\nsecond resulting in sendfile() being about **2.5x faster**. These are the\nresults I get on my Linux 2.6.38 box, AMD dual-core 1.6 GHz:\n\n*send()*\n\n+---------------+-----------------+\n| CPU time | 28.84 usec/pass |\n+---------------+-----------------+\n| transfer rate | 359.38 MB/sec |\n+---------------+-----------------+\n\n*sendfile()*\n\n+---------------+-----------------+\n| CPU time | 11.28 usec/pass |\n+---------------+-----------------+\n| transfer rate | 860.88 MB/sec |\n+---------------+-----------------+\n\n===========================\nWhen do you want to use it?\n===========================\n\nBasically any application sending files over the network can take advantage of\nsendfile(2). HTTP and FTP servers are a typical example.\n`proftpd `__ and\n`vsftpd `__ are known to use it, so is\n`pyftpdlib `__.\n\n=================\nAPI documentation\n=================\n\nsendfile module provides a single function: sendfile().\n\n- ``sendfile.sendfile(out, in, offset, nbytes, header=\"\", trailer=\"\", flags=0)``\n\n Copy *nbytes* bytes from file descriptor *in* (a regular file) to file\n descriptor *out* (a socket) starting at *offset*. Return the number of\n bytes just being sent. When the end of file is reached return 0.\n On Linux, if *offset* is given as *None*, the bytes are read from the current\n position of *in* and the position of *in* is updated.\n *headers* and *trailers* are strings that are written before and after the\n data from *in* is written. In cross platform applications their usage is\n discouraged\n (`send() `__ or\n `sendall() `__\n can be used instead). On Solaris, _out_ may be the file descriptor of a\n regular file or the file descriptor of a socket. On all other platforms,\n *out* must be the file descriptor of an open socket.\n *flags* argument is only supported on FreeBSD.\n\n- ``sendfile.SF_NODISKIO``\n- ``sendfile.SF_MNOWAIT``\n- ``sendfile.SF_SYNC``\n\n Parameters for the _flags_ argument, if the implementation supports it. They\n are available on FreeBSD platforms. See `FreeBSD's man sendfile(2) `__.\n\n=======================\nDifferences with send()\n=======================\n\n- sendfile(2) works with regular (mmap-like) files only (e.g. you can't use it\n with a `StringIO `__ object).\n- Also, it must be clear that the file can only be sent \"as is\" (e.g. you\n can't modify the content while transmitting).\n There might be problems with non regular filesystems such as NFS,\n SMBFS/Samba and CIFS. For this please refer to\n `proftpd documentation `__.\n- `OSError `__\n is raised instead of `socket.error `__.\n The accompaining `error codes `__\n have the same meaning though: EAGAIN, EWOULDBLOCK, EBUSY meaning you are\n supposed to retry, ECONNRESET, ENOTCONN, ESHUTDOWN, ECONNABORTED in case of\n disconnection. Some examples:\n `benchmark script `__,\n `test suite `__,\n `pyftpdlib wrapper `__.\n\n===================\nSupported platforms\n===================\n\nThis module works with Python versions from **2.5** to **3.4**. The supported platforms are:\n\n- **Linux**\n- **Mac OSX**\n- **FreeBSD**\n- **Dragon Fly BSD**\n- **Sun OS**\n- **AIX** (not properly tested)\n\n=======\nSupport\n=======\n\nFeel free to mail me at *g.rodola [AT] gmail [DOT] com* or post on the the\nmailing list: http://groups.google.com/group/py-sendfile.\n\n======\nStatus\n======\n\nAs of now the code includes a solid `test suite `__ and its ready for production use.\nIt's been included in `pyftpdlib `__\nproject and used in production environments for almost a year now without any\nproblem being reported so far.\n\n=======\nAuthors\n=======\n\npysendfile was originally written by *Ben Woolley* including Linux, FreeBSD and\nDragonFly BSD support. Later on *Niklas Edmundsson* took over maintenance and\nadded AIX support. After a couple of years of project stagnation\n`Giampaolo Rodola' `__ took over\nmaintenance and rewrote it from scratch adding support for:\n\n- Python 3\n- non-blocking sockets\n- `large file `__ support\n- Mac OSX\n- Sun OS\n- FreeBSD flag argument\n- multiple threads (release GIL)\n- a simple benchmark suite\n- unit tests\n- documentation", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/giampaolo/pysendfile", "keywords": "sendfile,python,performance,ftp", "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "pysendfile", "package_url": "https://pypi.org/project/pysendfile/", "platform": "UNIX", "project_url": "https://pypi.org/project/pysendfile/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/giampaolo/pysendfile" }, "release_url": "https://pypi.org/project/pysendfile/2.0.1/", "requires_dist": null, "requires_python": null, "summary": "A Python interface to sendfile(2)", "version": "2.0.1" }, "last_serial": 1105484, "releases": { "0.2.0": [], "2.0.0": [ { "comment_text": "", "digests": { "md5": "bd8d96b64e7c5c5b91ea2a6659aa82a3", "sha256": "b7b747e47f2b1b16942d8db9139e0f0f70546ca80a06460bd5abfdc7a1bca957" }, "downloads": -1, "filename": "pysendfile-2.0.0.tar.gz", "has_sig": false, "md5_digest": "bd8d96b64e7c5c5b91ea2a6659aa82a3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11155, "upload_time": "2014-04-30T21:51:04", "url": "https://files.pythonhosted.org/packages/16/f1/7a275a36a0ab8ddc6851c8b71b119cfe67aa8ea493c1480d86a58989965f/pysendfile-2.0.0.tar.gz" } ], "2.0.1": [ { "comment_text": "", "digests": { "md5": "e7b301eddd703ab74a48c59a8fda1f97", "sha256": "510a414b270986fba3c79cb76d90a4c910c701bfb43ff983a5d4e92846050e17" }, "downloads": -1, "filename": "pysendfile-2.0.1.tar.gz", "has_sig": false, "md5_digest": "e7b301eddd703ab74a48c59a8fda1f97", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19315, "upload_time": "2014-05-27T11:07:29", "url": "https://files.pythonhosted.org/packages/cd/3f/4aa268afd0252f06b3b487c296a066a01ddd4222a46b7a3748599c8fc8c3/pysendfile-2.0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "e7b301eddd703ab74a48c59a8fda1f97", "sha256": "510a414b270986fba3c79cb76d90a4c910c701bfb43ff983a5d4e92846050e17" }, "downloads": -1, "filename": "pysendfile-2.0.1.tar.gz", "has_sig": false, "md5_digest": "e7b301eddd703ab74a48c59a8fda1f97", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19315, "upload_time": "2014-05-27T11:07:29", "url": "https://files.pythonhosted.org/packages/cd/3f/4aa268afd0252f06b3b487c296a066a01ddd4222a46b7a3748599c8fc8c3/pysendfile-2.0.1.tar.gz" } ] }