{ "info": { "author": "Ximin Luo, Ceridwen", "author_email": "infinity0@debian.org, ceridwenv@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Topic :: Utilities" ], "description": "reprotest\n=========\n\nreprotest builds the same source code twice in different environments, and then\nchecks the binaries produced by each build for differences. If any are found,\nthen ``diffoscope(1)`` (or if unavailable then ``diff(1)``) is used to display\nthem in detail for later analysis.\n\nSee the ``COMMAND-LINE EXAMPLES`` section further below to get you\nstarted, as well as more detailed explanations of all the command-line\noptions. The same information is also available in\n``/usr/share/doc/reprotest/README.rst`` or similar.\n\n.. raw:: manpage\n\n .\\\" the below hack gets rid of the python \"usage\" message in favour of the\n .\\\" the synopsis we manually defined in doc/$(PACKAGE).h2m.0\n .SS positional arguments:\n .\\\" end_of_description_header\n\nCommand-line examples\n=====================\n\nThe easiest way to run reprotest is via our presets::\n\n # Build the current directory in a null server (/tmp)\n $ reprotest .\n $ reprotest . -vv -- null -d # for very verbose output\n\n # Build the given Debian source package in an schroot\n # See https://wiki.debian.org/sbuild for instructions on setting that up.\n $ reprotest reprotest_0.3.3.dsc -- schroot unstable-amd64-sbuild\n\nCurrently, we only support this for Debian packages, but are keen on\nadding more. If we don't have knowledge on how to build your file or\ndirectory, you can send a patch to us on adding this intelligence - see\nthe reprotest.presets python module, and adapt the existing logic.\n\nIn the meantime, you can use other parts of the CLI to build arbitrary things.\nYou'll need to give two mandatory arguments, the build command to run and the\nbuild artifact file/pattern to test after running the build. For example::\n\n $ reprotest 'python3 setup.py bdist' 'dist/*.tar.gz'\n\nThis runs the command on ``.``, the current working directory. To run it on a\nproject located elsewhere::\n\n $ reprotest -s ../path/to/other/project 'python3 setup.py bdist' 'dist/*.tar.gz'\n $ reprotest -c 'python3 setup.py bdist' ../path/to/other/project 'dist/*.tar.gz'\n\nThese two invocations are equivalent; you can pick the most convenient one\nfor your use-case. When using these from a shell:\n\n * If the build command has spaces, you will need to quote them, e.g.\n ``reprotest \"dpkg-buildpackage -b --no-sign\" [..]``.\n\n * If you want to use several build artifact patterns, or if you want to\n use shell wildcards as a pattern, you will also need to quote them, e.g.\n ``reprotest [..] \"*.tar.gz *.tar.xz\"``.\n\n * If your build artifacts have spaces in their names, you will need to\n quote these twice, e.g. ``'\"a file with spaces.gz\"'`` for a single\n artifact or ``'\"dir 1\"/* \"dir 2\"/*'`` for multiple patterns.\n\nTo get more help for the CLI, including documentation on optional\narguments and what they do, run::\n\n $ reprotest --help\n\n\nRunning in a virtual server\n===========================\n\nYou can also run the build inside what is called a \"virtual server\".\nThis could be a container, a chroot, etc. You run them like this::\n\n $ reprotest 'python3 setup.py bdist_wheel' 'dist/*.whl' -- qemu /path/to/qemu.img\n $ reprotest 'dpkg-buildpackage -b --no-sign' '../*.deb' -- schroot unstable-amd64\n\nThere are different server types available. See ``--help`` for a list of\nthem, which appears near the top, in the \"virtual\\_server\\_args\" part of\nthe \"positional arguments\" section.\n\nFor each virtual server (e.g. \"schroot\"), you see which extra arguments\nit supports::\n\n $ reprotest --help schroot\n\nWhen running builds inside a virtual server, you will probably have to\ngive extra commands, in order to set up your build dependencies inside\nthe virtual server. For example, to take you through what the \"Debian\ndirectory\" preset would look like, if we ran it using the full CLI::\n\n # \"Debian directory\" preset\n $ reprotest . -- schroot unstable-amd64-sbuild\n # This is exactly equivalent to this:\n $ reprotest -c auto . -- schroot unstable-amd64-sbuild\n # In the non-preset full CLI, this is roughly similar to:\n $ reprotest \\\n --testbed-init 'apt-get -y --no-install-recommends install \\\n disorderfs faketime locales-all sudo util-linux; \\\n test -c /dev/fuse || mknod -m 666 /dev/fuse c 10 229; \\\n test -f /etc/mtab || ln -s ../proc/self/mounts /etc/mtab' \\\n --testbed-build-pre 'apt-get -y --no-install-recommends build-dep ./' \\\n --build-command 'dpkg-buildpackage --no-sign -b' \\\n . \\\n '../*.deb' \\\n -- \\\n schroot unstable-amd64-sbuild\n\nThe ``--testbed-init`` argument is needed to set up basic tools, which\nreprotest needs in order to make the variations in the first place. This\nshould be the same regardless of what package is being built, but might\ndiffer depending on what virtual\\_server is being used.\n\nNext, we have ``--testbed-build-pre``, then ``--build-command`` (or ``-c``).\nFor our Debian directory, we install build-dependencies using ``apt-get``,\nthen we run the actual build command itself using ``dpkg-buildpackage(1)``.\n\nThen, we have the ``source_root`` and the ``artifact_pattern``. For\nreproducibility, we're only interested in the binary packages.\n\nFinally, we specify that this is to take place in the \"schroot\"\nvirtual\\_server with arguments \"unstable-amd64-sbuild\".\n\nOf course, all of this is a burden to remember, if you must run the same\nthing many times. So that is why adding new presets for new package types\nwould be good.\n\nHere is a more complex example. It tells reprotest to store the build products\ninto ``./artifacts`` to analyse later; and also tweaks the \"Debian dsc\" preset\nso that it uses our `experimental toolchain\n`__::\n\n $ reprotest --store-dir=artifacts \\\n --auto-preset-expr '_.prepend.testbed_init(\"apt-get install -y wget; \\\n echo deb http://reproducible.alioth.debian.org/debian/ ./ >> /etc/apt/sources.list; \\\n wget -q -O- https://reproducible.alioth.debian.org/reproducible.asc | apt-key add -; \\\n apt-get update; apt-get upgrade -y; \")' \\\n ./bash_4.4-4.0~reproducible1.dsc \\\n -- \\\n schroot unstable-amd64-sbuild\n\nAlternatively, you can clone your unstable-amd64-sbuild chroot, add our repo to\nthe cloned chroot, then use this chroot in place of \"unstable-amd64-sbuild\".\nThat would allow you to omit the long ``--auto-preset-expr`` flag above.\n\n\nConfig File\n===========\n\nYou can also give options to reprotest via a config file. This is a\ntime-saving measure similar to ``auto`` presets; the difference is that\nthese are more suited for local builds that are suited to your personal\npurposes. (You may use both presets and config files in the same build.)\n\nThe config file takes exactly the same options as the command-line interface,\nbut with the additional restriction that the section name must match the ones\ngiven in the --help output. Whitespace is allowed if and only if the same\ncommand-line option allows whitespace. Finally, it is not possible to give\npositional arguments via this mechanism.\n\nReprotest by default does not load any config file. You can tell it to load one\nwith the ``--config-file`` or ``-f`` command line options. If you give it a\ndirectory such as ``.``, it will load ``.reprotestrc`` within that directory.\n\nA sample config file is below::\n\n [basics]\n verbosity = 1\n variations =\n environment\n build_path\n user_group.available+=builduser:builduser\n fileordering\n home\n kernel\n locales\n exec_path\n time\n timezone\n umask\n store_dir =\n /home/foo/build/reprotest-artifacts\n\n [diff]\n diffoscope_arg =\n --debug\n\n\nAnalysing diff output\n=====================\n\nNormally when diffoscope compares directories, it also compares the metadata of\nfiles in those directories - file permissions, owners, and so on.\n\nHowever depending on the circumstance, this filesystem-level metadata may or\nmay not be intended to be distributed to other systems. For example: (1) for\nmost distros' package builders, we don't care about the metadata of the output\npackage files; only the file contents will be distributed to other systems. On\nthe other hand, (2) when running something like `make install`, we *do* care\nabout the metadata, because this is what will be recreated on another system.\n\nIn developing reprotest, our experience has been that case (1) is more common\nand so we pass ``--exclude-directory-metadata`` by default to diffoscope. If\nyou find that you are using reprotest for case (2) then you should pass\n``--diffoscope-args=--no-exclude-directory-metadata`` to reprotest, to tell\ndiffoscope to not ignore the metadata since it will be distributed and should\ntherefore be reproducible. Otherwise, you may get a false-positive result.\n\n\nVariations\n==========\n\nThe --vary and --variations flags in their simple forms, are a comma-separated\nlist of variation names that indicate which variations to apply. The full list\nof names is given in the --help text for --variations.\n\n| \\\n| In full detail, the flags are a comma-separated list of actions, as follows:\n|\n| +$variation (or $variation with no explicit operator)\n| -$variation\n| Enable or disable a variation\n|\n| @$variation\n| Enable a variation, resetting its parameters (see below) to default values.\n|\n| $variation.$param=$value\n| $variation.$param+=$value\n| $variation.$param-=$value\n| Set/add/remove $value as/to/from the current value of the $param parameter\n of the $variation.\n|\n| $variation.$param++\n| $variation.$param--\n| Increment/decrement the value of the $param parameter of the $variation.\n\nMost variations do not have parameters, and for them only the + and - operators\nare relevant. The variations that accept parameters are:\n\ndomain_host.use_sudo\n An integer, whether to use sudo(1) together with unshare(1) to change the\n system hostname and domainname. 0 means don't use sudo; any non-zero value\n means to use sudo. Default is 0, however this is not recommended and make\n may your build fail, see \"Varying the domain and host names\" for details.\nenvironment.variables\n A semicolon-separated ordered set, specifying environment variables that\n reprotest should try to vary. Default is \"REPROTEST_CAPTURE_ENVIRONMENT\".\n Supports regex-based syntax e.g.\n\n - PID=\\\\d{1,6}\n - HOME=(/\\\\w{3,12}){1,4}\n - (GO|PYTHON|)PATH=(/\\\\w{3,12}){1,4}(:(/\\\\w{3,12}){1,4}){0,4}\n\n Special cases:\n\n - $VARNAME= (empty RHS) to tell reprotest to delete the variable\n - $VARNAME=.{0} to tell reprotest to actually set an empty value\n - \\\\x2c and \\\\x3b to match or generate , and ; respectively.\nuser_group.available\n A semicolon-separated ordered set, specifying the available user+group\n combinations that reprotest can ``sudo(1)`` to. Default is empty, in which\n case the variation is a no-op, and you'll see a warning about this. Each\n user+group should be given in the form $user:$group where either component\n can be omitted, or else if there is no colon then it is interpreted as only\n a $user, with no $group variation.\ntime.faketimes\n A semicolon-separated ordered set, specifying possible ``faketime(1)`` time\n descriptors to use. Default is empty, in which case we randomly choose a\n time: either now (if the latest file-modtime in ``source_root`` is older\n than about half-a-year) or more than half-a-year in the future.\n\n Note that the clock continues to run during the build. It is possible for\n ``faketime(1)`` to freeze it, but we don't yet support that yet; it has a\n higher chance of causing your build to fail or misbehave.\n\nThe difference between --vary and --variations is that the former appends onto\nprevious values but the latter resets them. Furthermore, the last value set for\n--variations is treated as the zeroth --vary argument. For example::\n\n reprotest --vary=-user_group\n\nmeans to vary +all (the default value for --variations) and -user_group (the\ngiven value for --vary), whereas::\n\n reprotest --variations=-all,locales --variations=home,time --vary=timezone --vary=-time\n\nmeans to vary home, time (the last given value for --variations), timezone, and\n-time (the given multiple values for --vary), i.e. home and timezone.\n\n\nNotes on variations\n===================\n\nreprotest tries hard to perform variations without assuming it has full root\naccess to the system. It also assumes other software may be running on the same\nsystem, so it does not perform system-level modifications that would affect\nother processes. Due to these assumptions, some variations are implemented\nusing hacks at various levels of dirtiness, which are documented below.\n\nWe will hopefully lift these assumptions for certain virtual_server contexts,\nin future. That would likely allow for smoother operation in those contexts.\nThe assumptions will remain for the \"null\" (default) virtual_server however.\n\nNumber of CPUs\n--------------\n\nThe control build uses only 1 CPU in order to try to reduce nondeterminism that\nmight exist due to multithreading or multiprocessing. If you are sure your\nbuild is not affected by this (and good builds ought not to be), you can give\n--min-cpus=99999 to use all available cores for both builds.\n\nDomain or host\n--------------\n\nDoing this without sudo *may* result in your build failing.\n\nFailure is likely if your build must do system-related things - as opposed to\nonly processing bits and bytes. This is because it runs in a separate namespace\nwhere your non-privileged user looks like it is \"root\", but this prevents the\nfilesystem from recognising files owned by the real \"root\" user, amongst other\nthings. This is a limitation of unshare(1) and it is not possible work around\nthis in reprotest without heavy effort.\n\nTherefore, it is recommended to run this variation with use_sudo=1. To avoid\npassword prompts, see the section \"Avoid sudo(1) password prompts\" below.\n\nWhen running inside a virtual-server:\n\nThe non-sudo method fails with \"Operation not permitted\", even if you edited\n``/proc/sys/kernel/unprivileged_userns_clone``. The cause is currently unknown.\n\nThe sudo method works only if you take measures to avoid sudo password prompts,\nsince containers don't have a method to input this.\n\nUser or group\n-------------\n\nIf you also vary fileordering at the same time (this is the case by default),\neach user you use needs to be in the \"fuse\" group. Do that by running `usermod\n-aG fuse $OTHERUSER` as root.\n\nTo avoid sudo(1) password prompts, see the section \"Avoid sudo(1) password\nprompts\" below.\n\nTime\n----\n\nThe \"time\" variation uses ``faketime(1)`` which *sometimes* causes weird and\nhard-to-diagnose problems. In the past, this has included:\n\n- builds taking an infinite amount of time; though this should be fixed in\n recent versions of reprotest.\n\n- builds with implausibly huge differences caused by ./configure scripts\n producing different results with and without faketime. This still affects\n bash and probably certain other packages using autotools.\n\n- builds accessing the network failing due to certificate expiration errors\n and/or other time-related security errors. (Transparent builds of FOSS should\n not access the network in the first place, but it's outside of reprotest's\n scope to audit or prevent this.)\n\nIf you see a difference that you really think should not be there, try passing\n``--variations=-time`` to reprotest, and/or check our results on\nhttps://tests.reproducible-builds.org/ which use a different (more reliable)\nmechanism to vary the system time.\n\n\nAvoid sudo(1) password prompts\n==============================\n\nThere is currently no good way to do this. The following is an EXPERIMENTAL\nsolution and is brittle and unclean. You will have to decide for yourself if\nit's worth it for your use-case::\n\n $ reprotest --print-sudoers \\\n --variations=user_group.available+=guest-builder,domain_host.use_sudo=1 \\\n | sudo EDITOR=tee visudo -f /etc/sudoers.d/local-reprotest\n\nMake sure you set the variations you actually want to use. Obviously, don't\npick privileged users for this purpose, such as root.\n\n(Simplifying the output using wildcards, would open up passwordless access to\nchown anything on your system, because wildcards here match whitespace. I don't\nknow what the sudo authors were thinking.)\n\nNo, this is not nice at all - suggestions and patches welcome.\n\nIf you want to use this in a virtual server such as a chroot, you'll need to\ncopy (or mount or otherwise map) the resulting sudoers file into your chroot.\n\nFor example, for an schroot, you should (1) login to the source schroot and\ncreate an empty file `/etc/sudoers.d/local-reprotest` (this is important) and\nthen (2) add the line:\n\n /etc/sudoers.d/local-reprotest /etc/sudoers.d/local-reprotest none bind 0 0\n\nto your schroot's fstab.\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://salsa.debian.org/reproducible-builds/reprotest", "keywords": "", "license": "GPL-3+", "maintainer": "", "maintainer_email": "", "name": "reprotest", "package_url": "https://pypi.org/project/reprotest/", "platform": "", "project_url": "https://pypi.org/project/reprotest/", "project_urls": { "Homepage": "https://salsa.debian.org/reproducible-builds/reprotest" }, "release_url": "https://pypi.org/project/reprotest/0.7.8/", "requires_dist": null, "requires_python": "", "summary": "Build packages and check them for reproducibility.", "version": "0.7.8" }, "last_serial": 4043645, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "d8e185a391f94edc5009a0bbdc8ee28b", "sha256": "f5982b3746a287bedb7b15c24efc29459c8a1b2df42ba33980a56b5cfbbb3115" }, "downloads": -1, "filename": "reprotest-0.1.tar.gz", "has_sig": false, "md5_digest": "d8e185a391f94edc5009a0bbdc8ee28b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5609, "upload_time": "2016-06-14T16:41:31", "url": "https://files.pythonhosted.org/packages/d5/91/4b7f2dc133d97c982d13b042c0ed012be0dbc655a483b6ee311426564c50/reprotest-0.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "bd95ec5f7abe324cb4ac4ad5ee621417", "sha256": "9f0fe656008e6e386d3ab4ea6770c06e355e4c3eeaa05dae02e4dcb617392eb6" }, "downloads": -1, "filename": "reprotest-0.2.tar.gz", "has_sig": false, "md5_digest": "bd95ec5f7abe324cb4ac4ad5ee621417", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 70766, "upload_time": "2016-07-25T18:51:03", "url": "https://files.pythonhosted.org/packages/d8/62/07a83c61a791a3a5bcdf5b9192335396c21c3f8eb501b2f940349ff11e17/reprotest-0.2.tar.gz" } ], "0.5": [ { "comment_text": "", "digests": { "md5": "dc3e82cce2b69482499d383b6262db56", "sha256": "0b62a1a9e9ce43ef2b33bd9fd66f94d148736198593e947639a70c29937c99eb" }, "downloads": -1, "filename": "reprotest-0.5.tar.gz", "has_sig": true, "md5_digest": "dc3e82cce2b69482499d383b6262db56", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 57803, "upload_time": "2017-01-06T20:56:00", "url": "https://files.pythonhosted.org/packages/2f/be/a8afd0cce990158fa290166fda387f2d2c7952da454ddc698c4921d2f104/reprotest-0.5.tar.gz" } ], "0.7.8": [ { "comment_text": "", "digests": { "md5": "cbc80f1aec6b413587cc451cce6edad8", "sha256": "a321d5f5aac17022e101e7690185b94cab3979cee342f1ff8c527ee4fccd4924" }, "downloads": -1, "filename": "reprotest-0.7.8.tar.gz", "has_sig": true, "md5_digest": "cbc80f1aec6b413587cc451cce6edad8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 107044, "upload_time": "2018-07-09T13:37:59", "url": "https://files.pythonhosted.org/packages/02/b3/fbc845e7464827bf2f705fd2f4f99fe419fdd4e74d66c0f4ffc4a0e80aa8/reprotest-0.7.8.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "cbc80f1aec6b413587cc451cce6edad8", "sha256": "a321d5f5aac17022e101e7690185b94cab3979cee342f1ff8c527ee4fccd4924" }, "downloads": -1, "filename": "reprotest-0.7.8.tar.gz", "has_sig": true, "md5_digest": "cbc80f1aec6b413587cc451cce6edad8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 107044, "upload_time": "2018-07-09T13:37:59", "url": "https://files.pythonhosted.org/packages/02/b3/fbc845e7464827bf2f705fd2f4f99fe419fdd4e74d66c0f4ffc4a0e80aa8/reprotest-0.7.8.tar.gz" } ] }