{
    "info": {
        "author": "Bence Nagy",
        "author_email": "bence@underyx.me",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Environment :: Console",
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3.5"
        ],
        "description": "Tracks: Simple A/B testing of your backend\n==========================================\n\nHave you ever wanted to A/B test multiple code paths on your backend? Consider\nthe following: management wants you to cut down on server costs. You come to\nterms with the fact that you really don't need a 128 GB memory 32 core beast\nfor your coconut store \u2014 but how low can you go? Let's test how a slower\ncoconut catalog page affects conversion! (First, without ``tracks``.)\n\n.. code-block:: python\n\n    response = render_coconut_catalog()\n    test_value = random.random()\n    if test_value < (1 / 3):\n        # control group\n        sleep_sec = 0\n    elif test_value < (2 / 3):\n        sleep_sec = 0.5\n    else:\n        sleep_sec = 2\n    time.sleep(sleep_sec)\n    cur.execute('INSERT INTO test_runs (user_id, variant) VALUES (%s, %s)', user_id, sleep_sec)\n    return response\n\nNow, this kinda works, but it's already a bit messy and can easily get out of\nhand if you consider that right now:\n\n - The thing you test is really, really simple\n - You don't lock a user's version to make sure they always get the same test\n   variant, making your data more mushy with each page load.\n - You don't exclude your most important users from testing to avoid hurting\n   conversion among coconut addicts in your initial test run. (Of course you\n   will need to include them later to make an informed decision.)\n - You already have a few more ideas about things you want to A/B test. Maybe\n   a hundred.\n - You can add another 5 engineers to the project and they will probably each\n   implement these tests slightly differently, and that's no good.\n\nEnter ``tracks``. First we define our variants:\n\n.. code-block:: python\n\n    class DelayTracks(tracks.SimpleTrackSet):\n\n        @staticmethod\n        def short_track():\n            time.sleep(0.5)\n\n        @staticmethod\n        def long_track():\n            time.sleep(2)\n\nAnd to use it:\n\n.. code-block:: python\n\n    response = render_coconut_catalog()\n    with DelayTracks() as track:\n        track()\n        cur.execute('INSERT INTO test_runs (user_id, variant) VALUES (%s, %s)', user_id, track.name)\n    return response\n\nAlready a bit nicer, but look what happens if we go through the list of\nproblems above one by one. Firstly, we can easily make the tests more complex:\n\n.. code-block:: python\n\n    class PricingTracks(tracks.SimpleTrackSet):\n\n        @staticmethod\n        def expensive_track(response):\n            for coconut in response['coconuts']:\n                coconut['price'] += 1\n\n        @staticmethod\n        def cheap_track(response):\n            for coconut in response['coconuts']:\n                coconut['price'] -= 1\n\nDidn't get that much worse, now did it? Let's take it even further with more\nvariants:\n\n.. code-block:: python\n\n    class PricingTracks(tracks.ParamTrackSet):\n\n        params = [{'price_delta': n} for n in range(-5, 6)]\n        add_control_track = False  # {'price_delta': 0} is our control group\n\n        @staticmethod\n        def track_name(price_delta):\n            return 'price_adjusted_by_{0}'.format(price_delta)\n\n        @staticmethod\n        def track(response, price_delta):\n            for coconut in response['coconuts']:\n                coconut['price'] += price_delta\n\nTons of tests! Let's move on to the second bullet point. How do we lock the\nvariant served to users? Just change your usage to pass a user key to tracks:\n\n\n.. code-block:: python\n\n    response = render_coconut_catalog()\n    with DelayTracks(key=user_id) as track:\n        track()\n        cur.execute('INSERT INTO test_runs (user_id, variant) VALUES (%s, %s)', user_id, track.name)\n    return response\n\nThe key will be serialized to a string and the variant to use will be derived\nfrom that string. The key of course can be anything; in most cases it might be\nthe user ID, but you could use a combination of the user's country and the\narticle ID for instance. (Not sure why you would want this specific example,\nbut you get the point.)\n\nSo, with that solved, list item #3, here we come! What if we're worried about\nour top customers being mad at us for testing things on them? Easy peasy.\n\n.. code-block:: python\n\n    class DelayTracks(tracks.SimpleTrackSet):\n\n        @property\n        def is_eligible(self):\n            return not self.context['user']['is_vip']\n\n        # code of variants trimmed\n\n    response = render_coconut_catalog()\n    with DelayTracks(context={'user': user_dict}) as track:\n        track()  # will always be control for VIPs (even with `add_control_group = False`)\n        if track.is_eligible:\n            cur.execute('INSERT INTO test_runs (user_id, variant) VALUES (%s, %s)', (user_id, track.name))\n    return response\n\nAnd finally, let's try running the delay and pricing tests at the same time!\n\n.. code-block:: python\n\n    tracksets = [DelayTracks, PricingTracks]\n\n    response = render_coconut_catalog()\n    with tracks.MultiTrackSet(tracksets, key=user_id) as multitrack:\n        multitrack()\n        cur.execute(\n            'INSERT INTO test_runs (user_id, test, variant) VALUES (%s, %s)',\n            (user_id, multitrack.trackset.name, multitrack.track.name),\n        )\n    return response\n\nSo, with this, ``tracks`` will gather all tests the user is eligible for, and\nchoose one of them based on the given key. We could also set\n``DelayTracks.weight`` to 1000 to make that one ten times as likely to be used\nas the pricing one (the default weight is 100.)\n\nWe got pretty far with just a few lines of code, didn't we?",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "https://github.com/underyx/tracks/releases",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/underyx/tracks",
        "keywords": null,
        "license": "UNKNOWN",
        "maintainer": null,
        "maintainer_email": null,
        "name": "tracks",
        "package_url": "https://pypi.org/project/tracks/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/tracks/",
        "project_urls": {
            "Download": "https://github.com/underyx/tracks/releases",
            "Homepage": "https://github.com/underyx/tracks"
        },
        "release_url": "https://pypi.org/project/tracks/0.1.1/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Simple A/B testing for your backend",
        "version": "0.1.1"
    },
    "last_serial": 2176944,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a8355102e4dab9706cfb3c51c05ad910",
                    "sha256": "8852d5f44f66610e0f8b973e865827f70b6c207c01d856e59643486dc1ab53b3"
                },
                "downloads": -1,
                "filename": "tracks-0.1.0-py2.py3-none-any.whl",
                "has_sig": true,
                "md5_digest": "a8355102e4dab9706cfb3c51c05ad910",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 7467,
                "upload_time": "2016-06-15T14:25:34",
                "url": "https://files.pythonhosted.org/packages/e7/3e/7ff763e831b4cdd73c0d957e29b5e9eaebf04753198dc5446ab3e6e63bfd/tracks-0.1.0-py2.py3-none-any.whl"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "693bad960361dea1421a104b9b12db79",
                    "sha256": "fcd114c22951e3fd1d925314525a32919ad102c2c0edb607b94b4bcad846e8dc"
                },
                "downloads": -1,
                "filename": "tracks-0.1.1-py2.py3-none-any.whl",
                "has_sig": true,
                "md5_digest": "693bad960361dea1421a104b9b12db79",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 7504,
                "upload_time": "2016-06-20T10:25:04",
                "url": "https://files.pythonhosted.org/packages/28/36/4832cfef74182abae15b7a6d044bfb2856d0e1b41d93754a23e26f6dbf20/tracks-0.1.1-py2.py3-none-any.whl"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "693bad960361dea1421a104b9b12db79",
                "sha256": "fcd114c22951e3fd1d925314525a32919ad102c2c0edb607b94b4bcad846e8dc"
            },
            "downloads": -1,
            "filename": "tracks-0.1.1-py2.py3-none-any.whl",
            "has_sig": true,
            "md5_digest": "693bad960361dea1421a104b9b12db79",
            "packagetype": "bdist_wheel",
            "python_version": "2.7",
            "requires_python": null,
            "size": 7504,
            "upload_time": "2016-06-20T10:25:04",
            "url": "https://files.pythonhosted.org/packages/28/36/4832cfef74182abae15b7a6d044bfb2856d0e1b41d93754a23e26f6dbf20/tracks-0.1.1-py2.py3-none-any.whl"
        }
    ]
}