{ "info": { "author": "Presley Pizzo", "author_email": "ppizzo@linguist.umass.edu", "bugtrack_url": null, "classifiers": [ "Programming Language :: Python :: 2.7" ], "description": "#Speriment\n\n##Making experiments easier to express\n\n###What is Speriment?\nSperiment is a package, inspired by [SurveyMan](https://github.com/SurveyMan/SurveyMan), to help you write an online experiment for use with\n[PsiTurk](https://psiturk.org/). PsiTurk describes itself as a tape player - it can run any \"tape\",\nthat is, JavaScript program, as an online experiment on Mechanical Turk. But\nyou have to provide the tape. Speriment allows you to write a simple Python\nscript, using only the most basic programming skills, to create that JavaScript\nprogram. Instead of writing your own code to shuffle items, display HTML,\nrecord answers, and so on, you simply describe the structure and contents of\nyour experiment.\n\nIt's currently in beta. It should work on any Unix machine but has only been\ntested on a Mac. Please add an Issue if you find any bugs.\n\n###The structure of a Speriment\n\nSperiment experiments are made by nesting Python objects.\n\nThe outermost object is an `Experiment.`\n\n`Experiment`s contain `Block`s.\n\n`Block`s can contain other `Block`s, or `Page`s, or lists of `Page`s (called groups, but these have no special constructor).\n\n`Page`s that pose questions contain `Option`s, and those that are instructions do not.\n\nThere are a few other objects and a handful of functions that you may also want to use to either process your data or build up your experiment.\n\nJust call `install` on your `Experiment` at the end of your script to write your experiment to a JavaScript file which Speriment and PsiTurk can use to run your experiment online.\n\nCheck out the API at [RawGit](http://rawgit.com/presleyp/Speriment/master/doc/speriment.html).\n\nCheck out an [example script](https://github.com/presleyp/Speriment/blob/master/doc/example.py) to see some of the trickier ideas in action.\n\n###What kinds of experiments can Speriment run?\nHere are a few things Speriment can handle:\n\n- counterbalancing. Say you have questions that belong to treatment A and\n questions that belong to treatment B, and you want half of your participants\n to get treatment A first, and half to get treatment B first. Speriment can\n handle this in a probabilistic way. Put your A questions in one block and\n your B questions in another block and specify both of them as exchangeable\n blocks. It can also handle it in a deterministic way --- use `counterbalance` instead of `exchangeable`.\n You'll also need to set your PsiTurk config file's `num_counters` variable in this case.\n- Latin squares. For each item set (conditions 1 to n of an item), make a group\n (a list of Pages). Keep the order of the conditions the same in each group.\n Then set the block containing the groups to `latin_square = True`. This feature\n uses the `num_conds` variable set by PsiTurk, so remember to set that to your\n number of conditions.\n- pseudorandomization. Specify the `condition` of each Page in a block, and then\n set `pseudorandom = True`. The block will not run two items of the same\n condition in a row.\n- training loops. Give the relevant pages in a block (or their options)\n `correct` attributes. Then set a criterion for the block, as explained in the\n Python API. The block will rerun itself until the participant performs as\n well as you specified in the criterion. You may want to specify `feedback` on\n each Page or Option to tell participants how they're doing.\n- presentation of items conditioned on previous responses. Create a RunIf\n object and put it in the block that you want to run only under a certain\n condition.\n- distribution of text and resources to pages on a per-participant basis. This uses\n `banks` defined in a block or the experiment, and `SampleFrom` objects in place of\n the string to be sampled from a bank.\n\n###How do I run an experiment?\nYou'll still need to follow all of the [instructions for using PsiTurk](https://psiturk.readthedocs.org).\n\nBut this is what the workflow will look like in your terminal:\n\n1. Install PsiTurk and the Python component of Speriment. This only has to be done once, whereas future steps are done once per experiment.\nAnother one-time installation you may want to consider is of a database like MySQL, as the database PsiTurk uses by default may lead to corrupt data\nif multiple participants try to submit data at once. See the PsiTurk documentation.\n\n `sudo pip install psiturk`\n\n `sudo pip install speriment`\n\n2. Make a project directory for this experiment. In this case I'm calling it `myproject`.\n \n `psiturk-setup-example`\n\n `mv psiturk-example/ myproject/`\n\n3. Edit your [configuration files](http://psiturk.readthedocs.org/en/latest/configuration.html) and the following files in `templates`:\n - [ad.html](http://psiturk.readthedocs.org/en/latest/file_desc/ad_html.html)\n - [complete.html](http://psiturk.readthedocs.org/en/latest/file_desc/complete_html.html)\n - [consent.html](http://psiturk.readthedocs.org/en/latest/file_desc/consent_html.html)\n - [default.html](http://psiturk.readthedocs.org/en/latest/file_desc/default_html.html)\n - [error.html](http://psiturk.readthedocs.org/en/latest/file_desc/error_html.html)\n\n Note that [exp.html](http://psiturk.readthedocs.org/en/latest/file_desc/exp_html.html) is required for PsiTurk, but you should not edit it, because Speriment does so automatically. Speriment also automatically edits `static/task.js`, so do not delete it.\n\n4. Write a Python script to generate a Speriment from a csv of your experimental\nmaterials. Put the csv file and the Python script in `myproject` (or whatever\nyou called the directory).\n\n5. Install the JavaScript component of Speriment.\n \n `cd ~/myproject/static/lib`\n \n `npm install speriment`\n\n6. Run your Python script. It's important to do this after installing Speriment. If you accidentally do it in the wrong order, you can always rerun the script.\n \n `cd ~/myproject`\n \n `python myscript`\n\n7. Enter the PsiTurk shell. If you're using a MySQL database, start its server first with `mysql.server start`.\n \n `psiturk`\n\n8. In the PsiTurk shell, turn on the server and, if you're using a tunnel, open a tunnel.\n\n `server on`\n \n `tunnel open`\n\n9. Debug your experiment in your browser.\n \n `debug`\n\n10. Try out your experiment in the Mechanical Turk Sandbox. This command will ask you questions and then give you two links; follow the Sandbox link.\n \n `hit create`\n\n11. When you're ready, switch to live mode and make a HIT to put on the real Mechanical Turk.\n \n `mode`\n \n `hit create`\n\n The PsiTurk shell also has other useful commands, so check out its documentation.\n\n12. Check on your experiment as it's running with PsiTurk commands like `hit list` and `worker list.` When the HIT is reviewable, you can run `worker approve` to pay workers.\n\n13. Finally, use Speriment to retrieve and format your data, writing it to a\n csv in your project directory that you can load into Python or R. Speriment\n comes with a command-line tool `speriment-output` to make this easy. It\n takes one required argument and one optional list of arguments. The required argument is\n the name of a file to write the results to. The optional one is --exclude\n or -e. It should be used if you know already that you want to exclude data\n from certain workers. You can exclude them after looking at the data, so\n this is just a convenience. Follow the flag with the worker IDs of the\n workers you want to exclude.\n\n Here, I'm imagining I want to exclude the data that I generated\n when I was debugging the experiment and was given worker ID debugALHLUO.\n\n `speriment-output myproject_results.csv -e debugALHLUO`\n\n\n###What data does Speriment record?\n\nIn terms of PsiTurk, Speriment only records trial data, not unstructured data.\nPsiTurk records event data automatically, but only for a few kinds of events.\n\nSperiment records the following trial data and `speriment-output` gives it these column names:\n\n- PageID: ID given or automatically generated for the page.\n- PageText: Text displayed on the page.\n- PageResources: Resources displayed on the page.\n- BlockIDs: IDs of all blocks that enclose this page\n- StartTime: The time when the page displayed. This is in milliseconds since\n 1/1/1970, which makes it easy to do math on.\n- EndTime: The time when the participant clicked Next.\n- ReactionTime: The difference between StartTime and EndTime.\n- Iteration: The number of times, counting from 1, that this page was\n displayed. Will always be 1 unless the page is in a block with a criterion.\n- Condition: The experimental condition supplied or sampled for the page.\n- OptionOrder: The option IDs in the order in which they were displayed.\n Options are shuffled and you may want to look at how they appeared on the\n page.\n- OptionTexts: The text of each option displayed on this page in the order in which it displayed.\n- OptionResources: Resources displayed with the options on this page, grouped\n by option in the order in which the options displayed.\n- SelectedID: The IDs of any options that the participant selected.\n- SelectedPosition: The position, left-to-right starting from 0 at the left, of\n any selected options.\n- SelectedText: The text of any options that the participant selected.\n- Correct: The information you supplied about whether the option is correct or\n what a correct text answer will match.\n\n\n`speriment-output` also returns the following columns from PsiTurk data:\n- UniqueID: The HIT ID and Worker ID\n- TrialNumber: Starting from 0, the number of this trial. Every page gets a\n number, including instructions and feedback.\n- Version: If you set the `num_conds` variable in config.txt, this is the\n version of the experiment that the participant saw. Used in Latin Square\n designs.\n- Permutation: If you set the `num_counters` variable in config.txt, this\n determines the ordering of the blocks you counterbalanced.\n- HIT: HIT ID\n- WorkerID: Worker ID of the participant\n\nFinally, it returns the tags you included in your Python script:\n- User-defined columns: There will then be a column for each page tag you\n supplied and a column for each option tag you supplied. Option tag values\n will be grouped by option and giving in the order in which the options were\n displayed.\n\nPsiTurk provides information about the version of the experiment (which they\ncall condition) that was used for the purpose of Latin squares, the worker ID,\nand the trial number. Note that it also supplies, with each trial, a field\ncalled \"datetime\", which is the time the trial was saved. All trials are saved\nat the end of the experiment, so this number is not informative for reaction\ntimes and does not reliably show trial order.\n\n###How do I contribute?\n\nContributions are super welcome. But before you start coding, start an Issue or\ncomment on an existing one. There are lots of new features to add, and we want to\nmake sure they'll play nice together before anyone spends time implementing stuff.\n", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/presleyp/speriment", "keywords": "experiments psychology linguistics", "license": "GPL 2", "maintainer": null, "maintainer_email": null, "name": "speriment", "package_url": "https://pypi.org/project/speriment/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/speriment/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/presleyp/speriment" }, "release_url": "https://pypi.org/project/speriment/0.8.2/", "requires_dist": null, "requires_python": null, "summary": "Making experiments easier to express", "version": "0.8.2" }, "last_serial": 2045835, "releases": { "0.1": [], "0.2.0": [ { "comment_text": "", "digests": { "md5": "7d0ee277203a9aff1b16e0543d993b58", "sha256": "2cff7c8f0dce92b9b26d8430dc35cc42816b72c6c70cb6e002647572c3a85b5d" }, "downloads": -1, "filename": "speriment-0.2.0.tar.gz", "has_sig": false, "md5_digest": "7d0ee277203a9aff1b16e0543d993b58", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24607, "upload_time": "2015-01-06T02:26:23", "url": "https://files.pythonhosted.org/packages/c0/c2/a77c4312526fa08943f1bcddb6f3f8b49ae843042f4d834fc75655021742/speriment-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "74c18b64cb77f622ed1afdc88f20b054", "sha256": "83a140018e154cd09a6be81cdefb1f49ef523d9941e7e556c7b64ff11de9f5ca" }, "downloads": -1, "filename": "speriment-0.2.1.tar.gz", "has_sig": false, "md5_digest": "74c18b64cb77f622ed1afdc88f20b054", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24650, "upload_time": "2015-01-06T04:17:35", "url": "https://files.pythonhosted.org/packages/38/ed/cc283eb203d3c293fa1a078390e6738b3b24f39e676b89a7f2fc20830053/speriment-0.2.1.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "418a53c50290ff3364f0f897fd88f093", "sha256": "6897b499daf38531f27ac14d91f8a0e83c8a24a5ddbe306446bc9f8c906c2355" }, "downloads": -1, "filename": "speriment-0.3.0.tar.gz", "has_sig": false, "md5_digest": "418a53c50290ff3364f0f897fd88f093", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 25145, "upload_time": "2015-01-09T21:51:57", "url": "https://files.pythonhosted.org/packages/7f/58/3d599d1064b1be804230df588b9067dc111d8a201b5a951273de47de64b8/speriment-0.3.0.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "56c1949c33a9b9d1c52f1e1f3f9df605", "sha256": "fb7890dcfe1d37a498f25d5463486918aa39697614d26f1d9f890cdf2167fe45" }, "downloads": -1, "filename": "speriment-0.4.0.tar.gz", "has_sig": false, "md5_digest": "56c1949c33a9b9d1c52f1e1f3f9df605", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16188, "upload_time": "2015-01-14T07:00:59", "url": "https://files.pythonhosted.org/packages/af/7c/517285933a809b8593838837d6a9059df2b1c01e75a6b580d7d5104f2673/speriment-0.4.0.tar.gz" } ], "0.5.0": [ { "comment_text": "", "digests": { "md5": "91db1f2e6529ecf8e1a20704ac742c84", "sha256": "423ecdf8ee7dc07cd4751fb51db338fb9efa1f984b67a984dec649d9128055f1" }, "downloads": -1, "filename": "speriment-0.5.0.tar.gz", "has_sig": false, "md5_digest": "91db1f2e6529ecf8e1a20704ac742c84", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17456, "upload_time": "2015-02-05T21:55:41", "url": "https://files.pythonhosted.org/packages/d6/9d/00fff02b5537ac966c7b7fe150bbc6b583070e39b33460ba96109acfc534/speriment-0.5.0.tar.gz" } ], "0.5.1": [ { "comment_text": "", "digests": { "md5": "2433913c6b76f5ddbb2ee61b06d67369", "sha256": "145e4fff913c4e5dced347be749a620f678bfc23b048f3910bbd4134d3fbc9f5" }, "downloads": -1, "filename": "speriment-0.5.1.tar.gz", "has_sig": false, "md5_digest": "2433913c6b76f5ddbb2ee61b06d67369", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17664, "upload_time": "2015-02-11T20:00:24", "url": "https://files.pythonhosted.org/packages/02/26/ab98e1e34c0649a3f5e7819c400bf937057b4dbdeb206486d4e3b77ffa4e/speriment-0.5.1.tar.gz" } ], "0.6.0": [ { "comment_text": "", "digests": { "md5": "e1b5a8c598a907353c39510152726e17", "sha256": "7d88156d1f3cbc0607b81dfc9bbe0076e4e6946ca798e7ec01aa7008c8e2d838" }, "downloads": -1, "filename": "speriment-0.6.0.tar.gz", "has_sig": false, "md5_digest": "e1b5a8c598a907353c39510152726e17", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18271, "upload_time": "2015-04-09T03:41:54", "url": "https://files.pythonhosted.org/packages/60/ce/461becf63d4420c616d9f6f5092d75b59dc7f9ec3ed529a49d8d5d223a45/speriment-0.6.0.tar.gz" } ], "0.7.0": [ { "comment_text": "", "digests": { "md5": "21dca3caccd7b56cd5d802b2cf01feef", "sha256": "102be4aa598735acd0f595f355c02f4463b561f14da12cd8f3481353580c85e9" }, "downloads": -1, "filename": "speriment-0.7.0.tar.gz", "has_sig": false, "md5_digest": "21dca3caccd7b56cd5d802b2cf01feef", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18429, "upload_time": "2015-05-07T18:08:32", "url": "https://files.pythonhosted.org/packages/6a/9d/1047abd6b5c0bcbc0db0ebf5678c5b796fd2d5b5a957116da4ba4a81094a/speriment-0.7.0.tar.gz" } ], "0.8.0": [ { "comment_text": "", "digests": { "md5": "91d718c0c4b2ed54da56a11ea0623c46", "sha256": "6251e85fa73b1629967bdea7a4cb6b2704c7b5e11cb89a5e32c2843c425933ee" }, "downloads": -1, "filename": "speriment-0.8.0.tar.gz", "has_sig": false, "md5_digest": "91d718c0c4b2ed54da56a11ea0623c46", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10004, "upload_time": "2016-03-26T02:19:31", "url": "https://files.pythonhosted.org/packages/4f/6a/c2eae0161e0bc669604214b2f31defa78e1a12d22e6750ed2896e803528b/speriment-0.8.0.tar.gz" } ], "0.8.1": [ { "comment_text": "", "digests": { "md5": "39225215089884c2221fedbf18b4da45", "sha256": "b9d7eb484e82ea260b75c26a11c51bcf56912e4d4e7a4f6c65719a871f1f4ae2" }, "downloads": -1, "filename": "speriment-0.8.1.tar.gz", "has_sig": false, "md5_digest": "39225215089884c2221fedbf18b4da45", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17037, "upload_time": "2016-03-27T21:53:56", "url": "https://files.pythonhosted.org/packages/0c/92/4b249aec37a9d95960a68af40621be75e3d5d280f73ded395b553a5f2f2b/speriment-0.8.1.tar.gz" } ], "0.8.2": [ { "comment_text": "", "digests": { "md5": "edf165b5f82e2ba14a27c846a2d2ca8c", "sha256": "3764b652ac7a5130ffeb14c8c484e731f910119d339db0b97555b7de4026e450" }, "downloads": -1, "filename": "speriment-0.8.2.tar.gz", "has_sig": false, "md5_digest": "edf165b5f82e2ba14a27c846a2d2ca8c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19192, "upload_time": "2016-04-04T21:50:12", "url": "https://files.pythonhosted.org/packages/c3/e4/be445b753788235976f44cda6feeefc44208ebea4d7d60095f0711c0a500/speriment-0.8.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "edf165b5f82e2ba14a27c846a2d2ca8c", "sha256": "3764b652ac7a5130ffeb14c8c484e731f910119d339db0b97555b7de4026e450" }, "downloads": -1, "filename": "speriment-0.8.2.tar.gz", "has_sig": false, "md5_digest": "edf165b5f82e2ba14a27c846a2d2ca8c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19192, "upload_time": "2016-04-04T21:50:12", "url": "https://files.pythonhosted.org/packages/c3/e4/be445b753788235976f44cda6feeefc44208ebea4d7d60095f0711c0a500/speriment-0.8.2.tar.gz" } ] }