{
"info": {
"author": "Michael Murray",
"author_email": "m.j.murray123@gmail.com",
"bugtrack_url": null,
"classifiers": [
"License :: OSI Approved :: MIT License",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3"
],
"description": "# The MAME RL Algorithm Training Toolkit\n\n## About\nThis Python library has the to potential to train your reinforcement learning algorithm on almost any arcade game. It is currently available on Linux systems and works as a wrapper around [MAME](http://mamedev.org/). The toolkit allows your algorithm to step through gameplay while recieving the frame data and internal memory address values for tracking the games state, along with sending actions to interact with the game.\n\n## Installation\nYou can use `pip` to install the library, just run:\n```bash\npip install MAMEToolkit\n```\n\n**DISCLAIMER: We are unable to provide you with any game ROMs. It is the users own legal responsibility to acquire a game ROM for emulation. This library should only be used for non-commercial research purposes.**\n\nThere are some free ROMs available at: [https://www.mamedev.org/roms/]\n\n## Street Fighter Random Agent Demo\nThe toolkit has currently been applied to Street Fighter III Third Strike: Fight for the Future (Japan 990608, NO CD), but can modified for any game available on MAME. The following demonstrates how a random agent can be written for a street fighter environment.\n```python\nimport random\nfrom MAMEToolkit.sf_environment import Environment\n\nroms_path = \"roms/\" # Replace this with the path to your ROMs\nenv = Environment(\"env1\", roms_path)\nenv.start()\nwhile True:\n move_action = random.randint(0, 8)\n attack_action = random.randint(0, 9)\n frames, reward, round_done, stage_done, game_done = env.step(move_action, attack_action)\n if game_done:\n env.new_game()\n elif stage_done:\n env.next_stage()\n elif round_done:\n env.next_round()\n```\n\nThe toolkit also supports hogwild training:\n```python\nfrom multiprocessing import Process\nimport random\nfrom MAMEToolkit.sf_environment import Environment\n\n\ndef run_env(worker_id, roms_path):\n env = Environment(f\"env{worker_id}\", roms_path)\n env.start()\n while True:\n move_action = random.randint(0, 8)\n attack_action = random.randint(0, 9)\n frames, reward, round_done, stage_done, game_done = env.step(move_action, attack_action)\n if game_done:\n env.new_game()\n elif stage_done:\n env.next_stage()\n elif round_done:\n env.next_round()\n\n\nworkers = 8\n# Environments must be created outside of the threads\nroms_path = \"roms/\" # Replace this with the path to your ROMs\nthreads = [Process(target=run_env, args=(i, roms_path)) for i in range(workers)]\n[thread.start() for thread in threads]\n```\n\n![](pics/hogwild3.gif \"Hogwild Random Agents\")\n\n## Setting Up Your Own Game Environment\n\n**Game ID's**
\nTo create an emulation of the game you must first have the ROM for the game you are emulating and know the game ID used by MAME, for example for this version of street fighter it is 'sfiii3n'. \nThe id of your game can be found by running:\n```python\nfrom MAMEToolkit.emulator import see_games\nsee_games()\n```\nThis will bring up the MAME emulator. You can search through the list of games to find the one you want. The id of the game is always in brackets at the end of the game title.\n\n**Memory Addresses**
\nIt doesn't take much to interact with the emulator itself using the toolkit, however the challenge comes from finding the memory address values associated with the internal state you care about, and tracking said state with your environment class.\nThe internal memory states of a game can be tracked using the [MAME Cheat Debugger](http://docs.mamedev.org/debugger/cheats.html), which allows you to track how the memory address values of the game change over time.\n\n\nThe cheat debugger can be run using the following:\n```python\nfrom MAMEToolkit.emulator import run_cheat_debugger\nroms_path = \"roms/\" # Replace this with the path to your ROMs\ngame_id = \"sfiii3n\"\nrun_cheat_debugger(roms_path, game_id)\n```\nFor information about using the debugger, see the Memory dump section of this tutorial [https://www.dorkbotpdx.org/blog/skinny/use_mames_debugger_to_reverse_engineer_and_extend_old_games]\n\n\nOnce you have determined the memory addresses you wish to track you can start the emulation using:\n```python\nfrom MAMEToolkit.emulator import Emulator\nfrom MAMEToolkit.emulator import Address\n\nroms_path = \"roms/\" # Replace this with the path to your ROMs\ngame_id = \"sfiii3n\"\nmemory_addresses = {\n \"fighting\": Address('0x0200EE44', 'u8'),\n \"winsP1\": Address('0x02011383', 'u8'),\n \"winsP2\": Address('0x02011385', 'u8'),\n \"healthP1\": Address('0x02068D0B', 's8'),\n \"healthP2\": Address('0x020691A3', 's8')\n }\n\nemulator = Emulator(\"env1\", roms_path, \"sfiii3n\", memory_addresses)\n```\nThis will immediately start the emulation and halt it when the toolkit has linked to the emulator process. \n\n**Stepping the emulator**
\nOnce the toolkit is linked, you can step the emulator along using the step function:\n```python\ndata = emulator.step([])\n\nframe = data[\"frame\"]\nis_fighting = data[\"fighting\"]\nplayer1_wins = data[\"winsP1\"]\nplayer2_wins = data[\"winsP2\"]\nplayer1_health = data[\"healthP1\"]\nplayer2_health = data[\"healthP2\"]\n```\nThe step function returns the frame data as a NumPy matrix, along with all of the memory address integer values from that timestep.\n\n**Sending inputs**\nTo send actions to the emulator you also need to determine which input ports and fields the game supports. For example, with street fighter to insert a coin the following code is required:\n```python\nfrom MAMEToolkit.emulator import Action\n\ninsert_coin = Action(':INPUTS', 'Coin 1')\ndata = emulator.step([insert_coin])\n```\nTo identify which ports are availble use the list actions command:\n```python\nfrom MAMEToolkit.emulator import list_actions\n\nroms_path = \"roms/\" # Replace this with the path to your ROMs\ngame_id = \"sfiii3n\"\nprint(list_actions(roms_path, game_id))\n```\nwhich for street fighter returns the list with all the ports and fields available for sending actions to the step function:\n```python\n[\n {'port': ':scsi:1:cdrom:SCSI_ID', 'field': 'SCSI ID'}, \n {'port': ':INPUTS', 'field': 'P2 Jab Punch'}, \n {'port': ':INPUTS', 'field': 'P1 Left'}, \n {'port': ':INPUTS', 'field': 'P2 Fierce Punch'}, \n {'port': ':INPUTS', 'field': 'P1 Down'}, \n {'port': ':INPUTS', 'field': 'P2 Down'}, \n {'port': ':INPUTS', 'field': 'P2 Roundhouse Kick'}, \n {'port': ':INPUTS', 'field': 'P2 Strong Punch'}, \n {'port': ':INPUTS', 'field': 'P1 Strong Punch'}, \n {'port': ':INPUTS', 'field': '2 Players Start'}, \n {'port': ':INPUTS', 'field': 'Coin 1'}, \n {'port': ':INPUTS', 'field': '1 Player Start'}, \n {'port': ':INPUTS', 'field': 'P2 Right'}, \n {'port': ':INPUTS', 'field': 'Service 1'}, \n {'port': ':INPUTS', 'field': 'Coin 2'}, \n {'port': ':INPUTS', 'field': 'P1 Jab Punch'}, \n {'port': ':INPUTS', 'field': 'P2 Up'}, \n {'port': ':INPUTS', 'field': 'P1 Up'}, \n {'port': ':INPUTS', 'field': 'P1 Right'}, \n {'port': ':INPUTS', 'field': 'Service Mode'}, \n {'port': ':INPUTS', 'field': 'P1 Fierce Punch'}, \n {'port': ':INPUTS', 'field': 'P2 Left'}, \n {'port': ':EXTRA', 'field': 'P2 Short Kick'}, \n {'port': ':EXTRA', 'field': 'P2 Forward Kick'}, \n {'port': ':EXTRA', 'field': 'P1 Forward Kick'}, \n {'port': ':EXTRA', 'field': 'P1 Roundhouse Kick'}, \n {'port': ':EXTRA', 'field': 'P1 Short Kick'}\n]\n```\nWe advise you to create an enum of all the possible actions, then send their action values to the emulator, see [the example Actions Enum](MAMEToolkit/sf_environment/Actions.py)\n\nThere is also the problem of transitioning games between non-learnable gameplay screens such as the title screen and character select. To see how this can be implemented please look at the provided [Steps script](MAMEToolkit/sf_environment/Steps.py) and the [Example Street Fighter III Third Strike: Fight for the Future Environment Implementation](MAMEToolkit/sf_environment/Environment.py)\n\nThe emulator class also has a frame_ratio argument which can be used for adjusting the frame rate seen by your algorithm. By default MAME generates frames at 60 frames per second, however, this may be too many frames for your algorithm. The toolkit by default will use a frame_ratio of 3, which means that 1 in 3 frames are sent through the toolkit, this converts the frame rate to 20 frames per second. Using a higher frame_ratio also increases the performance of the toolkit.\n```Python\nfrom MAMEToolkit.emulator import Emulator\n\nemulator = Emulator(roms_path, game_id, memory_addresses, frame_ratio=3)\n```\n\n## Library Performance Benchmarks with PC Specs\nThe development and testing of this toolkit have been completed on an 8-core AMD FX-8300 3.3GHz CPU along with a 3GB GeForce GTX 1060 GPU.\nWith a single random agent, the street fighter environment can be run at 600%+ the normal gameplay speed. And For hogwild training with 8 random agents, the environment can be run at 300%+ the normal gameplay speed.\n\n## Simple ConvNet Agent\nTo ensure that the toolkit is able to train algorithms, a simple 5 layer ConvNet was setup with minimal tuning. The algorithm was able to successfully learn some simple mechanics of Street Fighter, such as combos and blocking. The Street Fighter gameplay works by having the player fight different opponents across 10 stages of increasing difficulty. Initially, the algorithm would reach stage 2 on average, but eventually could reach stage 5 on average after 2200 episodes of training. The learning rate was tracked using the net damage done vs damage taken of a single playthough for each episode.\n\n![](pics/chart.png \"ConvNet Results\")\n\n## MAME Changes\nThe library works by acting as a wrapper around a modified MAME implementation.\nThe following changes were made:\n* Updated the lua console to allow for the retrieval of the format of frame data\n* Update the lua console to allow for the retrieval of the current frames data\n* Disabled game start warnings\n\nThe following files are affected:\n* src/emu/machine.cpp\n* src/emu/video.cpp\n* src/emu/video.h\n* src/frontend/mame/luaengine.cpp\n* src/frontend/mame/ui/ui.cpp\n* src/osd/sdl/window.cpp\n\n**The modified MAME implementation can be found at [https://github.com/M-J-Murray/mame]**\n\n\n\n\n\n",
"description_content_type": "text/markdown",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/M-J-Murray/MAMEToolkit",
"keywords": "",
"license": "",
"maintainer": "",
"maintainer_email": "",
"name": "MAMEToolkit",
"package_url": "https://pypi.org/project/MAMEToolkit/",
"platform": "",
"project_url": "https://pypi.org/project/MAMEToolkit/",
"project_urls": {
"Homepage": "https://github.com/M-J-Murray/MAMEToolkit"
},
"release_url": "https://pypi.org/project/MAMEToolkit/1.0.2/",
"requires_dist": null,
"requires_python": "",
"summary": "A library to train your RL algorithms against MAME arcade games on Linux",
"version": "1.0.2"
},
"last_serial": 4458615,
"releases": {
"1.0.0": [
{
"comment_text": "",
"digests": {
"md5": "efecfc826c642a8097a0b4c54e9b0dab",
"sha256": "e63b0eb3643280b801f7d0cf1e9b4134c669972781c4f3918ea36053e18f5eb0"
},
"downloads": -1,
"filename": "MAMEToolkit-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "efecfc826c642a8097a0b4c54e9b0dab",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 60297770,
"upload_time": "2018-11-01T23:05:26",
"url": "https://files.pythonhosted.org/packages/cd/30/8bb19ae9429684f32e722ed4bee9254443ba709474d4a8fdb48a079c23e3/MAMEToolkit-1.0.0-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "8b6ac8036f2d4670d56943b78d88a277",
"sha256": "d91a9d81e3e18c762555e43621b0219455fbce12e7e5e209e2001a558d981c62"
},
"downloads": -1,
"filename": "MAMEToolkit-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "8b6ac8036f2d4670d56943b78d88a277",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 59908974,
"upload_time": "2018-11-01T23:06:53",
"url": "https://files.pythonhosted.org/packages/83/8e/0bcf2b5efc0ca19aaa873cc395d87de240d3b557bc0895c4a83ff2643214/MAMEToolkit-1.0.0.tar.gz"
}
],
"1.0.1": [
{
"comment_text": "",
"digests": {
"md5": "aaa2c09f55ba11bd0e97c3779deab6ed",
"sha256": "bec24885c5637bbb449aeed5998c548d369e9f27f0c434e379befeb5e8fa24ef"
},
"downloads": -1,
"filename": "MAMEToolkit-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "aaa2c09f55ba11bd0e97c3779deab6ed",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 60298735,
"upload_time": "2018-11-04T14:24:19",
"url": "https://files.pythonhosted.org/packages/2c/e6/3a854b26aaa37ca65b8c32584ac7774e4fe27f39ac342d20b83f2bd15e8e/MAMEToolkit-1.0.1-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "eeb50ea24bbf0c0e02d105818bc29c21",
"sha256": "de052c0bace91ca4d06c8bb72fc9aa5afa907e48d50eddb486165409ad27a7f8"
},
"downloads": -1,
"filename": "MAMEToolkit-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "eeb50ea24bbf0c0e02d105818bc29c21",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 59908467,
"upload_time": "2018-11-04T14:25:52",
"url": "https://files.pythonhosted.org/packages/db/ad/dfa33e9ec2e96001f73b99b528af06fe32f39722a27123393f034bc4d038/MAMEToolkit-1.0.1.tar.gz"
}
],
"1.0.2": [
{
"comment_text": "",
"digests": {
"md5": "72f37eb0984e453f587b658db691fa83",
"sha256": "226751f2b8806802f23dc135bad06ec6b916042fa6f872a51dff2e1c8b4379b5"
},
"downloads": -1,
"filename": "MAMEToolkit-1.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "72f37eb0984e453f587b658db691fa83",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 60298827,
"upload_time": "2018-11-06T19:14:09",
"url": "https://files.pythonhosted.org/packages/e9/ad/39864cd62b6aa1ef9bfcf53df21c8922ffab7f353fdde6daefcf6a69816a/MAMEToolkit-1.0.2-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "c2f5483610d4295976d12d8e44efcc5d",
"sha256": "835147d37c6efa51b0fcca774a635fa49c62ba9f00e0482f50511212b76e7860"
},
"downloads": -1,
"filename": "MAMEToolkit-1.0.2.tar.gz",
"has_sig": false,
"md5_digest": "c2f5483610d4295976d12d8e44efcc5d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 59908400,
"upload_time": "2018-11-06T19:15:57",
"url": "https://files.pythonhosted.org/packages/62/87/4149d43f31029b423f1fb9d41a589d8fe429205f0945cad07f09b9d54a59/MAMEToolkit-1.0.2.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "72f37eb0984e453f587b658db691fa83",
"sha256": "226751f2b8806802f23dc135bad06ec6b916042fa6f872a51dff2e1c8b4379b5"
},
"downloads": -1,
"filename": "MAMEToolkit-1.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "72f37eb0984e453f587b658db691fa83",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 60298827,
"upload_time": "2018-11-06T19:14:09",
"url": "https://files.pythonhosted.org/packages/e9/ad/39864cd62b6aa1ef9bfcf53df21c8922ffab7f353fdde6daefcf6a69816a/MAMEToolkit-1.0.2-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "c2f5483610d4295976d12d8e44efcc5d",
"sha256": "835147d37c6efa51b0fcca774a635fa49c62ba9f00e0482f50511212b76e7860"
},
"downloads": -1,
"filename": "MAMEToolkit-1.0.2.tar.gz",
"has_sig": false,
"md5_digest": "c2f5483610d4295976d12d8e44efcc5d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 59908400,
"upload_time": "2018-11-06T19:15:57",
"url": "https://files.pythonhosted.org/packages/62/87/4149d43f31029b423f1fb9d41a589d8fe429205f0945cad07f09b9d54a59/MAMEToolkit-1.0.2.tar.gz"
}
]
}