{
    "info": {
        "author": "Simon Larsson",
        "author_email": "simonlarsson0@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python :: 3 :: Only",
            "Topic :: Scientific/Engineering"
        ],
        "description": "[![PyPI version](https://badge.fury.io/py/ballbeam-gym.svg)](https://pypi.python.org/pypi/ballbeam-gym/) \n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/simon-larsson/ballbeam-gym/blob/master/LICENSE)\n\n# Ball & Beam Gym\nBall & beam simulation as OpenAI gym environments.\n\n---\n\n## Installation\n\nRun command:\n\n    pip install ballbeam-gym\n\nor clone the repository and run the following inside the folder:\n\n    pip install -e .\n\n---\n\n## System Dynamics\nSimulated as a frictionless first order system that takes the beam angle as input. The equation that describe the system is as follows:\n\n    dx/dt = v(t)\n    dv/dt = -m*g*sin(theta(t))/((I + 1)*m)\n\n[![visualization](ballbeam.png)](https://github.com/simon-larsson/ballbeam-gym)\n\n---\n\n## Environments\n- [BallBeamBalanceEnv](https://github.com/simon-larsson/ballbeam-gym#ballbeambalanceenv) - Objective is to not drop the ball from the beam using key state variables as observation space.\n- [VisualBallBeamBalanceEnv](https://github.com/simon-larsson/ballbeam-gym#visualballbeambalanceenv) - Same as above but only using simulation plot as observation space.\n- [BallBeamSetpointEnv](https://github.com/simon-larsson/ballbeam-gym#ballbeamsetpointenv) - Objective is to keep the ball as close to a set position on the beam as possible using key state variables as observation space.\n- [VisualBallBeamSetpointEnv](https://github.com/simon-larsson/ballbeam-gym#visualballbeamsetpointenv) - Same as above but only using simulation plot as observation space.\n- [BallBeamThrowEnv](https://github.com/simon-larsson/ballbeam-gym#ballbeamthrowenv) - Objective is to throw the ball as far as possible to the right possible using key state variables as observation space.\n- [VisualBallBeamThrowEnv](https://github.com/simon-larsson/ballbeam-gym#visualballbeamthrowenv) - Same as above but only using simulation plot as observation space.\n\n#### Alias\n- `BallBeamBalance-v0`\n- `VisualBallBeamBalance-v0`\n- `BallBeamSetpoint-v0`\n- `VisualBallBeamSetpoint-v0`\n- `BallBeamThrow-v0`\n- `VisualBallBeamThrow-v0`\n\n---\n\n## API\n\nThe environments use the same API and inherits from OpenAI gyms.\n- `step(action)` - Simulate one timestep.\n- `reset()` - Reset environment to start conditions.\n- `render(mode='human')` - Visualize one timestep.\n- `seed(seed)` - Make environment deterministic.\n\n---\n\n### BallBeamBalanceEnv\n\nBall is given a random or set initial velocity and it is the agent's job to stabilize the ball on the beam using a set of key state variables.\n\n**Parameters**\n- `timestep` - Length of a timestep.\n- `beam_length` - Length of beam.\n- `max_angle` - Max abs(angle) of beam.\n- `init_velocity` - Initial speed of ball (`None` for random).\n- `max_timesteps` - Max timesteps in an episode (`None` for infinate).\n- `action_mode` - Continuous or discrete action space.\n\n**Observation Space** \n- Beam angle.\n- Ball position on beam.\n- Ball velocity.\n\n**Action Space**\n\nContinuous:\n- Set angle.\n\nDiscrete:\n- Increase angle.\n- Keep angle.\n- Descrease angle.\n\n**Rewards**\n\nA reward of 1 is given for each timestep ball stays on beam.\n\n**Reset**\n\nResets when ball falls of beam or max timesteps are reached.\n\n---\n\n### VisualBallBeamBalanceEnv\n\nBall is given a random or set initial velocity and it is the agent's job to stabilize the ball on the beam using a image data from the simulation plot.\n\n**Parameters**\n- `timestep` - Length of a timestep.\n- `beam_length` - Length of beam.\n- `max_angle` - Max abs(angle) of beam.\n- `init_velocity` - Initial speed of ball (`None` for random).\n- `max_timesteps` - Max timesteps in an episode (`None` for infinate).\n- `action_mode` - Continuous or discrete action space.\n\n**Observation Space** \n- RGB image [350x260x3].\n\n**Action Space**\n\nContinuous:\n- Set angle.\n\nDiscrete:\n- Increase angle.\n- Keep angle.\n- Descrease angle.\n\n**Rewards**\n\nA reward of 1 is given for each timestep ball stays on beam.\n\n**Reset**\n\nResets when ball falls of beam or max timesteps are reached.\n\n---\n\n### BallBeamSetpointEnv\n\nThe agent's job is to keep the ball's position as close as possible to a setpoint using a set of key state variables.\n\n**Parameters**\n- `timestep` - Length of a timestep.\n- `beam_length` - Length of beam.\n- `max_angle` - Max abs(angle) of beam.\n- `init_velocity` - Initial speed of ball (`None` for random).\n- `max_timesteps` - Max timesteps in an episode (`None` for infinate).\n- `action_mode` - Continuous or discrete action space.\n- `setpoint` - Target position of ball (`None` for random).\n\n**Observation Space** \n- Beam angle.\n- Ball position.\n- Ball velocity.\n- Setpoint position.\n\n**Action Space**\n\nContinuous:\n- Set angle.\n\nDiscrete:\n- Increase angle.\n- Keep angle.\n- Descrease angle.\n\n**Rewards**\n\nAt each timestep the agent is rewarded with the squared proximity between the ball and the setpoint: \n\n`reward = (1 - (setpoint - ball_position)/beam_length)^2`.\n\n**Reset**\n\nResets when ball falls of beam or max timesteps are reached.\n\n---\n\n### VisualBallBeamSetpointEnv\n\nThe agent's job is to keep the ball's position as close as possible to a setpoint using a image data from the simulation plot.\n\n**Parameters**\n- `timestep` - Length of a timestep.\n- `beam_length` - Length of beam.\n- `max_angle` - Max abs(angle) of beam.\n- `init_velocity` - Initial speed of ball (`None` for random).\n- `max_timesteps` - Max timesteps in an episode (`None` for infinate).\n- `action_mode` - Continuous or discrete action space.\n- `setpoint` - Target position of ball (`None` for random).\n\n**Observation Space** \n- RGB image [350x260x3].\n\n**Action Space**\n\nContinuous:\n- Set angle.\n\nDiscrete:\n- Increase angle.\n- Keep angle.\n- Descrease angle.\n\n**Rewards**\n\nAt each timestep the agent is rewarded with the squared proximity between the ball and the setpoint: \n\n`reward = (1 - (setpoint - ball_position)/beam_length)^2`.\n\n**Reset**\n\nResets when ball falls of beam or max timesteps are reached.\n\n---\n\n### BallBeamThrowEnv\n\nThe agent's job is to throw the ball as far as possible to the right using a set of key state variables.\n\n**Parameters**\n- `timestep` - Length of a timestep.\n- `beam_length` - Length of beam.\n- `max_angle` - Max abs(angle) of beam.\n- `init_velocity` - Initial speed of ball (`None` for random).\n- `max_timesteps` - Max timesteps in an episode (`None` for infinate).\n- `action_mode` - Continuous or discrete action space.\n\n**Observation Space** \n- Beam angle.\n- Ball position on beam.\n- Ball velocity.\n\n**Action Space**\n\nContinuous:\n- Set angle.\n\nDiscrete:\n- Increase angle.\n- Keep angle.\n- Descrease angle.\n\n**Rewards**\n\nIs rewarded the calculated distance the ball would travel in x direction when it leaves the beam. Reward for a negative distance is set to 0.\n\n**Reset**\n\nResets when ball leaves the beam or max timesteps are reached.\n\n---\n\n### VisualBallBeamThrowEnv\n\nThe agent's job is to throw the ball as far as possible to the right using a image data from the simulation plot.\n\n**Parameters**\n- `timestep` - Length of a timestep.\n- `beam_length` - Length of beam.\n- `max_angle` - Max abs(angle) of beam.\n- `init_velocity` - Initial speed of ball (`None` for random).\n- `max_timesteps` - Max timesteps in an episode (`None` for infinate).\n- `action_mode` - Continuous or discrete action space.\n\n**Observation Space** \n- RGB image [350x260x3].\n\n**Action Space**\n\nContinuous:\n- Set angle.\n\nDiscrete:\n- Increase angle.\n- Keep angle.\n- Descrease angle.\n\n**Rewards**\n\nIs rewarded the calculated distance the ball would travel in x direction when it leaves the beam. Reward for a negative distance is set to 0.\n\n**Reset**\n\nResets when ball leaves the beam or max timesteps are reached.\n\n---\n\n## Example: PID Controller\n```python\nimport gym\nimport ballbeam_gym\n\n# pass env arguments as kwargs\nkwargs = {'timestep': 0.05, \n          'setpoint': 0.4,\n          'beam_length': 1.0,\n          'max_angle': 0.2,\n          'init_velocity': 0.0,\n          'action_mode': 'continuous'}\n\n# create env\nenv = gym.make('BallBeamSetpoint-v0', **kwargs)\n\n# constants for PID calculation\nKp = 2.0\nKd = 1.0\n\n# simulate 1000 steps\nfor i in range(1000):   \n    # control theta with a PID controller\n    env.render()\n    theta = Kp*(env.bb.x - env.setpoint) + Kd*(env.bb.v)\n    obs, reward, done, info = env.step(theta)\n\n    if done:\n        env.reset()\n\n```\n\n## Example: Reinforcement Learning\n```python\nimport gym\nimport ballbeam_gym\nfrom stable_baselines.common.policies import MlpPolicy\nfrom stable_baselines.common.vec_env import DummyVecEnv\nfrom stable_baselines import PPO2\n\n# pass env arguments as kwargs\nkwargs = {'timestep': 0.05, \n          'setpoint': 0.4,\n          'beam_length': 1.0,\n          'max_angle': 0.2,\n          'init_velocity': 0.0,\n          'action_mode': 'discrete'}\n\n# create env\nenv = gym.make('BallBeamSetpoint-v0', **kwargs)\n\n# train a mlp policy agent\nenv = DummyVecEnv([lambda: env])\nmodel = PPO2(MlpPolicy, env, verbose=1)\nmodel.learn(total_timesteps=20000)\n\nobs = env.reset()\nenv.render()\n\n# test agent on 1000 steps\nfor i in range(1000):\n    action, _ = model.predict(obs)\n    obs, reward, done, info = env.step(action)\n    env.render()\n    if done:\n        env.reset()",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/simon-larsson/ballbeam-gym",
        "keywords": "",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "ballbeam-gym",
        "package_url": "https://pypi.org/project/ballbeam-gym/",
        "platform": "",
        "project_url": "https://pypi.org/project/ballbeam-gym/",
        "project_urls": {
            "Homepage": "https://github.com/simon-larsson/ballbeam-gym"
        },
        "release_url": "https://pypi.org/project/ballbeam-gym/0.0.4/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Ball & beam environments for OpenAI gym",
        "version": "0.0.4"
    },
    "last_serial": 5645326,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9a3e63618c57b130e8a18cbfd0996a83",
                    "sha256": "656ebcb2ad365cbe85d7ec115269db2aeed025d6f6d7da47b51e7c09262614cd"
                },
                "downloads": -1,
                "filename": "ballbeam_gym-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "9a3e63618c57b130e8a18cbfd0996a83",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5220,
                "upload_time": "2019-07-23T08:18:46",
                "url": "https://files.pythonhosted.org/packages/a7/50/4604d3adf8bd9e140701ad2ca7fa49822ae0d4cbc62a845a196bcd673c3c/ballbeam_gym-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "450cd77ef5d341813de0b9c675f8d899",
                    "sha256": "df8647253e06cbb45eb12d1a6503288a07a129d50c87676c7fa4abe0b4f327ce"
                },
                "downloads": -1,
                "filename": "ballbeam_gym-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "450cd77ef5d341813de0b9c675f8d899",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7315,
                "upload_time": "2019-07-26T09:41:49",
                "url": "https://files.pythonhosted.org/packages/52/a1/2b95d63a147faf464de09d159507deb0124f114408aff6658223be8be669/ballbeam_gym-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3b91dd1666c5e790ec2105fccaa8f543",
                    "sha256": "b3895898a61fb28239118c73852ef22c933e3e74cf0ce82840da8f4bec5112b9"
                },
                "downloads": -1,
                "filename": "ballbeam_gym-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "3b91dd1666c5e790ec2105fccaa8f543",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7847,
                "upload_time": "2019-08-06T11:12:13",
                "url": "https://files.pythonhosted.org/packages/00/95/fc0ad3abb21fb74e5b89e2c153e4179cbd4589bf8227f16fd931eb72fdb6/ballbeam_gym-0.0.3.tar.gz"
            }
        ],
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "72ebab3536219d6752c184c6166d7dbf",
                    "sha256": "9b89c20b22503cdfe774ca6ef1559c6e4a8a430e8d807a883772285ba2aba8fd"
                },
                "downloads": -1,
                "filename": "ballbeam_gym-0.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "72ebab3536219d6752c184c6166d7dbf",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8477,
                "upload_time": "2019-08-07T14:30:35",
                "url": "https://files.pythonhosted.org/packages/09/84/41352d9e9d166757572fdeb5fd8f452e7f2a0d6fdd5d4e48d6b198b41b32/ballbeam_gym-0.0.4.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "72ebab3536219d6752c184c6166d7dbf",
                "sha256": "9b89c20b22503cdfe774ca6ef1559c6e4a8a430e8d807a883772285ba2aba8fd"
            },
            "downloads": -1,
            "filename": "ballbeam_gym-0.0.4.tar.gz",
            "has_sig": false,
            "md5_digest": "72ebab3536219d6752c184c6166d7dbf",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 8477,
            "upload_time": "2019-08-07T14:30:35",
            "url": "https://files.pythonhosted.org/packages/09/84/41352d9e9d166757572fdeb5fd8f452e7f2a0d6fdd5d4e48d6b198b41b32/ballbeam_gym-0.0.4.tar.gz"
        }
    ]
}