{
    "info": {
        "author": "Ken Elkabany",
        "author_email": "ken@elkabany.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Natural Language :: English",
            "Operating System :: POSIX :: Linux",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7",
            "Topic :: System :: Installation/Setup",
            "Topic :: System :: Systems Administration"
        ],
        "description": "# Marchitect [![Latest Version]][PyPI]\n\n[Latest Version]: https://img.shields.io/pypi/v/marchitect.svg\n[PyPI]: https://pypi.org/project/marchitect/\n\nA tool for uploading files to and running commands on remote hosts.\n\n## Install\n\n```bash\n$ pip3 install marchitect\n```\n\n## Example\n\nLet's write a couple of files to your machine assuming that it could easily be\nmade to be a remote machine.\n\n```python\nfrom marchitect.site_plan import SitePlan, Step\nfrom marchitect.whiteprint import Whiteprint\n\nclass HelloWorldWhiteprint(Whiteprint):\n\n    name = 'hello_world'  \n\n    def _execute(self, mode: str) -> None:\n        if mode == 'install':\n            # Write file by running remote shell commands.\n            self.exec('echo \"hello, world.\" > /tmp/helloworld1')\n            # Write file by uploading\n            self.scp_up_from_bytes(\n                b'hello, world.', '/tmp/helloworld2')\n\nclass MyMachine(SitePlan):\n    plan = [\n        Step(HelloWorldWhiteprint)\n    ]\n\nif __name__ == '__main__':\n    # SSH into your own machine, prompting you for your password.\n    import getpass\n    import os\n    user = os.getlogin()\n    password = getpass.getpass('%s@localhost password: ' % user)\n    sp = MyMachine.from_password('localhost', 22, user, password, {}, [])\n    # If you want to auth by private key, use the below:\n    # (Note: The password prompt will be for your private key, empty for none.)\n    #sp = MyMachine.from_private_key(\n    #    'localhost', 22, user, '/home/%s/.ssh/id_rsa' % user, password, {}, [])\n    sp.install()  # Sets the mode of _execute() to install.\n```\n\nThis example requires that you can SSH into your machine via password. To use\nyour SSH key instead, uncomment the lines above. After execution, you should\nhave `/tmp/helloworld1` and `/tmp/helloworld2` on your machine.\n\nHopefully it's clear that whiteprints let you run commands and upload files to a\ntarget machine. A whiteprint should contain all the operations for a common\npurpose. A site plan contains all the whiteprints that should be run on a\nsingle machine class.\n\nSteps for deploying your code repository to a machine would make for a good\nwhiteprint. A site plan for a machine that runs your web servers might use that\nwhiteprint and others.\n\n## Goals\n\n* Easy to get started.\n* Templating of configuration files.\n* Mix of imperative and declarative styles.\n* Arbitrary execution modes (install, update, clean, start, stop, ...).\n* Interface for validating machine state.\n* Be lightweight because most complex configurations are happening in\n  containers anyway.\n\n## Non-goals\n\n* Making whiteprints and site plans share-able with other people and companies.\n* Non-Linux deployment targets.\n\n## Concepts\n\n### Whiteprint\n\nTo create a whiteprint, extend `Whiteprint` and define a `name` class variable\nand an `_execute()` method; optionally define a `validate()` method. `name`\nshould be a reasonable name for the whiteprint. In the example above, the\n`HelloWorldWhiteprint` class's name is simply `hello_world`. `name` is\nimportant for file resolution which is discussed below.\n\n`_execute()` is where all the magic happens. The method takes a string called\n`mode`. Out of convention, your whiteprints should handle the following modes:\n\n* `install` (installing software)\n* `update` (updating software)\n* `clean` (removing software, if needed)\n* `start` (starting services)\n* `stop` (stopping services).\n\nDespite this convention, `mode` can be anything as you'll choosing the modes\nto execute your site plans with.\n\nWithin `_execute()`, you're given all the freedom to shoot yourself in the\nfoot. Use `self.exec()` to run any command on the target machine.\n\n`exec()` returns an `ExecOutput` object with variables `exit_status` (int),\n`stdout` (bytes), and `stderr` (bytes). You can use these outputs to control\nflow. If the exit status is non-zero, a `RemoteExecError` is raised. To\nsuppress the exception, set `error_ok=True`.\n\n`_execute()` has access to `self.cfg` which are the config variables for the\nwhiteprint. See the Templates & Config Vars section below.\n\nUse the variety of functions to copy files to and from the host:\n\n* `scp_up()` - Upload a file from the local host to the target.\n* `sp_up_from_bytes()` - Create a file on the target host from the bytes arg.\n* `scp_down()` - Download a file from the target to the local host.\n* `scp_down_to_bytes()` - Download a file from the target and return it.\n\n#### Templates & Config Vars\n\nYou can upload files that are [jinja2](http://jinja.pocoo.org) templates. The\ntemplates will be filled by the config variables passed to the whiteprint.\nConfig variables can be set in numerous ways, which we'll now explore.\n\nHere's a sample `test.toml` file that uses the jinja2 notation to specify a\nname variable with a default of `John Doe`:\n\n```toml\nname = \"{{ name|default('John Doe') }}\"\n```\n\nA whiteprint can populate a template for upload as follows:\n\n```python\nfrom marchitect.whiteprint import Whiteprint\n\nclass WhiteprintExample(Whiteprint):\n\n    default_cfg = {'name': 'Alice'}\n\n    def _execute(self, mode: str) -> None:\n        if mode == 'install':\n            self.scp_up_template('/path/to/test.toml', '~/test.toml')\n```\n\nA whiteprint can also upload a populated template that's stored in a string\nrather than a file:\n\n```python\nfrom marchitect.whiteprint import Whiteprint\n\nclass WhiteprintExample(Whiteprint):\n\n    default_cfg = {'name': 'Alice'}\n\n    def _execute(self, mode: str) -> None:\n        if mode == 'install':\n            self.scp_up_template_from_str(\n                'name = \"{{ name }}\"', '~/test.toml')\n```\n\nA config var can be overriden in `scp_up_template_from_str`:\n\n```python\nfrom marchitect.whiteprint import Whiteprint\n\nclass WhiteprintExample(Whiteprint):\n\n    default_cfg = {'name': 'Alice'}\n\n    def _execute(self, mode: str) -> None:\n        if mode == 'install':\n            # 'Bob' overrides 'Alice'\n            self.scp_up_template_from_str(\n                'name = \"{{ name }}\"', '~/test.toml',\n                cfg_override={'name': 'Bob'})\n```\n\nConfig vars can also be set by the `SitePlan` in the plan or during\ninstantiation.\n\n```python\nfrom marchitect.site_plan import Step, SitePlan\n\nclass MyMachine(SitePlan):\n    plan = [\n        Step(WhiteprintExample, {'name': 'Eve'})\n    ]\n\nif __name__ == '__main__':\n    MyMachine.from_password(..., cfg_override={'name': 'Foo'})\n```\n\nIn the above, `Foo` takes precedence over `Eve` which takes precedence over any\nvalues for `name` defined in the whiteprint.\n\nFinally, a `Step` can be given an alias as another identifier for specifying\nconfig vars. This is useful when a whiteprint is used multiple times in a site\nplan.\n\n```python\nfrom marchitect.site_plan import Step, SitePlan\n\nclass MyMachine(SitePlan):\n    plan = [\n        Step(WhiteprintExample, alias=\"ex1\"),\n        Step(WhiteprintExample, alias=\"ex2\"),\n    ]\n\nif __name__ == '__main__':\n    MyMachine.from_password(..., cfg_override={'ex1': 'Eve', 'ex2': 'Foo'})\n```\n\nIn the above, the first `WhiteprintExample` uploads `Eve` and the second\nreplaces it with `Foo`.\n\nThere are also config variables that are auto-derived and always available.\nThese are stored in `self.cfg['_target']`:\n\n* `user`: The login user for the SSH connection.\n* `host`: The target host for the SSH connection.\n* `kernel`: The kernel version of the target host. Ex: `4.15.0-43-generic`\n* `distro`: The Linux distribution of the target host. Ex: `ubuntu`\n* `disto_version`: The version of the Linux distribution. Ex: `18.04`\n* `hostname`: The hostname of the target host.\n* `fqdn`: The fully-qualified domain name of the target host.\n* `cpu_count`: The number of CPUs on the target host. Ex: `8`\n\n\n#### File Resolution\n\nMethods that upload local files (`scp_up()` and `scp_up_template()`) will\nsearch for the files according to the `rsrc_paths` argument in the `SitePlan`\nconstructor. The search proceeds in order of the `rsrc_paths` and the name of\nthe whiteprint is expected to be the name of a subfolder in the `rsrc_path`.\n\nFor example, assume `rsrc_paths` is `[Path('/srv/rsrcs')]`, the whiteprint\nhas a name of `foobar`, and the file `c` is referenced as `a/b/c`. The resolver\nwill look for the existence of `/srv/rsrcs/foobar/a/b/c`.\n\nIf a file path is specified as absolute, say `/a/b/c`, no `rsrc_path` will be\nprefixed. However, this form is not encouraged for portability across machines\nas resources may live in different folders on different machines.\n\n#### Idempotence\n\nIt's important to strive for the idempotence of your whiteprints. In other\nwords, assume your whiteprint in any mode (install, update, ...) can be\ninterrupted at any point. Can your whiteprint be re-applied successfully\nwithout any problems?\n\nIf so, your whiteprint is idempotent and is therefore resilient to connection\nerrors and software hiccups. Error handling will be as easy as retrying your\nwhiteprint a bounded number of times. If not, you'll need to figure out an\nerror handling strategy. In the extreme case, you can terminate servers that\nproduce errors and start over with a fresh one, assuming that you're in a cloud\nenvironment.\n\n### Prefab\n\nPrefabs are built-in idempotent components you can add to your whiteprints.\nThese make it easy to add common functionality with the `_execute()` and\n`_validate()` methods already defined. These are available out-of-thebox:\n\n* `Apt`: Common Linux package manager.\n* `Pip3`: Python package manager.\n* `FolderExists`: Ensures a folder exists at the specified path.\n* `LineInFile`: Ensures the specified line exists in the specified file.\n\nAn example:\n\n```python\nfrom marchitect.prefab import Apt\nfrom marchitect.whiteprint import Whiteprint\n\nclass HelloWorld2Whiteprint(Whiteprint):\n\n    prefabs = [\n        Apt(['curl']),\n    ]\n\n    def _execute(self, mode: str) -> None:\n        if mode == 'install':\n            self.exec('curl https://www.nytimes.com > /tmp/nytimes')\n```\n\nPrefabs are executed before your `_execute()` method.\n\nIf a prefab depends on a config variable, define a `_compute_prefabs()` class\nmethod:\n\n```python\nfrom typing import Any, Dict, List\nfrom marchitect.prefab import FolderExists, Prefab\nfrom marchitect.whiteprint import Whiteprint\n\nclass ExampleWhiteprint(Whiteprint):\n\n    @classmethod\n    def _compute_prefabs(cls, cfg: Dict[str, Any]) -> List[Prefab]:\n        return [FolderExists(cfg['temp_folder'])]\n```\n\nThe prefabs returned by`_compute_prefabs()` will be executed after those\nspecified in the `prefabs` class variable.\n\n### Site Plan\n\nSite plans are collections of whiteprints. You likely have distinct roles for\nthe machines in your infrastructure: web hosts, api hosts, database hosts, ...\nEach of these should map to their own site plan which will install the\nappropriate whiteprints (postgres for database hosts, uwsgi for web hosts, ...).\n\n## Testing\n\nTests are run against real SSH connections, which unfortunately makes it\ndifficult to run in a CI environment. Travis CI is not enabled for this reason.\nWhen running tests through `py.test` or `tox`, you can specify SSH credentials\neither as a user/pass pair or user/private_key. For example:\n\n```\nSSH_USER=username SSH_HOST=localhost SSH_PRIVATE_KEY=~/.ssh/id_rsa SSH_PRIVATE_KEY_PASSWORD=*** py.test\nSSH_USER=username SSH_HOST=localhost SSH_PASSWORD=*** py.test -sx\n```\n\nWill likely move to mocking SSH commands, but it will be painful to reliably\nmock the interfaces for `ssh2-python`.\n\n`mypy` and `lint` are also supported: `tox -e mypy,lint`\n\n## TODO\n* [] Add \"common\" dependencies to minimize invocations of commands like\n  `apt update` to once per site plan.\n* [] Write a log of applied site plans and whiteprints to the target host\n  for easy debugging.\n* [] Add documentation for `validate()` method.\n* [] Verify speed wins by using `ssh2-python` instead of `paramiko`.\n* [] Document `SitePlan.one_off_exec()`.\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://www.github.com/kelkabany/marchitect",
        "keywords": "",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "marchitect",
        "package_url": "https://pypi.org/project/marchitect/",
        "platform": "",
        "project_url": "https://pypi.org/project/marchitect/",
        "project_urls": {
            "Homepage": "https://www.github.com/kelkabany/marchitect"
        },
        "release_url": "https://pypi.org/project/marchitect/0.3/",
        "requires_dist": [
            "jinja2 (>=2.10)",
            "ssh2-python (>=0.17.0)"
        ],
        "requires_python": "",
        "summary": "Machine architect for software deployment.",
        "version": "0.3"
    },
    "last_serial": 5402281,
    "releases": {
        "0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9f164d09c5f102eabde5ba4932b3f600",
                    "sha256": "d8083412633a6b7ccd464c39ac047e2b8a487955d00941e9f40ca3e2a8150219"
                },
                "downloads": -1,
                "filename": "marchitect-0.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9f164d09c5f102eabde5ba4932b3f600",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 15102,
                "upload_time": "2019-06-14T07:04:11",
                "url": "https://files.pythonhosted.org/packages/7c/8d/2573d10d8b08831b0eac8736784c75aab9fbf4acfc51df07b3a6c6e1667a/marchitect-0.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "1643a60b17965f59ff389a7cb04311f2",
                    "sha256": "ad7ce4d146d89b0b614e7d310e69d6ab360df6acc16709246514c8371647a3c4"
                },
                "downloads": -1,
                "filename": "marchitect-0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "1643a60b17965f59ff389a7cb04311f2",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17939,
                "upload_time": "2019-06-14T07:04:13",
                "url": "https://files.pythonhosted.org/packages/2c/ea/7c5186017c2f822f39b486db137862bdf2d58efc8d15f17f05f8e54149bd/marchitect-0.2.tar.gz"
            }
        ],
        "0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b3c6fca6824fed2d3ef24f8c3b10b9d9",
                    "sha256": "9ff018d44dba59a5bc5863d486e7034e6653bfa3593330e51050ca5075a98810"
                },
                "downloads": -1,
                "filename": "marchitect-0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b3c6fca6824fed2d3ef24f8c3b10b9d9",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 15959,
                "upload_time": "2019-06-14T21:43:56",
                "url": "https://files.pythonhosted.org/packages/da/fc/c8212d7c4938b27507744af2eb76ef6a7b96d71b9be80189a5e2ecf128bc/marchitect-0.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3c0bb21876d9264dfc1610d42b6cd436",
                    "sha256": "327aa1ecb622c5eb1b6bc5d78859012770dff8485b33326a3e3f0f182af10c3f"
                },
                "downloads": -1,
                "filename": "marchitect-0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "3c0bb21876d9264dfc1610d42b6cd436",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17993,
                "upload_time": "2019-06-14T21:43:59",
                "url": "https://files.pythonhosted.org/packages/2f/82/7d0a2f57d86faf8ba5b15dfcdb2dd9a76a0c86a0c4bbacc7705eaa089d5b/marchitect-0.3.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "b3c6fca6824fed2d3ef24f8c3b10b9d9",
                "sha256": "9ff018d44dba59a5bc5863d486e7034e6653bfa3593330e51050ca5075a98810"
            },
            "downloads": -1,
            "filename": "marchitect-0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "b3c6fca6824fed2d3ef24f8c3b10b9d9",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 15959,
            "upload_time": "2019-06-14T21:43:56",
            "url": "https://files.pythonhosted.org/packages/da/fc/c8212d7c4938b27507744af2eb76ef6a7b96d71b9be80189a5e2ecf128bc/marchitect-0.3-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "3c0bb21876d9264dfc1610d42b6cd436",
                "sha256": "327aa1ecb622c5eb1b6bc5d78859012770dff8485b33326a3e3f0f182af10c3f"
            },
            "downloads": -1,
            "filename": "marchitect-0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "3c0bb21876d9264dfc1610d42b6cd436",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 17993,
            "upload_time": "2019-06-14T21:43:59",
            "url": "https://files.pythonhosted.org/packages/2f/82/7d0a2f57d86faf8ba5b15dfcdb2dd9a76a0c86a0c4bbacc7705eaa089d5b/marchitect-0.3.tar.gz"
        }
    ]
}