{ "info": { "author": "Propertyshelf, Inc.", "author_email": "development@propertyshelf.com", "bugtrack_url": null, "classifiers": [ "Programming Language :: Python" ], "description": "mls.fabfile\n===========\n\nThis project contains a bunch of fabric commands we use at `Propertyshelf`_\nto deploy and maintain our MLS systems.\n\n\nRequirements\n------------\n\n`mls.fabfile` currently uses `knife`_ to communicate with Rackspace servers.\nPlease make sure `knife` is installed and configured successfully on your\nsystem.\n\n\nInstall\n-------\n\nYou can install `mls.fabfile` with PIP::\n\n pip install mls.fabfile\n\nAll required dependencies will be installed automatically.\n\n\nUsage\n-----\n\nFirst, we need a working `knife.rb` file to interact with our Chef server and\nthe Rackspace cloud eco system. Below is an example `knife.rb` file that gets\nall its required info from environment variables. This way you can add this\n`knife.rb` file inside a `.chef` directory to your project and savely put it\nunder version control::\n\n # Logging.\n log_level :info\n log_location STDOUT\n\n # Chef server configuration.\n chef_server_url \"#{ENV['KNIFE_CHEF_SERVER']}\"\n client_key \"#{ENV['KNIFE_CLIENT_KEY']}\"\n node_name \"#{ENV['KNIFE_NODE_NAME']}\"\n validation_client_name \"#{ENV['KNIFE_VALIDATION_CLIENT_NAME']}\"\n validation_key \"#{ENV['KNIFE_VALIDATION_CLIENT_KEY']}\"\n encrypted_data_bag_secret \"#{ENV['ENCRYPTED_DATA_BAG_SECRET_FILE']}\"\n\n # Rackspace API configuration.\n knife[:rackspace_api_key] = \"#{ENV['RACKSPACE_API_KEY']}\"\n knife[:rackspace_api_username] = \"#{ENV['RACKSPACE_USERNAME']}\"\n knife[:rackspace_endpoint] = \"#{ENV['RACKSPACE_ENDPOINT']}\"\n knife[:rackspace_version] = \"#{ENV['RACKSPACE_VERSION']}\"\n\n\nNext, we need a `fabfile.py`. All we need to do is to import `mls.fabfile`\nto make the fabric commands available and the available environments we can\nwork with from `propertyshelf.fabfile.common`.::\n\n # -*- coding: utf-8 -*-\n \"\"\"Sample MLS deployment script.\"\"\"\n\n from fabric import api\n\n from mls.fabfile import *\n from propertyshelf.fabfile.common.environments import *\n\n\n # Definition of role names to be used.\n api.env.role_database = 'mls_db'\n api.env.role_frontend = 'mls_fe'\n api.env.role_staging = 'mls_staging'\n api.env.role_worker = 'mls_app'\n\n # Definition of used Rackspace flavors (server sized) for our servers.\n api.env.flavor_database = '5'\n api.env.flavor_frontend = '2'\n api.env.flavor_staging = '2'\n api.env.flavor_worker = '3'\n\n # Definition of node names to be used.\n api.env.nodename_database = 'mls-db'\n api.env.nodename_frontend = 'mls-fe'\n api.env.nodename_staging = 'mls-staging'\n api.env.nodename_worker = 'mls-app'\n\n # The Rackspace server image we use. This is a Debian 6.0.6.\n api.env.os_image = '92a28e50-181d-4fc7-a071-567d26fc95f6'\n\n # MLS specific configuration.\n api.env.domain = 'mls-example.com'\n api.env.mls_customizations = ['mlsext.realtorcom', ]\n api.env.mls_policy_enabled = True\n api.env.mls_policy_package = 'mlspolicy.example'\n api.env.mls_policy_package_url = 'git https://github.com/yourname/mlspolicy.example'\n\nYou can now use fabric to manage your MLS::\n\n $ fab -l\n Sample MLS deployment script.\n\n Available commands:\n\n development Work locally with vagrant.\n production Work with the production environment.\n staging Work with the staging environment.\n bootstrap.bundle_db_fe_app Bootstrap a new MLS bundle: Database, Frontend, App Worker.\n bootstrap.database Bootstrap a new standalone database server.\n bootstrap.frontend Bootstrap a new standalone frontend server.\n bootstrap.staging Bootstrap a staging system.\n bootstrap.worker Bootstrap a new standalone application worker.\n client.remove Remove an existing MLS application client.\n client.restart Restart the application client component.\n client.update Update the client packages.\n database.backup Perform a backup of Zope's data on the server.\n database.download_blobs Download blob part of Zope's data from the server.\n database.download_data Download the database files from the server.\n database.download_zodb Download ZODB part of Zope's data from the server.\n database.restart Restart the database component.\n database.restore Restore an existing backup of Zope's data on the server.\n database.upload_blob Upload blob part of Zope's data to the server.\n database.upload_data Upload the database files to the server.\n database.upload_zodb Upload ZODB part of Zope's data to the server.\n frontend.restart Restart the frontend components.\n frontend.restart_haproxy Restart the HA-Proxy load balancer component.\n frontend.restart_nginx Restart the NginX web server component.\n frontend.restart_varnish Restart the Varnish caching proxy component.\n roles.check Check if the required roles are available.\n roles.create_missing Create missing roles on the chef server.\n\nBefore we can start it is a good idea to check if all roles we defined are\navailable on the chef server::\n\n $ fab roles.check\n Role mls_fe NOT available.\n Role mls_db NOT available.\n Role mls_staging NOT available.\n Role mls_app NOT available.\n\n Done.\n\nTo create the missing roles based on our configuration, we simply have to do::\n\n $ fab roles.create_missing\n Created role mls_db\n Created role mls_fe\n Created role mls_app\n Created role mls_staging\n\n Done.\n\nNow we can create our staging system::\n\n $ fab bootstrap.staging\n [localhost] local: knife rackspace server create -S mls-staging -N mls-staging -f 5 -I 92a28e50-181d-4fc7-a071-567d26fc95f6 -r role[rackspace],role[mls_staging] -E staging\n\n ...\n\n Done.\n\nNote that there can only be one staging system. If you try to add another one\nwith the same name, you'll get an error message::\n\n $ fab bootstrap.staging\n\n Fatal error: Server \"mls-staging\" already exists in environment \"staging\".\n\n Aborting.\n\nIf you need a second one, you can adjust the node name manually::\n\n $ fab bootstrap.staging:nodename=mls-staging2\n [localhost] local: knife rackspace server create -S mls-staging2 -N mls-staging2 -f 5 -I 92a28e50-181d-4fc7-a071-567d26fc95f6 -r role[rackspace],role[mls_ni_staging] -E staging\n\n ...\n\n Done.\n\nYou can now manage the single components::\n\n $ fab staging frontend.restart\n [x.x.x.x] Executing task 'frontend.restart'\n [x.x.x.x] sudo: /etc/init.d/haproxy restart\n [x.x.x.x] out: sudo password:\n\n [x.x.x.x] out: Restarting haproxy: haproxy.\n [x.x.x.x] out:\n\n [x.x.x.x] sudo: /etc/init.d/varnish restart\n [x.x.x.x] out: sudo password:\n [x.x.x.x] out: Stopping HTTP accelerator: varnishd.\n [x.x.x.x] out: Starting HTTP accelerator: varnishd.\n [x.x.x.x] out:\n\n [x.x.x.x] sudo: /etc/init.d/nginx restart\n [x.x.x.x] out: sudo password:\n [x.x.x.x] out: Restarting nginx: nginx.\n [x.x.x.x] out:\n\n\n Done.\n Disconnecting from x.x.x.x... done.\n\nWe also support download of the database files for local testing::\n\n $ fab production database.download_data\n [x.x.x.x] Executing task 'database.download_data'\n This will overwrite your local Data.fs. Are you sure you want to continue? [Y/n]\n [localhost] local: mkdir -p var/filestorage\n [localhost] local: mv var/filestorage/Data.fs var/filestorage/Data.fs.bak\n [x.x.x.x] out: sudo password:\n [x.x.x.x] sudo: rsync -a var/filestorage/Data.fs /tmp/Data.fs\n [x.x.x.x] out: sudo password:\n [x.x.x.x] out:\n [x.x.x.x] download: /Volumes/Work/Propertyshelf/MLS/Provisioning/var/filestorage/Data.fs <- /tmp/Data.fs\n This will overwrite your local blob files. Are you sure you want to continue? [Y/n]\n [localhost] local: rm -rf var/blobstorage_bak\n [localhost] local: mv var/blobstorage var/blobstorage_bak\n [x.x.x.x] sudo: rsync -a ./var/blobstorage /tmp/\n [x.x.x.x] out: sudo password:\n [x.x.x.x] out:\n [x.x.x.x] sudo: tar czf blobstorage.tgz blobstorage\n [x.x.x.x] out: sudo password:\n [x.x.x.x] out:\n [x.x.x.x] download: /Volumes/Work/Propertyshelf/MLS/Provisioning/var/blobstorage.tgz <- /tmp/blobstorage.tgz\n\n Warning: Local file /Volumes/Work/Propertyshelf/MLS/Provisioning/var/blobstorage.tgz already exists and is being overwritten.\n\n [localhost] local: tar xzf blobstorage.tgz\n\n Done.\n Disconnecting from x.x.x.x... done.\n\nOnce we have local data files, we can upload them to our development environment\n(a vagrant VM)::\n\n $ fab development database.upload_data client.restart\n [localhost] local: vagrant ssh-config | grep IdentityFile\n [127.0.0.1:2222] Executing task 'database.upload_data'\n This will overwrite your remote Data.fs. Are you sure you want to continue? [y/N] y\n [127.0.0.1:2222] sudo: mkdir -p /tmp/upload\n [127.0.0.1:2222] put: var/filestorage/Data.fs -> /tmp/upload/Data.fs\n [127.0.0.1:2222] sudo: chown mls /tmp/upload/Data.fs\n [127.0.0.1:2222] sudo: supervisorctl stop zeoserver\n [127.0.0.1:2222] out: zeoserver: stopped\n [127.0.0.1:2222] out:\n\n [127.0.0.1:2222] sudo: mv var/filestorage/Data.fs var/filestorage/Data.fs.bak\n [127.0.0.1:2222] sudo: mv /tmp/upload/Data.fs var/filestorage/Data.fs\n This will overwrite your remote blob files. Are you sure you want to continue? [y/N] y\n [127.0.0.1:2222] sudo: mkdir -p /tmp/upload\n [localhost] local: tar czf blobstorage_upload.tgz blobstorage\n [127.0.0.1:2222] put: var/blobstorage_upload.tgz -> /tmp/upload/blobstorage.tgz\n [127.0.0.1:2222] sudo: chown mls /tmp/upload/blobstorage.tgz\n [127.0.0.1:2222] sudo: tar xzf blobstorage.tgz\n [127.0.0.1:2222] sudo: supervisorctl stop zeoserver\n [127.0.0.1:2222] out: zeoserver: ERROR (not running)\n [127.0.0.1:2222] out:\n\n [127.0.0.1:2222] sudo: mv var/blobstorage var/blobstorage_bak\n [127.0.0.1:2222] sudo: mv /tmp/upload/blobstorage var\n [127.0.0.1:2222] sudo: supervisorctl start zeoserver\n [127.0.0.1:2222] out: zeoserver: started\n [127.0.0.1:2222] out:\n\n [127.0.0.1:2222] Executing task 'client.restart'\n [127.0.0.1:2222] sudo: supervisorctl restart application\n [127.0.0.1:2222] out: application: stopped\n [127.0.0.1:2222] out: application: started\n [127.0.0.1:2222] out:\n\n\n Done.\n Disconnecting from 127.0.0.1:2222... done.\n\nWe can also get a list of nodes for already defined roles::\n\n $ fab roles.list_nodes\n Role: mls_fe\n - mls-fe: x.x.x.x\n\n Role: mls_db\n - mls-db: x.x.x.x\n\n Role: mls_staging\n - mls-staging: x.x.x.x\n - vagrant-mls-staging: 10.0.2.15\n\n Role: mls_app\n - mls-app-01: x.x.x.x\n\n\n Done.\n\nThis can be useful if we want to execute a task only for a given node::\n\n $ fab frontend.restart_nginx:hosts=x.x.x.x\n [x.x.x.x] Executing task 'frontend.restart_nginx'\n [x.x.x.x] sudo: /etc/init.d/nginx restart\n [x.x.x.x] out: sudo password:\n [x.x.x.x] out: Restarting nginx: nginx.\n [x.x.x.x] out:\n\n\n Done.\n Disconnecting from x.x.x.x... done.\n\n\n.. _`Propertyshelf`: http://propertyshelf.com\n.. _`knife`: http://docs.opscode.com/knife.html\n\nContributors\n============\n\nThomas Massmann, Author\n\nChangelog\n=========\n\n0.4 (2014-04-10)\n----------------\n\n- Added snapshotbackup option for database roles.\n- Ability to update the maintenance client as well.\n\n\n0.3 (2013-12-21)\n----------------\n\n- Moved common functionality to propertyshelf.fabfile.common.\n- New tasks: rebuild client, list nodes for roles.\n\n\n0.2 (2013-11-19)\n----------------\n\n- Set custom hostname for development environment so chef search works.\n- Enabled data upload (Data.fs and blobs).\n\n\n0.1.1 (2013-11-19)\n------------------\n\n- Added missing files.\n\n\n0.1 (2013-11-19)\n----------------\n\n- Work with development, staging and production environments.\n- Check for chef roles and create missing ones from configurations.\n- Bootstrap MLS systems.\n- Remove MLS clients if not needed any more.\n- Manage database, frontend and worker components.\n- Update client buildout packages.\n- Download database files for local testing.", "description_content_type": null, "docs_url": null, "download_url": "http://pypi.python.org/pypi/mls.fabfile", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/propertyshelf/mls.fabfile", "keywords": "fabric python mls deployment", "license": "BSD", "maintainer": null, "maintainer_email": null, "name": "mls.fabfile", "package_url": "https://pypi.org/project/mls.fabfile/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/mls.fabfile/", "project_urls": { "Download": "http://pypi.python.org/pypi/mls.fabfile", "Homepage": "https://github.com/propertyshelf/mls.fabfile" }, "release_url": "https://pypi.org/project/mls.fabfile/0.4/", "requires_dist": null, "requires_python": null, "summary": "Deploy and manage Propertyshelf MLS applications using Fabric.", "version": "0.4" }, "last_serial": 1832853, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "8c248bd11a2cff9e49d9b0651f9ea49a", "sha256": "f73d33c0b5cb3074c20b6082ce85e4bafdcda9264bca2602fc7aa8f1f58b6637" }, "downloads": -1, "filename": "mls.fabfile-0.1.tar.gz", "has_sig": false, "md5_digest": "8c248bd11a2cff9e49d9b0651f9ea49a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11622, "upload_time": "2013-11-19T09:41:15", "url": "https://files.pythonhosted.org/packages/db/dc/150d6c25481b9a6c70e074bd36bc625a2ff82c73022d3a226d653012ed4e/mls.fabfile-0.1.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "b4aa180faf1baa25668c9e3550a9f7c6", "sha256": "4ab778318ca9f329c8015b2b23372499da4199f09c99ea7ec2de249c7b4704d9" }, "downloads": -1, "filename": "mls.fabfile-0.1.1.tar.gz", "has_sig": false, "md5_digest": "b4aa180faf1baa25668c9e3550a9f7c6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11794, "upload_time": "2013-11-19T09:48:34", "url": "https://files.pythonhosted.org/packages/52/50/f75b06cbcb741a1b8ed1ebe6f3724d27ba9716211d875c0ba7288b51a0b9/mls.fabfile-0.1.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "9504a4ac4655b0cdc9e55d6c3b6720f8", "sha256": "76c93b2864efaf7b2c6ea3ce11c3307822d5b1af8b05e84fc4406e818d9df0a8" }, "downloads": -1, "filename": "mls.fabfile-0.2.tar.gz", "has_sig": false, "md5_digest": "9504a4ac4655b0cdc9e55d6c3b6720f8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13474, "upload_time": "2013-11-19T16:26:28", "url": "https://files.pythonhosted.org/packages/cc/19/73c953bbf927ba1b1c58b254cd3570d7a9b8276bee41b32f47a8ebe38640/mls.fabfile-0.2.tar.gz" } ], "0.3": [ { "comment_text": "", "digests": { "md5": "2b964f257eee540c6a0c653edbed0225", "sha256": "849feb462ed503de12b8663b7ffb1da242192d158be041862b840aafb9b4fa70" }, "downloads": -1, "filename": "mls.fabfile-0.3.tar.gz", "has_sig": false, "md5_digest": "2b964f257eee540c6a0c653edbed0225", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11133, "upload_time": "2013-12-21T09:50:00", "url": "https://files.pythonhosted.org/packages/69/8c/331eb6ee7f51a4f26fcb6bae820f407940c83393808946f1f44a6509c5d0/mls.fabfile-0.3.tar.gz" } ], "0.4": [ { "comment_text": "", "digests": { "md5": "b31b018577a724fe5ed64f795e009ac2", "sha256": "8502612d71f0c5fb1cabde1e8cd622122ec9265c6ef2413d2897b3c01b3755a3" }, "downloads": -1, "filename": "mls.fabfile-0.4.tar.gz", "has_sig": false, "md5_digest": "b31b018577a724fe5ed64f795e009ac2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11304, "upload_time": "2014-04-10T14:29:43", "url": "https://files.pythonhosted.org/packages/ce/80/83e3506eef52243bfb65f77dd3dc1fe5076e6f7491a6352059f3c2812b48/mls.fabfile-0.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "b31b018577a724fe5ed64f795e009ac2", "sha256": "8502612d71f0c5fb1cabde1e8cd622122ec9265c6ef2413d2897b3c01b3755a3" }, "downloads": -1, "filename": "mls.fabfile-0.4.tar.gz", "has_sig": false, "md5_digest": "b31b018577a724fe5ed64f795e009ac2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11304, "upload_time": "2014-04-10T14:29:43", "url": "https://files.pythonhosted.org/packages/ce/80/83e3506eef52243bfb65f77dd3dc1fe5076e6f7491a6352059f3c2812b48/mls.fabfile-0.4.tar.gz" } ] }