Metadata-Version: 1.1
Name: p01.build
Version: 0.5.1
Summary: A build system supporting application deploy and installation
Home-page: http://pypi.python.org/pypi/p01.build
Author: Roger Ineichen, Projekt01 GmbH
Author-email: dev@projekt01.ch
License: ZPL 2.1
Description: This package provides a product build, release and deploy system based on
        package versions using zc.buildout. The core concept is taken from keas.build
        and provides the same configuration syntax. The main difference is that we
        don't setup the install script as ``install`` because there is a conflict with
        the gnu install script call on ubuntu (pycairo, python waf install) used by a popen recipe (p01.recipe.setup:popen). This implementation will offer the
        install script as a ``deploy`` entry_point. The deploy entry point called deploy
        was removed and is not supported. Use salt or another concept for calling the deploy method your your server.
        
        
        ======
        README
        ======
        
        ``p01.build`` is a command line tool for quickly creating new (egg) releases.
        
        
        Sample
        ------
        
        As an example, consider a web application called MyServer:
        
        - ``p01.cdn`` - cdn resources like css and javascript files
        
        - ``p01.core`` - a package providing your application model
        
        - ``p01.web`` - a web front end for the ``p01.core`` package and using the
          p01.cdn resources
        
        Using ``p01.build`` you will be able to manage the lifecycle of these
        python packages and any number of deployment configurations coherently:
        
        - Allow you to define a project, which is a collection of
          interdependent eggs that generally get released together.
        
        - Automatically deploy new egg releases of each package when necessary.
        
        - Upload new eggs to a private egg repository (see mpypi).
        
        - Generate versioned buildout configuration files that combine the
          eggs properly.
        
        - Upload the buildout configuration files to a private configuration server.
          (mypypi has built in support such uploads)
        
        - Upload dependent buildout configuration files to a private configuration
          server (by checking the extends= chain, mypypi support such uploads)
        
        
        Installation
        ------------
        
        Install the ``deploy`` script with easy_install::
        
          $ easy_install p01.build
        
        You can also install a developer version of p01.build
        
        Checkout the code::
        
          $ svn checkout svn://svn.zope.org/repos/main/p01.build/trunk p01.build
          $ cd p01.build
        
        Run the bootstrap script and buildout::
        
          $ python bootstrap.py
          $ ./bin/buildout
        
        
        Release
        -------
        
        Once you have working developer version available, you should be able to run
        the ``build-package`` script. (Found in *./bin/build-package* with developer
        installations)::
        
          $ build-package
          Usage: build-package [options]
        
          Options:
            -h, --help            show this help message and exit
            -c FILE, --config-file=FILE
                                  The file containing the configuration of the project.
            -q, --quiet           When specified, no messages are displayed.
            -v, --verbose         When specified, debug information is created.
            -d, --use-defaults    When specified, no user input is required and the defaults are used.
            -o, --offline-mode    When set, no server commands are executed.
            -n, --next-version    When set, the system guesses the next version to
                                  generate.
            -b BRANCH, --use-branch=BRANCH
                                  When specified, this branch will be always used.
            -i BRANCHES --independent-branches=BRANCH1 BRANCH2,
                                  When specified, the system guesses the next version from all this branches. This is important to set if
                                  you release from different branches. It will prevent
                                  that a package get used which was released from
                                  another branch !!!
            --no-upload           When set, the generated configuration files are not
                                  uploaded.
            --no-branch-update    When set, the branch is not updated with a new version
                                  after a release is created.
            -s PATH, --storage-path=PATH
                                  Store the generated files in that folder and not in
                                  the global root directory. This is just for provide
                                  a clean setup and prevent to put all the generated config files
        
        
        Getting Started
        ---------------
        
        Assumptions; First we need to start with some assumptions that p01.build has
        about the way packages are laid out. When developing multiple packages in
        tandem, it often makes sense to have your subversion repository laid
        out like so::
        
          SVNROOT/MyServer/packages/
              branches/
                  Branch-0.x/
                      ...
                  Branch-1.x/
                      p01.cdn/
                      p01.core/
                      p01.web/
              tags/
                  p01.cdn-0.5.0/
                  p01.core-0.5.0/
                  p01.core-0.5.1/
                  p01.web-0.5.0/
                  p01.web-0.5.1/
                  p01.web-0.5.2/
                  ...
              trunk/
                  p01.cdn/
                  p01.core/
                  p01.web/
        
        The important thing to note is that each package does *not* have its
        own branches/ tags/ trunk/ directories, but rather there is just one
        set of the entire "project."
        
        
        Project setup
        -------------
        
        Before you can really do anything with the ``build-package`` script,
        you have to define a configuration file. Project configuration files
        use the INI [#ini]_ file format.  Every project configuration file
        must have a ``[build]`` section.  The project configuration file for
        the MyServer would look something like this::
        
        
          # MyServer.cfg
        
          [build]
          name = MyServer  # this has nothing to do with the package namespace
          version = +
          template = release.cfg
          tag-layout = subfolder
          upload-type = setup.py
          package-index = https://pypi.projekt01.ch/private
          package-index-username = username
          package-index-password = password
          buildout-server = https://pypi.projekt01.ch/++projects++/
          buildout-server-username = username
          buildout-server-password = password
          svn-repos = https://svn.projekt01.ch/svn/myserver/packages/
          svn-repos-username = somesvnuser
          svn-repos-password = somepass
          packages = p01.cdn
                     p01.core
                     p01.web
        
        Let's go over each of the settings in the ``build`` section of
        *MyServer.cfg*.
        
        - **name** - This is the name of the project. It can be anything you
          want and has nothing to do with the packages that make up the
          project.  The name will be part of the generation buildout
          configuration files.
        
        - **version** - This is the version to use when making a new release
          of the Project.  The version number becomes part of the filename for
          the generated buildout configuration files.
        
          - Using **+** as the version will simply increment the version
            number of the project from the versions that have already been
            released.
        
        - **template** - This is a base buildout configuration file to use for
          all deployments.  When a new Project release is created, the
          ``[versions]`` section will automatically be updated with the
          correct versions of each of the ``p01.*`` packages.  More on
          this later.
        
        - **tag-layout** - Choose from ``flat`` or ``subfolder``
        
          - **flat** Tags will be created in svn as /tags/package-version
            This is the default setting.
        
          - **subfolder** Tags will be created in svn as /tags/package/version
        
        - **upload-type** - Choose from ``internal`` or ``setup.py``
        
          - **internal** Upload packages to a WebDAV enabled web server using the
            below credetials. This is the default setting.
            (actually does a ``python setup.py sdist`` and uploads the result)
        
          - **setup.py** Executes ``python setup.py sdist register upload``,
            does nothing else as this command should take care of the upload.
        
        - **package-index** - The url to a WebDAV [#webdav]_ enabled web
          server where generated eggs for each of the ``p01.*`` packages
          should be uploaded. Used for upload only if ``upload-type`` is ``internal``.
          Also used to check/get existing versions of packages.
        
        - **package-index-username** - The username for accessing the WebDAV
          server
        
        - **package-index-password** - The password for accessing the WebDAV
          server
        
        - **buildout-upload-type** - Choose from ``webdav``, ``local`` or ``mypypi``
        
          - **webdav** Upload generated buildout files to the url specified by
            ``buildout-server`` with the WebDAV protocol.
        
          - **local** Just generate buildout files, don't upload them.
            If ``buildout-server`` is given buildout files will be copied to that
            folder.
        
          - **mypypi** Upload generated buildout files to the url specified by
            ``buildout-server``. The url should point to the mypypi upload page.
            (Something like http://yourhost/++projects++/)
        
        - **buildout-server** - The url to a WebDAV enabled web
          server where generated buildout files should be uploaded.
          If ``buildout-upload-type`` is ``local`` this is a path on the local
          filesystem. Buildout files wil be copied to this folder.
          If not given, the process stops after releasing the packages.
        
        - **buildout-server-username** - The username for accessing the WebDAV
          server
        
        - **buildout-server-password** - The password for accessing the WebDAV
          server
        
        - **svn-repos** - The url for the subversion repository where all the
          source code lives, including release tags.
        
        - **svn-repos-username** - The username for the url repository.
          Use the command line option ``--force-svnauth`` to force all svn operations
          to use this credential.
          Otherwise cached authentication will be used.
        
        - **svn-repos-password** - The password for the url repository.
        
        - **hash-config-files** - Add hashes based on file content to dependent config
          filenames.
        
        - **packages** - a list of packages that are part of the project.
          These are the packages that live in the svn repository and that
          should be released in conjunction with each other.
        
        
        Defining a Release Template
        ---------------------------
        
        As we saw in the previous section, *MyServer.cfg* refers to a file
        called *release.cfg*.  This is just a base buildout configuration. Additional
        to this, we can also define different configuration data as define in stage and
        production section. Such sections can get used in a product deployment as
        additional (shared) variables.  For the MyServer project, it might look like
        this::
        
          # release.cfg
        
          [buildout]
          extends = http://download.zope.org/zope3.4/3.4.0/versions.cfg
          parts = test
          find-links = https://pypi.projekt01.ch/private
        
          [test]
          recipe = zc.recipe.testrunner
          eggs = p01.cdn
                 p01.core
                 p01.web
        
          [app]
          recipe = zc.zope3recipes:app
          servers = zserver
          site.zcml = <include package="p01.web" file="app.zcml" />
          eggs = p01.web
        
          [zope3]
          location =
        
          [stage]
          memcached = 127.0.0.1:11211
        
          [production]
          memcached = 10.0.0.1:11211
        
        
        When a new release of the MyServer project is made, a ``[versions]``
        section will be added to this configuration file with all the
        correct ``p01.*`` versions pinned down.
        
        
        Defining Multiple Deployment Configurations
        -------------------------------------------
        
        Each time you release a Project, you may want to generate different
        buildout configuration files for all the different deployment
        environments you might have.  For example, you may have three
        different environments: Development, Stage, and Production. These are
        called variants.  Each environment may need to have the application
        run on different ports, at different log levels, or have other small
        differences.
        
        We can easily generate additional configuration variants by adding
        addtional sections to the *MyServer.cfg* file::
        
          # MyServer.cfg
        
          [Development]
          template = instance.cfg
          vars = stage
          port = 9080
          logdir = /opt/myserver/dev/logs
          install-dir = /opt/myserver/dev
          loglevel = debug
          cache-size = 1000
        
          [Stage]
          template = instance.cfg
          vars = stage
          port = 9082
          logdir = /opt/myserver/stage/logs
          install-dir = /opt/myserver/stage
          loglevel = info
          cache-size = 1000
        
          [Production]
          template = instance.cfg
          vars = production
          port = 8080
          logdir = /var/log/myserver
          install-dir = /opt/myserver/
          loglevel = warn
          cache-size = 200000
        
        We can then have a single *instance.cfg* file that uses python's built in
        string templating to access the variables we set in *MyServer.cfg*.  For the
        MyServer project, it might look like this::
        
          # instance.cfg
        
          [buildout]
          parts += server
          directory = %(install-dir)s
        
          [database]
          recipe = zc.recipe.filestorage
        
          [server]
          recipe = zc.zope3recipes:instance
          application = server-app
          zope.conf =
        
            <product-config memcached>
              memcached %(memcached)s
            </product-config>
        
            <zodb>
              cache-size %(cache-size)s
              <filestorage>
                path ${database:path}
              </filestorage>
            </zodb>
        
            <server>
              type WSGI-HTTP
              address %(port)s
            </server>
        
            <eventlog>
              level %(loglevel)s
              <logfile>
                formatter zope.exceptions.log.Formatter
                path %(logdir)s/server.log
              </logfile>
            </eventlog>
        
            <accesslog>
              <logfile>
                level info
                path %(logdir)s/server-access.log
              </logfile>
            </accesslog>
        
        As you can see, the MyServer.cfg uses additional vars (stage, production) which
        makes it very simple to define a lagrge amount of shared attributes in a
        release template and use them in the instance template. Note, a side effect
        from the python config parser is, that you will inherit arguments define in a
        duplicated section define in an template loaded via (buildout) extends.
        
        
        Releasing a Project
        -------------------
        
        Once you have created all the necessary configuration files, you can
        make your first project release.  This is where the ``build-package``
        script comes in to play.  The first time you run the ``build-package``
        script, the only option you need to pass in will be the configuration
        file.
        
        The ``build-package`` script will prompt you for version information
        about each of the packages it will be releasing as part of the
        MyServer.cfg project.  Your first interaction with the script might look
        like this::
        
          $ build-package -c MyServer.cfg --quiet
          Version for `p01.cdn` : 1.0.0
          The release p01.cdn-1.0.0 does not exist.
          Do you want to create it? yes/no [yes]: yes
          Version for `p01.core` : 1.0.0
          The release p01.core-1.0.0 does not exist.
          Do you want to create it? yes/no [yes]: yes
          Version for `p01.web` : 1.0.0
          The release p01.web-1.0.0 does not exist.
          Do you want to create it? yes/no [yes]: yes
        
        The next time you make a release, you can set the ``-n`` flag for
        ``build-package`` to automatically guess the next version that should
        be released.  It does this by first looking for all the release tags
        of a given package and finding the last changed revision for the trunk
        of a given package.  If any code for the given package was changed
        since the last time it was released, it will bump the most minor
        version number automatically.  If no change has occured, it will
        choose the latest existing release.
        
        You can also use the ``-d`` flag to make ``build-package`` not prompt
        you before creating a new release.
        
        If you need to make a new release from a particular branch, you can
        use the ``-b`` option.  For example, if bug fixes have been made to the
        MyServer-1.x branch, we can create a new release using code from this
        branch like this::
        
          $ build-package -c MyServer.cfg -nb MyServer-1.x
        
        When the new package versions are calculated, they will be versioned
        along the 1.x line, even if you have since created 2.x releases, by
        analyzing the name of the branch.
        
        Caveat when using ``-n`` and ``-d`` on a branch with a branchname ending
        with the version number is that you'll need to have the package versions
        matching the branch version.
        E.g. having a branch: ``branches/myserver-1.9`` will suppose packages like
        ``p01.core-1.9.x`` and ``p01.web-1.9.x`` and so on.
        You should be aware of this also when releasing packages from the trunk.
        Most probably you'll drive development on the trunk and branch out for a
        stable. In this case package versions on the branch should be kept inline.
        
        
        Install a Released Project
        ---------------------------
        
        ``p01.build`` also comes with a very simple installation script that
        can be used to quickly install any variant of the released project::
        
          $ deploy --help
          Usage: deploy [options]
        
          Options:
            -h, --help            show this help message and exit
            -u URL, --url=URL     The base URL at which the releases can be found.
            -p PROJECT, --project=PROJECT
                                  The name of the project to be installed.
            -V VARIANT, --variant=VARIANT
                                  The variant of the project to be installed.
            -v VERSION, --version=VERSION
                                  The version of the project to be installed.
            -l, --latest          When specified, the latest version will be chosen.
            --username=USER       The username needed to access the site.
            --password=PASSWORD   The password needed to access the site.
            -b PATH, --buildout-path=PATH
                                  The path to the buildout executable.
            --quiet               When specified, no messages are displayed.
            --verbose             When specified, debug information is created.
        
        For example, to install the latest Stage version of the ``MyServer`` project,
        you would run:
        
          $ deploy -u https://pypi.projekt01.ch/++projects++/ -p MyServer -V Stage --latest
        
        
        Creating Helper Scripts
        -----------------------
        
        Sometimes it can be a pain to remember what all the command line
        options are that you need to pass for building your project.
        Fortunately, it is really easy to create helper scripts that just set
        some defaults for you.
        
        For example, to create a build-myserver script, you would add the
        following to a buildout configuration file::
        
          [uploads]
          recipe = p01.recipe.setup:mkdir
          path = ${buildout:directory}/parts/uploads
        
          [build-myserver]
          recipe = zc.recipe.egg
          eggs = p01.build
          scripts = build=build-myserver
          initialization =
              sys.argv[1:1] = ['-c', 'MyServer.cfg',
                               '-o', '${buildout:directory}/parts/uploads']
        
        The possibilities are endless!
        
        Footnotes
        =========
        
        .. [#ini] See http://en.wikipedia.org/wiki/INI_file
        .. [#webdav] See http://en.wikipedia.org/wiki/WebDAV
        
        
        =======
        CHANGES
        =======
        
        0.5.1 (2015-08-26)
        ------------------
        
        - feature: added option -s, --storage-path which defines a directory where we
          store the created files before upload to pypi, webdav etc. This prevents that
          all files get generated into the package root.
        
        - cleaup code, get rid of pkg_resources.parse_version deprecation message.
          Ported the parse_version method for now.
        
        - removed p01/build/install.py and move code to p01/build/deploy.py. Switch the
          entry point for deploy to deploy.py
        
        
        0.5.0 (2015-04-15)
        ------------------
        
        - bugfix: parse given buildout parts content as is and only rstrip continue
          lines. This allows to use any content in parts like script content etc.
          without any intend troubles. The parse concept is similar and compatible with
          the one used in zc.buildout > 2.0
        
        - This package provides a product build, release and installation system based
          on package versions using zc.buildout. The core concept is taken from
          keas.build and provides the same configuration syntax. The main difference
          is that we don't setup the install script as ``install`` because there is a
          conflict with the gnu install script call on ubuntu (pycairo, python waf
          install) used by a popen recipe (p01.recipe.setup:popen). This implementation
          will offer the install script as a ``deploy`` entry_point. You can simply
          install the deploy script with ``easy_install p01.build`` on your server.
          For more information see p01/build/README.txt.
        
        - initial branch of keas.build 0.4.1
        
Keywords: Zope3 z3c p01 build release install
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Zope Public License
Classifier: Programming Language :: Python
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Framework :: Zope3
