Metadata-Version: 1.0
Name: zc.async
Version: 1.5.4
Summary: Schedule durable tasks across multiple processes and machines.
Home-page: http://packages.python.org/zc.async
Author: Gary Poster
Author-email: gary.poster@canonical.com
License: ZPL
Description: ===========
        What is it?
        ===========
        
        The ``zc.async`` package provides **an easy-to-use Python tool that schedules
        work persistently and reliably across multiple processes and machines.**
        
        For instance...
        
        - *Web apps*: maybe your web application lets users request the creation of a
          large PDF, or some other expensive task.
        
        - *Postponed work*: maybe you have a job that needs to be done at a certain time,
          not right now.
        
        - *Parallel processing*: maybe you have a long-running problem that can be made
          to complete faster by splitting it up into discrete parts, each performed in
          parallel, across multiple machines.
        
        - *Serial processing*: maybe you want to decompose and serialize a job.
        
        High-level features include the following:
        
        - easy to use;
        
        - flexible configuration, changeable dynamically in production;
        
        - reliable;
        
        - supports high availability;
        
        - good debugging tools;
        
        - well-tested; and
        
        - friendly to testing.
        
        While developed as part of the Zope project, zc.async can be used stand-alone.
        
        =================
        How does it work?
        =================
        
        The system uses the Zope Object Database (ZODB), a transactional, pickle-based
        Python object database, for communication and coordination among participating
        processes.
        
        zc.async participants can each run in their own process, or share a process
        (run in threads) with other code.
        
        The Twisted framework supplies some code (failures and reactor implementations,
        primarily) and some concepts to the package.
        
        ======================
        Where can I read more?
        ======================
        
        Quickstarts and in-depth documentation are available in the package and in
        the `new and exciting on-line documentation`_.
        
        .. _`new and exciting on-line documentation`: http://packages.python.org/zc.async/1.5.0/
        
        
        
        =======
        Changes
        =======
        
        1.5.4 (2011-03-03)
        ==================
        
        - Rearrange ftesting.setUp to avoid provoking a DemoStorage bug
          present in ZODB <= 3.9.3.
        
        - Resolved the following testing issues: signal handlers weren't cleaned
          up properly in some tests, Twisted was leaking file descriptors during
          ftesting tearDown (http://twistedmatrix.com/trac/ticket/3063), and
          the twisted.txt regression test was not repeatable due to
          Twisted's recalcitrance when it comes to stopping and subsequently
          starting a reactor instance.
        
        - Fix two undefined variables that could trigger exceptions in corner
          cases.
        
        
        1.5.3 (2009-11-15)
        ==================
        
        - Made zc.async.subscribers.ThreadedDispatcherInstaller keep track of signal
          handlers it installs in a module global "signal_handlers."
        
        - Made zc.async.ftesting.tearDown restore the signal handlers that were
          replaced by ThreadedDispatcherInstaller.
        
        - Fix a bug in zc.async.ftesting.setUp and zc.async.testing.print_logs
          which would result in the default argument for log_file becoming
          "fixated" with an incorrect value across tests.
        
        - Make the ftesting.txt test exercise the 'zc.async' logger in
          addition to 'zc.async.event'.
        
        - zc.async.utils.dt_to_long coerces return value to long (test pass on 64-bit
          Python).
        
        - Tests pass on Python 2.6
        
        - The callable of a zc.async.job.Job (or one of its subclasses) can be a
          method on the Job itself.
        
        1.5.2 (2009-07-22)
        ==================
        
        - Fix a bug where zc.async.testing._datetime.now did not accept the same
          keyword arguments as datetime.datetime, added tests.
        
        - Fix a bug where zc.async.testing._datetime.astimezone did not accept the same
          keyword arguments as datetime.datetime, added tests.
        
        1.5.1 (2008-10-13)
        ==================
        
        - Add a performance optimization with ``isinstance`` before ``providedBy``.
        
        - Add support for filters to ``AgentInstaller``.
        
        - Fix a bug which caused a condition to be always true in ``monitor.Encoder``.
        
        1.5.0 (2008-09-21)
        ==================
        
        - Documentation improvements.  Converted documentation into Sphinx system.
        
        - Made "other" commit errors for the ``RetryCommonForever`` retry policy have
          an incremental backoff.  By default, this starts at 0 seconds, and increments
          by a second to a maximum of 60 seconds.
        
        - Work around a memory leak in zope.i18nmessageid
          (https://bugs.launchpad.net/zope3/+bug/257657).  The change should be
          backward-compatible.  It also will produce slightly smaller pickles for Jobs,
          but that was really not a particular goal.
        
        - Added zc.async.partial.Partial for backward compatibility purposes.
        
        - Fix support for Twisted installed reactor.
        
        - Fix retry behavior for parallel and serial jobs
        
        - Tweaked the uuid.txt to mention zdaemon/supervisor rather than Zope 3.
        
        - Fixed some bugs in egg creation.
        
        - Changed quotas to not use a container that has conflict resolution, since
          these values should be a strict maximum.
        
        - We only want to claim a job if we are activated.   Make the agent check the
          ``activated`` and ``dead`` attributes of the parent dispatcher before
          claiming.
        
        - When activating, also clean out jobs from the dispatcher's agents, just as
          with deactivating.  This should protect from unusual race conditions in
          which the dispatcher got a job after being deactivated.
        
        - Change dispatcher to ping before claiming jobs.
        
        - when a ping reactivates a dispatcher, use new method ``reactivate`` rather
          than ``activate``.  This fires a new ``DispatcherReactivated`` event.
        
        - It's still theoretically possible (for instance, with a
          badly-behaved long commit that causes a sibling to believe that the
          process is dead) that an async worker process would be working on a
          job that it shouldn't be.  For instance, the job has been taken away,
          and is another process' responsibility now.  Now, whenever a
          process is about to start any work (especially a retry), it should
          double-check that the job is registered as being performed by itself.
          If not, the process should abort the transaction, make an error
          log, and give up on the job.  Write conflict errors on the job should
          protect us from the edge cases in this story.
        
        - The dispatcher's ``getActiveJobs`` method now actually tells you information
          about what's going on in the threads at this instant, rather than what's
          going on in the database.  The poll's ``active jobs`` keys continues to
          report what was true *in the database* as of *the last poll*.  This change
          also affects the ``async jobs`` monitor command.
        
        - The dispatcher method ``getJobInfo`` (and the monitor command ``async job``)
          now returns the name of the queue for the job, the name of the agent for the
          job, and whether the job has been, or was reassigned.
        
        - zc.async events inherit from 'zc.component.interfaces.IObjectEvent' instead
          of a zc.async specific IObjectEvent (thanks to Satchit Haridas).
        
        - Added new monitoring and introspection tools: the ``asyncdb`` zc.monitor
          command (and, for Python, the code in monitordb.py).  This code provides
          easy spellings to examine the database's view of what is happening in
          zc.async.  Because it is the database, it also has a much longer historical
          view than the ``async`` tools.  The best way to learn about these tools is
          to read the extensive documentation provided within zc.monitor by
          using ``asyncdb help`` and ``asyncdb help <TOOL NAME>``.
        
        - Added new preferred way of filtering agent choices: the new ``filter``
          attribute.  Using filters, rather than "choosers," allows several ``asyncdb``
          tools to filter pending jobs based on what an agent is willing to do.  It
          also is a smaller contract, and so a filter requires less code than a chooser
          in the common case.  On the other hand, using a filter alone doesn't allow
          the agent to try to *prefer* certain tasks.
        
        - Deprecated agent.chooseFirst.  It is no longer necesary, since an agent
          without a chooser and with a filter of None has the same behavior.  It is
          retained for legacy databases.
        
        - Moved deprecated legacy code to new ``legacy`` module.
        
        - Tried to be significantly reduce the chance of spurious timing errors in the
          tests, at the expense of causing the tests to take longer to run.
        
        - monitoring support depends on the new zc.monitor package, which is not Zope
          specific.  This means non-Zope 3 apps can take advantage of the monitoring
          support.  To use, use the [monitor] target; this only adds simplejson,
          zc.ngi, and zc.monitor to the basic dependencies.
        
        - Make ftesting try to join worker threads, in addition to polling thread,
          to try to eliminate intermittent test-runner warnings in ftests that a
          thread is left behind.  If the threads do not end, inform the user what jobs
          are not letting go.  (thanks to Patrick Strawderman)
        
        1.4.2 (2009-07-17)
        ==================
        
        - Fix a bug where zc.async.testing._datetime.now did not accept the same
          keyword arguments as datetime.datetime, added tests.
        
        1.4.1 (2008-07-30)
        ==================
        
        - The new ``serial`` and ``parallel`` helpers did not allow the
          ``postprocess`` argument to be a partial closure, and were being naughty.
          Fixed.
        
        - Added tests and demos for advanced features of ``serial`` and ``parallel``.
        
        - More tweaks to the new Quickstart S5 document.
        
        1.4.0 (2008-07-30)
        ==================
        
        - Mentioned in ftesting.txt that Zope 3 users should uses zope.app.testing
          3.4.2 or newer.  Also added a summary section at the beginning of that file.
        
        - Added logging of critical messages to __stdout__ for ``ftesting.setUp``.
          This can help discovering problems in callback transactions.  This uses a new
          helper function , ``print_logs``, in zc.async.testing, which is primarily
          intended to be used for quick and dirty debugging
        
        - Changed testing.wait_for_result and testing.wait_for_annotation to ignore
          ReadConflictErrors, so they can be used more reliably in tests that use
          MappingStorage, and other storages without MVCC.
        
        - Support <type 'builtin_function_or_method'> for adaptation to Job.
        
        - Add warning about long commits to tips and tricks.
        
        - After complaining about a polling dispatcher that is deactivated not really
          being dead in the logs, reactivate.
        
        - No longer use intermediate job to implement the success/failure addCallbacks
          behavior.  Introduce an ICallbackProxy that can be used for this kind of
          behavior instead.  This change was driven by two desires.
        
          - Don't log the intermediate result.  It makes logs harder to read with
            unnecessary duplications of pertinent data hidden within unimportant
            differences in the log entries.
        
          - Don't unnecessarily remember errors in success/failure callbacks.  This can
            cause unnecessary failures in unusual situations.
        
          The callback proxy accepts callbacks, which are added to the selected job
          (success or failure) when the job is selected.
        
          This change introduces some hopefully trivial incompatibilities, which
          basically come down to the callback being a proxy, not a real job. Use the
          convenience properties ``success`` and ``failure`` on the proxy to look at
          the respective jobs. After the proxy is evaluated, the ``job`` attribute
          will hold the job that was actually run. ``status`` and ``result`` are
          conveniences to get the status and result of the selected job.
        
        - Add ``parallel`` and ``serial`` convenience functions to zc.async.job to make
          it trivial to schedule and process decomposed jobs.
        
        - Add ``start`` convenience function to zc.async.configure to make it trivial
          to start up a common-case configuration of a zc.async dispatcher.
        
        - No longer use protected attributes of callbacks in ``resumeCallbacks``.
        
        - The "local" code is now moved out from the dispatcher module to
          threadlocal.  This is to recognize that the local code is now modified
          outside of the dispatcher module, as described in the next bullet.
        
        - Jobs, when called, are responsible for setting the "local" job value.  This
          means that zc.async.local.getJob() always returns the currently running job,
          whether it is a top-level job (as before) or a callback (now).
        
        - Start on S5 QuickStart presentation (see QUICKSTART_1_VIRTUALENV.txt in
          package).
        
        1.3 (2008-07-04)
        ================
        
        - added "Tips and Tricks" and incorporated into the PyPI page.
        
        - added ``setUp`` and ``tearDown`` hooks to Job class so that code can run
          before and after the main job's code.  The output of ``setUp`` is passed as
          an argument to ``tearDown`` so that one can pass state to the other, if
          needed. ``setUp`` is run immediately before the actual job call.
          ``tearDown`` runs after the transaction is committed, or after it was aborted
          if there was a failure.   A retry requested by a retry policy causes the
          methods to be run again.  A failure in ``setUp`` is considered to be a
          failure in the job, as far as the retryPolicy is concerned (i.e., the job
          calls the retry policy's ``jobError`` method).  If ``setUp`` fails, the job
          is not called, bit ``tearDown`` is.  ``tearDown`` will fail with a critical
          log message, but then processing will continue.
        
        - using the new ``setUp`` and ``tearDown`` hooks, added a Zope 3-specific Job
          subclass (see zc.async.z3.Job) that remembers the zope.app.component site and
          interaction participants when instantiated. These can be mutated. Then, when
          the job is run, the ``setUp`` sets up the site and a security interaction
          with the old participants, and then the ``tearDown`` tears it all down after
          the transaction has committed.
        
        - changed retry policy logs to "WARNING" level, from "INFO" level.
        
        - changed many dispatcher errors to "CRITICAL" level from "ERROR" level.
        
        - added "CRITICAL" level logs for "other" commit retries on the
          RetryCommonForever retry policy.
        
        - added ``remove`` method on queue.
        
        - added helpers for setting up and tearing down Zope 3 functional tests
          (ftesting.py), and a discussion of how to write Zope 3 functional tests with
          layers (zope.app.testing.functional) in ftesting.txt.
        
        - remove obsolete retry approach for success/failure callbacks
          (``completeStartedJobArguments``): it is now handled by retry policies.
        
        - remove odd full-path self-references within the utils module.
        
        - renamed ``zc.async.utils.try_transaction_five_times`` to
          ``zc.async.utils.try_five_times``.
        
        - doc improvements and fixes (thanks to Zvezdan Petkovic and Gintautas
          Miliauskas).
        
        - the ``z3`` "extra" distutils target now explicitly depends on zope.security,
          zope.app.security, and zope.app.component.  This almost certainly does not
          increase the practical dependencies of the ``z3`` extras, but it does reflect
          new direct dependencies of the z3-specific modules in the package.
        
        1.2 (2008-06-20)
        ================
        
        - made the log for finding an activated agent report the pertinent queue's oid
          as an unpacked integer, rather than the packed string blob. Use
          ``ZODB.utils.p64`` to convert back to an oid that the ZODB will recognize.
        
        - Bugfix: in failing a job, the job thought it was in its old agent, and the
          ``fail`` call failed. This is now tested by the first example in new doctest
          ``catastrophes.txt``.
        
        - jobs no longer default to a ``begin_by`` value of one hour after the
          ``begin_after``.  The default now is no limit.
        
        - Made dispatcher much more robust to transaction errors and ZEO
          ClientDisconnected errors.
        
        - Jobs now use an IRetryPolicy to decide what to do on failure within a job,
          within the commit of the result, and if the job is interrupted.  This allows
          support of transactional jobs, transactional jobs that critically must be
          run to completion, and non-transactional jobs such as communicating with an
          external service.
        
        - The default retry policy supports retries for ClientDisconnected errors,
          transaction errors, and interruptions.
        
        - ``job.txt`` has been expanded significantly to show error handling and the
          use of retry policies. New file ``catastrophes.txt`` shows handling of other
          catastrophes, such as interruptions to polling.
        
        - job errors now go in the main zc.async.event log rather than in the
          zc.async.trace log.  Successes continue to go in the trace log.
        
        - callback failures go to the main log as a CRITICAL error, by default.
        
        - ``handleInterrupt`` is the new protocol on jobs to inform them that they were
          active in a dispatcher that is now dead. They either fail or reschedule,
          depending on the associated IRetryPolicy for the job. If they reschedule,
          this should either be a datetime or timedelta. The job calls the agent's
          ``reschedule`` method. If the timedelta is empty or negative, or the datetime
          is earlier than now, the job is put back in the queue with a new ``putBack``
          method on the queue. This is intended to be the opposite of ``claim``. Jobs
          put in the queue with ``putBack`` will be pulled out before any others.
        
        - convert to using zope.minmax rather than locally defined ``Atom``.
        
        - Fix (and simplify) last_ping code so as to reduce unnecessarily writing the
          state of the parent DispatcherAgents collection to the database whenever the
          atom changed.
        
        - Depends on new release of zc.twist (1.3)
        
        - Switched dispatcher's in-memory storage of job and poll information to be per
          job or per poll, respectively, rather than per time period, so as to try and
          make memory usage more predictable (for instance, whether a dispatcher is
          whipping through lots of jobs quickly, or doing work more slowly).
        
        1.1.1 (2008-05-14)
        ==================
        
        - more README tweaks.
        
        - converted all reports from the dispatcher, including the monitor output,
          to use "unpacked" integer oids.  This addresses a problem that simplejson
          was having in trying to interpret the packed string blobs as unicode, and
          then making zc.ngi fall over.  To get the object, then, you'll need to
          use ``ZODB.utils.p64``, like this:
          ``connection.get(ZODB.utils.p64(INTEGER_OID))``, where ``INTEGER_OID``
          indicates the integer oid of the object you want to examine.
        
        - added several more tests for the monitor code.
        
        - made the ``async jobs`` monitor command be "up to the minute".  Before, it
          included all of the new and active jobs from the previous poll; now, it
          also filters out those that have since completed.
        
        - The ``async job`` command was broken, as revealed by a new monitor test.
          Fixed, which also means we need a new version of zope.bforest (1.2) for a new
          feature there.
        
        1.1 (2008-04-24)
        ================
        
        - Fired events when the IQueues and IQueue objects are installed by the
          QueueInstaller (thanks to Fred Drake).
        
        - Dispatchers make agent threads keep their connections, so each connection's
          object cache use is optimized if the agent regularly requests jobs with
          the same objects.
        
        - README improved (thanks to Benji York and Sebastian Ware).
        
        - Callbacks are logged at start in the trace log.
        
        - All job results (including callbacks) are logged, including verbose
          tracebacks if the callback generated a failure.
        
        - Had the ThreadedDispatcherInstaller subscriber stash the thread on the
          dispatcher, so you can shut down tests like this:
        
          >>> import zc.async.dispatcher
          >>> dispatcher = zc.async.dispatcher.get()
          >>> dispatcher.reactor.callFromThread(dispatcher.reactor.stop)
          >>> dispatcher.thread.join(3)
        
        - Added ``getQueue`` to zc.async.local as a convenience (it does what you
          could already do: ``zc.async.local.getJob().queue``).
        
        - Clarified that ``IQueue.pull`` is the approved way of removing scheduled jobs
          from a queue in interfaces and README.
        
        - reports in the logs of a job's success or failure come before callbacks are
          started.
        
        - Added a section showing how the basic_dispatcher_policy.zcml worked, which
          then pushed the former README_3 examples into README_3b.
        
        - Put ZPL everywhere I was supposed to.
        
        - Moved a number of helpful testing functions out of footnotes and into
          zc.async.testing, both so that zc.async tests don't have to redefine them
          and client packages can reuse them.
        
        1.0 (2008-04-09)
        ================
        
        Initial release.
        
        
        
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Zope Public License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Framework :: ZODB
