The subscribers module provides several conveniences for starting and
configuring zc.async.  Let's assume we have a database and all of the
necessary adapters and utilities registered [#setUp]_.

The first helper we'll discuss is ``threaded_dispatcher_installer``.  This can be
used as a subscriber to a DatabaseOpened event, as defined by zope.app.appsetup
if you are using it, and defined by zc.async.interfaces if you are not. It is
an instance of ``ThreadedDispatcherInstaller``, which, as the name implies, is
a class to create handlers that install a threaded dispatcher.

We will install a dispatcher that polls a bit faster than the default five
seconds, so that we can have an easier time in running this doctest.

    >>> import zc.async.subscribers
    >>> import zc.async.interfaces
    >>> import zope.event
    >>> import zope.component
    >>> isinstance(zc.async.subscribers.threaded_dispatcher_installer,
    ...            zc.async.subscribers.ThreadedDispatcherInstaller)
    True
    >>> zc.async.subscribers.threaded_dispatcher_installer.poll_interval
    5
    >>> threaded_installer = zc.async.subscribers.ThreadedDispatcherInstaller(
    ...     poll_interval=0.5)
    >>> zope.component.provideHandler(threaded_installer)
    >>> zope.event.notify(zc.async.interfaces.DatabaseOpened(db))

Now a dispatcher is installed and running.  (The get_poll helper is a testing
helper.)

    >>> import zc.async.dispatcher
    >>> dispatcher = zc.async.dispatcher.get()
    >>> dispatcher.poll_interval
    0.5

    >>> from zc.async.testing import get_poll
    >>> get_poll(dispatcher, 0)
    {}

The function also stashed the thread on the dispatcher, so we can write tests
that can grab it easily.

    >>> import threading
    >>> isinstance(dispatcher.thread, threading.Thread)
    True

The function also installs some signal handlers to optimize shutdown.  We'll
look at them soon.  For now, let's install some queues.

The subscribers module also includes helpers to install a queues collection
and zero or more queues.  The QueueInstaller class lets you specify an
iterable of names of queues to install, defaulting to ('',); a factory to
generate queues, defaulting to something that generates a zc.async.queue.Queue;
and a db_name if the queues collection should be placed in another database of
the given name, for a multi-database setup, defaulting to None, indicating that
the queues should be placed in the same database.

Two instances of this class are already instantiated in the module; one with
the defaults, and one specifying an additional database.

    >>> isinstance(zc.async.subscribers.queue_installer,
    ...            zc.async.subscribers.QueueInstaller)
    True
    >>> zc.async.subscribers.queue_installer.queues
    ('',)
    >>> print zc.async.subscribers.queue_installer.db_name
    None
    >>> isinstance(zc.async.subscribers.multidb_queue_installer,
    ...            zc.async.subscribers.QueueInstaller)
    True
    >>> zc.async.subscribers.multidb_queue_installer.queues
    ('',)
    >>> zc.async.subscribers.multidb_queue_installer.db_name
    'async'

Let's try the multidb variation out.  We'll need another database, and the
proper data structure set up on the two of them.  The first footnote of this
file sets the necessary data structures up.

The subscribers generated by this class expect to get the same event we fired
above, an IDatabaseOpenedEvent. Normally only one of these events fires, since
the database generally opens once, but for the purposes of our example we will
fire it again in a moment.

While we're at it, we'll use the other handler: ``AgentInstaller``.  This class
generates a subscriber that installs agents in the queues it finds when
dispatcher agent activation events fire.  You must specify an agent name to use;
and can specify a chooser (a way to choose the tasks this agent should perform),
a filter (a recent and a better alternative to defining a chooser), a size (the
number of concurrent jobs this agent should hand out), and specific queue names
in which the agent should be installed, defaulting to None, or all queues.

The agent_installer installs an agent named 'main' for the active dispatcher
in all queues, with a default FIFO chooser.

    >>> isinstance(zc.async.subscribers.agent_installer,
    ...            zc.async.subscribers.AgentInstaller)
    True
    >>> zc.async.subscribers.agent_installer.agent_name
    'main'
    >>> print zc.async.subscribers.agent_installer.queue_names
    None
    >>> print zc.async.subscribers.agent_installer.chooser
    None
    >>> print zc.async.subscribers.agent_installer.filter
    None
    >>> zc.async.subscribers.agent_installer.size
    3

Now we can install the subscribers and give it a try.  As we said above,
normally the database opened event only fires once; this is just for purpose of
demonstration.  We unregister the previous handler so nothing gets confused.

    >>> zope.component.getGlobalSiteManager().unregisterHandler(
    ...     threaded_installer)
    True
    >>> zope.component.provideHandler(
    ...     zc.async.subscribers.multidb_queue_installer)
    >>> zope.component.provideHandler(
    ...     zc.async.subscribers.agent_installer)
    >>> zope.event.notify(zc.async.interfaces.DatabaseOpened(db))


Now if we look in the database, we'll find a queues collection in another
database, with a queue, with a dispatcher, with an agent.

    >>> import pprint
    >>> pprint.pprint(get_poll(dispatcher))
    {'': {'main': {'active jobs': [],
                   'error': None,
                   'len': 0,
                   'new jobs': [],
                   'size': 3}}}
    >>> conn = db.open()
    >>> root = conn.root()
    >>> root._p_jar is conn
    True
    >>> queues = root[zc.async.interfaces.KEY]
    >>> root[zc.async.interfaces.KEY]._p_jar is conn
    False
    >>> queues.keys()
    ['']
    >>> queue = queues['']
    >>> len(queue.dispatchers)
    1
    >>> da = queue.dispatchers.values()[0]
    >>> list(da)
    ['main']
    >>> bool(da.activated)
    True


We discuss a few other use-cases for the AgentInstaller in a separate footnote
[#filters_and_choosers]_.

When an IQueues or IQueue is installed, an event is fired that provides the
object being added, the container it is added to, and the name under which it
is added.  Therefore, two ObjectAdded events have fired now, one for a queues
collection and one for a queue.

    >>> from zope.component import eventtesting
    >>> for event in eventtesting.getEvents(zc.async.interfaces.IObjectAdded):
    ...     print "----"
    ...     print event.object.__class__, "object"
    ...     print "added to", event.parent.__class__
    ...     print "with name", repr(event.name)
    ...
    ----
    <class 'zc.async.queue.Queues'> object
    added to <class 'persistent.mapping.PersistentMapping'>
    with name 'zc.async'
    ----
    <class 'zc.async.queue.Queue'> object
    added to <class 'zc.async.queue.Queues'>
    with name ''


Note that all the events inherit from IObjectEvent.

    >>> import zope.component.interfaces
    >>> for event in eventtesting.getEvents(
    ...         zope.component.interfaces.IObjectEvent):
    ...     print event.__class__
    <class 'zope.component.interfaces.Unregistered'>
    <class 'zc.async.interfaces.ObjectAdded'>
    <class 'zc.async.interfaces.ObjectAdded'>
    <class 'zc.async.interfaces.DispatcherRegistered'>
    <class 'zc.async.interfaces.DispatcherActivated'>
    <class 'zope.component.interfaces.Unregistered'>
    <class 'zc.async.interfaces.DispatcherActivated'>
    <class 'zope.component.interfaces.Unregistered'>
    <class 'zc.async.interfaces.DispatcherActivated'>


Finally, we mentioned at the start that the threaded dispatcher installer also
installed some signal handlers.  Let's show a SIGINT (CTRL-C, usually), and
how it deactivates the dispatcher's agents collection in the ZODB.

    >>> import signal
    >>> import os
    >>> if getattr(os, 'getpid', None) is not None: # UNIXEN, not Windows
    ...     pid = os.getpid()
    ...     try:
    ...         os.kill(pid, signal.SIGINT)
    ...     except KeyboardInterrupt:
    ...         if dispatcher.activated:
    ...             assert False, 'dispatcher did not deactivate'
    ...     else:
    ...         print "failed to send SIGINT, or something"
    ... else:
    ...     dispatcher.reactor.callFromThread(dispatcher.reactor.stop)
    ...     for i in range(30):
    ...         if not dispatcher.activated:
    ...             break
    ...         time.sleep(0.1)
    ...     else:
    ...         assert False, 'dispatcher did not deactivate'
    ...
    >>> import transaction
    >>> t = transaction.begin() # sync
    >>> bool(da.activated)
    False

The dispatcher installer keeps track of the signal handlers it has
registered, as well as the signal handlers that were replaced.  This makes
it more convenient to restore the old signal handlers if needed.

References to the handlers are stored in the module global signal_handlers,
which is keyed by id(dispatcher).

    >>> signal_handlers = zc.async.subscribers.signal_handlers[id(dispatcher)]

The value is a mapping of signal to a tuple containing the old and new
signal handlers.

    >>> signal.SIGINT in signal_handlers
    True
    >>> signal.SIGTERM in signal_handlers
    True
    >>> has_sigbreak = hasattr(signal, "SIGBREAK")
    >>> not has_sigbreak or signal.SIGBREAK in signal_handlers
    True

zc.async.ftesting.tearDown will restore the old signal handlers, unless
the currently registered handler is different than the one that was
registered by the installer.

Let's register a newer SIGTERM handler to exercise this.

    >>> old_sigterm_handler = signal.getsignal(signal.SIGTERM)
    >>> def new_sigterm_handler(*args):
    ...     old_sigterm_handler(*args)
    >>> ignored = signal.signal(signal.SIGTERM, new_sigterm_handler)

    >>> import zc.async.ftesting
    >>> zc.async.ftesting.tearDown()

The signal_handlers mapping has been cleared.

    >>> zc.async.subscribers.signal_handlers
    {}

The old signal handlers have been restored.

    >>> prev, curr = signal_handlers[signal.SIGINT]
    >>> signal.getsignal(signal.SIGINT) is prev
    True
    >>> if has_sigbreak:
    ...     prev, curr = signal_handlers[signal.SIGBREAK]
    ...     signal.getsignal(signal.SIGBREAK) is prev
    ... else:
    ...     True
    True

...except for the SIGTERM handler, since we registered a newer one.

    >>> prev, curr = signal_handlers[signal.SIGTERM]
    >>> old_sigterm_handler is curr
    True
    >>> signal.getsignal(signal.SIGTERM) is prev
    False
    >>> signal.getsignal(signal.SIGTERM) is new_sigterm_handler
    True
    >>> ignore = signal.signal(signal.SIGTERM, prev) # restore original handler





.. ......... ..
.. Footnotes ..
.. ......... ..

.. [#setUp] Below we set up a database, provide the adapters and utilities
    that the code expects, and then define some helper functions we'll use in
    the examples.  See README_2 for a discussion of what is going on with the
    configuration.

    >>> databases = {}
    >>> import ZODB.FileStorage
    >>> storage = ZODB.FileStorage.FileStorage(
    ...     'main.fs', create=True)

    >>> async_storage = ZODB.FileStorage.FileStorage(
    ...     'async.fs', create=True)

    >>> from ZODB.DB import DB
    >>> databases[''] = db = DB(storage)
    >>> databases['async'] = async_db = DB(async_storage)
    >>> async_db.databases = db.databases = databases
    >>> db.database_name = ''
    >>> async_db.database_name = 'async'

    >>> import zc.async.configure
    >>> zc.async.configure.base()


.. [#filters_and_choosers] AgentInstaller also takes a number of other
    parameters such as 'chooser' and 'filter' that sets up the agent to handle
    only certain kinds of jobs. Both of these parameters expect a callable that
    will be used to select jobs that the agent will handle. Note that in the
    following we do not demonstrate the functionality of filters or
    choosers. That is discussed in detail in agent.txt. But we do look at the
    functionality of the installer itself.

    We first create an agent that will only accept jobs if the callable is a
    method named 'mock_work'.

    >>> def mock_work_filter(job):
    ...     return job.callable.__name__ == 'mock_work'
    >>> filtering_agent_installer = zc.async.subscribers.AgentInstaller(
    ...     'filtering_agent', filter=mock_work_filter)


    >>> isinstance(filtering_agent_installer,
    ...            zc.async.subscribers.AgentInstaller)
    True
    >>> filtering_agent_installer.agent_name
    'filtering_agent'
    >>> print filtering_agent_installer.queue_names
    None
    >>> print filtering_agent_installer.chooser
    None
    >>> filtering_agent_installer.filter.__name__
    'mock_work_filter'
    >>> filtering_agent_installer.size
    3

    We now register the new agent-installer, unregistering the earlier one at
    the same time.

    >>> zope.component.getGlobalSiteManager().unregisterHandler(
    ...     zc.async.subscribers.agent_installer)
    True
    >>> zope.component.provideHandler(filtering_agent_installer)

    Since the queue has already been installed and we want our new agent
    installer to run, we fire a DispatcherActivated event, which is the event
    that our installer subscribes to.

    >>> zope.event.notify(zc.async.interfaces.DispatcherActivated(da))

    Let's check the new agent is setup correctly:

    >>> 'filtering_agent' in da
    True
    >>> da['filtering_agent'].filter.__name__
    'mock_work_filter'
    >>> print da['filtering_agent'].chooser
    None


    A chooser can be used instead of a filter, but using the latter is preferred
    as the zc.async monitoring code provides hooks for filters (and not
    choosers). We now create an agent that uses a chooser instead of a filter.

    >>> def mock_work_chooser(agent):
    ...     return agent.queue.claim(lambda j: j.callable == mock_work)
    ...

    >>> choosing_agent_installer = zc.async.subscribers.AgentInstaller(
    ...     'choosing_agent', chooser=mock_work_chooser)


    >>> isinstance(choosing_agent_installer,
    ...            zc.async.subscribers.AgentInstaller)
    True
    >>> choosing_agent_installer.agent_name
    'choosing_agent'
    >>> print choosing_agent_installer.queue_names
    None
    >>> print choosing_agent_installer.filter
    None
    >>> choosing_agent_installer.chooser.__name__
    'mock_work_chooser'
    >>> choosing_agent_installer.size
    3

    Like before, we register the new handler and check the agents are setup
    correctly.

    >>> zope.component.getGlobalSiteManager().unregisterHandler(
    ...     filtering_agent_installer)
    True
    >>> zope.component.provideHandler(choosing_agent_installer)
    >>> zope.event.notify(zc.async.interfaces.DispatcherActivated(da))

    >>> 'choosing_agent' in da
    True
    >>> da['choosing_agent'].chooser.__name__
    'mock_work_chooser'
    >>> print da['choosing_agent'].filter
    None

    A chooser and a filter cannot be both provided the the same time.

    >>> corrupt_installer = zc.async.subscribers.AgentInstaller(
    ...     'filtering_agent', chooser=mock_work_chooser,
    ...     filter=mock_work_filter)
    Traceback (most recent call last):
    ...
    ValueError: cannot set both chooser and filter to non-None
