Metadata-Version: 1.1
Name: pymdeco
Version: 0.1.0
Summary: PyMDECO - Python Meta Data Extractor and Collection Organizer library
Home-page: https://bitbucket.org/todor/pymdeco/
Author: Todor Bukov
Author-email: dev.todor@gmail.com
License: GNU Lesser General Public License version 3 or later(LGPLv3)
Description: PyMDECO - Python Meta Data Extractor and Collection Organizer
        =============================================================
        
        PyMDECO is a library which aims to facilitate the metadata extraction,
        storage and manipulation of a large collection of photos and multimedia
        (audio and video) files.
        Rather than developing own metadata parsers for the multitude of file
        types, it depends on third-party libraries and tools to extract the
        metadata from the files.
        
        
        Example usage
        --------------
        
        Example with a video file:
        
            >>> from pymdeco import services
            >>> srv = services.FileMetadataService()
            >>> meta = srv.get_metadata('/tests/big_buck_bunny_720p_surround.avi')
            >>> import json
            >>> print json.dumps(meta, indent=2) # to pretty print the metadata
            {
              "file_name": "big_buck_bunny_720p_surround.avi",
              "file_type": "video",
              "file_size": 332243668,
              "mime_type": "video/x-msvideo",
              "file_hash": {
                "value": "b957d6e6212638441b52d3b620af157cc8d40c2a0342669294854a06edcd528c",
                "algorithm": "sha256"
              },
              "file_timestamps": {
                "modified": "2008-06-11 13:29:26",
                "created": "2008-06-13 21:13:00"
              },
              "video_metadata": {
                "streams": [
                  {
                    "sample_aspect_ratio": "1:1",
                    "codec_type": "video",
                    "codec_name": "mpeg4",
                    "duration": "596.457",
                    "nb_frames": "14315",
                    "codec_time_base": "1/24",
                    "index": 0,
                    "width": 1280,
                    "divx_packed": "0",
                    "pix_fmt": "yuv420p",
            ... # output truncated
        
        
        Example with an audio file:
        
            >>> from pymdeco import services
            >>> srv = services.FileMetadataService()
            >>> meta = srv.get_metadata('/tests/jonobacon-freesoftwaresong2.mp3')
            >>> import json
            >>> print(json.dumps(meta, indent=2)) # to pretty print the metadata
            {
              "file_name": "jonobacon-freesoftwaresong2.mp3",
              "file_type": "audio",
              "file_size": 3169160,
              "mime_type": "audio/mpeg",
              "file_hash": {
                "value": "d7ebc161d5d8fb802659fea949204af2958906b91913ca7577cfaeece90ffb78",
                "algorithm": "sha256"
              },
              "file_timestamps": {
                "modified": "2012-01-09 20:02:23",
                "created": "2012-01-09 20:02:23"
              },
              "audio_metadata": {
                "streams": [
                  {
                    "index": 0,
                    "sample_fmt": "s16",
                    "codec_tag": "0x0000",
                    "bits_per_sample": 0,
                    "r_frame_rate": "0/0",
                    "start_time": "0.000",
                    "time_base": "1/14112000",
                    "codec_tag_string": "[0][0][0][0]",
                    "codec_type": "audio",
                    "channels": 2,
                    "codec_long_name": "MP3 (MPEG audio layer 3)",
                    "codec_name": "mp3",
            ... # output truncated
        
        Example with an image file:
        
            >>> from pymdeco import services
            >>> srv = services.FileMetadataService()
            >>> meta = srv.get_metadata('/tests/some_image.jpg')
            >>> print(json.dumps(meta, indent=2)) # to pretty print the metadata
            {
              "file_name": "some_image.jpg",
              "file_type": "image",
              "file_size": 159894,
              "mime_type": "image/jpeg",
              "file_hash": {
                "value": "844a8750f2c9e1a24175c8f158abb6d204ec2b79fc49aba512cded3cdb3a0111",
                "algorithm": "sha256"
              },
              "file_timestamps": {
                "modified": "2012-01-09 20:43:12",
                "created": "2012-01-09 20:43:12"
              },
              "image_metadata": {
                "Exif": {
                  "Photo": {
                    "LightSource": 0,
                    "PixelXDimension": 900,
                    "SubSecTime": "16",
            ... # output truncated ... Image details follow
            "Image": {
                    "YResolution": 72,
                    "ResolutionUnit": 2,
                    "Orientation": 1,
                    "Copyright": "Some Rights Reserved                                  ",
                    "Artist": "Yovko Lambrev                       ",
                    "Make": "NIKON CORPORATION",
                    "DateTime": "2008-09-13 11:26:41",
            ... # output truncated ... XMP details follow
                "Xmp": {
                  "iptc": {
                    "CreatorContactInfo/Iptc4xmpCore:CiEmailWork": "yovko@simplestudio.org",
                    "CreatorContactInfo/Iptc4xmpCore:CiAdrCity": "Sofia",
                    "CreatorContactInfo/Iptc4xmpCore:CiUrlWork": "http://simplestudio.org",
            ... # output truncated ... IPTC tags follow
                "Iptc": {
                  "Application2": {
                    "CountryName": "['Bulgaria']",
                    "Byline": "['Yovko Lambrev']",
                    "BylineTitle": "['Photographer']",
            ... # output truncated
        
        Example with a binary file:
        
            >>> from pymdeco import services
            >>> srv = services.FileMetadataService()
            >>> meta = srv.get_metadata('/tests/minimal_ubuntu_11.04_natty.iso')
            >>> import json
            >>> print (json.dumps(meta, indent=2))
            {
              "file_name": "minimal_ubuntu_11.04_natty.iso",
              "file_type": "application",
              "file_size": 19922944,
              "mime_type": "application/x-iso9660-image",
              "file_hash": {
                "value": "8607e2c06090db13b06a216efbeb65d7aeff4ca8666904e6874cf4a5960f2366",
                "algorithm": "sha256"
              },
              "file_timestamps": {
                "modified": "2011-05-02 17:24:18",
                "created": "2011-05-02 17:24:18"
              }
            }
        
        
        .. _installation_reference:
        
        Installation and building documentation
        ========================================
        
        Getting the latest development version:
        
            :command:`hg clone https://bitbucket.org/todor/pymdeco`
        
        
        Installing from source:
        
            :command:`python setup.py install`
        
        
        Generating the HTML documentation:
        
            :command:`python setup.py build_doc`
        
        
        If the installation is successfull the following code can be used to
        confirm the installation is ok and all dependencies are present:
        
        Checking dependencies:
        
            >>> from pymdeco.utils import check_dependencies
            >>> print json.dumps(check_dependencies(), indent=2)
            {
              "pyexiv2": "pyexiv2: 0.3.2",
              "ffprobe": "/usr/bin/ffprobe"
            }
        
        If any of the dictionary values are None, then PyMDECO is unable to find
        some of the dependencies.
        
        
        .. _dependencies_reference:
        
        Dependencies
        -------------
        
        *   Python 2.7+ or Python 3.2+
        *   `pyxeiv2 <http://tilloy.net/dev/pyexiv2/>`_ library (which is the Python
            bidnings to `exiv2 <http://exiv2.org/>`_). It is used to extract
            `EXIF <http://en.wikipedia.org/wiki/Exchangeable_image_file_format>`_,
            `XMP <http://en.wikipedia.org/wiki/Extensible_Metadata_Platform>`_ and
            `IPTC <http://en.wikipedia.org/wiki/Extensible_Metadata_Platform>`_
            tags from photos and images (JPEG, PNG, GIF, etc).
        *   :program:`ffprobe` binary (from `ffmpeg suite <http://ffmpeg.org/>`_)
            (version 0.9.0 or later) located in the system's :envvar:`PATH`.
            Older versions of the :program:`ffprobe` (prior 0.9.x) cannot
            output in JSON format.
        *   Sphinx (>= 1.0.0) - for building the documentation
        
        .. note::
        
            After the installation of the library one please ensure that
            :program:`ffprobe` executable is installed and available to the library
            (the executable file must be available in one of the directories listed in
            :envvar:`PATH` environmental variable.
        
            Refer to *ffmpeg* download page on the best way to install ffmpeg suite
            (which :program:`ffprobe` is part of):
            `http://ffmpeg.org/download.html <http://ffmpeg.org/download.html>`_.
            Under Windows it is recommended to use the static version of the program.
        
        
        Tutorial
        ========
        
        Assuming that all external dependencies have been satisfied (see
        :ref:`dependencies_reference` section for details) the easiest way to use
        the library is to instantiate a service from :mod:`pymdeco.services` module:
        
            Example:
        
                >>> import os, json
                >>> from pymdeco.services import FileMetadataService
                >>> srv = FileMetadataService()
                >>> for root, dirs, files in os.walk(r'/some/multimedia/files/'):
                ...   for afile in files:
                ...     meta = srv.get_metadata(os.path.join(root,afile))
                ...     print(json.dumps(meta, indent=2)) # to prettify the output
                ...
                # long prinout of metadata in nested dictionaries
        
        In PyMDECO, there is a separation between the various components for
        extracting metadata:
        
        * *Extractors* - The low-level functions which call the third-party libraries
          or invoke the external executable programs and parse their output. These
          functions expect the third-party libraries and tool to be present on the
          system, otherwise they won't work.
        
        * *Scanners* - Mid-level classes which can perform additional checks during
          the initializations and identify, load and check for the external
          dependencies and/or provide alternatives if they are not present.
          Scanners usually specialize in the extraction of only certain type of
          files i.e. only in audio or only in image metadata extraction.
        
        * *Services* - High-level classes which can combine multiple scanners to allow
          scanning of different types files. They also provide fall-back method if
          some of the scanners can not work due to missing dependencies.
        
        Each of these class of components sit in their own module. There is a clear
        separation of the of their duties and the components from a module only know
        and use the classes from the lower layers i.e.:
        *Services only use Scanners. Scanners only use Extractors.*
        
        Each Service and Scanner use the file's MIME
        (`Multipurpose Internet Mail Extensions <http://en.wikipedia.org/wiki/MIME>`_)
        to identify what kind of data it is likely to contain and then runs the
        appropriate function to get the metadata.
        
        The Services are the most convenient way to extract data as they know which
        Scanner to use for particular file type. However, sometimes the library user
        may wish to exert more control over which Scanner class is used, so it is
        possible to construct the service manually:
        
        Example:
        
            >>> from pymdeco import scanners, services
            >>> ais = scanners.AudioInfoScanner() # using the default MIME 'audio/*'
            >>> ais.pre_checks() # see API documentation for details
            >>> srv = services.FileMetadataService(scanners_list=[ais]) # using the default scanner for */*
            >>> srv.available_scanners()
            {'*/*': <pymdeco.scanners.FileInfoScanner object at 0x02A76A30>, \
            'audio/*': <pymdeco.scanners.AudioInfoScanner object at 0x02A768D0>}
            >>> vis = scanners.VideoInfoScanner()
            >>> vis.pre_checks()
            >>> srv.register_scanner("video/mpeg", vis) # registering with specific MIME
            >>> srv.available_scanners()
            {'video/mpeg': <pymdeco.scanners.VideoInfoScanner object at 0x02A76A90>, \
            '*/*': <pymdeco.scanners.FileInfoScanner object at 0x02A76A30>, \
            'audio/*': <pymdeco.scanners.AudioInfoScanner object at 0x02A768D0>}
            >>> srv.set_default_scanner(ais)
            >>> srv.available_scanners()
            {'video/mpeg': <pymdeco.scanners.VideoInfoScanner object at 0x02A76A90>, \
            '*/*': <pymdeco.scanners.AudioInfoScanner object at 0x02A768D0>, \
            'audio/*': <pymdeco.scanners.AudioInfoScanner object at 0x02A768D0>}
            >>> meta = srv.get_metadata('/some/video/file.mpeg')
            >>> # do something with the metadata
        
        Regardless of which method has been used to obtain the metadata, the
        result is always a (nested) Python dictionary with keys - undescore separated
        meaningfull descriptions of the metadata tag and values - either strings or
        numbers.
        The result can then easily be converted to JSON for transmission via HTTP or
        stored to document-oriented database (like
        `MongoDB <http://www.mongodb.org/>`_).
        
        
        API documentation
        -------------------
        
        Downloading the source and building the documentation (see
        :ref:`installation_reference` section) will create HTML version of the
        API documentation.
        
        
        History and future directions
        =============================
        
        The library was created out of necessity to manage large collection of photos
        (tens of thousands) and video clips (many thousands) collected over a large
        period of time from different sources and created by various models of
        photo cameras, camcorders and webcams.
        
        I quickly found out that to identify duplicates and effectively manage the
        backups I needed good and reliable metadata handling and storage.
        After experimenting with half a dozen Python packages and tools I settled
        on building a common API which makes use of well-established third-party
        libraries and external tools.
        
        So far only the metadata extraction part has been mostly completed, but
        development of some sort of persistent storage for that metadata is on the
        roadmap (involving the usage of MongoDB or Sqlite as potential backends). This
        persistent storage will be used to build collections with metadata and then
        constructing queries like "is this file already in the collection", "is there
        a *similar* file in the collection", "get all files with this metadata tag" or
        "list all the files created by the same device on the same date as this file".
        
        See TODO.txt for more details.
        
        
        Getting involved
        =================
        
        This library is at early stage, but is already capable to do enough useful
        metadata collection. If you are interested in improving it further and want
        to get involved, please send a mail to **pymdeco at googlegroups dot com**.
        
        
        License
        =======
        
        PyMDECO is licenced under GNU LGP (Lesser General Public License) version 3 or
        later. For details please refer to LICENSE.txt included in the source
        distribution.
        
Platform: any
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Plugins
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: Intended Audience :: End Users/Desktop
Classifier: License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)
Classifier: Operating System :: OS Independent
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX
Classifier: Operating System :: Unix
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3.2
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Multimedia
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires: pyexiv2 (>=0.3.0)
Requires: sphinx (>=1.0.0)
