{ "info": { "author": "Thierry Zamofing", "author_email": "thierry.zamofing@psi.ch", "bugtrack_url": null, "classifiers": [], "description": "Detector X-Ray Python Package\n=============================\n\n**most of these python scripts can be called with option '-h' to get more detailed help.**\n\nlibDetXR.py\n-----------\nwrapper to the libDetXR.(dll,so) library. These contains following C-functions:\n - Compression Algorithms\n - Imaging Algorithms\n - BitManip Functions\n\nfor details s.a.:\nC/C++: `../src/README.rst <../src/README.rst>`_\n\nSPEC2hdf5.py\n------------\n:Input: \\*.cbf files and \\*.dat files\n:Output: one \\*.hdf5 file\n:Speedup: muptiprocess single node\n\ncollects data of a scan on the cSAXS beamline and stores it in one HDF5 file\n\nThis will analyse and merge following data into one hdf5 file::\n\n specES1/dat-files/specES1_started_2013_04_26_1604.dat\n mcs/S00000-00999/S00033/e14472_00033.dat\n eiger/S00000-00999/S00033/e14472_1_00033_00000_00001.cbf and many more\n pilatus_1/S00000-00999/S00033/e14472_1_00033_00000_00001.cbf and many more\n pilatus_2/S00000-00999/S00033/e14472_1_00033_00000_00001.cbf and many more\n\nthis can be speedup on one compute node using all available cores.\n\nImgSource.py\n------------\nImgSource is a helper class to provide image. It is used among others in zmqImageSource, procMoment, procRoiStat, procSTXM\nCurrently following sources are supported::\n\n Random values:\n --src rand X Y Z --type=DATATYPE\n --src rand 2560 2160 3 --type=uint16\n Incremental values:\n --src inc X Y Z --type=uint16\n --src inc 2560 2160 6 --type=uint16\n Raw data from files *.raw in a directory:\n --src raw X Y PATH --type=DATATYPE\n --src raw 2560 2160 /scratch/detectorData/PCO/ --type=uint16\n TIFF data from files *.tif[f] in a directory:\n --src tif PATH\n --src tif /scratch/detectorData/PCO/\n CBF data from files *.cbf in a directory:\n --src cbf PATH\n --src cbf /scratch/detectorData/e14472/pilatus_1/S00000-00999/S00033/\n HDF file from a given object:\n --src hdf FILE ELEM\n --src hdf myfile.hdf5 entry/mygrp/dataset\n ZMQ source stream (now only uncompressed 'chunk'-type is supported)\n --src zmq JSON-config (with server,optional (queueSz, ifType and timeout))\n --src zmq '{\"server\":\"tcp://localhost:8080\",\"queueSz\":4,\"ifType\":\"PULL\"}',\n\nzmqImageSource.py\n-----------------\n\n| ZeroMQ PUSH or PUB Server:\n| the datasource can be one of the types that ImgSource supports. It then sends this images as ZMQ messages (chunk type, raw or compressed).\n\n:Input: a data source: \\*.cbf, \\*.tiff, \\*.hdf5 \\*.raw zmqStream etc. files\n:Output: one zmq-Streams with 'chunk' header (raw or compressed)\n:Speedup: only single process\n\nprocMoment.py\n-------------\nProcess the moments m00,m11,m01,m02,m10,m20 of an image source (cbf, tiff,raw files or hdf5 file) and stores the result in an output hdf5 file.\n\n:Input: ImgSource.py data: (raw, tiff, cbf, hdf5, zmq-steam, etc.), optional MATLAB Mask file for valid pixels\n:Output: one \\*.hdf5 file or a zmq Stream\n:Speedup: only single process\n\n| Further a visualization of the moments is possible.\n| Different implementation can be taken: python/opencv or C\n| Multi process speedup is not yet implemented but feasible on request.\n| SIMD implementation would also speedup on request.\n\nSingle image processing (32 bit, 1679x1475 pilatus_2 image, on pc 9477):\n:python: 0.5sec\n:pyFast: 0.03sec\n:openCV: 0.005sec\n:c: 0.005sec\nThe speed now is mostly memory bandwidth bound.\n\nprocRoiStat.py\n--------------\nProcess statistics of multiple ROI of an image source (cbf, tiff,raw files or hdf5 file) and stores the result (avg,avgstd,sumsq) for each ROI in an output hdf5 file.\n\n:Input: ImgSource.py data: (raw, tiff, cbf, hdf5, zmq-steam, etc.), and a MATLAB Roi file\n:Output: one \\*.hdf5 file or a zmq Stream\n:Speedup: only single process\n\nThe input for ROI definition is a MATLAB Roi file (e.g. pilatus_integration_mask.mat) as used at the cSAXS beamline.\n\n| Currently only the avg is implemented.\n| The speed is much higher as the origin MATLAB implementation.\n| Multi process speedup is not yet implemented but feasible on request.\n| SIMD implementation would also speedup on request.\n\n| Single image processing (32 bit, 1679x1475 pilatus_2 image, on pc 9477):\n| The speedup depends on the mask. The mask used is a 16 segments pilatus_integration_mask.mat\n| Speedup C-implementation vesus python: 41-42 x faster\n| The python implementation is allready much faster than the original MATLAB.\n\n:python: 0.235 sec\n:c: 0.006sec\n\nTherefore we can achive a speedup of 100-500x compared to the MATLAB code.\nThe speed now is mostly memory bandwidth bound.\n\nprocSTXM.py\n-----------\nSTXM processing of an image source and visualization. The current implementation is very basic, just as a template to later have a faster and more flexible STXM processing script.\n\n:Input: ImgSource.py data: (raw, tiff, cbf, hdf5, zmq-steam, etc.), optional MATLAB Mask file for valid pixels\n:Output: x,y,t during processing on console. STXM viewer and /tmp/result.npz file when processing is finished.\n:Speedup: only single process, multiproc-single node for hdf5->hdf5 processing\n\nzmqWriter.py\n------------\nZmqWriter connects to a ZMQ server that sends json- and binary data messages.\n\n:Input: one or multiple zmq-Streams with 'chunk' or 'pilatus'(cbf-files) header\n:Output: one \\*.hdf5 file\n:Speedup: muptiprocess single node (compression). Single process writing to file.\n\n| These data is stored into a hdf5 file or it can just copy ``*.cbf`` files.\n| The program can also be started as a REST server .\n| The REST-server and each writer is a independent process.\n\nMulti process compression speedup for the writer process is not foreseen, because the zmqWriter is intended to receive compressed chunks. Anyhow: the zmqWriter can convert images to a desired compression scheme, but this will work single threaded and could be therefore time critical::\n\n 1. *.cbf-Files -> *.cbf-Files\n 2. raw image -> hdf5-File (uncompressed, byte-shuffle, zlib lz4, lzf)\n 3. *.cbf-Files -> hdf5-File (uncompressed, byte-shuffle, zlib lz4, lzf)\n 4. compressed image-chunk -> hdf5-File (compressed as the received image-chunk)\n\nThe modes used in productions are 1 for cbf files as from the Pilatus and 4 for hdf5 files with the Eiger.\n\nThe modes 2 and 3 are for tests and preparation to hdf5 format and not recommanded for production. They are not performant\n\nCbf2CrystFELhdf5.py\n-------------------\nconverts cbf files to CrystFEL hdf5 files.\nthe program searches all \\*.cbf files of a directory and converts them.\nS.a. http://www.desy.de/~twhite/crystfel/\n\n\nFurther helper classes\n======================\n:utilities.py:\n utilities to create HDF5 objects\n\n:CbfParser.py:\n class to parse ``*.cbf`` files. It gets the header and decompress the binary part to a numpy array\n\n:DatParser.py:\n class to parse ``*.dat`` files at the cSAXS beamline.\n\n:zmq2imgGL.py:\n | a zmq sink to show an show a colored image that is received.\n | ``*.cbf`` zmq messages, uncompressed and compressed chunks are supported.\n\n:FileNameGen.py:\n generate filenames with a given filename pattern. (Currently not used)\n\n:hdf5vis.py:\n Simple Test application to show movies of image series in a hdf5-file\n\n:libDetXRTester.py:\n sample code to test the libDetXR functionality.\n\n:testPerfHdf5ChunkWrite.py:\n | Test the performance of Chunk writing:\n | all source data is first read into ram, then a HDF5 file is written.\n | diffferent interprocess communications are tested and compared:\n | ``'Pipe1','Pipe1b','Pipe2','Pool1','ShMem1','ShMem2','ShMem3','ShPool2'``", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "www.psi.ch", "keywords": null, "license": "(c) 2013 PSI", "maintainer": null, "maintainer_email": null, "name": "libDetXR", "package_url": "https://pypi.org/project/libDetXR/", "platform": "Linux,Windows", "project_url": "https://pypi.org/project/libDetXR/", "project_urls": { "Download": "UNKNOWN", "Homepage": "www.psi.ch" }, "release_url": "https://pypi.org/project/libDetXR/0.0.3.78/", "requires_dist": null, "requires_python": null, "summary": "(git:638683e) Detector X-Ray python Library", "version": "0.0.3.78" }, "last_serial": 2086648, "releases": { "0.0.3.78": [] }, "urls": [] }