\n\nPyGlow is a Python package which attempts to implement Keras like API struture on PyTorch backend. It provides functionalities which supports information theoretic methods in deep learning. These methods are relevant for understanding neural network dynamics in information plane. The package is equipped with a number of state-of-the-art algorithms for estimating and calculating various kinds of information theoretic measures. The package also provides intensive support for information bottleneck based methods in deep learning. \n\n\n\n# What's new with PyGlow ?\n\nPyGlow provide support for the following features of the 'IB theory of Deep Learning':\n\n- Functionalities for experimental IB-based DNN analysis\n\n- Test IB-based DNN performance bounds\n\n- Flexible API structure to test theoretical ideas and hypothesis related to IB-theory or information theory in general\n\n- IB-based training paradigms for DNN\n\n- Flexible internal pipeline which allows for the implementation of custom user-defined dependence criterions or IB-based loss functions \n\n\n\n# Examples\n\nLet's see with an example how you can really analyse the dynamics of DNNs using either in-built criterion or your own custom 'user-defined' criterion\n\n```python\n# importing glow module\nimport glow\nfrom glow.layers import Dense, Dropout, Conv2d, Flatten\nfrom glow.datasets import mnist\nfrom glow.models import IBSequential\nfrom glow.information_bottleneck.estimator import HSIC\n\n# define hyperparameters\n# hyperparameter\nbatch_size = 64\nnum_workers = 3\nvalidation_split = 0.2\nnum_epochs = 10\n\n# load the dataset\ntrain_loader, val_loader, test_loader = mnist.load_data(\n batch_size=batch_size, num_workers=num_workers, validation_split=validation_split\n)\n\n# define your model\nmodel = IBSequential(input_shape=(1, 28, 28), gpu=False, track_dynamics=False, save_dynamics=True)\nmodel.add(Conv2d(filters=16, kernel_size=3, stride=1, padding=1))\nmodel.add(Flatten())\nmodel.add(Dropout(0.4))\nmodel.add(Dense(500, activation='relu'))\nmodel.add(Dropout(0.4))\nmodel.add(Dense(200, activation='relu'))\nmodel.add(Dropout(0.2))\nmodel.add(Dense(10, activation='softmax'))\n```\n\nYes the code structure looks exactly like Keras ! so that you can really try out new ideas of your own without worrying much about the code and debugging !\n\nNow lets compile the model and attach HSIC criterion (more on this can be found on the [link]()) evaluator to it and finally train it ! To use your own custom built criterion visit this notebook [custom_criterion]().\n\n```python\n# compile the model\nmodel.compile(optimizer='SGD', loss='cross_entropy', metrics=['accuracy'])\n\n# attach evaluator for criterion (HSIC in example) used while calculating coordinates of dynamics\nmodel.attach_evaluator(HSIC(kernel='gaussian', gpu=True, sigma=5))\n\n# train the model\nmodel.fit_generator(train_loader, val_loader, num_epochs)\n```\n\nThat's it ! you can now extract the evaluated dynamics from the model. For complete code checkout this [notebook]().\n\nNow that you have seen the exciting implications of PyGlow and with how much clean interface you can implement information theoretic methods for your own model, just head over to install it on your local machine and to really open the box of deep learning !\n\nYou can find more examples on either [docs page](http://pyglow.000webhostapp.com/jupyternotebooks.html) or related notebooks in github repo at [examples]().\n\n\n\n# Requirements\n\nBefore installation there are some dependencies of PyGlow which needs to be taken care off for smooth installation.\n\nPyGlow requires the following Python packages:\n\n- NumPy, for basic numerical data structure\n- PyTorch, for supporting backend engine\n- tdqm, for progress bar functionalities\n\n\n\n# Installation\n\n## Installing dependencies\n\n### PyTorch\n\nPyGlow requires PyTorch backend, so to install it first run the following command (if already installed then skip this section).\n\n```console\npip install torch\n```\nNow its time to install PyGlow on your system !\n\n## Installing PyGlow\n\nCurrently the package is in development phase and can be installed from either Test PyPI or PyPI .\n\nFrom TestPyPI\n*************\n\n```console\npip install -i https://test.pypi.org/simple/ PyGlow\n```\n\nFrom PyPI\n*********\n```console\n pip install PyGlow\n```\n\n\n# Documentation\n\nEntire documentation is available on [Read the Docs](http://pyglow.000webhostapp.com/).\n\n\n\n# Problems\n\nIf something unexpected happens while installing or using the package, do report it into the [issue tracker](https://github.com/spino17/PyGlow/issues).\n\n\n\n# Contributing\n\nPyGlow is a community project and we definitely look forward for contributions from all of the enthusiastic people out there ! \n\nWhile you contribute, let us know that you are up for it by creating an issue or finding an existing one for the functionality or the feature you want to integrate into the core. Introduce your budding idea in the issue comments panel and discuss with us how are you planning to do it. \n\nSending direct PR without following the above steps is highly discouraged and might end up with a rejected PR.\n\n\n\n# Contact\n\nFeel free to contact regarding any PyGlow related affair at bhavyabhatt17@gmail.com, we would love to receive queries from your side. \n\n\n\n# License\n\nPyGlow is BSD-style licensed, including PyTorch license requirements as found in the LICENSE file.\n\n\n",
"description_content_type": "text/markdown",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/spino17/PyGlow",
"keywords": "",
"license": "BSD 3-clause",
"maintainer": "",
"maintainer_email": "",
"name": "PyGlow",
"package_url": "https://pypi.org/project/PyGlow/",
"platform": "",
"project_url": "https://pypi.org/project/PyGlow/",
"project_urls": {
"Homepage": "https://github.com/spino17/PyGlow"
},
"release_url": "https://pypi.org/project/PyGlow/0.1.7/",
"requires_dist": [
"torch (>=1.1.0)",
"numpy (>=1.15.0)",
"torchvision",
"tqdm (>=4.30.0)"
],
"requires_python": "",
"summary": "Information Theory of Deep Learning",
"version": "0.1.7"
},
"last_serial": 5824982,
"releases": {
"0.0.1": [
{
"comment_text": "",
"digests": {
"md5": "a52775b5e199383e488fb245ada33ddb",
"sha256": "bdc7c380dbef4c6c5e8f3937b33886bf1b5646eeaa12adea15e25fe50e1c3d11"
},
"downloads": -1,
"filename": "PyGlow-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "a52775b5e199383e488fb245ada33ddb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 954,
"upload_time": "2019-07-09T14:28:52",
"url": "https://files.pythonhosted.org/packages/f6/e4/a70e0f25b150dceb4fdd3f482446a3ad2b4cc7c69bf66e0b5ca9e8f63d25/PyGlow-0.0.1.tar.gz"
}
],
"0.1.6": [
{
"comment_text": "",
"digests": {
"md5": "4c5236ae17fed648401e7becde448314",
"sha256": "1785f0adc740c8a9cd60e2ebdb1da25fa6f010176d95fbbaf40ef774462e8175"
},
"downloads": -1,
"filename": "PyGlow-0.1.6-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "4c5236ae17fed648401e7becde448314",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 28555,
"upload_time": "2019-09-11T19:45:58",
"url": "https://files.pythonhosted.org/packages/27/99/5b7cf03adfa9451a763608a3fa60c8533866ab4ac3793945964d112c06a8/PyGlow-0.1.6-py2.py3-none-any.whl"
}
],
"0.1.7": [
{
"comment_text": "",
"digests": {
"md5": "4be2da101e52316cbc179ff85304b858",
"sha256": "728b3310c2399580d3413bde6a3abe3c632abf0050da728c38cf37ea00ca6329"
},
"downloads": -1,
"filename": "PyGlow-0.1.7-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "4be2da101e52316cbc179ff85304b858",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 28824,
"upload_time": "2019-09-13T10:53:26",
"url": "https://files.pythonhosted.org/packages/47/f7/1658494bbfce636d6559feeb5e9779ca3ba814b26d58ac8e8d2f9ac2d8c3/PyGlow-0.1.7-py2.py3-none-any.whl"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "4be2da101e52316cbc179ff85304b858",
"sha256": "728b3310c2399580d3413bde6a3abe3c632abf0050da728c38cf37ea00ca6329"
},
"downloads": -1,
"filename": "PyGlow-0.1.7-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "4be2da101e52316cbc179ff85304b858",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 28824,
"upload_time": "2019-09-13T10:53:26",
"url": "https://files.pythonhosted.org/packages/47/f7/1658494bbfce636d6559feeb5e9779ca3ba814b26d58ac8e8d2f9ac2d8c3/PyGlow-0.1.7-py2.py3-none-any.whl"
}
]
}
\n