{ "info": { "author": "Philippe Remy", "author_email": "", "bugtrack_url": null, "classifiers": [], "description": "# Conditional RNN (Tensorflow)\nConditions time series on categorical data.\n\n[![Downloads](https://pepy.tech/badge/cond-rnn)](https://pepy.tech/project/cond-rnn)\n[![Downloads](https://pepy.tech/badge/cond-rnn/month)](https://pepy.tech/project/cond-rnn/month)\n\n```\npip install cond-rnn\n```\n\n## TL;DR\n\n

\n \n

\n\nUseful if you have time series data with other inputs that do not depend on time. For example, weather data from two different cities: Paris and San Francisco. You want to predict the next temperature based on historical data. But at the same time, you expect the weather to change based on the city. You can either:\n- Combine the auxiliary features with the time series data (ugly!).\n- Concatenate the auxiliary features with the output of the RNN layer. It's some kind of post-RNN adjustment since the RNN layer won't see this auxiliary info.\n- Or just use this library! Long story short, initialize the RNN states with a learned representation of the condition (e.g. Paris or San Francisco). This way you model *elegantly* `P(x_{t+1}|x_{0:t}, cond)`.\n\n## API\n\n```python\noutputs = cond_rnn.ConditionalRNN(units=NUM_CELLS, cell='GRU', cond=cond)(inputs)\n```\n\nThe conditional RNN expects those parameters:\n\n- `units`: int, The number of units in the RNN Cell.\n- `cell`: string, cell class or object (pre-instantiated). In the case of string, 'GRU', 'LSTM' and 'RNN' are supported.\n- `inputs`: `3-D` Tensor with shape `[batch_size, timesteps, input_dim]`.\n- `cond`: `2-D` Tensor or list of tensors with shape `[batch_size, cond_dim]`. In the case of a list, the tensors can have a different `cond_dim`.\n- ` *args, **kwargs`: Any parameters of the [tf.keras.layers.RNN](https://www.tensorflow.org/api_docs/python/tf/keras/layers/RNN) class, such as `return_sequences`, `return_state`, `stateful`, `unroll`...\n\nHere is a self contained example of how it works:\n\n```python\nimport numpy as np\nimport tensorflow as tf\nfrom cond_rnn import ConditionalRNN\n\ntf.enable_eager_execution() # just for the example to make it short.\nbatch_size, time_steps, input_dim, cond_dim, num_cells = 16, 10, 1, 2, 8\ninputs = tf.constant(dtype=tf.float32, value=np.random.uniform(size=(batch_size, time_steps, input_dim)))\ncond = tf.constant(dtype=tf.float32, value=np.random.standard_normal(size=(batch_size, cond_dim)))\noutputs = ConditionalRNN(num_cells, cell='LSTM', cond=cond, return_sequences=True)(inputs)\nprint(outputs.shape) # (batch_size, time_steps, num_cells)\n# tf.Tensor([[[ 3.66962641e-01, ..., 3.78204376e-01]]], shape=(16, 10, 8), dtype=float32)\n```\n\nFor more examples, browse to [examples/](examples/)\n\n## Background\n\nThis implementation was inspired from the very good answer: [Adding Features To Time Series Model LSTM](https://datascience.stackexchange.com/a/17139), which I quote below. The option 3 was implemented in this library (with a slight modification: we do not add \ud835\udc63\u20d7 to the hidden state but rather overwrite the hidden state by \ud835\udc63\u20d7. We can argue that this is almost exactly the same thing as \ud835\udc63\u20d7 is obtained \ud835\udc63\u20d7 =\ud835\udc16\ud835\udc65\u20d7 +\ud835\udc4f\u20d7 where \ud835\udc4f\u20d7 could be the hidden state).\n\nFor RNNs (e.g., LSTMs and GRUs), the layer input is a list of timesteps, and each timestep is a feature tensor. That means that you could have a input tensor like this (in Pythonic notation):\n\n```\n# Input tensor to RNN\n[\n # Timestep 1\n [ temperature_in_paris, value_of_nasdaq, unemployment_rate ],\n # Timestep 2\n [ temperature_in_paris, value_of_nasdaq, unemployment_rate ],\n # Timestep 3\n [ temperature_in_paris, value_of_nasdaq, unemployment_rate ],\n ...\n]\n```\n\nSo absolutely, you can have multiple features at each timestep. In my mind, weather is a time series feature: where I live, it happens to be a function of time. So it would be quite reasonable to encode weather information as one of your features in each timestep (with an appropriate encoding, like cloudy=0, sunny=1, etc.).\n\nIf you have non-time-series data, then it doesn't really make sense to pass it through the LSTM, though. Maybe the LSTM will work anyway, but even if it does, it will probably come at the cost of higher loss / lower accuracy per training time.\n\nAlternatively, you can introduce this sort of \"extra\" information into your model outside of the LSTM by means of additional layers. You might have a data flow like this:\n\n```\nTIME_SERIES_INPUT ------> LSTM -------\\\n *---> MERGE ---> [more processing]\nAUXILIARY_INPUTS --> [do something] --/\n```\n\nSo you would merge your auxiliary inputs into the LSTM outputs, and continue your network from there. Now your model is simply multi-input.\n\nFor example, let's say that in your particular application, you only keep the last output of the LSTM output sequence. Let's say that it is a vector of length 10. You auxiliary input might be your encoded weather (so a scalar). Your merge layer could simply append the auxiliary weather information onto the end of the LSTM output vector to produce a single vector of length 11. But you don't need to just keep the last LSTM output timestep: if the LSTM outputted 100 timesteps, each with a 10-vector of features, you could still tack on your auxiliary weather information, resulting in 100 timesteps, each consisting of a vector of 11 datapoints.\n\nThe Keras documentation on its functional API has a good overview of this.\n\nIn other cases, you may want to condition the LSTM on non-temporal data. For example, predict the weather tomorrow, given location. In this case, here are three suggestions, each with positive/negatives:\n\n1. Have the first timestep contain your conditioning data, since it will effectively \"set\" the internal/hidden state of your RNN. Frankly, I would not do this, for a bunch of reasons: your conditioning data needs to be the same shape as the rest of your features, makes it harder to create stateful RNNs (in terms of being really careful to track how you feed data into the network), the network may \"forget\" the conditioning data with enough time (e.g., long training sequences, or long prediction sequences), etc.\n\n2. Include the data as part of the temporal data itself. So each feature vector at a particular timestep includes \"mostly\" time-series data, but then has the conditioning data appended to the end of each feature vector. Will the network learn to recognize this? Probably, but even then, you are creating a harder learning task by polluting the sequence data with non-sequential information. So I would also discourage this.\n\n3. Probably the best approach would be to directly affect the hidden state of the RNN at time zero. This is the approach taken by Karpathy and Fei-Fei and by Vinyals et al. This is how it works:\n\n * For each training sample, take your condition variables \ud835\udc65\u20d7 .\n * Transform/reshape your condition variables with an affine transformation to get it into the right shape as the internal state of the RNN: \ud835\udc63\u20d7 =\ud835\udc16\ud835\udc65\u20d7 +\ud835\udc4f\u20d7 (these \ud835\udc16 and \ud835\udc4f\u20d7 are trainable weights). You can obtain it with a Dense layer in keras.\n * For the very first timestep, add \ud835\udc63\u20d7 to the hidden state of the RNN when calculating its value.\nThis approach is the most \"theoretically\" correct, since it properly conditions the RNN on your non-temporal inputs, naturally solves the shape problem, and also avoids polluting your inputs timesteps with additional, non-temporal information. The downside is that this approach often requires graph-level control of your architecture, so if you are using a higher-level abstraction like Keras, you will find it hard to implement unless you add your own layer type.\n\nReferences:\n\n- https://adventuresinmachinelearning.com/recurrent-neural-networks-lstm-tutorial-tensorflow/\n- https://datascience.stackexchange.com/a/17139\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "cond-rnn", "package_url": "https://pypi.org/project/cond-rnn/", "platform": "", "project_url": "https://pypi.org/project/cond-rnn/", "project_urls": null, "release_url": "https://pypi.org/project/cond-rnn/1.4/", "requires_dist": null, "requires_python": "", "summary": "Conditional RNN", "version": "1.4" }, "last_serial": 5510623, "releases": { "1.0": [ { "comment_text": "", "digests": { "md5": "7fb657c45db8cd4c1997bc96e6833ef2", "sha256": "64bdaa4bce2756444cea8767fd6ce87cad131d1c96ae81d2c7c85510afd3ebde" }, "downloads": -1, "filename": "cond_rnn-1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "7fb657c45db8cd4c1997bc96e6833ef2", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 2228, "upload_time": "2019-06-12T06:15:56", "url": "https://files.pythonhosted.org/packages/47/75/4ac6ae1a8f8c7a84555026b2d3a38425df4ebecfef1bb86a1ec3a29f7a7d/cond_rnn-1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1b511962990856983cfb373c0f09778a", "sha256": "dc375efa1bc5153a41aa153500a58eb1440efe5d8d08a626fff037676686723e" }, "downloads": -1, "filename": "cond-rnn-1.0.tar.gz", "has_sig": false, "md5_digest": "1b511962990856983cfb373c0f09778a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1712, "upload_time": "2019-06-12T06:15:59", "url": "https://files.pythonhosted.org/packages/20/04/5f80ab3a8fed28adec7a6785bee891c0f5dc0e4e2d73e82927fd33478a85/cond-rnn-1.0.tar.gz" } ], "1.1": [ { "comment_text": "", "digests": { "md5": "5a1462c4876f840b7b5aa52a80a61338", "sha256": "9992b6ac2f1c2c97d9edc38ea5947063338be858b3820083562dd34c0ff2e81b" }, "downloads": -1, "filename": "cond_rnn-1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "5a1462c4876f840b7b5aa52a80a61338", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 5053, "upload_time": "2019-06-12T10:41:22", "url": "https://files.pythonhosted.org/packages/2c/62/2abb33ece73652c7aeadd772e48f61624053d583b0bdaab89fa04e84d301/cond_rnn-1.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "dd31e44a707a16dffbd033569ae19961", "sha256": "aaa6a5a7cf95c300439bd493598c3299c47afb4dcaff1504803ece8e9de6d263" }, "downloads": -1, "filename": "cond-rnn-1.1.tar.gz", "has_sig": false, "md5_digest": "dd31e44a707a16dffbd033569ae19961", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4853, "upload_time": "2019-06-12T10:41:25", "url": "https://files.pythonhosted.org/packages/b5/e2/2ae1c8b935e68c3edbffc8f1011db8f0fdc4b4973932d8c5425d52e7c9d5/cond-rnn-1.1.tar.gz" } ], "1.2": [ { "comment_text": "", "digests": { "md5": "5fb9afefe90b196da880a449c0e014b6", "sha256": "1ee822fb57f81f58f0186fbf032b6547e76c40e5b54cada7ab7f0594035aa7d2" }, "downloads": -1, "filename": "cond_rnn-1.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "5fb9afefe90b196da880a449c0e014b6", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 5608, "upload_time": "2019-06-13T04:39:32", "url": "https://files.pythonhosted.org/packages/d1/fe/ba919c2b4d690beae5b14fe1086c5ef2683792344f0ba2fd6c1671d118d6/cond_rnn-1.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "19816cc25db8eff6641ab5cc47c4d116", "sha256": "fcbd228b05538eb6f4639acfd2af3089c40d2f95796d676db5386e9b47877028" }, "downloads": -1, "filename": "cond-rnn-1.2.tar.gz", "has_sig": false, "md5_digest": "19816cc25db8eff6641ab5cc47c4d116", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5427, "upload_time": "2019-06-13T04:39:34", "url": "https://files.pythonhosted.org/packages/84/63/4d9fff0d1412863ebd5b95d02342eeb5194a3f753191e3856cd08dded761/cond-rnn-1.2.tar.gz" } ], "1.3": [ { "comment_text": "", "digests": { "md5": "3d615fd3f668b906dd4c115278a28798", "sha256": "e42c05e886be199105a1ad193501a532b68e532fb1412c99885c4ea065046806" }, "downloads": -1, "filename": "cond_rnn-1.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "3d615fd3f668b906dd4c115278a28798", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 5866, "upload_time": "2019-06-18T07:23:31", "url": "https://files.pythonhosted.org/packages/6d/d0/9cbaea1214ce4e7dafc29af8960f7b2586676d6c6e4e6ac26c8154fc3910/cond_rnn-1.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6e9041a90aaf7baa743bc5c696a0c3f6", "sha256": "19a83039fda4e92c7f8266b6ebb369f1cd727ceaa9178a57bd17e2806d486692" }, "downloads": -1, "filename": "cond-rnn-1.3.tar.gz", "has_sig": false, "md5_digest": "6e9041a90aaf7baa743bc5c696a0c3f6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5532, "upload_time": "2019-06-18T07:23:34", "url": "https://files.pythonhosted.org/packages/e6/c6/5c28a456a34a83381019b2f2a82f3265ce9f03d500e965d21b37a6859ffc/cond-rnn-1.3.tar.gz" } ], "1.4": [ { "comment_text": "", "digests": { "md5": "f06bb79261522efe99ec12a53d72cf2d", "sha256": "79a32e9f731392d93b5d9d7553768085edee802c5f36c5d6a23f5de31bb57f5b" }, "downloads": -1, "filename": "cond_rnn-1.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f06bb79261522efe99ec12a53d72cf2d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 5948, "upload_time": "2019-07-10T07:46:50", "url": "https://files.pythonhosted.org/packages/0f/f3/6dc58eca3e5d454e39c22d23dcb192a901a98107d111a75996a7735aff16/cond_rnn-1.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "7fee3db4961c561d1b56ac1a89165ff3", "sha256": "cfcdc43d8284147ecd482307e78dc8f82784b7e5f89175e3bab7831fddb715fe" }, "downloads": -1, "filename": "cond-rnn-1.4.tar.gz", "has_sig": false, "md5_digest": "7fee3db4961c561d1b56ac1a89165ff3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5686, "upload_time": "2019-07-10T07:46:52", "url": "https://files.pythonhosted.org/packages/c8/f5/1f85b64adf7a549b180cc1df6af75018efe09db378b70ff19d69ebccbb7f/cond-rnn-1.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "f06bb79261522efe99ec12a53d72cf2d", "sha256": "79a32e9f731392d93b5d9d7553768085edee802c5f36c5d6a23f5de31bb57f5b" }, "downloads": -1, "filename": "cond_rnn-1.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f06bb79261522efe99ec12a53d72cf2d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 5948, "upload_time": "2019-07-10T07:46:50", "url": "https://files.pythonhosted.org/packages/0f/f3/6dc58eca3e5d454e39c22d23dcb192a901a98107d111a75996a7735aff16/cond_rnn-1.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "7fee3db4961c561d1b56ac1a89165ff3", "sha256": "cfcdc43d8284147ecd482307e78dc8f82784b7e5f89175e3bab7831fddb715fe" }, "downloads": -1, "filename": "cond-rnn-1.4.tar.gz", "has_sig": false, "md5_digest": "7fee3db4961c561d1b56ac1a89165ff3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5686, "upload_time": "2019-07-10T07:46:52", "url": "https://files.pythonhosted.org/packages/c8/f5/1f85b64adf7a549b180cc1df6af75018efe09db378b70ff19d69ebccbb7f/cond-rnn-1.4.tar.gz" } ] }