{ "info": { "author": "Erwan Gilbert", "author_email": "gilberterwan@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 1 - Planning", "License :: OSI Approved", "Natural Language :: French", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Topic :: Communications" ], "description": "\ufeffOracle_river - transfert de donnees depuis oracle vers elasticsearch en SQL incr\u00e9mental\r\n=======================================================================================\r\n\r\nCe module permet de recuperer des requetes SQL en mode incremental dans une base de donnees Oracle et de les stocker\r\nsous format JSON dans elasticsearch.\r\nL incr\u00e9mental des donn\u00e9es sera gere via une base MySQL (date de derniere execution)\r\n\r\nVous pouvez l'installer avec pip:\r\n\r\n pip install oracle_river\t\r\n\r\nsi vous rencontrez un message d erreur c est probablement que vous n avez pas le module pyelasticsearch \r\nqui sert de d\u00e9pendance \u00e0 oracle_river.\r\nInstallez le et reit\u00e9rez la commande pip install oracle_river\r\n\r\n\r\n |Oracle| => SQL incremental + serialisation JSON => | elasticsearch |\r\n \r\n\r\nExemple d'usage:\r\n\r\n 1) CREER UNE BASE MYSQL\r\n creez une base au nom de votre choix\r\n creez une table histo_connexion avec le format ci-dessous:\r\n\tID_CONNEXION / NOM_BASE / NOM_SCHEMA / DTM_DERNIERE_EXEC\r\n autoincrement/string 256/string256 /Timestamp\t\t\t\t\t\t\r\n\r\n\r\n 2) REALISER LES IMPORTS :\r\n\r\n\tcreez un fichier python .py dans lequel vous inscrivez les lignes de codes suivantes :\r\n\t(vous devez au pr\u00e9alable installer les API de connexion cx_Oralce et mysql.connector)\r\n\r\n #!/usr/bin/env python\r\n # coding: utf-8\r\n\r\n from datetime import datetime\r\n import cx_Oracle\r\n import mysql.connector\r\n from pyelasticsearch import ElasticSearch\r\n from oracle_river import Load_Data\r\n\r\n 3) DEFINIR LES PARAMETRES DE CONNEXION:\r\n\r\n # ouverture de la connexion \u00e0 la base oracle\r\n DB = 'pythonhol/welcome@127.0.0.1/orcl' (voir http://www.oracle.com/technetwork/articles/dsl/python-091105.html)\r\n OracleCnx = cx_Oracle.connect(DB)\r\n # param\u00e8tres de connexion elasticseach\r\n es = ElasticSearch('http://localhost:9200/')\r\n # ouverture de la connexion MySQL\r\n MySQLCnx = mysql.connector.connect(user='root', password='', host='localhost', database='elasticsearch_river')\r\n\r\n 4) PREPARER LA REQUETE:\r\n\r\n\trenseignez les param\u00e8tre ci-dessous :\r\n\r\n NomTable = 'CUSTOMER' (pour la base d'administration MySql)\r\n DtmInit = '2013-09-01 00:00:00' (si vous executez la requete pour la premi\u00e8re fois)\r\n DocType = 'CUSTOMER_ID' (cl\u00e9 primaire de la table oracle utilis\u00e9 comme type d index sur elasticsearch)\r\n Index = 'dbcustomer' (pour elasticsearch, minuscules obligatoire par elasticsearch)\r\n NomBase = 'DBBUSINESS' (pour la base d'administration MySql)\r\n NomSchema = 'ACTIVITY' (pour la base d'administration MySql)\r\n Query = (\"SELECT * FROM ACTIVITY.CUSTOMER WHERE CREATION_DTM >= to_timestamp('\\\r\n {0}', 'YYYY-MM-DD HH24:MI:SS') OR LAST_UPDATE_DTM>= to_timestamp('\\\r\n {0}', 'YYYY-MM-DD HH24:MI:SS')\")# les param\u00e9tres s'\u00e9crivent comme cela : \"{0},{1},{n}\".format(valeur0, valeur1, valeurn)\r\n\r\n 5) EXECUTER LA REQUETE VIA LA FONCTION PRESENTE DANS LE MODULE:\r\n\r\n Load_Data(MySQLCnx, es, OracleCnx, Query, DtmInit, DocType, Index, NomBase, NomSchema, NomTable)\r\n\r\n\tla requete defini plus haut va s executer et les donnees vont etre transferees a elasticseearch\r\n\tpar morceaux via le protocol http \r\n\r\nCe code est sous licence WTFPL.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/oneairone/oracle_river", "keywords": "", "license": "WTFPL", "maintainer": "", "maintainer_email": "", "name": "oracle_river", "package_url": "https://pypi.org/project/oracle_river/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/oracle_river/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/oneairone/oracle_river" }, "release_url": "https://pypi.org/project/oracle_river/1.0.2/", "requires_dist": null, "requires_python": null, "summary": "Charger des donnees oracle vers Elasticsearch / Load data from oracle to Elasticsearch", "version": "1.0.2" }, "last_serial": 2049360, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "28b81b39982ea45467e45b4db061dd6d", "sha256": "865a718708e0b688ceb5015013338663763afa3dd4ec018b65ca0932403a671f" }, "downloads": -1, "filename": "oracle_river-0.0.1.zip", "has_sig": false, "md5_digest": "28b81b39982ea45467e45b4db061dd6d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33313, "upload_time": "2016-04-01T17:29:01", "url": "https://files.pythonhosted.org/packages/40/52/d06c2e6833ca50e2011f076fe441eff8cc8f382118cf2b5edb9217752bb9/oracle_river-0.0.1.zip" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "2ec0ccf9776fe12c57aced88aa1ace7e", "sha256": "dfb1bdeef33096180954c0b1345c8d5ba463448ebafa4dda31379e5e0eadfcf3" }, "downloads": -1, "filename": "oracle_river-0.0.2.zip", "has_sig": false, "md5_digest": "2ec0ccf9776fe12c57aced88aa1ace7e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33396, "upload_time": "2016-04-01T18:45:49", "url": "https://files.pythonhosted.org/packages/ba/e7/0e98984eef5df0e09e01fc9dbc59d2ab1318a2558c7f0866b8f41658704e/oracle_river-0.0.2.zip" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "7f0264d1da2675cd996b6d48e586d9d8", "sha256": "83427a1191f1db4f4e122c316055ff5259e3bad21609e2dd7e207e2ef4fdfb84" }, "downloads": -1, "filename": "oracle_river-0.0.3.zip", "has_sig": false, "md5_digest": "7f0264d1da2675cd996b6d48e586d9d8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34649, "upload_time": "2016-04-01T19:00:03", "url": "https://files.pythonhosted.org/packages/09/98/0107ebda6e41abed6fa54100c6216e92f81b4267859db6bb8ac5743650e7/oracle_river-0.0.3.zip" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "58b8776e0022886fee97723e59cfb395", "sha256": "0a736a16880779feba26132680ceab74352111ccf7c6655338b1f2c46c62af05" }, "downloads": -1, "filename": "oracle_river-0.0.4.zip", "has_sig": false, "md5_digest": "58b8776e0022886fee97723e59cfb395", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 35325, "upload_time": "2016-04-03T13:29:06", "url": "https://files.pythonhosted.org/packages/ca/b6/66753c03665b7b5e508b52db053900cc4428764dbec621de8dc30e42759b/oracle_river-0.0.4.zip" } ], "1.0.0": [ { "comment_text": "", "digests": { "md5": "00c33cd7468c54b3ce17e21ce5d83a4c", "sha256": "589ddf832cdbf3291384f93f88dc89edf313e9a7a8086f2841bc5ef898a10505" }, "downloads": -1, "filename": "oracle_river-1.0.0.zip", "has_sig": false, "md5_digest": "00c33cd7468c54b3ce17e21ce5d83a4c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 36284, "upload_time": "2016-04-05T10:12:57", "url": "https://files.pythonhosted.org/packages/a3/f0/7af4df93bcec7aaecaafc727d2f926648536b7640c5da94ee13b0b137b64/oracle_river-1.0.0.zip" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "cbf2c99e9b96ff322ed5bde2283d5e2e", "sha256": "0fbaea84a8005db99ad181bd468317d3b95846f372bb3363e155c65d9fa49422" }, "downloads": -1, "filename": "oracle_river-1.0.1.zip", "has_sig": false, "md5_digest": "cbf2c99e9b96ff322ed5bde2283d5e2e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 36218, "upload_time": "2016-04-06T15:14:32", "url": "https://files.pythonhosted.org/packages/70/d5/b9c93b791f59e0f5be9ca09e6c7e207f38cc4cebbf9ffc89badde66f53ca/oracle_river-1.0.1.zip" } ], "1.0.2": [ { "comment_text": "", "digests": { "md5": "b6ae0a82a790e822e07fa035e87ca2f7", "sha256": "3e2edcd208b8d62d4a782c42c4091848a7cba48a75c0ed844833a1a1e207e4b5" }, "downloads": -1, "filename": "oracle_river-1.0.2.zip", "has_sig": false, "md5_digest": "b6ae0a82a790e822e07fa035e87ca2f7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 36225, "upload_time": "2016-04-05T11:20:53", "url": "https://files.pythonhosted.org/packages/9d/71/88423fff1faa84deb70c5217688e050e2002ded45a4a219d626c2883def9/oracle_river-1.0.2.zip" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "b6ae0a82a790e822e07fa035e87ca2f7", "sha256": "3e2edcd208b8d62d4a782c42c4091848a7cba48a75c0ed844833a1a1e207e4b5" }, "downloads": -1, "filename": "oracle_river-1.0.2.zip", "has_sig": false, "md5_digest": "b6ae0a82a790e822e07fa035e87ca2f7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 36225, "upload_time": "2016-04-05T11:20:53", "url": "https://files.pythonhosted.org/packages/9d/71/88423fff1faa84deb70c5217688e050e2002ded45a4a219d626c2883def9/oracle_river-1.0.2.zip" } ] }