{ "info": { "author": "", "author_email": "", "bugtrack_url": null, "classifiers": [ "Topic :: System :: Distributed Computing", "Topic :: System :: Systems Administration" ], "description": "# c7n-log-exporter: Cloud watch log exporter automation\n\nA small serverless app to archive cloud logs across accounts to an archive bucket. It utilizes\ncloud log export to s3 feature for historical exports.\n\nIt also supports kinesis streams / firehose to move to realtime exports in the same format\nas the periodic historical exports.\n\n\n## Features\n\n - Log group filtering by regex\n - Incremental support based on previously synced dates\n - Incremental support based on last log group write time\n - Cross account via sts role assume\n - Lambda and CLI support.\n - Day based log segmentation (output keys look\n like $prefix/$account_id/$group/$year/$month/$day/$export_task_uuid/$stream/$log)\n \n\n## Assumptions\n\n - The archive bucket has already has appropriate bucket policy permissions.\n For details see:\n https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasks.html#S3Permissions\n - Default periodicity for log group archival into s3 is daily.\n - Exporter is run with account credentials that have access to the archive s3 bucket.\n - Catch up archiving is not run in lambda (do a cli run first)\n\n\n## Cli usage\n\n```\nmake install\n```\n\nYou can run on a single account / log group via the export subcommand\n```\nc7n-log-exporter export --help\n```\n\n## Config format\n\nTo ease usage when running across multiple accounts, a config file can be specified, as\nan example.\n\n```\ndestination:\n bucket: custodian-log-archive\n prefix: logs2\n\naccounts:\n - name: custodian-demo\n role: \"arn:aws:iam::111111111111:role/CloudCustodianRole\"\n groups:\n - \"/aws/lambda/*\"\n - \"vpc-flow-logs\"\n```\n\n## Multiple accounts via cli\n\nTo run on the cli across multiple accounts, edit the config.yml to specify multiple\naccounts and log groups.\n\n```\nc7n-log-exporter run --config config.yml\n```\n\n## Serverless Usage\n\nEdit config.yml to specify the accounts, archive bucket, and log groups you want to\nuse.\n\n```\nmake install\nmake deploy\n```", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/cloud-custodian/cloud-custodian", "keywords": "", "license": "Apache-2.0", "maintainer": "", "maintainer_email": "", "name": "c7n-logexporter", "package_url": "https://pypi.org/project/c7n-logexporter/", "platform": "", "project_url": "https://pypi.org/project/c7n-logexporter/", "project_urls": { "Homepage": "https://github.com/cloud-custodian/cloud-custodian" }, "release_url": "https://pypi.org/project/c7n-logexporter/0.3.2/", "requires_dist": null, "requires_python": "", "summary": "Cloud Custodian - Cloud Watch Log S3 exporter", "version": "0.3.2" }, "last_serial": 5876471, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "f80c0ab972d8945a04492ac7d4789483", "sha256": "bc14edefadb6798f4524d5c316736b36dc48894d5bf2234cc68a2fbf9a52c580" }, "downloads": -1, "filename": "c7n_logexporter-0.1.tar.gz", "has_sig": false, "md5_digest": "f80c0ab972d8945a04492ac7d4789483", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1303, "upload_time": "2017-05-16T07:50:00", "url": "https://files.pythonhosted.org/packages/ca/91/16fc9ed7d5974143b12dfeeaa461df1c3f7bbf8e9c20a7ffb74bec354d68/c7n_logexporter-0.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "e02413e5e2acc402bb531380722f2fbb", "sha256": "fc1793ba15bcbc609cfc38ac8ae86d39f6d4d279b8b8b0fd18cc2ff01a413b81" }, "downloads": -1, "filename": "c7n_logexporter-0.2.tar.gz", "has_sig": false, "md5_digest": "e02413e5e2acc402bb531380722f2fbb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2202, "upload_time": "2018-02-14T12:55:31", "url": "https://files.pythonhosted.org/packages/82/2a/ef2588d6b56156b95c4526f29883739c5da6d9757e3249069b7b3904f9c4/c7n_logexporter-0.2.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "144ad6b2e028b0cb55862b5f1c0e8481", "sha256": "5d860d94db586669ff877d2d16f334406eff6c97201165bf6869bcaa5f013c5f" }, "downloads": -1, "filename": "c7n_logexporter-0.3.1.tar.gz", "has_sig": false, "md5_digest": "144ad6b2e028b0cb55862b5f1c0e8481", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2245, "upload_time": "2019-09-23T22:34:38", "url": "https://files.pythonhosted.org/packages/ce/35/d68de08cc131f58b01c954833dd9ce49ebaf5f715200362b308e04770870/c7n_logexporter-0.3.1.tar.gz" } ], "0.3.2": [ { "comment_text": "", "digests": { "md5": "8a5c31e9d7ae8b0977ab3449f4252449", "sha256": "75d27eb8d515203f570cda2301382a45a85da0bc51f0c72c42cdc71f478566da" }, "downloads": -1, "filename": "c7n_logexporter-0.3.2.tar.gz", "has_sig": false, "md5_digest": "8a5c31e9d7ae8b0977ab3449f4252449", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2565, "upload_time": "2019-09-23T22:42:13", "url": "https://files.pythonhosted.org/packages/10/54/87702b80244fab03be65cbb5ce85b72a0db45d2b4324e8c1fd1eefbdcb66/c7n_logexporter-0.3.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "8a5c31e9d7ae8b0977ab3449f4252449", "sha256": "75d27eb8d515203f570cda2301382a45a85da0bc51f0c72c42cdc71f478566da" }, "downloads": -1, "filename": "c7n_logexporter-0.3.2.tar.gz", "has_sig": false, "md5_digest": "8a5c31e9d7ae8b0977ab3449f4252449", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2565, "upload_time": "2019-09-23T22:42:13", "url": "https://files.pythonhosted.org/packages/10/54/87702b80244fab03be65cbb5ce85b72a0db45d2b4324e8c1fd1eefbdcb66/c7n_logexporter-0.3.2.tar.gz" } ] }