![](\"figs/logo_01.png\"){kind=logo}
\n\n\n\n\n[](https://travis-ci.com/snorkel-team/snorkel?branch=master)\n[](https://snorkel.readthedocs.io/en/master)\n[](https://codecov.io/gh/snorkel-team/snorkel/branch/master)\n[](https://opensource.org/licenses/Apache-2.0)\n[](https://spectrum.chat/snorkel)\n\n***Programmatically Build and Manage Training Data***\n\n## Announcement\n\n**The Snorkel team is now focusing their efforts on Snorkel Flow, an end-to-end AI application development platform based on the core ideas behind Snorkel\u2014you can check it out [here](https://snorkel.ai) or [join us](www.snorkel.ai/careers) in building it!**\n\nThe [Snorkel project](https://snorkel.ai/how-to-use-snorkel-to-build-ai-applications/) started at Stanford in 2015 with a simple technical bet: that it would increasingly be the **training data**, not the models, algorithms, or infrastructure, that decided whether a machine learning project succeeded or failed. Given this premise, we set out to explore the radical idea that you could bring mathematical and systems structure to the messy and often entirely manual process of training data creation and management, starting by empowering users to **programmatically label, build, and manage** training data.\n\nTo say that the Snorkel project succeeded and expanded beyond what we had ever expected would be an understatement. The basic goals of a research repo like Snorkel are to provide a minimum viable framework for testing and validating hypotheses. Four years later, we\u2019ve been fortunate to do not just this, but to develop and deploy early versions of Snorkel in partnership with some of the world\u2019s leading organizations like [Google](https://ai.googleblog.com/2019/03/harnessing-organizational-knowledge-for.html), [Intel](https://dl.acm.org/doi/abs/10.1145/3329486.3329492), [Stanford Medicine](https://www.cell.com/patterns/fulltext/S2666-3899(20)30019-2), and many more; author over [thirty-six peer-reviewed publications](https://snorkel.ai/technology.html) on our findings around Snorkel and related innovations in weak supervision modeling, data augmentation, multi-task learning, and more; be included in courses at top-tier universities; support production deployments in systems that you\u2019ve likely used in the last few hours; and work with an amazing community of researchers and practitioners from industry, medicine, government, academia, and beyond.\n\nHowever, we realized increasingly\u2013from conversations with users in weekly office hours, workshops, online discussions, and industry partners\u2013that the Snorkel project was just the very first step. The ideas behind Snorkel change not just how you label training data, but so much of the entire lifecycle and pipeline of building, deploying, and managing ML: how users inject their knowledge; how models are constructed, trained, inspected, versioned, and monitored; how entire pipelines are developed iteratively; and how the full set of stakeholders in any ML deployment, from subject matter experts to ML engineers, are incorporated into the process.\n\nOver the last year, we have been building the platform to support this broader vision: [Snorkel Flow](https://snorkel.ai/platform.html), an end-to-end machine learning platform for developing and deploying AI applications. Snorkel Flow incorporates many of the concepts of the Snorkel project with a range of newer techniques around weak supervision modeling, data augmentation, multi-task learning, data slicing and structuring, monitoring and analysis, and more, all of which integrate in a way that is greater than the sum of its parts\u2013and that we believe makes ML truly faster, more flexible, and more practical than ever before.\n\nMoving forward, we will be focusing our efforts on Snorkel Flow. We are extremely grateful for all of you that have contributed to the Snorkel project, and are excited for you to check out our next chapter [here](https://snorkel.ai).\n\n\n# Quick Links\n* [Snorkel website](https://snorkel.org)\n* [Snorkel tutorials](https://github.com/snorkel-team/snorkel-tutorials)\n* [Snorkel documentation](https://snorkel.readthedocs.io/)\n* [Snorkel community forum](https://spectrum.chat/snorkel)\n* [Snorkel mailing list](https://groups.google.com/forum/#!forum/snorkel-ml)\n* [Snorkel Twitter](https://twitter.com/SnorkelAI)\n\n# Getting Started\nThe quickest way to familiarize yourself with the Snorkel library is to walk through the [Get Started](https://snorkel.org/get-started/) page on the Snorkel website, followed by the full-length tutorials in the [Snorkel tutorials](https://github.com/snorkel-team/snorkel-tutorials) repository.\nThese tutorials demonstrate a variety of tasks, domains, labeling techniques, and integrations that can serve as templates as you apply Snorkel to your own applications.\n\n\n# Installation\n\nSnorkel requires Python 3.6 or later. To install Snorkel, we recommend using `pip`:\n\n```bash\npip install snorkel\n```\n\nor `conda`:\n\n```bash\nconda install snorkel -c conda-forge\n```\n\nFor information on installing from source and contributing to Snorkel, see our\n[contributing guidelines](./CONTRIBUTING.md).\n\n\n\nThe following example commands give some more color on installing with `conda`.\nThese commands assume that your `conda` installation is Python 3.6,\nand that you want to use a virtual environment called `snorkel-env`.\n\n```bash\n# [OPTIONAL] Activate a virtual environment called \"snorkel\"\nconda create --yes -n snorkel-env python=3.6\nconda activate snorkel-env\n\n# We specify PyTorch here to ensure compatibility, but it may not be necessary.\nconda install pytorch==1.1.0 -c pytorch\nconda install snorkel==0.9.0 -c conda-forge\n```\n\n
\n\n\nIf you're using Windows, we highly recommend using Docker\n(you can find an example in our\n[tutorials repo](https://github.com/snorkel-team/snorkel-tutorials/blob/master/Dockerfile))\nor the [Linux subsystem](https://docs.microsoft.com/en-us/windows/wsl/faq).\nWe've done limited testing on Windows, so if you want to contribute instructions\nor improvements, feel free to open a PR!\n\n
\n