{
"info": {
"author": "Six Feet Up, Inc.",
"author_email": "info@sixfeetup.com",
"bugtrack_url": null,
"classifiers": [
"Framework :: Plone :: 4.3",
"Framework :: Plone :: 5.0",
"Framework :: Zope2",
"Intended Audience :: Developers",
"License :: OSI Approved :: GNU General Public License (GPL)",
"Topic :: Software Development :: Libraries :: Python Modules"
],
"description": "Introduction\n============\n\n.. image:: http://www.sixfeetup.com/logos/solr-index.png\n :height: 111\n :width: 327\n :alt: SolrIndex\n :align: left\n\nSolrIndex is a product for Plone/Zope that provides enhanced searching capabilities by leveraging Solr, the popular open source enterprise search platform from the Apache Lucene project. It is compatible with Plone 4 and Plone 5.\n\nOut of the box, SolrIndex brings in more relevant search results by replacing Plone's default full-text indexing with Solr-based search features, and including the ability to assign weights to certain fields.\n\nLeveraging Solr's advanced search algorithms, SolrIndex comes with exciting features, such as the ability to use stopwords and synonyms. Stopwords allow to control which words the search mechanism should ignore, and synonyms make it possible to extend a query by including additional matches.\n\nSolrIndex also comes with blazing fast and highly scalable search capabilities. SolrIndex is extensible by design, which means it has the ability to integrate with other indexes and catalogs. This is good news for sites that need to provide search capabilities across multiple repositories.\n\nWith additional customization, SolrIndex also has the ability to provide faceted search, highlighting of query terms, spelling suggestions and \"more like this\" suggestions.\n\nThanks to SolrIndex, Plone and Zope-powered sites now benefit from truly enterprise search capabilities.\n\nUseful Links\n============\n\n- Solr: http://lucene.apache.org/solr/\n- PyPI: http://pypi.python.org/pypi/alm.solrindex\n- issue tracker: https://github.com/collective/alm.solrindex/issues\n- git repository: https://github.com/collective/alm.solrindex\n\n\nSpecial Thanks\n==============\n\nSix Feet Up would especially like to thank Shane Hathaway for his key contribution to SolrIndex.\n\nDetailed Documentation\n======================\n\n\nInstallation\n------------\n\nInclude this package in your Zope 2 or Plone buildout. If you are using\nthe ``plone.recipe.zope2instance`` recipe, add ``alm.solrindex`` to the\n``eggs`` parameter and the ``zcml`` parameter. See the ``buildout.cfg``\nin this package for an example. The example also shows how to use the\n``collective.recipe.solrinstance`` recipe to build a working Solr\ninstance with little extra effort.\n\nOnce Zope is running with this package installed, you can visit a\nZCatalog and add ``SolrIndex`` as an index. You should only add one\nSolrIndex to a ZCatalog, but a single SolrIndex can take the place of\nmultiple ZCatalog indexes.\n\n\nThe Solr Schema\n---------------\n\nConfigure the Solr schema to store an integer unique key. Add fields\nwith names matching the attributes of objects you want to index in Solr.\nYou should avoid creating a Solr field that will index the same data\nas what will be indexed in ZODB by another ZCatalog index. In other\nwords, if you add a ``Description`` field to Solr, you probably ought\nto remove the index named ``Description`` from ZCatalog, so that you\ndon't force your system to index descriptions twice.\n\nOnce the SolrIndex is installed, you can query all of the fields\ndescribed by the Solr schema, even if there is no ZCatalog index with\na matching name. For example, if you have configured a ``Description``\nfield in the Solr schema, then you can issue catalog queries against\nthe ``Description`` field using the same syntax you would use with\nother ZCatalog indexes. For example::\n\n results = portal.portal_catalog(Description={'query': 'waldo'})\n\nQueries of this form pass through a configurable translation layer made\nof field handler objects. When you need more flexibility than the field\nhandlers provide, you can either write your own field handlers (see the\n\"Writing Your Own Field Handlers\" section) or you can provide Solr\nparameters that do not get translated (see the \"Translucent Solr\nQueries\" section).\n\n\nTranslucent Solr Queries\n------------------------\n\nYou can issue a Solr query through a ZCatalog containing a SolrIndex by\nproviding a ``solr_params`` dictionary in the ZCatalog query. For\nexample, if you have a SolrIndex installed in portal_catalog, this call\nwill query Solr::\n\n results = portal.portal_catalog(solr_params={'q': 'waldo'})\n\nThe SolrIndex in the catalog will issue the query parameters specified\nin ``solr_params`` to Solr. Each parameter value can be a string\n(including unicode) or a list of strings. If you provide query\nparameters for other Solr fields, the parameters passed to Solr will be\nmixed with parameters generated for the other fields. Note that Solr\nrequires some value for the '``q``' parameter, so if you provide Solr\nparameters but no value for '``q``', SolrIndex will supply '``*:*``' as the\nvalue for '``q``'.\n\nSolr will return to the SolrIndex a list of matching document IDs and\nscores, then the SolrIndex will pass the document IDs and scores to\nZCatalog, then ZCatalog will intersect the document IDs with results\nfrom other indexes. Finally, ZCatalog will return a sorted list of\nresult objects (\"brain\" objects) to application code.\n\nIf you need access to the Solr response object, provide a\n``solr_callback`` function in the catalog query. After Solr sends its\nresponse, the SolrIndex will call the callback function with the parsed\nSolr response object. The response object conforms with the\ndocumentation of the ``solrpy`` package.\n\n\nHighlighting\n------------\n\nHighlighting data may be requested for any field marked as ``stored``\nin the Solr schema. To enable this feature, pass a ``highlight`` value of\neither ``True``, or a list of field names to highlight. A value of ``queried``\nwill cause Solr to return highlighting data for the list of queried columns.\nIf you pass in a sequence of field names, the requested highlighting data\nwill be limited to that list. You can also enable it by default in your Solr\nconfig file. If you do enable it by default in the config file, but don't\nwant it for a particular query, you must pass ``hl``:``off`` in solr_params.\n\nThe retrieved data is stored in the ``highlighting`` attribute on the\nreturned brain. To use the custom ``HighlightingBrain``, the index needs to\nbe able to connect to its parent catalog. The code attempts to retrieve a\nnamed utility for this, and will attempt to use Acquisition to find the id\nof its immediate parent. Failing that, it defaults to using ``portal_catalog``.\nIf the code cannot determine the name of your catalog automatically and you\nwant to use highlighting, you will need to change the ``catalog_name``\nproperty of the SolrIndex to reflect the correct value.\n\nTo retrieve the highlighting data, the brain will have a ``getHighlighting``\nmethod. By default, this is set to return the highlighting data for all\nfields in a single list. You can limit this to specific fields, and change\nthe return format to a dictionary keyed on field name by passing\n``combine_fields=False``.\n\nExample:\n\n results = portal.portal_catalog(SearchableText='lincoln',\n solr_params={'highlight': True})\n \n results[0].getHighlighting()\n [u'lincoln-collections Lincoln ',\n u'The collection of Lincoln plates']\n \n results[0].getHighlighting(combine_fields=False)\n {'SearchableText': [u'lincoln-collections Lincoln ']}\n 'Description': [u'The collection of Lincoln plates']}\n \n results[0].getHighlighting('Description')\n [u'The collection of Lincoln plates']\n \n results[0].getHighlighting('Description', combine_fields=False)\n {'Description': [u'The collection of Lincoln plates']}\n\nThe number of snippets returned, how the search terms are highlighted, and\nseveral other settings can all be tweaked in your Solr config.\n\nhttp://wiki.apache.org/solr/HighlightingParameters\n\n\nEncoding\n--------\n\nAll data submitted to Solr for indexing or as a query must be encoded as\nUTF-8. To this end, the SolrIndex has an ``expected_encodings`` lines\nproperty that details the list of encodings for it to try to decode data\nfrom before transcoding to UTF-8. If you submit data to be indexed or\nqueries with strings in a different encoding, you need to add that\nencoding to this list, before UTF-8.\n\nhttp://wiki.apache.org/solr/FAQ#Why_don.27t_International_Characters_Work.3F\n\n\nSorting\n-------\n\nSolrIndex only provides document IDs and scores, while ZCatalog retains\nthe responsibility for sorting the results. To sort the results from a\nquery involving SolrIndex, use the ``sort_on`` parameter like you\nnormally would with ZCatalog. At this time, you can not use a SolrIndex\nas the index to sort on, but that could change in the future.\n\n\nWriting Your Own Field Handlers\n-------------------------------\n\nField handlers serve two functions. They parse object attributes for\nindexing, and they translate field-specific catalog queries to Solr\nqueries. They are registered as utilities, so you can write your own\nhandlers and register them using ZCML.\n\nTo determine the field handler for a Solr field, ``alm.solrindex`` first\nlooks for an ``ISolrFieldHandler`` utility with a name matching the field\nname. If it doesn't find one, it looks for an ``ISolrFieldHandler`` utility\nwith a name matching the name of the Java class that handles the field\nin Solr. If that also fails, it retrieves the ``ISolrFieldHandler`` with no\nname.\n\nSee the documentation of the ``ISolrFieldHandler`` interface and the examples\nin handlers.py.\n\n\nIntegration with ZCatalog\n-------------------------\n\nOne ``SolrIndex`` can take the place of several ZCatalog indexes. In\ntheory, you could replace all of the catalog indexes with just a single\n``SolrIndex``. Don't do that yet, though, because this package needs\nmore maturity before it's ready to take on that many responsibilities.\n\nFurthermore, replacing all ZCatalog indexes might not be the right\ngoal. ZCatalog indexes are under appreciated. ZCatalog indexes are built\non the excellent transaction-aware object cache provided by ZODB. This\ngives them certain inherent performance advantages over network bound\nsearch engines like Solr. Any communication with Solr incurs a delay on\nthe order of a millisecond, while a ZCatalog index can often answer a\nquery in a few microseconds. ZCatalog indexes also simplify cluster\ndesign. The ZODB cache allows cluster nodes to perform searches without\nrelying on a large central search engine.\n\nWhere ZCatalog indexes currently fall short, however, is in the realm\nof indexing text. None of the text indexes available for ZCatalog match\nthe features and performance of text search engines like Solr.\n\nTherefore, one good way to use this package is to move all text indexes\nto Solr. That way, queries that don't need the text engine will avoid\nthe expense of invoking Solr. You can also move other kinds of indexes\nto Solr.\n\n\nHow This Package Maintains Persistent Connections\n-------------------------------------------------\n\nThis package uses a new method of maintaining an external database\nconnection from a ZODB object. Previous approaches included storing\n``_v_`` (volatile) attributes, keeping connections in a thread local\nvariable, and reusing the multi-database support inside ZODB, but\nthose approaches each have significant drawbacks.\n\nThe new method is to add dictionary called ``foreign_connections`` to\nthe ZODB Connection object (the ``_p_jar`` attribute of any persisted\nobject). Each key in the dictionary is the OID of the object that needs\nto maintain a persistent connection. Each value is an\nimplementation-dependent database connection or connection wrapper. If\nit is possible to write to the external database, the database\nconnection or connection wrapper should implement the ``IDataManager``\ninterface so that it can be included in transaction commit or abort.\n\nWhen a SolrIndex needs a connection to Solr, it first looks in the\n``foreign_connections`` dictionary to see if a connection has already\nbeen made. If no connection has been made, the SolrIndex makes the\nconnection immediately. Each ZODB connection has its own\n``foreign_connections`` attribute, so database connections are not\nshared by concurrent threads, making this a thread safe solution.\n\nThis solution is better than ``_v_`` attributes because connections will\nnot be dropped due to ordinary object deactivation. This solution is\nbetter than thread local variables because it allows the object\ndatabase to hold any number of external connections and it does not\nbreak when you pass control between threads. This solution is better\nthan using multi-database support because participants in a\nmulti-database are required to fulfill a complex contract that is\nirrelevant to databases other than ZODB.\n\nOther packages that maintain an external database connection should try\nout this scheme to see if it improves reliability or readability. Other\npackages should use the same ZODB Connection attribute name,\n``foreign_connections``, which should not cause any clashes, since\nOIDs can not be shared.\n\nAn implementation note: when ZODB objects are first created, they are\nnot stored in any database, so there is no simple way for the object to\nget a ``foreign_connections`` dictionary. During that time, one way to hold\na database connection is to temporarily fall back to the volatile\nattribute solution. That is what SolrIndex does (see the ``_v_temp_cm``\nattribute).\n\n\nTroubleshooting\n---------------\n\nIf the Solr index is preventing you from accessing Zope for some reason,\nyou can set ``DISABLE_SOLR=YES`` in the environment, causing the SolrIndex\nclass to bypass Solr for all queries and updates.\n\n\nChangelog\n=========\n\n1.2.0 (2016-10-15)\n------------------\n\n- Fix typo in solrpycore.\n [davidblewett]\n\n- Thanks to: \"Schorr, Dr. Thomas\" for the following\n encoding fixes, refs ticket #1:\n\n - Added a `expected_encodings` property to `SolrIndex` that lists the encodings\n to expect text in; each is tried in turn to decode each parameter sent to\n Solr. If none succeeds in decoding the text, we fall back to UTF8 and replace\n failing characters.\n http://wiki.apache.org/solr/FAQ#Why_don.27t_International_Characters_Work.3F\n [davidblewett]\n\n - Added `_encode_param` method to `SolrIndex` to encode a given string to UTF8.\n [davidblewett]\n\n - Modified `SolrIndex`'s '_apply_index` to send all parameters through the\n `_encode_param` method.\n [davidblewett]\n\n - Added a `test__apply_index_with_unicode` to ensure unicode queries are\n handled correctly.\n [davidblewett]\n\n- Initial highlighting support:\n\n - Imported `getToolByName` from `Products.CMFCore`, to be used on import failure.\n - Updated `SolrIndex` to pass any fields from the Solr schema that have stored=True to be highlighted.\n - Updated `SolrIndex` to store highlighting data returned from Solr in a `_highlighting` attribute.\n - Added a `HighlightingBrain` class that subclasses `AbstractCatalogBrain` that looks up the highlighted data in `SolrIndex`.\n - Added a `test__apply_index_with_highlighting` test; unfortunately, calling the `portal_catalog`\n is not working in the tests currently.\n\n [davidblewett]\n\n- Fixed : IIBTree needs integer keys\n http://plone.org/products/alm.solrindex/issues/3\n [thomasdesvenain]\n\n- Quick Plone 4 compatibility fixes\n [thomasdesvenain]\n\n- Search using ZCTextIndex '*' key character works with alm.solrindex.\n Makes livesearch works with solrindex as SearchableText index.\n [thomasdesvenain]\n\n- Highlighting is not activated by default because there can be severe performance issues.\n Pass 'highlight' parameter in solr_params to force it,\n and pass 'queried' as 'highlight' value to force highlight on queried fields only.\n [thomasdesvenain]\n\n- Improved unicode handling to correctly handle dictionaries passed in as a field search,\n in `SolrIndex._decode_param`.\n [davidblewett]\n\n- Extended ZCTextIndex support when a dictionary is passed in as a field search.\n [davidblewett]\n\n- Update test setup so that it is testing against Solr 1.4\n [claytron]\n\n- Handle empty ``dismax`` queries since a ``*:*`` value for ``q`` is not\n interpreted for the ``dismax`` query handler and returns no results\n rather than all results.\n [claytron]\n\n- Add uninstall profile, restoring the default Plone indizes.\n [thet]\n\n- Give the SolrIndex a meta_type 'SolrIndex' and register\n ATSimpleStringCriterion for it, otherwise Collections cannot add\n SearchableText criteria.\n [maurits]\n\n- Ensure that only one 'q' parameter is sent to Solr.\n [claytron]\n\n- Plone 4.1 compatibility.\n [timo]\n\n- Add missing elementtree import\n [saily]\n\n- Fix stale cached highlighting information that \n lead to in inconsistent results.\n [nrb]\n\n- Plone 4.3 compatibility.\n [cguardia]\n\n- Add support for solr.TrieDateField\n [mjpieters]\n\n- Fix decoding of query requests so that lists are not stringified\n before getting sent to field handlers.\n [davisagli]\n\n- Implement getIndexQueryNames which is now part of IPluggableIndex.\n [davisagli]\n\n- Add support for range queries to the DateFieldHandler.\n [davisagli]\n\n- Don't turn wildcard queries into fuzzy queries.\n [davisagli]\n\n- Confirm compatibility with Plone 5\n [witekdev, davisagli]\n\n\n1.1.1 (2010-11-04)\n------------------\n\n- Fix up links to issue tracker and Plone product page\n [clayton]\n\n1.1 (2010-10-12)\n----------------\n\n- Added `z3c.autoinclude` support for Plone\n [claytron]\n\n1.0 (2010-05-27)\n----------------\n\n- Initial public release\n\n- Clean up docs in prep for release.\n [claytron]\n\n- Fix up reST errors.\n [claytron]\n\n0.14 (2010-05-11)\n-----------------\n\n- Updated SolrConnectionManager to have a dummy savepoint\n implementation, refs #2451.\n [davidb]\n\n0.13 (2010-03-01)\n-----------------\n\n- commit to cleanup version #'s\n\n0.12 (2010-03-01)\n-----------------\n\n- PEP8 cleanup\n [clayton]\n\n0.11 (2009-11-27)\n-----------------\n\n- A commit after an aborted index update no longer breaks with an\n assertion error. Refs #1340\n\n0.10 (2009-10-15)\n-----------------\n\n- Filter out invalid XML characters from indexed documents.\n\n0.9 (2009-10-14)\n----------------\n\n- Fixed test failure by going to the login_form to log in, instead of\n the front page, where we get ambiguity errors.\n [maurits]\n\n- Fixed the catalog object information page. Solr was unable to parse\n a negative number in the query.\n\n\n0.8 (2009-09-18)\n----------------\n\n- Added support for Solr boolean fields.\n\n- GenericSetup profiles now have the option of clearing the\n index.\n\n- Made the waituri script wait up to 90 seconds by default,\n pause a little more between polls, and accept a timeout\n parameter.\n\n0.7 (2009-09-13)\n----------------\n\n- The Solr URI can now be provided by an environment variable,\n so that catalog.xml does not need to hard code the URI.\n\n0.6 (2009-09-11)\n----------------\n\n- Added narrative documentation.\n\n- Don't clear the index when running GenericSetup. Clearing\n indexes turns out to be a long-standing problem with GenericSetup;\n in this case the easy solution is to just not clear it.\n\n0.5 (2009-09-10)\n----------------\n\n- Added a script that waits for Solr to start up.\n\n- Brought in a private copy of solrpy to fix some bugs:\n\n - The connection retry code reconnected, but wasn't\n actually retrying the request.\n\n - The raw_query method should not assume the parameter\n values are unicode (they could be lists of unicode).\n\n0.4 (2009-09-10)\n----------------\n\n- Purge Solr when importing a SolrIndex via GenericSetup.\n\n0.3 (2009-09-10)\n----------------\n\n- Made field handlers more flexible. Now they can add any\n kind of query parameter to the Solr query.\n\n- The default field handler now generates \"fq\" parameters\n instead of \"q\" parameters. This seems to fit the intent of\n the Solr authors much better.\n\n- Renamed \"solr_additional\" to \"solr_params\".\n\n0.2 (2009-09-09)\n----------------\n\n- Added a GenericSetup profile that replaces SearchableText\n with a SolrIndex.\n\n- Renamed the catalog parameter for passing extra args to Solr\n \"solr_additional\". Also renamed the response callback\n parameter to \"solr_callback\".\n\n0.1 (2009-09-09)\n----------------\n\n- First release",
"description_content_type": null,
"docs_url": null,
"download_url": "UNKNOWN",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "UNKNOWN",
"keywords": "zope zcatalog solr plone",
"license": "BSD",
"maintainer": null,
"maintainer_email": null,
"name": "alm.solrindex",
"package_url": "https://pypi.org/project/alm.solrindex/",
"platform": "UNKNOWN",
"project_url": "https://pypi.org/project/alm.solrindex/",
"project_urls": {
"Download": "UNKNOWN",
"Homepage": "UNKNOWN"
},
"release_url": "https://pypi.org/project/alm.solrindex/1.2.0/",
"requires_dist": null,
"requires_python": null,
"summary": "A ZCatalog multi-index that uses Solr",
"version": "1.2.0"
},
"last_serial": 2401575,
"releases": {
"1.0": [
{
"comment_text": "",
"digests": {
"md5": "6217ee47f37380d32467d03eccdf3839",
"sha256": "1934f32175b39abca7398c7f41af9806d8d4cbf8954aa753d99ba795d27a093a"
},
"downloads": -1,
"filename": "alm.solrindex-1.0.zip",
"has_sig": false,
"md5_digest": "6217ee47f37380d32467d03eccdf3839",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 61663,
"upload_time": "2010-05-28T01:10:54",
"url": "https://files.pythonhosted.org/packages/47/6b/90ecbad4ce95a651ce9aa6c0ea83b19a72cd95e859d4b0258dd7fe24a58c/alm.solrindex-1.0.zip"
}
],
"1.1": [
{
"comment_text": "",
"digests": {
"md5": "d66e139916e19a6f722d16433fa2efd2",
"sha256": "85cd0c89267ff058a8fce94eb142e0e92849540117bac077a0e56ceda618f175"
},
"downloads": -1,
"filename": "alm.solrindex-1.1.zip",
"has_sig": false,
"md5_digest": "d66e139916e19a6f722d16433fa2efd2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 61869,
"upload_time": "2010-10-12T22:46:51",
"url": "https://files.pythonhosted.org/packages/84/f2/885c0c8b83e3b21b77fa80a7d3f09301fa3a31c3abb34b48f84638ba93f9/alm.solrindex-1.1.zip"
}
],
"1.1.1": [
{
"comment_text": "",
"digests": {
"md5": "d96fbfd3144eb6124ebd165982f8fa05",
"sha256": "849fa3544b6c880df75574ad2ef509be07f278d5db44086198c00c9f149730db"
},
"downloads": -1,
"filename": "alm.solrindex-1.1.1.zip",
"has_sig": false,
"md5_digest": "d96fbfd3144eb6124ebd165982f8fa05",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 62148,
"upload_time": "2010-11-05T14:45:17",
"url": "https://files.pythonhosted.org/packages/e2/1d/2b84fffa85022d37394be2b046b61a53fa37b023a880d4d252c83a325297/alm.solrindex-1.1.1.zip"
}
],
"1.2.0": [
{
"comment_text": "",
"digests": {
"md5": "0b03d21178c40632f6ae187511cecc5e",
"sha256": "1a061f11d52ca938d24008003cac04fd5ef30c5cf53651c48267e1ff2fc76fa4"
},
"downloads": -1,
"filename": "alm.solrindex-1.2.0.zip",
"has_sig": false,
"md5_digest": "0b03d21178c40632f6ae187511cecc5e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 74445,
"upload_time": "2016-10-15T20:03:58",
"url": "https://files.pythonhosted.org/packages/01/e2/165b37bd61ef9b9355840d5c66061b83654d725542c5e1e35e6aeb2c57bb/alm.solrindex-1.2.0.zip"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "0b03d21178c40632f6ae187511cecc5e",
"sha256": "1a061f11d52ca938d24008003cac04fd5ef30c5cf53651c48267e1ff2fc76fa4"
},
"downloads": -1,
"filename": "alm.solrindex-1.2.0.zip",
"has_sig": false,
"md5_digest": "0b03d21178c40632f6ae187511cecc5e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 74445,
"upload_time": "2016-10-15T20:03:58",
"url": "https://files.pythonhosted.org/packages/01/e2/165b37bd61ef9b9355840d5c66061b83654d725542c5e1e35e6aeb2c57bb/alm.solrindex-1.2.0.zip"
}
]
}