├── README.md
├── license.txt
├── package.lisp
├── solr.asd
├── solr.lisp
├── solr.test.asd
├── test.lisp
└── triple-index
├── conf
├── admin-extra.html
├── elevate.xml
├── mapping-FoldToASCII.txt
├── mapping-ISOLatin1Accent.txt
├── protwords.txt
├── schema.xml
├── scripts.conf
├── solrconfig.xml
├── spellings.txt
├── stopwords.txt
├── synonyms.txt
├── velocity
│ ├── VM_global_library.vm
│ ├── browse.vm
│ ├── cluster.vm
│ ├── clusterResults.vm
│ ├── doc.vm
│ ├── facet_dates.vm
│ ├── facet_fields.vm
│ ├── facet_queries.vm
│ ├── facet_ranges.vm
│ ├── facets.vm
│ ├── footer.vm
│ ├── head.vm
│ ├── header.vm
│ ├── hit.vm
│ ├── jquery.autocomplete.css
│ ├── jquery.autocomplete.js
│ ├── layout.vm
│ ├── main.css
│ ├── query.vm
│ ├── querySpatial.vm
│ ├── suggest.vm
│ └── tabs.vm
└── xslt
│ ├── example.xsl
│ ├── example_atom.xsl
│ ├── example_rss.xsl
│ └── luke.xsl
└── solr.xml
/README.md:
--------------------------------------------------------------------------------
1 | This is a Solr binding for Allegro CL. Solr is an open-source
2 | freetext indexing/searching platform from the Apache Lucene project.
3 | See the following URL for its details.
4 |
5 | http://lucene.apache.org/solr/
6 |
7 | This package allows Allegro CL applications to communicate with a
8 | running Solr server, add and delete documents, and run queries to
9 | retrieve indexed records.
10 |
11 | The package comes with a solr.asd file. To use it just load :solr.
12 |
13 | (push #p"path/to/solr/source/directory" asdf:*central-registry*)
14 | (asdf:load-system :solr)
15 |
16 | Accessing the database
17 | ----------------------
18 |
19 | The Solr server should be running. To access the server,
20 | you need to create an instance of solr with the endpoint url.
21 | For example, if the server is running on localhost:8983,
22 | you can say:
23 |
24 | (defvar *solr* (make-instance 'solr :uri "http://localhost:8983/solr"))
25 |
26 | This action itself doesn't actually connect to the database, but the
27 | instance *solr* can be passed to other solr APIs to access to the
28 | database.
29 |
30 |
31 | Adding documents
32 | ----------------
33 |
34 | To add a document, you can use solr-add and solr-add* API.
35 |
36 | (solr-add *solr* '((:id . 123) (:name . "foobar") (:author . "xyzzy")))
37 |
38 | (solr-add* *solr* list-of-records)
39 |
40 | This adds the document with id=123, name="foobar" and author="xyzzy".
41 | The document record is semantically an unordered collection of named
42 | fields; you can pass an alist or a hashtable as a record. Field names
43 | are represented by keywords. Field values are mapped as follows:
44 |
45 | Lisp numbers => Solr numbers
46 | Lisp date-time object => Solr Datetime
47 | Lisp strings => Solr text
48 | Lisp t and nil => Solr boolean
49 |
50 | Non-empty Lisp lists can be used to represent set of values.
51 |
52 | By default, solr-add and solr-add* do not commit the change.
53 | solr-commit commits pending changes:
54 |
55 | (solr-commit *solr*)
56 |
57 | Or, you can discard uncommitted changes by solr-rollback:
58 |
59 | (solr-rollback *solr*)
60 |
61 | For convenience, solr-add and solr-add* accept a keyword argument
62 | commit, that automatically commits the change before returning:
63 |
64 | (solr-add* *solr* list-of-records :commit t)
65 |
66 | If you're adding large amount of documents, it is a good idea to send
67 | a bunch of documents together before committing using solr-add*,
68 | because it is much faster than adding documents one by one.
69 |
70 | Occasionally you may want to call solr-optimize to optimize indices
71 | for faster query performance:
72 |
73 | (solr-optimize *solr*)
74 |
75 | Deleting documents
76 | ------------------
77 |
78 | You can delete documents by listing document ids, or specifying
79 | queries:
80 |
81 | (solr-delete *solr* :ids '(1 13 17))
82 |
83 | (solr-delete *solr* :queries '("author:Shiro"))
84 |
85 | The deletion takes effects after committing. solr-delete accepts the
86 | commit keyword argument for autocommit.
87 |
88 |
89 | Querying documents
90 | ------------------
91 |
92 | solr-query does the query, and takes large number of keyword arguments
93 | to customize the query. See the Solr documentation for the full set
94 | of features. Here's an example of solr-query call:
95 |
96 | (solr-query *solr* :query "author:Shiro"
97 | :fields "id,name,author"
98 | :param-alist '((:rows . 100)))
99 |
100 | It returns LXML, a S-expression representaton of XML. It's up
101 | to the caller to extract necessary information from the returned
102 | LXML, but we provide a few convenience procedures for some
103 | basic extraction.
104 |
105 | (solr-result->response-count lxml)
106 |
107 | This returns three values: the total number of matching documents, the
108 | starting record number, and the number of documents included in the
109 | response. Note that Solr does pagenation by default--if you don't
110 | pass the :rows keyword it will only return the first 10 matching
111 | records. To retrieve other documents you need to pass the :start
112 | keyword to solr-query.
113 |
114 | Information on matching documents is in :doc XML elements.
115 |
116 | (solr-result->doc-nodes lxml)
117 |
118 | This procedure extracts and returns the list of doc elements,
119 | on which you can map to dig further information.
120 |
121 | The extracted doc elements are still LXML. The following procedure
122 | further converts the field values to Lisp objects according to the
123 | LXML attributes:
124 |
125 | (solr-result->doc-alist lxml)
126 |
127 |
128 | Error handling
129 | --------------
130 |
131 | When the Solr server returns an error (e.g. invalid query format), a
132 | condition solr-error is raised. It contains the Solr response status
133 | code, response headers and response body. The response body is in
134 | LXML.
135 |
136 | When the API failed to communicate with Solr server (e.g. the server
137 | isn't running), a socket-error condition is raised.
138 |
139 |
140 |
--------------------------------------------------------------------------------
/license.txt:
--------------------------------------------------------------------------------
1 | Eclipse Public License - v 1.0
2 |
3 | THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
4 |
5 | 1. DEFINITIONS
6 |
7 | "Contribution" means:
8 |
9 | a) in the case of the initial Contributor, the initial code and documentation distributed under this Agreement, and
10 | b) in the case of each subsequent Contributor:
11 | i) changes to the Program, and
12 | ii) additions to the Program;
13 | where such changes and/or additions to the Program originate from and are distributed by that particular Contributor. A Contribution 'originates' from a Contributor if it was added to the Program by such Contributor itself or anyone acting on such Contributor's behalf. Contributions do not include additions to the Program which: (i) are separate modules of software distributed in conjunction with the Program under their own license agreement, and (ii) are not derivative works of the Program.
14 | "Contributor" means any person or entity that distributes the Program.
15 |
16 | "Licensed Patents" mean patent claims licensable by a Contributor which are necessarily infringed by the use or sale of its Contribution alone or when combined with the Program.
17 |
18 | "Program" means the Contributions distributed in accordance with this Agreement.
19 |
20 | "Recipient" means anyone who receives the Program under this Agreement, including all Contributors.
21 |
22 | 2. GRANT OF RIGHTS
23 |
24 | a) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, distribute and sublicense the Contribution of such Contributor, if any, and such derivative works, in source code and object code form.
25 | b) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free patent license under Licensed Patents to make, use, sell, offer to sell, import and otherwise transfer the Contribution of such Contributor, if any, in source code and object code form. This patent license shall apply to the combination of the Contribution and the Program if, at the time the Contribution is added by the Contributor, such addition of the Contribution causes such combination to be covered by the Licensed Patents. The patent license shall not apply to any other combinations which include the Contribution. No hardware per se is licensed hereunder.
26 | c) Recipient understands that although each Contributor grants the licenses to its Contributions set forth herein, no assurances are provided by any Contributor that the Program does not infringe the patent or other intellectual property rights of any other entity. Each Contributor disclaims any liability to Recipient for claims brought by any other entity based on infringement of intellectual property rights or otherwise. As a condition to exercising the rights and licenses granted hereunder, each Recipient hereby assumes sole responsibility to secure any other intellectual property rights needed, if any. For example, if a third party patent license is required to allow Recipient to distribute the Program, it is Recipient's responsibility to acquire that license before distributing the Program.
27 | d) Each Contributor represents that to its knowledge it has sufficient copyright rights in its Contribution, if any, to grant the copyright license set forth in this Agreement.
28 | 3. REQUIREMENTS
29 |
30 | A Contributor may choose to distribute the Program in object code form under its own license agreement, provided that:
31 |
32 | a) it complies with the terms and conditions of this Agreement; and
33 | b) its license agreement:
34 | i) effectively disclaims on behalf of all Contributors all warranties and conditions, express and implied, including warranties or conditions of title and non-infringement, and implied warranties or conditions of merchantability and fitness for a particular purpose;
35 | ii) effectively excludes on behalf of all Contributors all liability for damages, including direct, indirect, special, incidental and consequential damages, such as lost profits;
36 | iii) states that any provisions which differ from this Agreement are offered by that Contributor alone and not by any other party; and
37 | iv) states that source code for the Program is available from such Contributor, and informs licensees how to obtain it in a reasonable manner on or through a medium customarily used for software exchange.
38 | When the Program is made available in source code form:
39 |
40 | a) it must be made available under this Agreement; and
41 | b) a copy of this Agreement must be included with each copy of the Program.
42 | Contributors may not remove or alter any copyright notices contained within the Program.
43 |
44 | Each Contributor must identify itself as the originator of its Contribution, if any, in a manner that reasonably allows subsequent Recipients to identify the originator of the Contribution.
45 |
46 | 4. COMMERCIAL DISTRIBUTION
47 |
48 | Commercial distributors of software may accept certain responsibilities with respect to end users, business partners and the like. While this license is intended to facilitate the commercial use of the Program, the Contributor who includes the Program in a commercial product offering should do so in a manner which does not create potential liability for other Contributors. Therefore, if a Contributor includes the Program in a commercial product offering, such Contributor ("Commercial Contributor") hereby agrees to defend and indemnify every other Contributor ("Indemnified Contributor") against any losses, damages and costs (collectively "Losses") arising from claims, lawsuits and other legal actions brought by a third party against the Indemnified Contributor to the extent caused by the acts or omissions of such Commercial Contributor in connection with its distribution of the Program in a commercial product offering. The obligations in this section do not apply to any claims or Losses relating to any actual or alleged intellectual property infringement. In order to qualify, an Indemnified Contributor must: a) promptly notify the Commercial Contributor in writing of such claim, and b) allow the Commercial Contributor to control, and cooperate with the Commercial Contributor in, the defense and any related settlement negotiations. The Indemnified Contributor may participate in any such claim at its own expense.
49 |
50 | For example, a Contributor might include the Program in a commercial product offering, Product X. That Contributor is then a Commercial Contributor. If that Commercial Contributor then makes performance claims, or offers warranties related to Product X, those performance claims and warranties are such Commercial Contributor's responsibility alone. Under this section, the Commercial Contributor would have to defend claims against the other Contributors related to those performance claims and warranties, and if a court requires any other Contributor to pay any damages as a result, the Commercial Contributor must pay those damages.
51 |
52 | 5. NO WARRANTY
53 |
54 | EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Each Recipient is solely responsible for determining the appropriateness of using and distributing the Program and assumes all risks associated with its exercise of rights under this Agreement , including but not limited to the risks and costs of program errors, compliance with applicable laws, damage to or loss of data, programs or equipment, and unavailability or interruption of operations.
55 |
56 | 6. DISCLAIMER OF LIABILITY
57 |
58 | EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
59 |
60 | 7. GENERAL
61 |
62 | If any provision of this Agreement is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this Agreement, and without further action by the parties hereto, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable.
63 |
64 | If Recipient institutes patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Program itself (excluding combinations of the Program with other software or hardware) infringes such Recipient's patent(s), then such Recipient's rights granted under Section 2(b) shall terminate as of the date such litigation is filed.
65 |
66 | All Recipient's rights under this Agreement shall terminate if it fails to comply with any of the material terms or conditions of this Agreement and does not cure such failure in a reasonable period of time after becoming aware of such noncompliance. If all Recipient's rights under this Agreement terminate, Recipient agrees to cease use and distribution of the Program as soon as reasonably practicable. However, Recipient's obligations under this Agreement and any licenses granted by Recipient relating to the Program shall continue and survive.
67 |
68 | Everyone is permitted to copy and distribute copies of this Agreement, but in order to avoid inconsistency the Agreement is copyrighted and may only be modified in the following manner. The Agreement Steward reserves the right to publish new versions (including revisions) of this Agreement from time to time. No one other than the Agreement Steward has the right to modify this Agreement. The Eclipse Foundation is the initial Agreement Steward. The Eclipse Foundation may assign the responsibility to serve as the Agreement Steward to a suitable separate entity. Each new version of the Agreement will be given a distinguishing version number. The Program (including Contributions) may always be distributed subject to the version of the Agreement under which it was received. In addition, after a new version of the Agreement is published, Contributor may elect to distribute the Program (including its Contributions) under the new version. Except as expressly stated in Sections 2(a) and 2(b) above, Recipient receives no rights or licenses to the intellectual property of any Contributor under this Agreement, whether expressly, by implication, estoppel or otherwise. All rights in the Program not expressly granted under this Agreement are reserved.
69 |
70 | This Agreement is governed by the laws of the State of New York and the intellectual property laws of the United States of America. No party to this Agreement will bring a legal action under this Agreement more than one year after the cause of action arose. Each party waives its rights to a jury trial in any resulting litigation.
71 |
--------------------------------------------------------------------------------
/package.lisp:
--------------------------------------------------------------------------------
1 | ;; -*- mode: common-lisp -*-
2 | ;; copyright (c) 2011-2016 Franz Inc, Oakland, CA - All rights reserved.
3 | ;; This program and the accompanying materials are made available under the
4 | ;; terms of the Eclipse Public License v1.0 which accompanies this
5 | ;; distribution (see license.txt), and is available at
6 | ;; http://www.eclipse.org/legal/epl-v10.html
7 |
8 | (eval-when (:compile-toplevel :load-toplevel :execute)
9 | (require :aserve)
10 | (require :datetime)
11 | (require :net-xml-generator)
12 | (require :pxml-sax)
13 | )
14 |
15 | (defpackage :solr
16 | (:use :cl :excl
17 | :util.date-time
18 | :net.aserve.client
19 | :net.xml.generator
20 | :net.xml.parser)
21 | (:export :solr
22 | :solr-error
23 | :solr-add
24 | :solr-add*
25 | :solr-commit
26 | :solr-optimize
27 | :solr-rollback
28 | :solr-delete
29 | :solr-query
30 | :solr-result->response-count
31 | :solr-result->doc-nodes
32 | :solr-result->doc-alist
33 | ))
34 |
--------------------------------------------------------------------------------
/solr.asd:
--------------------------------------------------------------------------------
1 | ;;;; -*- mode: common-lisp -*-
2 |
3 | (in-package :cl-user)
4 |
5 | (asdf:defsystem :solr
6 | :name "Solr"
7 | :author "Shiro Kawai / Franz Inc."
8 | :version "0.1"
9 |
10 | :components ((:file "package")
11 | (:file "solr" :depends-on ("package")))
12 |
13 | :in-order-to ((test-op (test-op :solr.test))))
14 |
15 | (defmethod asdf:perform ((op asdf:test-op)
16 | (system (eql (asdf:find-system :solr))))
17 | (asdf:load-system :solr.test)
18 | (funcall (find-symbol (symbol-name '#:run-test) :solr.test)))
19 |
--------------------------------------------------------------------------------
/solr.lisp:
--------------------------------------------------------------------------------
1 | ;; -*- mode: common-lisp -*-
2 | ;; copyright (c) 2011-2016 Franz Inc, Oakland, CA - All rights reserved.
3 | ;; This program and the accompanying materials are made available under the
4 | ;; terms of the Eclipse Public License v1.0 which accompanies this
5 | ;; distribution (see license.txt), and is available at
6 | ;; http://www.eclipse.org/legal/epl-v10.html
7 |
8 | (in-package :solr)
9 |
10 | (eval-when (compile eval)
11 | (setq *readtable* (excl:named-readtable :xml)))
12 |
13 | ;; Solr API
14 | ;;
15 | ;; Example usage:
16 | ;;
17 | ;; (defvar *solr* (make-instance 'solr :uri "http://localhost:8983/solr"))
18 | ;;
19 | ;; (solr-add *solr* '((:id . 123) (:name . "foobar") (:author . "xyzzy")))
20 | ;;
21 | ;; (solr-commit *solr*)
22 | ;;
23 | ;; (solr-query *solr* :query "name:foobar")
24 | ;;
25 | ;; (solr-delete *solr* :ids '(123))
26 | ;;
27 | ;; Condition(s):
28 | ;;
29 | ;; solr-error
30 | ;;
31 | ;; When Solr server returns an error (response whose status is not 200),
32 | ;; this condition is thrown. Slots are:
33 | ;;
34 | ;; status-code - the response status, e.g. 400
35 | ;; response-headers - assoc list of parsed response headers
36 | ;; response-body - LXML format of response body.
37 | ;;
38 | ;; Solr record representation:
39 | ;;
40 | ;; solr-add and solr-add* takes a record to represent a document.
41 | ;; Semantically, a solr record is a collection of named fields.
42 | ;; In the lisp world, it can be represented as an assoc list or
43 | ;; a hashtable.
44 | ;; Field names are represented by keywords.
45 | ;; Field values mapping:
46 | ;; Multiple values in Solr record are represented in Lisp list.
47 | ;; Numbers are mapped to Lisp numbers.
48 | ;; Datetime is mapped to date-time class.
49 | ;; Text is mapped to Lisp strings.
50 | ;; Boolean value is mapped to Lisp nil and t.
51 |
52 | ;;;
53 | ;;; Connection representation and condition
54 | ;;;
55 |
56 | ;; Public
57 | (defclass solr ()
58 | ((uri :initarg :uri
59 | :reader solr-uri
60 | :documentation "URI of Solr REST API endpoint, e.g. http://localhost:8983/solr")
61 | )
62 | (:documentation "An object holding Solr endpoint"))
63 |
64 | (defmethod print-object ((solr solr) stream)
65 | (print-unreadable-object (solr stream :type t)
66 | (princ (solr-uri solr) stream)))
67 |
68 | ;; Public
69 | (define-condition solr-error (error)
70 | ((status-code :initarg :status-code)
71 | (response-headers :initarg :response-headers)
72 | (response-body :initarg :response-body)))
73 |
74 | (defmethod print-object ((o solr-error) stream)
75 | (print-unreadable-object (o stream :type t)
76 | (let ((code (and (slot-boundp o 'status-code)
77 | (slot-value o 'status-code)))
78 | (body (and (slot-boundp o 'response-body)
79 | (slot-value o 'response-body)))
80 | (headers (and (slot-boundp o 'response-headers)
81 | (slot-value o 'response-headers)))
82 | (length-cutoff 40))
83 | (format stream "code: ~a, ~:d header~:p, response: ~a"
84 | code
85 | (length headers)
86 | (if (and body (stringp body))
87 | (if (> (length body) length-cutoff)
88 | (format nil "\"(starts with) ~a...\"" (subseq body 0 length-cutoff))
89 | body)
90 | body)))))
91 |
92 | ;; a utility macro
93 | (defmacro xml->string (&body body)
94 | (let ((s (gensym)))
95 | `(with-output-to-string (,s)
96 | (let ((*print-pretty* nil))
97 | (with-xml-generation (,s)
98 | ,@body)))))
99 |
100 | ;;;
101 | ;;; Updating
102 | ;;;
103 |
104 | ;; API
105 | (defmethod solr-add ((solr solr) doc &key (commit nil)
106 | (overwrite t))
107 | "Add a new document to the Solr pointed by SOLR.
108 | DOC can be a hashtable or an assoc list.
109 | If COMMIT is true, the record is committed immediately.
110 | If OVERWRITE is true, an existing record with the same key field will be
111 | replaced with DOC, if any.
112 |
113 | The value associated with each key can be a string, symbol, boolean,
114 | real number, date-time, or a nonempty list of them. Boolean value is
115 | converted to 'true' or 'false'. Strings and symbols are passed to Solr
116 | as strings. Reals are passed as numbers, and Data-time is converted to
117 | iso8601 format Solr expects. If it is a nonempty list, it is passed
118 | as multiple values with the same key. (An empty list is treated as a
119 | boolean false).
120 |
121 | Example:
122 | (solr-add solr '((:id . 1234) (:name . \"foo\")
123 | (:text . \"Lorem ipsum dolor sit amet, consectetur
124 | adipisicing elit, sed do eiusmod tempor incididunt ut labore et
125 | dolore magna aliqua.\"))
126 | :commit t)
127 |
128 | On success, returns LXML representation of the Solr server response.
129 | "
130 | (let ((msg (xml->string
131 | ^((add @overwrite (xbool overwrite))
132 | ^(doc (render-record doc))))))
133 | (post-request solr msg `((commit . ,(xbool commit))))))
134 |
135 | ;; API
136 | (defmethod solr-add* ((solr solr) docs &key (commit nil)
137 | (overwrite t))
138 | "Add a new documents to the Solr pointed by SOLR.
139 | DOCS is a list of hashtables or assoc lists.
140 | If COMMIT is true, the record is committed immediately.
141 | If OVERWRITE is true, an existing record with the same key field will be
142 | replaced with DOC, if any.
143 | On success, returns LXML representation of the Solr server response."
144 | (let ((msg (xml->string
145 | ^((add @overwrite (xbool overwrite))
146 | (dolist (doc docs)
147 | ^(doc (render-record doc)))))))
148 | (post-request solr msg `((commit . ,(xbool commit))))))
149 |
150 | ;; API
151 | (defmethod solr-commit ((solr solr) &key (wait-searcher t)
152 | (expunge-deletes nil))
153 | "Send COMMIT command.
154 | WAIT-SEARCHER controls whether the request watis until searcher objects
155 | to be warmed for use; default is T.
156 | EXPUNGE-DELETS controls whether sergments with deletes are merged away;
157 | default is NIL.
158 | On success, returns LXML representation of the Solr server response."
159 | (let ((msg (xml->string
160 | ^((commit @waitSearcher (xbool wait-searcher)
161 | @expungeDeletes (xbool expunge-deletes))))))
162 | (post-request solr msg)))
163 |
164 | ;; API
165 | (defmethod solr-optimize ((solr solr) &key (wait-searcher t)
166 | (max-segments 1))
167 | "Send OPTIMIZE command.
168 | WAIT-SEARCHER controls whether the request waits until searcher objects
169 | to be warmed for use; default is T.
170 | MAX-SEGMENTS sets the maximum number of segments to optimize down;
171 | default is 1.
172 | On success, returns LXML representation of the Solr server response."
173 | (let ((msg (xml->string
174 | ^((optimize @waitSearcher (xbool wait-searcher)
175 | @maxSegments max-segments)))))
176 | (post-request solr msg)))
177 |
178 | ;; API
179 | (defmethod solr-rollback ((solr solr))
180 | "Send ROLLBACK command.
181 | On success, returns LXML representation of the Solr server response."
182 | (post-request solr ""))
183 |
184 | ;; API
185 | (defmethod solr-delete ((solr solr) &key (ids nil) (queries nil) (commit nil))
186 | "Deletes the documents matching given IDs or queries.
187 | IDS takes a list of numeric ids; documents with matching uniqueKey field
188 | defined in schema are deleted.
189 | QUERIES takes a list of queies in strings. A simple one is :,
190 | such as \"author:Shiro\".
191 | If COMMIT is T, deletes are committed immediately.
192 | On success, returns LXML representation of the Solr server response."
193 | (let ((msg (xml->string
194 | ^(delete
195 | (dolist (id ids) ^(id @id))
196 | (dolist (q queries) ^(query @q))))))
197 | (post-request solr msg `((commit . ,(xbool commit))))))
198 |
199 | ;;;
200 | ;;; Query
201 | ;;;
202 |
203 | ;; API
204 | (defmethod solr-query ((solr solr) &key (query "*:*")
205 | (fields "*")
206 | (search-name "select")
207 | (score t)
208 | (sort nil)
209 | (param-alist nil)
210 | (result-type :whole) ;for backward comaptibility
211 | )
212 | "Searches documents according to the given QUERY.
213 | Returns Solr response in LXML.
214 | If Solr server returns an error, solr-error condition is raised.
215 |
216 | FIELDS specifies which fields to be included in the results;
217 | the default is \"*\". You can list multiple fields separated
218 | by comma, e.g. \"id,name\".
219 |
220 | SEARCH-NAME names the name of the customized search; if omitted,
221 | the default \"select\" search is used.
222 |
223 | SORT takes Solr sort specification in a string, e.g. \"name asc\"
224 | to sort by ascending name order, or \"inStock asc, price desc\"
225 | for combined sort.
226 |
227 | PARAM-ALIST can be used for passing additional query commands
228 | and parameters. For example, the following enables faceted search
229 | with \"cat\" and \"inStock\" categories:
230 |
231 | :param-alist '((:facet . t) (:facet.field \"cat\" \"inStock\"))
232 |
233 | Or, the following enables highlighting for the field \"name\" and
234 | \"features\".
235 |
236 | :param-alist '((:hl . t) (:hl.fl . \"name,features\"))
237 |
238 | By default, Solr returns the first 10 results. You can see the
239 | total number of results by :numFound attribute of the :result LXML node.
240 | To retrieve subsequent results, you need to pass :start parameter
241 | as follows:
242 |
243 | :param-alist '((:start . 10))
244 |
245 | This will return 11th to 20th results (or less if the result is exhausted).
246 | Alternatively, you can increase the number of results returned by one
247 | query by :rows parameter:
248 |
249 | :param-alist '((:rows . 1000))
250 | "
251 | (let ((uri (format nil "~a/~a" (solr-uri solr) search-name))
252 | (q `((q . ,query)
253 | (fl . ,fields)
254 | (score . ,(xbool score))
255 | ,@(if sort `((sort . ,sort)))
256 | ,@(loop for (k . v) in param-alist
257 | if (consp v)
258 | append (mapcar (lambda (vv) (cons k (render-value vv))) v)
259 | else
260 | collect (cons k (render-value v))
261 | end))))
262 | (multiple-value-bind (body status headers)
263 | (do-http-request/retry uri
264 | :method :get :query q :external-format #+allegro (crlf-base-ef :utf-8)
265 | #-allegro :utf-8)
266 | (translate-result
267 | (parse-response body status headers)
268 | result-type))))
269 |
270 | (defun translate-result (lxml type)
271 | (ecase type
272 | ((:whole) lxml)
273 | ((:nodes) (solr-result->doc-nodes lxml))
274 | ((:alist) (solr-result->doc-alist lxml))))
275 |
276 | ;; This woulb be a one-liner if we could use XPath, but I [SK] don't
277 | ;; want to depend on CL-XML just for that.
278 | (defun extract-response-node (lxml)
279 | (labels ((search-result (lxml)
280 | (cond ((not (consp lxml)) nil)
281 | ((and (consp lxml) (consp (car lxml))
282 | (eq (caar lxml) :result)
283 | (equal (cadr (member :name (cdar lxml))) "response"))
284 | lxml) ;found
285 | (t (dolist (node (cdr lxml))
286 | (let ((r (search-result node)))
287 | (when r (return-from extract-response-node r))))))))
288 | (search-result lxml)))
289 |
290 | (defun doc-node->alist (node)
291 | (labels ((get-name (n)
292 | (intern (cadr (member :name (cdar n))) :keyword))
293 | (get-value (n)
294 | (let ((type (if (consp (car n)) (caar n) (car n)))
295 | (vals (cdr n)))
296 | (ecase type
297 | ((:str) (car vals))
298 | ((:arr :lis) (mapcar #'get-value vals))
299 | ((:int :long) (parse-integer (car vals)))
300 | ((:float :double)
301 | (let* ((*read-default-float-format* (if (eq type :float)
302 | 'single-float
303 | 'double-float))
304 | (v (read-from-string (car vals))))
305 | (unless (realp v)
306 | (error "Invalid ~a number:" type (car vals)))
307 | v))
308 | ((:bool) (not (equal (car vals) "false")))
309 | ((:date) (parse-iso8601 (car vals)))))))
310 | (mapcar (lambda (n) (cons (get-name n) (get-value n))) (cdr node))))
311 |
312 | ;;
313 | ;; Result extractors
314 | ;;
315 |
316 | ;; API
317 | (defun solr-result->response-count (lxml)
318 | "From the LXML result of solr-query response, extract and returns three values: total number of hits, the start record number of the current response, and the number of records in this response."
319 | (let ((node (extract-response-node lxml)))
320 | (and node
321 | (values (parse-integer (getf (cdar node) :numFound))
322 | (parse-integer (getf (cdar node) :start))
323 | (length (cdr node))))))
324 |
325 | ;; API
326 | (defun solr-result->doc-nodes (lxml)
327 | "From the LXML result of solr-query response, extract and returns a list of :doc elements in LXML format."
328 | (cdr (extract-response-node lxml)))
329 |
330 | ;; API
331 | (defun solr-result->doc-alist (lxml)
332 | "From the LXML result of solr-query response, extract and returns a list of :doc elements in alist format.
333 | Values in the nodes are converted back to CL objects."
334 | (mapcar #'doc-node->alist (solr-result->doc-nodes lxml)))
335 |
336 | ;;;
337 | ;;; Some utilities
338 | ;;;
339 |
340 | ;; Retry if we get EADDRNOTAVAIL - it means we've consumed local ports
341 | ;; faster than the system reclaims it, so it is reasonable to retry
342 | ;;
343 | (defun do-http-request/retry (uri &rest keys)
344 | (loop
345 | (handler-case
346 | (return (apply #'do-http-request uri keys))
347 | (socket-error (condition)
348 | (if* (eq (stream-error-identifier condition) :address-not-available)
349 | then (sleep 0.01)
350 | else (error condition))))))
351 |
352 | ;; Common procedure for request-response
353 | (defun post-request (solr body &optional query-alist)
354 | (multiple-value-bind (body status headers)
355 | (do-http-request/retry (update-endpoint solr query-alist)
356 | :method :post :content body :content-type "text/xml"
357 | :external-format #+allegro (crlf-base-ef :utf-8) #-allegro :utf-8)
358 | (parse-response body status headers)))
359 |
360 | ;; Parse response
361 | (defun parse-response (body status headers)
362 | (destructuring-bind ((param content-type &optional charset))
363 | (net.aserve::parse-header-value (cdr (assoc :content-type headers)))
364 | (declare (ignore param charset))
365 | (let ((lxml (if* (string-equal content-type "application/xml")
366 | then (let ((*package* (find-package :keyword)))
367 | ;; the pxml parser returns ((:xml..) (:response ..))
368 | ;; but pxml-sax returns ((:response ..))
369 | ;; by prepending :xml the extract-response-node
370 | ;; function will accept the new or old parsed form
371 | (cons :xml (parse-xml body)))
372 | else body)))
373 | (when (not (eql status 200))
374 | (error 'solr-error :status-code status :response-headers headers
375 | :response-body lxml))
376 | lxml)))
377 |
378 | ;; Some Solr POST message can take optional parameters via url query string.
379 | ;; We can't use :query argument of do-http-request, for we have to use
380 | ;; both url query string and POST message body, while do-http-request
381 | ;; assumes the query string to be the POST message body.
382 | (defun update-endpoint (solr &optional query-params)
383 | (let ((uri (solr-uri solr)))
384 | (if query-params
385 | (format nil "~a/update?~a" uri
386 | (net.aserve:query-to-form-urlencoded query-params :external-format
387 | #+allegro (crlf-base-ef :utf-8)
388 | #-allegro :utf-8))
389 | (format nil "~a/update" uri))))
390 |
391 | ;; Rendering record to xml. Needs to be called within the dynamic
392 | ;; extent of with-xml-generation.
393 | (defun render-record (rec)
394 | (if* (hash-table-p rec)
395 | then (maphash #'render-field rec)
396 | else (loop for (key . val) in rec do (render-field key val))))
397 |
398 | (defun render-field (key val)
399 | (cond
400 | ((consp key) ^(doc (render-record (cons key val))))
401 | ((consp val) (dolist (v val) (render-field key v)))
402 | ((hash-table-p val) ^(doc (render-record val)))
403 | (t ^((field @name key) @(render-value val)))))
404 |
405 | (defun render-value (val)
406 | (etypecase val
407 | ;; emit double-floats with an #\E for exponentChar instead of #\D so solr can parse them
408 | (double-float (let ((*read-default-float-format* 'double-float))
409 | (format nil "~e" val)))
410 | (number val)
411 | (boolean (xbool val))
412 | (string val)
413 | (symbol (symbol-name val))
414 | (date-time
415 | (with-output-to-string (s)
416 | (let ((*date-time-fmt* "%Y-%m-%dT%H:%M:%SZ"))
417 | ;; ensure we use UTC
418 | (princ (ut-to-date-time (date-time-to-ut val) :time-zone 0) s))))))
419 |
420 | (defun xbool (val) (if val 'true 'false))
421 |
--------------------------------------------------------------------------------
/solr.test.asd:
--------------------------------------------------------------------------------
1 | (in-package :cl-user)
2 |
3 | (asdf:defsystem :solr.test
4 | :name "Unit test for Solr"
5 | :author "Shiro Kawai / Franz Inc."
6 | :version "0.1"
7 | :components ((:file "test"))
8 | :depends-on (:solr))
9 |
10 |
--------------------------------------------------------------------------------
/test.lisp:
--------------------------------------------------------------------------------
1 | ;; copyright (c) 2011-2016 Franz Inc, Oakland, CA - All rights reserved.
2 | ;; This program and the accompanying materials are made available under the
3 | ;; terms of the Eclipse Public License v1.0 which accompanies this
4 | ;; distribution (see license.txt), and is available at
5 | ;; http://www.eclipse.org/legal/epl-v10.html
6 | ;;
7 | ;; Testing Solr binding
8 | ;;
9 | ;; To run the test, we need a Solr server running with example data
10 | ;; loaded. If you get Solr binary distribution, go down to example/
11 | ;; directory and run the following command:
12 | ;;
13 | ;; $ java -jar start.jar
14 | ;;
15 | ;; This runs Solr example server in foreground.
16 | ;;
17 | ;; If this is the first time, you need to populate the example database.
18 | ;; In another shell window, go down to example/exampledocs/ directory
19 | ;; and run the following command:
20 | ;;
21 | ;; $ java -jar post.jar *.xml
22 | ;;
23 | ;; See http://lucene.apache.org/solr/tutorial.html for the details.
24 | ;;
25 |
26 | (cl:eval-when (:compile-toplevel :load-toplevel)
27 | (cl:require :tester)
28 | (cl:require :regexp2))
29 |
30 | (cl:defpackage #:solr.test
31 | (:use #:cl #:excl #:util.test #:solr)
32 | (:export #:run-test #:run-test-solr-server))
33 |
34 | (cl:in-package #:solr.test)
35 |
36 | (defvar *solr-port* 8983
37 | "default port used by example")
38 |
39 | (defvar *e* nil
40 | "with-solr macro binds this to a condition when socket-error occurs.
41 | Intended for diagnostics.")
42 |
43 | (defmacro with-solr ((var uri) &body body)
44 | (let ((uri_ (gensym)) (e_ (gensym)))
45 | `(let* ((,uri_ ,uri)
46 | (,var (make-instance 'solr :uri ,uri_)))
47 | (declare (ignorable ,var))
48 | (setf *e* nil)
49 | (handler-case
50 | (progn ,@body)
51 | (socket-error (,e_)
52 | (setf *e* ,e_)
53 | (error "Can't connect to the Solr server at ~a: Maybe it is not running? (original socket error=~s)"
54 | ,uri_ (slot-value ,e_ 'excl::identifier)))))))
55 |
56 | ;; run-test is run by (asdf:oos 'asdf:test-op :solr)
57 | (defun run-test (&key (port *solr-port*))
58 | (with-tests (:name "solr")
59 | (with-solr (solr (format nil "http://localhost:~a/solr" port))
60 | ;; Response count
61 | (let ((r (solr-query solr)))
62 | (test '(17 0 10) (solr-result->response-count r)
63 | :multiple-values t)
64 | (test 10 (length (solr-result->doc-nodes r)))
65 | (test 10 (length (solr-result->doc-alist r))))
66 | (let ((r (solr-query solr :param-alist '((:rows . 20)))))
67 | (test '(17 0 17) (solr-result->response-count r)
68 | :multiple-values t))
69 | (let ((r (solr-query solr :param-alist '((:start 10 :rows 20)))))
70 | (test '(17 10 7) (solr-result->response-count r)
71 | :multiple-values t))
72 | )))
73 |
74 |
--------------------------------------------------------------------------------
/triple-index/conf/admin-extra.html:
--------------------------------------------------------------------------------
1 |
17 |
18 |
32 |
--------------------------------------------------------------------------------
/triple-index/conf/elevate.xml:
--------------------------------------------------------------------------------
1 |
2 |
18 |
19 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
32 |
33 |
34 |
35 |
36 |
37 |
--------------------------------------------------------------------------------
/triple-index/conf/mapping-ISOLatin1Accent.txt:
--------------------------------------------------------------------------------
1 | # The ASF licenses this file to You under the Apache License, Version 2.0
2 | # (the "License"); you may not use this file except in compliance with
3 | # the License. You may obtain a copy of the License at
4 | #
5 | # http://www.apache.org/licenses/LICENSE-2.0
6 | #
7 | # Unless required by applicable law or agreed to in writing, software
8 | # distributed under the License is distributed on an "AS IS" BASIS,
9 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
10 | # See the License for the specific language governing permissions and
11 | # limitations under the License.
12 |
13 | # Syntax:
14 | # "source" => "target"
15 | # "source".length() > 0 (source cannot be empty.)
16 | # "target".length() >= 0 (target can be empty.)
17 |
18 | # example:
19 | # "À" => "A"
20 | # "\u00C0" => "A"
21 | # "\u00C0" => "\u0041"
22 | # "ß" => "ss"
23 | # "\t" => " "
24 | # "\n" => ""
25 |
26 | # À => A
27 | "\u00C0" => "A"
28 |
29 | # Á => A
30 | "\u00C1" => "A"
31 |
32 | # Â => A
33 | "\u00C2" => "A"
34 |
35 | # Ã => A
36 | "\u00C3" => "A"
37 |
38 | # Ä => A
39 | "\u00C4" => "A"
40 |
41 | # Å => A
42 | "\u00C5" => "A"
43 |
44 | # Æ => AE
45 | "\u00C6" => "AE"
46 |
47 | # Ç => C
48 | "\u00C7" => "C"
49 |
50 | # È => E
51 | "\u00C8" => "E"
52 |
53 | # É => E
54 | "\u00C9" => "E"
55 |
56 | # Ê => E
57 | "\u00CA" => "E"
58 |
59 | # Ë => E
60 | "\u00CB" => "E"
61 |
62 | # Ì => I
63 | "\u00CC" => "I"
64 |
65 | # Í => I
66 | "\u00CD" => "I"
67 |
68 | # Î => I
69 | "\u00CE" => "I"
70 |
71 | # Ï => I
72 | "\u00CF" => "I"
73 |
74 | # IJ => IJ
75 | "\u0132" => "IJ"
76 |
77 | # Ð => D
78 | "\u00D0" => "D"
79 |
80 | # Ñ => N
81 | "\u00D1" => "N"
82 |
83 | # Ò => O
84 | "\u00D2" => "O"
85 |
86 | # Ó => O
87 | "\u00D3" => "O"
88 |
89 | # Ô => O
90 | "\u00D4" => "O"
91 |
92 | # Õ => O
93 | "\u00D5" => "O"
94 |
95 | # Ö => O
96 | "\u00D6" => "O"
97 |
98 | # Ø => O
99 | "\u00D8" => "O"
100 |
101 | # Œ => OE
102 | "\u0152" => "OE"
103 |
104 | # Þ
105 | "\u00DE" => "TH"
106 |
107 | # Ù => U
108 | "\u00D9" => "U"
109 |
110 | # Ú => U
111 | "\u00DA" => "U"
112 |
113 | # Û => U
114 | "\u00DB" => "U"
115 |
116 | # Ü => U
117 | "\u00DC" => "U"
118 |
119 | # Ý => Y
120 | "\u00DD" => "Y"
121 |
122 | # Ÿ => Y
123 | "\u0178" => "Y"
124 |
125 | # à => a
126 | "\u00E0" => "a"
127 |
128 | # á => a
129 | "\u00E1" => "a"
130 |
131 | # â => a
132 | "\u00E2" => "a"
133 |
134 | # ã => a
135 | "\u00E3" => "a"
136 |
137 | # ä => a
138 | "\u00E4" => "a"
139 |
140 | # å => a
141 | "\u00E5" => "a"
142 |
143 | # æ => ae
144 | "\u00E6" => "ae"
145 |
146 | # ç => c
147 | "\u00E7" => "c"
148 |
149 | # è => e
150 | "\u00E8" => "e"
151 |
152 | # é => e
153 | "\u00E9" => "e"
154 |
155 | # ê => e
156 | "\u00EA" => "e"
157 |
158 | # ë => e
159 | "\u00EB" => "e"
160 |
161 | # ì => i
162 | "\u00EC" => "i"
163 |
164 | # í => i
165 | "\u00ED" => "i"
166 |
167 | # î => i
168 | "\u00EE" => "i"
169 |
170 | # ï => i
171 | "\u00EF" => "i"
172 |
173 | # ij => ij
174 | "\u0133" => "ij"
175 |
176 | # ð => d
177 | "\u00F0" => "d"
178 |
179 | # ñ => n
180 | "\u00F1" => "n"
181 |
182 | # ò => o
183 | "\u00F2" => "o"
184 |
185 | # ó => o
186 | "\u00F3" => "o"
187 |
188 | # ô => o
189 | "\u00F4" => "o"
190 |
191 | # õ => o
192 | "\u00F5" => "o"
193 |
194 | # ö => o
195 | "\u00F6" => "o"
196 |
197 | # ø => o
198 | "\u00F8" => "o"
199 |
200 | # œ => oe
201 | "\u0153" => "oe"
202 |
203 | # ß => ss
204 | "\u00DF" => "ss"
205 |
206 | # þ => th
207 | "\u00FE" => "th"
208 |
209 | # ù => u
210 | "\u00F9" => "u"
211 |
212 | # ú => u
213 | "\u00FA" => "u"
214 |
215 | # û => u
216 | "\u00FB" => "u"
217 |
218 | # ü => u
219 | "\u00FC" => "u"
220 |
221 | # ý => y
222 | "\u00FD" => "y"
223 |
224 | # ÿ => y
225 | "\u00FF" => "y"
226 |
227 | # ff => ff
228 | "\uFB00" => "ff"
229 |
230 | # fi => fi
231 | "\uFB01" => "fi"
232 |
233 | # fl => fl
234 | "\uFB02" => "fl"
235 |
236 | # ffi => ffi
237 | "\uFB03" => "ffi"
238 |
239 | # ffl => ffl
240 | "\uFB04" => "ffl"
241 |
242 | # ſt => ft
243 | "\uFB05" => "ft"
244 |
245 | # st => st
246 | "\uFB06" => "st"
247 |
--------------------------------------------------------------------------------
/triple-index/conf/protwords.txt:
--------------------------------------------------------------------------------
1 | # The ASF licenses this file to You under the Apache License, Version 2.0
2 | # (the "License"); you may not use this file except in compliance with
3 | # the License. You may obtain a copy of the License at
4 | #
5 | # http://www.apache.org/licenses/LICENSE-2.0
6 | #
7 | # Unless required by applicable law or agreed to in writing, software
8 | # distributed under the License is distributed on an "AS IS" BASIS,
9 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
10 | # See the License for the specific language governing permissions and
11 | # limitations under the License.
12 |
13 | #-----------------------------------------------------------------------
14 | # Use a protected word file to protect against the stemmer reducing two
15 | # unrelated words to the same base word.
16 |
17 | # Some non-words that normally won't be encountered,
18 | # just to test that they won't be stemmed.
19 | dontstems
20 | zwhacky
21 |
22 |
--------------------------------------------------------------------------------
/triple-index/conf/schema.xml:
--------------------------------------------------------------------------------
1 |
2 |
18 |
19 |
47 |
48 |
49 |
58 |
59 |
60 |
67 |
68 |
69 |
70 |
71 |
72 |
73 |
74 |
75 |
76 |
89 |
90 |
93 |
94 |
95 |
96 |
97 |
98 |
108 |
109 |
110 |
111 |
112 |
113 |
135 |
136 |
137 |
138 |
139 |
140 |
141 |
150 |
151 |
152 |
153 |
154 |
155 |
156 |
157 |
167 |
168 |
169 |
170 |
171 |
172 |
173 |
184 |
185 |
186 |
197 |
198 |
204 |
205 |
206 |
207 |
208 |
209 |
210 |
211 |
212 |
221 |
222 |
223 |
224 |
227 |
231 |
236 |
237 |
238 |
239 |
240 |
241 |
242 |
243 |
244 |
249 |
250 |
251 |
252 |
253 |
254 |
255 |
256 |
257 |
259 |
260 |
261 |
262 |
263 |
264 |
265 |
266 |
267 |
268 |
270 |
271 |
272 |
273 |
274 |
275 |
276 |
277 |
278 |
279 |
280 |
281 |
282 |
283 |
284 |
285 |
286 |
291 |
292 |
293 |
294 |
295 |
296 |
297 |
300 |
301 |
302 |
303 |
304 |
305 |
306 |
308 |
309 |
310 |
311 |
312 |
317 |
318 |
319 |
320 |
321 |
322 |
323 |
331 |
332 |
336 |
337 |
338 |
341 |
342 |
345 |
346 |
347 |
348 |
359 |
362 |
363 |
364 |
365 |
366 |
367 |
368 |
369 |
370 |
371 |
372 |
373 |
374 |
375 |
386 |
387 |
388 |
389 |
390 |
391 |
392 |
393 |
394 |
395 |
396 |
397 |
398 |
399 |
400 |
401 |
402 |
403 |
404 |
406 |
407 |
408 |
419 |
420 |
421 |
422 |
423 |
424 |
428 |
429 |
430 |
431 |
432 |
433 |
455 |
456 |
457 |
458 |
459 |
460 |
464 |
465 |
466 |
467 |
468 |
471 | id
472 |
473 |
474 | text
475 |
476 |
477 |
478 |
479 |
482 |
483 |
487 |
492 |
493 |
494 |
495 |
--------------------------------------------------------------------------------
/triple-index/conf/scripts.conf:
--------------------------------------------------------------------------------
1 | # Licensed to the Apache Software Foundation (ASF) under one or more
2 | # contributor license agreements. See the NOTICE file distributed with
3 | # this work for additional information regarding copyright ownership.
4 | # The ASF licenses this file to You under the Apache License, Version 2.0
5 | # (the "License"); you may not use this file except in compliance with
6 | # the License. You may obtain a copy of the License at
7 | #
8 | # http://www.apache.org/licenses/LICENSE-2.0
9 | #
10 | # Unless required by applicable law or agreed to in writing, software
11 | # distributed under the License is distributed on an "AS IS" BASIS,
12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | # See the License for the specific language governing permissions and
14 | # limitations under the License.
15 |
16 | user=
17 | solr_hostname=localhost
18 | solr_port=8983
19 | rsyncd_port=18983
20 | data_dir=
21 | webapp_name=solr
22 | master_host=
23 | master_data_dir=
24 | master_status_dir=
25 |
--------------------------------------------------------------------------------
/triple-index/conf/solrconfig.xml:
--------------------------------------------------------------------------------
1 |
2 |
18 |
19 |
23 |
24 |
31 |
32 |
42 | ${solr.abortOnConfigurationError:true}
43 |
44 |
50 | LUCENE_31
51 |
52 |
66 |
70 |
71 |
75 |
76 |
77 |
78 |
79 |
82 |
83 |
84 |
87 |
90 |
91 |
98 | ${solr.data.dir:}
99 |
100 |
101 |
107 |
109 |
110 |
111 |
119 |
120 |
121 | false
122 |
123 | 10
124 |
127 | 32
128 |
131 |
132 |
133 | 10000
134 | 1000
135 | 10000
136 |
137 |
150 |
153 |
154 |
161 |
164 |
165 |
184 | native
185 |
186 |
189 |
190 |
191 |
192 |
197 |
198 |
199 | false
200 | 32
201 | 10
202 |
203 |
212 | false
213 |
214 |
217 | true
218 |
219 |
233 |
234 |
235 | 1
236 |
237 | 0
238 |
242 |
246 |
247 |
248 |
256 | false
257 |
258 |
259 |
260 |
269 |
270 |
273 |
274 |
275 |
277 |
278 |
279 |
280 |
281 |
296 |
302 |
303 |
311 |
321 |
325 |
334 |
335 |
336 |
358 |
363 |
366 |
372 |
373 |
374 |
375 |
388 | 1024
389 |
390 |
391 |
402 |
403 |
422 |
426 |
427 |
432 |
436 |
437 |
443 |
447 |
448 |
454 |
460 |
461 |
470 |
479 |
480 |
481 |
489 | true
490 |
491 |
504 |
507 |
508 |
517 | 20
518 |
519 |
522 | 200
523 |
524 |
540 |
543 |
544 |
545 |
549 |
550 |
551 |
552 |
553 |
554 | static firstSearcher warming in solrconfig.xml
555 |
556 |
557 |
558 |
559 |
566 | false
567 |
568 |
577 | 2
578 |
579 |
580 |
581 |
582 |
597 |
598 |
616 |
618 |
619 |
626 |
627 |
636 |
641 |
666 |
672 |
673 |
674 |
693 |
702 |
703 |
706 |
707 | explicit
708 | 10
709 |
710 |
714 |
723 |
728 |
744 |
752 |
756 |
762 |
763 |
764 |
773 |
774 |
775 | explicit
776 |
777 |
778 | velocity
779 |
780 | browse
781 | layout
782 | Solritas
783 |
784 | edismax
785 | *:*
786 | 10
787 | *,score
788 |
789 | text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
790 |
791 | text,features,name,sku,id,manu,cat
792 | 3
793 |
794 |
795 | text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
796 |
797 |
798 | on
799 | cat
800 | manu_exact
801 | ipod
802 | GB
803 | 1
804 | cat,inStock
805 | price
806 | 0
807 | 600
808 | 50
809 | after
810 | manufacturedate_dt
811 | NOW/YEAR-10YEARS
812 | NOW
813 | +1YEAR
814 | before
815 | after
816 |
817 |
818 |
819 | on
820 | text features name
821 | 0
822 | name
823 |
824 |
825 | spellcheck
826 |
827 |
830 |
831 |
832 |
843 |
845 |
849 |
854 |
855 |
858 |
860 |
861 |
864 |
867 |
868 |
871 |
874 |
875 |
880 |
883 |
884 |
886 | text
887 | true
888 | ignored_
889 |
890 |
891 | true
892 | links
893 | ignored_
894 |
895 |
896 |
897 |
915 |
918 |
919 |
920 |
950 |
953 |
954 |
959 |
961 |
962 |
970 |
973 |
982 |
983 |
984 |
985 |
986 | search
987 | solrpingquery
988 | all
989 |
990 |
991 |
992 |
993 |
994 |
995 | explicit
996 | true
997 |
998 |
999 |
1000 |
1011 |
1024 |
1025 |
1067 |
1068 |
1075 |
1076 |
1077 | textSpell
1078 |
1079 |
1082 |
1083 |
1086 |
1087 | default
1088 | name
1089 | spellchecker
1090 |
1091 |
1092 |
1093 |
1103 |
1104 |
1111 |
1119 |
1120 |
1121 |
1130 |
1131 |
1132 |
1145 |
1146 |
1147 | false
1148 | false
1149 | 1
1150 |
1151 |
1152 | spellcheck
1153 |
1154 |
1155 |
1156 |
1160 |
1161 |
1162 |
1169 |
1170 |
1171 | true
1172 |
1173 |
1174 | tvComponent
1175 |
1176 |
1177 |
1178 |
1189 |
1192 |
1193 |
1194 |
1195 | default
1196 |
1206 | org.carrot2.clustering.lingo.LingoClusteringAlgorithm
1207 |
1216 | 20
1217 |
1218 |
1223 | ENGLISH
1224 |
1225 |
1226 | stc
1227 | org.carrot2.clustering.stc.STCClusteringAlgorithm
1228 |
1229 |
1230 |
1231 |
1238 |
1242 |
1243 | true
1244 | default
1245 | true
1246 |
1247 | name
1248 | id
1249 |
1250 | features
1251 |
1252 | true
1253 |
1254 |
1255 |
1256 | false
1257 |
1258 | edismax
1259 |
1260 | text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
1261 |
1262 | *:*
1263 | 10
1264 | *,score
1265 |
1266 |
1267 | clustering
1268 |
1269 |
1270 |
1271 |
1278 |
1279 |
1280 |
1281 |
1282 |
1283 | true
1284 |
1285 |
1286 | terms
1287 |
1288 |
1289 |
1290 |
1291 |
1299 |
1300 |
1301 | string
1302 | elevate.xml
1303 |
1304 |
1305 |
1306 |
1307 |
1308 | explicit
1309 |
1310 |
1311 | elevator
1312 |
1313 |
1314 |
1315 |
1319 |
1320 |
1321 |
1322 |
1323 |
1326 |
1327 | 100
1328 |
1329 |
1330 |
1331 |
1334 |
1336 |
1337 |
1338 | 70
1339 |
1340 | 0.5
1341 |
1342 | [-\w ,/\n\"']{20,200}
1343 |
1344 |
1345 |
1346 |
1347 |
1350 |
1351 | ]]>
1352 | ]]>
1353 |
1354 |
1355 |
1356 |
1357 |
1360 |
1361 |
1362 |
1365 |
1366 |
1367 |
1369 |
1370 |
1371 |
1374 |
1379 |
1380 |
1381 |
1382 |
1384 |
1385 | ,,
1387 | ,,
1388 | ,,
1389 | ,,
1390 | ,]]>
1391 | ]]>
1392 |
1393 |
1394 |
1395 |
1396 |
1397 |
1406 |
1415 |
1428 |
1429 |
1440 |
1443 |
1455 |
1458 |
1461 |
1462 |
1466 |
1467 | 5
1468 |
1469 |
1470 |
1478 |
1479 |
1482 |
1483 |
1490 |
1491 |
1495 |
1496 |
1497 |
1498 | *:*
1499 |
1500 |
1503 |
1506 |
1507 |
1508 |
1509 |
--------------------------------------------------------------------------------
/triple-index/conf/spellings.txt:
--------------------------------------------------------------------------------
1 | pizza
2 | history
--------------------------------------------------------------------------------
/triple-index/conf/stopwords.txt:
--------------------------------------------------------------------------------
1 | # Licensed to the Apache Software Foundation (ASF) under one or more
2 | # contributor license agreements. See the NOTICE file distributed with
3 | # this work for additional information regarding copyright ownership.
4 | # The ASF licenses this file to You under the Apache License, Version 2.0
5 | # (the "License"); you may not use this file except in compliance with
6 | # the License. You may obtain a copy of the License at
7 | #
8 | # http://www.apache.org/licenses/LICENSE-2.0
9 | #
10 | # Unless required by applicable law or agreed to in writing, software
11 | # distributed under the License is distributed on an "AS IS" BASIS,
12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | # See the License for the specific language governing permissions and
14 | # limitations under the License.
15 |
16 | #-----------------------------------------------------------------------
17 | # a couple of test stopwords to test that the words are really being
18 | # configured from this file:
19 | stopworda
20 | stopwordb
21 |
22 | #Standard english stop words taken from Lucene's StopAnalyzer
23 | a
24 | an
25 | and
26 | are
27 | as
28 | at
29 | be
30 | but
31 | by
32 | for
33 | if
34 | in
35 | into
36 | is
37 | it
38 | no
39 | not
40 | of
41 | on
42 | or
43 | s
44 | such
45 | t
46 | that
47 | the
48 | their
49 | then
50 | there
51 | these
52 | they
53 | this
54 | to
55 | was
56 | will
57 | with
58 |
59 |
--------------------------------------------------------------------------------
/triple-index/conf/synonyms.txt:
--------------------------------------------------------------------------------
1 | # The ASF licenses this file to You under the Apache License, Version 2.0
2 | # (the "License"); you may not use this file except in compliance with
3 | # the License. You may obtain a copy of the License at
4 | #
5 | # http://www.apache.org/licenses/LICENSE-2.0
6 | #
7 | # Unless required by applicable law or agreed to in writing, software
8 | # distributed under the License is distributed on an "AS IS" BASIS,
9 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
10 | # See the License for the specific language governing permissions and
11 | # limitations under the License.
12 |
13 | #-----------------------------------------------------------------------
14 | #some test synonym mappings unlikely to appear in real input text
15 | aaafoo => aaabar
16 | bbbfoo => bbbfoo bbbbar
17 | cccfoo => cccbar cccbaz
18 | fooaaa,baraaa,bazaaa
19 |
20 | # Some synonym groups specific to this example
21 | GB,gib,gigabyte,gigabytes
22 | MB,mib,megabyte,megabytes
23 | Television, Televisions, TV, TVs
24 | #notice we use "gib" instead of "GiB" so any WordDelimiterFilter coming
25 | #after us won't split it into two words.
26 |
27 | # Synonym mappings can be used for spelling correction too
28 | pixima => pixma
29 |
30 |
--------------------------------------------------------------------------------
/triple-index/conf/velocity/VM_global_library.vm:
--------------------------------------------------------------------------------
1 |
2 | #macro(param $key)$request.params.get($key)#end
3 |
4 | #macro(url_for_solr)/solr#if($request.core.name != "")/$request.core.name#end#end
5 | #macro(url_for_home)#url_for_solr/browse#end
6 |
7 | #macro(q)&q=$!{esc.url($params.get('q'))}#end
8 |
9 | #macro(fqs $p)#foreach($fq in $p)#if($velocityCount>1){end}fq=$esc.url($fq)#end#end
10 |
11 | #macro(debug)#if($request.params.get('debugQuery'))&debugQuery=true#end#end
12 |
13 | #macro(boostPrice)#if($request.params.get('bf') == 'price')&bf=price#end#end
14 |
15 | #macro(annotate)#if($request.params.get('annotateBrowse'))&annotateBrowse=true#end#end
16 |
17 | #macro(annTitle $msg)#if($annotate == true)title="$msg"#end#end
18 |
19 | #macro(spatial)#if($request.params.get('sfield'))&sfield=store#end#if($request.params.get('pt'))&pt=$request.params.get('pt')#end#if($request.params.get('d'))&d=$request.params.get('d')#end#end
20 |
21 | #macro(qOpts)#set($queryOpts = $request.params.get("queryOpts"))#if($queryOpts && $queryOpts != "")&queryOpts=$queryOpts#end#end
22 |
23 | #macro(lensNoQ)?#if($request.params.getParams('fq') and $list.size($request.params.getParams('fq')) > 0)fqs($request.params.getParams('fq'))#end#debug#boostPrice#annotate#spatial#qOpts#end
24 | #macro(lens)#lensNoQ#q#end
25 |
26 |
27 | #macro(url_for_lens)#{url_for_home}#lens#end
28 |
29 | #macro(url_for_start $start)#url_for_home#lens&start=$start#end
30 |
31 | #macro(url_for_filters $p)#url_for_home?#q#boostPrice#spatial#qOpts#if($list.size($p) > 0)fqs($p)#end#debug#end
32 |
33 | #macro(url_for_nested_facet_query $field)#url_for_home#lens&fq=$esc.url($field)#end
34 |
35 | ## TODO: convert to use {!raw f=$field}$value (with escaping of course)
36 | #macro(url_for_facet_filter $field $value)#url_for_home#lens&fq=$esc.url($field):%22$esc.url($value)%22#end
37 |
38 | #macro(url_for_facet_date_filter $field $value)#url_for_home#lens&fq=$esc.url($field):$esc.url($value)#end
39 |
40 | #macro(url_for_facet_range_filter $field $value)#url_for_home#lens&fq=$esc.url($field):$esc.url($value)#end
41 |
42 |
43 | #macro(link_to_previous_page $text)
44 | #if($page.current_page_number > 1)
45 | #set($prev_start = $page.start - $page.results_per_page)
46 | $text
47 | #end
48 | #end
49 |
50 | #macro(link_to_next_page $text)
51 | #if($page.current_page_number < $page.page_count)
52 | #set($next_start = $page.start + $page.results_per_page)
53 | $text
54 | #end
55 | #end
56 |
57 | #macro(link_to_page $page_number $text)
58 | #if($page_number == $page.current_page_number)
59 | $text
60 | #else
61 | #if($page_number <= $page.page_count)
62 | #set($page_start = $page_number * $page.results_per_page - $page.results_per_page)
63 | $text
64 | #end
65 | #end
66 | #end
67 |
68 | #macro(display_facet_query $field, $display, $fieldName)
69 | #if($field.size() > 0)
70 | $display
71 |
81 | #end
82 | #end
83 |
84 | #macro(display_facet_range_date $field, $display, $fieldName)
85 | $display
86 | ##Note: even if mincount is 1, you can still get a '0' before & after
87 | ##Note: We assume facet.range.include='lower'
88 |
16 | #if($response.response.get('grouped'))
17 | $response.response.get('grouped').size() group(s) found in ${response.responseHeader.QTime} ms
18 | #else$page.results_found results found in ${response.responseHeader.QTime} ms
19 | Page $page.current_page_number of $page.page_count#end
21 |
Disclaimer: The locations displayed in this demonstration are purely fictional. It is more than likely that no store with the items listed actually exists at that location!
--------------------------------------------------------------------------------
/triple-index/conf/velocity/head.vm:
--------------------------------------------------------------------------------
1 |
2 | ## An example of using an arbitrary request parameter
3 |
19 |
20 | #param('title')
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
--------------------------------------------------------------------------------
/triple-index/conf/velocity/header.vm:
--------------------------------------------------------------------------------
1 |