SOLR-7610: fix example/files README and other minor improvements/localization

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1683187 13f79535-47bb-0310-9956-ffa450edef68

solr/CHANGES.txt

@@ -277,7 +277,7 @@ New Features
   which blends RPT indexes for speed with serialized geometry for accuracy.  Includes a Lucene segment based
   in-memory shape cache. (David Smiley)
 
-* SOLR-7465: New file indexing example, under example/files.  (Esther Quansah, Erik Hatcher)
+* SOLR-7465, SOLR-7610: New file indexing example, under example/files.  (Esther Quansah, Erik Hatcher)
 
 * SOLR-7468: Kerberos authenticaion plugin for Solr. This would allow running a Kerberized Solr.
   (Noble Paul, Ishan Chattopadhyaya, Gregory Chanan, Anshum Gupta)

solr/example/files/README.txt

@@ -40,7 +40,7 @@ For further explanations, see the frequently asked questions at the end of the g
 
 		bin/solr create -c files -d example/files/conf
 
-* Now you’ve created a core called “files” using a configuration tuned for indexing and query rich text files.
+* Now you’ve created a core called “files” using a configuration tuned for indexing and querying rich text files.
 
 * You should see the following response:
 
@@ -66,6 +66,10 @@ For further explanations, see the frequently asked questions at the end of the g
 		COMMITting Solr index changes to http://localhost:8983/solr/files/update...
 		Time spent: <some amount of time>
 		
+* To see a list of accepted file types, do:
+  	  	bin/post -h
+	
+
 <hr>
 ##BROWSING DOCUMENTS
 
@@ -73,7 +77,7 @@ For further explanations, see the frequently asked questions at the end of the g
 
 * To view your document information in the HTML interface view, adjust the URL in your address bar to [http://localhost:8983/solr/files/browse](http://localhost:8983/solr/files/browse)
 
-* To view your document information in XML or other formats, add &wt (for writer type) to the end of that URL. i.e.:
+* To view your document information in XML or other formats, add &wt (for writer type) to the end of that URL. i.e. To view your results in xml format direct your browser to:
 	[http://localhost:8983/solr/files/browse?&wt=xml](http://localhost:8983/solr/files/browse?&wt=xml)
 
 <hr>
@@ -81,9 +85,9 @@ For further explanations, see the frequently asked questions at the end of the g
 
 * Another way to verify that your core has been created is to view it in the Admin User Interface.
 
-You can use the Admin_UI as a visual tool for most of the things you’ll be doing with your cores/collections in Solr.
+	- The Admin_UI serves as a visual tool for indexing and querying your index in Solr.
 
-* To access the Admin UI, go to your browser visit :
+* To access the Admin UI, go to your browser and visit :
 	[http://localhost:8983/solr/](http://localhost:8983/solr/)
 
 	- <i>The Admin UI is only accessible when Solr is running</i>
@@ -92,13 +96,14 @@ You can use the Admin_UI as a visual tool for most of the things you’ll be doi
 * Alternatively, you could just go to the core page directly by visiting : [http://localhost:8983/solr/#/files](http://localhost:8983/solr/#/files)
 
 * Now you’ve opened the core page. On this page there are a multitude of different tools you can use to analyze and search your core. You will make use of these features after indexing your documents.
+* Take note of the "Num Docs" field in your core Statistics. If after indexing your documents, it shows Num Docs to be 0, that means there was a problem indexing.
 
 <hr>
 ##QUERYING INDEX
 
 * In the Admin UI, enter a term in the query box to see which documents contain the word. 
 
-* You can filter the results by switching between the different content type tabs.
+* You can filter the results by switching between the different content type tabs. To view an international version of this interface, hover over the globe icon in the top right hand section of the page.
 
 * Notice the tag cloud on the right side, which facets by top phrases extracted during indexing.
   Click on the phrases to see which documents contain them.
@@ -130,27 +135,18 @@ You can use the Admin_UI as a visual tool for most of the things you’ll be doi
 
 * How can I change the /browse UI?
 
-	The primary templates are under example/files/conf/velocity.  In order to edit those files in place (without having to
+	The primary templates are under example/files/conf/velocity.  **In order to edit those files in place (without having to
-	re-create or patch a core/collection with an updated configuration), Solr can be started with a special system property
+	re-create or patch a core/collection with an updated configuration)**, Solr can be started with a special system property
 	set to the _absolute_ path to the conf/velocity directory, like this: 
 	
+		bin/solr start -Dvelocity.template.base.dir=</full/path/to>/example/files/conf/velocity/
 	
-	 bin/solr start -Dvelocity.template.base.dir=</full/path/to>/example/files/conf/velocity/
+        If you want to adjust the browse templates for an existing collection, edit the core’s configuration
+        under server/solr/files/conf/velocity.
 
 
-bin/solr stop
+=======
-rm -Rf server/solr/files/
 
-# templates extracted with:
+* Provenance of free images used in this example:
-#    unzip  -j dist/solr-velocity-*.jar velocity/* -x *.properties -d example/files/templates/
+  - Globe icon: visualpharm.com
-bin/solr start -Dvelocity.template.base.dir=<absolute path to example/files/templates>
+  - Flag icons: freeflagicons.com
-# TODO: make it so an install dir relative path can be used somehow?
-bin/solr create_core -c files
-bin/post -c files ~/Documents
-curl http://localhost:8983/solr/files/config/params -H 'Content-type:application/json'  -d '{
-"update" : {
-  "facets": {
-    "facet.field":"content_type"
-    }
-  }
-}'

solr/example/files/browse-resources/velocity/resources.properties

@@ -0,0 +1,24 @@
+# Title: "<Solr logo> Powered File Search"
+powered_file_search=Powered File Search
+
+# Search box and results
+find=Find
+submit=Submit
+page_of=Page <span class="page-num">{0}</span> of <span class="page-count">{1}</span>
+previous=previous
+next=next
+results_found_in=results found in {0}ms
+results_found=results found
+ 
+# Facets
+top_phrases=Top Phrases
+ 
+# Type labels
+type.all=All Types
+type.doc.label=Document
+type.html.label=HTML
+type.pdf.label=PDF
+type.presentation.label=Presentation
+type.spreadsheet.label=Spreadsheet
+type.text.label=text
+type.unknown=unknown

solr/example/files/browse-resources/velocity/resources_de_DE.properties

@@ -0,0 +1,17 @@
+find=Durchsuchen
+page_of=Page <span class="page-num">{0}</span> von <span class="page-count">{1}</span>
+previous=vorherige Seite
+next=n\u00e4chste Seite
+results_found_in=Ergebnisse in {0}ms gefunden
+results_found=Ergebnisse gefunden
+powered_file_search= betriebene Dateisuche
+type.text.label=Text
+type.pdf.label=PDF
+type.html.label=HTML
+type.presentation.label=Pr\u00e4sentation
+type.doc.label=Dokument
+type.spreadsheet.label=Kalkulationstabelle
+type.unknown=unbekannt
+type.all=alle Arten
+top_phrases=Schl\u00fcssels\u00e4tze
+submit=einreichen

solr/example/files/browse-resources/velocity/resources_fr_FR.properties

@@ -0,0 +1,19 @@
+find=Recherche
+page_of=Page <span class="page-num">{0}</span> de <span class="page-count">{1}</span>
+previous=pr\u00e9c\u00e9dent
+next=suivant
+results_found_in=resultas ficher en {0}ms
+results_found=resultas ficher
+powered_file_search=Recherches de Fichiers
+type.text.label=Texte
+type.pdf.label=PDF
+type.html.label=HTML
+type.presentation.label=Pr\u00e9sentation
+type.doc.label=Documents
+type.spreadsheet.label=Tableur
+type.unknown=inconnu
+type.all=Tous les Types
+top_phrases=Phrases Cl\u00e9s
+submit=Recherche
+
+

solr/example/files/conf/email_url_types.txt

@@ -0,0 +1,2 @@
+<URL>
+<EMAIL>

solr/example/files/conf/managed-schema

@@ -398,6 +398,13 @@
   <fieldType name="tlong" class="solr.TrieLongField" positionIncrementGap="0" precisionStep="8"/>
   <fieldType name="tlongs" class="solr.TrieLongField" positionIncrementGap="0" multiValued="true" precisionStep="8"/>
 
+  <fieldType name="text_email_url" class="solr.TextField">
+    <analyzer>
+      <tokenizer class="solr.UAX29URLEmailTokenizerFactory"/>
+      <filter class="solr.TypeTokenFilterFactory" types="email_url_types.txt" useWhitelist="true"/>
+    </analyzer>
+  </fieldType>
+
   <fieldType name="text_shingles" class="solr.TextField" positionIncrementGap="100" multiValued="true">
     <analyzer type="index">
       <tokenizer class="solr.StandardTokenizerFactory"/>

solr/example/files/conf/params.json

@@ -18,6 +18,7 @@
     "type_fq":"{!field f=doc_type v=$type}",
     "hl":"on",
     "hl.fl":"content",
+    "v.locale":"${locale}",
     "debug":"true",
     "":{"v":0}},
   "velocity":{

solr/example/files/conf/solrconfig.xml

@@ -82,6 +82,7 @@
   <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-langid-\d.*\.jar" />
 
   <lib dir="${solr.install.dir:../../../..}/contrib/velocity/lib" regex=".*\.jar" />
+  <lib path="${solr.install.dir:../../../..}/example/files/browse-resources"/>
   <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-velocity-\d.*\.jar" />
   <!-- an exact 'path' can be used instead of a 'dir' to specify a 
        specific jar file.  This will cause a serious error to be logged 

solr/example/files/conf/update-script.js

@@ -57,17 +57,27 @@ function processAdd(cmd) {
         break;
     }
 
-
     // TODO: error handling needed?   What if there is no slash?
     if(doc_type) { doc.setField("doc_type", doc_type); }
     doc.setField("content_type_type_s", ct_type);
     doc.setField("content_type_subtype_s", ct_subtype);
-
-// doc, image, unknown, ...
-    // application/pdf => doc
-    // application/msword => doc
-    // image/* => image
   }
+
+    var analyzer =
+         req.getCore().getLatestSchema()
+         .getFieldTypeByName("text_email_url")
+         .getIndexAnalyzer();
+
+  var token_stream =
+       analyzer.tokenStream("content", new java.io.StringReader(doc.getFieldValue("content")));
+  var term_att = token_stream.getAttribute(org.apache.lucene.analysis.tokenattributes.CharTermAttribute.class);
+  var type_att = token_stream.getAttribute(org.apache.lucene.analysis.tokenattributes.TypeAttribute.class);
+  token_stream.reset();
+  while (token_stream.incrementToken()) {
+    doc.addField(type_att.type().replace(/\<|\>/g,'').toLowerCase()+"_ss", term_att.toString());
+  }
+  token_stream.end();
+  token_stream.close();
 }
 
 function processDelete(cmd) {

solr/example/files/conf/velocity/browse.vm

@@ -2,12 +2,13 @@
   <form id="query-form" action="#{url_for_home}" method="GET">
     $resource.find:
     <input type="text" id="q" name="q" style="width: 50%" value="$!esc.html($request.params.get('q'))"/>
-    <input type="submit"/>
+    <input type="submit" value="$resource.submit"/>
     <div id="debug_query" class="debug">
       <span id="parsed_query">$esc.html($response.response.debug.parsedquery)</span>
     </div>
 
     <input type="hidden" name="type" value="#current_type"/>
+    #if("#current_locale"!="")<input type="hidden" value="locale" value="#current_locale"/>#end
     #foreach($fq in $response.responseHeader.params.getAll("fq"))
       <input type="hidden" name="fq" id="allFQs" value="$esc.html($fq)"/>
     #end
@@ -33,7 +34,7 @@
 
   <div id="results_list">
     <div class="pagination">
-      <span class="results-found">$page.results_found</span> results found in ${response.responseHeader.QTime}ms
+      <span class="results-found">$page.results_found</span> $resource.results_found_in.insert(${response.responseHeader.QTime})
       $resource.page_of.insert($page.current_page_number,$page.page_count)
     </div>
 
@@ -41,7 +42,7 @@
 
     <div class="pagination">
       #link_to_previous_page
-      <span class="results-found">$page.results_found</span> results found.
+      <span class="results-found">$page.results_found</span> $resource.results_found.
       $resource.page_of.insert($page.current_page_number,$page.page_count)
       #link_to_next_page
     </div>

solr/example/files/conf/velocity/dropit.js

@@ -0,0 +1,97 @@
+/*
+ * Dropit v1.1.0
+ * http://dev7studios.com/dropit
+ *
+ * Copyright 2012, Dev7studios
+ * Free to use and abuse under the MIT license.
+ * http://www.opensource.org/licenses/mit-license.php
+ */
+
+;(function($) {
+
+    $.fn.dropit = function(method) {
+
+        var methods = {
+
+            init : function(options) {
+                this.dropit.settings = $.extend({}, this.dropit.defaults, options);
+                return this.each(function() {
+                    var $el = $(this),
+                         el = this,
+                         settings = $.fn.dropit.settings;
+
+                    // Hide initial submenus
+                    $el.addClass('dropit')
+                    .find('>'+ settings.triggerParentEl +':has('+ settings.submenuEl +')').addClass('dropit-trigger')
+                    .find(settings.submenuEl).addClass('dropit-submenu').hide();
+
+                    // Open on click
+                    $el.off(settings.action).on(settings.action, settings.triggerParentEl +':has('+ settings.submenuEl +') > '+ settings.triggerEl +'', function(){
+                        // Close click menu's if clicked again
+                        if(settings.action == 'click' && $(this).parents(settings.triggerParentEl).hasClass('dropit-open')){
+                            settings.beforeHide.call(this);
+                            $(this).parents(settings.triggerParentEl).removeClass('dropit-open').find(settings.submenuEl).hide();
+                            settings.afterHide.call(this);
+                            return false;
+                        }
+
+                        // Hide open menus
+                        settings.beforeHide.call(this);
+                        $('.dropit-open').removeClass('dropit-open').find('.dropit-submenu').hide();
+                        settings.afterHide.call(this);
+
+                        // Open this menu
+                        settings.beforeShow.call(this);
+                        $(this).parents(settings.triggerParentEl).addClass('dropit-open').find(settings.submenuEl).show();
+                        settings.afterShow.call(this);
+
+                        return false;
+                    });
+
+                    // Close if outside click
+                    $(document).on('click', function(){
+                        settings.beforeHide.call(this);
+                        $('.dropit-open').removeClass('dropit-open').find('.dropit-submenu').hide();
+                        settings.afterHide.call(this);
+                    });
+
+                    // If hover
+                    if(settings.action == 'mouseenter'){
+                        $el.on('mouseleave', '.dropit-open', function(){
+                            settings.beforeHide.call(this);
+                            $(this).removeClass('dropit-open').find(settings.submenuEl).hide();
+                            settings.afterHide.call(this);
+                        });
+                    }
+
+                    settings.afterLoad.call(this);
+                });
+            }
+
+        };
+
+        if (methods[method]) {
+            return methods[method].apply(this, Array.prototype.slice.call(arguments, 1));
+        } else if (typeof method === 'object' || !method) {
+            return methods.init.apply(this, arguments);
+        } else {
+            $.error( 'Method "' +  method + '" does not exist in dropit plugin!');
+        }
+
+    };
+
+    $.fn.dropit.defaults = {
+        action: 'mouseenter', // The open action for the trigger
+        submenuEl: 'ul', // The submenu element
+        triggerEl: 'a', // The trigger element
+        triggerParentEl: 'li', // The trigger parent element
+        afterLoad: function(){}, // Triggers when plugin has loaded
+        beforeShow: function(){}, // Triggers before submenu is shown
+        afterShow: function(){}, // Triggers after submenu is shown
+        beforeHide: function(){}, // Triggers before submenu is hidden
+        afterHide: function(){} // Triggers before submenu is hidden
+    };
+
+    $.fn.dropit.settings = {};
+
+})(jQuery);

solr/example/files/conf/velocity/facet_text_shingles.vm

@@ -1,5 +1,5 @@
 <div id="facet_$field.name">
-  <span class="facet-field">Top Phrases</span><br/>
+  <span class="facet-field">$resource.top_phrases</span><br/>
 
   <ul id="tagcloud">
     #foreach($facet in $sort.sort($field.values,"name"))

solr/example/files/conf/velocity/head.vm

@@ -9,13 +9,19 @@
 
 <script type="text/javascript" src="#{url_root}/js/lib/jquery-1.7.2.min.js"></script>
 <script type="text/javascript" src="#{url_for_solr}/admin/file?file=/velocity/jquery.tx3-tag-cloud.js&contentType=text/javascript"></script>
+   <script type="text/javascript" src="#{url_for_solr}/admin/file?file=/velocity/dropit.js&contentType=text/javascript"></script>
 
 <script type="text/javascript">
   $(document).ready(function(){
     $("#tagcloud").tx3TagCloud({
       multiplier: 5
     });
+
+
+     $('.menu').dropit();
   });
+
+
 </script>
 
 <style>
@@ -33,6 +39,11 @@
     font-size: 20pt;
   }
 
+  #header2{
+
+  margin-left:1200px;
+  }
+
   #logo {
     width: 115px;
     margin: 0px 0px 0px 0px;
@@ -42,6 +53,9 @@
   a {
     color: #305CB3;
   }
+  a.hidden{
+  display:none;
+  }
 
   em {
     color: #FF833D;
@@ -187,4 +201,22 @@
     transition: color 250ms linear;
   }
 
+  .dropit {
+  list-style: none;
+  padding: 0;
+  margin: 0;
+  }
+  .dropit .dropit-trigger { position: relative; }
+  .dropit .dropit-submenu {
+  position: absolute;
+  top: 100%;
+  left: 0; /* dropdown left or right */
+  z-index: 1000;
+  display: none;
+  min-width: 150px;
+  list-style: none;
+  padding: 0;
+  margin: 0;
+  }
+  .dropit .dropit-open .dropit-submenu { display: block; }
 </style>

solr/example/files/conf/velocity/layout.vm

@@ -4,9 +4,25 @@
 </head>
   <body>
     <div id="header">
-      <a href="#url_for_home"><img src="#{url_root}/img/solr.svg" id="logo" title="Solr"/></a> Powered File Search
+      <a href="#url_for_home"><img src="#{url_root}/img/solr.svg" id="logo" title="Solr"/></a> $resource.powered_file_search
     </div>
 
+    <div id="header2" onclick="javascript:locale_select()">
+      <ul class="menu">
+
+        <li>
+          <a href="#"><img src="#{url_for_solr}/admin/file?file=/velocity/img/globe_256.png&contentType=image/png" id="locale_pic" title="locale_select" width="30px" height="27px"/></a>
+          <ul>
+            <li><a href="#url_for_locale('fr_FR')" #if("#current_locale"=="fr_FR")class="hidden"#end>
+              <img src="#{url_for_solr}/admin/file?file=/velocity/img/france_640.png&contentType=image/png" id="french_flag"  width="40px" height="40px"/>Fran&ccedil;ais</a></li>
+            <li><a href="#url_for_locale('de_DE')" #if("#current_locale"=="de_DE")class="hidden"#end>
+              <img src="#{url_for_solr}/admin/file?file=/velocity/img/germany_640.png&contentType=image/png" id="german_flag"  width="40px" height="40px"/>Deutsch</a></li>
+            <li><a href="#url_for_locale('')" #if("#current_locale"=="")class="hidden"#end>
+              <img src="#{url_for_solr}/admin/file?file=/velocity/img/english_640.png&contentType=image/png" id="english_flag"  width="40px" height="40px"/>English</a></li>
+          </ul>
+        </li>
+      </ul>
+    </div>
 
     #if($response.response.error.code)
       <div class="error">

solr/example/files/conf/velocity/macros.vm

@@ -1,7 +1,7 @@
 #macro(lensNoQ)?#if($list.size($response.responseHeader.params.getAll("fq")) > 0)&#fqs($response.responseHeader.params.getAll("fq"))#end#sort($request.params.getParams('sort'))#end
 
 ## lens modified for example/files - to use fq from responseHeader rather than request, and #debug removed too as it is built into browse params now, also added type to lens
-#macro(lens)#lensNoQ#q&type=#current_type#end
+#macro(lens)#lensNoQ#q&type=#current_type#if("#current_locale"!="")&locale=#current_locale#end#end
 
 
 # TODO: make this parameterized fully, no context sensitivity
@@ -44,5 +44,7 @@ $pad$v##  #TODO: $esc.html() or maybe make that optional?
 #end
 
 ## Macros defined custom for the "files" example
-#macro(url_for_type $type)#url_for_home#lensNoQ#q&type=$type#end
+#macro(url_for_type $type)#url_for_home#lensNoQ#q&type=$type#if("#current_locale"!="")&locale=#current_locale#end#end
 #macro(current_type)#if($response.responseHeader.params.type)${response.responseHeader.params.type}#{else}all#end#end
+#macro(url_for_locale $locale)#url_for_home#lensNoQ#q&type=#current_type#if($locale!="")&locale=$locale#end#end
+#macro(current_locale)$!{response.responseHeader.params.locale}#end

solr/example/files/conf/velocity/resources.properties

@@ -1,6 +0,0 @@
-find=Find
-page_of=Page <span class="page-num">{0}</span> of <span class="page-count">{1}</span>
-previous=previous
-next=next
-
-

solr/example/files/conf/velocity/results_list.vm

@@ -1,11 +1,11 @@
 <ul id="tabs">
-  <li><a href="#url_for_type('all')" #if("#current_type"=="all")class="selected"#end>All Types ($response.response.facet_counts.facet_queries.all_types)</a></li>
+  <li><a href="#url_for_type('all')" #if("#current_type"=="all")class="selected"#end>$resource.type.all ($response.response.facet_counts.facet_queries.all_types)</a></li>
   #foreach($type in $response.response.facet_counts.facet_fields.doc_type)
     #if($type.key)
-      <li><a href="#url_for_type($type.key)" #if("#current_type"==$type.key)class="selected"#end>$type.key ($type.value)</a></li>
+      <li><a href="#url_for_type($type.key)" #if("#current_type"==$type.key)class="selected"#end> #if($resource.get("type.${type.key}.label"))$resource.get("type.${type.key}.label")#else$type.key#end ($type.value)</a></li>
     #else
       #if($type.value > 0)
-        <li><a href="#url_for_type('unknown')" #if("#current_type"=="unknown")class="selected"#end>Unknown ($type.value)</a></li>
+        <li><a href="#url_for_type('unknown')" #if("#current_type"=="unknown")class="selected"#end>$resource.type.unknown ($type.value)</a></li>
       #end
     #end
   #end
@@ -17,3 +17,5 @@
     #parse("hit.vm")
   #end
 </div>
+
+