Apache
/
lucene
mirror of https://github.com/apache/lucene.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Added new web application demo for contrib's XmlQueryParser. 

This change involves:
* Adding Tomcat's Servlet jar into the lib directory and appropriate entry in NOTICE.txt following the lead from Solr's packaging
* Adding new "demo" directory to XmlQueryParser src directory
* Changing XMLQueryParser's build file to create a demo War file
* Changing the main build to include the demo War file (and any other future contrib/*/war files) in the binary distributions

The packaged source distribution has NOT been changed currently to add a lib directory with the servlet.jar so building from a cut-down src distro as opposed to the full subversion /trunk directory will not currently build the war file (the xml query parser build file detects the absence of servlet.jar). Not sure if this is a problem currently.

TODO: 
Now that the servlet jar is available in Subversion I would recommend that the other existing WAR file,"luceneweb.war", is changed to move much of the java code which is currently embedded in JSP files into servlet .java files. This would ensure that the build system will check that the code in this application compiles cleanly with the latest Lucene APIs - otherwise any issue will only become apparent when a user tries to run a JSP.



git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@712318 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Mark Harwood 2008-11-08 00:20:32 +00:00
parent 9136b7a0e0
commit de7ab5b898
12 changed files with 431 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
CHANGES.txt
View File

@ -47,6 +47,9 @@ New features
overlapping tokens (tokens with 0 position increment) should be
counted in lengthNorm. (Andrzej Bialecki via Mike McCandless)
5. Added web-based demo of functionality in contrib's XML Query Parser
packaged as War file (Mark Harwood)
Optimizations
1. LUCENE-1427: Fixed QueryWrapperFilter to not waste time computing

2
NOTICE.txt
View File

@ -14,3 +14,5 @@ The Arabic stemmer (contrib/analyzer) comes with a default
stopword list that is BSD-licensed created by Jacques Savoy. The file
resides in contrib/analyzers/src/java/org/apache/lucene/analysis/ar/stopwords.txt
See http://members.unine.ch/jacques.savoy/clef/index.html.
Includes lib/servlet-api-2.4.jar from Apache Tomcat

2
build.xml
View File

@ -61,7 +61,7 @@
excludes="contrib/db/*/lib/,contrib/*/ext-libs/,src/site/build/,contrib/benchmark/temp/**,contrib/benchmark/work/**"
/>
<patternset id="binary.build.dist.patterns"
includes="${final.name}.jar,${demo.war.name}.war,${demo.name}.jar,docs/,contrib/*/*.jar"
includes="${final.name}.jar,${demo.war.name}.war,${demo.name}.jar,docs/,contrib/*/*.jar,contrib/*/*.war"
/>
<patternset id="binary.root.dist.patterns"
includes="src/demo/,src/jsp/,docs/,*.txt,contrib/*/README*,**/CHANGES.txt"

37
contrib/xml-query-parser/build.xml
View File

@ -26,7 +26,10 @@
<import file="../contrib-build.xml"/>
<property name="queries.jar" location="${common.dir}/build/contrib/queries/lucene-queries-${version}.jar"/>
<property name="servlet.jar" location="${common.dir}/lib/servlet-api-2.4.jar"/>
<available property="queries.jar.present" type="file" file="${queries.jar}"/>
<available property="servlet.jar.present" type="file" file="${servlet.jar}"/>
<path id="classpath">
<pathelement path="${lucene.jar}"/>
@ -34,12 +37,44 @@
<pathelement path="${project.classpath}"/>
</path>
<path id="web-classpath">
<pathelement path="${lucene.jar}"/>
<pathelement path="${queries.jar}"/>
<pathelement path="${servlet.jar}"/>
<pathelement path="${build.dir}/${final.name}.jar"/>
</path>
<target name="compile-core" depends="build-queries, common.compile-core" />
<target name="build-queries" unless="queries.jar.present">
<echo>XML Parser building dependency ${queries.jar}</echo>
<ant antfile="../queries/build.xml" target="default" inheritall="true" dir="../queries" />
<ant antfile="../queries/build.xml" target="default" inheritall="false" dir="../queries" />
</target>
<!-- override contrib-build.xml target to also build web demo -->
<target name="build-artifacts-and-tests" depends="jar, compile-test,build-web-demo" />
<target name="default" depends="jar-core,build-web-demo"/>
<target name="build-web-demo" description="Compiles demo" if="servlet.jar.present">
<echo>XML Parser compiling web demo</echo>
<compile
srcdir="src/demo/java"
destdir="${build.dir}/classes/webdemo">
<classpath refid="web-classpath"/>
</compile>
<war destfile="${build.dir}/xml-query-demo.war" webxml="src/demo/WebContent/WEB-INF/web.xml">
<fileset dir="src/demo/WebContent"/>
<lib dir="${build.dir}">
<include name="${final.name}.jar"/>
</lib>
<lib file="${queries.jar}"/>
<lib file="${lucene.jar}"/>
<classes dir="${build.dir}/classes/webdemo"/>
</war>
</target>
</project>

3
contrib/xml-query-parser/src/demo/WebContent/META-INF/MANIFEST.MF Normal file
View File

@ -0,0 +1,3 @@
Manifest-Version: 1.0
Class-Path:

5
contrib/xml-query-parser/src/demo/WebContent/WEB-INF/data.tsv Normal file
View File

@ -0,0 +1,5 @@
South 100 Contract Java developer required to work within a small development group. Minimum 3+ years experience developing web applications in Java with exposure to Open Source technologies such as Spring, Hibernate, Eclipse, Struts, Lucene, Tomcat
North 078 Permanent Seeking developer with VB.NET, HTML, CSS, JavaScript, ASP. NET, SQL Query Analyzer, Visual Studio. NET, SQL Profiler
East 100 Permanent Project Manager - currently seeking a Project Manager to be based in London with experience of running multiple projects within budget. Candidate will come with a strong project management background, ideally from a technical background with web related experience and project management methodology such as Prince 2
West 085 Contract Oracle DBA required to provide 3rd line support, maintenance and database restore for company's production systems. Experienced in SQL, PL/SQL Oracle databases (9i & 10GR2), Oracle RAC, RMAN and Data Guard. Ideally with, Linux and Windows experience
North 099 Permanent Search engine developer required with experience in the following technologies: Java, Lucene, Solr, Spring, JSP, MySQL, Tomcat, JavaScript, Ant / Ivy, Subversion
1 South 100 Contract Java developer required to work within a small development group. Minimum 3+ years experience developing web applications in Java with exposure to Open Source technologies such as Spring, Hibernate, Eclipse, Struts, Lucene, Tomcat
2 North 078 Permanent Seeking developer with VB.NET, HTML, CSS, JavaScript, ASP. NET, SQL Query Analyzer, Visual Studio. NET, SQL Profiler
3 East 100 Permanent Project Manager - currently seeking a Project Manager to be based in London with experience of running multiple projects within budget. Candidate will come with a strong project management background, ideally from a technical background with web related experience and project management methodology such as Prince 2
4 West 085 Contract Oracle DBA required to provide 3rd line support, maintenance and database restore for company's production systems. Experienced in SQL, PL/SQL Oracle databases (9i & 10GR2), Oracle RAC, RMAN and Data Guard. Ideally with, Linux and Windows experience
5 North 099 Permanent Search engine developer required with experience in the following technologies: Java, Lucene, Solr, Spring, JSP, MySQL, Tomcat, JavaScript, Ant / Ivy, Subversion

58
contrib/xml-query-parser/src/demo/WebContent/WEB-INF/query.xsl Normal file
View File

@ -0,0 +1,58 @@
<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/Document">
<BooleanQuery>
<!-- Clause if user selects a preference for type of job - apply choice of
permanent/contract filter and cache -->
<xsl:if test="type">
<Clause occurs="must">
<ConstantScoreQuery>
<CachedFilter>
<TermsFilter fieldName="type"><xsl:value-of select="type"/></TermsFilter>
</CachedFilter>
</ConstantScoreQuery>
</Clause>
</xsl:if>
<!-- Use standard Lucene query parser for any job description input -->
<xsl:if test="description">
<Clause occurs="must">
<UserQuery fieldName="description"><xsl:value-of select="description"/></UserQuery>
</Clause>
</xsl:if>
<!-- If any of the location fields are set OR them ALL in a Boolean filter and cache individual filters -->
<xsl:if test="South|North|East|West">
<Clause occurs="must">
<ConstantScoreQuery>
<BooleanFilter>
<xsl:for-each select="South|North|East|West">
<Clause occurs="should">
<CachedFilter>
<TermsFilter fieldName="location"><xsl:value-of select="name()"/></TermsFilter>
</CachedFilter>
</Clause>
</xsl:for-each>
</BooleanFilter>
</ConstantScoreQuery>
</Clause>
</xsl:if>
<!-- Use XSL functions to split and zero pad salary range value -->
<xsl:if test="salaryRange">
<Clause occurs="must">
<ConstantScoreQuery>
<RangeFilter fieldName="salary" >
<xsl:attribute name="lowerTerm">
<xsl:value-of select='format-number( substring-before(salaryRange,"-"), "000" )' />
</xsl:attribute>
<xsl:attribute name="upperTerm">
<xsl:value-of select='format-number( substring-after(salaryRange,"-"), "000" )' />
</xsl:attribute>
</RangeFilter>
</ConstantScoreQuery>
</Clause>
</xsl:if>
</BooleanQuery>
</xsl:template>
</xsl:stylesheet>

33
contrib/xml-query-parser/src/demo/WebContent/WEB-INF/web.xml Normal file
View File

@ -0,0 +1,33 @@
<?xml version="1.0" encoding="UTF-8"?>
<web-app id="WebApp_ID" version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">
<display-name>
LuceneXmlQueryWebDemo</display-name>
<servlet>
<description>
Servlet demonstrating XMLQueryParser</description>
<display-name>
FormBasedXmlQueryDemo</display-name>
<servlet-name>FormBasedXmlQueryDemo</servlet-name>
<servlet-class>
org.apache.lucene.xmlparser.webdemo.FormBasedXmlQueryDemo</servlet-class>
<init-param>
<description>
Name of query file held in /WEB-INF</description>
<param-name>xslFile</param-name>
<param-value>query.xsl</param-value>
</init-param>
<init-param>
<description>
Default field used in standard Lucene QueryParser used in UserQuery tag</description>
<param-name>defaultStandardQueryParserField</param-name>
<param-value>jobDescription</param-value>
</init-param>
</servlet>
<servlet-mapping>
<servlet-name>FormBasedXmlQueryDemo</servlet-name>
<url-pattern>/FormBasedXmlQueryDemo</url-pattern>
</servlet-mapping>
<welcome-file-list>
<welcome-file>index.jsp</welcome-file>
</welcome-file-list>
</web-app>

129
contrib/xml-query-parser/src/demo/WebContent/index.jsp Normal file
View File

@ -0,0 +1,129 @@
<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
import="org.apache.lucene.search.*,org.apache.lucene.document.*"
pageEncoding="ISO-8859-1"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<link rel="stylesheet" type="text/css" href="stylesheet.css">
<title>XML Query Parser demo</title>
</head>
<body>
<h1>Job Search</h1>
<%
// Load form variables
String description=request.getParameter("description");
String type=request.getParameter("type");
String salaryRange=request.getParameter("salaryRange");
%>
<form method="POST" action="FormBasedXmlQueryDemo">
<table >
<tr>
<th class="formHeader">Description</th>
<td>
<input name="description" value="<%=description==null?"":description%>"/>
</td>
</tr>
<tr>
<th class="formHeader">Type</th>
<td>
<select name="type">
<option value="" <%=type==null?"selected":""%>>Any</option>
<option value="Contract" <%="Contract".equals(type)?"selected":""%>>Contract</option>
<option value="Permanent" <%="Permanent".equals(type)?"selected":""%>>Permanent</option>
</select>
</td>
</tr>
<tr>
<th class="formHeader">Salary</th>
<td>
<select name="salaryRange">
<option value="" <%=salaryRange==null?"selected":""%>>Any</option>
<%
String ranges[]={"20","30","40", "50","60","70","80","90","100","110","120","150","200"};
for(int i=1;i<ranges.length;i++)
{
String rangeText=ranges[i-1]+"-"+ranges[i];
%>
<option value="<%=rangeText%>" <%=rangeText.equals(salaryRange)?"selected":""%>><%=ranges[i-1]%> to <%=ranges[i]%>k</option>
<%
}
%>
</select>
</td>
</tr>
<tr>
<th class="formHeader">Locations</th>
<td>
<%
String locs[]={"South","North","East","West"};
boolean allLocsBlank=true;
for(int i=0;i<locs.length;i++)
{
if(request.getParameter(locs[i])!=null)
{
allLocsBlank=false;
}
}
for(int i=0;i<locs.length;i++)
{
%>
<input id='cb<%=locs[i]%>'
name="<%=locs[i]%>"
<%
if((allLocsBlank)||("on".equals(request.getParameter(locs[i])))) {
%>
checked="checked"
<% }
%>
type="checkbox"/>
<label for="cb<%=locs[i]%>"><%=locs[i]%></label>
<%
}
%>
</td>
</tr>
<tr>
<th></th>
<td>
<input type="submit" value="search"/>
</td>
</tr>
</table>
</form>
<%
Document[] results=(Document[])request.getAttribute("results");
if(results!=null)
{
%>
<table width="600">
<tr>
<th class="resultsHeader">Type</th>
<th class="resultsHeader">Location</th>
<th class="resultsHeader">Salary</th>
<th class="resultsHeader">Description</th>
</tr>
<%
for (int i = 0; i < results.length; i++)
{
Document doc = results[i];
%>
<tr class="resultsRow">
<td><%=doc.get("type")%></td>
<td><%=doc.get("location")%></td>
<td class="resultNum"><%=doc.get("salary")%>,000</td>
<td><%=doc.get("description")%></td>
</tr>
<%
}
%>
</table>
<%
}//end if has results
%>
</body>
</html>

23
contrib/xml-query-parser/src/demo/WebContent/stylesheet.css Normal file
View File

@ -0,0 +1,23 @@
BODY {font: 10pt Tahoma; color: #000000; background-color: #FFFFFF}
P {font: 10pt Tahoma}
BIG {font: 14pt Tahoma}
#A { color: #FFFFFF;text-decoration: none underline}
A { text-decoration: none underline}
#A:hover {color: #ff33ff; text-decoration: none}
A:hover {color: #9A00C0; text-decoration: none}
.resultsHeader {font: bold 10pt Tahoma; color: #000000; background-color: #DCE2EE}
.formHeader {font: bold 10pt Tahoma; text-align:right; color: #000000; }
TD {font: 10pt Tahoma; color: #000000; }
TR.resultsRow:hover {font: 10pt Tahoma; color: #000000; background-color: #ECF2FE}
.resultNum {text-align:right}
FORM {display: inline}
H1 {font: bold 16pt Tahoma}
H2 {font: bold 14pt Tahoma}
H3 {font: bold 12pt Tahoma}
SPAN.h1 {font: bold 22pt Tahoma}
SPAN.h2 {font: bold 14pt Tahoma}
SMALL {font: 8pt Tahoma}
SELECT {font: 10pt Tahoma; }

133
contrib/xml-query-parser/src/demo/java/org/apache/lucene/xmlparser/webdemo/FormBasedXmlQueryDemo.java Normal file
View File

@ -0,0 +1,133 @@
package org.apache.lucene.xmlparser.webdemo;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.Enumeration;
import java.util.Properties;
import java.util.StringTokenizer;
import javax.servlet.RequestDispatcher;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.store.RAMDirectory;
import org.apache.lucene.xmlparser.CorePlusExtensionsParser;
import org.apache.lucene.xmlparser.QueryTemplateManager;
public class FormBasedXmlQueryDemo extends HttpServlet {
private QueryTemplateManager queryTemplateManager;
private CorePlusExtensionsParser xmlParser;
private IndexSearcher searcher;
private Analyzer analyzer=new StandardAnalyzer();
public void init(ServletConfig config) throws ServletException {
super.init(config);
try {
openExampleIndex();
//load servlet configuration settings
String xslFile=config.getInitParameter("xslFile");
String defaultStandardQueryParserField = config.getInitParameter("defaultStandardQueryParserField");
//Load and cache choice of XSL query template using QueryTemplateManager
queryTemplateManager=new QueryTemplateManager(
getServletContext().getResourceAsStream("/WEB-INF/"+xslFile));
//initialize an XML Query Parser for use by all threads
xmlParser=new CorePlusExtensionsParser(defaultStandardQueryParserField,analyzer);
} catch (Exception e) {
throw new ServletException("Error loading query template",e);
}
}
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
//Take all completed form fields and add to a Properties object
Properties completedFormFields=new Properties();
Enumeration pNames = request.getParameterNames();
while(pNames.hasMoreElements()){
String propName=(String) pNames.nextElement();
String value=request.getParameter(propName);
if((value!=null)&&(value.trim().length()>0)){
completedFormFields.setProperty(propName, value);
}
}
try{
//Create an XML query by populating template with given user criteria
org.w3c.dom.Document xmlQuery=queryTemplateManager.getQueryAsDOM(completedFormFields);
//Parse the XML to produce a Lucene query
Query query=xmlParser.getQuery(xmlQuery.getDocumentElement());
//Run the query
TopDocs topDocs = searcher.search(query,10);
//and package the results and forward to JSP
if(topDocs!=null) {
ScoreDoc[] sd = topDocs.scoreDocs;
Document[] results=new Document[sd.length];
for (int i = 0; i < results.length; i++) {
results[i]=searcher.doc(sd[i].doc);
request.setAttribute("results", results);
}
}
RequestDispatcher dispatcher = getServletContext().getRequestDispatcher("/index.jsp");
dispatcher.forward(request,response);
}
catch(Exception e){
throw new ServletException("Error processing query",e);
}
}
private void openExampleIndex() throws CorruptIndexException, IOException {
//Create a RAM-based index from our test data file
RAMDirectory rd=new RAMDirectory();
IndexWriter writer=new IndexWriter (rd,analyzer,IndexWriter.MaxFieldLength.LIMITED);
InputStream dataIn=getServletContext().getResourceAsStream("/WEB-INF/data.tsv");
BufferedReader br = new BufferedReader(new InputStreamReader(dataIn));
String line = br.readLine();
while(line!=null)
{
line=line.trim();
if(line.length()>0)
{
//parse row and create a document
StringTokenizer st=new StringTokenizer(line,"\t");
Document doc=new Document();
doc.add(new Field("location",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED_NO_NORMS));
doc.add(new Field("salary",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED_NO_NORMS));
doc.add(new Field("type",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED_NO_NORMS));
doc.add(new Field("description",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED));
writer.addDocument(doc);
}
line=br.readLine();
}
writer.close();
//open searcher
searcher=new IndexSearcher(rd);
}
}

2
lib/servlet-api-2.4.jar Normal file
View File

@ -0,0 +1,2 @@
AnyObjectId[018d6effad3823d0ea59f1b58ab154fc2652f418] was removed in git history.
Apache SVN contains full history.