Mailing List Archive: RE: announce: web spider/search tool built w/Lucene

RE: announce: web spider/search tool built w/Lucene

Apr 1, 2002, 9:31 AM

Post #1 of 3 (395 views)

We are working on a search application that is an extension to lucene. It would be able to crawl more than web sites and would have a plugable architecture.

I haven't downloaded the code yet, but I like what you have done with spindle. What is the chance of using it as a starting point for the lucene extension?

Mark Tucker

-----Original Message-----
From: James Cooper [mailto:pixel@bitmechanic.com]
Sent: Saturday, March 30, 2002 1:52 PM
To: lucene-user@jakarta.apache.org
Subject: announce: web spider/search tool built w/Lucene

hi,

I've recently released a free web spider/search tool that uses
Lucene. It's available from:

http://www.bitmechanic.com/projects/spindle/

It's the first project I've built with Lucene, so there's probably room
for improvment as far as how indexes are built and whatnot, but hopefully
others may find it useful.

cheers

-- James

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>

RE: announce: web spider/search tool built w/Lucene [ In reply to ]

pixel at bitmechanic

Apr 1, 2002, 11:31 AM

Post #2 of 3 (388 views)

Permalink

On Mon, 1 Apr 2002, Mark Tucker wrote:

> We are working on a search application that is an extension to lucene.
> It would be able to crawl more than web sites and would have a
> plugable architecture.
>
> I haven't downloaded the code yet, but I like what you have done with
> spindle. What is the chance of using it as a starting point for the
> lucene extension?

That sounds good to me. We may want to swap out the HTML Parser -- the
code I used was GPL, so I had to release spindle under the GPL.

It's a small amount of code though, so we could rewrite it without too
much trouble (or use a different library), since I assume we need to be
able to release under the Apache License.

Let me know if there's anything you'd like me to do.

cheers

-- James

--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>

Re: announce: web spider/search tool built w/Lucene [ In reply to ]

acoliver at apache

Apr 1, 2002, 12:33 PM

Post #3 of 3 (382 views)

Permalink

What I'd like to do if we ever finish the "where to do it" discussion,
is put all 3 submissions under /scratchpad/(X,Y,Z)

Then have a

/scratchapad/luceneapp

and refactor the bits and pieces into there. That way we can get he
best of all three approaches, continue to build all three throughough
the process so that users can get their hands on things early (with the
hope they'll help contribute) and then when we've stabilized the
sources, move the luceneapp into the lucene core as a seperate build
target (for proper encapsulation).

-Andy

James Cooper wrote:

>On Mon, 1 Apr 2002, Mark Tucker wrote:
>
>>We are working on a search application that is an extension to lucene.
>>It would be able to crawl more than web sites and would have a
>>plugable architecture.
>>
>>I haven't downloaded the code yet, but I like what you have done with
>>spindle. What is the chance of using it as a starting point for the
>>lucene extension?
>>
>
>That sounds good to me. We may want to swap out the HTML Parser -- the
>code I used was GPL, so I had to release spindle under the GPL.
>
>It's a small amount of code though, so we could rewrite it without too
>much trouble (or use a different library), since I assume we need to be
>able to release under the Apache License.
>
>Let me know if there's anything you'd like me to do.
>
>cheers
>
>-- James
>
>
>--
>To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
>For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
>
>

--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>