Mailing List Archive

lucene in applet
Hi folks,

I´m working on a site-grabber to wich I would like to add offline search
funktionality. I experimented with lucene, and it covers all my needs. I´m
planning to realize the search functionality with an applet.

Problem: I cannot access the index created with lucene from my applet. The
only way to access resources on a remote host (wich could be the file-system
as well) is with a stream.

//Something like this
URL source = new URL(getCodeBase(), "path_to_index");
BufferedReader in = new BufferedReader(new
InputStreamReader(source.openStream()));
...

From what I tested lucene I found that the only possibilities to access an
existing index is directly with a 'org.apache.lucene.store.Directory' or a
'java.io.File' or a 'java.lang.String' containing the path to the index.

With this interface it seems not to be possible to acces an index from an
applet.

Has anyone encountered a similar problem, and maybe found a solution for it?

It seems to me that maybe lucene should add this possibility to access an
index?

I would appreciate comments on this.


Thank you,

Christoph Breidert
Software - Development

<<<<<<<<<<<<<<<<<<<<<<<<<<<
sitewaerts GmbH
Hebelstraße 15
D-76133 Karlsruhe

Tel: +49 (721) 920 918 0
Fax: +49 (721) 920 918 29
http://www.sitewaerts.de
>>>>>>>>>>>>>>>>>>>>>>>>>>>



--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
Re: lucene in applet [ In reply to ]
Christopher,

> I?m working on a site-grabber to wich I would like to add offline search
> funktionality. I experimented with lucene, and it covers all my needs. I?m
> planning to realize the search functionality with an applet.
>
> Problem: I cannot access the index created with lucene from my applet. The
> only way to access resources on a remote host (wich could be the file-system
> as well) is with a stream.

This sounds about right. It's a question of the applet sandbox,
not of lucene itself.

I've done a little applet work in the past; my general advice is
that a "site-grabber with offline search functionality" is not
compatible with the constraints an applet must run under.
Specifically applets have no access to the local file system, and
applets may only open network connections back to the server they were
downloaded from.

I strongly suggest you ditch the applet aspect entirely, and
maybe look into sun's Java Web Start, which attempts to give you a
good combination of the features of an applet and an application.

> //Something like this
> URL source = new URL(getCodeBase(), "path_to_index");
> BufferedReader in = new BufferedReader(new
> InputStreamReader(source.openStream()));
> ...
>
> >From what I tested lucene I found that the only possibilities to access an
> existing index is directly with a 'org.apache.lucene.store.Directory' or a
> 'java.io.File' or a 'java.lang.String' containing the path to the index.

What you say in this sentence could be rewritten more correctly
as, "it is only possible to access an existing index stored on a disk
by using one of three classes that access the disk." Do you see the
tautology there?

> It seems to me that maybe lucene should add this possibility to access an
> index?

You do bring up one interesting possibility, which is that
perhaps Lucene should have some option to retrieve an index file from
a URL. I'm not sure how wise this would be - if you're retrieving the
index file from a URL, why not just run the search at that URL? - but
it's something to consider. You should think carefully about what
your goal is, and whether such an approach would help, and do a little
experimentation with the lucene code - after all, you have the source,
so you can create a new class to load an index from some remote
location. If you have good results, maybe post the code here.

Steven J. Owens
puff@darksleep.com


--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
AW: lucene in applet [ In reply to ]
Hi Steven,

thanks for your reply.

Here is my szenario:
1) I want to grab a the website "xy.com" to the local disc at C:\xy
2) While exporting I want to index the content to C:\xy\index
3) I put applet.class at C:\xy\index

If I open that applet from the (file) location C:\xy\index it can access any
file in that
directory with a method like this one. This method reads the content of
sFile.

private String read(String sFileName)
{
StringBuffer sb = new StringBuffer();
try
{
URL source = new URL(getCodeBase(), sFileName);
BufferedReader in = new BufferedReader(new
InputStreamReader(source.openStream()));
while(true)
{
String s = in.readLine();
if(s==null)
break;
sb.append(s);
}
in.close();
}
catch(Exception e)
{
//errorhandling
}
return sb.toString();
}

It seems to me that the security-constraints of the sandbox do not prevent
me from using an applet
to open any file in the directory C:\xy\index. With this framework I can
nicely create an applet to
search an exported site.

It would be nice if lucene had the functionality to open content from an
URL.

Like in this little method above, there should not be any problem reading
the index files from the
URL and perfom a search with the content of these files?

>if you're retrieving the
>index file from a URL, why not just run the search at that URL?

I cannot run a JVM at C:\xy\index. For this I want to use the browser. This
gives me the possibility to
entirely grab a site and burn it on a CD maybe, and the index will still
work.

Yours,
Christoph Breidert

breidert@sitewaerts.de

-----Ursprüngliche Nachricht-----
Von: puffmail@darksleep.com [mailto:puffmail@darksleep.com]
Gesendet: Sonntag, 25. November 2001 20:27
An: Lucene Users List
Betreff: Re: lucene in applet


Christopher,

> I?m working on a site-grabber to wich I would like to add offline search
> funktionality. I experimented with lucene, and it covers all my needs. I?m
> planning to realize the search functionality with an applet.
>
> Problem: I cannot access the index created with lucene from my applet. The
> only way to access resources on a remote host (wich could be the
file-system
> as well) is with a stream.

This sounds about right. It's a question of the applet sandbox,
not of lucene itself.

I've done a little applet work in the past; my general advice is
that a "site-grabber with offline search functionality" is not
compatible with the constraints an applet must run under.
Specifically applets have no access to the local file system, and
applets may only open network connections back to the server they were
downloaded from.

I strongly suggest you ditch the applet aspect entirely, and
maybe look into sun's Java Web Start, which attempts to give you a
good combination of the features of an applet and an application.

> //Something like this
> URL source = new URL(getCodeBase(), "path_to_index");
> BufferedReader in = new BufferedReader(new
> InputStreamReader(source.openStream()));
> ...
>
> >From what I tested lucene I found that the only possibilities to access
an
> existing index is directly with a 'org.apache.lucene.store.Directory' or a
> 'java.io.File' or a 'java.lang.String' containing the path to the index.

What you say in this sentence could be rewritten more correctly
as, "it is only possible to access an existing index stored on a disk
by using one of three classes that access the disk." Do you see the
tautology there?

> It seems to me that maybe lucene should add this possibility to access an
> index?

You do bring up one interesting possibility, which is that
perhaps Lucene should have some option to retrieve an index file from
a URL. I'm not sure how wise this would be - if you're retrieving the
index file from a URL, why not just run the search at that URL? - but
it's something to consider. You should think carefully about what
your goal is, and whether such an approach would help, and do a little
experimentation with the lucene code - after all, you have the source,
so you can create a new class to load an index from some remote
location. If you have good results, maybe post the code here.

Steven J. Owens
puff@darksleep.com


--
To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>

<<<<<<<<<<<<<<<<<<<<<<<<<<<
sitewaerts GmbH
Hebelstraße 15
D-76133 Karlsruhe

Tel: +49 (721) 920 918 0
Fax: +49 (721) 920 918 29
http://www.sitewaerts.de
>>>>>>>>>>>>>>>>>>>>>>>>>>>



--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
Re: AW: lucene in applet [ In reply to ]
Christopher,

> Here is my szenario:
> 1) I want to grab a the website "xy.com" to the local disc at C:\xy
> 2) While exporting I want to index the content to C:\xy\index
> 3) I put applet.class at C:\xy\index
>
> If I open that applet from the (file) location C:\xy\index it can
> access any file in that directory with a method like this one. This
> method reads the content of sFile.
>
> It seems to me that the security-constraints of the sandbox do not
> prevent me from using an applet to open any file in the directory
> C:\xy\index. With this framework I can nicely create an applet to
> search an exported site.
>
> >if you're retrieving the
> >index file from a URL, why not just run the search at that URL?

> I cannot run a JVM at C:\xy\index. For this I want to use the
> browser. This gives me the possibility to entirely grab a site and
> burn it on a CD maybe, and the index will still work.

Have you tested that with an applet and a non-lucene file? It
looks a little weird to me, but, I've never tried to run an applet
locally and access the local disk, so I don't know.

Regardless, this part of the discussion is in the area of applet
stuff, not lucene stuff, so it's inappropriate for this mailing list.

> It would be nice if lucene had the functionality to open content
> from an URL.

Based on 2) above, you're saving the index to the local
directory, so why not just load it from the local directory?

> Like in this little method above, there should not be any problem
> reading the index files from the URL and perfom a search with the
> content of these files?

As I said last message, it shouldn't be difficult to modify your
lucene source to load from a URL. Then you could post your changes
here, and Doug might decide to incorporate them into lucene in
general. I'm still not really sure why you want to do it that way,
but give it a try.

Steven J. Owens
puff@darksleep.com

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
AW: lucene in applet [ In reply to ]
Hi folks,

I found a solution to use an index in an applet. The index files are stored
an an URL.

Since you cannot acces the disc on the client from an applet I used the
RAMDirectory. Here´s my code:

//Variable to store the names of the indexfiles in
Stack stackFileNames = new Stack();

//open the file with the filenames
//you need this, because you cannot search a directory for files an
an URL
URL source = new URL(getCodeBase(), "filenames.txt");
BufferedReader in = new BufferedReader(new
InputStreamReader(source.openStream()));
while(true)
{
String s = in.readLine();
if(s==null)
break;
stackFileNames.push(s);
}
in.close();

//open the index
while(!stackFileNames.empty())
{
String sFileName = (String)stackFileNames.pop();
//streaming the file to the RAMDirectory
org.apache.lucene.store.OutputStream os =
ramDir.createFile(sFileName);
java.io.InputStream is = new URL(getCodeBase(),
"repository/"+sFileName).openStream();

byte[] baBuffer = new byte[1024];
while(true)
{
synchronized(baBuffer)
{
int iBytesRead = is.read(baBuffer);
if (iBytesRead == -1)
break;
os.writeBytes(baBuffer, iBytesRead);
}
}
is.close();
os.close();
}

//IndexReader
IndexReader ir = IndexReader.open(ramDir);


With this I can perform searchen in the Applet. Work´s fine.


Have fun,

Christoph Breidert
www.sitewaerts.de

-----Ursprüngliche Nachricht-----
Von: Christoph Breidert [mailto:breidert@sitewaerts.de]
Gesendet: Montag, 26. November 2001 13:09
An: 'Lucene Users List'
Betreff: AW: lucene in applet


Hi Steven,

thanks for your reply.

Here is my szenario:
1) I want to grab a the website "xy.com" to the local disc at C:\xy
2) While exporting I want to index the content to C:\xy\index
3) I put applet.class at C:\xy\index

If I open that applet from the (file) location C:\xy\index it can access any
file in that
directory with a method like this one. This method reads the content of
sFile.

private String read(String sFileName)
{
StringBuffer sb = new StringBuffer();
try
{
URL source = new URL(getCodeBase(), sFileName);
BufferedReader in = new BufferedReader(new
InputStreamReader(source.openStream()));
while(true)
{
String s = in.readLine();
if(s==null)
break;
sb.append(s);
}
in.close();
}
catch(Exception e)
{
//errorhandling
}
return sb.toString();
}

It seems to me that the security-constraints of the sandbox do not prevent
me from using an applet
to open any file in the directory C:\xy\index. With this framework I can
nicely create an applet to
search an exported site.

It would be nice if lucene had the functionality to open content from an
URL.

Like in this little method above, there should not be any problem reading
the index files from the
URL and perfom a search with the content of these files?

>if you're retrieving the
>index file from a URL, why not just run the search at that URL?

I cannot run a JVM at C:\xy\index. For this I want to use the browser. This
gives me the possibility to
entirely grab a site and burn it on a CD maybe, and the index will still
work.

Yours,
Christoph Breidert

breidert@sitewaerts.de

-----Ursprüngliche Nachricht-----
Von: puffmail@darksleep.com [mailto:puffmail@darksleep.com]
Gesendet: Sonntag, 25. November 2001 20:27
An: Lucene Users List
Betreff: Re: lucene in applet


Christopher,

> I?m working on a site-grabber to wich I would like to add offline search
> funktionality. I experimented with lucene, and it covers all my needs. I?m
> planning to realize the search functionality with an applet.
>
> Problem: I cannot access the index created with lucene from my applet. The
> only way to access resources on a remote host (wich could be the
file-system
> as well) is with a stream.

This sounds about right. It's a question of the applet sandbox,
not of lucene itself.

I've done a little applet work in the past; my general advice is
that a "site-grabber with offline search functionality" is not
compatible with the constraints an applet must run under.
Specifically applets have no access to the local file system, and
applets may only open network connections back to the server they were
downloaded from.

I strongly suggest you ditch the applet aspect entirely, and
maybe look into sun's Java Web Start, which attempts to give you a
good combination of the features of an applet and an application.

> //Something like this
> URL source = new URL(getCodeBase(), "path_to_index");
> BufferedReader in = new BufferedReader(new
> InputStreamReader(source.openStream()));
> ...
>
> >From what I tested lucene I found that the only possibilities to access
an
> existing index is directly with a 'org.apache.lucene.store.Directory' or a
> 'java.io.File' or a 'java.lang.String' containing the path to the index.

What you say in this sentence could be rewritten more correctly
as, "it is only possible to access an existing index stored on a disk
by using one of three classes that access the disk." Do you see the
tautology there?

> It seems to me that maybe lucene should add this possibility to access an
> index?

You do bring up one interesting possibility, which is that
perhaps Lucene should have some option to retrieve an index file from
a URL. I'm not sure how wise this would be - if you're retrieving the
index file from a URL, why not just run the search at that URL? - but
it's something to consider. You should think carefully about what
your goal is, and whether such an approach would help, and do a little
experimentation with the lucene code - after all, you have the source,
so you can create a new class to load an index from some remote
location. If you have good results, maybe post the code here.

Steven J. Owens
puff@darksleep.com


--
To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>

<<<<<<<<<<<<<<<<<<<<<<<<<<<
sitewaerts GmbH
Hebelstraße 15
D-76133 Karlsruhe

Tel: +49 (721) 920 918 0
Fax: +49 (721) 920 918 29
http://www.sitewaerts.de
>>>>>>>>>>>>>>>>>>>>>>>>>>>



--
To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>

<<<<<<<<<<<<<<<<<<<<<<<<<<<
sitewaerts GmbH
Hebelstraße 15
D-76133 Karlsruhe

Tel: +49 (721) 920 918 0
Fax: +49 (721) 920 918 29
http://www.sitewaerts.de
>>>>>>>>>>>>>>>>>>>>>>>>>>>



--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>