Mailing List Archive

Remove Duplicates
Hello team,
I have indexed a set of files based on some categories but I find
the urls to crawl that I've given has a lot of duplication . How
can I remove them .I want to refine the hit results too.
Secondly can I index also some database values along with the file
contents.
Regards,
Suneetha