Mailing List Archive: Solr webpage SEO

Solr webpage SEO

Mar 4, 2021, 1:04 AM

Post #1 of 6 (438 views)

Hi, sending to this list since dev@solr list is not yet announced properly.

We have a few days of traffic to the new site and can see the most visited pages at https://uls.apache.org/exports/solr.apache.org.yaml (see copy below).
When I search google for "solr query parser", I get the 6.6 guide on top, which is probably why /guide/6_6/the-standard-query-parser.html shows up, and the same for the other /guide/6_6/ links.
Some questions:

How can we make Google forget about version 6.6? I know we had a bunch of redirects from Confluence to the 6.6 guide, are they still in place?
Why is /docs/6_6_0/solr-core/index.html the 2nd most visited page? Anywhere that links to it?
Why is /docs/4_8_1/solr-solrj/index.html so high? Ahywhere that links to it?
The /mirrors-solr-latest-redir.html redirect was not working. I just pushed a fix

Sheet3:
Name: Most visited pages, past month
Values:
/index.html: 443
/docs/6_6_0/solr-core/index.html: 281
/guide/8_8/solr-tutorial.html: 104
/news.html: 92
/guide/solr-tutorial.html: 91
/resources.html: 75
/features.html: 69
/docs/8_7_0/solr-core/index.html: 68
/downloads.html: 68
/guide/6_6/the-standard-query-parser.html: 65
/docs/4_8_1/solr-solrj/index.html: 62
/guide/6_6/common-query-parameters.html: 50
/guide/index.html: 46
/docs/8_8_1/solr-solrj/index.html: 44
/docs/8_7_0/solr-solrj/index.html: 38
/docs/8_8_1/solr-core/index.html: 37
/community.html: 24
/guide/8_8/: 23
/guide/6_6/uploading-data-with-index-handlers.html: 22
/docs/8_6_3/solr-core/index.html: 21
/guide/6_6/filter-descriptions.html: 21
/guide/6_6/collections-api.html: 18
/docs/8_6_2/solr-solrj/overview-summary.html: 16
/guide/6_6/faceting.html: 16
/mirrors-solr-latest-redir.html: 15
/whoweare.html: 15
/guide/6_6/solrcloud.html: 14
/guide/6_6/tokenizers.html: 13
/guide/7_0/solr-configuration-files.html: 13
/guide/8_8/query-syntax-and-parsing.html: 13
/security.html: 13
/guide/6_6/introduction-to-solr-indexing.html: 12
/guide/8_8/solr-upgrade-notes.html: 12
/docs/8_0_0/solr-solrj/allclasses-frame.html: 11
/docs/8_6_3/solr-solrj/index.html: 11
/guide/6_6/the-dismax-query-parser.html: 11
/guide/8_8/getting-started.html: 11
/guide/solr-upgrade-notes.html: 11
/docs/8_0_0/solr-solrj/overview-summary.html: 10
/docs/8_6_2/solr-solrj/index.html: 10
/guide/6_6/running-solr.html: 10
/docs/7_2_1/solr-solrj/overview-summary.html: 9
/docs/8_0_0/solr-solrj/overview-frame.html: 9
/guide/6_6/format-of-solr-xml.html: 9
/guide/6_6/index.html: 9
/guide/6_6/learning-to-rank.html: 9
/guide/6_6/making-and-restoring-backups.html: 9
/guide/6_6/working-with-dates.html: 9
/guide/8_0/reindexing.html: 9
/docs/7_2_1/solr-solrj/allclasses-frame.html: 8

Re: Solr webpage SEO [ In reply to ]

ichattopadhyaya at gmail

Mar 4, 2021, 1:21 AM

Post #2 of 6 (437 views)

Permalink

We can add robots.txt to stop Google from indexing/showing in results.

On Thu, 4 Mar, 2021, 2:34 pm Jan Høydahl, <jan.asf@cominvent.com> wrote:

> Hi, sending to this list since dev@solr list is not yet announced
> properly.
>
> We have a few days of traffic to the new site and can see the most visited
> pages at https://uls.apache.org/exports/solr.apache.org.yaml (see copy
> below).
> When I search google for "solr query parser", I get the 6.6 guide on top,
> which is probably why /guide/6_6/the-standard-query-parser.html shows up,
> and the same for the other /guide/6_6/ links.
> Some questions:
>
>
> - How can we make Google forget about version 6.6? I know we had a
> bunch of redirects from Confluence to the 6.6 guide, are they still in
> place?
> - Why is /docs/6_6_0/solr-core/index.html the 2nd most visited page?
> Anywhere that links to it?
> - Why is /docs/4_8_1/solr-solrj/index.html so high? Ahywhere that
> links to it?
> - The /mirrors-solr-latest-redir.html redirect was not working. I just
> pushed a fix
>
>
>
> Sheet3:
> Name: Most visited pages, past month
> Values:
> /index.html: 443
> /docs/6_6_0/solr-core/index.html: 281
> /guide/8_8/solr-tutorial.html: 104
> /news.html: 92
> /guide/solr-tutorial.html: 91
> /resources.html: 75
> /features.html: 69
> /docs/8_7_0/solr-core/index.html: 68
> /downloads.html: 68
> /guide/6_6/the-standard-query-parser.html: 65
> /docs/4_8_1/solr-solrj/index.html: 62
> /guide/6_6/common-query-parameters.html: 50
> /guide/index.html: 46
> /docs/8_8_1/solr-solrj/index.html: 44
> /docs/8_7_0/solr-solrj/index.html: 38
> /docs/8_8_1/solr-core/index.html: 37
> /community.html: 24
> /guide/8_8/: 23
> /guide/6_6/uploading-data-with-index-handlers.html: 22
> /docs/8_6_3/solr-core/index.html: 21
> /guide/6_6/filter-descriptions.html: 21
> /guide/6_6/collections-api.html: 18
> /docs/8_6_2/solr-solrj/overview-summary.html: 16
> /guide/6_6/faceting.html: 16
> /mirrors-solr-latest-redir.html: 15
> /whoweare.html: 15
> /guide/6_6/solrcloud.html: 14
> /guide/6_6/tokenizers.html: 13
> /guide/7_0/solr-configuration-files.html: 13
> /guide/8_8/query-syntax-and-parsing.html: 13
> /security.html: 13
> /guide/6_6/introduction-to-solr-indexing.html: 12
> /guide/8_8/solr-upgrade-notes.html: 12
> /docs/8_0_0/solr-solrj/allclasses-frame.html: 11
> /docs/8_6_3/solr-solrj/index.html: 11
> /guide/6_6/the-dismax-query-parser.html: 11
> /guide/8_8/getting-started.html: 11
> /guide/solr-upgrade-notes.html: 11
> /docs/8_0_0/solr-solrj/overview-summary.html: 10
> /docs/8_6_2/solr-solrj/index.html: 10
> /guide/6_6/running-solr.html: 10
> /docs/7_2_1/solr-solrj/overview-summary.html: 9
> /docs/8_0_0/solr-solrj/overview-frame.html: 9
> /guide/6_6/format-of-solr-xml.html: 9
> /guide/6_6/index.html: 9
> /guide/6_6/learning-to-rank.html: 9
> /guide/6_6/making-and-restoring-backups.html: 9
> /guide/6_6/working-with-dates.html: 9
> /guide/8_0/reindexing.html: 9
> /docs/7_2_1/solr-solrj/allclasses-frame.html: 8
>
>
>

Re: Solr webpage SEO [ In reply to ]

jan.asf at cominvent

Mar 4, 2021, 2:41 AM

Post #3 of 6 (437 views)

Permalink

Sure we could do robots.

But I suspect that we put ourselves in this situation through https://issues.apache.org/jira/browse/SOLR-10595 ourselves
Check out the attachment solr_redirects.conf on that JIRA (also here https://gist.github.com/janhoy/a3149e1ed27df020194a2de1a7fa2c16)

Here, we explicitly map all pages that used to be in Confluence (which there are a ton of links to on the net), to version 6.6 of the guide.
Of course, if we change those to "latest", some of those links will break, but perhaps it would still be better?
Or can we be more intelligent in the rewrite rules on Solr site - that if you try a "/guide/foo.html" link and it is not found, that you display a custom error page or go to front page of latest guide?

Jan

> 4. mar. 2021 kl. 10:21 skrev Ishan Chattopadhyaya <ichattopadhyaya@gmail.com>:
>
> We can add robots.txt to stop Google from indexing/showing in results.
>
> On Thu, 4 Mar, 2021, 2:34 pm Jan Høydahl, <jan.asf@cominvent.com <mailto:jan.asf@cominvent.com>> wrote:
> Hi, sending to this list since dev@solr list is not yet announced properly.
>
> We have a few days of traffic to the new site and can see the most visited pages at https://uls.apache.org/exports/solr.apache.org.yaml <https://uls.apache.org/exports/solr.apache.org.yaml> (see copy below).
> When I search google for "solr query parser", I get the 6.6 guide on top, which is probably why /guide/6_6/the-standard-query-parser.html shows up, and the same for the other /guide/6_6/ links.
> Some questions:
>
> How can we make Google forget about version 6.6? I know we had a bunch of redirects from Confluence to the 6.6 guide, are they still in place?
> Why is /docs/6_6_0/solr-core/index.html the 2nd most visited page? Anywhere that links to it?
> Why is /docs/4_8_1/solr-solrj/index.html so high? Ahywhere that links to it?
> The /mirrors-solr-latest-redir.html redirect was not working. I just pushed a fix
>
>
> Sheet3:
> Name: Most visited pages, past month
> Values:
> /index.html: 443
> /docs/6_6_0/solr-core/index.html: 281
> /guide/8_8/solr-tutorial.html: 104
> /news.html: 92
> /guide/solr-tutorial.html: 91
> /resources.html: 75
> /features.html: 69
> /docs/8_7_0/solr-core/index.html: 68
> /downloads.html: 68
> /guide/6_6/the-standard-query-parser.html: 65
> /docs/4_8_1/solr-solrj/index.html: 62
> /guide/6_6/common-query-parameters.html: 50
> /guide/index.html: 46
> /docs/8_8_1/solr-solrj/index.html: 44
> /docs/8_7_0/solr-solrj/index.html: 38
> /docs/8_8_1/solr-core/index.html: 37
> /community.html: 24
> /guide/8_8/: 23
> /guide/6_6/uploading-data-with-index-handlers.html: 22
> /docs/8_6_3/solr-core/index.html: 21
> /guide/6_6/filter-descriptions.html: 21
> /guide/6_6/collections-api.html: 18
> /docs/8_6_2/solr-solrj/overview-summary.html: 16
> /guide/6_6/faceting.html: 16
> /mirrors-solr-latest-redir.html: 15
> /whoweare.html: 15
> /guide/6_6/solrcloud.html: 14
> /guide/6_6/tokenizers.html: 13
> /guide/7_0/solr-configuration-files.html: 13
> /guide/8_8/query-syntax-and-parsing.html: 13
> /security.html: 13
> /guide/6_6/introduction-to-solr-indexing.html: 12
> /guide/8_8/solr-upgrade-notes.html: 12
> /docs/8_0_0/solr-solrj/allclasses-frame.html: 11
> /docs/8_6_3/solr-solrj/index.html: 11
> /guide/6_6/the-dismax-query-parser.html: 11
> /guide/8_8/getting-started.html: 11
> /guide/solr-upgrade-notes.html: 11
> /docs/8_0_0/solr-solrj/overview-summary.html: 10
> /docs/8_6_2/solr-solrj/index.html: 10
> /guide/6_6/running-solr.html: 10
> /docs/7_2_1/solr-solrj/overview-summary.html: 9
> /docs/8_0_0/solr-solrj/overview-frame.html: 9
> /guide/6_6/format-of-solr-xml.html: 9
> /guide/6_6/index.html: 9
> /guide/6_6/learning-to-rank.html: 9
> /guide/6_6/making-and-restoring-backups.html: 9
> /guide/6_6/working-with-dates.html: 9
> /guide/8_0/reindexing.html: 9
> /docs/7_2_1/solr-solrj/allclasses-frame.html: 8
>
>

Re: Solr webpage SEO [ In reply to ]

uwe at thetaphi

Mar 4, 2021, 3:31 AM

Post #4 of 6 (437 views)

Permalink

We can change the confluence redirects to use the url without Version number. The htaccess of Solr webpage redirects then automatically to latest version of refguide. This is done by the pelican variable on website deployment.

This link redirects automatically, so if we change the confluence redirects to use no version number it works fine:
https://solr.apache.org/guide/understanding-analyzers-tokenizers-and-filters.html (also with hashes!)

We can't do any dynamic tricks (does it exist?), as redirects can't be dynamic at Apache's Servers. We have no php or similar to implement a dynamic 404 page.

Uwe

Am March 4, 2021 10:41:52 AM UTC schrieb "Jan Høydahl" <jan.asf@cominvent.com>:
>Sure we could do robots.
>
>But I suspect that we put ourselves in this situation through
>https://issues.apache.org/jira/browse/SOLR-10595 ourselves
>Check out the attachment solr_redirects.conf on that JIRA (also here
>https://gist.github.com/janhoy/a3149e1ed27df020194a2de1a7fa2c16)
>
>Here, we explicitly map all pages that used to be in Confluence (which
>there are a ton of links to on the net), to version 6.6 of the guide.
>Of course, if we change those to "latest", some of those links will
>break, but perhaps it would still be better?
>Or can we be more intelligent in the rewrite rules on Solr site - that
>if you try a "/guide/foo.html" link and it is not found, that you
>display a custom error page or go to front page of latest guide?
>
>Jan
>
>> 4. mar. 2021 kl. 10:21 skrev Ishan Chattopadhyaya
><ichattopadhyaya@gmail.com>:
>>
>> We can add robots.txt to stop Google from indexing/showing in
>results.
>>
>> On Thu, 4 Mar, 2021, 2:34 pm Jan Høydahl, <jan.asf@cominvent.com
><mailto:jan.asf@cominvent.com>> wrote:
>> Hi, sending to this list since dev@solr list is not yet announced
>properly.
>>
>> We have a few days of traffic to the new site and can see the most
>visited pages at https://uls.apache.org/exports/solr.apache.org.yaml
><https://uls.apache.org/exports/solr.apache.org.yaml> (see copy below).
>> When I search google for "solr query parser", I get the 6.6 guide on
>top, which is probably why /guide/6_6/the-standard-query-parser.html
>shows up, and the same for the other /guide/6_6/ links.
>> Some questions:
>>
>> How can we make Google forget about version 6.6? I know we had a
>bunch of redirects from Confluence to the 6.6 guide, are they still in
>place?
>> Why is /docs/6_6_0/solr-core/index.html the 2nd most visited page?
>Anywhere that links to it?
>> Why is /docs/4_8_1/solr-solrj/index.html so high? Ahywhere that links
>to it?
>> The /mirrors-solr-latest-redir.html redirect was not working. I just
>pushed a fix
>>
>>
>> Sheet3:
>> Name: Most visited pages, past month
>> Values:
>> /index.html: 443
>> /docs/6_6_0/solr-core/index.html: 281
>> /guide/8_8/solr-tutorial.html: 104
>> /news.html: 92
>> /guide/solr-tutorial.html: 91
>> /resources.html: 75
>> /features.html: 69
>> /docs/8_7_0/solr-core/index.html: 68
>> /downloads.html: 68
>> /guide/6_6/the-standard-query-parser.html: 65
>> /docs/4_8_1/solr-solrj/index.html: 62
>> /guide/6_6/common-query-parameters.html: 50
>> /guide/index.html: 46
>> /docs/8_8_1/solr-solrj/index.html: 44
>> /docs/8_7_0/solr-solrj/index.html: 38
>> /docs/8_8_1/solr-core/index.html: 37
>> /community.html: 24
>> /guide/8_8/: 23
>> /guide/6_6/uploading-data-with-index-handlers.html: 22
>> /docs/8_6_3/solr-core/index.html: 21
>> /guide/6_6/filter-descriptions.html: 21
>> /guide/6_6/collections-api.html: 18
>> /docs/8_6_2/solr-solrj/overview-summary.html: 16
>> /guide/6_6/faceting.html: 16
>> /mirrors-solr-latest-redir.html: 15
>> /whoweare.html: 15
>> /guide/6_6/solrcloud.html: 14
>> /guide/6_6/tokenizers.html: 13
>> /guide/7_0/solr-configuration-files.html: 13
>> /guide/8_8/query-syntax-and-parsing.html: 13
>> /security.html: 13
>> /guide/6_6/introduction-to-solr-indexing.html: 12
>> /guide/8_8/solr-upgrade-notes.html: 12
>> /docs/8_0_0/solr-solrj/allclasses-frame.html: 11
>> /docs/8_6_3/solr-solrj/index.html: 11
>> /guide/6_6/the-dismax-query-parser.html: 11
>> /guide/8_8/getting-started.html: 11
>> /guide/solr-upgrade-notes.html: 11
>> /docs/8_0_0/solr-solrj/overview-summary.html: 10
>> /docs/8_6_2/solr-solrj/index.html: 10
>> /guide/6_6/running-solr.html: 10
>> /docs/7_2_1/solr-solrj/overview-summary.html: 9
>> /docs/8_0_0/solr-solrj/overview-frame.html: 9
>> /guide/6_6/format-of-solr-xml.html: 9
>> /guide/6_6/index.html: 9
>> /guide/6_6/learning-to-rank.html: 9
>> /guide/6_6/making-and-restoring-backups.html: 9
>> /guide/6_6/working-with-dates.html: 9
>> /guide/8_0/reindexing.html: 9
>> /docs/7_2_1/solr-solrj/allclasses-frame.html: 8
>>
>>

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

Re: Solr webpage SEO [ In reply to ]

uwe at thetaphi

Mar 4, 2021, 3:34 AM

Post #5 of 6 (437 views)

Permalink

I disagree with excluding older Java docs or refguide using robots! When I look for documentation of a class I generally enter class name and version number into google.

We can maybe handle this with priorities inside a sitemap.xml or custom http headers (X-Robots) using a htaccess rule. I can check this out.

Uwe

Am March 4, 2021 10:41:52 AM UTC schrieb "Jan Høydahl" <jan.asf@cominvent.com>:
>Sure we could do robots.
>
>But I suspect that we put ourselves in this situation through
>https://issues.apache.org/jira/browse/SOLR-10595 ourselves
>Check out the attachment solr_redirects.conf on that JIRA (also here
>https://gist.github.com/janhoy/a3149e1ed27df020194a2de1a7fa2c16)
>
>Here, we explicitly map all pages that used to be in Confluence (which
>there are a ton of links to on the net), to version 6.6 of the guide.
>Of course, if we change those to "latest", some of those links will
>break, but perhaps it would still be better?
>Or can we be more intelligent in the rewrite rules on Solr site - that
>if you try a "/guide/foo.html" link and it is not found, that you
>display a custom error page or go to front page of latest guide?
>
>Jan
>
>> 4. mar. 2021 kl. 10:21 skrev Ishan Chattopadhyaya
><ichattopadhyaya@gmail.com>:
>>
>> We can add robots.txt to stop Google from indexing/showing in
>results.
>>
>> On Thu, 4 Mar, 2021, 2:34 pm Jan Høydahl, <jan.asf@cominvent.com
><mailto:jan.asf@cominvent.com>> wrote:
>> Hi, sending to this list since dev@solr list is not yet announced
>properly.
>>
>> We have a few days of traffic to the new site and can see the most
>visited pages at https://uls.apache.org/exports/solr.apache.org.yaml
><https://uls.apache.org/exports/solr.apache.org.yaml> (see copy below).
>> When I search google for "solr query parser", I get the 6.6 guide on
>top, which is probably why /guide/6_6/the-standard-query-parser.html
>shows up, and the same for the other /guide/6_6/ links.
>> Some questions:
>>
>> How can we make Google forget about version 6.6? I know we had a
>bunch of redirects from Confluence to the 6.6 guide, are they still in
>place?
>> Why is /docs/6_6_0/solr-core/index.html the 2nd most visited page?
>Anywhere that links to it?
>> Why is /docs/4_8_1/solr-solrj/index.html so high? Ahywhere that links
>to it?
>> The /mirrors-solr-latest-redir.html redirect was not working. I just
>pushed a fix
>>
>>
>> Sheet3:
>> Name: Most visited pages, past month
>> Values:
>> /index.html: 443
>> /docs/6_6_0/solr-core/index.html: 281
>> /guide/8_8/solr-tutorial.html: 104
>> /news.html: 92
>> /guide/solr-tutorial.html: 91
>> /resources.html: 75
>> /features.html: 69
>> /docs/8_7_0/solr-core/index.html: 68
>> /downloads.html: 68
>> /guide/6_6/the-standard-query-parser.html: 65
>> /docs/4_8_1/solr-solrj/index.html: 62
>> /guide/6_6/common-query-parameters.html: 50
>> /guide/index.html: 46
>> /docs/8_8_1/solr-solrj/index.html: 44
>> /docs/8_7_0/solr-solrj/index.html: 38
>> /docs/8_8_1/solr-core/index.html: 37
>> /community.html: 24
>> /guide/8_8/: 23
>> /guide/6_6/uploading-data-with-index-handlers.html: 22
>> /docs/8_6_3/solr-core/index.html: 21
>> /guide/6_6/filter-descriptions.html: 21
>> /guide/6_6/collections-api.html: 18
>> /docs/8_6_2/solr-solrj/overview-summary.html: 16
>> /guide/6_6/faceting.html: 16
>> /mirrors-solr-latest-redir.html: 15
>> /whoweare.html: 15
>> /guide/6_6/solrcloud.html: 14
>> /guide/6_6/tokenizers.html: 13
>> /guide/7_0/solr-configuration-files.html: 13
>> /guide/8_8/query-syntax-and-parsing.html: 13
>> /security.html: 13
>> /guide/6_6/introduction-to-solr-indexing.html: 12
>> /guide/8_8/solr-upgrade-notes.html: 12
>> /docs/8_0_0/solr-solrj/allclasses-frame.html: 11
>> /docs/8_6_3/solr-solrj/index.html: 11
>> /guide/6_6/the-dismax-query-parser.html: 11
>> /guide/8_8/getting-started.html: 11
>> /guide/solr-upgrade-notes.html: 11
>> /docs/8_0_0/solr-solrj/overview-summary.html: 10
>> /docs/8_6_2/solr-solrj/index.html: 10
>> /guide/6_6/running-solr.html: 10
>> /docs/7_2_1/solr-solrj/overview-summary.html: 9
>> /docs/8_0_0/solr-solrj/overview-frame.html: 9
>> /guide/6_6/format-of-solr-xml.html: 9
>> /guide/6_6/index.html: 9
>> /guide/6_6/learning-to-rank.html: 9
>> /guide/6_6/making-and-restoring-backups.html: 9
>> /guide/6_6/working-with-dates.html: 9
>> /guide/8_0/reindexing.html: 9
>> /docs/7_2_1/solr-solrj/allclasses-frame.html: 8
>>
>>

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

Re: Solr webpage SEO [ In reply to ]

casstargett at gmail

Mar 8, 2021, 11:10 AM

Post #6 of 6 (432 views)

Permalink

For the Ref Guide a good amount of the traffic is probably the old redirect rules in Confluence that explicitly went to 6_6 for a good reason *at the time*. A blanket fix to those rules (it was a discrete list of pages in a text file) to remove the URL in the path would cause them to be automatically redirected to the latest. I’d honestly be fine just removing the redirect rules entirely…it’s been almost 4 years, when do we get to move on?

A lot of the 6.6 traffic is also coming from Google itself since that version of a page frequently appears at the top of search results, possibly at least partially due to that Confluence redirect rule, but I’m sure there are also other factors. IIRC people have proposed general ideas about how to fix that, but I don’t recall seeing actual code or PRs yet to actually address the issue.
On Mar 4, 2021, 5:34 AM -0600, Uwe Schindler <uwe@thetaphi.de>, wrote:
> I disagree with excluding older Java docs or refguide using robots! When I look for documentation of a class I generally enter class name and version number into google.
>
> We can maybe handle this with priorities inside a sitemap.xml or custom http headers (X-Robots) using a htaccess rule. I can check this out.
>
> Uwe
>
> Am March 4, 2021 10:41:52 AM UTC schrieb "Jan Høydahl" <jan.asf@cominvent.com>:
> > Sure we could do robots.
> >
> > But I suspect that we put ourselves in this situation through https://issues.apache.org/jira/browse/SOLR-10595 ourselves
> > Check out the attachment solr_redirects.conf on that JIRA (also here https://gist.github.com/janhoy/a3149e1ed27df020194a2de1a7fa2c16)
> >
> > Here, we explicitly map all pages that used to be in Confluence (which there are a ton of links to on the net), to version 6.6 of the guide.
> > Of course, if we change those to "latest", some of those links will break, but perhaps it would still be better?
> > Or can we be more intelligent in the rewrite rules on Solr site - that if you try a "/guide/foo.html" link and it is not found, that you display a custom error page or go to front page of latest guide?
> >
> > Jan
> >
> > > 4. mar. 2021 kl. 10:21 skrev Ishan Chattopadhyaya <ichattopadhyaya@gmail.com>:
> > >
> > > We can add robots.txt to stop Google from indexing/showing in results.
> > >
> > > On Thu, 4 Mar, 2021, 2:34 pm Jan Høydahl, <jan.asf@cominvent.com> wrote:
> > > > Hi, sending to this list since dev@solr list is not yet announced properly.
> > > >
> > > > We have a few days of traffic to the new site and can see the most visited pages at https://uls.apache.org/exports/solr.apache.org.yaml (see copy below).
> > > > When I search google for "solr query parser", I get the 6.6 guide on top, which is probably why /guide/6_6/the-standard-query-parser.html shows up, and the same for the other /guide/6_6/ links.
> > > > Some questions:
> > > >
> > > > • How can we make Google forget about version 6.6? I know we had a bunch of redirects from Confluence to the 6.6 guide, are they still in place?
> > > > • Why is /docs/6_6_0/solr-core/index.html the 2nd most visited page? Anywhere that links to it?
> > > > • Why is /docs/4_8_1/solr-solrj/index.html so high? Ahywhere that links to it?
> > > > • The /mirrors-solr-latest-redir.html redirect was not working. I just pushed a fix
> > > >
> > > >
> > > > Sheet3:
> > > >   Name: Most visited pages, past month
> > > >   Values:
> > > >    /index.html: 443
> > > >    /docs/6_6_0/solr-core/index.html: 281
> > > >    /guide/8_8/solr-tutorial.html: 104
> > > >    /news.html: 92
> > > >    /guide/solr-tutorial.html: 91
> > > >    /resources.html: 75
> > > >    /features.html: 69
> > > >    /docs/8_7_0/solr-core/index.html: 68
> > > >    /downloads.html: 68
> > > >    /guide/6_6/the-standard-query-parser.html: 65
> > > >    /docs/4_8_1/solr-solrj/index.html: 62
> > > >    /guide/6_6/common-query-parameters.html: 50
> > > >    /guide/index.html: 46
> > > >    /docs/8_8_1/solr-solrj/index.html: 44
> > > >    /docs/8_7_0/solr-solrj/index.html: 38
> > > >    /docs/8_8_1/solr-core/index.html: 37
> > > >    /community.html: 24
> > > >    /guide/8_8/: 23
> > > >    /guide/6_6/uploading-data-with-index-handlers.html: 22
> > > >    /docs/8_6_3/solr-core/index.html: 21
> > > >    /guide/6_6/filter-descriptions.html: 21
> > > >    /guide/6_6/collections-api.html: 18
> > > >    /docs/8_6_2/solr-solrj/overview-summary.html: 16
> > > >    /guide/6_6/faceting.html: 16
> > > >    /mirrors-solr-latest-redir.html: 15
> > > >    /whoweare.html: 15
> > > >    /guide/6_6/solrcloud.html: 14
> > > >    /guide/6_6/tokenizers.html: 13
> > > >    /guide/7_0/solr-configuration-files.html: 13
> > > >    /guide/8_8/query-syntax-and-parsing.html: 13
> > > >    /security.html: 13
> > > >    /guide/6_6/introduction-to-solr-indexing.html: 12
> > > >    /guide/8_8/solr-upgrade-notes.html: 12
> > > >    /docs/8_0_0/solr-solrj/allclasses-frame.html: 11
> > > >    /docs/8_6_3/solr-solrj/index.html: 11
> > > >    /guide/6_6/the-dismax-query-parser.html: 11
> > > >    /guide/8_8/getting-started.html: 11
> > > >    /guide/solr-upgrade-notes.html: 11
> > > >    /docs/8_0_0/solr-solrj/overview-summary.html: 10
> > > >    /docs/8_6_2/solr-solrj/index.html: 10
> > > >    /guide/6_6/running-solr.html: 10
> > > >    /docs/7_2_1/solr-solrj/overview-summary.html: 9
> > > >    /docs/8_0_0/solr-solrj/overview-frame.html: 9
> > > >    /guide/6_6/format-of-solr-xml.html: 9
> > > >    /guide/6_6/index.html: 9
> > > >    /guide/6_6/learning-to-rank.html: 9
> > > >    /guide/6_6/making-and-restoring-backups.html: 9
> > > >    /guide/6_6/working-with-dates.html: 9
> > > >    /guide/8_0/reindexing.html: 9
> > > >    /docs/7_2_1/solr-solrj/allclasses-frame.html: 8
> > > >
> > > >
> >
>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de