Mailing List Archive

This already exists to some degree in Mediawiki by default. The
"noindex,nofollow" tags can be assigned per-namespace, so the
Wikipedia: namespace can easily be removed from search indexers
(ones that follow the standard, at least).

As far as assigning it on a per-page basis (ie: for the mainspace if
need be), I would think that either a magic word (like you said Jimbo:
{{NOINDEX}} or somesuch), or an extension (such as the url
blacklists) would be the way to go.

-Chad

On Tue, Apr 29, 2008 at 11:07 AM, Jimmy Wales <jwales@wikia.com> wrote:
> Charles Matthews wrote:
> > Well, maybe we should discuss the downside first. Not having project
> > pages on Google would certainly impede my work. You know, some of us
> > still develop articles, and so on.
>
> This is a valid point, and I think it needs to be addressed in a couple
> of ways:
>
> 1. First, narrowing the scope of the noindex request.
>
> 2. Second, finding out what it would take to improve internal search to
> make it more usable for people developing articles and so on.
>
> > Ah, but I do. Isn't it a better solution to blank some AfDs, than to
> > say "the mission has to come second"? After all, really negative
> > material should be off the site, not just harder to find.
>
> I would support that for some kinds of pages, blanking should be the
> default upon the close of discussion.
>
> -----
>
> I wonder how hard it would be to have a technical change whereby
> articles could be tagged with a {{noindex}} template which would set the
> meta headers appropriately. This could be liberally applied to project
> pages that may be magnets for bad behavior.
>
> And then user space could be the only thing removed from google by default.
>
> Thoughts?
>
> --Jimbo
>
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

I would be very reluctant to use NOINDEX on any mainspace article
pages. If it is appropriate to have the article content, then it is
appropriate to index it in the most public way.
Other namespaces--even article talk pages--that's another matter. They
should normally be public, but in my view some of them not searchable
by google

On Tue, Apr 29, 2008 at 11:11 AM, Chad <innocentkiller@gmail.com> wrote:
> This already exists to some degree in Mediawiki by default. The
> "noindex,nofollow" tags can be assigned per-namespace, so the
> Wikipedia: namespace can easily be removed from search indexers
> (ones that follow the standard, at least).
>
> As far as assigning it on a per-page basis (ie: for the mainspace if
> need be), I would think that either a magic word (like you said Jimbo:
> {{NOINDEX}} or somesuch), or an extension (such as the url
> blacklists) would be the way to go.
>
> -Chad
>
>
>
> On Tue, Apr 29, 2008 at 11:07 AM, Jimmy Wales <jwales@wikia.com> wrote:
> > Charles Matthews wrote:
> > > Well, maybe we should discuss the downside first. Not having project
> > > pages on Google would certainly impede my work. You know, some of us
> > > still develop articles, and so on.
> >
> > This is a valid point, and I think it needs to be addressed in a couple
> > of ways:
> >
> > 1. First, narrowing the scope of the noindex request.
> >
> > 2. Second, finding out what it would take to improve internal search to
> > make it more usable for people developing articles and so on.
> >
> > > Ah, but I do. Isn't it a better solution to blank some AfDs, than to
> > > say "the mission has to come second"? After all, really negative
> > > material should be off the site, not just harder to find.
> >
> > I would support that for some kinds of pages, blanking should be the
> > default upon the close of discussion.
> >
> > -----
> >
> > I wonder how hard it would be to have a technical change whereby
> > articles could be tagged with a {{noindex}} template which would set the
> > meta headers appropriately. This could be liberally applied to project
> > pages that may be magnets for bad behavior.
> >
> > And then user space could be the only thing removed from google by default.
> >
> > Thoughts?
> >
> > --Jimbo
> >
> >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>



--
David Goodman, Ph.D, M.L.S.
http://en.wikipedia.org/wiki/User_talk:DGG

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

2008/4/29 David Goodman <dgoodmanny@gmail.com>:

> I would be very reluctant to use NOINDEX on any mainspace article
> pages. If it is appropriate to have the article content, then it is
> appropriate to index it in the most public way.


Absolutely. The solution to bad living bios in article space, for
example, is to fix, stub or remove them.


> Other namespaces--even article talk pages--that's another matter. They
> should normally be public, but in my view some of them not searchable
> by google


Yeah. Our internal search needs to be a lot better. Fortunately, this
is happening. Just not yesterday.


- d.

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

So - uh - the question naturally arises "Does anyone with the
knowledge of how to do this feel BOLD?" Maybe just noindex the
various talk spaces and see how it goes?

WilyD

On Tue, Apr 29, 2008 at 11:11 AM, Chad <innocentkiller@gmail.com> wrote:
> This already exists to some degree in Mediawiki by default. The
> "noindex,nofollow" tags can be assigned per-namespace, so the
> Wikipedia: namespace can easily be removed from search indexers
> (ones that follow the standard, at least).
>
> As far as assigning it on a per-page basis (ie: for the mainspace if
> need be), I would think that either a magic word (like you said Jimbo:
> {{NOINDEX}} or somesuch), or an extension (such as the url
> blacklists) would be the way to go.
>
> -Chad
>
>
>
> On Tue, Apr 29, 2008 at 11:07 AM, Jimmy Wales <jwales@wikia.com> wrote:
> > Charles Matthews wrote:
> > > Well, maybe we should discuss the downside first. Not having project
> > > pages on Google would certainly impede my work. You know, some of us
> > > still develop articles, and so on.
> >
> > This is a valid point, and I think it needs to be addressed in a couple
> > of ways:
> >
> > 1. First, narrowing the scope of the noindex request.
> >
> > 2. Second, finding out what it would take to improve internal search to
> > make it more usable for people developing articles and so on.
> >
> > > Ah, but I do. Isn't it a better solution to blank some AfDs, than to
> > > say "the mission has to come second"? After all, really negative
> > > material should be off the site, not just harder to find.
> >
> > I would support that for some kinds of pages, blanking should be the
> > default upon the close of discussion.
> >
> > -----
> >
> > I wonder how hard it would be to have a technical change whereby
> > articles could be tagged with a {{noindex}} template which would set the
> > meta headers appropriately. This could be liberally applied to project
> > pages that may be magnets for bad behavior.
> >
> > And then user space could be the only thing removed from google by default.
> >
> > Thoughts?
> >
> > --Jimbo
> >
> >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

Jimmy Wales ha scritto:
> I would support that for some kinds of pages, blanking should be the
> default upon the close of discussion.
>
Should history be left for documentation purposes?
> -----
>
> I wonder how hard it would be to have a technical change whereby
> articles could be tagged with a {{noindex}} template which would set the
> meta headers appropriately. This could be liberally applied to project
> pages that may be magnets for bad behavior.
>
> And then user space could be the only thing removed from google by default.
>
> Thoughts?
>
>
What about Wikipedia clones? We had experience of AfD pages that get
copied by clones and google indexes them (they have a {{noindex}} on
it.wikipedia). People that were declared non-notable don't like this
fact to be their first googlehit.

Cruccone

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

Marco Chiesa wrote:
> Jimmy Wales ha scritto:
>> I would support that for some kinds of pages, blanking should be the
>> default upon the close of discussion.
>>
> Should history be left for documentation purposes?

In most cases, I think that blanking is sufficient. Full deletion is
also called for in some cases.

--Jimbo

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

The way to do it would be to open a bugzilla request.

-Chad

On Tue, Apr 29, 2008 at 12:11 PM, Wily D <wilydoppelganger@gmail.com> wrote:
> So - uh - the question naturally arises "Does anyone with the
> knowledge of how to do this feel BOLD?" Maybe just noindex the
> various talk spaces and see how it goes?
>
> WilyD
>
>
> On Tue, Apr 29, 2008 at 11:11 AM, Chad <innocentkiller@gmail.com> wrote:
>
>
> > This already exists to some degree in Mediawiki by default. The
> > "noindex,nofollow" tags can be assigned per-namespace, so the
> > Wikipedia: namespace can easily be removed from search indexers
> > (ones that follow the standard, at least).
> >
> > As far as assigning it on a per-page basis (ie: for the mainspace if
> > need be), I would think that either a magic word (like you said Jimbo:
> > {{NOINDEX}} or somesuch), or an extension (such as the url
> > blacklists) would be the way to go.
> >
> > -Chad
> >
> >
> >
> > On Tue, Apr 29, 2008 at 11:07 AM, Jimmy Wales <jwales@wikia.com> wrote:
> > > Charles Matthews wrote:
> > > > Well, maybe we should discuss the downside first. Not having project
> > > > pages on Google would certainly impede my work. You know, some of us
> > > > still develop articles, and so on.
> > >
> > > This is a valid point, and I think it needs to be addressed in a couple
> > > of ways:
> > >
> > > 1. First, narrowing the scope of the noindex request.
> > >
> > > 2. Second, finding out what it would take to improve internal search to
> > > make it more usable for people developing articles and so on.
> > >
> > > > Ah, but I do. Isn't it a better solution to blank some AfDs, than to
> > > > say "the mission has to come second"? After all, really negative
> > > > material should be off the site, not just harder to find.
> > >
> > > I would support that for some kinds of pages, blanking should be the
> > > default upon the close of discussion.
> > >
> > > -----
> > >
> > > I wonder how hard it would be to have a technical change whereby
> > > articles could be tagged with a {{noindex}} template which would set the
> > > meta headers appropriately. This could be liberally applied to project
> > > pages that may be magnets for bad behavior.
> > >
> > > And then user space could be the only thing removed from google by default.
> > >
> > > Thoughts?
> > >
> > > --Jimbo
> > >
> > >
> > > _______________________________________________
> > > foundation-l mailing list
> > > foundation-l@lists.wikimedia.org
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >
> >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

On Tue, Apr 29, 2008 at 1:25 PM, Chad <innocentkiller@gmail.com> wrote:

> The way to do it would be to open a bugzilla request.
>
> -Chad
>

Right now, robots.txt excludes certain areas, including XfD, from external
search. There are two problems with this:

1. Internal search indexes them.
2. Developers are forced to work on them on a case-by-case basis, which
takes a lot of time to work on. For example, bug 13398, regarding
well-intentioned bot reports that tend to seem like nasty condemnations of
sites as "spammers", has been open for over a month, but there are many
other important bugs open, and so this and other robots.txt bugs have not
yet been fixed.

For the former problem, it seems fairly straightforward to have logged-out
users search only mainspace pages, either by default behavior of the search
box, or by banning them entirely from searching these pages. Logged-in
users would retain their current defaults and ability to search user pages,
etc.

For the latter problem, I think the community needs to indicate that these
bugzilla requests are important, and should be handled much faster. I have
concerns about blocking namespaces because of the possible affects on other
languages who'd rather not block these namespaces (because robots.txt works
globally), but frankly I think that a full block of other namespaces would
be a good idea, and should be considered on meta.

--
[[User:Ral315]]
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

Alright, as soon as I can convince Bugzilla to let me have an account ...

WilyD

On Tue, Apr 29, 2008 at 1:25 PM, Chad <innocentkiller@gmail.com> wrote:
> The way to do it would be to open a bugzilla request.
>
> -Chad
>
>
>
> On Tue, Apr 29, 2008 at 12:11 PM, Wily D <wilydoppelganger@gmail.com> wrote:
> > So - uh - the question naturally arises "Does anyone with the
> > knowledge of how to do this feel BOLD?" Maybe just noindex the
> > various talk spaces and see how it goes?
> >
> > WilyD
> >
> >
> > On Tue, Apr 29, 2008 at 11:11 AM, Chad <innocentkiller@gmail.com> wrote:
> >
> >
> > > This already exists to some degree in Mediawiki by default. The
> > > "noindex,nofollow" tags can be assigned per-namespace, so the
> > > Wikipedia: namespace can easily be removed from search indexers
> > > (ones that follow the standard, at least).
> > >
> > > As far as assigning it on a per-page basis (ie: for the mainspace if
> > > need be), I would think that either a magic word (like you said Jimbo:
> > > {{NOINDEX}} or somesuch), or an extension (such as the url
> > > blacklists) would be the way to go.
> > >
> > > -Chad
> > >
> > >
> > >
> > > On Tue, Apr 29, 2008 at 11:07 AM, Jimmy Wales <jwales@wikia.com> wrote:
> > > > Charles Matthews wrote:
> > > > > Well, maybe we should discuss the downside first. Not having project
> > > > > pages on Google would certainly impede my work. You know, some of us
> > > > > still develop articles, and so on.
> > > >
> > > > This is a valid point, and I think it needs to be addressed in a couple
> > > > of ways:
> > > >
> > > > 1. First, narrowing the scope of the noindex request.
> > > >
> > > > 2. Second, finding out what it would take to improve internal search to
> > > > make it more usable for people developing articles and so on.
> > > >
> > > > > Ah, but I do. Isn't it a better solution to blank some AfDs, than to
> > > > > say "the mission has to come second"? After all, really negative
> > > > > material should be off the site, not just harder to find.
> > > >
> > > > I would support that for some kinds of pages, blanking should be the
> > > > default upon the close of discussion.
> > > >
> > > > -----
> > > >
> > > > I wonder how hard it would be to have a technical change whereby
> > > > articles could be tagged with a {{noindex}} template which would set the
> > > > meta headers appropriately. This could be liberally applied to project
> > > > pages that may be magnets for bad behavior.
> > > >
> > > > And then user space could be the only thing removed from google by default.
> > > >
> > > > Thoughts?
> > > >
> > > > --Jimbo
> > > >
> > > >
> > > > _______________________________________________
> > > > foundation-l mailing list
> > > > foundation-l@lists.wikimedia.org
> > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > > >
> > >
> > > _______________________________________________
> > > foundation-l mailing list
> > > foundation-l@lists.wikimedia.org
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >
> >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

2008/4/29 Jimmy Wales <jwales@wikia.com>:
> Charles Matthews wrote:
> > Well, maybe we should discuss the downside first. Not having project
> > pages on Google would certainly impede my work. You know, some of us
> > still develop articles, and so on.
>
> This is a valid point, and I think it needs to be addressed in a couple
> of ways:
>
> 1. First, narrowing the scope of the noindex request.

Create a new name space no index it and move problematical pages there.

> 2. Second, finding out what it would take to improve internal search to
> make it more usable for people developing articles and so on.

Spelling. Page rank based on internal links. Page rank based on
external links with proper weighting for authority domains. You are
not going to be able to match google's searching ability.



--
geni

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

FYI https://bugzilla.wikimedia.org/show_bug.cgi?id=8068

MinuteElectron.

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

Well, if the entire project namespace were set to nofollow/noindex,
then it would certainly cut down on the individual requests for
specific pages.

As far as a NOINDEX magic word for the mainspace, I don't think this
is a good idea necessarily, I was just saying how it could be done.

-Chad

On Tue, Apr 29, 2008 at 1:52 PM, Ryan <wiki.ral315@gmail.com> wrote:
> On Tue, Apr 29, 2008 at 1:25 PM, Chad <innocentkiller@gmail.com> wrote:
>
> > The way to do it would be to open a bugzilla request.
> >
> > -Chad
> >
>
> Right now, robots.txt excludes certain areas, including XfD, from external
> search. There are two problems with this:
>
> 1. Internal search indexes them.
> 2. Developers are forced to work on them on a case-by-case basis, which
> takes a lot of time to work on. For example, bug 13398, regarding
> well-intentioned bot reports that tend to seem like nasty condemnations of
> sites as "spammers", has been open for over a month, but there are many
> other important bugs open, and so this and other robots.txt bugs have not
> yet been fixed.
>
> For the former problem, it seems fairly straightforward to have logged-out
> users search only mainspace pages, either by default behavior of the search
> box, or by banning them entirely from searching these pages. Logged-in
> users would retain their current defaults and ability to search user pages,
> etc.
>
> For the latter problem, I think the community needs to indicate that these
> bugzilla requests are important, and should be handled much faster. I have
> concerns about blocking namespaces because of the possible affects on other
> languages who'd rather not block these namespaces (because robots.txt works
> globally), but frankly I think that a full block of other namespaces would
> be a good idea, and should be considered on meta.
>
> --
> [[User:Ral315]]
>
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

The relevant bug request exists :
https://bugzilla.wikimedia.org/show_bug.cgi?id=13864 but there's some
politicing about implementing it, so it seems allowing each Wiki to
choose a set of namespaces might allow for a quicker consensus. Sounds
also like devs might just change the specific robots.txt if consensus
existed, though I'll be hogtied and thricefuck'd if I've any idea how
to do that.

If people think noindexing is worthwhile, they might indicate this is
worth having as an option by voting for this bug.

WilyD

On Tue, Apr 29, 2008 at 1:25 PM, Chad <innocentkiller@gmail.com> wrote:
> The way to do it would be to open a bugzilla request.
>
> -Chad
>
>
>
> On Tue, Apr 29, 2008 at 12:11 PM, Wily D <wilydoppelganger@gmail.com> wrote:
> > So - uh - the question naturally arises "Does anyone with the
> > knowledge of how to do this feel BOLD?" Maybe just noindex the
> > various talk spaces and see how it goes?
> >
> > WilyD
> >
> >
> > On Tue, Apr 29, 2008 at 11:11 AM, Chad <innocentkiller@gmail.com> wrote:
> >
> >
> > > This already exists to some degree in Mediawiki by default. The
> > > "noindex,nofollow" tags can be assigned per-namespace, so the
> > > Wikipedia: namespace can easily be removed from search indexers
> > > (ones that follow the standard, at least).
> > >
> > > As far as assigning it on a per-page basis (ie: for the mainspace if
> > > need be), I would think that either a magic word (like you said Jimbo:
> > > {{NOINDEX}} or somesuch), or an extension (such as the url
> > > blacklists) would be the way to go.
> > >
> > > -Chad
> > >
> > >
> > >
> > > On Tue, Apr 29, 2008 at 11:07 AM, Jimmy Wales <jwales@wikia.com> wrote:
> > > > Charles Matthews wrote:
> > > > > Well, maybe we should discuss the downside first. Not having project
> > > > > pages on Google would certainly impede my work. You know, some of us
> > > > > still develop articles, and so on.
> > > >
> > > > This is a valid point, and I think it needs to be addressed in a couple
> > > > of ways:
> > > >
> > > > 1. First, narrowing the scope of the noindex request.
> > > >
> > > > 2. Second, finding out what it would take to improve internal search to
> > > > make it more usable for people developing articles and so on.
> > > >
> > > > > Ah, but I do. Isn't it a better solution to blank some AfDs, than to
> > > > > say "the mission has to come second"? After all, really negative
> > > > > material should be off the site, not just harder to find.
> > > >
> > > > I would support that for some kinds of pages, blanking should be the
> > > > default upon the close of discussion.
> > > >
> > > > -----
> > > >
> > > > I wonder how hard it would be to have a technical change whereby
> > > > articles could be tagged with a {{noindex}} template which would set the
> > > > meta headers appropriately. This could be liberally applied to project
> > > > pages that may be magnets for bad behavior.
> > > >
> > > > And then user space could be the only thing removed from google by default.
> > > >
> > > > Thoughts?
> > > >
> > > > --Jimbo
> > > >
> > > >
> > > > _______________________________________________
> > > > foundation-l mailing list
> > > > foundation-l@lists.wikimedia.org
> > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > > >
> > >
> > > _______________________________________________
> > > foundation-l mailing list
> > > foundation-l@lists.wikimedia.org
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >
> >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

Apparently, the ability to disable indexing of various namespaces
already exists at the project level

http://www.mediawiki.org/wiki/Manual:LocalSettings.php
LocalSettings.php can be amended with the line
$wgNamespaceRobotPolicies = array( NS_TALK => 'noindex' ); (here
substitute "TALK" "USER_TALK" "WIKIPEDIA_TALK" or whatever namespace
you think should be noindex'd for NS_TALK) and it won't be indexed
anymore. I gather we need to a) develop a local consensus to do this,
and b) beg the devs to do it for us. For this, I think, a volunteer
roundup is needed. Anyone think we can develop a consensus to noindex
the talk spaces on en.wiki as a start?

WilyD


On Tue, Apr 29, 2008 at 3:40 PM, Wily D <wilydoppelganger@gmail.com> wrote:
> The relevant bug request exists :
> https://bugzilla.wikimedia.org/show_bug.cgi?id=13864 but there's some
> politicing about implementing it, so it seems allowing each Wiki to
> choose a set of namespaces might allow for a quicker consensus. Sounds
> also like devs might just change the specific robots.txt if consensus
> existed, though I'll be hogtied and thricefuck'd if I've any idea how
> to do that.
>
> If people think noindexing is worthwhile, they might indicate this is
> worth having as an option by voting for this bug.
>
> WilyD
>
>
>
> On Tue, Apr 29, 2008 at 1:25 PM, Chad <innocentkiller@gmail.com> wrote:
> > The way to do it would be to open a bugzilla request.
> >
> > -Chad
> >
> >
> >
> > On Tue, Apr 29, 2008 at 12:11 PM, Wily D <wilydoppelganger@gmail.com> wrote:
> > > So - uh - the question naturally arises "Does anyone with the
> > > knowledge of how to do this feel BOLD?" Maybe just noindex the
> > > various talk spaces and see how it goes?
> > >
> > > WilyD
> > >
> > >
> > > On Tue, Apr 29, 2008 at 11:11 AM, Chad <innocentkiller@gmail.com> wrote:
> > >
> > >
> > > > This already exists to some degree in Mediawiki by default. The
> > > > "noindex,nofollow" tags can be assigned per-namespace, so the
> > > > Wikipedia: namespace can easily be removed from search indexers
> > > > (ones that follow the standard, at least).
> > > >
> > > > As far as assigning it on a per-page basis (ie: for the mainspace if
> > > > need be), I would think that either a magic word (like you said Jimbo:
> > > > {{NOINDEX}} or somesuch), or an extension (such as the url
> > > > blacklists) would be the way to go.
> > > >
> > > > -Chad
> > > >
> > > >
> > > >
> > > > On Tue, Apr 29, 2008 at 11:07 AM, Jimmy Wales <jwales@wikia.com> wrote:
> > > > > Charles Matthews wrote:
> > > > > > Well, maybe we should discuss the downside first. Not having project
> > > > > > pages on Google would certainly impede my work. You know, some of us
> > > > > > still develop articles, and so on.
> > > > >
> > > > > This is a valid point, and I think it needs to be addressed in a couple
> > > > > of ways:
> > > > >
> > > > > 1. First, narrowing the scope of the noindex request.
> > > > >
> > > > > 2. Second, finding out what it would take to improve internal search to
> > > > > make it more usable for people developing articles and so on.
> > > > >
> > > > > > Ah, but I do. Isn't it a better solution to blank some AfDs, than to
> > > > > > say "the mission has to come second"? After all, really negative
> > > > > > material should be off the site, not just harder to find.
> > > > >
> > > > > I would support that for some kinds of pages, blanking should be the
> > > > > default upon the close of discussion.
> > > > >
> > > > > -----
> > > > >
> > > > > I wonder how hard it would be to have a technical change whereby
> > > > > articles could be tagged with a {{noindex}} template which would set the
> > > > > meta headers appropriately. This could be liberally applied to project
> > > > > pages that may be magnets for bad behavior.
> > > > >
> > > > > And then user space could be the only thing removed from google by default.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > --Jimbo
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > foundation-l mailing list
> > > > > foundation-l@lists.wikimedia.org
> > > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > > > >
> > > >
> > > > _______________________________________________
> > > > foundation-l mailing list
> > > > foundation-l@lists.wikimedia.org
> > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > > >
> > >
> > > _______________________________________________
> > > foundation-l mailing list
> > > foundation-l@lists.wikimedia.org
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >
> >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
>

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

Please contribute if you're interested:
http://en.wikipedia.org/wiki/Wikipedia:Noindexing_Talk_Spaces

On Tue, Apr 29, 2008 at 5:41 PM, Wily D <wilydoppelganger@gmail.com> wrote:
> Apparently, the ability to disable indexing of various namespaces
> already exists at the project level
>
> http://www.mediawiki.org/wiki/Manual:LocalSettings.php
> LocalSettings.php can be amended with the line
> $wgNamespaceRobotPolicies = array( NS_TALK => 'noindex' ); (here
> substitute "TALK" "USER_TALK" "WIKIPEDIA_TALK" or whatever namespace
> you think should be noindex'd for NS_TALK) and it won't be indexed
> anymore. I gather we need to a) develop a local consensus to do this,
> and b) beg the devs to do it for us. For this, I think, a volunteer
> roundup is needed. Anyone think we can develop a consensus to noindex
> the talk spaces on en.wiki as a start?
>
> WilyD
>
>
>
>
> On Tue, Apr 29, 2008 at 3:40 PM, Wily D <wilydoppelganger@gmail.com> wrote:
> > The relevant bug request exists :
> > https://bugzilla.wikimedia.org/show_bug.cgi?id=13864 but there's some
> > politicing about implementing it, so it seems allowing each Wiki to
> > choose a set of namespaces might allow for a quicker consensus. Sounds
> > also like devs might just change the specific robots.txt if consensus
> > existed, though I'll be hogtied and thricefuck'd if I've any idea how
> > to do that.
> >
> > If people think noindexing is worthwhile, they might indicate this is
> > worth having as an option by voting for this bug.
> >
> > WilyD
> >
> >
> >
> > On Tue, Apr 29, 2008 at 1:25 PM, Chad <innocentkiller@gmail.com> wrote:
> > > The way to do it would be to open a bugzilla request.
> > >
> > > -Chad
> > >
> > >
> > >
> > > On Tue, Apr 29, 2008 at 12:11 PM, Wily D <wilydoppelganger@gmail.com> wrote:
> > > > So - uh - the question naturally arises "Does anyone with the
> > > > knowledge of how to do this feel BOLD?" Maybe just noindex the
> > > > various talk spaces and see how it goes?
> > > >
> > > > WilyD
> > > >
> > > >
> > > > On Tue, Apr 29, 2008 at 11:11 AM, Chad <innocentkiller@gmail.com> wrote:
> > > >
> > > >
> > > > > This already exists to some degree in Mediawiki by default. The
> > > > > "noindex,nofollow" tags can be assigned per-namespace, so the
> > > > > Wikipedia: namespace can easily be removed from search indexers
> > > > > (ones that follow the standard, at least).
> > > > >
> > > > > As far as assigning it on a per-page basis (ie: for the mainspace if
> > > > > need be), I would think that either a magic word (like you said Jimbo:
> > > > > {{NOINDEX}} or somesuch), or an extension (such as the url
> > > > > blacklists) would be the way to go.
> > > > >
> > > > > -Chad
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Apr 29, 2008 at 11:07 AM, Jimmy Wales <jwales@wikia.com> wrote:
> > > > > > Charles Matthews wrote:
> > > > > > > Well, maybe we should discuss the downside first. Not having project
> > > > > > > pages on Google would certainly impede my work. You know, some of us
> > > > > > > still develop articles, and so on.
> > > > > >
> > > > > > This is a valid point, and I think it needs to be addressed in a couple
> > > > > > of ways:
> > > > > >
> > > > > > 1. First, narrowing the scope of the noindex request.
> > > > > >
> > > > > > 2. Second, finding out what it would take to improve internal search to
> > > > > > make it more usable for people developing articles and so on.
> > > > > >
> > > > > > > Ah, but I do. Isn't it a better solution to blank some AfDs, than to
> > > > > > > say "the mission has to come second"? After all, really negative
> > > > > > > material should be off the site, not just harder to find.
> > > > > >
> > > > > > I would support that for some kinds of pages, blanking should be the
> > > > > > default upon the close of discussion.
> > > > > >
> > > > > > -----
> > > > > >
> > > > > > I wonder how hard it would be to have a technical change whereby
> > > > > > articles could be tagged with a {{noindex}} template which would set the
> > > > > > meta headers appropriately. This could be liberally applied to project
> > > > > > pages that may be magnets for bad behavior.
> > > > > >
> > > > > > And then user space could be the only thing removed from google by default.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > --Jimbo
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > foundation-l mailing list
> > > > > > foundation-l@lists.wikimedia.org
> > > > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > > > > >
> > > > >
> > > > > _______________________________________________
> > > > > foundation-l mailing list
> > > > > foundation-l@lists.wikimedia.org
> > > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > > > >
> > > >
> > > > _______________________________________________
> > > > foundation-l mailing list
> > > > foundation-l@lists.wikimedia.org
> > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > > >
> > >
> > > _______________________________________________
> > > foundation-l mailing list
> > > foundation-l@lists.wikimedia.org
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >
> >
>

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

Hello,

Ryan wrote:
> Right now, robots.txt excludes certain areas, including XfD, from external
> search. There are two problems with this:
>
> 1. Internal search indexes them.
> 2. Developers are forced to work on them on a case-by-case basis, which
> takes a lot of time to work on. For example, bug 13398, regarding
> well-intentioned bot reports that tend to seem like nasty condemnations of
> sites as "spammers", has been open for over a month, but there are many
> other important bugs open, and so this and other robots.txt bugs have not
> yet been fixed.
>
> For the former problem, it seems fairly straightforward to have logged-out
> users search only mainspace pages, either by default behavior of the search
> box, or by banning them entirely from searching these pages. Logged-in
> users would retain their current defaults and ability to search user pages,
> etc.

I think that this is a very bad idea. It is necessary for anonymous
users to be able to search the Meta and Talk namespaces / pages.
However, banning then from Google would be OK.

Regards,

Yann

> For the latter problem, I think the community needs to indicate that these
> bugzilla requests are important, and should be handled much faster. I have
> concerns about blocking namespaces because of the possible affects on other
> languages who'd rather not block these namespaces (because robots.txt works
> globally), but frankly I think that a full block of other namespaces would
> be a good idea, and should be considered on meta.
>
> --
> [[User:Ral315]]

--
http://www.non-violence.org/ | Site collaboratif sur la non-violence
http://www.forget-me.net/ | Alternatives sur le Net
http://fr.wikisource.org/ | Bibliothèque libre
http://wikilivres.info | Documents libres

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l