Hi All, welcome to the monthly MediaWiki Insights email!
Like last time
<https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Reports/January_2024>,
we’re starting this email by celebrating volunteer contributors who got
their first patch merged in MW core, WMF deployed extensions or services
over the past month:
A big thanks to GergesShamon and Agent Isai for their contributions!
Welcome :-)
Many thanks also to the reviewers - volunteers and staff - of patches by
new contributors. Your support is invaluable for helping people contribute
to MediaWiki effectively and make it a fun first experience!
The focus of this edition of the monthly MW Insights email lies on our
multi-year initiatives: Migration of MediaWiki from bare metal to Kubernetes
<https://wikitech.wikimedia.org/wiki/MediaWiki_On_Kubernetes> in Wikimedia
Production, Parser Unification
<https://www.mediawiki.org/wiki/Parsoid/Parser_Unification> and RESTBase
deprecation <https://phabricator.wikimedia.org/project/profile/6289/>.
These initiatives involve many people and projects, require orchestrated
efforts and coordination, and will help us in many different ways. Another
thing that these initiatives have in common is that we’re getting closer to
being able to benefit from these efforts :-).
Project Snapshots: Milestones reached on 3 multi-year projects
MediaWiki on Kubernetes migration is 75% complete, completing migration of
all internal traffic, all scheduled jobs and reaches 50% global traffic
milestone!
Often shortened as mw-on-k8s, MediaWiki on Kubernetes is a multi-year
effort to move all of the MediaWiki deployments running on WMF production
infrastructure to the new WikiKube platform.
The migration is underway and very recently a new milestone was hit where
the WikiKube platform now serves 50% of end user requests
<https://phabricator.wikimedia.org/T290536>. At half mark, having more
percentage of traffic on WikiKube, Service Operations is working with
Release Engineering to make changes to monitoring tools to surface any
issues during deployments to continue with further ramps. Also, almost the
entirety of what is called internal traffic
<https://phabricator.wikimedia.org/T333120>, that is traffic that is
generated by applications running in the infrastructure and reaching out to
MediaWiki for various purposes is migrated to WikiKube. A special mention
should go to MediaWiki Jobs fully migrated just a week ago
<https://phabricator.wikimedia.org/T349796>. Feel free to follow the higher
umbrella task <https://phabricator.wikimedia.org/T290536> and/or
MediaWiki_On_Kubernetes
on Wikitech <https://wikitech.wikimedia.org/wiki/MediaWiki_On_Kubernetes>.
This migration will unleash the ability to deploy multiple versions of code
simultaneously. Also, this will help enhance platform capabilities to build
dockerized isolated environments for coding, testing and even production
debugging.
MediaWiki on Kubernetes will allow us to deprecate and eventually remove a
lot of our in-house developed code. Another benefit is that we will be able
to react to sudden traffic spikes, like newsworthy events better, as the
flexing up and down is a matter of configuration change. This enables
efficient placement of workloads, packing workloads in a more
environmentally friendly way, increasing hardware utilization.
It takes a village to get this far: A big thanks to the Service Operations
team (Clément Goubert, Giuseppe Lavagetto, Alexandros Kosiaris, Kamila
Sou?ková, Hugh Nowlan, Effie Mouzeli, Reuven Lazarus, Jannis Mayboom and
Kavitha Appakayala) for their leadership on this project, the Release
Engineering team (specifically: Dan Duvall, Jeena Huneidi, Tyler Cipriani
and Ahmon Dancy) for their work on Blubber
<https://wikitech.wikimedia.org/wiki/Blubber>, the Deployment Pipeline
<https://wikitech.wikimedia.org/wiki/Deployment_pipeline> and Scap
<https://wikitech.wikimedia.org/wiki/Scap>, Dom Walden from Quality and
Test Engineering for crafting and executing a plan to test the first
deployment of MediaWiki on Kubernetes, and everyone else who has
contributed in the one way or the other! <3
More milestones:
We’re seeing the first lights at the end of the tunnel of another
multi-year initiative: The *parser unification*
<https://www.mediawiki.org/wiki/Parsoid/Parser_Unification>. One week
ago, Parsoid
Read Views was rolled out to first wikis: Parsoid is now the default read
views renderer on the Foundations’ Office Wiki and Wikitech
DiscussionTools. This early experimentation allows us to find issues in a
limited space, which will help us evaluate readiness of the feature
and increase
our confidence for future rollouts
<http://mediawiki.org/wiki/Parsoid/Parser_Unification/Confidence_Framework>.
A huge thanks to Subbu Sastry, Mateus Santos, CScott, Isabelle
Hubert-Pallatin, Arlo Breault, Shannon Bailey, Yiannis Giannelos and Sérgio
Lopes for making this work! Many thanks also to Daniel Kinzler: His work on
the RESTBase deprecation directly helped us get to this milestone :-)
*RESTBase deprecation*: We have been continuously working on decoupling
services from RESTBase, aiming for the modernisation and sustainability of
Wikimedia products in our services platform. The MediaWiki Interfaces team
finished up reimplementation of Reading Lists endpoints in MW REST API
<https://phabricator.wikimedia.org/T348491> and are now confirming with
affected callers <https://phabricator.wikimedia.org/T357478> that the new
endpoints meet their needs before rerouting calls
<https://phabricator.wikimedia.org/T348493> and retiring old code
<https://phabricator.wikimedia.org/T348494>. The overall effort
<https://phabricator.wikimedia.org/T336693> will not only move us forward
on RESTBase retirement, but also reduce the total amount of code we have to
maintain. Many thanks to Bill Pirkle, Atieno Njira, Wendy Quarshie, and
Daniel Kinzler for making this work! We also fully turned off Parsoid cache
storage in RESTBase - clients will get outputs direct from MediaWiki and
the cache will be handled by ParserCache. Next, we will re-route clients
directly to MediaWiki and fully remove Parsoid from RESTBase (T344944
<https://phabricator.wikimedia.org/T344944>). The Page Content Service
(PCS) will also handle its own cache
<https://phabricator.wikimedia.org/T348995> and we are ready to test the
new capabilities in staging soon. Many thanks to Yiannis Giannelos and the
Content Transform team and to Daniel Kinzler for his efforts and support on
decoupling Parsoid from RESTBase!
All of these multi-year initiatives help us increase sustainability and
maintainability of the platform, streamline engineering and developer
workflows, and unlock the path for new and improved platform capabilities
and product opportunities.
Outlook: Knowledge Platform in the annual plan 2024/2025
The Wikimedia Foundation has recently published the draft objectives by the
Product & Technology department
<https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2024-2025/Goals/Infrastructure>
for the next annual plan on Meta, alongside an introduction by Selena
Deckelmann and a few questions that we’re exploring
<https://meta.wikimedia.org/wiki/Talk:Wikimedia_Foundation_Annual_Plan/2024-2025/Goals/Infrastructure#Questions>.
Your input on these questions is very welcome!
The draft objectives include “Knowledge Platform I” - centered around
MediaWiki platform evolution and “Knowledge Platform II” - centered around
developer/engineering services and workflows. The objectives show only the
high-level direction for next year. The draft “key results” (currently work
in progress) will give a better idea of what areas of work we’re thinking
about. We’ll be publishing these in March and share the link + invitation
for feedback with this list again.
Thanks all for reading,
Birgit
--
Birgit Müller (she/her)
Director of Product, MediaWiki and Developer Experiences
Wikimedia Foundation <https://wikimediafoundation.org/>
Like last time
<https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Reports/January_2024>,
we’re starting this email by celebrating volunteer contributors who got
their first patch merged in MW core, WMF deployed extensions or services
over the past month:
A big thanks to GergesShamon and Agent Isai for their contributions!
Welcome :-)
Many thanks also to the reviewers - volunteers and staff - of patches by
new contributors. Your support is invaluable for helping people contribute
to MediaWiki effectively and make it a fun first experience!
The focus of this edition of the monthly MW Insights email lies on our
multi-year initiatives: Migration of MediaWiki from bare metal to Kubernetes
<https://wikitech.wikimedia.org/wiki/MediaWiki_On_Kubernetes> in Wikimedia
Production, Parser Unification
<https://www.mediawiki.org/wiki/Parsoid/Parser_Unification> and RESTBase
deprecation <https://phabricator.wikimedia.org/project/profile/6289/>.
These initiatives involve many people and projects, require orchestrated
efforts and coordination, and will help us in many different ways. Another
thing that these initiatives have in common is that we’re getting closer to
being able to benefit from these efforts :-).
Project Snapshots: Milestones reached on 3 multi-year projects
MediaWiki on Kubernetes migration is 75% complete, completing migration of
all internal traffic, all scheduled jobs and reaches 50% global traffic
milestone!
Often shortened as mw-on-k8s, MediaWiki on Kubernetes is a multi-year
effort to move all of the MediaWiki deployments running on WMF production
infrastructure to the new WikiKube platform.
The migration is underway and very recently a new milestone was hit where
the WikiKube platform now serves 50% of end user requests
<https://phabricator.wikimedia.org/T290536>. At half mark, having more
percentage of traffic on WikiKube, Service Operations is working with
Release Engineering to make changes to monitoring tools to surface any
issues during deployments to continue with further ramps. Also, almost the
entirety of what is called internal traffic
<https://phabricator.wikimedia.org/T333120>, that is traffic that is
generated by applications running in the infrastructure and reaching out to
MediaWiki for various purposes is migrated to WikiKube. A special mention
should go to MediaWiki Jobs fully migrated just a week ago
<https://phabricator.wikimedia.org/T349796>. Feel free to follow the higher
umbrella task <https://phabricator.wikimedia.org/T290536> and/or
MediaWiki_On_Kubernetes
on Wikitech <https://wikitech.wikimedia.org/wiki/MediaWiki_On_Kubernetes>.
This migration will unleash the ability to deploy multiple versions of code
simultaneously. Also, this will help enhance platform capabilities to build
dockerized isolated environments for coding, testing and even production
debugging.
MediaWiki on Kubernetes will allow us to deprecate and eventually remove a
lot of our in-house developed code. Another benefit is that we will be able
to react to sudden traffic spikes, like newsworthy events better, as the
flexing up and down is a matter of configuration change. This enables
efficient placement of workloads, packing workloads in a more
environmentally friendly way, increasing hardware utilization.
It takes a village to get this far: A big thanks to the Service Operations
team (Clément Goubert, Giuseppe Lavagetto, Alexandros Kosiaris, Kamila
Sou?ková, Hugh Nowlan, Effie Mouzeli, Reuven Lazarus, Jannis Mayboom and
Kavitha Appakayala) for their leadership on this project, the Release
Engineering team (specifically: Dan Duvall, Jeena Huneidi, Tyler Cipriani
and Ahmon Dancy) for their work on Blubber
<https://wikitech.wikimedia.org/wiki/Blubber>, the Deployment Pipeline
<https://wikitech.wikimedia.org/wiki/Deployment_pipeline> and Scap
<https://wikitech.wikimedia.org/wiki/Scap>, Dom Walden from Quality and
Test Engineering for crafting and executing a plan to test the first
deployment of MediaWiki on Kubernetes, and everyone else who has
contributed in the one way or the other! <3
More milestones:
We’re seeing the first lights at the end of the tunnel of another
multi-year initiative: The *parser unification*
<https://www.mediawiki.org/wiki/Parsoid/Parser_Unification>. One week
ago, Parsoid
Read Views was rolled out to first wikis: Parsoid is now the default read
views renderer on the Foundations’ Office Wiki and Wikitech
DiscussionTools. This early experimentation allows us to find issues in a
limited space, which will help us evaluate readiness of the feature
and increase
our confidence for future rollouts
<http://mediawiki.org/wiki/Parsoid/Parser_Unification/Confidence_Framework>.
A huge thanks to Subbu Sastry, Mateus Santos, CScott, Isabelle
Hubert-Pallatin, Arlo Breault, Shannon Bailey, Yiannis Giannelos and Sérgio
Lopes for making this work! Many thanks also to Daniel Kinzler: His work on
the RESTBase deprecation directly helped us get to this milestone :-)
*RESTBase deprecation*: We have been continuously working on decoupling
services from RESTBase, aiming for the modernisation and sustainability of
Wikimedia products in our services platform. The MediaWiki Interfaces team
finished up reimplementation of Reading Lists endpoints in MW REST API
<https://phabricator.wikimedia.org/T348491> and are now confirming with
affected callers <https://phabricator.wikimedia.org/T357478> that the new
endpoints meet their needs before rerouting calls
<https://phabricator.wikimedia.org/T348493> and retiring old code
<https://phabricator.wikimedia.org/T348494>. The overall effort
<https://phabricator.wikimedia.org/T336693> will not only move us forward
on RESTBase retirement, but also reduce the total amount of code we have to
maintain. Many thanks to Bill Pirkle, Atieno Njira, Wendy Quarshie, and
Daniel Kinzler for making this work! We also fully turned off Parsoid cache
storage in RESTBase - clients will get outputs direct from MediaWiki and
the cache will be handled by ParserCache. Next, we will re-route clients
directly to MediaWiki and fully remove Parsoid from RESTBase (T344944
<https://phabricator.wikimedia.org/T344944>). The Page Content Service
(PCS) will also handle its own cache
<https://phabricator.wikimedia.org/T348995> and we are ready to test the
new capabilities in staging soon. Many thanks to Yiannis Giannelos and the
Content Transform team and to Daniel Kinzler for his efforts and support on
decoupling Parsoid from RESTBase!
All of these multi-year initiatives help us increase sustainability and
maintainability of the platform, streamline engineering and developer
workflows, and unlock the path for new and improved platform capabilities
and product opportunities.
Outlook: Knowledge Platform in the annual plan 2024/2025
The Wikimedia Foundation has recently published the draft objectives by the
Product & Technology department
<https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2024-2025/Goals/Infrastructure>
for the next annual plan on Meta, alongside an introduction by Selena
Deckelmann and a few questions that we’re exploring
<https://meta.wikimedia.org/wiki/Talk:Wikimedia_Foundation_Annual_Plan/2024-2025/Goals/Infrastructure#Questions>.
Your input on these questions is very welcome!
The draft objectives include “Knowledge Platform I” - centered around
MediaWiki platform evolution and “Knowledge Platform II” - centered around
developer/engineering services and workflows. The objectives show only the
high-level direction for next year. The draft “key results” (currently work
in progress) will give a better idea of what areas of work we’re thinking
about. We’ll be publishing these in March and share the link + invitation
for feedback with this list again.
Thanks all for reading,
Birgit
--
Birgit Müller (she/her)
Director of Product, MediaWiki and Developer Experiences
Wikimedia Foundation <https://wikimediafoundation.org/>