Hello,
We have several more changes to the rdbms library’s interface to announce.
We previously made such announcements in May 2023
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/GATRFC63FPQJ6Z43WZDIKSHYZQJCI62H/>
and February 2023
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/YNLVJVTYTK3IGQF4HY7ITGMIJ2W5Q7CG/>
.
Query builders for all common query types
We now have query builders for all common query types:
-
SELECT
-
INSERT
-
UPDATE
-
DELETE
-
UPSERT
-
REPLACE
We encourage you to migrate calls using the old methods (such as
Database::insert) to use the query builders instead. Starting from 1.41,
all the old methods are considered internal and might change without prior
notice.
See T335377 <https://phabricator.wikimedia.org/T335377> and previous
announcements such as the one in May 2023
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/GATRFC63FPQJ6Z43WZDIKSHYZQJCI62H/>
for more information.
Script for migrating to SelectQueryBuilder
Given that there are many calls to Database::select in extensions we wrote
a simple script to help with the migration to SelectQueryBuilder. It is not
perfect and doesn’t cover many aspects such as joins but it takes away a
lot of tedious work.
Using that script we made hundreds of migrations in the MediaWiki core. As
of now, we have more than 800 calls to newSelectQueryBuilder() and only
roughly 100 direct calls to Database::select() left in core.
For more information, check out
https://gitlab.wikimedia.org/ladsgroup/migrateselect
Access to external clusters simplified
If you ever had to make database queries against external cluster databases
(such as extension1, otherwise known as x1), you will like this change.
Many of our extensions store their data in x1, for example: Echo,
UrlShortener, Cognate, Translate, ContentTranslation, GrowthExperiments,
Campaigns and many more. And it requires a lot of complexity with many
caveats (see T330590 <https://phabricator.wikimedia.org/T330590> for more
information).
Now, an extension can introduce one or more virtual domains in their
extension.json and then if you set the configuration such as:
$wgVirtualDomainsMapping['urlshortener'] = [ 'cluster' => 'extension1',
'db' => 'wikishared' ];
then $lbf->getPrimaryDatabase( 'urlshortener' ) will make the right
connection. If it’s not set in the mapping of virtual domains
configuration, it simply makes the connection to the local database. You
don’t need to do any special coding anymore which simplifies the logic a
lot and removes the need for extra configuration variables. This also helps
in facilitating proper support of external clusters in the database updater
and improvements in testing in the future.
You can take a look at the example of url shortener (gerrit:963293
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/UrlShortener/+/963293>)
for how adoption is done.
Thank you,
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
We have several more changes to the rdbms library’s interface to announce.
We previously made such announcements in May 2023
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/GATRFC63FPQJ6Z43WZDIKSHYZQJCI62H/>
and February 2023
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/YNLVJVTYTK3IGQF4HY7ITGMIJ2W5Q7CG/>
.
Query builders for all common query types
We now have query builders for all common query types:
-
SELECT
-
INSERT
-
UPDATE
-
DELETE
-
UPSERT
-
REPLACE
We encourage you to migrate calls using the old methods (such as
Database::insert) to use the query builders instead. Starting from 1.41,
all the old methods are considered internal and might change without prior
notice.
See T335377 <https://phabricator.wikimedia.org/T335377> and previous
announcements such as the one in May 2023
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/GATRFC63FPQJ6Z43WZDIKSHYZQJCI62H/>
for more information.
Script for migrating to SelectQueryBuilder
Given that there are many calls to Database::select in extensions we wrote
a simple script to help with the migration to SelectQueryBuilder. It is not
perfect and doesn’t cover many aspects such as joins but it takes away a
lot of tedious work.
Using that script we made hundreds of migrations in the MediaWiki core. As
of now, we have more than 800 calls to newSelectQueryBuilder() and only
roughly 100 direct calls to Database::select() left in core.
For more information, check out
https://gitlab.wikimedia.org/ladsgroup/migrateselect
Access to external clusters simplified
If you ever had to make database queries against external cluster databases
(such as extension1, otherwise known as x1), you will like this change.
Many of our extensions store their data in x1, for example: Echo,
UrlShortener, Cognate, Translate, ContentTranslation, GrowthExperiments,
Campaigns and many more. And it requires a lot of complexity with many
caveats (see T330590 <https://phabricator.wikimedia.org/T330590> for more
information).
Now, an extension can introduce one or more virtual domains in their
extension.json and then if you set the configuration such as:
$wgVirtualDomainsMapping['urlshortener'] = [ 'cluster' => 'extension1',
'db' => 'wikishared' ];
then $lbf->getPrimaryDatabase( 'urlshortener' ) will make the right
connection. If it’s not set in the mapping of virtual domains
configuration, it simply makes the connection to the local database. You
don’t need to do any special coding anymore which simplifies the logic a
lot and removes the need for extra configuration variables. This also helps
in facilitating proper support of external clusters in the database updater
and improvements in testing in the future.
You can take a look at the example of url shortener (gerrit:963293
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/UrlShortener/+/963293>)
for how adoption is done.
Thank you,
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>