Mailing List Archive

It's possible that I'm very out of touch, but I'll ask anyway :)

As far as I know, the main place where editors of Wikimedia's wiki sites
actually see ORES in action is the filtering and highlighting functionality
on Recent Changes. This functionality is enabled in a limited number of
wikis, but at least in some of those wikis, it works pretty well; I've just
done a quick and informal poll on the Hebrew Wikipedia village pump, and
the responses till now were that this is a good feature that helps with
patrolling.

The email says that "All ML models currently accessible on ORES are also
currently accessible on Lift Wing", and if I understand correctly, this
means that this feature in Recent Changes will keep working. Do I
understand correctly? :)

In addition, I have some followup questions:

1. The MediaWiki extension that implements the frontend in Recent Changes
is itself named "ORES". It's an internal name that isn't seen much by wiki
editors except if they go to Special:Version or to translatewiki.
Nevertheless, as the time goes by, seeing the old name may start getting
weird. So what's the plan about it? Will this extension remain as is? Will
it be renamed? Will it be replaced with a new frontend extension in the
foreseeable future?

2. Back when ORES was originally developed and deployed around 2017,
several wiki editors' communities participated in the development by
adapting the product to the needs of their wikis and languages by
translating the ORES extension's user interface and, more importantly, by
labelling a sample of several thousands of diffs from their wiki using the
Wikilabels tool. The communities that did that whole process were, more or
less, the communities to which this Recent Changes enhancement was
deployed. Will anything like that have to be done again along with the move
away from ORES?

3. Will this change open up the possibility of deploying this Recent
Changes enhancement, or a newer version thereof, to more wikis and
languages?

If you think that my questions show a wrong understanding of something,
please let me know—as I said in the beginning, its quite possible :)

Thanks!

?????? ??? ??, 3 ????? 2023, 17:16, ??? Chris Albon ?<calbon@wikimedia.org>:

> Hi everybody,
>
> TL;DR We would like users of ORES models to migrate to our new open source
> ML infrastructure, Lift Wing, within the next five months. We are available
> to help you do that, from advice to making code commits. It is important to
> note: All ML models currently accessible on ORES are also currently
> accessible on Lift Wing.
>
> As part of the Machine Learning Modernization Project (
> https://www.mediawiki.org/wiki/Machine_Learning/Modernization), the
> Machine Learning team has deployed a Wikimedia’s new machine learning
> inference infrastructure, called Lift Wing (
> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing). Lift Wing
> brings a lot of new features such as support for GPU-based models, open
> source LLM hosting, auto-scaling, stability, and ability to host a larger
> number of models.
>
> With the creation of Lift Wing, the team is turning its attention to
> deprecating the current machine learning infrastructure, ORES. ORES served
> us really well over the years, it was a successful project but it came
> before radical changes in technology like Docker, Kubernetes and more
> recently MLOps. The servers that run ORES are at the end of their planned
> lifespan and so to save cost we are going to shut them down in early 2024.
>
> We have outlined a deprecation path on Wikitech (
> https://wikitech.wikimedia.org/wiki/ORES), please read the page if you
> are a maintainer of a tool or code that uses the ORES endpoint
> https://ores.wikimedia.org/). If you have any doubt or if you need
> assistance in migrating to Lift Wing, feel free to contact the ML team via:
>
> - Email: ml@wikimedia.org
> - Phabricator: #Machine-Learning-Team tag
> - IRC (Libera): #wikimedia-ml
>
> The Machine Learning team is available to help projects migrate, from
> offering advice to making code commits. We want to make this as easy as
> possible for folks.
>
> High Level timeline:
>
> **By September 30th 2023: *Infrastructure powering the ORES API endpoint
> will be migrated from ORES to Lift Wing. For users, the API endpoint will
> remain the same, and most users won’t notice any change. Rather just the
> backend services powering the endpoint will change.
>
> Details: We'd like to add a DNS CNAME that points ores.wikimedia.org to
> ores-legacy.wikimedia.org, a new endpoint that offers a almost complete
> replacement of the ORES API calling Lift Wing behind the scenes. In an
> ideal world we'd migrate all tools to Lift Wing before decommissioning the
> infrastructure behind ores.wikimedia.org, but it turned out to be really
> challenging so to avoid disrupting users we chose to implement a transition
> layer/API.
>
> To summarize, if you don't have time to migrate before September to Lift
> Wing, your code/tool should work just fine on ores-legacy.wikimedia.org
> and you'll not have to change a line in your code thanks to the DNS CNAME.
> The ores-legacy endpoint is not a 100% replacement for ores, we removed
> some very old and not used features, so we highly recommend at least test
> the new endpoint for your use case to avoid surprises when we'll make the
> switch. In case you find anything weird, please report it to us using the
> aforementioned channels.
>
> **September to January: *We will be reaching out to every user of ORES we
> can identify and working with them to make the migration process as easy as
> possible.
>
> **By January 2024: *If all goes well, we would like zero traffic on the
> ORES API endpoint so we can turn off the ores-legacy API.
>
> If you want more information about Lift Wing, please check
> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>
> Thanks in advance for the patience and the help!
>
> Regards,
>
> The Machine Learning Team
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

Hi Amir!

Answering inline:

On Thu, Aug 3, 2023 at 10:11?PM Amir E. Aharoni <
amir.aharoni@mail.huji.ac.il> wrote:

>
> The email says that "All ML models currently accessible on ORES are also
> currently accessible on Lift Wing", and if I understand correctly, this
> means that this feature in Recent Changes will keep working. Do I
> understand correctly? :)
>

Definitely yes, we are working on migrating the ORES extension to Lift
Wing, without any change required for users. The tracking task is
https://phabricator.wikimedia.org/T319170. At the moment all wikis with the
ORES extension enabled, except fi/en/wikidata, are already using models
from Lift Wing.


> In addition, I have some followup questions:
>
> 1. The MediaWiki extension that implements the frontend in Recent Changes
> is itself named "ORES". It's an internal name that isn't seen much by wiki
> editors except if they go to Special:Version or to translatewiki.
> Nevertheless, as the time goes by, seeing the old name may start getting
> weird. So what's the plan about it? Will this extension remain as is? Will
> it be renamed? Will it be replaced with a new frontend extension in the
> foreseeable future?
>

This is a good question and we don't have a definitive answer at the
moment. Our understanding is that renaming extensions in MediaWiki is a
long and complicated process, so we'll likely not be able to rename it in
the foreseeable future. We would definitely like to add more models to RC
Filters, for example Revert Risk (for the curious, see
https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-agnostic_revert_risk),
but we are not sure yet if it is worth to create a new extension or just to
expand the ORES one. We'll get back to this list as soon as we have a
better plan :)


> 2. Back when ORES was originally developed and deployed around 2017,
> several wiki editors' communities participated in the development by
> adapting the product to the needs of their wikis and languages by
> translating the ORES extension's user interface and, more importantly, by
> labelling a sample of several thousands of diffs from their wiki using the
> Wikilabels tool. The communities that did that whole process were, more or
> less, the communities to which this Recent Changes enhancement was
> deployed. Will anything like that have to be done again along with the move
> away from ORES?
>

The first goal of Lift Wing is to provide a more modern and easy-to-use
infrastructure to host models at the WMF, for internal teams and for the
community. The focus of the Machine Learning team is to provide
infrastructure to run models on, so other teams and the community will be
able to propose what to host and we'll vet what is possible and what not
(following strict criteria like security of data and PII, model
architecture feasibility, etc..). Once a model is deployed on Lift Wing,
there will be a team or a community group owning it, namely responsible for
its development in terms of features etc.. (more info in
https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing#Hosting_a_model
).
To summarize:
* All the work done so far with ORES models will be preserved, it is
already available on Lift Wing and anybody can use it. We hope that it is
now easier to play with model servers and improve them (for WMF and the
community), but we are open to any suggestion and feedback about how to
improve it. For the curious, more details in the Lift Wing Wikitech page (
https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing).
* The future work will be split into two main areas (as I see it):
** The ML team will keep working on improving the infrastructure,
documentation, performance, etc.. of Lift Wing, to provide better tools and
data access for any new idea related to models and their usage. We'll
maintain the infrastructure with monitoring/alarms/etc.., so the day-to-day
ops will not fall on the model owners (WMF and community), so that they
will be able to concentrate themselves only on the models and their future
steps.
** Other WMF teams like Research will propose and work on new models that
the community needs, but we'll also focus on improving what is currently
being used. For example, most of the ORES traffic is for the goodfaith and
damaging models that worked very well over the years but they rely on old
training data and architectures. The Revert Risk models (for example,
https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-agnostic_revert_risk)
are an attempt to improve the reliability and performance of the
aforementioned models, using a single score instead of multiple ones.


> 3. Will this change open up the possibility of deploying this Recent
> Changes enhancement, or a newer version thereof, to more wikis and
> languages?
>

It may be possible in the future to enhance even more the RC Filters, at
the moment we are concentrating on migrating the current ones to Lift Wing,
but after that we'll start figuring out what is the next step. Any
suggestion or advice is really welcome! (see
https://wikitech.wikimedia.org/wiki/ORES#Machine_Learning_contacts)


If you think that my questions show a wrong understanding of something,
> please let me know—as I said in the beginning, its quite possible :)
>

Thanks a lot for the questions, I hope I answered your doubts, feel free to
follow up if anything is missing!

Luca (on behalf of the ML team)

Great, thank you!

?????? ??? ??, 4 ????? 2023, 11:49, ??? Luca Toscano ?<
ltoscano@wikimedia.org>:

> Hi Amir!
>
> Answering inline:
>
> On Thu, Aug 3, 2023 at 10:11?PM Amir E. Aharoni <
> amir.aharoni@mail.huji.ac.il> wrote:
>
>>
>> The email says that "All ML models currently accessible on ORES are also
>> currently accessible on Lift Wing", and if I understand correctly, this
>> means that this feature in Recent Changes will keep working. Do I
>> understand correctly? :)
>>
>
> Definitely yes, we are working on migrating the ORES extension to Lift
> Wing, without any change required for users. The tracking task is
> https://phabricator.wikimedia.org/T319170. At the moment all wikis with
> the ORES extension enabled, except fi/en/wikidata, are already using models
> from Lift Wing.
>
>
>> In addition, I have some followup questions:
>>
>> 1. The MediaWiki extension that implements the frontend in Recent Changes
>> is itself named "ORES". It's an internal name that isn't seen much by wiki
>> editors except if they go to Special:Version or to translatewiki.
>> Nevertheless, as the time goes by, seeing the old name may start getting
>> weird. So what's the plan about it? Will this extension remain as is? Will
>> it be renamed? Will it be replaced with a new frontend extension in the
>> foreseeable future?
>>
>
> This is a good question and we don't have a definitive answer at the
> moment. Our understanding is that renaming extensions in MediaWiki is a
> long and complicated process, so we'll likely not be able to rename it in
> the foreseeable future. We would definitely like to add more models to RC
> Filters, for example Revert Risk (for the curious, see
> https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-agnostic_revert_risk),
> but we are not sure yet if it is worth to create a new extension or just to
> expand the ORES one. We'll get back to this list as soon as we have a
> better plan :)
>
>
>> 2. Back when ORES was originally developed and deployed around 2017,
>> several wiki editors' communities participated in the development by
>> adapting the product to the needs of their wikis and languages by
>> translating the ORES extension's user interface and, more importantly, by
>> labelling a sample of several thousands of diffs from their wiki using the
>> Wikilabels tool. The communities that did that whole process were, more or
>> less, the communities to which this Recent Changes enhancement was
>> deployed. Will anything like that have to be done again along with the move
>> away from ORES?
>>
>
> The first goal of Lift Wing is to provide a more modern and easy-to-use
> infrastructure to host models at the WMF, for internal teams and for the
> community. The focus of the Machine Learning team is to provide
> infrastructure to run models on, so other teams and the community will be
> able to propose what to host and we'll vet what is possible and what not
> (following strict criteria like security of data and PII, model
> architecture feasibility, etc..). Once a model is deployed on Lift Wing,
> there will be a team or a community group owning it, namely responsible for
> its development in terms of features etc.. (more info in
> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing#Hosting_a_model
> ).
> To summarize:
> * All the work done so far with ORES models will be preserved, it is
> already available on Lift Wing and anybody can use it. We hope that it is
> now easier to play with model servers and improve them (for WMF and the
> community), but we are open to any suggestion and feedback about how to
> improve it. For the curious, more details in the Lift Wing Wikitech page (
> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing).
> * The future work will be split into two main areas (as I see it):
> ** The ML team will keep working on improving the infrastructure,
> documentation, performance, etc.. of Lift Wing, to provide better tools and
> data access for any new idea related to models and their usage. We'll
> maintain the infrastructure with monitoring/alarms/etc.., so the day-to-day
> ops will not fall on the model owners (WMF and community), so that they
> will be able to concentrate themselves only on the models and their future
> steps.
> ** Other WMF teams like Research will propose and work on new models that
> the community needs, but we'll also focus on improving what is currently
> being used. For example, most of the ORES traffic is for the goodfaith and
> damaging models that worked very well over the years but they rely on old
> training data and architectures. The Revert Risk models (for example,
> https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-agnostic_revert_risk)
> are an attempt to improve the reliability and performance of the
> aforementioned models, using a single score instead of multiple ones.
>
>
>> 3. Will this change open up the possibility of deploying this Recent
>> Changes enhancement, or a newer version thereof, to more wikis and
>> languages?
>>
>
> It may be possible in the future to enhance even more the RC Filters, at
> the moment we are concentrating on migrating the current ones to Lift Wing,
> but after that we'll start figuring out what is the next step. Any
> suggestion or advice is really welcome! (see
> https://wikitech.wikimedia.org/wiki/ORES#Machine_Learning_contacts)
>
>
> If you think that my questions show a wrong understanding of something,
>> please let me know—as I said in the beginning, its quite possible :)
>>
>
> Thanks a lot for the questions, I hope I answered your doubts, feel free
> to follow up if anything is missing!
>
> Luca (on behalf of the ML team)
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

> Our understanding is that renaming extensions in MediaWiki is a long and
complicated process, so we'll likely not be able to rename it in the
foreseeable future.

Why?

Renaming is usually a bad thing because it often confuses the hell out of
users, but from a technical perspective it is pretty trivial.

--
Bawolff

On Friday, August 4, 2023, Luca Toscano <ltoscano@wikimedia.org> wrote:

> Hi Amir!
>
> Answering inline:
>
> On Thu, Aug 3, 2023 at 10:11?PM Amir E. Aharoni <
> amir.aharoni@mail.huji.ac.il> wrote:
>
>>
>> The email says that "All ML models currently accessible on ORES are also
>> currently accessible on Lift Wing", and if I understand correctly, this
>> means that this feature in Recent Changes will keep working. Do I
>> understand correctly? :)
>>
>
> Definitely yes, we are working on migrating the ORES extension to Lift
> Wing, without any change required for users. The tracking task is
> https://phabricator.wikimedia.org/T319170. At the moment all wikis with
> the ORES extension enabled, except fi/en/wikidata, are already using models
> from Lift Wing.
>
>
>> In addition, I have some followup questions:
>>
>> 1. The MediaWiki extension that implements the frontend in Recent Changes
>> is itself named "ORES". It's an internal name that isn't seen much by wiki
>> editors except if they go to Special:Version or to translatewiki.
>> Nevertheless, as the time goes by, seeing the old name may start getting
>> weird. So what's the plan about it? Will this extension remain as is? Will
>> it be renamed? Will it be replaced with a new frontend extension in the
>> foreseeable future?
>>
>
> This is a good question and we don't have a definitive answer at the
> moment. Our understanding is that renaming extensions in MediaWiki is a
> long and complicated process, so we'll likely not be able to rename it in
> the foreseeable future. We would definitely like to add more models to RC
> Filters, for example Revert Risk (for the curious, see
> https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-
> agnostic_revert_risk), but we are not sure yet if it is worth to create a
> new extension or just to expand the ORES one. We'll get back to this list
> as soon as we have a better plan :)
>
>
>> 2. Back when ORES was originally developed and deployed around 2017,
>> several wiki editors' communities participated in the development by
>> adapting the product to the needs of their wikis and languages by
>> translating the ORES extension's user interface and, more importantly, by
>> labelling a sample of several thousands of diffs from their wiki using the
>> Wikilabels tool. The communities that did that whole process were, more or
>> less, the communities to which this Recent Changes enhancement was
>> deployed. Will anything like that have to be done again along with the move
>> away from ORES?
>>
>
> The first goal of Lift Wing is to provide a more modern and easy-to-use
> infrastructure to host models at the WMF, for internal teams and for the
> community. The focus of the Machine Learning team is to provide
> infrastructure to run models on, so other teams and the community will be
> able to propose what to host and we'll vet what is possible and what not
> (following strict criteria like security of data and PII, model
> architecture feasibility, etc..). Once a model is deployed on Lift Wing,
> there will be a team or a community group owning it, namely responsible for
> its development in terms of features etc.. (more info in
> https://wikitech.wikimedia.org/wiki/Machine_Learning/
> LiftWing#Hosting_a_model).
> To summarize:
> * All the work done so far with ORES models will be preserved, it is
> already available on Lift Wing and anybody can use it. We hope that it is
> now easier to play with model servers and improve them (for WMF and the
> community), but we are open to any suggestion and feedback about how to
> improve it. For the curious, more details in the Lift Wing Wikitech page (
> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing).
> * The future work will be split into two main areas (as I see it):
> ** The ML team will keep working on improving the infrastructure,
> documentation, performance, etc.. of Lift Wing, to provide better tools and
> data access for any new idea related to models and their usage. We'll
> maintain the infrastructure with monitoring/alarms/etc.., so the day-to-day
> ops will not fall on the model owners (WMF and community), so that they
> will be able to concentrate themselves only on the models and their future
> steps.
> ** Other WMF teams like Research will propose and work on new models that
> the community needs, but we'll also focus on improving what is currently
> being used. For example, most of the ORES traffic is for the goodfaith and
> damaging models that worked very well over the years but they rely on old
> training data and architectures. The Revert Risk models (for example,
> https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/
> Language-agnostic_revert_risk) are an attempt to improve the reliability
> and performance of the aforementioned models, using a single score instead
> of multiple ones.
>
>
>> 3. Will this change open up the possibility of deploying this Recent
>> Changes enhancement, or a newer version thereof, to more wikis and
>> languages?
>>
>
> It may be possible in the future to enhance even more the RC Filters, at
> the moment we are concentrating on migrating the current ones to Lift Wing,
> but after that we'll start figuring out what is the next step. Any
> suggestion or advice is really welcome! (see https://wikitech.
> wikimedia.org/wiki/ORES#Machine_Learning_contacts)
>
>
> If you think that my questions show a wrong understanding of something,
>> please let me know—as I said in the beginning, its quite possible :)
>>
>
> Thanks a lot for the questions, I hope I answered your doubts, feel free
> to follow up if anything is missing!
>
> Luca (on behalf of the ML team)
>

Am Fr., 4. Aug. 2023 um 12:53 Uhr schrieb Brian Wolff <bawolff@gmail.com>:

> > Our understanding is that renaming extensions in MediaWiki is a long
> and complicated process, so we'll likely not be able to rename it in the
> foreseeable future.
>
> Why?
>
> Renaming is usually a bad thing because it often confuses the hell out of
> users, but from a technical perspective it is pretty trivial.
>

Renaming an extension that's deployed to production is basically
impossible. e.g. attempt of renaming Extension:Flow to
StructuredDiscussions.

Basically the only viable option is to undeploy the extension, rename the
extension, and deploy it again.


> --
> Bawolff
>
> On Friday, August 4, 2023, Luca Toscano <ltoscano@wikimedia.org> wrote:
>
>> Hi Amir!
>>
>> Answering inline:
>>
>> On Thu, Aug 3, 2023 at 10:11?PM Amir E. Aharoni <
>> amir.aharoni@mail.huji.ac.il> wrote:
>>
>>>
>>> The email says that "All ML models currently accessible on ORES are also
>>> currently accessible on Lift Wing", and if I understand correctly, this
>>> means that this feature in Recent Changes will keep working. Do I
>>> understand correctly? :)
>>>
>>
>> Definitely yes, we are working on migrating the ORES extension to Lift
>> Wing, without any change required for users. The tracking task is
>> https://phabricator.wikimedia.org/T319170. At the moment all wikis with
>> the ORES extension enabled, except fi/en/wikidata, are already using models
>> from Lift Wing.
>>
>>
>>> In addition, I have some followup questions:
>>>
>>> 1. The MediaWiki extension that implements the frontend in Recent
>>> Changes is itself named "ORES". It's an internal name that isn't seen much
>>> by wiki editors except if they go to Special:Version or to translatewiki.
>>> Nevertheless, as the time goes by, seeing the old name may start getting
>>> weird. So what's the plan about it? Will this extension remain as is? Will
>>> it be renamed? Will it be replaced with a new frontend extension in the
>>> foreseeable future?
>>>
>>
>> This is a good question and we don't have a definitive answer at the
>> moment. Our understanding is that renaming extensions in MediaWiki is a
>> long and complicated process, so we'll likely not be able to rename it in
>> the foreseeable future. We would definitely like to add more models to RC
>> Filters, for example Revert Risk (for the curious, see
>> https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-agnostic_revert_risk),
>> but we are not sure yet if it is worth to create a new extension or just to
>> expand the ORES one. We'll get back to this list as soon as we have a
>> better plan :)
>>
>>
>>> 2. Back when ORES was originally developed and deployed around 2017,
>>> several wiki editors' communities participated in the development by
>>> adapting the product to the needs of their wikis and languages by
>>> translating the ORES extension's user interface and, more importantly, by
>>> labelling a sample of several thousands of diffs from their wiki using the
>>> Wikilabels tool. The communities that did that whole process were, more or
>>> less, the communities to which this Recent Changes enhancement was
>>> deployed. Will anything like that have to be done again along with the move
>>> away from ORES?
>>>
>>
>> The first goal of Lift Wing is to provide a more modern and easy-to-use
>> infrastructure to host models at the WMF, for internal teams and for the
>> community. The focus of the Machine Learning team is to provide
>> infrastructure to run models on, so other teams and the community will be
>> able to propose what to host and we'll vet what is possible and what not
>> (following strict criteria like security of data and PII, model
>> architecture feasibility, etc..). Once a model is deployed on Lift Wing,
>> there will be a team or a community group owning it, namely responsible for
>> its development in terms of features etc.. (more info in
>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing#Hosting_a_model
>> ).
>> To summarize:
>> * All the work done so far with ORES models will be preserved, it is
>> already available on Lift Wing and anybody can use it. We hope that it is
>> now easier to play with model servers and improve them (for WMF and the
>> community), but we are open to any suggestion and feedback about how to
>> improve it. For the curious, more details in the Lift Wing Wikitech page (
>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing).
>> * The future work will be split into two main areas (as I see it):
>> ** The ML team will keep working on improving the infrastructure,
>> documentation, performance, etc.. of Lift Wing, to provide better tools and
>> data access for any new idea related to models and their usage. We'll
>> maintain the infrastructure with monitoring/alarms/etc.., so the day-to-day
>> ops will not fall on the model owners (WMF and community), so that they
>> will be able to concentrate themselves only on the models and their future
>> steps.
>> ** Other WMF teams like Research will propose and work on new models that
>> the community needs, but we'll also focus on improving what is currently
>> being used. For example, most of the ORES traffic is for the goodfaith and
>> damaging models that worked very well over the years but they rely on old
>> training data and architectures. The Revert Risk models (for example,
>> https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-agnostic_revert_risk)
>> are an attempt to improve the reliability and performance of the
>> aforementioned models, using a single score instead of multiple ones.
>>
>>
>>> 3. Will this change open up the possibility of deploying this Recent
>>> Changes enhancement, or a newer version thereof, to more wikis and
>>> languages?
>>>
>>
>> It may be possible in the future to enhance even more the RC Filters, at
>> the moment we are concentrating on migrating the current ones to Lift Wing,
>> but after that we'll start figuring out what is the next step. Any
>> suggestion or advice is really welcome! (see
>> https://wikitech.wikimedia.org/wiki/ORES#Machine_Learning_contacts)
>>
>>
>> If you think that my questions show a wrong understanding of something,
>>> please let me know—as I said in the beginning, its quite possible :)
>>>
>>
>> Thanks a lot for the questions, I hope I answered your doubts, feel free
>> to follow up if anything is missing!
>>
>> Luca (on behalf of the ML team)
>>
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/



--
Amir (he/him)

?????? ??? ??, 4 ????? 2023, 13:53, ??? Brian Wolff ?<bawolff@gmail.com>:

> > Our understanding is that renaming extensions in MediaWiki is a long
> and complicated process, so we'll likely not be able to rename it in the
> foreseeable future.
>
> Why?
>
> Renaming is usually a bad thing because it often confuses the hell out of
> users, but from a technical perspective it is pretty trivial.
>
> --
>

I'm not the biggest expert on MediaWiki, but from the little I do know, the
truth is closer to what Bawolff says. It's just a bit of careful searching
and replacing. And in this case, it probably doesn't affect the users very
much, because, as I've already written above, the name is not seen by most
users in the frontend.

Although precisely because of that, it's not the most important thing to do
either. It will just become a bit confusing in the long run that the ORES
technology is declared as deprecated and the servers are turned off, but
the ORES extension is still installed on some big wikis.

If this extension only adds the Recent Changes filtering and highlighting,
perhaps it can be given a name that describes its function, such as
"MLRevisionLabels" or something like that.

>
>

Hi Chris & ML team,

Good to see LiftWing is finally becoming a reality. There are a few things
in the documentation that I would like to clarify.

1. In [1], the bot owner is encouraged to move to the revertrisk score.
However, in [2], it's explicitly mentioned that the model should not be
used for "Auto-removing edits that a user makes without another editor in
the loop". So, should bot owners currently reverting based on goodfaith and
damaging scores explore the new models? If so, do you have any suggestions
on how to automatically match thresholds between the old and new models?
2. I could not find any reference regarding the ores scores exposed through
other APIs (specifically the RC API [3]). Will those be available going
forward? Under which names?
3. Will it still be possible to (re-)train existing and new model for a
specific wiki? How and when?

Thanks,
Strainu

[1]
https://wikitech.wikimedia.org/wiki/ORES#Example:_migrating_a_Bot_from_ORES_to_Lift_Wing
[2]
https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-agnostic_revert_risk#Users_and_uses
[3]
https://ro.wikipedia.org/w/api.php?action=query&format=json&list=recentchanges&rcnamespace=0%7C4%7C6%7C8%7C10&
*rcprop=*title%7Ctimestamp%7Cids%7C*oresscores*
%7Ctags%7Cpatrolled&rcshow=unpatrolled&rclimit=50&rctype=edit%7Cnew%7Ccategorize

În joi, 3 aug. 2023 la 17:16, Chris Albon <calbon@wikimedia.org> a scris:

> Hi everybody,
>
> TL;DR We would like users of ORES models to migrate to our new open source
> ML infrastructure, Lift Wing, within the next five months. We are available
> to help you do that, from advice to making code commits. It is important to
> note: All ML models currently accessible on ORES are also currently
> accessible on Lift Wing.
>
> As part of the Machine Learning Modernization Project (
> https://www.mediawiki.org/wiki/Machine_Learning/Modernization), the
> Machine Learning team has deployed a Wikimedia’s new machine learning
> inference infrastructure, called Lift Wing (
> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing). Lift Wing
> brings a lot of new features such as support for GPU-based models, open
> source LLM hosting, auto-scaling, stability, and ability to host a larger
> number of models.
>
> With the creation of Lift Wing, the team is turning its attention to
> deprecating the current machine learning infrastructure, ORES. ORES served
> us really well over the years, it was a successful project but it came
> before radical changes in technology like Docker, Kubernetes and more
> recently MLOps. The servers that run ORES are at the end of their planned
> lifespan and so to save cost we are going to shut them down in early 2024.
>
> We have outlined a deprecation path on Wikitech (
> https://wikitech.wikimedia.org/wiki/ORES), please read the page if you
> are a maintainer of a tool or code that uses the ORES endpoint
> https://ores.wikimedia.org/). If you have any doubt or if you need
> assistance in migrating to Lift Wing, feel free to contact the ML team via:
>
> - Email: ml@wikimedia.org
> - Phabricator: #Machine-Learning-Team tag
> - IRC (Libera): #wikimedia-ml
>
> The Machine Learning team is available to help projects migrate, from
> offering advice to making code commits. We want to make this as easy as
> possible for folks.
>
> High Level timeline:
>
> **By September 30th 2023: *Infrastructure powering the ORES API endpoint
> will be migrated from ORES to Lift Wing. For users, the API endpoint will
> remain the same, and most users won’t notice any change. Rather just the
> backend services powering the endpoint will change.
>
> Details: We'd like to add a DNS CNAME that points ores.wikimedia.org to
> ores-legacy.wikimedia.org, a new endpoint that offers a almost complete
> replacement of the ORES API calling Lift Wing behind the scenes. In an
> ideal world we'd migrate all tools to Lift Wing before decommissioning the
> infrastructure behind ores.wikimedia.org, but it turned out to be really
> challenging so to avoid disrupting users we chose to implement a transition
> layer/API.
>
> To summarize, if you don't have time to migrate before September to Lift
> Wing, your code/tool should work just fine on ores-legacy.wikimedia.org
> and you'll not have to change a line in your code thanks to the DNS CNAME.
> The ores-legacy endpoint is not a 100% replacement for ores, we removed
> some very old and not used features, so we highly recommend at least test
> the new endpoint for your use case to avoid surprises when we'll make the
> switch. In case you find anything weird, please report it to us using the
> aforementioned channels.
>
> **September to January: *We will be reaching out to every user of ORES we
> can identify and working with them to make the migration process as easy as
> possible.
>
> **By January 2024: *If all goes well, we would like zero traffic on the
> ORES API endpoint so we can turn off the ores-legacy API.
>
> If you want more information about Lift Wing, please check
> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>
> Thanks in advance for the patience and the help!
>
> Regards,
>
> The Machine Learning Team
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org> wrote:

> Hi everybody,
>
> TL;DR We would like users of ORES models to migrate to our new open source
> ML infrastructure, Lift Wing, within the next five months. We are available
> to help you do that, from advice to making code commits. It is important to
> note: All ML models currently accessible on ORES are also currently
> accessible on Lift Wing.
>
> As part of the Machine Learning Modernization Project (
> https://www.mediawiki.org/wiki/Machine_Learning/Modernization), the
> Machine Learning team has deployed a Wikimedia’s new machine learning
> inference infrastructure, called Lift Wing (
> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing). Lift Wing
> brings a lot of new features such as support for GPU-based models, open
> source LLM hosting, auto-scaling, stability, and ability to host a larger
> number of models.
>

This sounds quite exciting! What's the best place to read up on that
planned support for GPU-based models and open source LLMs? (I also saw in
the recent NYT article[1] that the team is "in the process of adapting A.I.
models that are 'off the shelf; — essentially models that have been made
available by researchers for anyone to freely customize — so that
Wikipedia’s editors can use them for their work.")

I'm aware of the history[2] of not being able to use NVIDIA GPUs due to
their CUDA drivers being proprietary. It was mentioned recently in the
Wikimedia AI Telegram group that this is still a serious limitation,
despite some new explorations with AMD GPUs[3] - to the point that e.g. the
WMF's Language team has resorted to using models without GPU support (CPU
only).[4]
It sounds like there is reasonable hope that this situation could change
fairly soon? Would it also mean both at the same time, i.e. open source
LLMs running with GPU support (considering that at least some
well-known ones appear to require torch.cuda.is_available() == True for
that)?

Regards, Tilman

[1] https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
[2]
https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
[3] https://phabricator.wikimedia.org/T334583 etc.
[4]
https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental but, I
understand, written to be deployable on WMF infrastructure)


>
> With the creation of Lift Wing, the team is turning its attention to
> deprecating the current machine learning infrastructure, ORES. ORES served
> us really well over the years, it was a successful project but it came
> before radical changes in technology like Docker, Kubernetes and more
> recently MLOps. The servers that run ORES are at the end of their planned
> lifespan and so to save cost we are going to shut them down in early 2024.
>
> We have outlined a deprecation path on Wikitech (
> https://wikitech.wikimedia.org/wiki/ORES), please read the page if you
> are a maintainer of a tool or code that uses the ORES endpoint
> https://ores.wikimedia.org/). If you have any doubt or if you need
> assistance in migrating to Lift Wing, feel free to contact the ML team via:
>
> - Email: ml@wikimedia.org
> - Phabricator: #Machine-Learning-Team tag
> - IRC (Libera): #wikimedia-ml
>
> The Machine Learning team is available to help projects migrate, from
> offering advice to making code commits. We want to make this as easy as
> possible for folks.
>
> High Level timeline:
>
> **By September 30th 2023: *Infrastructure powering the ORES API endpoint
> will be migrated from ORES to Lift Wing. For users, the API endpoint will
> remain the same, and most users won’t notice any change. Rather just the
> backend services powering the endpoint will change.
>
> Details: We'd like to add a DNS CNAME that points ores.wikimedia.org to
> ores-legacy.wikimedia.org, a new endpoint that offers a almost complete
> replacement of the ORES API calling Lift Wing behind the scenes. In an
> ideal world we'd migrate all tools to Lift Wing before decommissioning the
> infrastructure behind ores.wikimedia.org, but it turned out to be really
> challenging so to avoid disrupting users we chose to implement a transition
> layer/API.
>
> To summarize, if you don't have time to migrate before September to Lift
> Wing, your code/tool should work just fine on ores-legacy.wikimedia.org
> and you'll not have to change a line in your code thanks to the DNS CNAME.
> The ores-legacy endpoint is not a 100% replacement for ores, we removed
> some very old and not used features, so we highly recommend at least test
> the new endpoint for your use case to avoid surprises when we'll make the
> switch. In case you find anything weird, please report it to us using the
> aforementioned channels.
>
> **September to January: *We will be reaching out to every user of ORES we
> can identify and working with them to make the migration process as easy as
> possible.
>
> **By January 2024: *If all goes well, we would like zero traffic on the
> ORES API endpoint so we can turn off the ores-legacy API.
>
> If you want more information about Lift Wing, please check
> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>
> Thanks in advance for the patience and the help!
>
> Regards,
>
> The Machine Learning Team
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

Hi Strainu,

Here Diego from the WMF Research team,

> 1. In [1], the bot owner is encouraged to move to the revertrisk score.
However, in [2], it's explicitly mentioned that the model should not be
used for "Auto-removing edits that a user makes without another editor in
the loop". So, should bot owners currently reverting based on goodfaith and
damaging scores explore the new models? If so, do you have any suggestions
on how to automatically match thresholds between the old and new models?

Sorry for the confusion, we have updated this model card. You can use this
model for "automatically reverting content" as you were using ORES. Here
<https://phabricator.wikimedia.org/F37149700> you can see the model's
performance comparison.

Our current recommendation is to use the Language Agnostic
<https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-agnostic_revert_risk>model
for this task (patrolling bots) The Multilingual
<https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Multilingual_revert_risk>
model is performing better for IP Edits, but we are still working on
improving its stability. Within the next 3 months we expect to improve
Language Agnostic accuracy in anonymous edits, and also Multilingual model
stability.

Best,
Diego

Hi Tilman,

Most of the work is still very experimental. We have hosted a few LLMs on
Lift Wing already (StarCoder for example) but they were just running on
CPU, far too slow for real use cases. But it proves that we can easily host
LLMs on Lift Wing. We have been pretty quiet about it while we focus on the
ORES migration, but it is our next big project. More soon hopefully!

Where we are now is that we have budget for a big GPU purchase (~10-20 GPUs
depending on cost), the question we will try to answer after the ORES
migration is complete is: what GPUs should we purchase? We are trying to
balance our strong preference to stay open source (i.e. AMD mROC) in a
world dominated by a single closed source vendor (i.e. Nvidia). In
addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
H100, etc) or a mix of big and small? We will need to figure out all this.

I wouldn't characterize WMF's Language Team using CPU as because of AMD,
rather at the time we didn't have the budget for GPUs so Lift Wing didn't
have any. Since then we have moved two GPUs onto Lift Wing for testing but
they are pretty old (2017ish). Once we make the big GPU purchase Lift Wing
will gain a lot of functionality for LLM and similar models.

Chris

On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com> wrote:

> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org> wrote:
>
>> Hi everybody,
>>
>> TL;DR We would like users of ORES models to migrate to our new open
>> source ML infrastructure, Lift Wing, within the next five months. We are
>> available to help you do that, from advice to making code commits. It is
>> important to note: All ML models currently accessible on ORES are also
>> currently accessible on Lift Wing.
>>
>> As part of the Machine Learning Modernization Project (
>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization), the
>> Machine Learning team has deployed a Wikimedia’s new machine learning
>> inference infrastructure, called Lift Wing (
>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing). Lift
>> Wing brings a lot of new features such as support for GPU-based models,
>> open source LLM hosting, auto-scaling, stability, and ability to host a
>> larger number of models.
>>
>
> This sounds quite exciting! What's the best place to read up on that
> planned support for GPU-based models and open source LLMs? (I also saw in
> the recent NYT article[1] that the team is "in the process of adapting A.I.
> models that are 'off the shelf; — essentially models that have been made
> available by researchers for anyone to freely customize — so that
> Wikipedia’s editors can use them for their work.")
>
> I'm aware of the history[2] of not being able to use NVIDIA GPUs due to
> their CUDA drivers being proprietary. It was mentioned recently in the
> Wikimedia AI Telegram group that this is still a serious limitation,
> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
> WMF's Language team has resorted to using models without GPU support (CPU
> only).[4]
> It sounds like there is reasonable hope that this situation could change
> fairly soon? Would it also mean both at the same time, i.e. open source
> LLMs running with GPU support (considering that at least some
> well-known ones appear to require torch.cuda.is_available() == True for
> that)?
>
> Regards, Tilman
>
> [1] https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
> [2]
> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
> [3] https://phabricator.wikimedia.org/T334583 etc.
> [4]
> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental but, I
> understand, written to be deployable on WMF infrastructure)
>
>
>>
>> With the creation of Lift Wing, the team is turning its attention to
>> deprecating the current machine learning infrastructure, ORES. ORES served
>> us really well over the years, it was a successful project but it came
>> before radical changes in technology like Docker, Kubernetes and more
>> recently MLOps. The servers that run ORES are at the end of their planned
>> lifespan and so to save cost we are going to shut them down in early 2024.
>>
>> We have outlined a deprecation path on Wikitech (
>> https://wikitech.wikimedia.org/wiki/ORES), please read the page if you
>> are a maintainer of a tool or code that uses the ORES endpoint
>> https://ores.wikimedia.org/). If you have any doubt or if you need
>> assistance in migrating to Lift Wing, feel free to contact the ML team via:
>>
>> - Email: ml@wikimedia.org
>> - Phabricator: #Machine-Learning-Team tag
>> - IRC (Libera): #wikimedia-ml
>>
>> The Machine Learning team is available to help projects migrate, from
>> offering advice to making code commits. We want to make this as easy as
>> possible for folks.
>>
>> High Level timeline:
>>
>> **By September 30th 2023: *Infrastructure powering the ORES API endpoint
>> will be migrated from ORES to Lift Wing. For users, the API endpoint will
>> remain the same, and most users won’t notice any change. Rather just the
>> backend services powering the endpoint will change.
>>
>> Details: We'd like to add a DNS CNAME that points ores.wikimedia.org to
>> ores-legacy.wikimedia.org, a new endpoint that offers a almost complete
>> replacement of the ORES API calling Lift Wing behind the scenes. In an
>> ideal world we'd migrate all tools to Lift Wing before decommissioning the
>> infrastructure behind ores.wikimedia.org, but it turned out to be really
>> challenging so to avoid disrupting users we chose to implement a transition
>> layer/API.
>>
>> To summarize, if you don't have time to migrate before September to Lift
>> Wing, your code/tool should work just fine on ores-legacy.wikimedia.org
>> and you'll not have to change a line in your code thanks to the DNS CNAME.
>> The ores-legacy endpoint is not a 100% replacement for ores, we removed
>> some very old and not used features, so we highly recommend at least test
>> the new endpoint for your use case to avoid surprises when we'll make the
>> switch. In case you find anything weird, please report it to us using the
>> aforementioned channels.
>>
>> **September to January: *We will be reaching out to every user of ORES
>> we can identify and working with them to make the migration process as easy
>> as possible.
>>
>> **By January 2024: *If all goes well, we would like zero traffic on the
>> ORES API endpoint so we can turn off the ores-legacy API.
>>
>> If you want more information about Lift Wing, please check
>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>
>> Thanks in advance for the patience and the help!
>>
>> Regards,
>>
>> The Machine Learning Team
>> _______________________________________________
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

Hi Chris,

On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org> wrote:

> Hi Tilman,
>
> Most of the work is still very experimental. We have hosted a few LLMs on
> Lift Wing already (StarCoder for example) but they were just running on
> CPU, far too slow for real use cases. But it proves that we can easily host
> LLMs on Lift Wing. We have been pretty quiet about it while we focus on the
> ORES migration, but it is our next big project. More soon hopefully!
>
Understood. Looking forward to learning more later!


> Where we are now is that we have budget for a big GPU purchase (~10-20
> GPUs depending on cost), the question we will try to answer after the ORES
> migration is complete is: what GPUs should we purchase? We are trying to
> balance our strong preference to stay open source (i.e. AMD mROC) in a
> world dominated by a single closed source vendor (i.e. Nvidia). In
> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
> H100, etc) or a mix of big and small? We will need to figure out all this.
>
I see. On that matter, what do you folks make of the recent announcements
of AMD's partnerships with Hugging Face and Pytorch[5]? (which, I
understand, came after the ML team had already launched the aforementioned
new AMD explorations)

"Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch to take
on Nvidia [...]
Both partnerships involve AMD’s ROCm AI software stack, the company’s
answer to Nvidia’s proprietary CUDA platform and application-programming
interface. AMD called ROCm an open and portable AI system with
out-of-the-box support that can port to existing AI models. [...B]oth AMD
and Hugging Face are dedicating engineering resources to each other and
sharing data to ensure that the constantly updated AI models from Hugging
Face, which might not otherwise run well on AMD hardware, would be
“guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
will fully upstream the ROCm software stack and “provide immediate ‘day
zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
accelerators,” which is meant to appeal to those customers looking to
switch from Nvidia’s software ecosystem."


In their own announcement, Hugging Face offered further details, including
a pretty impressive list of models to be supported:[6]


"We intend to support state-of-the-art transformer architectures for
natural language processing, computer vision, and speech, such as BERT,
DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
support more traditional computer vision models, like ResNet and ResNext,
and deep learning recommendation models, a first for us. [..] We'll do our
best to test and validate these models for PyTorch, TensorFlow, and ONNX
Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
seamlessly in our open-source libraries, starting with the transformers
library."


Do you think this may promise too much, or could it point to a possible
solution of the Foundation's conundrum?
In any case, this seems to be an interesting moment where many in AI are
trying to move away from Nvidia's proprietary CUDA platform. Most of them
probably more for financial and availability reasons though, given the
current GPU shortages[7] (which the ML team is undoubtedly aware of
already; mentioning this as context for others on this list. See also
Marketwatch's remarks about current margins[5]).

Regards, Tilman


[5]
https://archive.ph/2023.06.15-173527/https://www.marketwatch.com/amp/story/open-source-ai-amd-looks-to-hugging-face-and-meta-spinoff-pytorch-to-take-on-nvidia-e4738f87
[6] https://huggingface.co/blog/huggingface-and-amd
[7] See e.g. https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/
(avoid playing the song though. Don't say I didn't warn you)


> I wouldn't characterize WMF's Language Team using CPU as because of AMD,
> rather at the time we didn't have the budget for GPUs so Lift Wing didn't
> have any. Since then we have moved two GPUs onto Lift Wing for testing but
> they are pretty old (2017ish). Once we make the big GPU purchase Lift Wing
> will gain a lot of functionality for LLM and similar models.
>
> Chris
>
> On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com> wrote:
>
>> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org> wrote:
>>
>>> Hi everybody,
>>>
>>> TL;DR We would like users of ORES models to migrate to our new open
>>> source ML infrastructure, Lift Wing, within the next five months. We are
>>> available to help you do that, from advice to making code commits. It is
>>> important to note: All ML models currently accessible on ORES are also
>>> currently accessible on Lift Wing.
>>>
>>> As part of the Machine Learning Modernization Project (
>>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization), the
>>> Machine Learning team has deployed a Wikimedia’s new machine learning
>>> inference infrastructure, called Lift Wing (
>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing). Lift
>>> Wing brings a lot of new features such as support for GPU-based models,
>>> open source LLM hosting, auto-scaling, stability, and ability to host a
>>> larger number of models.
>>>
>>
>> This sounds quite exciting! What's the best place to read up on that
>> planned support for GPU-based models and open source LLMs? (I also saw in
>> the recent NYT article[1] that the team is "in the process of adapting A.I.
>> models that are 'off the shelf; — essentially models that have been made
>> available by researchers for anyone to freely customize — so that
>> Wikipedia’s editors can use them for their work.")
>>
>> I'm aware of the history[2] of not being able to use NVIDIA GPUs due to
>> their CUDA drivers being proprietary. It was mentioned recently in the
>> Wikimedia AI Telegram group that this is still a serious limitation,
>> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
>> WMF's Language team has resorted to using models without GPU support (CPU
>> only).[4]
>> It sounds like there is reasonable hope that this situation could change
>> fairly soon? Would it also mean both at the same time, i.e. open source
>> LLMs running with GPU support (considering that at least some
>> well-known ones appear to require torch.cuda.is_available() == True for
>> that)?
>>
>> Regards, Tilman
>>
>> [1] https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
>> [2]
>> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
>> [3] https://phabricator.wikimedia.org/T334583 etc.
>> [4]
>> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
>> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental but, I
>> understand, written to be deployable on WMF infrastructure)
>>
>>
>>>
>>> With the creation of Lift Wing, the team is turning its attention to
>>> deprecating the current machine learning infrastructure, ORES. ORES served
>>> us really well over the years, it was a successful project but it came
>>> before radical changes in technology like Docker, Kubernetes and more
>>> recently MLOps. The servers that run ORES are at the end of their planned
>>> lifespan and so to save cost we are going to shut them down in early 2024.
>>>
>>> We have outlined a deprecation path on Wikitech (
>>> https://wikitech.wikimedia.org/wiki/ORES), please read the page if you
>>> are a maintainer of a tool or code that uses the ORES endpoint
>>> https://ores.wikimedia.org/). If you have any doubt or if you need
>>> assistance in migrating to Lift Wing, feel free to contact the ML team via:
>>>
>>> - Email: ml@wikimedia.org
>>> - Phabricator: #Machine-Learning-Team tag
>>> - IRC (Libera): #wikimedia-ml
>>>
>>> The Machine Learning team is available to help projects migrate, from
>>> offering advice to making code commits. We want to make this as easy as
>>> possible for folks.
>>>
>>> High Level timeline:
>>>
>>> **By September 30th 2023: *Infrastructure powering the ORES API
>>> endpoint will be migrated from ORES to Lift Wing. For users, the API
>>> endpoint will remain the same, and most users won’t notice any change.
>>> Rather just the backend services powering the endpoint will change.
>>>
>>> Details: We'd like to add a DNS CNAME that points ores.wikimedia.org to
>>> ores-legacy.wikimedia.org, a new endpoint that offers a almost complete
>>> replacement of the ORES API calling Lift Wing behind the scenes. In an
>>> ideal world we'd migrate all tools to Lift Wing before decommissioning the
>>> infrastructure behind ores.wikimedia.org, but it turned out to be
>>> really challenging so to avoid disrupting users we chose to implement a
>>> transition layer/API.
>>>
>>> To summarize, if you don't have time to migrate before September to Lift
>>> Wing, your code/tool should work just fine on ores-legacy.wikimedia.org
>>> and you'll not have to change a line in your code thanks to the DNS CNAME.
>>> The ores-legacy endpoint is not a 100% replacement for ores, we removed
>>> some very old and not used features, so we highly recommend at least test
>>> the new endpoint for your use case to avoid surprises when we'll make the
>>> switch. In case you find anything weird, please report it to us using the
>>> aforementioned channels.
>>>
>>> **September to January: *We will be reaching out to every user of ORES
>>> we can identify and working with them to make the migration process as easy
>>> as possible.
>>>
>>> **By January 2024: *If all goes well, we would like zero traffic on the
>>> ORES API endpoint so we can turn off the ores-legacy API.
>>>
>>> If you want more information about Lift Wing, please check
>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>>
>>> Thanks in advance for the patience and the help!
>>>
>>> Regards,
>>>
>>> The Machine Learning Team
>>> _______________________________________________
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> _______________________________________________
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

Hi Tilman!

On Tue, Aug 8, 2023 at 5:45?AM Tilman Bayer <haebwiki@gmail.com> wrote:

>
> Hi Chris,
>
> On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org> wrote:
>
>> Hi Tilman,
>>
>> Most of the work is still very experimental. We have hosted a few LLMs on
>> Lift Wing already (StarCoder for example) but they were just running on
>> CPU, far too slow for real use cases. But it proves that we can easily host
>> LLMs on Lift Wing. We have been pretty quiet about it while we focus on the
>> ORES migration, but it is our next big project. More soon hopefully!
>>
> Understood. Looking forward to learning more later!
>
>
>> Where we are now is that we have budget for a big GPU purchase (~10-20
>> GPUs depending on cost), the question we will try to answer after the ORES
>> migration is complete is: what GPUs should we purchase? We are trying to
>> balance our strong preference to stay open source (i.e. AMD mROC) in a
>> world dominated by a single closed source vendor (i.e. Nvidia). In
>> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
>> H100, etc) or a mix of big and small? We will need to figure out all this.
>>
> I see. On that matter, what do you folks make of the recent announcements
> of AMD's partnerships with Hugging Face and Pytorch[5]? (which, I
> understand, came after the ML team had already launched the aforementioned
> new AMD explorations)
>
> "Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch to
> take on Nvidia [...]
> Both partnerships involve AMD’s ROCm AI software stack, the company’s
> answer to Nvidia’s proprietary CUDA platform and application-programming
> interface. AMD called ROCm an open and portable AI system with
> out-of-the-box support that can port to existing AI models. [...B]oth AMD
> and Hugging Face are dedicating engineering resources to each other and
> sharing data to ensure that the constantly updated AI models from Hugging
> Face, which might not otherwise run well on AMD hardware, would be
> “guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
> will fully upstream the ROCm software stack and “provide immediate ‘day
> zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
> accelerators,” which is meant to appeal to those customers looking to
> switch from Nvidia’s software ecosystem."
>
>
> In their own announcement, Hugging Face offered further details, including
> a pretty impressive list of models to be supported:[6]
>
>
> "We intend to support state-of-the-art transformer architectures for
> natural language processing, computer vision, and speech, such as BERT,
> DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
> generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
> LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
> support more traditional computer vision models, like ResNet and ResNext,
> and deep learning recommendation models, a first for us. [..] We'll do our
> best to test and validate these models for PyTorch, TensorFlow, and ONNX
> Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
> seamlessly in our open-source libraries, starting with the transformers
> library."
>
>
> Do you think this may promise too much, or could it point to a possible
> solution of the Foundation's conundrum?
>

In https://phabricator.wikimedia.org/T334583 we experimented with LLMs and
AMD GPUs on Lift Wing, and we confirmed the good results that Pytorch
announced, We were able to run bloom-3b, bloom-560m, nllb-200 and falcon-7b
on Lift Wing, having issues only with the last one since the GPU VRAM was
not enough (16GB are low for Falcon-7b). So we can confirm that AMD ROCm
works really well with Pytorch :)


> In any case, this seems to be an interesting moment where many in AI are
> trying to move away from Nvidia's proprietary CUDA platform.
>

This is my own view, not my team's, so I can't speak up for what the WMF
will decide, but I think we should keep going with AMD and avoid Nvidia as
much as possible. Our strong stand against proprietary software should
hold, even if it means more efforts and work to advance in the ML field. I
completely get the frustration when common libraries and tools have more
difficulty to run on AMD than Nvidia, but our communities should align (in
my opinion) to the most open source solution and contribute (where
possible) so that more and more people adopt the same.
Adding proprietary software to the WMF infrastructure and practices is also
something that is technically difficult for various reasons (from the Linux
Kernel maintenance to Debian package upload), meanwhile we already have
everything set up and working for AMD (that works nicely with our
infrastructure). Moreover Debian upstream has recently created a team to
maintain AMD ROCm packages (https://lists.debian.org/debian-ai/), so it
will be interesting to see what their direction will be (so far it seems
aligned to ours).

Thanks!

Luca

Hi Strainu!

On Fri, Aug 4, 2023 at 3:25?PM Strainu <strainu10@gmail.com> wrote:

> Hi Chris & ML team,
>
> Good to see LiftWing is finally becoming a reality. There are a few things
> in the documentation that I would like to clarify.
>
> 1. In [1], the bot owner is encouraged to move to the revertrisk score.
> However, in [2], it's explicitly mentioned that the model should not be
> used for "Auto-removing edits that a user makes without another editor in
> the loop". So, should bot owners currently reverting based on goodfaith and
> damaging scores explore the new models? If so, do you have any suggestions
> on how to automatically match thresholds between the old and new models?
>

Diego (from the Research team) answered this bit afaics, but I read it in
another Wikitech-l thread (maybe it is my email reader, not sure, but I
wanted to point it out in case you missed it).
Quoting Diego:
"""
Sorry for the confusion, we have updated this model card. You can use this
model for "automatically reverting content" as you were using ORES. Here
you can see the model's performance comparison.

Our current recommendation is to use the Language Agnostic model for this
task (patrolling bots). The Multilingual model is performing better for IP
Edits, but we are still working on improving its stability. Within the
next 3 months we expect to improve Language Agnostic accuracy in anonymous
edits, and also Multilingual model stability.
"""

2. I could not find any reference regarding the ores scores exposed through
> other APIs (specifically the RC API [3]). Will those be available going
> forward? Under which names?
>

I am very ignorant about RC APIs, but if you want to explore this part more
please open a task in Phabricator with the Machine-Learning-team tag, we'll
try to research what is possible and get back to you. We'd be also curious
to know the use case, to figure out how to best support it.

3. Will it still be possible to (re-)train existing and new model for a
> specific wiki? How and when?
>

So far the ML team concentrated all the efforts in the serving
infrastructure (Lift Wing), meanwhile the training part is still to be
decided. In [1] we added info about how to request to host a model on Lift
Wing, but we didn't provide any automated way to train or retrain the
models over time. It is a big effort that we'll tackle in the future, we'll
keep this list updated as much as possible. All the models that we host now
have been trained on big nodes like the Analytics statistics ones [2], but
every re-train is manual and ad-hoc for a specific use case. We are also
strongly encouraging people to migrate away from Revscoring models
(goodfaith, damaging, etc..) as much as possible, we'd prefer not to to
retrain those (where possible) and migrate people to more modern solutions
(like Revert Risk). Having said this, if you have any specific request
please open a Phabricator task with the Machine-Learning-team tag and we'll
evaluate the use case.

Thanks!

Luca

[1]:
https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing#Hosting_a_model
[2]: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Clients

On Tue, 8 Aug 2023, 10:45 Tilman Bayer, <haebwiki@gmail.com> wrote:

>
> Hi Chris,
>
> On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org> wrote:
>
>> Hi Tilman,
>>
>> Most of the work is still very experimental. We have hosted a few LLMs on
>> Lift Wing already (StarCoder for example) but they were just running on
>> CPU, far too slow for real use cases. But it proves that we can easily host
>> LLMs on Lift Wing. We have been pretty quiet about it while we focus on the
>> ORES migration, but it is our next big project. More soon hopefully!
>>
> Understood. Looking forward to learning more later!
>
>
>> Where we are now is that we have budget for a big GPU purchase (~10-20
>> GPUs depending on cost), the question we will try to answer after the ORES
>> migration is complete is: what GPUs should we purchase? We are trying to
>> balance our strong preference to stay open source (i.e. AMD mROC) in a
>> world dominated by a single closed source vendor (i.e. Nvidia). In
>> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
>> H100, etc) or a mix of big and small? We will need to figure out all this.
>>
> I see. On that matter, what do you folks make of the recent announcements
> of AMD's partnerships with Hugging Face and Pytorch[5]? (which, I
> understand, came after the ML team had already launched the aforementioned
> new AMD explorations)
>
> "Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch to
> take on Nvidia [...]
> Both partnerships involve AMD’s ROCm AI software stack, the company’s
> answer to Nvidia’s proprietary CUDA platform and application-programming
> interface. AMD called ROCm an open and portable AI system with
> out-of-the-box support that can port to existing AI models. [...B]oth AMD
> and Hugging Face are dedicating engineering resources to each other and
> sharing data to ensure that the constantly updated AI models from Hugging
> Face, which might not otherwise run well on AMD hardware, would be
> “guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
> will fully upstream the ROCm software stack and “provide immediate ‘day
> zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
> accelerators,” which is meant to appeal to those customers looking to
> switch from Nvidia’s software ecosystem."
>
>
> In their own announcement, Hugging Face offered further details, including
> a pretty impressive list of models to be supported:[6]
>
>
> "We intend to support state-of-the-art transformer architectures for
> natural language processing, computer vision, and speech, such as BERT,
> DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
> generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
> LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
> support more traditional computer vision models, like ResNet and ResNext,
> and deep learning recommendation models, a first for us. [..] We'll do our
> best to test and validate these models for PyTorch, TensorFlow, and ONNX
> Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
> seamlessly in our open-source libraries, starting with the transformers
> library."
>
>
> Do you think this may promise too much, or could it point to a possible
> solution of the Foundation's conundrum?
> In any case, this seems to be an interesting moment where many in AI are
> trying to move away from Nvidia's proprietary CUDA platform. Most of them
> probably more for financial and availability reasons though, given the
> current GPU shortages[7] (which the ML team is undoubtedly aware of
> already; mentioning this as context for others on this list. See also
> Marketwatch's remarks about current margins[5]).
>
> Regards, Tilman
>
>
> [5]
> https://archive.ph/2023.06.15-173527/https://www.marketwatch.com/amp/story/open-source-ai-amd-looks-to-hugging-face-and-meta-spinoff-pytorch-to-take-on-nvidia-e4738f87
> [6] https://huggingface.co/blog/huggingface-and-amd
> [7] See e.g.
> https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/ (avoid
> playing the song though. Don't say I didn't warn you)
>
>
>> I wouldn't characterize WMF's Language Team using CPU as because of AMD,
>> rather at the time we didn't have the budget for GPUs so Lift Wing didn't
>> have any. Since then we have moved two GPUs onto Lift Wing for testing but
>> they are pretty old (2017ish). Once we make the big GPU purchase Lift Wing
>> will gain a lot of functionality for LLM and similar models.
>>
>> Chris
>>
>> On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com> wrote:
>>
>>> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org> wrote:
>>>
>>>> Hi everybody,
>>>>
>>>> TL;DR We would like users of ORES models to migrate to our new open
>>>> source ML infrastructure, Lift Wing, within the next five months. We are
>>>> available to help you do that, from advice to making code commits. It is
>>>> important to note: All ML models currently accessible on ORES are also
>>>> currently accessible on Lift Wing.
>>>>
>>>> As part of the Machine Learning Modernization Project (
>>>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization), the
>>>> Machine Learning team has deployed a Wikimedia’s new machine learning
>>>> inference infrastructure, called Lift Wing (
>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing). Lift
>>>> Wing brings a lot of new features such as support for GPU-based models,
>>>> open source LLM hosting, auto-scaling, stability, and ability to host a
>>>> larger number of models.
>>>>
>>>
>>> This sounds quite exciting! What's the best place to read up on that
>>> planned support for GPU-based models and open source LLMs? (I also saw in
>>> the recent NYT article[1] that the team is "in the process of adapting A.I.
>>> models that are 'off the shelf; — essentially models that have been made
>>> available by researchers for anyone to freely customize — so that
>>> Wikipedia’s editors can use them for their work.")
>>>
>>> I'm aware of the history[2] of not being able to use NVIDIA GPUs due to
>>> their CUDA drivers being proprietary. It was mentioned recently in the
>>> Wikimedia AI Telegram group that this is still a serious limitation,
>>> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
>>> WMF's Language team has resorted to using models without GPU support (CPU
>>> only).[4]
>>> It sounds like there is reasonable hope that this situation could change
>>> fairly soon? Would it also mean both at the same time, i.e. open source
>>> LLMs running with GPU support (considering that at least some
>>> well-known ones appear to require torch.cuda.is_available() == True for
>>> that)?
>>>
>>> Regards, Tilman
>>>
>>> [1]
>>> https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
>>> [2]
>>> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
>>> [3] https://phabricator.wikimedia.org/T334583 etc.
>>> [4]
>>> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
>>> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental but, I
>>> understand, written to be deployable on WMF infrastructure)
>>>
>>>
>>>>
>>>> With the creation of Lift Wing, the team is turning its attention to
>>>> deprecating the current machine learning infrastructure, ORES. ORES served
>>>> us really well over the years, it was a successful project but it came
>>>> before radical changes in technology like Docker, Kubernetes and more
>>>> recently MLOps. The servers that run ORES are at the end of their planned
>>>> lifespan and so to save cost we are going to shut them down in early 2024.
>>>>
>>>> We have outlined a deprecation path on Wikitech (
>>>> https://wikitech.wikimedia.org/wiki/ORES), please read the page if you
>>>> are a maintainer of a tool or code that uses the ORES endpoint
>>>> https://ores.wikimedia.org/). If you have any doubt or if you need
>>>> assistance in migrating to Lift Wing, feel free to contact the ML team via:
>>>>
>>>> - Email: ml@wikimedia.org
>>>> - Phabricator: #Machine-Learning-Team tag
>>>> - IRC (Libera): #wikimedia-ml
>>>>
>>>> The Machine Learning team is available to help projects migrate, from
>>>> offering advice to making code commits. We want to make this as easy as
>>>> possible for folks.
>>>>
>>>> High Level timeline:
>>>>
>>>> **By September 30th 2023: *Infrastructure powering the ORES API
>>>> endpoint will be migrated from ORES to Lift Wing. For users, the API
>>>> endpoint will remain the same, and most users won’t notice any change.
>>>> Rather just the backend services powering the endpoint will change.
>>>>
>>>> Details: We'd like to add a DNS CNAME that points ores.wikimedia.org
>>>> to ores-legacy.wikimedia.org, a new endpoint that offers a almost
>>>> complete replacement of the ORES API calling Lift Wing behind the scenes.
>>>> In an ideal world we'd migrate all tools to Lift Wing before
>>>> decommissioning the infrastructure behind ores.wikimedia.org, but it
>>>> turned out to be really challenging so to avoid disrupting users we chose
>>>> to implement a transition layer/API.
>>>>
>>>> To summarize, if you don't have time to migrate before September to
>>>> Lift Wing, your code/tool should work just fine on
>>>> ores-legacy.wikimedia.org and you'll not have to change a line in your
>>>> code thanks to the DNS CNAME. The ores-legacy endpoint is not a 100%
>>>> replacement for ores, we removed some very old and not used features, so we
>>>> highly recommend at least test the new endpoint for your use case to avoid
>>>> surprises when we'll make the switch. In case you find anything weird,
>>>> please report it to us using the aforementioned channels.
>>>>
>>>> **September to January: *We will be reaching out to every user of ORES
>>>> we can identify and working with them to make the migration process as easy
>>>> as possible.
>>>>
>>>> **By January 2024: *If all goes well, we would like zero traffic on
>>>> the ORES API endpoint so we can turn off the ores-legacy API.
>>>>
>>>> If you want more information about Lift Wing, please check
>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>>>
>>>> Thanks in advance for the patience and the help!
>>>>
>>>> Regards,
>>>>
>>>> The Machine Learning Team
>>>> _______________________________________________
>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>
>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>
>>> _______________________________________________
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> _______________________________________________
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

Hello!


As a next step in the deprecation process of ORES
https://wikitech.wikimedia.org/wiki/ORES the Machine Learning team will
switch the backend of ores.wikimedia.org to ores-legacy, a k8s application
meant to provide a compatibility layer between ORES and Lift Wing so users
that have not yet migrated to Lift Wing will be transparently migrated.
Ores-legacy is an application that has the same API as ORES but in the
background makes requests to Lift Wing, allowing us to decommission the
ORES servers until all clients have moved.

This change is planned to take place on Monday 25th of September. If you
have a client/application that is still using ORES we expect that this
switch is going to be transparent for you.

However keep in mind that ores-legacy is not a 100% replacement for ORES as
some old and unused features are no longer supported.

If you see anything out of the ordinary, feel free to contact the Machine
Learning team:

IRC libera: #wikimedia-ml

Phabricator: Machine-Learning-team tag

Thank you!


On Wed, Aug 9, 2023 at 1:22?PM Chaloemphon Praphuchakang <
yoshrakpraphu@gmail.com> wrote:

>
> On Tue, 8 Aug 2023, 10:45 Tilman Bayer, <haebwiki@gmail.com> wrote:
>
>>
>> Hi Chris,
>>
>> On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org> wrote:
>>
>>> Hi Tilman,
>>>
>>> Most of the work is still very experimental. We have hosted a few LLMs
>>> on Lift Wing already (StarCoder for example) but they were just running on
>>> CPU, far too slow for real use cases. But it proves that we can easily host
>>> LLMs on Lift Wing. We have been pretty quiet about it while we focus on the
>>> ORES migration, but it is our next big project. More soon hopefully!
>>>
>> Understood. Looking forward to learning more later!
>>
>>
>>> Where we are now is that we have budget for a big GPU purchase (~10-20
>>> GPUs depending on cost), the question we will try to answer after the ORES
>>> migration is complete is: what GPUs should we purchase? We are trying to
>>> balance our strong preference to stay open source (i.e. AMD mROC) in a
>>> world dominated by a single closed source vendor (i.e. Nvidia). In
>>> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
>>> H100, etc) or a mix of big and small? We will need to figure out all this.
>>>
>> I see. On that matter, what do you folks make of the recent announcements
>> of AMD's partnerships with Hugging Face and Pytorch[5]? (which, I
>> understand, came after the ML team had already launched the aforementioned
>> new AMD explorations)
>>
>> "Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch to
>> take on Nvidia [...]
>> Both partnerships involve AMD’s ROCm AI software stack, the company’s
>> answer to Nvidia’s proprietary CUDA platform and application-programming
>> interface. AMD called ROCm an open and portable AI system with
>> out-of-the-box support that can port to existing AI models. [...B]oth AMD
>> and Hugging Face are dedicating engineering resources to each other and
>> sharing data to ensure that the constantly updated AI models from Hugging
>> Face, which might not otherwise run well on AMD hardware, would be
>> “guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
>> will fully upstream the ROCm software stack and “provide immediate ‘day
>> zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
>> accelerators,” which is meant to appeal to those customers looking to
>> switch from Nvidia’s software ecosystem."
>>
>>
>> In their own announcement, Hugging Face offered further details,
>> including a pretty impressive list of models to be supported:[6]
>>
>>
>> "We intend to support state-of-the-art transformer architectures for
>> natural language processing, computer vision, and speech, such as BERT,
>> DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
>> generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
>> LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
>> support more traditional computer vision models, like ResNet and ResNext,
>> and deep learning recommendation models, a first for us. [..] We'll do our
>> best to test and validate these models for PyTorch, TensorFlow, and ONNX
>> Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
>> seamlessly in our open-source libraries, starting with the transformers
>> library."
>>
>>
>> Do you think this may promise too much, or could it point to a possible
>> solution of the Foundation's conundrum?
>> In any case, this seems to be an interesting moment where many in AI are
>> trying to move away from Nvidia's proprietary CUDA platform. Most of them
>> probably more for financial and availability reasons though, given the
>> current GPU shortages[7] (which the ML team is undoubtedly aware of
>> already; mentioning this as context for others on this list. See also
>> Marketwatch's remarks about current margins[5]).
>>
>> Regards, Tilman
>>
>>
>> [5]
>> https://archive.ph/2023.06.15-173527/https://www.marketwatch.com/amp/story/open-source-ai-amd-looks-to-hugging-face-and-meta-spinoff-pytorch-to-take-on-nvidia-e4738f87
>> [6] https://huggingface.co/blog/huggingface-and-amd
>> [7] See e.g.
>> https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/ (avoid
>> playing the song though. Don't say I didn't warn you)
>>
>>
>>> I wouldn't characterize WMF's Language Team using CPU as because of AMD,
>>> rather at the time we didn't have the budget for GPUs so Lift Wing didn't
>>> have any. Since then we have moved two GPUs onto Lift Wing for testing but
>>> they are pretty old (2017ish). Once we make the big GPU purchase Lift Wing
>>> will gain a lot of functionality for LLM and similar models.
>>>
>>> Chris
>>>
>>> On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com> wrote:
>>>
>>>> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org>
>>>> wrote:
>>>>
>>>>> Hi everybody,
>>>>>
>>>>> TL;DR We would like users of ORES models to migrate to our new open
>>>>> source ML infrastructure, Lift Wing, within the next five months. We are
>>>>> available to help you do that, from advice to making code commits. It is
>>>>> important to note: All ML models currently accessible on ORES are also
>>>>> currently accessible on Lift Wing.
>>>>>
>>>>> As part of the Machine Learning Modernization Project (
>>>>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization), the
>>>>> Machine Learning team has deployed a Wikimedia’s new machine learning
>>>>> inference infrastructure, called Lift Wing (
>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing). Lift
>>>>> Wing brings a lot of new features such as support for GPU-based models,
>>>>> open source LLM hosting, auto-scaling, stability, and ability to host a
>>>>> larger number of models.
>>>>>
>>>>
>>>> This sounds quite exciting! What's the best place to read up on that
>>>> planned support for GPU-based models and open source LLMs? (I also saw in
>>>> the recent NYT article[1] that the team is "in the process of adapting A.I.
>>>> models that are 'off the shelf; — essentially models that have been made
>>>> available by researchers for anyone to freely customize — so that
>>>> Wikipedia’s editors can use them for their work.")
>>>>
>>>> I'm aware of the history[2] of not being able to use NVIDIA GPUs due to
>>>> their CUDA drivers being proprietary. It was mentioned recently in the
>>>> Wikimedia AI Telegram group that this is still a serious limitation,
>>>> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
>>>> WMF's Language team has resorted to using models without GPU support (CPU
>>>> only).[4]
>>>> It sounds like there is reasonable hope that this situation could
>>>> change fairly soon? Would it also mean both at the same time, i.e. open
>>>> source LLMs running with GPU support (considering that at least some
>>>> well-known ones appear to require torch.cuda.is_available() == True for
>>>> that)?
>>>>
>>>> Regards, Tilman
>>>>
>>>> [1]
>>>> https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
>>>> [2]
>>>> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
>>>> [3] https://phabricator.wikimedia.org/T334583 etc.
>>>> [4]
>>>> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
>>>> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental but, I
>>>> understand, written to be deployable on WMF infrastructure)
>>>>
>>>>
>>>>>
>>>>> With the creation of Lift Wing, the team is turning its attention to
>>>>> deprecating the current machine learning infrastructure, ORES. ORES served
>>>>> us really well over the years, it was a successful project but it came
>>>>> before radical changes in technology like Docker, Kubernetes and more
>>>>> recently MLOps. The servers that run ORES are at the end of their planned
>>>>> lifespan and so to save cost we are going to shut them down in early 2024.
>>>>>
>>>>> We have outlined a deprecation path on Wikitech (
>>>>> https://wikitech.wikimedia.org/wiki/ORES), please read the page if
>>>>> you are a maintainer of a tool or code that uses the ORES endpoint
>>>>> https://ores.wikimedia.org/). If you have any doubt or if you need
>>>>> assistance in migrating to Lift Wing, feel free to contact the ML team via:
>>>>>
>>>>> - Email: ml@wikimedia.org
>>>>> - Phabricator: #Machine-Learning-Team tag
>>>>> - IRC (Libera): #wikimedia-ml
>>>>>
>>>>> The Machine Learning team is available to help projects migrate, from
>>>>> offering advice to making code commits. We want to make this as easy as
>>>>> possible for folks.
>>>>>
>>>>> High Level timeline:
>>>>>
>>>>> **By September 30th 2023: *Infrastructure powering the ORES API
>>>>> endpoint will be migrated from ORES to Lift Wing. For users, the API
>>>>> endpoint will remain the same, and most users won’t notice any change.
>>>>> Rather just the backend services powering the endpoint will change.
>>>>>
>>>>> Details: We'd like to add a DNS CNAME that points ores.wikimedia.org
>>>>> to ores-legacy.wikimedia.org, a new endpoint that offers a almost
>>>>> complete replacement of the ORES API calling Lift Wing behind the scenes.
>>>>> In an ideal world we'd migrate all tools to Lift Wing before
>>>>> decommissioning the infrastructure behind ores.wikimedia.org, but it
>>>>> turned out to be really challenging so to avoid disrupting users we chose
>>>>> to implement a transition layer/API.
>>>>>
>>>>> To summarize, if you don't have time to migrate before September to
>>>>> Lift Wing, your code/tool should work just fine on
>>>>> ores-legacy.wikimedia.org and you'll not have to change a line in
>>>>> your code thanks to the DNS CNAME. The ores-legacy endpoint is not a 100%
>>>>> replacement for ores, we removed some very old and not used features, so we
>>>>> highly recommend at least test the new endpoint for your use case to avoid
>>>>> surprises when we'll make the switch. In case you find anything weird,
>>>>> please report it to us using the aforementioned channels.
>>>>>
>>>>> **September to January: *We will be reaching out to every user of
>>>>> ORES we can identify and working with them to make the migration process as
>>>>> easy as possible.
>>>>>
>>>>> **By January 2024: *If all goes well, we would like zero traffic on
>>>>> the ORES API endpoint so we can turn off the ores-legacy API.
>>>>>
>>>>> If you want more information about Lift Wing, please check
>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>>>>
>>>>> Thanks in advance for the patience and the help!
>>>>>
>>>>> Regards,
>>>>>
>>>>> The Machine Learning Team
>>>>> _______________________________________________
>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>
>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>
>>>> _______________________________________________
>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>
>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>
>>> _______________________________________________
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> _______________________________________________
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

Does the new ores-legacy support the same feature set. E.g. features
output, injection, and threshold optimizations. Or is it just prediction?
This will affect some of the systems I need to migrate.

On Fri, Sep 22, 2023, 06:21 Ilias Sarantopoulos <
isarantopoulos@wikimedia.org> wrote:

> Hello!
>
>
> As a next step in the deprecation process of ORES
> https://wikitech.wikimedia.org/wiki/ORES the Machine Learning team will
> switch the backend of ores.wikimedia.org to ores-legacy, a k8s
> application meant to provide a compatibility layer between ORES and Lift
> Wing so users that have not yet migrated to Lift Wing will be
> transparently migrated. Ores-legacy is an application that has the same API
> as ORES but in the background makes requests to Lift Wing, allowing us to
> decommission the ORES servers until all clients have moved.
>
> This change is planned to take place on Monday 25th of September. If you
> have a client/application that is still using ORES we expect that this
> switch is going to be transparent for you.
>
> However keep in mind that ores-legacy is not a 100% replacement for ORES
> as some old and unused features are no longer supported.
>
> If you see anything out of the ordinary, feel free to contact the Machine
> Learning team:
>
> IRC libera: #wikimedia-ml
>
> Phabricator: Machine-Learning-team tag
>
> Thank you!
>
>
> On Wed, Aug 9, 2023 at 1:22?PM Chaloemphon Praphuchakang <
> yoshrakpraphu@gmail.com> wrote:
>
>>
>> On Tue, 8 Aug 2023, 10:45 Tilman Bayer, <haebwiki@gmail.com> wrote:
>>
>>>
>>> Hi Chris,
>>>
>>> On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org>
>>> wrote:
>>>
>>>> Hi Tilman,
>>>>
>>>> Most of the work is still very experimental. We have hosted a few LLMs
>>>> on Lift Wing already (StarCoder for example) but they were just running on
>>>> CPU, far too slow for real use cases. But it proves that we can easily host
>>>> LLMs on Lift Wing. We have been pretty quiet about it while we focus on the
>>>> ORES migration, but it is our next big project. More soon hopefully!
>>>>
>>> Understood. Looking forward to learning more later!
>>>
>>>
>>>> Where we are now is that we have budget for a big GPU purchase (~10-20
>>>> GPUs depending on cost), the question we will try to answer after the ORES
>>>> migration is complete is: what GPUs should we purchase? We are trying to
>>>> balance our strong preference to stay open source (i.e. AMD mROC) in a
>>>> world dominated by a single closed source vendor (i.e. Nvidia). In
>>>> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
>>>> H100, etc) or a mix of big and small? We will need to figure out all this.
>>>>
>>> I see. On that matter, what do you folks make of the recent
>>> announcements of AMD's partnerships with Hugging Face and Pytorch[5]?
>>> (which, I understand, came after the ML team had already launched the
>>> aforementioned new AMD explorations)
>>>
>>> "Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch to
>>> take on Nvidia [...]
>>> Both partnerships involve AMD’s ROCm AI software stack, the company’s
>>> answer to Nvidia’s proprietary CUDA platform and application-programming
>>> interface. AMD called ROCm an open and portable AI system with
>>> out-of-the-box support that can port to existing AI models. [...B]oth AMD
>>> and Hugging Face are dedicating engineering resources to each other and
>>> sharing data to ensure that the constantly updated AI models from Hugging
>>> Face, which might not otherwise run well on AMD hardware, would be
>>> “guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
>>> will fully upstream the ROCm software stack and “provide immediate ‘day
>>> zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
>>> accelerators,” which is meant to appeal to those customers looking to
>>> switch from Nvidia’s software ecosystem."
>>>
>>>
>>> In their own announcement, Hugging Face offered further details,
>>> including a pretty impressive list of models to be supported:[6]
>>>
>>>
>>> "We intend to support state-of-the-art transformer architectures for
>>> natural language processing, computer vision, and speech, such as BERT,
>>> DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
>>> generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
>>> LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
>>> support more traditional computer vision models, like ResNet and ResNext,
>>> and deep learning recommendation models, a first for us. [..] We'll do our
>>> best to test and validate these models for PyTorch, TensorFlow, and ONNX
>>> Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
>>> seamlessly in our open-source libraries, starting with the transformers
>>> library."
>>>
>>>
>>> Do you think this may promise too much, or could it point to a possible
>>> solution of the Foundation's conundrum?
>>> In any case, this seems to be an interesting moment where many in AI are
>>> trying to move away from Nvidia's proprietary CUDA platform. Most of them
>>> probably more for financial and availability reasons though, given the
>>> current GPU shortages[7] (which the ML team is undoubtedly aware of
>>> already; mentioning this as context for others on this list. See also
>>> Marketwatch's remarks about current margins[5]).
>>>
>>> Regards, Tilman
>>>
>>>
>>> [5]
>>> https://archive.ph/2023.06.15-173527/https://www.marketwatch.com/amp/story/open-source-ai-amd-looks-to-hugging-face-and-meta-spinoff-pytorch-to-take-on-nvidia-e4738f87
>>> [6] https://huggingface.co/blog/huggingface-and-amd
>>> [7] See e.g.
>>> https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/ (avoid
>>> playing the song though. Don't say I didn't warn you)
>>>
>>>
>>>> I wouldn't characterize WMF's Language Team using CPU as because of
>>>> AMD, rather at the time we didn't have the budget for GPUs so Lift Wing
>>>> didn't have any. Since then we have moved two GPUs onto Lift Wing for
>>>> testing but they are pretty old (2017ish). Once we make the big GPU
>>>> purchase Lift Wing will gain a lot of functionality for LLM and similar
>>>> models.
>>>>
>>>> Chris
>>>>
>>>> On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com> wrote:
>>>>
>>>>> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org>
>>>>> wrote:
>>>>>
>>>>>> Hi everybody,
>>>>>>
>>>>>> TL;DR We would like users of ORES models to migrate to our new open
>>>>>> source ML infrastructure, Lift Wing, within the next five months. We are
>>>>>> available to help you do that, from advice to making code commits. It is
>>>>>> important to note: All ML models currently accessible on ORES are also
>>>>>> currently accessible on Lift Wing.
>>>>>>
>>>>>> As part of the Machine Learning Modernization Project (
>>>>>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization), the
>>>>>> Machine Learning team has deployed a Wikimedia’s new machine learning
>>>>>> inference infrastructure, called Lift Wing (
>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing). Lift
>>>>>> Wing brings a lot of new features such as support for GPU-based models,
>>>>>> open source LLM hosting, auto-scaling, stability, and ability to host a
>>>>>> larger number of models.
>>>>>>
>>>>>
>>>>> This sounds quite exciting! What's the best place to read up on that
>>>>> planned support for GPU-based models and open source LLMs? (I also saw in
>>>>> the recent NYT article[1] that the team is "in the process of adapting A.I.
>>>>> models that are 'off the shelf; — essentially models that have been made
>>>>> available by researchers for anyone to freely customize — so that
>>>>> Wikipedia’s editors can use them for their work.")
>>>>>
>>>>> I'm aware of the history[2] of not being able to use NVIDIA GPUs due
>>>>> to their CUDA drivers being proprietary. It was mentioned recently in the
>>>>> Wikimedia AI Telegram group that this is still a serious limitation,
>>>>> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
>>>>> WMF's Language team has resorted to using models without GPU support (CPU
>>>>> only).[4]
>>>>> It sounds like there is reasonable hope that this situation could
>>>>> change fairly soon? Would it also mean both at the same time, i.e. open
>>>>> source LLMs running with GPU support (considering that at least some
>>>>> well-known ones appear to require torch.cuda.is_available() == True for
>>>>> that)?
>>>>>
>>>>> Regards, Tilman
>>>>>
>>>>> [1]
>>>>> https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
>>>>> [2]
>>>>> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
>>>>> [3] https://phabricator.wikimedia.org/T334583 etc.
>>>>> [4]
>>>>> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
>>>>> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental but, I
>>>>> understand, written to be deployable on WMF infrastructure)
>>>>>
>>>>>
>>>>>>
>>>>>> With the creation of Lift Wing, the team is turning its attention to
>>>>>> deprecating the current machine learning infrastructure, ORES. ORES served
>>>>>> us really well over the years, it was a successful project but it came
>>>>>> before radical changes in technology like Docker, Kubernetes and more
>>>>>> recently MLOps. The servers that run ORES are at the end of their planned
>>>>>> lifespan and so to save cost we are going to shut them down in early 2024.
>>>>>>
>>>>>> We have outlined a deprecation path on Wikitech (
>>>>>> https://wikitech.wikimedia.org/wiki/ORES), please read the page if
>>>>>> you are a maintainer of a tool or code that uses the ORES endpoint
>>>>>> https://ores.wikimedia.org/). If you have any doubt or if you need
>>>>>> assistance in migrating to Lift Wing, feel free to contact the ML team via:
>>>>>>
>>>>>> - Email: ml@wikimedia.org
>>>>>> - Phabricator: #Machine-Learning-Team tag
>>>>>> - IRC (Libera): #wikimedia-ml
>>>>>>
>>>>>> The Machine Learning team is available to help projects migrate, from
>>>>>> offering advice to making code commits. We want to make this as easy as
>>>>>> possible for folks.
>>>>>>
>>>>>> High Level timeline:
>>>>>>
>>>>>> **By September 30th 2023: *Infrastructure powering the ORES API
>>>>>> endpoint will be migrated from ORES to Lift Wing. For users, the API
>>>>>> endpoint will remain the same, and most users won’t notice any change.
>>>>>> Rather just the backend services powering the endpoint will change.
>>>>>>
>>>>>> Details: We'd like to add a DNS CNAME that points ores.wikimedia.org
>>>>>> to ores-legacy.wikimedia.org, a new endpoint that offers a almost
>>>>>> complete replacement of the ORES API calling Lift Wing behind the scenes.
>>>>>> In an ideal world we'd migrate all tools to Lift Wing before
>>>>>> decommissioning the infrastructure behind ores.wikimedia.org, but it
>>>>>> turned out to be really challenging so to avoid disrupting users we chose
>>>>>> to implement a transition layer/API.
>>>>>>
>>>>>> To summarize, if you don't have time to migrate before September to
>>>>>> Lift Wing, your code/tool should work just fine on
>>>>>> ores-legacy.wikimedia.org and you'll not have to change a line in
>>>>>> your code thanks to the DNS CNAME. The ores-legacy endpoint is not a 100%
>>>>>> replacement for ores, we removed some very old and not used features, so we
>>>>>> highly recommend at least test the new endpoint for your use case to avoid
>>>>>> surprises when we'll make the switch. In case you find anything weird,
>>>>>> please report it to us using the aforementioned channels.
>>>>>>
>>>>>> **September to January: *We will be reaching out to every user of
>>>>>> ORES we can identify and working with them to make the migration process as
>>>>>> easy as possible.
>>>>>>
>>>>>> **By January 2024: *If all goes well, we would like zero traffic on
>>>>>> the ORES API endpoint so we can turn off the ores-legacy API.
>>>>>>
>>>>>> If you want more information about Lift Wing, please check
>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>>>>>
>>>>>> Thanks in advance for the patience and the help!
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> The Machine Learning Team
>>>>>> _______________________________________________
>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>
>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>
>>>>> _______________________________________________
>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>
>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>
>>>> _______________________________________________
>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>
>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>
>>> _______________________________________________
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> _______________________________________________
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

Hi Aaron!

Thanks for following up. The API is almost compatible with what ORES
currently does, but there are limitations (like the max number of revisions
in a batch etc..). The API clearly states when something is not supported,
so you can check its compatibility now making some requests to:

https://ores-legacy.wikimedia.org

If you open a task with a list of systems that you need to migrate we can
definitely take a look and help. So far the traffic being served by ORES
has been reduced to few clients, and all of them don't run with
recognizable UAs (see https://meta.wikimedia.org/wiki/User-Agent_policy) so
we'll try our best to support them. The migration to Lift Wing has been
widely publicized, a lot of documentation is available to migrate. We'd
suggest trying Lift Wing for your systems instead (see
https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Usage).

The Machine Learning plan is to eventually deprecate ores-legacy too, to
maintain only one system (namely Lift Wing). There is no final date yet,
we'll try to reach out to all remaining users first, so if you plan to keep
using ores-legacy please follow up with us first :)

Thanks!

Luca (on behalf of the ML Team)

On Fri, Sep 22, 2023 at 5:10?PM Aaron Halfaker <aaron.halfaker@gmail.com>
wrote:

> Does the new ores-legacy support the same feature set. E.g. features
> output, injection, and threshold optimizations. Or is it just prediction?
> This will affect some of the systems I need to migrate.
>
> On Fri, Sep 22, 2023, 06:21 Ilias Sarantopoulos <
> isarantopoulos@wikimedia.org> wrote:
>
>> Hello!
>>
>>
>> As a next step in the deprecation process of ORES
>> https://wikitech.wikimedia.org/wiki/ORES the Machine Learning team will
>> switch the backend of ores.wikimedia.org to ores-legacy, a k8s
>> application meant to provide a compatibility layer between ORES and Lift
>> Wing so users that have not yet migrated to Lift Wing will be
>> transparently migrated. Ores-legacy is an application that has the same API
>> as ORES but in the background makes requests to Lift Wing, allowing us to
>> decommission the ORES servers until all clients have moved.
>>
>> This change is planned to take place on Monday 25th of September. If you
>> have a client/application that is still using ORES we expect that this
>> switch is going to be transparent for you.
>>
>> However keep in mind that ores-legacy is not a 100% replacement for ORES
>> as some old and unused features are no longer supported.
>>
>> If you see anything out of the ordinary, feel free to contact the Machine
>> Learning team:
>>
>> IRC libera: #wikimedia-ml
>>
>> Phabricator: Machine-Learning-team tag
>>
>> Thank you!
>>
>>
>> On Wed, Aug 9, 2023 at 1:22?PM Chaloemphon Praphuchakang <
>> yoshrakpraphu@gmail.com> wrote:
>>
>>>
>>> On Tue, 8 Aug 2023, 10:45 Tilman Bayer, <haebwiki@gmail.com> wrote:
>>>
>>>>
>>>> Hi Chris,
>>>>
>>>> On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org>
>>>> wrote:
>>>>
>>>>> Hi Tilman,
>>>>>
>>>>> Most of the work is still very experimental. We have hosted a few LLMs
>>>>> on Lift Wing already (StarCoder for example) but they were just running on
>>>>> CPU, far too slow for real use cases. But it proves that we can easily host
>>>>> LLMs on Lift Wing. We have been pretty quiet about it while we focus on the
>>>>> ORES migration, but it is our next big project. More soon hopefully!
>>>>>
>>>> Understood. Looking forward to learning more later!
>>>>
>>>>
>>>>> Where we are now is that we have budget for a big GPU purchase (~10-20
>>>>> GPUs depending on cost), the question we will try to answer after the ORES
>>>>> migration is complete is: what GPUs should we purchase? We are trying to
>>>>> balance our strong preference to stay open source (i.e. AMD mROC) in a
>>>>> world dominated by a single closed source vendor (i.e. Nvidia). In
>>>>> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
>>>>> H100, etc) or a mix of big and small? We will need to figure out all this.
>>>>>
>>>> I see. On that matter, what do you folks make of the recent
>>>> announcements of AMD's partnerships with Hugging Face and Pytorch[5]?
>>>> (which, I understand, came after the ML team had already launched the
>>>> aforementioned new AMD explorations)
>>>>
>>>> "Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch to
>>>> take on Nvidia [...]
>>>> Both partnerships involve AMD’s ROCm AI software stack, the company’s
>>>> answer to Nvidia’s proprietary CUDA platform and application-programming
>>>> interface. AMD called ROCm an open and portable AI system with
>>>> out-of-the-box support that can port to existing AI models. [...B]oth AMD
>>>> and Hugging Face are dedicating engineering resources to each other and
>>>> sharing data to ensure that the constantly updated AI models from Hugging
>>>> Face, which might not otherwise run well on AMD hardware, would be
>>>> “guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
>>>> will fully upstream the ROCm software stack and “provide immediate ‘day
>>>> zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
>>>> accelerators,” which is meant to appeal to those customers looking to
>>>> switch from Nvidia’s software ecosystem."
>>>>
>>>>
>>>> In their own announcement, Hugging Face offered further details,
>>>> including a pretty impressive list of models to be supported:[6]
>>>>
>>>>
>>>> "We intend to support state-of-the-art transformer architectures for
>>>> natural language processing, computer vision, and speech, such as BERT,
>>>> DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
>>>> generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
>>>> LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
>>>> support more traditional computer vision models, like ResNet and ResNext,
>>>> and deep learning recommendation models, a first for us. [..] We'll do our
>>>> best to test and validate these models for PyTorch, TensorFlow, and ONNX
>>>> Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
>>>> seamlessly in our open-source libraries, starting with the transformers
>>>> library."
>>>>
>>>>
>>>> Do you think this may promise too much, or could it point to a possible
>>>> solution of the Foundation's conundrum?
>>>> In any case, this seems to be an interesting moment where many in AI
>>>> are trying to move away from Nvidia's proprietary CUDA platform. Most of
>>>> them probably more for financial and availability reasons though, given the
>>>> current GPU shortages[7] (which the ML team is undoubtedly aware of
>>>> already; mentioning this as context for others on this list. See also
>>>> Marketwatch's remarks about current margins[5]).
>>>>
>>>> Regards, Tilman
>>>>
>>>>
>>>> [5]
>>>> https://archive.ph/2023.06.15-173527/https://www.marketwatch.com/amp/story/open-source-ai-amd-looks-to-hugging-face-and-meta-spinoff-pytorch-to-take-on-nvidia-e4738f87
>>>> [6] https://huggingface.co/blog/huggingface-and-amd
>>>> [7] See e.g.
>>>> https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/ (avoid
>>>> playing the song though. Don't say I didn't warn you)
>>>>
>>>>
>>>>> I wouldn't characterize WMF's Language Team using CPU as because of
>>>>> AMD, rather at the time we didn't have the budget for GPUs so Lift Wing
>>>>> didn't have any. Since then we have moved two GPUs onto Lift Wing for
>>>>> testing but they are pretty old (2017ish). Once we make the big GPU
>>>>> purchase Lift Wing will gain a lot of functionality for LLM and similar
>>>>> models.
>>>>>
>>>>> Chris
>>>>>
>>>>> On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi everybody,
>>>>>>>
>>>>>>> TL;DR We would like users of ORES models to migrate to our new open
>>>>>>> source ML infrastructure, Lift Wing, within the next five months. We are
>>>>>>> available to help you do that, from advice to making code commits. It is
>>>>>>> important to note: All ML models currently accessible on ORES are also
>>>>>>> currently accessible on Lift Wing.
>>>>>>>
>>>>>>> As part of the Machine Learning Modernization Project (
>>>>>>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization), the
>>>>>>> Machine Learning team has deployed a Wikimedia’s new machine learning
>>>>>>> inference infrastructure, called Lift Wing (
>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing).
>>>>>>> Lift Wing brings a lot of new features such as support for GPU-based
>>>>>>> models, open source LLM hosting, auto-scaling, stability, and ability to
>>>>>>> host a larger number of models.
>>>>>>>
>>>>>>
>>>>>> This sounds quite exciting! What's the best place to read up on that
>>>>>> planned support for GPU-based models and open source LLMs? (I also saw in
>>>>>> the recent NYT article[1] that the team is "in the process of adapting A.I.
>>>>>> models that are 'off the shelf; — essentially models that have been made
>>>>>> available by researchers for anyone to freely customize — so that
>>>>>> Wikipedia’s editors can use them for their work.")
>>>>>>
>>>>>> I'm aware of the history[2] of not being able to use NVIDIA GPUs due
>>>>>> to their CUDA drivers being proprietary. It was mentioned recently in the
>>>>>> Wikimedia AI Telegram group that this is still a serious limitation,
>>>>>> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
>>>>>> WMF's Language team has resorted to using models without GPU support (CPU
>>>>>> only).[4]
>>>>>> It sounds like there is reasonable hope that this situation could
>>>>>> change fairly soon? Would it also mean both at the same time, i.e. open
>>>>>> source LLMs running with GPU support (considering that at least some
>>>>>> well-known ones appear to require torch.cuda.is_available() == True for
>>>>>> that)?
>>>>>>
>>>>>> Regards, Tilman
>>>>>>
>>>>>> [1]
>>>>>> https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
>>>>>> [2]
>>>>>> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
>>>>>> [3] https://phabricator.wikimedia.org/T334583 etc.
>>>>>> [4]
>>>>>> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
>>>>>> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental but,
>>>>>> I understand, written to be deployable on WMF infrastructure)
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> With the creation of Lift Wing, the team is turning its attention to
>>>>>>> deprecating the current machine learning infrastructure, ORES. ORES served
>>>>>>> us really well over the years, it was a successful project but it came
>>>>>>> before radical changes in technology like Docker, Kubernetes and more
>>>>>>> recently MLOps. The servers that run ORES are at the end of their planned
>>>>>>> lifespan and so to save cost we are going to shut them down in early 2024.
>>>>>>>
>>>>>>> We have outlined a deprecation path on Wikitech (
>>>>>>> https://wikitech.wikimedia.org/wiki/ORES), please read the page if
>>>>>>> you are a maintainer of a tool or code that uses the ORES endpoint
>>>>>>> https://ores.wikimedia.org/). If you have any doubt or if you need
>>>>>>> assistance in migrating to Lift Wing, feel free to contact the ML team via:
>>>>>>>
>>>>>>> - Email: ml@wikimedia.org
>>>>>>> - Phabricator: #Machine-Learning-Team tag
>>>>>>> - IRC (Libera): #wikimedia-ml
>>>>>>>
>>>>>>> The Machine Learning team is available to help projects migrate,
>>>>>>> from offering advice to making code commits. We want to make this as easy
>>>>>>> as possible for folks.
>>>>>>>
>>>>>>> High Level timeline:
>>>>>>>
>>>>>>> **By September 30th 2023: *Infrastructure powering the ORES API
>>>>>>> endpoint will be migrated from ORES to Lift Wing. For users, the API
>>>>>>> endpoint will remain the same, and most users won’t notice any change.
>>>>>>> Rather just the backend services powering the endpoint will change.
>>>>>>>
>>>>>>> Details: We'd like to add a DNS CNAME that points ores.wikimedia.org
>>>>>>> to ores-legacy.wikimedia.org, a new endpoint that offers a almost
>>>>>>> complete replacement of the ORES API calling Lift Wing behind the scenes.
>>>>>>> In an ideal world we'd migrate all tools to Lift Wing before
>>>>>>> decommissioning the infrastructure behind ores.wikimedia.org, but
>>>>>>> it turned out to be really challenging so to avoid disrupting users we
>>>>>>> chose to implement a transition layer/API.
>>>>>>>
>>>>>>> To summarize, if you don't have time to migrate before September to
>>>>>>> Lift Wing, your code/tool should work just fine on
>>>>>>> ores-legacy.wikimedia.org and you'll not have to change a line in
>>>>>>> your code thanks to the DNS CNAME. The ores-legacy endpoint is not a 100%
>>>>>>> replacement for ores, we removed some very old and not used features, so we
>>>>>>> highly recommend at least test the new endpoint for your use case to avoid
>>>>>>> surprises when we'll make the switch. In case you find anything weird,
>>>>>>> please report it to us using the aforementioned channels.
>>>>>>>
>>>>>>> **September to January: *We will be reaching out to every user of
>>>>>>> ORES we can identify and working with them to make the migration process as
>>>>>>> easy as possible.
>>>>>>>
>>>>>>> **By January 2024: *If all goes well, we would like zero traffic on
>>>>>>> the ORES API endpoint so we can turn off the ores-legacy API.
>>>>>>>
>>>>>>> If you want more information about Lift Wing, please check
>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>>>>>>
>>>>>>> Thanks in advance for the patience and the help!
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> The Machine Learning Team
>>>>>>> _______________________________________________
>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>>
>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>
>>>>>> _______________________________________________
>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>
>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>
>>>>> _______________________________________________
>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>
>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>
>>>> _______________________________________________
>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>
>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>
>>> _______________________________________________
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> _______________________________________________
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

Do you have a tag for filing bugs against ORES-legacy? I can't seem to
find a relevant one in phab.

On Fri, Sep 22, 2023 at 8:39?AM Luca Toscano <ltoscano@wikimedia.org> wrote:

> Hi Aaron!
>
> Thanks for following up. The API is almost compatible with what ORES
> currently does, but there are limitations (like the max number of revisions
> in a batch etc..). The API clearly states when something is not supported,
> so you can check its compatibility now making some requests to:
>
> https://ores-legacy.wikimedia.org
>
> If you open a task with a list of systems that you need to migrate we can
> definitely take a look and help. So far the traffic being served by ORES
> has been reduced to few clients, and all of them don't run with
> recognizable UAs (see https://meta.wikimedia.org/wiki/User-Agent_policy)
> so we'll try our best to support them. The migration to Lift Wing has been
> widely publicized, a lot of documentation is available to migrate. We'd
> suggest trying Lift Wing for your systems instead (see
> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Usage).
>
> The Machine Learning plan is to eventually deprecate ores-legacy too, to
> maintain only one system (namely Lift Wing). There is no final date yet,
> we'll try to reach out to all remaining users first, so if you plan to keep
> using ores-legacy please follow up with us first :)
>
> Thanks!
>
> Luca (on behalf of the ML Team)
>
> On Fri, Sep 22, 2023 at 5:10?PM Aaron Halfaker <aaron.halfaker@gmail.com>
> wrote:
>
>> Does the new ores-legacy support the same feature set. E.g. features
>> output, injection, and threshold optimizations. Or is it just prediction?
>> This will affect some of the systems I need to migrate.
>>
>> On Fri, Sep 22, 2023, 06:21 Ilias Sarantopoulos <
>> isarantopoulos@wikimedia.org> wrote:
>>
>>> Hello!
>>>
>>>
>>> As a next step in the deprecation process of ORES
>>> https://wikitech.wikimedia.org/wiki/ORES the Machine Learning team will
>>> switch the backend of ores.wikimedia.org to ores-legacy, a k8s
>>> application meant to provide a compatibility layer between ORES and Lift
>>> Wing so users that have not yet migrated to Lift Wing will be
>>> transparently migrated. Ores-legacy is an application that has the same API
>>> as ORES but in the background makes requests to Lift Wing, allowing us to
>>> decommission the ORES servers until all clients have moved.
>>>
>>> This change is planned to take place on Monday 25th of September. If
>>> you have a client/application that is still using ORES we expect that this
>>> switch is going to be transparent for you.
>>>
>>> However keep in mind that ores-legacy is not a 100% replacement for ORES
>>> as some old and unused features are no longer supported.
>>>
>>> If you see anything out of the ordinary, feel free to contact the
>>> Machine Learning team:
>>>
>>> IRC libera: #wikimedia-ml
>>>
>>> Phabricator: Machine-Learning-team tag
>>>
>>> Thank you!
>>>
>>>
>>> On Wed, Aug 9, 2023 at 1:22?PM Chaloemphon Praphuchakang <
>>> yoshrakpraphu@gmail.com> wrote:
>>>
>>>>
>>>> On Tue, 8 Aug 2023, 10:45 Tilman Bayer, <haebwiki@gmail.com> wrote:
>>>>
>>>>>
>>>>> Hi Chris,
>>>>>
>>>>> On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Tilman,
>>>>>>
>>>>>> Most of the work is still very experimental. We have hosted a few
>>>>>> LLMs on Lift Wing already (StarCoder for example) but they were just
>>>>>> running on CPU, far too slow for real use cases. But it proves that we can
>>>>>> easily host LLMs on Lift Wing. We have been pretty quiet about it while we
>>>>>> focus on the ORES migration, but it is our next big project. More soon
>>>>>> hopefully!
>>>>>>
>>>>> Understood. Looking forward to learning more later!
>>>>>
>>>>>
>>>>>> Where we are now is that we have budget for a big GPU purchase
>>>>>> (~10-20 GPUs depending on cost), the question we will try to answer after
>>>>>> the ORES migration is complete is: what GPUs should we purchase? We are
>>>>>> trying to balance our strong preference to stay open source (i.e. AMD mROC)
>>>>>> in a world dominated by a single closed source vendor (i.e. Nvidia). In
>>>>>> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
>>>>>> H100, etc) or a mix of big and small? We will need to figure out all this.
>>>>>>
>>>>> I see. On that matter, what do you folks make of the recent
>>>>> announcements of AMD's partnerships with Hugging Face and Pytorch[5]?
>>>>> (which, I understand, came after the ML team had already launched the
>>>>> aforementioned new AMD explorations)
>>>>>
>>>>> "Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch to
>>>>> take on Nvidia [...]
>>>>> Both partnerships involve AMD’s ROCm AI software stack, the company’s
>>>>> answer to Nvidia’s proprietary CUDA platform and application-programming
>>>>> interface. AMD called ROCm an open and portable AI system with
>>>>> out-of-the-box support that can port to existing AI models. [...B]oth AMD
>>>>> and Hugging Face are dedicating engineering resources to each other and
>>>>> sharing data to ensure that the constantly updated AI models from Hugging
>>>>> Face, which might not otherwise run well on AMD hardware, would be
>>>>> “guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
>>>>> will fully upstream the ROCm software stack and “provide immediate ‘day
>>>>> zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
>>>>> accelerators,” which is meant to appeal to those customers looking to
>>>>> switch from Nvidia’s software ecosystem."
>>>>>
>>>>>
>>>>> In their own announcement, Hugging Face offered further details,
>>>>> including a pretty impressive list of models to be supported:[6]
>>>>>
>>>>>
>>>>> "We intend to support state-of-the-art transformer architectures for
>>>>> natural language processing, computer vision, and speech, such as BERT,
>>>>> DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
>>>>> generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
>>>>> LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
>>>>> support more traditional computer vision models, like ResNet and ResNext,
>>>>> and deep learning recommendation models, a first for us. [..] We'll do our
>>>>> best to test and validate these models for PyTorch, TensorFlow, and ONNX
>>>>> Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
>>>>> seamlessly in our open-source libraries, starting with the transformers
>>>>> library."
>>>>>
>>>>>
>>>>> Do you think this may promise too much, or could it point to a
>>>>> possible solution of the Foundation's conundrum?
>>>>> In any case, this seems to be an interesting moment where many in AI
>>>>> are trying to move away from Nvidia's proprietary CUDA platform. Most of
>>>>> them probably more for financial and availability reasons though, given the
>>>>> current GPU shortages[7] (which the ML team is undoubtedly aware of
>>>>> already; mentioning this as context for others on this list. See also
>>>>> Marketwatch's remarks about current margins[5]).
>>>>>
>>>>> Regards, Tilman
>>>>>
>>>>>
>>>>> [5]
>>>>> https://archive.ph/2023.06.15-173527/https://www.marketwatch.com/amp/story/open-source-ai-amd-looks-to-hugging-face-and-meta-spinoff-pytorch-to-take-on-nvidia-e4738f87
>>>>> [6] https://huggingface.co/blog/huggingface-and-amd
>>>>> [7] See e.g.
>>>>> https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/ (avoid
>>>>> playing the song though. Don't say I didn't warn you)
>>>>>
>>>>>
>>>>>> I wouldn't characterize WMF's Language Team using CPU as because of
>>>>>> AMD, rather at the time we didn't have the budget for GPUs so Lift Wing
>>>>>> didn't have any. Since then we have moved two GPUs onto Lift Wing for
>>>>>> testing but they are pretty old (2017ish). Once we make the big GPU
>>>>>> purchase Lift Wing will gain a lot of functionality for LLM and similar
>>>>>> models.
>>>>>>
>>>>>> Chris
>>>>>>
>>>>>> On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi everybody,
>>>>>>>>
>>>>>>>> TL;DR We would like users of ORES models to migrate to our new open
>>>>>>>> source ML infrastructure, Lift Wing, within the next five months. We are
>>>>>>>> available to help you do that, from advice to making code commits. It is
>>>>>>>> important to note: All ML models currently accessible on ORES are also
>>>>>>>> currently accessible on Lift Wing.
>>>>>>>>
>>>>>>>> As part of the Machine Learning Modernization Project (
>>>>>>>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization),
>>>>>>>> the Machine Learning team has deployed a Wikimedia’s new machine learning
>>>>>>>> inference infrastructure, called Lift Wing (
>>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing).
>>>>>>>> Lift Wing brings a lot of new features such as support for GPU-based
>>>>>>>> models, open source LLM hosting, auto-scaling, stability, and ability to
>>>>>>>> host a larger number of models.
>>>>>>>>
>>>>>>>
>>>>>>> This sounds quite exciting! What's the best place to read up on that
>>>>>>> planned support for GPU-based models and open source LLMs? (I also saw in
>>>>>>> the recent NYT article[1] that the team is "in the process of adapting A.I.
>>>>>>> models that are 'off the shelf; — essentially models that have been made
>>>>>>> available by researchers for anyone to freely customize — so that
>>>>>>> Wikipedia’s editors can use them for their work.")
>>>>>>>
>>>>>>> I'm aware of the history[2] of not being able to use NVIDIA GPUs due
>>>>>>> to their CUDA drivers being proprietary. It was mentioned recently in the
>>>>>>> Wikimedia AI Telegram group that this is still a serious limitation,
>>>>>>> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
>>>>>>> WMF's Language team has resorted to using models without GPU support (CPU
>>>>>>> only).[4]
>>>>>>> It sounds like there is reasonable hope that this situation could
>>>>>>> change fairly soon? Would it also mean both at the same time, i.e. open
>>>>>>> source LLMs running with GPU support (considering that at least some
>>>>>>> well-known ones appear to require torch.cuda.is_available() == True for
>>>>>>> that)?
>>>>>>>
>>>>>>> Regards, Tilman
>>>>>>>
>>>>>>> [1]
>>>>>>> https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
>>>>>>> [2]
>>>>>>> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
>>>>>>> [3] https://phabricator.wikimedia.org/T334583 etc.
>>>>>>> [4]
>>>>>>> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
>>>>>>> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental but,
>>>>>>> I understand, written to be deployable on WMF infrastructure)
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> With the creation of Lift Wing, the team is turning its attention
>>>>>>>> to deprecating the current machine learning infrastructure, ORES. ORES
>>>>>>>> served us really well over the years, it was a successful project but it
>>>>>>>> came before radical changes in technology like Docker, Kubernetes and more
>>>>>>>> recently MLOps. The servers that run ORES are at the end of their planned
>>>>>>>> lifespan and so to save cost we are going to shut them down in early 2024.
>>>>>>>>
>>>>>>>> We have outlined a deprecation path on Wikitech (
>>>>>>>> https://wikitech.wikimedia.org/wiki/ORES), please read the page if
>>>>>>>> you are a maintainer of a tool or code that uses the ORES endpoint
>>>>>>>> https://ores.wikimedia.org/). If you have any doubt or if you need
>>>>>>>> assistance in migrating to Lift Wing, feel free to contact the ML team via:
>>>>>>>>
>>>>>>>> - Email: ml@wikimedia.org
>>>>>>>> - Phabricator: #Machine-Learning-Team tag
>>>>>>>> - IRC (Libera): #wikimedia-ml
>>>>>>>>
>>>>>>>> The Machine Learning team is available to help projects migrate,
>>>>>>>> from offering advice to making code commits. We want to make this as easy
>>>>>>>> as possible for folks.
>>>>>>>>
>>>>>>>> High Level timeline:
>>>>>>>>
>>>>>>>> **By September 30th 2023: *Infrastructure powering the ORES API
>>>>>>>> endpoint will be migrated from ORES to Lift Wing. For users, the API
>>>>>>>> endpoint will remain the same, and most users won’t notice any change.
>>>>>>>> Rather just the backend services powering the endpoint will change.
>>>>>>>>
>>>>>>>> Details: We'd like to add a DNS CNAME that points
>>>>>>>> ores.wikimedia.org to ores-legacy.wikimedia.org, a new endpoint
>>>>>>>> that offers a almost complete replacement of the ORES API calling Lift Wing
>>>>>>>> behind the scenes. In an ideal world we'd migrate all tools to Lift Wing
>>>>>>>> before decommissioning the infrastructure behind ores.wikimedia.org,
>>>>>>>> but it turned out to be really challenging so to avoid disrupting users we
>>>>>>>> chose to implement a transition layer/API.
>>>>>>>>
>>>>>>>> To summarize, if you don't have time to migrate before September to
>>>>>>>> Lift Wing, your code/tool should work just fine on
>>>>>>>> ores-legacy.wikimedia.org and you'll not have to change a line in
>>>>>>>> your code thanks to the DNS CNAME. The ores-legacy endpoint is not a 100%
>>>>>>>> replacement for ores, we removed some very old and not used features, so we
>>>>>>>> highly recommend at least test the new endpoint for your use case to avoid
>>>>>>>> surprises when we'll make the switch. In case you find anything weird,
>>>>>>>> please report it to us using the aforementioned channels.
>>>>>>>>
>>>>>>>> **September to January: *We will be reaching out to every user of
>>>>>>>> ORES we can identify and working with them to make the migration process as
>>>>>>>> easy as possible.
>>>>>>>>
>>>>>>>> **By January 2024: *If all goes well, we would like zero traffic
>>>>>>>> on the ORES API endpoint so we can turn off the ores-legacy API.
>>>>>>>>
>>>>>>>> If you want more information about Lift Wing, please check
>>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>>>>>>>
>>>>>>>> Thanks in advance for the patience and the help!
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> The Machine Learning Team
>>>>>>>> _______________________________________________
>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>> To unsubscribe send an email to
>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>
>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>>
>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>
>>>>>> _______________________________________________
>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>
>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>
>>>>> _______________________________________________
>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>
>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>
>>>> _______________________________________________
>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>
>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>
>>> _______________________________________________
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> _______________________________________________
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

It looks like model_info is not implemented at all. E.g.
https://ores-legacy.wikimedia.org/v3/scores/enwiki?model_info=statistics.thresholds.true.%22maximum+recall+@+precision+%3E=+0.9%22&models=damaging

I get {"detail":{"error":{"code":"bad request","message":"model_info query
parameter is not supported by this endpoint anymore. For more information
please visit https://wikitech.wikimedia.org/wiki/ORES"}}}

But when I go to that page, nothing discusses model_info. Is there a way
to get this from LiftWing?

On Fri, Sep 22, 2023 at 8:53?AM Aaron Halfaker <aaron.halfaker@gmail.com>
wrote:

> Do you have a tag for filing bugs against ORES-legacy? I can't seem to
> find a relevant one in phab.
>
> On Fri, Sep 22, 2023 at 8:39?AM Luca Toscano <ltoscano@wikimedia.org>
> wrote:
>
>> Hi Aaron!
>>
>> Thanks for following up. The API is almost compatible with what ORES
>> currently does, but there are limitations (like the max number of revisions
>> in a batch etc..). The API clearly states when something is not supported,
>> so you can check its compatibility now making some requests to:
>>
>> https://ores-legacy.wikimedia.org
>>
>> If you open a task with a list of systems that you need to migrate we
>> can definitely take a look and help. So far the traffic being served by
>> ORES has been reduced to few clients, and all of them don't run with
>> recognizable UAs (see https://meta.wikimedia.org/wiki/User-Agent_policy)
>> so we'll try our best to support them. The migration to Lift Wing has been
>> widely publicized, a lot of documentation is available to migrate. We'd
>> suggest trying Lift Wing for your systems instead (see
>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Usage).
>>
>> The Machine Learning plan is to eventually deprecate ores-legacy too, to
>> maintain only one system (namely Lift Wing). There is no final date yet,
>> we'll try to reach out to all remaining users first, so if you plan to keep
>> using ores-legacy please follow up with us first :)
>>
>> Thanks!
>>
>> Luca (on behalf of the ML Team)
>>
>> On Fri, Sep 22, 2023 at 5:10?PM Aaron Halfaker <aaron.halfaker@gmail.com>
>> wrote:
>>
>>> Does the new ores-legacy support the same feature set. E.g. features
>>> output, injection, and threshold optimizations. Or is it just prediction?
>>> This will affect some of the systems I need to migrate.
>>>
>>> On Fri, Sep 22, 2023, 06:21 Ilias Sarantopoulos <
>>> isarantopoulos@wikimedia.org> wrote:
>>>
>>>> Hello!
>>>>
>>>>
>>>> As a next step in the deprecation process of ORES
>>>> https://wikitech.wikimedia.org/wiki/ORES the Machine Learning team will
>>>> switch the backend of ores.wikimedia.org to ores-legacy, a k8s
>>>> application meant to provide a compatibility layer between ORES and Lift
>>>> Wing so users that have not yet migrated to Lift Wing will be
>>>> transparently migrated. Ores-legacy is an application that has the same API
>>>> as ORES but in the background makes requests to Lift Wing, allowing us to
>>>> decommission the ORES servers until all clients have moved.
>>>>
>>>> This change is planned to take place on Monday 25th of September. If
>>>> you have a client/application that is still using ORES we expect that this
>>>> switch is going to be transparent for you.
>>>>
>>>> However keep in mind that ores-legacy is not a 100% replacement for
>>>> ORES as some old and unused features are no longer supported.
>>>>
>>>> If you see anything out of the ordinary, feel free to contact the
>>>> Machine Learning team:
>>>>
>>>> IRC libera: #wikimedia-ml
>>>>
>>>> Phabricator: Machine-Learning-team tag
>>>>
>>>> Thank you!
>>>>
>>>>
>>>> On Wed, Aug 9, 2023 at 1:22?PM Chaloemphon Praphuchakang <
>>>> yoshrakpraphu@gmail.com> wrote:
>>>>
>>>>>
>>>>> On Tue, 8 Aug 2023, 10:45 Tilman Bayer, <haebwiki@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>> On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Tilman,
>>>>>>>
>>>>>>> Most of the work is still very experimental. We have hosted a few
>>>>>>> LLMs on Lift Wing already (StarCoder for example) but they were just
>>>>>>> running on CPU, far too slow for real use cases. But it proves that we can
>>>>>>> easily host LLMs on Lift Wing. We have been pretty quiet about it while we
>>>>>>> focus on the ORES migration, but it is our next big project. More soon
>>>>>>> hopefully!
>>>>>>>
>>>>>> Understood. Looking forward to learning more later!
>>>>>>
>>>>>>
>>>>>>> Where we are now is that we have budget for a big GPU purchase
>>>>>>> (~10-20 GPUs depending on cost), the question we will try to answer after
>>>>>>> the ORES migration is complete is: what GPUs should we purchase? We are
>>>>>>> trying to balance our strong preference to stay open source (i.e. AMD mROC)
>>>>>>> in a world dominated by a single closed source vendor (i.e. Nvidia). In
>>>>>>> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
>>>>>>> H100, etc) or a mix of big and small? We will need to figure out all this.
>>>>>>>
>>>>>> I see. On that matter, what do you folks make of the recent
>>>>>> announcements of AMD's partnerships with Hugging Face and Pytorch[5]?
>>>>>> (which, I understand, came after the ML team had already launched the
>>>>>> aforementioned new AMD explorations)
>>>>>>
>>>>>> "Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch
>>>>>> to take on Nvidia [...]
>>>>>> Both partnerships involve AMD’s ROCm AI software stack, the company’s
>>>>>> answer to Nvidia’s proprietary CUDA platform and application-programming
>>>>>> interface. AMD called ROCm an open and portable AI system with
>>>>>> out-of-the-box support that can port to existing AI models. [...B]oth AMD
>>>>>> and Hugging Face are dedicating engineering resources to each other and
>>>>>> sharing data to ensure that the constantly updated AI models from Hugging
>>>>>> Face, which might not otherwise run well on AMD hardware, would be
>>>>>> “guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
>>>>>> will fully upstream the ROCm software stack and “provide immediate ‘day
>>>>>> zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
>>>>>> accelerators,” which is meant to appeal to those customers looking to
>>>>>> switch from Nvidia’s software ecosystem."
>>>>>>
>>>>>>
>>>>>> In their own announcement, Hugging Face offered further details,
>>>>>> including a pretty impressive list of models to be supported:[6]
>>>>>>
>>>>>>
>>>>>> "We intend to support state-of-the-art transformer architectures for
>>>>>> natural language processing, computer vision, and speech, such as BERT,
>>>>>> DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
>>>>>> generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
>>>>>> LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
>>>>>> support more traditional computer vision models, like ResNet and ResNext,
>>>>>> and deep learning recommendation models, a first for us. [..] We'll do our
>>>>>> best to test and validate these models for PyTorch, TensorFlow, and ONNX
>>>>>> Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
>>>>>> seamlessly in our open-source libraries, starting with the transformers
>>>>>> library."
>>>>>>
>>>>>>
>>>>>> Do you think this may promise too much, or could it point to a
>>>>>> possible solution of the Foundation's conundrum?
>>>>>> In any case, this seems to be an interesting moment where many in AI
>>>>>> are trying to move away from Nvidia's proprietary CUDA platform. Most of
>>>>>> them probably more for financial and availability reasons though, given the
>>>>>> current GPU shortages[7] (which the ML team is undoubtedly aware of
>>>>>> already; mentioning this as context for others on this list. See also
>>>>>> Marketwatch's remarks about current margins[5]).
>>>>>>
>>>>>> Regards, Tilman
>>>>>>
>>>>>>
>>>>>> [5]
>>>>>> https://archive.ph/2023.06.15-173527/https://www.marketwatch.com/amp/story/open-source-ai-amd-looks-to-hugging-face-and-meta-spinoff-pytorch-to-take-on-nvidia-e4738f87
>>>>>> [6] https://huggingface.co/blog/huggingface-and-amd
>>>>>> [7] See e.g.
>>>>>> https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/
>>>>>> (avoid playing the song though. Don't say I didn't warn you)
>>>>>>
>>>>>>
>>>>>>> I wouldn't characterize WMF's Language Team using CPU as because of
>>>>>>> AMD, rather at the time we didn't have the budget for GPUs so Lift Wing
>>>>>>> didn't have any. Since then we have moved two GPUs onto Lift Wing for
>>>>>>> testing but they are pretty old (2017ish). Once we make the big GPU
>>>>>>> purchase Lift Wing will gain a lot of functionality for LLM and similar
>>>>>>> models.
>>>>>>>
>>>>>>> Chris
>>>>>>>
>>>>>>> On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi everybody,
>>>>>>>>>
>>>>>>>>> TL;DR We would like users of ORES models to migrate to our new
>>>>>>>>> open source ML infrastructure, Lift Wing, within the next five months. We
>>>>>>>>> are available to help you do that, from advice to making code commits. It
>>>>>>>>> is important to note: All ML models currently accessible on ORES are also
>>>>>>>>> currently accessible on Lift Wing.
>>>>>>>>>
>>>>>>>>> As part of the Machine Learning Modernization Project (
>>>>>>>>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization),
>>>>>>>>> the Machine Learning team has deployed a Wikimedia’s new machine learning
>>>>>>>>> inference infrastructure, called Lift Wing (
>>>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing).
>>>>>>>>> Lift Wing brings a lot of new features such as support for GPU-based
>>>>>>>>> models, open source LLM hosting, auto-scaling, stability, and ability to
>>>>>>>>> host a larger number of models.
>>>>>>>>>
>>>>>>>>
>>>>>>>> This sounds quite exciting! What's the best place to read up on
>>>>>>>> that planned support for GPU-based models and open source LLMs? (I also saw
>>>>>>>> in the recent NYT article[1] that the team is "in the process of adapting
>>>>>>>> A.I. models that are 'off the shelf; — essentially models that have been
>>>>>>>> made available by researchers for anyone to freely customize — so that
>>>>>>>> Wikipedia’s editors can use them for their work.")
>>>>>>>>
>>>>>>>> I'm aware of the history[2] of not being able to use NVIDIA
>>>>>>>> GPUs due to their CUDA drivers being proprietary. It was mentioned recently
>>>>>>>> in the Wikimedia AI Telegram group that this is still a serious limitation,
>>>>>>>> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
>>>>>>>> WMF's Language team has resorted to using models without GPU support (CPU
>>>>>>>> only).[4]
>>>>>>>> It sounds like there is reasonable hope that this situation could
>>>>>>>> change fairly soon? Would it also mean both at the same time, i.e. open
>>>>>>>> source LLMs running with GPU support (considering that at least some
>>>>>>>> well-known ones appear to require torch.cuda.is_available() == True for
>>>>>>>> that)?
>>>>>>>>
>>>>>>>> Regards, Tilman
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
>>>>>>>> [2]
>>>>>>>> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
>>>>>>>> [3] https://phabricator.wikimedia.org/T334583 etc.
>>>>>>>> [4]
>>>>>>>> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
>>>>>>>> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental
>>>>>>>> but, I understand, written to be deployable on WMF infrastructure)
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> With the creation of Lift Wing, the team is turning its attention
>>>>>>>>> to deprecating the current machine learning infrastructure, ORES. ORES
>>>>>>>>> served us really well over the years, it was a successful project but it
>>>>>>>>> came before radical changes in technology like Docker, Kubernetes and more
>>>>>>>>> recently MLOps. The servers that run ORES are at the end of their planned
>>>>>>>>> lifespan and so to save cost we are going to shut them down in early 2024.
>>>>>>>>>
>>>>>>>>> We have outlined a deprecation path on Wikitech (
>>>>>>>>> https://wikitech.wikimedia.org/wiki/ORES), please read the page
>>>>>>>>> if you are a maintainer of a tool or code that uses the ORES endpoint
>>>>>>>>> https://ores.wikimedia.org/). If you have any doubt or if you
>>>>>>>>> need assistance in migrating to Lift Wing, feel free to contact the ML team
>>>>>>>>> via:
>>>>>>>>>
>>>>>>>>> - Email: ml@wikimedia.org
>>>>>>>>> - Phabricator: #Machine-Learning-Team tag
>>>>>>>>> - IRC (Libera): #wikimedia-ml
>>>>>>>>>
>>>>>>>>> The Machine Learning team is available to help projects migrate,
>>>>>>>>> from offering advice to making code commits. We want to make this as easy
>>>>>>>>> as possible for folks.
>>>>>>>>>
>>>>>>>>> High Level timeline:
>>>>>>>>>
>>>>>>>>> **By September 30th 2023: *Infrastructure powering the ORES API
>>>>>>>>> endpoint will be migrated from ORES to Lift Wing. For users, the API
>>>>>>>>> endpoint will remain the same, and most users won’t notice any change.
>>>>>>>>> Rather just the backend services powering the endpoint will change.
>>>>>>>>>
>>>>>>>>> Details: We'd like to add a DNS CNAME that points
>>>>>>>>> ores.wikimedia.org to ores-legacy.wikimedia.org, a new endpoint
>>>>>>>>> that offers a almost complete replacement of the ORES API calling Lift Wing
>>>>>>>>> behind the scenes. In an ideal world we'd migrate all tools to Lift Wing
>>>>>>>>> before decommissioning the infrastructure behind
>>>>>>>>> ores.wikimedia.org, but it turned out to be really challenging so
>>>>>>>>> to avoid disrupting users we chose to implement a transition layer/API.
>>>>>>>>>
>>>>>>>>> To summarize, if you don't have time to migrate before September
>>>>>>>>> to Lift Wing, your code/tool should work just fine on
>>>>>>>>> ores-legacy.wikimedia.org and you'll not have to change a line in
>>>>>>>>> your code thanks to the DNS CNAME. The ores-legacy endpoint is not a 100%
>>>>>>>>> replacement for ores, we removed some very old and not used features, so we
>>>>>>>>> highly recommend at least test the new endpoint for your use case to avoid
>>>>>>>>> surprises when we'll make the switch. In case you find anything weird,
>>>>>>>>> please report it to us using the aforementioned channels.
>>>>>>>>>
>>>>>>>>> **September to January: *We will be reaching out to every user of
>>>>>>>>> ORES we can identify and working with them to make the migration process as
>>>>>>>>> easy as possible.
>>>>>>>>>
>>>>>>>>> **By January 2024: *If all goes well, we would like zero traffic
>>>>>>>>> on the ORES API endpoint so we can turn off the ores-legacy API.
>>>>>>>>>
>>>>>>>>> If you want more information about Lift Wing, please check
>>>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>>>>>>>>
>>>>>>>>> Thanks in advance for the patience and the help!
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> The Machine Learning Team
>>>>>>>>> _______________________________________________
>>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>>> To unsubscribe send an email to
>>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>>
>>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>> To unsubscribe send an email to
>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>
>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>>
>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>
>>>>>> _______________________________________________
>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>
>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>
>>>>> _______________________________________________
>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>
>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>
>>>> _______________________________________________
>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>
>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>
>>> _______________________________________________
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> _______________________________________________
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
>

Hi!

The tag 'Machine-Learning-Team' is the one that we pay more attention to :)

Luca

On Fri, Sep 22, 2023 at 5:54?PM Aaron Halfaker <aaron.halfaker@gmail.com>
wrote:

> Do you have a tag for filing bugs against ORES-legacy? I can't seem to
> find a relevant one in phab.
>
> On Fri, Sep 22, 2023 at 8:39?AM Luca Toscano <ltoscano@wikimedia.org>
> wrote:
>
>> Hi Aaron!
>>
>> Thanks for following up. The API is almost compatible with what ORES
>> currently does, but there are limitations (like the max number of revisions
>> in a batch etc..). The API clearly states when something is not supported,
>> so you can check its compatibility now making some requests to:
>>
>> https://ores-legacy.wikimedia.org
>>
>> If you open a task with a list of systems that you need to migrate we
>> can definitely take a look and help. So far the traffic being served by
>> ORES has been reduced to few clients, and all of them don't run with
>> recognizable UAs (see https://meta.wikimedia.org/wiki/User-Agent_policy)
>> so we'll try our best to support them. The migration to Lift Wing has been
>> widely publicized, a lot of documentation is available to migrate. We'd
>> suggest trying Lift Wing for your systems instead (see
>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Usage).
>>
>> The Machine Learning plan is to eventually deprecate ores-legacy too, to
>> maintain only one system (namely Lift Wing). There is no final date yet,
>> we'll try to reach out to all remaining users first, so if you plan to keep
>> using ores-legacy please follow up with us first :)
>>
>> Thanks!
>>
>> Luca (on behalf of the ML Team)
>>
>> On Fri, Sep 22, 2023 at 5:10?PM Aaron Halfaker <aaron.halfaker@gmail.com>
>> wrote:
>>
>>> Does the new ores-legacy support the same feature set. E.g. features
>>> output, injection, and threshold optimizations. Or is it just prediction?
>>> This will affect some of the systems I need to migrate.
>>>
>>> On Fri, Sep 22, 2023, 06:21 Ilias Sarantopoulos <
>>> isarantopoulos@wikimedia.org> wrote:
>>>
>>>> Hello!
>>>>
>>>>
>>>> As a next step in the deprecation process of ORES
>>>> https://wikitech.wikimedia.org/wiki/ORES the Machine Learning team will
>>>> switch the backend of ores.wikimedia.org to ores-legacy, a k8s
>>>> application meant to provide a compatibility layer between ORES and Lift
>>>> Wing so users that have not yet migrated to Lift Wing will be
>>>> transparently migrated. Ores-legacy is an application that has the same API
>>>> as ORES but in the background makes requests to Lift Wing, allowing us to
>>>> decommission the ORES servers until all clients have moved.
>>>>
>>>> This change is planned to take place on Monday 25th of September. If
>>>> you have a client/application that is still using ORES we expect that this
>>>> switch is going to be transparent for you.
>>>>
>>>> However keep in mind that ores-legacy is not a 100% replacement for
>>>> ORES as some old and unused features are no longer supported.
>>>>
>>>> If you see anything out of the ordinary, feel free to contact the
>>>> Machine Learning team:
>>>>
>>>> IRC libera: #wikimedia-ml
>>>>
>>>> Phabricator: Machine-Learning-team tag
>>>>
>>>> Thank you!
>>>>
>>>>
>>>> On Wed, Aug 9, 2023 at 1:22?PM Chaloemphon Praphuchakang <
>>>> yoshrakpraphu@gmail.com> wrote:
>>>>
>>>>>
>>>>> On Tue, 8 Aug 2023, 10:45 Tilman Bayer, <haebwiki@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>> On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Tilman,
>>>>>>>
>>>>>>> Most of the work is still very experimental. We have hosted a few
>>>>>>> LLMs on Lift Wing already (StarCoder for example) but they were just
>>>>>>> running on CPU, far too slow for real use cases. But it proves that we can
>>>>>>> easily host LLMs on Lift Wing. We have been pretty quiet about it while we
>>>>>>> focus on the ORES migration, but it is our next big project. More soon
>>>>>>> hopefully!
>>>>>>>
>>>>>> Understood. Looking forward to learning more later!
>>>>>>
>>>>>>
>>>>>>> Where we are now is that we have budget for a big GPU purchase
>>>>>>> (~10-20 GPUs depending on cost), the question we will try to answer after
>>>>>>> the ORES migration is complete is: what GPUs should we purchase? We are
>>>>>>> trying to balance our strong preference to stay open source (i.e. AMD mROC)
>>>>>>> in a world dominated by a single closed source vendor (i.e. Nvidia). In
>>>>>>> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
>>>>>>> H100, etc) or a mix of big and small? We will need to figure out all this.
>>>>>>>
>>>>>> I see. On that matter, what do you folks make of the recent
>>>>>> announcements of AMD's partnerships with Hugging Face and Pytorch[5]?
>>>>>> (which, I understand, came after the ML team had already launched the
>>>>>> aforementioned new AMD explorations)
>>>>>>
>>>>>> "Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch
>>>>>> to take on Nvidia [...]
>>>>>> Both partnerships involve AMD’s ROCm AI software stack, the company’s
>>>>>> answer to Nvidia’s proprietary CUDA platform and application-programming
>>>>>> interface. AMD called ROCm an open and portable AI system with
>>>>>> out-of-the-box support that can port to existing AI models. [...B]oth AMD
>>>>>> and Hugging Face are dedicating engineering resources to each other and
>>>>>> sharing data to ensure that the constantly updated AI models from Hugging
>>>>>> Face, which might not otherwise run well on AMD hardware, would be
>>>>>> “guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
>>>>>> will fully upstream the ROCm software stack and “provide immediate ‘day
>>>>>> zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
>>>>>> accelerators,” which is meant to appeal to those customers looking to
>>>>>> switch from Nvidia’s software ecosystem."
>>>>>>
>>>>>>
>>>>>> In their own announcement, Hugging Face offered further details,
>>>>>> including a pretty impressive list of models to be supported:[6]
>>>>>>
>>>>>>
>>>>>> "We intend to support state-of-the-art transformer architectures for
>>>>>> natural language processing, computer vision, and speech, such as BERT,
>>>>>> DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
>>>>>> generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
>>>>>> LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
>>>>>> support more traditional computer vision models, like ResNet and ResNext,
>>>>>> and deep learning recommendation models, a first for us. [..] We'll do our
>>>>>> best to test and validate these models for PyTorch, TensorFlow, and ONNX
>>>>>> Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
>>>>>> seamlessly in our open-source libraries, starting with the transformers
>>>>>> library."
>>>>>>
>>>>>>
>>>>>> Do you think this may promise too much, or could it point to a
>>>>>> possible solution of the Foundation's conundrum?
>>>>>> In any case, this seems to be an interesting moment where many in AI
>>>>>> are trying to move away from Nvidia's proprietary CUDA platform. Most of
>>>>>> them probably more for financial and availability reasons though, given the
>>>>>> current GPU shortages[7] (which the ML team is undoubtedly aware of
>>>>>> already; mentioning this as context for others on this list. See also
>>>>>> Marketwatch's remarks about current margins[5]).
>>>>>>
>>>>>> Regards, Tilman
>>>>>>
>>>>>>
>>>>>> [5]
>>>>>> https://archive.ph/2023.06.15-173527/https://www.marketwatch.com/amp/story/open-source-ai-amd-looks-to-hugging-face-and-meta-spinoff-pytorch-to-take-on-nvidia-e4738f87
>>>>>> [6] https://huggingface.co/blog/huggingface-and-amd
>>>>>> [7] See e.g.
>>>>>> https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/
>>>>>> (avoid playing the song though. Don't say I didn't warn you)
>>>>>>
>>>>>>
>>>>>>> I wouldn't characterize WMF's Language Team using CPU as because of
>>>>>>> AMD, rather at the time we didn't have the budget for GPUs so Lift Wing
>>>>>>> didn't have any. Since then we have moved two GPUs onto Lift Wing for
>>>>>>> testing but they are pretty old (2017ish). Once we make the big GPU
>>>>>>> purchase Lift Wing will gain a lot of functionality for LLM and similar
>>>>>>> models.
>>>>>>>
>>>>>>> Chris
>>>>>>>
>>>>>>> On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi everybody,
>>>>>>>>>
>>>>>>>>> TL;DR We would like users of ORES models to migrate to our new
>>>>>>>>> open source ML infrastructure, Lift Wing, within the next five months. We
>>>>>>>>> are available to help you do that, from advice to making code commits. It
>>>>>>>>> is important to note: All ML models currently accessible on ORES are also
>>>>>>>>> currently accessible on Lift Wing.
>>>>>>>>>
>>>>>>>>> As part of the Machine Learning Modernization Project (
>>>>>>>>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization),
>>>>>>>>> the Machine Learning team has deployed a Wikimedia’s new machine learning
>>>>>>>>> inference infrastructure, called Lift Wing (
>>>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing).
>>>>>>>>> Lift Wing brings a lot of new features such as support for GPU-based
>>>>>>>>> models, open source LLM hosting, auto-scaling, stability, and ability to
>>>>>>>>> host a larger number of models.
>>>>>>>>>
>>>>>>>>
>>>>>>>> This sounds quite exciting! What's the best place to read up on
>>>>>>>> that planned support for GPU-based models and open source LLMs? (I also saw
>>>>>>>> in the recent NYT article[1] that the team is "in the process of adapting
>>>>>>>> A.I. models that are 'off the shelf; — essentially models that have been
>>>>>>>> made available by researchers for anyone to freely customize — so that
>>>>>>>> Wikipedia’s editors can use them for their work.")
>>>>>>>>
>>>>>>>> I'm aware of the history[2] of not being able to use NVIDIA
>>>>>>>> GPUs due to their CUDA drivers being proprietary. It was mentioned recently
>>>>>>>> in the Wikimedia AI Telegram group that this is still a serious limitation,
>>>>>>>> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
>>>>>>>> WMF's Language team has resorted to using models without GPU support (CPU
>>>>>>>> only).[4]
>>>>>>>> It sounds like there is reasonable hope that this situation could
>>>>>>>> change fairly soon? Would it also mean both at the same time, i.e. open
>>>>>>>> source LLMs running with GPU support (considering that at least some
>>>>>>>> well-known ones appear to require torch.cuda.is_available() == True for
>>>>>>>> that)?
>>>>>>>>
>>>>>>>> Regards, Tilman
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
>>>>>>>> [2]
>>>>>>>> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
>>>>>>>> [3] https://phabricator.wikimedia.org/T334583 etc.
>>>>>>>> [4]
>>>>>>>> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
>>>>>>>> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental
>>>>>>>> but, I understand, written to be deployable on WMF infrastructure)
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> With the creation of Lift Wing, the team is turning its attention
>>>>>>>>> to deprecating the current machine learning infrastructure, ORES. ORES
>>>>>>>>> served us really well over the years, it was a successful project but it
>>>>>>>>> came before radical changes in technology like Docker, Kubernetes and more
>>>>>>>>> recently MLOps. The servers that run ORES are at the end of their planned
>>>>>>>>> lifespan and so to save cost we are going to shut them down in early 2024.
>>>>>>>>>
>>>>>>>>> We have outlined a deprecation path on Wikitech (
>>>>>>>>> https://wikitech.wikimedia.org/wiki/ORES), please read the page
>>>>>>>>> if you are a maintainer of a tool or code that uses the ORES endpoint
>>>>>>>>> https://ores.wikimedia.org/). If you have any doubt or if you
>>>>>>>>> need assistance in migrating to Lift Wing, feel free to contact the ML team
>>>>>>>>> via:
>>>>>>>>>
>>>>>>>>> - Email: ml@wikimedia.org
>>>>>>>>> - Phabricator: #Machine-Learning-Team tag
>>>>>>>>> - IRC (Libera): #wikimedia-ml
>>>>>>>>>
>>>>>>>>> The Machine Learning team is available to help projects migrate,
>>>>>>>>> from offering advice to making code commits. We want to make this as easy
>>>>>>>>> as possible for folks.
>>>>>>>>>
>>>>>>>>> High Level timeline:
>>>>>>>>>
>>>>>>>>> **By September 30th 2023: *Infrastructure powering the ORES API
>>>>>>>>> endpoint will be migrated from ORES to Lift Wing. For users, the API
>>>>>>>>> endpoint will remain the same, and most users won’t notice any change.
>>>>>>>>> Rather just the backend services powering the endpoint will change.
>>>>>>>>>
>>>>>>>>> Details: We'd like to add a DNS CNAME that points
>>>>>>>>> ores.wikimedia.org to ores-legacy.wikimedia.org, a new endpoint
>>>>>>>>> that offers a almost complete replacement of the ORES API calling Lift Wing
>>>>>>>>> behind the scenes. In an ideal world we'd migrate all tools to Lift Wing
>>>>>>>>> before decommissioning the infrastructure behind
>>>>>>>>> ores.wikimedia.org, but it turned out to be really challenging so
>>>>>>>>> to avoid disrupting users we chose to implement a transition layer/API.
>>>>>>>>>
>>>>>>>>> To summarize, if you don't have time to migrate before September
>>>>>>>>> to Lift Wing, your code/tool should work just fine on
>>>>>>>>> ores-legacy.wikimedia.org and you'll not have to change a line in
>>>>>>>>> your code thanks to the DNS CNAME. The ores-legacy endpoint is not a 100%
>>>>>>>>> replacement for ores, we removed some very old and not used features, so we
>>>>>>>>> highly recommend at least test the new endpoint for your use case to avoid
>>>>>>>>> surprises when we'll make the switch. In case you find anything weird,
>>>>>>>>> please report it to us using the aforementioned channels.
>>>>>>>>>
>>>>>>>>> **September to January: *We will be reaching out to every user of
>>>>>>>>> ORES we can identify and working with them to make the migration process as
>>>>>>>>> easy as possible.
>>>>>>>>>
>>>>>>>>> **By January 2024: *If all goes well, we would like zero traffic
>>>>>>>>> on the ORES API endpoint so we can turn off the ores-legacy API.
>>>>>>>>>
>>>>>>>>> If you want more information about Lift Wing, please check
>>>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>>>>>>>>
>>>>>>>>> Thanks in advance for the patience and the help!
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> The Machine Learning Team
>>>>>>>>> _______________________________________________
>>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>>> To unsubscribe send an email to
>>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>>
>>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>> To unsubscribe send an email to
>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>
>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>>
>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>
>>>>>> _______________________________________________
>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>
>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>
>>>>> _______________________________________________
>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>
>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>
>>>> _______________________________________________
>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>
>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>
>>> _______________________________________________
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> _______________________________________________
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

Let's discuss the issue in a Phabricator task, it seems more appropriate
than here (so other folks can chime in etc.. more easily).

From our traffic analysis there is no current client using model_info, so
we didn't add it to the feature set. We are working on an equivalent
solution in Lift Wing for all hosted models, not only revscoring ones, but
we don't have anything available now (a sort of "explainer" for the model's
metadata basically).

Luca

On Fri, Sep 22, 2023 at 6:01?PM Aaron Halfaker <aaron.halfaker@gmail.com>
wrote:

> It looks like model_info is not implemented at all. E.g.
> https://ores-legacy.wikimedia.org/v3/scores/enwiki?model_info=statistics.thresholds.true.%22maximum+recall+@+precision+%3E=+0.9%22&models=damaging
>
> I get {"detail":{"error":{"code":"bad request","message":"model_info
> query parameter is not supported by this endpoint anymore. For more
> information please visit https://wikitech.wikimedia.org/wiki/ORES"}}}
>
> But when I go to that page, nothing discusses model_info. Is there a way
> to get this from LiftWing?
>
> On Fri, Sep 22, 2023 at 8:53?AM Aaron Halfaker <aaron.halfaker@gmail.com>
> wrote:
>
>> Do you have a tag for filing bugs against ORES-legacy? I can't seem to
>> find a relevant one in phab.
>>
>> On Fri, Sep 22, 2023 at 8:39?AM Luca Toscano <ltoscano@wikimedia.org>
>> wrote:
>>
>>> Hi Aaron!
>>>
>>> Thanks for following up. The API is almost compatible with what ORES
>>> currently does, but there are limitations (like the max number of revisions
>>> in a batch etc..). The API clearly states when something is not supported,
>>> so you can check its compatibility now making some requests to:
>>>
>>> https://ores-legacy.wikimedia.org
>>>
>>> If you open a task with a list of systems that you need to migrate we
>>> can definitely take a look and help. So far the traffic being served by
>>> ORES has been reduced to few clients, and all of them don't run with
>>> recognizable UAs (see https://meta.wikimedia.org/wiki/User-Agent_policy)
>>> so we'll try our best to support them. The migration to Lift Wing has been
>>> widely publicized, a lot of documentation is available to migrate. We'd
>>> suggest trying Lift Wing for your systems instead (see
>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Usage).
>>>
>>> The Machine Learning plan is to eventually deprecate ores-legacy too, to
>>> maintain only one system (namely Lift Wing). There is no final date yet,
>>> we'll try to reach out to all remaining users first, so if you plan to keep
>>> using ores-legacy please follow up with us first :)
>>>
>>> Thanks!
>>>
>>> Luca (on behalf of the ML Team)
>>>
>>> On Fri, Sep 22, 2023 at 5:10?PM Aaron Halfaker <aaron.halfaker@gmail.com>
>>> wrote:
>>>
>>>> Does the new ores-legacy support the same feature set. E.g. features
>>>> output, injection, and threshold optimizations. Or is it just prediction?
>>>> This will affect some of the systems I need to migrate.
>>>>
>>>> On Fri, Sep 22, 2023, 06:21 Ilias Sarantopoulos <
>>>> isarantopoulos@wikimedia.org> wrote:
>>>>
>>>>> Hello!
>>>>>
>>>>>
>>>>> As a next step in the deprecation process of ORES
>>>>> https://wikitech.wikimedia.org/wiki/ORES the Machine Learning team will
>>>>> switch the backend of ores.wikimedia.org to ores-legacy, a k8s
>>>>> application meant to provide a compatibility layer between ORES and Lift
>>>>> Wing so users that have not yet migrated to Lift Wing will be
>>>>> transparently migrated. Ores-legacy is an application that has the same API
>>>>> as ORES but in the background makes requests to Lift Wing, allowing us to
>>>>> decommission the ORES servers until all clients have moved.
>>>>>
>>>>> This change is planned to take place on Monday 25th of September. If
>>>>> you have a client/application that is still using ORES we expect that this
>>>>> switch is going to be transparent for you.
>>>>>
>>>>> However keep in mind that ores-legacy is not a 100% replacement for
>>>>> ORES as some old and unused features are no longer supported.
>>>>>
>>>>> If you see anything out of the ordinary, feel free to contact the
>>>>> Machine Learning team:
>>>>>
>>>>> IRC libera: #wikimedia-ml
>>>>>
>>>>> Phabricator: Machine-Learning-team tag
>>>>>
>>>>> Thank you!
>>>>>
>>>>>
>>>>> On Wed, Aug 9, 2023 at 1:22?PM Chaloemphon Praphuchakang <
>>>>> yoshrakpraphu@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>> On Tue, 8 Aug 2023, 10:45 Tilman Bayer, <haebwiki@gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>> Hi Chris,
>>>>>>>
>>>>>>> On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Tilman,
>>>>>>>>
>>>>>>>> Most of the work is still very experimental. We have hosted a few
>>>>>>>> LLMs on Lift Wing already (StarCoder for example) but they were just
>>>>>>>> running on CPU, far too slow for real use cases. But it proves that we can
>>>>>>>> easily host LLMs on Lift Wing. We have been pretty quiet about it while we
>>>>>>>> focus on the ORES migration, but it is our next big project. More soon
>>>>>>>> hopefully!
>>>>>>>>
>>>>>>> Understood. Looking forward to learning more later!
>>>>>>>
>>>>>>>
>>>>>>>> Where we are now is that we have budget for a big GPU purchase
>>>>>>>> (~10-20 GPUs depending on cost), the question we will try to answer after
>>>>>>>> the ORES migration is complete is: what GPUs should we purchase? We are
>>>>>>>> trying to balance our strong preference to stay open source (i.e. AMD mROC)
>>>>>>>> in a world dominated by a single closed source vendor (i.e. Nvidia). In
>>>>>>>> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
>>>>>>>> H100, etc) or a mix of big and small? We will need to figure out all this.
>>>>>>>>
>>>>>>> I see. On that matter, what do you folks make of the recent
>>>>>>> announcements of AMD's partnerships with Hugging Face and Pytorch[5]?
>>>>>>> (which, I understand, came after the ML team had already launched the
>>>>>>> aforementioned new AMD explorations)
>>>>>>>
>>>>>>> "Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch
>>>>>>> to take on Nvidia [...]
>>>>>>> Both partnerships involve AMD’s ROCm AI software stack, the
>>>>>>> company’s answer to Nvidia’s proprietary CUDA platform and
>>>>>>> application-programming interface. AMD called ROCm an open and portable AI
>>>>>>> system with out-of-the-box support that can port to existing AI models.
>>>>>>> [...B]oth AMD and Hugging Face are dedicating engineering resources to each
>>>>>>> other and sharing data to ensure that the constantly updated AI models from
>>>>>>> Hugging Face, which might not otherwise run well on AMD hardware, would be
>>>>>>> “guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
>>>>>>> will fully upstream the ROCm software stack and “provide immediate ‘day
>>>>>>> zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
>>>>>>> accelerators,” which is meant to appeal to those customers looking to
>>>>>>> switch from Nvidia’s software ecosystem."
>>>>>>>
>>>>>>>
>>>>>>> In their own announcement, Hugging Face offered further details,
>>>>>>> including a pretty impressive list of models to be supported:[6]
>>>>>>>
>>>>>>>
>>>>>>> "We intend to support state-of-the-art transformer architectures for
>>>>>>> natural language processing, computer vision, and speech, such as BERT,
>>>>>>> DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
>>>>>>> generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
>>>>>>> LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
>>>>>>> support more traditional computer vision models, like ResNet and ResNext,
>>>>>>> and deep learning recommendation models, a first for us. [..] We'll do our
>>>>>>> best to test and validate these models for PyTorch, TensorFlow, and ONNX
>>>>>>> Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
>>>>>>> seamlessly in our open-source libraries, starting with the transformers
>>>>>>> library."
>>>>>>>
>>>>>>>
>>>>>>> Do you think this may promise too much, or could it point to a
>>>>>>> possible solution of the Foundation's conundrum?
>>>>>>> In any case, this seems to be an interesting moment where many in AI
>>>>>>> are trying to move away from Nvidia's proprietary CUDA platform. Most of
>>>>>>> them probably more for financial and availability reasons though, given the
>>>>>>> current GPU shortages[7] (which the ML team is undoubtedly aware of
>>>>>>> already; mentioning this as context for others on this list. See also
>>>>>>> Marketwatch's remarks about current margins[5]).
>>>>>>>
>>>>>>> Regards, Tilman
>>>>>>>
>>>>>>>
>>>>>>> [5]
>>>>>>> https://archive.ph/2023.06.15-173527/https://www.marketwatch.com/amp/story/open-source-ai-amd-looks-to-hugging-face-and-meta-spinoff-pytorch-to-take-on-nvidia-e4738f87
>>>>>>> [6] https://huggingface.co/blog/huggingface-and-amd
>>>>>>> [7] See e.g.
>>>>>>> https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/
>>>>>>> (avoid playing the song though. Don't say I didn't warn you)
>>>>>>>
>>>>>>>
>>>>>>>> I wouldn't characterize WMF's Language Team using CPU as because of
>>>>>>>> AMD, rather at the time we didn't have the budget for GPUs so Lift Wing
>>>>>>>> didn't have any. Since then we have moved two GPUs onto Lift Wing for
>>>>>>>> testing but they are pretty old (2017ish). Once we make the big GPU
>>>>>>>> purchase Lift Wing will gain a lot of functionality for LLM and similar
>>>>>>>> models.
>>>>>>>>
>>>>>>>> Chris
>>>>>>>>
>>>>>>>> On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi everybody,
>>>>>>>>>>
>>>>>>>>>> TL;DR We would like users of ORES models to migrate to our new
>>>>>>>>>> open source ML infrastructure, Lift Wing, within the next five months. We
>>>>>>>>>> are available to help you do that, from advice to making code commits. It
>>>>>>>>>> is important to note: All ML models currently accessible on ORES are also
>>>>>>>>>> currently accessible on Lift Wing.
>>>>>>>>>>
>>>>>>>>>> As part of the Machine Learning Modernization Project (
>>>>>>>>>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization),
>>>>>>>>>> the Machine Learning team has deployed a Wikimedia’s new machine learning
>>>>>>>>>> inference infrastructure, called Lift Wing (
>>>>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing).
>>>>>>>>>> Lift Wing brings a lot of new features such as support for GPU-based
>>>>>>>>>> models, open source LLM hosting, auto-scaling, stability, and ability to
>>>>>>>>>> host a larger number of models.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This sounds quite exciting! What's the best place to read up on
>>>>>>>>> that planned support for GPU-based models and open source LLMs? (I also saw
>>>>>>>>> in the recent NYT article[1] that the team is "in the process of adapting
>>>>>>>>> A.I. models that are 'off the shelf; — essentially models that have been
>>>>>>>>> made available by researchers for anyone to freely customize — so that
>>>>>>>>> Wikipedia’s editors can use them for their work.")
>>>>>>>>>
>>>>>>>>> I'm aware of the history[2] of not being able to use NVIDIA
>>>>>>>>> GPUs due to their CUDA drivers being proprietary. It was mentioned recently
>>>>>>>>> in the Wikimedia AI Telegram group that this is still a serious limitation,
>>>>>>>>> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
>>>>>>>>> WMF's Language team has resorted to using models without GPU support (CPU
>>>>>>>>> only).[4]
>>>>>>>>> It sounds like there is reasonable hope that this situation could
>>>>>>>>> change fairly soon? Would it also mean both at the same time, i.e. open
>>>>>>>>> source LLMs running with GPU support (considering that at least some
>>>>>>>>> well-known ones appear to require torch.cuda.is_available() == True for
>>>>>>>>> that)?
>>>>>>>>>
>>>>>>>>> Regards, Tilman
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
>>>>>>>>> [2]
>>>>>>>>> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
>>>>>>>>> [3] https://phabricator.wikimedia.org/T334583 etc.
>>>>>>>>> [4]
>>>>>>>>> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
>>>>>>>>> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental
>>>>>>>>> but, I understand, written to be deployable on WMF infrastructure)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> With the creation of Lift Wing, the team is turning its attention
>>>>>>>>>> to deprecating the current machine learning infrastructure, ORES. ORES
>>>>>>>>>> served us really well over the years, it was a successful project but it
>>>>>>>>>> came before radical changes in technology like Docker, Kubernetes and more
>>>>>>>>>> recently MLOps. The servers that run ORES are at the end of their planned
>>>>>>>>>> lifespan and so to save cost we are going to shut them down in early 2024.
>>>>>>>>>>
>>>>>>>>>> We have outlined a deprecation path on Wikitech (
>>>>>>>>>> https://wikitech.wikimedia.org/wiki/ORES), please read the page
>>>>>>>>>> if you are a maintainer of a tool or code that uses the ORES endpoint
>>>>>>>>>> https://ores.wikimedia.org/). If you have any doubt or if you
>>>>>>>>>> need assistance in migrating to Lift Wing, feel free to contact the ML team
>>>>>>>>>> via:
>>>>>>>>>>
>>>>>>>>>> - Email: ml@wikimedia.org
>>>>>>>>>> - Phabricator: #Machine-Learning-Team tag
>>>>>>>>>> - IRC (Libera): #wikimedia-ml
>>>>>>>>>>
>>>>>>>>>> The Machine Learning team is available to help projects migrate,
>>>>>>>>>> from offering advice to making code commits. We want to make this as easy
>>>>>>>>>> as possible for folks.
>>>>>>>>>>
>>>>>>>>>> High Level timeline:
>>>>>>>>>>
>>>>>>>>>> **By September 30th 2023: *Infrastructure powering the ORES API
>>>>>>>>>> endpoint will be migrated from ORES to Lift Wing. For users, the API
>>>>>>>>>> endpoint will remain the same, and most users won’t notice any change.
>>>>>>>>>> Rather just the backend services powering the endpoint will change.
>>>>>>>>>>
>>>>>>>>>> Details: We'd like to add a DNS CNAME that points
>>>>>>>>>> ores.wikimedia.org to ores-legacy.wikimedia.org, a new endpoint
>>>>>>>>>> that offers a almost complete replacement of the ORES API calling Lift Wing
>>>>>>>>>> behind the scenes. In an ideal world we'd migrate all tools to Lift Wing
>>>>>>>>>> before decommissioning the infrastructure behind
>>>>>>>>>> ores.wikimedia.org, but it turned out to be really challenging
>>>>>>>>>> so to avoid disrupting users we chose to implement a transition layer/API.
>>>>>>>>>>
>>>>>>>>>> To summarize, if you don't have time to migrate before September
>>>>>>>>>> to Lift Wing, your code/tool should work just fine on
>>>>>>>>>> ores-legacy.wikimedia.org and you'll not have to change a line
>>>>>>>>>> in your code thanks to the DNS CNAME. The ores-legacy endpoint is not a
>>>>>>>>>> 100% replacement for ores, we removed some very old and not used features,
>>>>>>>>>> so we highly recommend at least test the new endpoint for your use case to
>>>>>>>>>> avoid surprises when we'll make the switch. In case you find anything
>>>>>>>>>> weird, please report it to us using the aforementioned channels.
>>>>>>>>>>
>>>>>>>>>> **September to January: *We will be reaching out to every user
>>>>>>>>>> of ORES we can identify and working with them to make the migration process
>>>>>>>>>> as easy as possible.
>>>>>>>>>>
>>>>>>>>>> **By January 2024: *If all goes well, we would like zero traffic
>>>>>>>>>> on the ORES API endpoint so we can turn off the ores-legacy API.
>>>>>>>>>>
>>>>>>>>>> If you want more information about Lift Wing, please check
>>>>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>>>>>>>>>
>>>>>>>>>> Thanks in advance for the patience and the help!
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> The Machine Learning Team
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>>>> To unsubscribe send an email to
>>>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>>>
>>>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>>> To unsubscribe send an email to
>>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>>
>>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>> To unsubscribe send an email to
>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>
>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>>
>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>
>>>>>> _______________________________________________
>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>
>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>
>>>>> _______________________________________________
>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>
>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>
>>>> _______________________________________________
>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>
>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>
>>> _______________________________________________
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

We could definitely file a task. However, it does seem like highlighting
the features that will no longer be available is an appropriate topic for a
discussion about migration in a technical mailing list.

Is there a good reference for which features have been excluded from
ores-legacy? It looks like https://wikitech.wikimedia.org/wiki/ORES covers
some of the excluded features/models, but not all of them.

I see now that it looks like the RevertRisk model will be replacing
the *damaging
*and *goodfaith *models that differentiate intentional damage from
unintentional damage. There's a large body of research on why this is
valuable and important to the social functioning of the wikis. This
literature also discusses why being reverted is not a very good signal for
damage/vandalism and can lead to problems when used as a signal for
patrolling. Was there a community discussion about this deprecation that I
missed? I have some preliminary results (in press) that demonstrate that
the RevertRisk model performs significantly worse than the damaging and
goodfaith models in English Wikipedia for patrolling work. Do you have
documentation for how you evaluated this model and compared it to
damaging/goodfaith?

FWIW, from my reading of these announcement threads, I believed that
generally functionality and models would be preserved in
ores-legacy/LiftWing. This is the first time I've realized the scale of
what will become unavailable.

On Fri, Sep 22, 2023 at 9:07?AM Luca Toscano <ltoscano@wikimedia.org> wrote:

> Let's discuss the issue in a Phabricator task, it seems more appropriate
> than here (so other folks can chime in etc.. more easily).
>
> From our traffic analysis there is no current client using model_info, so
> we didn't add it to the feature set. We are working on an equivalent
> solution in Lift Wing for all hosted models, not only revscoring ones, but
> we don't have anything available now (a sort of "explainer" for the model's
> metadata basically).
>
> Luca
>
> On Fri, Sep 22, 2023 at 6:01?PM Aaron Halfaker <aaron.halfaker@gmail.com>
> wrote:
>
>> It looks like model_info is not implemented at all. E.g.
>> https://ores-legacy.wikimedia.org/v3/scores/enwiki?model_info=statistics.thresholds.true.%22maximum+recall+@+precision+%3E=+0.9%22&models=damaging
>>
>> I get {"detail":{"error":{"code":"bad request","message":"model_info
>> query parameter is not supported by this endpoint anymore. For more
>> information please visit https://wikitech.wikimedia.org/wiki/ORES"}}}
>>
>> But when I go to that page, nothing discusses model_info. Is there a way
>> to get this from LiftWing?
>>
>> On Fri, Sep 22, 2023 at 8:53?AM Aaron Halfaker <aaron.halfaker@gmail.com>
>> wrote:
>>
>>> Do you have a tag for filing bugs against ORES-legacy? I can't seem to
>>> find a relevant one in phab.
>>>
>>> On Fri, Sep 22, 2023 at 8:39?AM Luca Toscano <ltoscano@wikimedia.org>
>>> wrote:
>>>
>>>> Hi Aaron!
>>>>
>>>> Thanks for following up. The API is almost compatible with what ORES
>>>> currently does, but there are limitations (like the max number of revisions
>>>> in a batch etc..). The API clearly states when something is not supported,
>>>> so you can check its compatibility now making some requests to:
>>>>
>>>> https://ores-legacy.wikimedia.org
>>>>
>>>> If you open a task with a list of systems that you need to migrate we
>>>> can definitely take a look and help. So far the traffic being served by
>>>> ORES has been reduced to few clients, and all of them don't run with
>>>> recognizable UAs (see https://meta.wikimedia.org/wiki/User-Agent_policy)
>>>> so we'll try our best to support them. The migration to Lift Wing has been
>>>> widely publicized, a lot of documentation is available to migrate. We'd
>>>> suggest trying Lift Wing for your systems instead (see
>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Usage).
>>>>
>>>> The Machine Learning plan is to eventually deprecate ores-legacy too,
>>>> to maintain only one system (namely Lift Wing). There is no final date yet,
>>>> we'll try to reach out to all remaining users first, so if you plan to keep
>>>> using ores-legacy please follow up with us first :)
>>>>
>>>> Thanks!
>>>>
>>>> Luca (on behalf of the ML Team)
>>>>
>>>> On Fri, Sep 22, 2023 at 5:10?PM Aaron Halfaker <
>>>> aaron.halfaker@gmail.com> wrote:
>>>>
>>>>> Does the new ores-legacy support the same feature set. E.g. features
>>>>> output, injection, and threshold optimizations. Or is it just prediction?
>>>>> This will affect some of the systems I need to migrate.
>>>>>
>>>>> On Fri, Sep 22, 2023, 06:21 Ilias Sarantopoulos <
>>>>> isarantopoulos@wikimedia.org> wrote:
>>>>>
>>>>>> Hello!
>>>>>>
>>>>>>
>>>>>> As a next step in the deprecation process of ORES
>>>>>> https://wikitech.wikimedia.org/wiki/ORES the Machine Learning team will
>>>>>> switch the backend of ores.wikimedia.org to ores-legacy, a k8s
>>>>>> application meant to provide a compatibility layer between ORES and Lift
>>>>>> Wing so users that have not yet migrated to Lift Wing will be
>>>>>> transparently migrated. Ores-legacy is an application that has the same API
>>>>>> as ORES but in the background makes requests to Lift Wing, allowing us to
>>>>>> decommission the ORES servers until all clients have moved.
>>>>>>
>>>>>> This change is planned to take place on Monday 25th of September. If
>>>>>> you have a client/application that is still using ORES we expect that this
>>>>>> switch is going to be transparent for you.
>>>>>>
>>>>>> However keep in mind that ores-legacy is not a 100% replacement for
>>>>>> ORES as some old and unused features are no longer supported.
>>>>>>
>>>>>> If you see anything out of the ordinary, feel free to contact the
>>>>>> Machine Learning team:
>>>>>>
>>>>>> IRC libera: #wikimedia-ml
>>>>>>
>>>>>> Phabricator: Machine-Learning-team tag
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>>
>>>>>> On Wed, Aug 9, 2023 at 1:22?PM Chaloemphon Praphuchakang <
>>>>>> yoshrakpraphu@gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>> On Tue, 8 Aug 2023, 10:45 Tilman Bayer, <haebwiki@gmail.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Hi Chris,
>>>>>>>>
>>>>>>>> On Mon, Aug 7, 2023 at 11:51?AM Chris Albon <calbon@wikimedia.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Tilman,
>>>>>>>>>
>>>>>>>>> Most of the work is still very experimental. We have hosted a few
>>>>>>>>> LLMs on Lift Wing already (StarCoder for example) but they were just
>>>>>>>>> running on CPU, far too slow for real use cases. But it proves that we can
>>>>>>>>> easily host LLMs on Lift Wing. We have been pretty quiet about it while we
>>>>>>>>> focus on the ORES migration, but it is our next big project. More soon
>>>>>>>>> hopefully!
>>>>>>>>>
>>>>>>>> Understood. Looking forward to learning more later!
>>>>>>>>
>>>>>>>>
>>>>>>>>> Where we are now is that we have budget for a big GPU purchase
>>>>>>>>> (~10-20 GPUs depending on cost), the question we will try to answer after
>>>>>>>>> the ORES migration is complete is: what GPUs should we purchase? We are
>>>>>>>>> trying to balance our strong preference to stay open source (i.e. AMD mROC)
>>>>>>>>> in a world dominated by a single closed source vendor (i.e. Nvidia). In
>>>>>>>>> addition, do we go for a few expensive GPUs better suited to LLMs (A1000,
>>>>>>>>> H100, etc) or a mix of big and small? We will need to figure out all this.
>>>>>>>>>
>>>>>>>> I see. On that matter, what do you folks make of the recent
>>>>>>>> announcements of AMD's partnerships with Hugging Face and Pytorch[5]?
>>>>>>>> (which, I understand, came after the ML team had already launched the
>>>>>>>> aforementioned new AMD explorations)
>>>>>>>>
>>>>>>>> "Open-source AI: AMD looks to Hugging Face and Meta spinoff PyTorch
>>>>>>>> to take on Nvidia [...]
>>>>>>>> Both partnerships involve AMD’s ROCm AI software stack, the
>>>>>>>> company’s answer to Nvidia’s proprietary CUDA platform and
>>>>>>>> application-programming interface. AMD called ROCm an open and portable AI
>>>>>>>> system with out-of-the-box support that can port to existing AI models.
>>>>>>>> [...B]oth AMD and Hugging Face are dedicating engineering resources to each
>>>>>>>> other and sharing data to ensure that the constantly updated AI models from
>>>>>>>> Hugging Face, which might not otherwise run well on AMD hardware, would be
>>>>>>>> “guaranteed” to work on hardware like the MI300X. [...] AMD said PyTorch
>>>>>>>> will fully upstream the ROCm software stack and “provide immediate ‘day
>>>>>>>> zero’ support for PyTorch 2.0 with ROCm release 5.4.2 on all AMD Instinct
>>>>>>>> accelerators,” which is meant to appeal to those customers looking to
>>>>>>>> switch from Nvidia’s software ecosystem."
>>>>>>>>
>>>>>>>>
>>>>>>>> In their own announcement, Hugging Face offered further details,
>>>>>>>> including a pretty impressive list of models to be supported:[6]
>>>>>>>>
>>>>>>>>
>>>>>>>> "We intend to support state-of-the-art transformer architectures
>>>>>>>> for natural language processing, computer vision, and speech, such as BERT,
>>>>>>>> DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. Of course,
>>>>>>>> generative AI models will be available too (e.g., GPT2, GPT-NeoX, T5, OPT,
>>>>>>>> LLaMA), including our own BLOOM and StarCoder models. Lastly, we will also
>>>>>>>> support more traditional computer vision models, like ResNet and ResNext,
>>>>>>>> and deep learning recommendation models, a first for us. [..] We'll do our
>>>>>>>> best to test and validate these models for PyTorch, TensorFlow, and ONNX
>>>>>>>> Runtime for the above platforms. [...] We will integrate the AMD ROCm SDK
>>>>>>>> seamlessly in our open-source libraries, starting with the transformers
>>>>>>>> library."
>>>>>>>>
>>>>>>>>
>>>>>>>> Do you think this may promise too much, or could it point to a
>>>>>>>> possible solution of the Foundation's conundrum?
>>>>>>>> In any case, this seems to be an interesting moment where many in
>>>>>>>> AI are trying to move away from Nvidia's proprietary CUDA platform. Most of
>>>>>>>> them probably more for financial and availability reasons though, given the
>>>>>>>> current GPU shortages[7] (which the ML team is undoubtedly aware of
>>>>>>>> already; mentioning this as context for others on this list. See also
>>>>>>>> Marketwatch's remarks about current margins[5]).
>>>>>>>>
>>>>>>>> Regards, Tilman
>>>>>>>>
>>>>>>>>
>>>>>>>> [5]
>>>>>>>> https://archive.ph/2023.06.15-173527/https://www.marketwatch.com/amp/story/open-source-ai-amd-looks-to-hugging-face-and-meta-spinoff-pytorch-to-take-on-nvidia-e4738f87
>>>>>>>> [6] https://huggingface.co/blog/huggingface-and-amd
>>>>>>>> [7] See e.g.
>>>>>>>> https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/
>>>>>>>> (avoid playing the song though. Don't say I didn't warn you)
>>>>>>>>
>>>>>>>>
>>>>>>>>> I wouldn't characterize WMF's Language Team using CPU as because
>>>>>>>>> of AMD, rather at the time we didn't have the budget for GPUs so Lift Wing
>>>>>>>>> didn't have any. Since then we have moved two GPUs onto Lift Wing for
>>>>>>>>> testing but they are pretty old (2017ish). Once we make the big GPU
>>>>>>>>> purchase Lift Wing will gain a lot of functionality for LLM and similar
>>>>>>>>> models.
>>>>>>>>>
>>>>>>>>> Chris
>>>>>>>>>
>>>>>>>>> On Sun, Aug 6, 2023 at 9:57?PM Tilman Bayer <haebwiki@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> On Thu, Aug 3, 2023 at 7:16?AM Chris Albon <calbon@wikimedia.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi everybody,
>>>>>>>>>>>
>>>>>>>>>>> TL;DR We would like users of ORES models to migrate to our new
>>>>>>>>>>> open source ML infrastructure, Lift Wing, within the next five months. We
>>>>>>>>>>> are available to help you do that, from advice to making code commits. It
>>>>>>>>>>> is important to note: All ML models currently accessible on ORES are also
>>>>>>>>>>> currently accessible on Lift Wing.
>>>>>>>>>>>
>>>>>>>>>>> As part of the Machine Learning Modernization Project (
>>>>>>>>>>> https://www.mediawiki.org/wiki/Machine_Learning/Modernization),
>>>>>>>>>>> the Machine Learning team has deployed a Wikimedia’s new machine learning
>>>>>>>>>>> inference infrastructure, called Lift Wing (
>>>>>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing).
>>>>>>>>>>> Lift Wing brings a lot of new features such as support for GPU-based
>>>>>>>>>>> models, open source LLM hosting, auto-scaling, stability, and ability to
>>>>>>>>>>> host a larger number of models.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This sounds quite exciting! What's the best place to read up on
>>>>>>>>>> that planned support for GPU-based models and open source LLMs? (I also saw
>>>>>>>>>> in the recent NYT article[1] that the team is "in the process of adapting
>>>>>>>>>> A.I. models that are 'off the shelf; — essentially models that have been
>>>>>>>>>> made available by researchers for anyone to freely customize — so that
>>>>>>>>>> Wikipedia’s editors can use them for their work.")
>>>>>>>>>>
>>>>>>>>>> I'm aware of the history[2] of not being able to use NVIDIA
>>>>>>>>>> GPUs due to their CUDA drivers being proprietary. It was mentioned recently
>>>>>>>>>> in the Wikimedia AI Telegram group that this is still a serious limitation,
>>>>>>>>>> despite some new explorations with AMD GPUs[3] - to the point that e.g. the
>>>>>>>>>> WMF's Language team has resorted to using models without GPU support (CPU
>>>>>>>>>> only).[4]
>>>>>>>>>> It sounds like there is reasonable hope that this situation could
>>>>>>>>>> change fairly soon? Would it also mean both at the same time, i.e. open
>>>>>>>>>> source LLMs running with GPU support (considering that at least some
>>>>>>>>>> well-known ones appear to require torch.cuda.is_available() == True for
>>>>>>>>>> that)?
>>>>>>>>>>
>>>>>>>>>> Regards, Tilman
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>> https://www.nytimes.com/2023/07/18/magazine/wikipedia-ai-chatgpt.html
>>>>>>>>>> [2]
>>>>>>>>>> https://techblog.wikimedia.org/2020/04/06/saying-no-to-proprietary-code-in-production-is-hard-work-the-gpu-chapter/
>>>>>>>>>> [3] https://phabricator.wikimedia.org/T334583 etc.
>>>>>>>>>> [4]
>>>>>>>>>> https://diff.wikimedia.org/2023/06/13/mint-supporting-underserved-languages-with-open-machine-translation/
>>>>>>>>>> or https://thottingal.in/blog/2023/07/21/wikiqa/ (experimental
>>>>>>>>>> but, I understand, written to be deployable on WMF infrastructure)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> With the creation of Lift Wing, the team is turning its
>>>>>>>>>>> attention to deprecating the current machine learning infrastructure, ORES.
>>>>>>>>>>> ORES served us really well over the years, it was a successful project but
>>>>>>>>>>> it came before radical changes in technology like Docker, Kubernetes and
>>>>>>>>>>> more recently MLOps. The servers that run ORES are at the end of their
>>>>>>>>>>> planned lifespan and so to save cost we are going to shut them down in
>>>>>>>>>>> early 2024.
>>>>>>>>>>>
>>>>>>>>>>> We have outlined a deprecation path on Wikitech (
>>>>>>>>>>> https://wikitech.wikimedia.org/wiki/ORES), please read the page
>>>>>>>>>>> if you are a maintainer of a tool or code that uses the ORES endpoint
>>>>>>>>>>> https://ores.wikimedia.org/). If you have any doubt or if you
>>>>>>>>>>> need assistance in migrating to Lift Wing, feel free to contact the ML team
>>>>>>>>>>> via:
>>>>>>>>>>>
>>>>>>>>>>> - Email: ml@wikimedia.org
>>>>>>>>>>> - Phabricator: #Machine-Learning-Team tag
>>>>>>>>>>> - IRC (Libera): #wikimedia-ml
>>>>>>>>>>>
>>>>>>>>>>> The Machine Learning team is available to help projects migrate,
>>>>>>>>>>> from offering advice to making code commits. We want to make this as easy
>>>>>>>>>>> as possible for folks.
>>>>>>>>>>>
>>>>>>>>>>> High Level timeline:
>>>>>>>>>>>
>>>>>>>>>>> **By September 30th 2023: *Infrastructure powering the ORES API
>>>>>>>>>>> endpoint will be migrated from ORES to Lift Wing. For users, the API
>>>>>>>>>>> endpoint will remain the same, and most users won’t notice any change.
>>>>>>>>>>> Rather just the backend services powering the endpoint will change.
>>>>>>>>>>>
>>>>>>>>>>> Details: We'd like to add a DNS CNAME that points
>>>>>>>>>>> ores.wikimedia.org to ores-legacy.wikimedia.org, a new endpoint
>>>>>>>>>>> that offers a almost complete replacement of the ORES API calling Lift Wing
>>>>>>>>>>> behind the scenes. In an ideal world we'd migrate all tools to Lift Wing
>>>>>>>>>>> before decommissioning the infrastructure behind
>>>>>>>>>>> ores.wikimedia.org, but it turned out to be really challenging
>>>>>>>>>>> so to avoid disrupting users we chose to implement a transition layer/API.
>>>>>>>>>>>
>>>>>>>>>>> To summarize, if you don't have time to migrate before September
>>>>>>>>>>> to Lift Wing, your code/tool should work just fine on
>>>>>>>>>>> ores-legacy.wikimedia.org and you'll not have to change a line
>>>>>>>>>>> in your code thanks to the DNS CNAME. The ores-legacy endpoint is not a
>>>>>>>>>>> 100% replacement for ores, we removed some very old and not used features,
>>>>>>>>>>> so we highly recommend at least test the new endpoint for your use case to
>>>>>>>>>>> avoid surprises when we'll make the switch. In case you find anything
>>>>>>>>>>> weird, please report it to us using the aforementioned channels.
>>>>>>>>>>>
>>>>>>>>>>> **September to January: *We will be reaching out to every user
>>>>>>>>>>> of ORES we can identify and working with them to make the migration process
>>>>>>>>>>> as easy as possible.
>>>>>>>>>>>
>>>>>>>>>>> **By January 2024: *If all goes well, we would like zero
>>>>>>>>>>> traffic on the ORES API endpoint so we can turn off the ores-legacy API.
>>>>>>>>>>>
>>>>>>>>>>> If you want more information about Lift Wing, please check
>>>>>>>>>>> https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
>>>>>>>>>>>
>>>>>>>>>>> Thanks in advance for the patience and the help!
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> The Machine Learning Team
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>>>>> To unsubscribe send an email to
>>>>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>>>>
>>>>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>>>> To unsubscribe send an email to
>>>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>>>
>>>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>>> To unsubscribe send an email to
>>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>>
>>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>>> To unsubscribe send an email to
>>>>>>>> wikitech-l-leave@lists.wikimedia.org
>>>>>>>>
>>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>>
>>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>>
>>>>>> _______________________________________________
>>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>>
>>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>>
>>>>> _______________________________________________
>>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>>
>>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>>
>>>> _______________________________________________
>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>>>
>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>>
>>> _______________________________________________
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

On Fri, Sep 22, 2023 at 8:59?PM Aaron Halfaker <aaron.halfaker@gmail.com>
wrote:

> We could definitely file a task. However, it does seem like highlighting
> the features that will no longer be available is an appropriate topic for a
> discussion about migration in a technical mailing list.
>

A specific question related to a functionality is the topic for a task, I
don't think that we should discuss every detail that differs from the ORES
API (Wikitech-l doesn't seem a good medium for it). We are already
following up on Phabricator, let's use tasks if possible to keep the
conversation as light and targeted as possible.

Is there a good reference for which features have been excluded from
> ores-legacy? It looks like https://wikitech.wikimedia.org/wiki/ORES covers
> some of the excluded features/models, but not all of them.
>

We spent the last months helping the community to migrate away from the
ORES API (to use Lift Wing instead), the remaining traffic is only related
to few low traffic IPs that we are not able to contact. We didn't add
feature injection or threshold optimization to ores-legacy, for example,
since there was no indication on our logs that users were relying on it. We
have always stated everywhere (including all emails sent in this mailing
list) that we are 100% open to add a functionality if it is backed up by a
valid use case.


> I see now that it looks like the RevertRisk model will be replacing the *damaging
> *and *goodfaith *models that differentiate intentional damage from
> unintentional damage. There's a large body of research on why this is
> valuable and important to the social functioning of the wikis. This
> literature also discusses why being reverted is not a very good signal for
> damage/vandalism and can lead to problems when used as a signal for
> patrolling. Was there a community discussion about this deprecation that I
> missed? I have some preliminary results (in press) that demonstrate that
> the RevertRisk model performs significantly worse than the damaging and
> goodfaith models in English Wikipedia for patrolling work. Do you have
> documentation for how you evaluated this model and compared it to
> damaging/goodfaith?
>

We have model cards related to both Revert Risk models, all of them linked
in the API portal docs (more info:
https://api.wikimedia.org/wiki/Lift_Wing_API). All the community folks that
migrated their bots/tools/etc.. to Revert Risk were very happy about the
change, and we haven't had any request to switch back since then.

The ML team provides all the models deployed on ORES on Lift Wing, so any
damaging and goodfaith variant is available in the new API. We chose to not
pursue the development of those models for several reasons:
- We haven't had any indication/request from the community about those
models in almost two years, except few Phabricator updates that we followed
up on.
- Managing several hundreds models for goodfaith and damaging is not very
scalable in a modern micro-service architecture like Lift Wing (since we
have a model for each supported wiki). We (both Research and ML) are
oriented on having fewer models that manage more languages at the same
time, and this is the direction that we are following at the moment. It may
not be the perfect one but so far it seems a good choice. If you want to
chime in and provide your inputs we are 100% available in hearing
suggestions/concerns/doubts/recommendations/etc.., please follow up in any
of our channels (IRC, mailing lists, Phabricator for example).
- Last but not the least, most of the damaging/goodfaith models have been
trained with data coming from years ago, and never re-trained. The efforts
to keep several hundreds models up-to-date with recent data versus doing
the same of few models (like revert risk) weights in favor of the latter
for a relatively small team of engineers like us.


> FWIW, from my reading of these announcement threads, I believed that
> generally functionality and models would be preserved in
> ores-legacy/LiftWing. This is the first time I've realized the scale of
> what will become unavailable.
>

This is the part that I don't get, since as mentioned before all the models
that currently run on ORES are available in both ores-legacy and Lift Wing.
What changes is that we don't expose anymore functionality that logs
clearly show are not used, and that would need to be maintained and
improved over time. We are open to improve and add any requirement that the
community needs, the only thing that we ask is to provide a valid use case
to support it.

I do think that Lift Wing is a great improvement for the community, we have
been working with all the folks that reached out to us, without hiding
anything (including deprecation plans and path forwards).

Thanks for following up!

Luca

All fine points. As you can see, I've filed some phab tasks where I saw a
clear opportunity to do so.

> as mentioned before all the models that currently run on ORES are
available in both ores-legacy and Lift Wing.

I thought I read that damaging and goodfaith models are going to be
replaced. Should I instead read that they are likely to remain available
for the foreseeable future? When I asked about a community discussion
about the transition from damaging/goodfaith to revertrisk, I was imagining
that many people who use those predictions might have an opinion about them
going away. E.g. people who use the relevant filters in RecentChanges.
Maybe I missed the discussions about that.

I haven't seen a mention of the article quality or article topic models in
the docs. Are those also going to remain available? I have some user
scripts that use these models and are relatively widely used. I didn't
notice anyone reaching out. ... So I checked and setting a User-Agent on my
user scripts doesn't actually change the User-Agent. I've read that you
need to set "Api-User-Agent" instead, but that causes a CORS error when
querying ORES. I'll file a bug.

On Fri, Sep 22, 2023 at 1:22?PM Luca Toscano <ltoscano@wikimedia.org> wrote:

>
>
> On Fri, Sep 22, 2023 at 8:59?PM Aaron Halfaker <aaron.halfaker@gmail.com>
> wrote:
>
>> We could definitely file a task. However, it does seem like highlighting
>> the features that will no longer be available is an appropriate topic for a
>> discussion about migration in a technical mailing list.
>>
>
> A specific question related to a functionality is the topic for a task, I
> don't think that we should discuss every detail that differs from the ORES
> API (Wikitech-l doesn't seem a good medium for it). We are already
> following up on Phabricator, let's use tasks if possible to keep the
> conversation as light and targeted as possible.
>
> Is there a good reference for which features have been excluded from
>> ores-legacy? It looks like https://wikitech.wikimedia.org/wiki/ORES covers
>> some of the excluded features/models, but not all of them.
>>
>
> We spent the last months helping the community to migrate away from the
> ORES API (to use Lift Wing instead), the remaining traffic is only related
> to few low traffic IPs that we are not able to contact. We didn't add
> feature injection or threshold optimization to ores-legacy, for example,
> since there was no indication on our logs that users were relying on it. We
> have always stated everywhere (including all emails sent in this mailing
> list) that we are 100% open to add a functionality if it is backed up by a
> valid use case.
>
>
>> I see now that it looks like the RevertRisk model will be replacing the *damaging
>> *and *goodfaith *models that differentiate intentional damage from
>> unintentional damage. There's a large body of research on why this is
>> valuable and important to the social functioning of the wikis. This
>> literature also discusses why being reverted is not a very good signal for
>> damage/vandalism and can lead to problems when used as a signal for
>> patrolling. Was there a community discussion about this deprecation that I
>> missed? I have some preliminary results (in press) that demonstrate that
>> the RevertRisk model performs significantly worse than the damaging and
>> goodfaith models in English Wikipedia for patrolling work. Do you have
>> documentation for how you evaluated this model and compared it to
>> damaging/goodfaith?
>>
>
> We have model cards related to both Revert Risk models, all of them linked
> in the API portal docs (more info:
> https://api.wikimedia.org/wiki/Lift_Wing_API). All the community folks
> that migrated their bots/tools/etc.. to Revert Risk were very happy about
> the change, and we haven't had any request to switch back since then.
>
> The ML team provides all the models deployed on ORES on Lift Wing, so any
> damaging and goodfaith variant is available in the new API. We chose to not
> pursue the development of those models for several reasons:
> - We haven't had any indication/request from the community about those
> models in almost two years, except few Phabricator updates that we followed
> up on.
> - Managing several hundreds models for goodfaith and damaging is not very
> scalable in a modern micro-service architecture like Lift Wing (since we
> have a model for each supported wiki). We (both Research and ML) are
> oriented on having fewer models that manage more languages at the same
> time, and this is the direction that we are following at the moment. It may
> not be the perfect one but so far it seems a good choice. If you want to
> chime in and provide your inputs we are 100% available in hearing
> suggestions/concerns/doubts/recommendations/etc.., please follow up in any
> of our channels (IRC, mailing lists, Phabricator for example).
> - Last but not the least, most of the damaging/goodfaith models have been
> trained with data coming from years ago, and never re-trained. The efforts
> to keep several hundreds models up-to-date with recent data versus doing
> the same of few models (like revert risk) weights in favor of the latter
> for a relatively small team of engineers like us.
>
>
>> FWIW, from my reading of these announcement threads, I believed that
>> generally functionality and models would be preserved in
>> ores-legacy/LiftWing. This is the first time I've realized the scale of
>> what will become unavailable.
>>
>
> This is the part that I don't get, since as mentioned before all the
> models that currently run on ORES are available in both ores-legacy and
> Lift Wing. What changes is that we don't expose anymore functionality that
> logs clearly show are not used, and that would need to be maintained and
> improved over time. We are open to improve and add any requirement that the
> community needs, the only thing that we ask is to provide a valid use case
> to support it.
>
> I do think that Lift Wing is a great improvement for the community, we
> have been working with all the folks that reached out to us, without hiding
> anything (including deprecation plans and path forwards).
>
> Thanks for following up!
>
> Luca
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/