Mailing List Archive

[Wikimedia-l] Re: 23 March: Invitation to Open Community Call on ChatGPT, generative AI, and Wikimedia
Some interesting updates on current developments

Open Assistant is open source ChatGPT clone with crowdsourced fine-tuning
- https://open-assistant.io

Redpajama is project for reproducing LLaMA and releasing the model under
open source licence. Current status is that they have released the
pre-training data
- https://www.together.xyz/blog/redpajama

Free-dolly is CC-BY-SA licenced fine-tuning dataset.
-
https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm

LLM chat which runs in web browser (non-open source Vicuna-7B)
- https://simonwillison.net/2023/Apr/16/web-llm/

Afaik all of these are clear steps towards full open source LLM stack.
Especially Open Assistant is especially interesting as it is focusing on
crowdsourcing.

Br,
-- Kimmo Virtanen, Zache

On Mon, Apr 3, 2023 at 8:43?PM Samuel Klein <meta.sj@gmail.com> wrote:

> At this point I guess I would recommend adding five or so
>> g2.cores8.ram36.disk20 flavor VPSs to WMCS, with between one and three
>> RTX A6000 GPUs each, plus a 1TB SSD each, which should cost under
>> $60k. That should allow for very widely multilingual models somewhere
>> between GPT-3.5 and 4 performance with current training rates.
>>
>
> Having part of the cluster for this makes sense, even as what it is used
> for changes over time.
>
>
>> These models can be quantized into int4 weights which run on cell
>> phones:
>> https://github.com/rupeshs/alpaca.cpp/tree/linux-android-build-support
>> It seems inevitable that we will someday include such LLMs with
>> Internet-in-a-Box, and, why not also the primary mobile apps
>>
>
> Eventually, yes. A good reason to renew attention to mobile as a canonical
> wiki experience.
>
>
> _______________________________________________
> Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines
> at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l
> Public archives at
> https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/R2KSXD3VE4MPGSIIYCPUMFXFPIY7D5CH/
> To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org