Mailing List Archive

Re: [Wikitech-l] Why don't we re-encode proprietary formats as Ogg?
I am definitely not opposed to adding in that functionality as I have
mentioned in the past:
see thread:
http://www.mail-archive.com/wikitech-l@lists.wikimedia.org/msg00888.html

You should take a look at the work Mike Baynton did back in summer of
code 07.

The issue that we have is both the Bottlenecks you mentioned. Where
possible we want to crowd source the transcoding costs and we have a
working firefogg support which we can more aggressively push out once
firefox 3.5 lands.

Essentially wikimedia commons is not designed to support hosting the raw
footage. Other like-minded organizations like archive.org that have
peta-bytes of storage across thousands of storage nodes are better
positioned to act as a host for raw footage and its derivatives.
Additional commons is a "strict" archive where files not licensed
properly often get removed where archive.org can act as a more permanent
storage space while license issues are sorted out.

Since wikimedia projects will shortly be supporting linline searching
time segment grabbing from archive.org material its maybe not so
critical that we create and host the transcoding infrastructure ourselves.

Although as you mention it would be nice to support transocoding on
wikimedias servers for uploading short clips from cell-phone type cases.

--michael

David Gerard wrote:
> [cc'd back to wikitech-l]
>
> 2009/6/8 Tim Starling <tstarling@wikimedia.org>:
>
>
>> It's been discussed since OggHandler was invented in 2007, and I've
>> always been in favour of it. But the code hasn't materialised, despite
>> a Google Summer of Code project come and gone that was meant to
>> implement a transcoding queue. The transcoding queue project was meant
>> to allow transformations in quality and size, but it would also allow
>> format changes without much trouble.
>>
>
>
> Ahhh, that's fantastic, so it is just a Simple Matter of Programming :-D
>
> (I'm tempted to bodge something together myself, despite my low
> opinion of my own coding abilities ;-) )
>
> Start simple. "Upload your phone and camera video files! We'll
> transcode them into Theora and store them." Pick suitable (tweakable)
> defaults. Get it doing that one job. Then we can think about
> size/quality transformations later. Sound like a vague plan?
>
> Bottlenecks: 1. CPU to transcode with. 2. Disk space for queued video.
>
>
> - d.
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>


_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Wikitech-l] Why don't we re-encode proprietary formats as Ogg? [ In reply to ]
I presume the WMF has a large amount of free disk space. How much?

On Mon, Jun 8, 2009 at 2:28 PM, Michael Dale <mdale@wikimedia.org> wrote:

> I am definitely not opposed to adding in that functionality as I have
> mentioned in the past:
> see thread:
> http://www.mail-archive.com/wikitech-l@lists.wikimedia.org/msg00888.html
>
> You should take a look at the work Mike Baynton did back in summer of
> code 07.
>
> The issue that we have is both the Bottlenecks you mentioned. Where
> possible we want to crowd source the transcoding costs and we have a
> working firefogg support which we can more aggressively push out once
> firefox 3.5 lands.
>
> Essentially wikimedia commons is not designed to support hosting the raw
> footage. Other like-minded organizations like archive.org that have
> peta-bytes of storage across thousands of storage nodes are better
> positioned to act as a host for raw footage and its derivatives.
> Additional commons is a "strict" archive where files not licensed
> properly often get removed where archive.org can act as a more permanent
> storage space while license issues are sorted out.
>
> Since wikimedia projects will shortly be supporting linline searching
> time segment grabbing from archive.org material its maybe not so
> critical that we create and host the transcoding infrastructure ourselves.
>
> Although as you mention it would be nice to support transocoding on
> wikimedias servers for uploading short clips from cell-phone type cases.
>
> --michael
>
> David Gerard wrote:
> > [cc'd back to wikitech-l]
> >
> > 2009/6/8 Tim Starling <tstarling@wikimedia.org>:
> >
> >
> >> It's been discussed since OggHandler was invented in 2007, and I've
> >> always been in favour of it. But the code hasn't materialised, despite
> >> a Google Summer of Code project come and gone that was meant to
> >> implement a transcoding queue. The transcoding queue project was meant
> >> to allow transformations in quality and size, but it would also allow
> >> format changes without much trouble.
> >>
> >
> >
> > Ahhh, that's fantastic, so it is just a Simple Matter of Programming :-D
> >
> > (I'm tempted to bodge something together myself, despite my low
> > opinion of my own coding abilities ;-) )
> >
> > Start simple. "Upload your phone and camera video files! We'll
> > transcode them into Theora and store them." Pick suitable (tweakable)
> > defaults. Get it doing that one job. Then we can think about
> > size/quality transformations later. Sound like a vague plan?
> >
> > Bottlenecks: 1. CPU to transcode with. 2. Disk space for queued video.
> >
> >
> > - d.
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
>
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Wikitech-l] Why don't we re-encode proprietary formats as Ogg? [ In reply to ]
2009/6/8 Brian <Brian.Mingus@colorado.edu>:
> I presume the WMF has a large amount of free disk space. How much?

Hard drives are cheap, the WMF can just buy more if that is all that is needed.

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Wikitech-l] Why don't we re-encode proprietary formats as Ogg? [ In reply to ]
I don't think that's all that's needed. There will be Wikimedians scouring
the Internet for all free video in all of its forms (of which there is quite
a lot) and uploading it to Commons. You'll need an entire encoding farm.
Hard drives are cheap, its true, but redundant storage is less cheap as a
function of reliability.

On Mon, Jun 8, 2009 at 3:17 PM, Thomas Dalton <thomas.dalton@gmail.com>wrote:

> 2009/6/8 Brian <Brian.Mingus@colorado.edu>:
> > I presume the WMF has a large amount of free disk space. How much?
>
> Hard drives are cheap, the WMF can just buy more if that is all that is
> needed.
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Wikitech-l] Why don't we re-encode proprietary formats as Ogg? [ In reply to ]
2009/6/8 Brian <Brian.Mingus@colorado.edu>:

> I don't think that's all that's needed. There will be Wikimedians scouring
> the Internet for all free video in all of its forms (of which there is quite
> a lot) and uploading it to Commons. You'll need an entire encoding farm.
> Hard drives are cheap, its true, but redundant storage is less cheap as a
> function of reliability.


Yes, that's true - easily re-encoding proprietary formats to an Ogg on
Commonsn will lead to people loading Commons with lovely video.

Mind you, the Wikimedians could do that with Firefogg.

OTOH, hands up all those who have FF 3.5? (raises hand) And who's
installed Firefogg? (puts hand down)


- d.

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l