Mailing List Archive

On Sep 18, 2017, at 1:58 PM, Max Semenik <maxsem.wiki@gmail.com> wrote:
>
> Today, the HHVM developers made an announcement[1] that they have plans of
> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> instead.
>
> While this does not mean that we need to take an action immediately,
> eventually we will have to decide something. As I see it, our options are:
>
> 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> PHP. This, however, is a dead end and will make things progressively harder
> as the implementations will diverge and various Composer libraries we use
> will start requiring some Zend-specific features.
>
> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
>
> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.

Do we have performance benchmarks on 5.3 vs. HHVM vs. 5.6 vs. 7?

>
> I personally think that 3) is the only viable option in the long run. What
> do you think?
>
> ----
> [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
>
> --
> Best regards,
> Max Semenik ([[User:MaxSem]])
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

I think option 2 will be impossible. As that will decrease the user base of mediawiki by a lot.
I think we should go with option 3 as php 7 has had increased in performances. On Monday, 18 September 2017, 22:03:20 BST, James Hare <jamesmhare@gmail.com> wrote:

On Sep 18, 2017, at 1:58 PM, Max Semenik <maxsem.wiki@gmail.com> wrote:
>
> Today, the HHVM developers made an announcement[1] that they have plans of
> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> instead.
>
> While this does not mean that we need to take an action immediately,
> eventually we will have to decide something. As I see it, our options are:
>
> 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> PHP. This, however, is a dead end and will make things progressively harder
> as the implementations will diverge and various Composer libraries we use
> will start requiring some Zend-specific features.
>
> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
>
> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.

Do we have performance benchmarks on 5.3 vs. HHVM vs. 5.6 vs. 7?

>
> I personally think that 3) is the only viable option in the long run. What
> do you think?
>
> ----
> [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
>
> --
> Best regards,
> Max Semenik ([[User:MaxSem]])
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Mon, Sep 18, 2017 at 4:58 PM, Max Semenik <maxsem.wiki@gmail.com> wrote:

> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
>

I actually predicted this and wrote up my suggestions here:
https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/Examples#Example_of_a_.22medium_level.22_position_statement

I will *not* be making this my position statement at the dev summit -- only
one per person, so many positions to take, this one didn't make the cut! --
but I would like to strongly encourage someone to submit a position
statement relating to this issue (could be any of max's three positions) to
the dev summit. I think it is an important issue for our foundation and
community to discuss; certainly very relevant to the next 15 years of the
project.
--scott
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Mon, Sep 18, 2017 at 4:58 PM, Max Semenik <maxsem.wiki@gmail.com> wrote:

> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
> while 5.6 and 7 provided nice performance improvements.
>

In particular, I've heard good things about PHP 7 performance. Someone less
lazy than I am at 5pm might want to do some research on that though.

I can say that PHP 7 locally runs unit tests significantly faster than PHP
5.6, although that's not really a representative workload for running a
website.


--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Monday, September 18, 2017, Max Semenik <maxsem.wiki@gmail.com> wrote:
> Today, the HHVM developers made an announcement[1] that they have plans of
> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> instead.
>
> While this does not mean that we need to take an action immediately,
> eventually we will have to decide something. As I see it, our options are:
>
> 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> PHP. This, however, is a dead end and will make things progressively
harder
> as the implementations will diverge and various Composer libraries we use
> will start requiring some Zend-specific features.
>
> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
>
> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
while
> 5.6 and 7 provided nice performance improvements.
>
> I personally think that 3) is the only viable option in the long run. What
> do you think?
>
> ----
> [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
>
> --
> Best regards,
> Max Semenik ([[User:MaxSem]])
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Well i agree that 3 seems likely the best long term option, we will
probably be doing 1 in the short term. However I think it would be prudent
to wait and see how things turn out in the short term before comitting to
any path. The landscape can still shift quite a lot before the time comes
when its impractical to continue doing 1.

--
bawolff
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Other tools we are using, such as Phabricator, will also be following HHVM
to Hack (presumably). Facebook, as the largest (by engineering budget)
production user of PHP, will certainly have an outsized influence on the
direction of the platform and surrounding ecosystem. If we follow path #3
we're probably also committing to supporting Zend development long-term as
the primary production user.
--scott

On Mon, Sep 18, 2017 at 5:26 PM, Brian Wolff <bawolff@gmail.com> wrote:

> On Monday, September 18, 2017, Max Semenik <maxsem.wiki@gmail.com> wrote:
> > Today, the HHVM developers made an announcement[1] that they have plans
> of
> > ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> > instead.
> >
> > While this does not mean that we need to take an action immediately,
> > eventually we will have to decide something. As I see it, our options
> are:
> >
> > 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> > PHP. This, however, is a dead end and will make things progressively
> harder
> > as the implementations will diverge and various Composer libraries we use
> > will start requiring some Zend-specific features.
> >
> > 2) Declare our loyalty to HHVM. This will result in most of our current
> > users being unable to upgrade, eventually producing what amounts to a
> > WMF-only product and lots of installations with outdated MediaWiki having
> > security holes. At least we will be able to convert to Hack eventually.
> > This is a very clean-cut case of vendor lock-in though, and if Facebook
> > decides to switch their code base to something shinier, we'll be deep in
> > trouble.
> >
> > 3) Revert WMF to Zend and forget about HHVM. This will result in
> > performance degradation, however it will not be that dramatic: when we
> > upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
> while
> > 5.6 and 7 provided nice performance improvements.
> >
> > I personally think that 3) is the only viable option in the long run.
> What
> > do you think?
> >
> > ----
> > [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
> >
> > --
> > Best regards,
> > Max Semenik ([[User:MaxSem]])
> > _______________________________________________
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> Well i agree that 3 seems likely the best long term option, we will
> probably be doing 1 in the short term. However I think it would be prudent
> to wait and see how things turn out in the short term before comitting to
> any path. The landscape can still shift quite a lot before the time comes
> when its impractical to continue doing 1.
>
> --
> bawolff
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Mon, Sep 18, 2017 at 2:33 PM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Other tools we are using, such as Phabricator, will also be following HHVM
> to Hack (presumably).


To the contrary, Phabricator has never supported HHVM.

--
Best regards,
Max Semenik ([[User:MaxSem]])
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> PHP. This, however, is a dead end and will make things progressively harder
> as the implementations will diverge and various Composer libraries we use
> will start requiring some Zend-specific features.

This will probably ultimately happen, but given the PHP version stats,
e.g. here:
https://seld.be/notes/php-versions-stats-2017-1-edition

I think we have several years at least before that starts becoming an
issue. Realistically, if you write distributable PHP code now, targeting
7.1 gets you only 17% of the users, so you'd do either 7.0 (gets you
about half) or more likely even 5.6. Extending this trend (I know,
dangerous, but let's assume) if 7.2 is released somewhere around Dec
2017-Jan 2018, 7.3 would probably not happen before around 2019. If that
would have features not supported in HHVM, that means we'd have to worry
around 2021 when people would start releasing components targeting it.
So we have about 3 years to get the solution - *if* 7.3 has features not
supported by HHVM.

Note that this statistics is for Composer users, which means it is
probably skewed towards modern versions, since people using PHP 5.3
probably don't use Composer too much in general. OTOH, since we do use
Composer, that appears to be appropriate for our case.

> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.

I don't think this is a good idea, for reasons that seem obvious to me
(but I can elaborate if necessary).

> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.

I think we should evaluate 7.1 or 7.2 (provided we don't have any
runtime issues with them) and see how performance looks like ASAP (with
opcache, of course). Of there's some help needed, or there are some
specific issues that are blockers, I think Zend team would be glad to
talk to us. If needed, I could probably help with establishing the
contacts.
--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> Other tools we are using, such as Phabricator, will also be following HHVM
> to Hack (presumably).

I am not sure this is the case. Mozilla recently declared they want to
use Phabricator[1], but I heard no mention about HHVM. Which makes me
think that one stays on PHP. Also, Phabricator is now independent from
Facebook, afaik, since it's developers have separate company, Phacility.

[1] https://wiki.mozilla.org/Phabricator

--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> I can say that PHP 7 locally runs unit tests significantly faster than PHP
> 5.6, although that's not really a representative workload for running a
> website.

PHP 7 is faster than 5.6, and probably be on almost any workload, from
my experience (the degree of speedup would vary of course). As for
comparison with hhvm, I heard various reports, but I think spending some
time on seriously testing it (I mean creating proper production setup,
and directing either captured/replayed or real traffic to it) is the
best way to go.
--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hey,

Going with option 2 would be a rather big f***-*** to the non-WMF part of
the MediaWiki community. Furthermore, it would limit usage of common PHP
libraries, which sooner or later will end up using features not available
in HHVM. This, together with the already outlined reasons, makes me concur
with HHVM not being a viable long term option.

If only WMF was running PHP 7.1 already... guess I'll have more luck asking
for a pony though :)

Cheers

--
Jeroen De Dauw | https://entropywins.wtf | https://keybase.io/jeroendedauw
Software craftsmanship advocate
~=[,,_,,]:3
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 19/09/17 06:58, Max Semenik wrote:
> Today, the HHVM developers made an announcement[1] that they have plans of
> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> instead.

The HHVM team did tell us privately that they were planning on
changing their strategy, basically as you describe it above. The
surprising things for me in this announcement were:

* The plan to also drop PHP 5 compatibility, on a short timeline (1 year).
* Rather than "drifting away" from PHP, their top priority plans
include removing core language features like references and destructors.

> While this does not mean that we need to take an action immediately,
> eventually we will have to decide something.

Actually, I think a year is a pretty short time for ops to switch to
PHP 7. I think we need to decide on this pretty much immediately.

> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.
>
> I personally think that 3) is the only viable option in the long run. What
> do you think?

Yes, I think it's the only viable option.

I'll run a benchmark, but I don't see how it could influence the
decision. It'll be more for capacity planning.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> * Rather than "drifting away" from PHP, their top priority plans
> include removing core language features like references and destructors.

Wow. I can see why they're doing it (those are sources of most
complications ans security issues in the language, references being
especially weird and tricky). But dropping those would certainly mean
very heavy incompatibility with PHP, by which point it'd be completely
separate language. Which probably excludes Max's #2 from consideration
altogether.

> Actually, I think a year is a pretty short time for ops to switch to
> PHP 7. I think we need to decide on this pretty much immediately.

Should it be on the TechCom agenda and should we have some public
discussion on IRC in RFC format for this soon?

--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Mon, Sep 18, 2017 at 5:44 PM Stas Malyshev <smalyshev@wikimedia.org>
wrote:

> Hi!
>
> > * Rather than "drifting away" from PHP, their top priority plans
> > include removing core language features like references and destructors.
>
> Wow. I can see why they're doing it (those are sources of most
> complications ans security issues in the language, references being
> especially weird and tricky). But dropping those would certainly mean
> very heavy incompatibility with PHP, by which point it'd be completely
> separate language. Which probably excludes Max's #2 from consideration
> altogether.
>
> > Actually, I think a year is a pretty short time for ops to switch to
> > PHP 7. I think we need to decide on this pretty much immediately.
>
> Should it be on the TechCom agenda and should we have some public
> discussion on IRC in RFC format for this soon?
>
>
I see zero reason for us to go through all the formalities, unless we want
to really. I have yet to see anyone (on list, or on IRC anywhere at all
today) where anyone suggested (2) was a good idea at all. It's a
horrifically bad idea. I don't consider it remotely viable and would do
everything possible to veto such a move.

(1) is impossible as a long-term goal.

So this basically means we're going the route of (3) which is the only way
we can actually expect people to use MediaWiki outside of Wikimedia. And
considering we never really implemented any HHVM/Hack specific features (as
far as I know) it means there's no reason we have to continue to support
HHVM at all once WMF has moved off of it.

It's been a fun experiment for the past couple of years, but it's time to
move on.

-Chad
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi,

On 09/18/2017 05:13 PM, Tim Starling wrote:
> * The plan to also drop PHP 5 compatibility, on a short timeline (1 year).
> * Rather than "drifting away" from PHP, their top priority plans
> include removing core language features like references and destructors.

On Reddit[1], a member of the HHVM team clarified they plan on dropping
support for destructors *from Hack* soon. (Not that I think it really
makes any difference in what our long-term plan should be.)

[1] https://www.reddit.com/r/PHP/comments/70wtky/the_future_of_hhvm/dn6skdn/

-- Legoktm

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 19/09/17 12:30, Legoktm wrote:
> Hi,
>
> On 09/18/2017 05:13 PM, Tim Starling wrote:
>> * The plan to also drop PHP 5 compatibility, on a short timeline (1 year).
>> * Rather than "drifting away" from PHP, their top priority plans
>> include removing core language features like references and destructors.
>
> On Reddit[1], a member of the HHVM team clarified they plan on dropping
> support for destructors *from Hack* soon. (Not that I think it really
> makes any difference in what our long-term plan should be.)

It's unclear how much difference that will make, since they are clear
about wanting to make HHVM be purely a Hack runtime. They'll carry on
supporting Composer and PHPUnit, but only until Hack has "its own
ecosystem of core frameworks".

They "will not be targeting PHP software beyond such libraries after
the 3.24 release", which presumably means that they will no longer run
MediaWiki or PHP unit tests against HHVM.

Also, they said that they want to remove destructors in order to
eliminate the performance overhead of reference counting, and I don't
think it is possible to get that performance benefit unless you remove
reference counts from the VM entirely. Maybe removing them from the
Hack language will be a first step, but we can't expect them to keep
them in the VM in the longer term.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 10:13:47AM +1000, Tim Starling wrote:
> On 19/09/17 06:58, Max Semenik wrote:
> > Today, the HHVM developers made an announcement[1] that they have plans of
> > ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> > instead.
>
> The HHVM team did tell us privately that they were planning on
> changing their strategy, basically as you describe it above. The
> surprising things for me in this announcement were:
>
> * The plan to also drop PHP 5 compatibility, on a short timeline (1 year).
> * Rather than "drifting away" from PHP, their top priority plans
> include removing core language features like references and destructors.
>
> > While this does not mean that we need to take an action immediately,
> > eventually we will have to decide something.
>
> Actually, I think a year is a pretty short time for ops to switch to
> PHP 7. I think we need to decide on this pretty much immediately.

The next step would be the upgrade of the mw* fleet to Debian stretch
while still using HHVM 3.18 (to minimise disruption since we've stabilised
3.18 in it's current build). That work is tracked at T174431. 3.18 will
be supported by upstream for at least another six months (and if the migration drags
further I can roll custom 3.18 security backports from later LTS releases)

Debian stretch ships PHP7, so that'd be a good stepstone to migrate
back to Zend.

Cheers,
Moritz

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Should we have a TechComm-driven meeting about this ASAP?

Like others, I don't expect that there will be disagreement about the way
to go, but there is a lot to discuss about what needs to be done,
resourcing, etc.

It would be nice to have Ori around for it, to pick his brains about any
undocumented or little-known knowledge about the HHVM migration that could
bite us when migrating to PHP 7.x if we don't know about it.

On Tue, Sep 19, 2017 at 9:07 AM, Moritz Muehlenhoff <
mmuhlenhoff@wikimedia.org> wrote:

> On Tue, Sep 19, 2017 at 10:13:47AM +1000, Tim Starling wrote:
> > On 19/09/17 06:58, Max Semenik wrote:
> > > Today, the HHVM developers made an announcement[1] that they have
> plans of
> > > ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> > > instead.
> >
> > The HHVM team did tell us privately that they were planning on
> > changing their strategy, basically as you describe it above. The
> > surprising things for me in this announcement were:
> >
> > * The plan to also drop PHP 5 compatibility, on a short timeline (1
> year).
> > * Rather than "drifting away" from PHP, their top priority plans
> > include removing core language features like references and destructors.
> >
> > > While this does not mean that we need to take an action immediately,
> > > eventually we will have to decide something.
> >
> > Actually, I think a year is a pretty short time for ops to switch to
> > PHP 7. I think we need to decide on this pretty much immediately.
>
> The next step would be the upgrade of the mw* fleet to Debian stretch
> while still using HHVM 3.18 (to minimise disruption since we've stabilised
> 3.18 in it's current build). That work is tracked at T174431. 3.18 will
> be supported by upstream for at least another six months (and if the
> migration drags
> further I can roll custom 3.18 security backports from later LTS releases)
>
> Debian stretch ships PHP7, so that'd be a good stepstone to migrate
> back to Zend.
>
> Cheers,
> Moritz
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Sounds like a good idea. +Daniel for scheduling.

Best,

Victoria


> On Sep 19, 2017, at 6:56 AM, Gilles Dubuc <gilles@wikimedia.org> wrote:
>
> Should we have a TechComm-driven meeting about this ASAP?
>
> Like others, I don't expect that there will be disagreement about the way
> to go, but there is a lot to discuss about what needs to be done,
> resourcing, etc.
>
> It would be nice to have Ori around for it, to pick his brains about any
> undocumented or little-known knowledge about the HHVM migration that could
> bite us when migrating to PHP 7.x if we don't know about it.
>
> On Tue, Sep 19, 2017 at 9:07 AM, Moritz Muehlenhoff <
> mmuhlenhoff@wikimedia.org> wrote:
>
>> On Tue, Sep 19, 2017 at 10:13:47AM +1000, Tim Starling wrote:
>>> On 19/09/17 06:58, Max Semenik wrote:
>>>> Today, the HHVM developers made an announcement[1] that they have
>> plans of
>>>> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
>>>> instead.
>>>
>>> The HHVM team did tell us privately that they were planning on
>>> changing their strategy, basically as you describe it above. The
>>> surprising things for me in this announcement were:
>>>
>>> * The plan to also drop PHP 5 compatibility, on a short timeline (1
>> year).
>>> * Rather than "drifting away" from PHP, their top priority plans
>>> include removing core language features like references and destructors.
>>>
>>>> While this does not mean that we need to take an action immediately,
>>>> eventually we will have to decide something.
>>>
>>> Actually, I think a year is a pretty short time for ops to switch to
>>> PHP 7. I think we need to decide on this pretty much immediately.
>>
>> The next step would be the upgrade of the mw* fleet to Debian stretch
>> while still using HHVM 3.18 (to minimise disruption since we've stabilised
>> 3.18 in it's current build). That work is tracked at T174431. 3.18 will
>> be supported by upstream for at least another six months (and if the
>> migration drags
>> further I can roll custom 3.18 security backports from later LTS releases)
>>
>> Debian stretch ships PHP7, so that'd be a good stepstone to migrate
>> back to Zend.
>>
>> Cheers,
>> Moritz
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Mon, Sep 18, 2017 at 9:51 PM, Chad <innocentkiller@gmail.com> wrote:

> I see zero reason for us to go through all the formalities, unless we want
> to really. I have yet to see anyone (on list, or on IRC anywhere at all
> today) where anyone suggested (2) was a good idea at all. It's a
> horrifically bad idea.
>

Technically, I did outline the arguments for (2), earlier on this thread.
It was a bit allegorical, though:
https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/Examples#Example_of_a_.22medium_level.22_position_statement
This should be seriously considered, not just dismissed.

I agree the short timeline seems to push us toward reverting to Zend. But
it is worth having a meaningful discussion about the long-term outlook.
Which VM is likely to be better supported in 15 years' time? Which VM
would we rather adopt and maintain indefinitely ourselves, if needed --
since in a 15 yr timeframe it's entirely possible that (a) Facebook could
abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining
control over our core runtime is important; I think we should at least
discuss long-term contingencies if either goes down. Obviously, our future
was most stable when we (briefly!) had a choice between two strong
runtimes... but that opportunity seems to be vanishing. Practically
speaking, it's not really a choice between "lock-in" and "no lock in" -- we
have to choose to align our futures with either Zend Technologies Ltd or
Facebook. One of these is *much* better funded than the other. It is
likely that the project with the most funding will continue to have the
better performance.

There are other big users of HHVM -- do we know what other members of the
larger community are doing? We've heard that Phabricator intends to follow
PHP 7. Etsy also shifted to HHVM, do we know what their plans are?
--scott

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

> On Sep 19, 2017, at 8:21 AM, C. Scott Ananian <cananian@wikimedia.org> wrote:
>
>> On Mon, Sep 18, 2017 at 9:51 PM, Chad <innocentkiller@gmail.com> wrote:
>>
>> I see zero reason for us to go through all the formalities, unless we want
>> to really. I have yet to see anyone (on list, or on IRC anywhere at all
>> today) where anyone suggested (2) was a good idea at all. It's a
>> horrifically bad idea.
>>
>
> Technically, I did outline the arguments for (2), earlier on this thread.
> It was a bit allegorical, though:
> https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/Examples#Example_of_a_.22medium_level.22_position_statement
> This should be seriously considered, not just dismissed.
>
> I agree the short timeline seems to push us toward reverting to Zend. But
> it is worth having a meaningful discussion about the long-term outlook.
> Which VM is likely to be better supported in 15 years' time? Which VM
> would we rather adopt and maintain indefinitely ourselves, if needed --
> since in a 15 yr timeframe it's entirely possible that (a) Facebook could
> abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining
> control over our core runtime is important; I think we should at least
> discuss long-term contingencies if either goes down. Obviously, our future
> was most stable when we (briefly!) had a choice between two strong
> runtimes... but that opportunity seems to be vanishing. Practically
> speaking, it's not really a choice between "lock-in" and "no lock in" -- we
> have to choose to align our futures with either Zend Technologies Ltd or
> Facebook. One of these is *much* better funded than the other. It is
> likely that the project with the most funding will continue to have the
> better performance.
>
> There are other big users of HHVM -- do we know what other members of the
> larger community are doing? We've heard that Phabricator intends to follow
> PHP 7. Etsy also shifted to HHVM, do we know what their plans are?
> --scott

This is what I want to know as well: who is the community of HHVM users outside of us? What are their own internal reactions? Keep in mind that businesses like Etsy don't have to support third party downstream users as we do; "I guess we'll use Hack now" is a more plausible response than it would be for us.

But if others are indeed like us and are balking at the plan to drop PHP, it would be worth looking into how we can pool resources to "save" HHVM, or create a runtime that maintains the performance improvements of HHVM while maintaining PHP support.

Incidentally, how much work has been done on incorporating HHVM's improvements back into Zend?

>
> --
> (http://cscott.net)
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 3:21 PM, C. Scott Ananian
<cananian@wikimedia.org> wrote:
> On Mon, Sep 18, 2017 at 9:51 PM, Chad <innocentkiller@gmail.com> wrote:
>
>> I see zero reason for us to go through all the formalities, unless we want
>> to really. I have yet to see anyone (on list, or on IRC anywhere at all
>> today) where anyone suggested (2) was a good idea at all. It's a
>> horrifically bad idea.
>>
>
> Technically, I did outline the arguments for (2), earlier on this thread.
> It was a bit allegorical, though:
> https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/Examples#Example_of_a_.22medium_level.22_position_statement
> This should be seriously considered, not just dismissed.
>
> I agree the short timeline seems to push us toward reverting to Zend. But
> it is worth having a meaningful discussion about the long-term outlook.
> Which VM is likely to be better supported in 15 years' time? Which VM
> would we rather adopt and maintain indefinitely ourselves, if needed --
> since in a 15 yr timeframe it's entirely possible that (a) Facebook could
> abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining
> control over our core runtime is important; I think we should at least
> discuss long-term contingencies if either goes down. Obviously, our future
> was most stable when we (briefly!) had a choice between two strong
> runtimes... but that opportunity seems to be vanishing. Practically
> speaking, it's not really a choice between "lock-in" and "no lock in" -- we
> have to choose to align our futures with either Zend Technologies Ltd or
> Facebook. One of these is *much* better funded than the other. It is
> likely that the project with the most funding will continue to have the
> better performance.
>
> There are other big users of HHVM -- do we know what other members of the
> larger community are doing? We've heard that Phabricator intends to follow
> PHP 7. Etsy also shifted to HHVM, do we know what their plans are?
> --scott
>
> --
> (http://cscott.net)
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

I disagree. I don't think its useful or possible to try to forecast
technical trends 15 years out. 15 years from now, it is just as likely
that facebook will be as relevant as myspace is today, as it is that
facebook will go full cyberpunk dystopia on us and rule the world. I
don't think we can realistically predict anything 15 years out.

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

*Continue* to have the best performance? A quick search for benchmarks suggests that 7.1 outperforms HHVM more often than not.

Betting our future on Facebook because they have most money right now seems unwise. Which of the two has the larger developer community?


> On 19 Sep 2017, at 16:21, C. Scott Ananian <cananian@wikimedia.org> wrote:
>
> On Mon, Sep 18, 2017 at 9:51 PM, Chad <innocentkiller@gmail.com> wrote:
>
>> I see zero reason for us to go through all the formalities, unless we want
>> to really. I have yet to see anyone (on list, or on IRC anywhere at all
>> today) where anyone suggested (2) was a good idea at all. It's a
>> horrifically bad idea.
>>
>
> Technically, I did outline the arguments for (2), earlier on this thread.
> It was a bit allegorical, though:
> https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/Examples#Example_of_a_.22medium_level.22_position_statement
> This should be seriously considered, not just dismissed.
>
> I agree the short timeline seems to push us toward reverting to Zend. But
> it is worth having a meaningful discussion about the long-term outlook.
> Which VM is likely to be better supported in 15 years' time? Which VM
> would we rather adopt and maintain indefinitely ourselves, if needed --
> since in a 15 yr timeframe it's entirely possible that (a) Facebook could
> abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining
> control over our core runtime is important; I think we should at least
> discuss long-term contingencies if either goes down. Obviously, our future
> was most stable when we (briefly!) had a choice between two strong
> runtimes... but that opportunity seems to be vanishing. Practically
> speaking, it's not really a choice between "lock-in" and "no lock in" -- we
> have to choose to align our futures with either Zend Technologies Ltd or
> Facebook. One of these is *much* better funded than the other. It is
> likely that the project with the most funding will continue to have the
> better performance.
>
> There are other big users of HHVM -- do we know what other members of the
> larger community are doing? We've heard that Phabricator intends to follow
> PHP 7. Etsy also shifted to HHVM, do we know what their plans are?
> --scott
>
> --
> (http://cscott.net)
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 9:21 AM, C. Scott Ananian
<cananian@wikimedia.org> wrote:
> There are other big users of HHVM -- do we know what other members of the
> larger community are doing? We've heard that Phabricator intends to follow
> PHP 7. Etsy also shifted to HHVM, do we know what their plans are?

Etsy 'experimented' with HHVM [0] and then eventually switched to PHP7
as their primary runtime. The blog posts about this are a little
scattered, but Rasmus spoke about it [1] and Etsy started the phan
project [2].

For what it's worth, my opinion is that PHP is an actual FLOSS
software project with years of history and core contributions from
Zend who make their living with PHP. HHVM is a well funded internal
project from Facebook that has experimented with FLOSS but ultimately
is controlled by the internal needs of Facebook. For me the choice
here is obviously to back the community driven FLOSS project and help
them continue to thrive.

[0]: https://codeascraft.com/2015/04/06/experimenting-with-hhvm-at-etsy/
[1]: https://codeascraft.com/speakers/rasmus-lerdorf-deploying-php-7/
[2]: https://github.com/phan/phan

Bryan
--
Bryan Davis Wikimedia Foundation <bd808@wikimedia.org>
[[m:User:BDavis_(WMF)]] Manager, Cloud Services Boise, ID USA
irc: bd808 v:415.839.6885 x6855

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

> On Sep 19, 2017, at 6:56 AM, Gilles Dubuc <gilles@wikimedia.org> wrote:
>
> Should we have a TechComm-driven meeting about this ASAP?
>
> Like others, I don't expect that there will be disagreement about the way
> to go, but there is a lot to discuss about what needs to be done,
> resourcing, etc.
>
> It would be nice to have Ori around for it, to pick his brains about any
> undocumented or little-known knowledge about the HHVM migration that could
> bite us when migrating to PHP 7.x if we don't know about it.
>
> On Tue, Sep 19, 2017 at 9:07 AM, Moritz Muehlenhoff <
> mmuhlenhoff@wikimedia.org> wrote:
>
>> On Tue, Sep 19, 2017 at 10:13:47AM +1000, Tim Starling wrote:
>>> On 19/09/17 06:58, Max Semenik wrote:
>>>> Today, the HHVM developers made an announcement[1] that they have
>> plans of
>>>> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
>>>> instead.
>>>
>>> The HHVM team did tell us privately that they were planning on
>>> changing their strategy, basically as you describe it above. The
>>> surprising things for me in this announcement were:
>>>
>>> * The plan to also drop PHP 5 compatibility, on a short timeline (1
>> year).
>>> * Rather than "drifting away" from PHP, their top priority plans
>>> include removing core language features like references and destructors.
>>>
>>>> While this does not mean that we need to take an action immediately,
>>>> eventually we will have to decide something.
>>>
>>> Actually, I think a year is a pretty short time for ops to switch to
>>> PHP 7. I think we need to decide on this pretty much immediately.
>>
>> The next step would be the upgrade of the mw* fleet to Debian stretch
>> while still using HHVM 3.18 (to minimise disruption since we've stabilised
>> 3.18 in it's current build). That work is tracked at T174431. 3.18 will
>> be supported by upstream for at least another six months (and if the
>> migration drags
>> further I can roll custom 3.18 security backports from later LTS releases)
>>
>> Debian stretch ships PHP7, so that'd be a good stepstone to migrate
>> back to Zend.
>>
>> Cheers,
>> Moritz
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 11:33 AM, bawolff <bawolff+wn@gmail.com> wrote:

> I disagree. I don't think its useful or possible to try to forecast
> technical trends 15 years out. 15 years from now, it is just as likely
> that facebook will be as relevant as myspace is today, as it is that
> facebook will go full cyberpunk dystopia on us and rule the world. I
> don't think we can realistically predict anything 15 years out.
>
>
That's why I argued we should consider the code quality of both, and
determine which we'd feel most comfortable supporting on our own in the
case our crystal balls are wrong and we end up holding the bag for the
runtime. (This is just one factor to consider; my argument is that we
should carefully enumerate the various factors before making an evaluation,
not just dismiss any of the options out of hand.)
--scott

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 11:34 AM, Bryan Davis <bd808@wikimedia.org> wrote:

> On Tue, Sep 19, 2017 at 9:21 AM, C. Scott Ananian
> <cananian@wikimedia.org> wrote:
> > There are other big users of HHVM -- do we know what other members of the
> > larger community are doing? We've heard that Phabricator intends to
> follow
> > PHP 7. Etsy also shifted to HHVM, do we know what their plans are?
>
> Etsy 'experimented' with HHVM [0] and then eventually switched to PHP7
> as their primary runtime. The blog posts about this are a little
> scattered, but Rasmus spoke about it [1] and Etsy started the phan
> project [2].
>

I got confirmation on twitter:
https://twitter.com/jazzdan/status/910162545805336576


> For what it's worth, my opinion is that PHP is an actual FLOSS
> software project with years of history and core contributions from
> Zend who make their living with PHP. HHVM is a well funded internal
> project from Facebook that has experimented with FLOSS but ultimately
> is controlled by the internal needs of Facebook. For me the choice
> here is obviously to back the community driven FLOSS project and help
> them continue to thrive.
>

Fair enough. My point is just that we should stop and reflect that this is
a major inflection point. Language choices are sticky, so this decision
will have significant long-term implications. We should at least stop to
evaluate PHP7 vs Hack and determine which is a better fit for our codebase,
and do due diligence on both sides (count how many engineers, how many open
source contributors, commit rates, etc). HHVM has been flirting with a
LLVM backend, and LLVM itself has quite a large and active community. The
PHP community has had issues with proper handling of security patches in
the past. I'm suggesting to proceed cautiously and have a proper
discussion of all the factors involved instead of over-simplifying this to
"community" vs "facebook".

For example, the top-line github stats are:
hhvm: 504 contributors (24,192 commits)
php-src: 496 contributors (104,566 commits)

HHVM seems to have a larger community of contributors despite a much
shorter active life. But note that the PHP github mirror has been broken
since Jul 29 (!). In the past 6 days I count 8 distinct contributors to
php-src, and 10 distinct contributors in the past *two days* to hhvm (one
of whom contributed an OCAML frontend(!)). These are just hand-wavy
figures; ideally we should try to determine how many of the recent
contributors to each project are employed by Facebook and/or Zend.

I think there's room for a reasonable debate.
--scott
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 9:40 AM C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Fair enough. My point is just that we should stop and reflect that this is
> a major inflection point. Language choices are sticky, so this decision
> will have significant long-term implications. We should at least stop to
> evaluate PHP7 vs Hack and determine which is a better fit for our codebase,
> and do due diligence on both sides (count how many engineers, how many open
> source contributors, commit rates, etc). HHVM has been flirting with a
> LLVM backend, and LLVM itself has quite a large and active community. The
> PHP community has had issues with proper handling of security patches in
> the past. I'm suggesting to proceed cautiously and have a proper
> discussion of all the factors involved instead of over-simplifying this to
> "community" vs "facebook".
>
>
I'm not trying to simplify this into community vs. facebook. And let's also
be clear: we never chose HHVM for Hack. We don't use Hack. The one
experiment I had at trying Hack never panned out. MediaWiki is in PHP, not
Hack.

The *only* reason we're having a language discussion is because HHVM has
announced that they're abandoning PHP in favor of Hack. If someone had some
to the list last week and said "Hey let's abandon PHP for XYZLang" they
would've been rightly laughed off.

The debate here is between runtimes for PHP, and on the long enough
timescale there's only one option. PHP has a long-standing history of being
a viable runtime. HHVM does not.

I don't see this as an A/B choice at all, but rather a clear path forward.
So sure: let's have an RfC/TechComm meeting to work out the details, but
let's not pretend that option #2 is even remotely viable. It is not.

-Chad
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 1:17 PM, Chad <innocentkiller@gmail.com> wrote:

> be clear: we never chose HHVM for Hack. We don't use Hack. The one
> experiment I had at trying Hack never panned out. MediaWiki is in PHP, not
> Hack.
>

To be super clear: MediaWiki is in *PHP5*. The choices are:

1) MediaWiki will always be in PHP5.
2) MediaWiki will eventually migrate to PHP7, or
3) MediaWiki will eventually migrate to Hack.

Is anyone arguing for #1?

So we've got two backwards-incompatible choices to make, eventually.
--scott

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 10:28 AM C. Scott Ananian <cananian@wikimedia.org>
wrote:

> On Tue, Sep 19, 2017 at 1:17 PM, Chad <innocentkiller@gmail.com> wrote:
>
> > be clear: we never chose HHVM for Hack. We don't use Hack. The one
> > experiment I had at trying Hack never panned out. MediaWiki is in PHP,
> not
> > Hack.
> >
>
> To be super clear: MediaWiki is in *PHP5*. The choices are:
>
> 1) MediaWiki will always be in PHP5.
> 2) MediaWiki will eventually migrate to PHP7, or
> 3) MediaWiki will eventually migrate to Hack.
>
> Is anyone arguing for #1?
>
> So we've got two backwards-incompatible choices to make, eventually.
>

I see everyone saying #1 for now and #2 down the road. You're the only one
here who seems to think #3 is even worth considering. The rest of us
have--rightly--dismissed it already.

-Chad
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 19 Sep 2017 5:40 pm, "C. Scott Ananian" <cananian@wikimedia.org> wrote:

I'm suggesting to proceed cautiously and have a proper
discussion of all the factors involved instead of over-simplifying this to
"community" vs "facebook".

For example, the top-line github stats are:
hhvm: 504 contributors (24,192 commits)
php-src: 496 contributors (104,566 commits)

HHVM seems to have a larger community of contributors despite a much
shorter active life.

By a difference of 8 contributors?

But note that the PHP github mirror has been broken since Jul 29 (!).

I'm not convinced an exclamation mark in brackets is required here.
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tuesday, September 19, 2017, C. Scott Ananian <cananian@wikimedia.org>
wrote:
> On Tue, Sep 19, 2017 at 1:17 PM, Chad <innocentkiller@gmail.com> wrote:
>
>> be clear: we never chose HHVM for Hack. We don't use Hack. The one
>> experiment I had at trying Hack never panned out. MediaWiki is in PHP,
not
>> Hack.
>>
>
> To be super clear: MediaWiki is in *PHP5*. The choices are:
>
> 1) MediaWiki will always be in PHP5.
> 2) MediaWiki will eventually migrate to PHP7, or
> 3) MediaWiki will eventually migrate to Hack.
>
> Is anyone arguing for #1?
>
> So we've got two backwards-incompatible choices to make, eventually.
> --scott
>
> --
> (http://cscott.net)
>

No, MediaWiki is written in a subset of php5 which is forwards compatible
with php7. The backwards incompatible things are by in large things no sane
code would rely on (http://php.net/manual/en/migration70.incompatible.php)
There are people who are using mediawiki with php7 right now without
complaint. Presumably eventually we will move to needing php7, just like
once we moved from php4 to php5, but thats really just a version
requirement increment.


Sure there may be backwards incompatible things that come up, but thats
nothing new. E.g. We used to have a class named Namespace. Then php 5.3
came along and its now MWNamespace. All signs point to php7
incompatibilities to be of this form.
--
Brian
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Chad: the more you argue that it's not even worth considering, the more I'm
going to push back and ask for actual numbers and facts. ;)

Alex: I admit 8 contributors is not significant, but 24,192 commits vs
104,566 commits is. The conclusion I was suggesting was "same number of
contributors over a much shorter time frame".

Re: github mirror broken: note that http://php.net/build-setup.php mentions
*only* the github mirror. And the github mirror has been broken for almost
two months now. There's no mention of the "real" repo. That merits an
exclamation mark, I think.
--scott
?
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi,

On 09/19/2017 10:27 AM, C. Scott Ananian wrote:
> To be super clear: MediaWiki is in *PHP5*. The choices are:

Well, kind of. MediaWiki has full support for PHP 7 while still
maintaining PHP 5 support.

> 1) MediaWiki will always be in PHP5.
> 2) MediaWiki will eventually migrate to PHP7, or
> 3) MediaWiki will eventually migrate to Hack.
>
> Is anyone arguing for #1?

I don't think anyone has argued for anything besides #2.

> So we've got two backwards-incompatible choices to make, eventually.

There's already an open RfC[1] about bumping the minimum PHP requirement
to 7.x.

[1] https://phabricator.wikimedia.org/T172165

-- Legoktm

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi,

On 09/19/2017 10:50 AM, C. Scott Ananian wrote:
> Chad: the more you argue that it's not even worth considering, the more I'm
> going to push back and ask for actual numbers and facts. ;)

Migrating to Hack simply isn't feasible for most users of MediaWiki.
Most distros don't have packages for HHVM, and it's not straightforward
to build AIUI.

Once you've figured that out, then you're going to quickly realize that
you need a cron job to regularly restart HHVM because it'll
OOM/crash/etc. So either you have downtime or multiple HHVM processes.

On the MediaWiki development side, we've been forced for fork all of the
external libraries we depend upon to support Hack. This includes basic
tooling like composer, phpunit, codesniffer, and so on.

There are probably more reasons that could be listed, but I think you
get the idea of why Chad considers it to be a non-starter.

-- Legoktm

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 10:50 AM C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Chad: the more you argue that it's not even worth considering, the more I'm
> going to push back and ask for actual numbers and facts. ;)
>
>
As I said in my prior message: it's been considered, and discarded rather
quickly. It doesn't need much further introspection than that. A couple of
major points:

* Brian W's point is correct: there's not much migration cost at all moving
to PHP7. Moving between PHP versions has always been pretty easy for us in
MW--we write very conservative code as it is.
* Various (there's lots from lots of people) benchmarks routinely show HHVM
and PHP7--especially 7.1+--performance to be roughly on par, depending on
workloads. We can get some firmer numbers here.
** A bunch of major projects have already dropped HHVM for PHP7+. Etsy,
Symphony, Wordpress (they no longer test against it)
* Swapping to HHVM/Hack would abandon many/most of our downstream users --
the stats Stas pointed to way far back shows world install base as well
under 0.5% of all PHP runtimes.
* Speaking of downstream users: HHVM has always been a second-class citizen
on non-Linux OSes. It's always been wonky on OSX (I would know, I was the
first person to ever get it built), and I don't think anyone's ever got it
working on Windows outside of Cygwin--please correct me if I'm wrong?
* It greatly complicates setup and development work for both WMF and
volunteers.
** Did I mention almost nobody has up to date packages for this?
* PHP has a long-documented history of working as a proper FLOSS project.
Facebook's track record here has been less than stellar--it comes in fits
and starts and there's a lot of "throwing code over the wall"
** We have a core PHP contributor (Stas) on this list. Other than
occasional patches we've shipped upstream to HHVM, I doubt anyone would
claim themselves a core HHVM contributor around here [maybe Tim early on,
heh]
* Choices HHVM is going to make upstream (references, destuctors) are a
/huge/ issue for us. The former is basically a requirement for our Hooks
system and the latter is used all over the dang place for profiling and
other fun scope-based tricks.
* Libraries we use -- composer, phpunit, etc can not be depended on to
support Hack in the long term, and there's no tooling in the HHVM world for
this (promises notwithstanding).

-Chad
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> I agree the short timeline seems to push us toward reverting to Zend. But
> it is worth having a meaningful discussion about the long-term outlook.
> Which VM is likely to be better supported in 15 years' time? Which VM

15 years is a lot to predict. 15 years ago Facebook, Twitter, Reddit and
Linkedin did not exist and Slashdot, Livejournal, etc. were all the
rage. We don't even know if Facebook as such would exist in 15 years or
would have budget to support its own language.

> would we rather adopt and maintain indefinitely ourselves, if needed --
> since in a 15 yr timeframe it's entirely possible that (a) Facebook could
> abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining

While (b) could happen, PHP project is not very dependent on Zend for
its existence. Zend owns none of the infrastructure or processes, and
while a lot of performance work on PHP 7 was conducted by Zend team (and
they are still working on improvements AFAIK), there are plenty of
community members that do not work for Zend and do not depend on Zend in
any way. Of course, it is possible that the whole community would
implode, but here we have many more stakeholders than in Hack case,
where the stakeholder is mostly a single - albeit large and currently
very successful - company.

> speaking, it's not really a choice between "lock-in" and "no lock in" -- we
> have to choose to align our futures with either Zend Technologies Ltd or
> Facebook. One of these is *much* better funded than the other. It is

Again, I do not think this is the right statement to make. The control
of Zend Tech as a company over the future of PHP is much less than
Facebook's one over Hack (which is pretty much absolute). PHP is guided
by the community, decisions are taken by using community processes in
which Zend does not have any special role, and PHP project could survive
reasonably survive without Zend, even if with less resources. Most PHP
infrastructure - Composer, debugging, IDEs, profiling, code quality,
frameworks, etc. - are completely independent of Zend (which also has a
number of tools, but it is not the only provider). So I do not think it
is an adequate comparison.

I am not sure if Hack has an open-source community outside Facebook (if
anybody has pointers to that, please share - commit numbers certainly
don't tell much) - but it is pretty clear to me that Facebook is in
absolute control over this platform. This is not the case with Zend and PHP.

--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Zend was testing against hhvm at one point, then decided they would stop testing against it. See https://www.phoronix.com/scan.php?page=news_item&px=PTS-PHP-7.1-Benchmarks
On Tuesday, 19 September 2017, 19:24:49 BST, Stas Malyshev <smalyshev@wikimedia.org> wrote:

Hi!

> I agree the short timeline seems to push us toward reverting to Zend.  But
> it is worth having a meaningful discussion about the long-term outlook.
> Which VM is likely to be better supported in 15 years' time?  Which VM

15 years is a lot to predict. 15 years ago Facebook, Twitter, Reddit and
Linkedin did not exist and Slashdot, Livejournal, etc. were all the
rage. We don't even know if Facebook as such would exist in 15 years or
would have budget to support its own language.

> would we rather adopt and maintain indefinitely ourselves, if needed --
> since in a 15 yr timeframe it's entirely possible that (a) Facebook could
> abandon Hack/HHVM, or (b) the PHP Zend team could implode.  Maintaining

While (b) could happen, PHP project is not very dependent on Zend for
its existence. Zend owns none of the infrastructure or processes, and
while a lot of performance work on PHP 7 was conducted by Zend team (and
they are still working on improvements AFAIK), there are plenty of
community members that do not work for Zend and do not depend on Zend in
any way. Of course, it is possible that the whole community would
implode, but here we have many more stakeholders than in Hack case,
where the stakeholder is mostly a single - albeit large and currently
very successful - company.

> speaking, it's not really a choice between "lock-in" and "no lock in" -- we
> have to choose to align our futures with either Zend Technologies Ltd or
> Facebook.  One of these is *much* better funded than the other.  It is

Again, I do not think this is the right statement to make. The control
of Zend Tech as a company over the future of PHP is much less than
Facebook's one over Hack (which is pretty much absolute). PHP is guided
by the community, decisions are taken by using community processes in
which Zend does not have any special role, and PHP project could survive
reasonably survive without Zend, even if with less resources. Most PHP
infrastructure - Composer, debugging, IDEs, profiling, code quality,
frameworks, etc. - are completely independent of Zend (which also has a
number of tools, but it is not the only provider). So I do not think it
is an adequate comparison.

I am not sure if Hack has an open-source community outside Facebook (if
anybody has pointers to that, please share - commit numbers certainly
don't tell much) - but it is pretty clear to me that Facebook is in
absolute control over this platform. This is not the case with Zend and PHP.

--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> Incidentally, how much work has been done on incorporating HHVM's
> improvements back into Zend?

Depends on which ones you're talking about. Syntax ones may or may not
find its way into PHP, but performance ones would probably be completely
different from HHVM - i.e. the resulting performance may or may not be
on par or better, but reusing most of the performance work on HHVM in
PHP would not be possible due to completely different engine internals.

So pretty much all that can be taken from HHVM into PHP would be "this
syntax looks like a good idea, let's reimplement it".
--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 09/19/2017 10:34 AM, Bryan Davis wrote:

> For what it's worth, my opinion is that PHP is an actual FLOSS
> software project with years of history and core contributions from
> Zend who make their living with PHP. HHVM is a well funded internal
> project from Facebook that has experimented with FLOSS but ultimately
> is controlled by the internal needs of Facebook. For me the choice
> here is obviously to back the community driven FLOSS project and help
> them continue to thrive.

+1.

As a practical choice, given the wide usage of PHP on the web, it makes
a lot more sense to throw our weight behind Zend development since it is
more likely to survive than HHVM which is more subject to the whims and
fancies of FB's internal choices which we are not privy to and cannot
influence in any meaningful way. I mean how big of a user besides
Mediawiki serving Wikipedia sites does it have to be for FB to consider
other user's needs?

Subbu.

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Chad: thanks for writing up all of those in one place. I'm not actually
trying to argue *for* Hack here. My argument is just that we should take
the time to lay out the arguments carefully on both sides, since this is a
decision that is likely to have long-lasting effects. Think of this like
the traditional [[devil's advocate]] process for canonizing a saint. I'm
just challenging us to take a hard look at our reasoning and make sure it
is well-founded technically.

In my ideal world we'd take your excellent point-by-point argument and try
to rigorously quantify each, just to satisfy ourselves that we've done due
diligence. You say "there's not much migration cost moving to PHP7" --
well, it would be nice to assign someone to go through the details in a
little more detail to double-check that. "Various benchmarks dhow HHVM and
PHP7 performance to be roughly on par" -- firmer numbers would be nice, as
you say. I think Tim was already going to do this? "A bunch of major
projects have already dropped HHVM" -- again, can we take the devil's
advocate position and ask the HHVM team who *hasn't* dropped HHVM ;) and
ask for their reasons? You also raise some points about packaging and
portability that I could probably counter by pointing out that HHVM has an
ARM and PowerPC JIT now (
http://hhvm.com/blog/2017/03/09/how-the-cyber-elephant-got-his-arm.html),
while Zend's JIT-in-progress only supports x86. We could also poke at the
HHVM folks and ask about their packaging plans, since they (according to
their announcement) are going to "reinvest in open source". You also
mentioned PHP's long history of FLOSS without also mentioning their long
history at sucking at security. And we could try to quantify how dependent
PHP is on Zend contributions vs how dependent HHVM is on facebook
contributions. We can get numbers on this, we don't have to assume things.

The HHVM announcement specifically mentioned that they will maintain
compatibility with composer and phpunit, so that seems to be a wash.

The references/destructors thing is actually really interesting as a
technical discussion. The reason given for removing these is all about
performance, in particular the performance hit you get from ref-counting as
opposed to pure GC. There's been a lot of really good GC work done in the
past two decades; see https://news.ycombinator.com/item?id=11759762
although I like to cite https://www.cs.princeton.edu/~appel/papers/45.ps as
foundational. It may be a good opportunity to take a hard look at our
Hooks system and figure out if its design is future-proof.
--scott

On Tue, Sep 19, 2017 at 2:04 PM, Chad <innocentkiller@gmail.com> wrote:

> On Tue, Sep 19, 2017 at 10:50 AM C. Scott Ananian <cananian@wikimedia.org>
> wrote:
>
> > Chad: the more you argue that it's not even worth considering, the more
> I'm
> > going to push back and ask for actual numbers and facts. ;)
> >
> >
> As I said in my prior message: it's been considered, and discarded rather
> quickly. It doesn't need much further introspection than that. A couple of
> major points:
>
> * Brian W's point is correct: there's not much migration cost at all moving
> to PHP7. Moving between PHP versions has always been pretty easy for us in
> MW--we write very conservative code as it is.
> * Various (there's lots from lots of people) benchmarks routinely show HHVM
> and PHP7--especially 7.1+--performance to be roughly on par, depending on
> workloads. We can get some firmer numbers here.
> ** A bunch of major projects have already dropped HHVM for PHP7+. Etsy,
> Symphony, Wordpress (they no longer test against it)
> * Swapping to HHVM/Hack would abandon many/most of our downstream users --
> the stats Stas pointed to way far back shows world install base as well
> under 0.5% of all PHP runtimes.
> * Speaking of downstream users: HHVM has always been a second-class citizen
> on non-Linux OSes. It's always been wonky on OSX (I would know, I was the
> first person to ever get it built), and I don't think anyone's ever got it
> working on Windows outside of Cygwin--please correct me if I'm wrong?
> * It greatly complicates setup and development work for both WMF and
> volunteers.
> ** Did I mention almost nobody has up to date packages for this?
> * PHP has a long-documented history of working as a proper FLOSS project.
> Facebook's track record here has been less than stellar--it comes in fits
> and starts and there's a lot of "throwing code over the wall"
> ** We have a core PHP contributor (Stas) on this list. Other than
> occasional patches we've shipped upstream to HHVM, I doubt anyone would
> claim themselves a core HHVM contributor around here [maybe Tim early on,
> heh]
> * Choices HHVM is going to make upstream (references, destuctors) are a
> /huge/ issue for us. The former is basically a requirement for our Hooks
> system and the latter is used all over the dang place for profiling and
> other fun scope-based tricks.
> * Libraries we use -- composer, phpunit, etc can not be depended on to
> support Hack in the long term, and there's no tooling in the HHVM world for
> this (promises notwithstanding).
>
> -Chad
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 2:41 PM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> source". You also mentioned PHP's long history of FLOSS without also
> mentioning their long history at sucking at security.
>

Whoops, I should have toned that down a bit before hitting send. To be
clear, I'm mostly talking about the ~2007 time frame where there was a lot
of tension between the PHP core team and various folks who wanted to make
PHP more secure in different ways. I don't actually know what the
present-day status is -- suhosin seems to be still around, but (for
instance) https://sektioneins.de/en/categories/php.html hasn't had any
particular complaints since 2015.

So to be super clear: I'm just pointing out that there used to be issues
here; sometimes the community's interests do not exactly align. Consider
me in the devil's advocate role again: I'd be interested to hear an
insider's opinion (Stas?) on how security issues are handled these days and
what the future outlook is,
https://www.cvedetails.com/vulnerability-list/vendor_id-74/product_id-128/PHP-PHP.html
doesn't look as nice as
http://www.cvedetails.com/vulnerability-list/vendor_id-7758/product_id-35896/Facebook-Hhvm.html
but
maybe the latter is misleading; older vulnerabilities seem to be at
http://www.cvedetails.com/vulnerability-list/vendor_id-7758/product_id-30684/Facebook-Hiphop-Virtual-Machine.html
for instance.
--scott

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 9:40 AM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Fair enough. My point is just that we should stop and reflect that this is
> a major inflection point. Language choices are sticky, so this decision
> will have significant long-term implications. We should at least stop to
> evaluate PHP7 vs Hack and determine which is a better fit for our codebase,
> and do due diligence on both sides (count how many engineers, how many open
> source contributors, commit rates, etc). HHVM has been flirting with a
> LLVM backend, and LLVM itself has quite a large and active community. The
> PHP community has had issues with proper handling of security patches in
> the past. I'm suggesting to proceed cautiously and have a proper
> discussion of all the factors involved instead of over-simplifying this to
> "community" vs "facebook".
>

PHP is an open-source language with mature tooling and major community
buy-in. Facebook has *promised* to turn Hack into an open-source language
with mature tooling and community buy-in; almost none of that exists
currently. Once it already does, a worthwhile discussion might be had about
switching to Hack. Right now it would be incredibly irresponsible.

Also, making PHP a viable language for third parties is the core business
model of Zend. Making Hack a viable language for third parties has
absolutely nothing to do with the business model of Facebook. At any point
they might decide it is a distraction they don't need. Comparing commit
numbers is not really meaningful without knowing what fraction of those
committers can disappear overnight if Facebook reconsiders its priorities.

IMO the more interesting discussion to be had is how little we invest into
the technology our whole platform is based on. You'd think the largest
production user of PHP would pay at least one part-time PHP developer, or
try to represent itself in standards and roadmap discussions, but we do
not. Is that normal?
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 11:41 AM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Think of this like
> the traditional [[devil's advocate]] process for canonizing a saint. I'm
> just challenging us to take a hard look at our reasoning and make sure it
> is well-founded technically.
>

?I for one appreciate your engagement here, even if it has not been
popular.

It is easy for a group to make a snap decision, or to end up in an echo
chamber, where decisions seem "obvious" to everyone who is participating.
Yes, sometimes there are easy or obvious decisions. But with the stakes so
high on this one, it seems prudent to think it through, enumerate the
arguments, and invite discussion.

If the decision is really so clearcut, the analysis and discussion should
be quick and painless. Hopefully it can also be civil and respectful,
especially toward those arguing the minority view (whether because they
believe it, or because they are just trying to help reach the best possible
outcome).

For what it's worth, I have added this topic to the TechCom internal
meeting agenda for this week. They can decide whether or not to schedule an
IRC meeting, or to take other steps.

Kevin
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> So to be super clear: I'm just pointing out that there used to be issues
> here; sometimes the community's interests do not exactly align. Consider
> me in the devil's advocate role again: I'd be interested to hear an
> insider's opinion (Stas?) on how security issues are handled these days and
> what the future outlook is,
> https://www.cvedetails.com/vulnerability-list/vendor_id-74/product_id-128/PHP-PHP.html

OK, so it's a big topic but I can do some quick survey and we can get
deeper if you want to. So:

Despite what you see there, there are not a lot of genuine security
issues in PHP. Unfortunately, CVE issuance process is such that it does
not involve any consultation with the vendor about classifying the issue
and assigning severity. You can just create CVE ID for anything and how
they assign severity is kinda mystery to me, but one thing is clear -
PHP core team is not consulted about it, at least not in any way I've
ever noticed. It doesn't mean those are all wrong, but some probably
are, and a lot are misclassified.

Now, for the substance of the issues. Many of them are unserialize()
issues. For this, without getting too much into the woods, I can only
say this: there seems to be no way to make unserialize() robust against
untrusted data, and it has to do with how references, object
construction, object destruction (notice the similarity with what Hack
intends to drop? It is not a coincidence) and serialization support
works in PHP. There is too much internal structure exposed to make it
work with untrusted data now. Maybe if we redesigned the whole thing
from scratch, it _could_ be possible, but even then I kinda doubt it, at
least not without sacrificing some feature support. For now, the best
approach to this is consider using any unserialize() on untrusted data
inherently insecure. It's just too low-level to be sure no corner case
ever does anything strange.

Some of those are genuine security problems, which now mostly
concentrate in several extensions, which has been either under-served or
have very wide exposed areas. Namely, wddx - obscure format that suffers
from issues similar to unserialize() and in addition lack
maintainership, phar - which wasn't really designed to deal with
untrusted scripts but seem to kinda be moving that direction, and gd -
which mostly relies on libgd and every issue there is also automatically
extends into PHP. The latest two would probably benefit from a good
fuzzing testing, but nobody took on it yet. I suspect if hhvm uses the
same or derived code - and it likely does, I don't think they
reimplemented libgd from scratch? - many of those would also be present
in hhvm if hhvm supports these (no idea).

There are also some long-standing debates about how randomness is
handled (basically, there are many ways to get randoms, and most of them
are not suitable for security-related randomness) and some DoS issues in
PHP hash tables - the latter are mostly resolved, but in kinda temporary
way, so there's more work to be done.

Some of these issues - like https://bugs.php.net/bug.php?id=74310 - are
definitely not security issues at all. Yes, you can create bad code that
hits some corner case and produces segfault. That shouldn't happen, but
this being complex C code which is 20 years old, it does. This has
absolutely nothing to do with security, and whoever decided to issue CVE
to it and assign 7.5 severity
(https://www.cvedetails.com/cve/CVE-2017-9119/) has done a very sloppy
job. As I said, this is done with zero communication with actual PHP
team as far as I know, which is very sad, but this is the state of
affairs. Some of those are, I am sorry to say, complete baloney, e.g.
https://www.cvedetails.com/cve/CVE-2017-8923/ says:

<<The zend_string_extend function in Zend/zend_string.h in PHP through
7.1.5 does not prevent changes to string objects that result in a
negative length, which allows remote attackers to cause a denial of
service (application crash) or possibly have unspecified other impact by
leveraging a script's use of .= with a long string.>>

This is nonsense - unless you run with no memory limit at all (nobody
sane does that) and specifically allow your code not only accept
infinite untrusted data, but have it fed specifically to certain
functions arranged into a specific code pattern, no remote attacker can
do it. There are many issues that have similar claims, none of them are
actual security issues.

Some are also assigned to PHP despite them being application issues,
e.g. https://www.cvedetails.com/cve/CVE-2017-9067/. To add insult to
injury, this is 2017 CVE about a version that was EOLed in 2014. The
data quality seems to be very sad there. I tried to figure out how to
make it better a while ago but pretty much gave up on it because I
couldn't find anybody responsible or at least concerned about this sad
state of things.

OK, this came out super-long and kinda ranty (sorry!), so I will stop
for now, but if you have any questions about it, please feel free to
ping me, and I will be glad to discuss it.
--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> IMO the more interesting discussion to be had is how little we invest into
> the technology our whole platform is based on. You'd think the largest
> production user of PHP would pay at least one part-time PHP developer, or
> try to represent itself in standards and roadmap discussions, but we do
> not. Is that normal?

I think this is a very good point. There are other ways to support PHP
too. Like assisting in regular testing of upcoming versions. Helping
writing the docs. Contributing to RFC discussions (we have a large
codebase, heavily visited site, and a lot of experience dealing with it,
surely there could be a thing or two we could contribute). Triaging the
bugs (one of the most necessary, thankless, and under-appreciated jobs
in an open-source project). Probably more...
--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 2:41 PM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> You say "there's not much migration cost moving to PHP7" --
> well, it would be nice to assign someone to go through the details in a
> little more detail to double-check that.


What migration costs there were for PHP7 have probably already been paid,
since as has already been noted several developers are already running on
PHP 7 (myself included). There's nothing much open on the NewPHP
workboard,[1] and what is still open seems to be false positives (T120336,
T173850, T173849, T120694), mostly done already (T153505), deprecations of
stuff we only still have for backwards compatibility (T120333, T143788),
irrelevant (T174199), or tracking backports (T174262).

[1]: https://phabricator.wikimedia.org/project/board/346/


> The HHVM announcement specifically mentioned that they will maintain
> compatibility with composer and phpunit, so that seems to be a wash.
>

They specifically mentioned they'd maintain compatibility with *current
versions* of composer and phpunit *until replacements exist*. No specific
criteria for whether a replacement is good enough have been supplied.

They also imply that they may not support full use of those tools, versus
only features of the tools required for whatever use cases they decide to
support.


> [... much discussion of garbage collection ...] It may be a good
> opportunity to take a hard look at our
> Hooks system and figure out if its design is future-proof.
>

I note our hook system has nothing to do with garbage collection or
destructors. It does rely on references, since that's how PHP handles
output parameters.[1] And in particular, explicit references are needed to
handle output parameters combined with call_user_func_array().

Garbage collection and destructors do make a major difference to the use of
RAII patterns[2] such as ScopedCallback and our database classes.

[1]: https://en.wikipedia.org/wiki/Output_parameter
[2]: https://en.wikipedia.org/wiki/RAII


--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 20/09/17 02:40, C. Scott Ananian wrote:
> For example, the top-line github stats are:
> hhvm: 504 contributors (24,192 commits)
> php-src: 496 contributors (104,566 commits)

There's a reason I've contributed loads of code to HHVM and hardly any
to PHP. Although the HHVM folks were sometimes tied up in internal
work and not available, when they were available, my interactions with
them were very pleasant. They were enthusiastic about my
contributions, and were happy to have long design discussions on IRC.
Code review sometimes had a lot of back and forth, but at least they
were positive and engaged. The people I dealt with in code review
usually had it as their job to accept community contributions. My bug
reports were respectfully treated.

It was a big contrast to my interactions with the PHP community, which
were so often negative. For example, Jani's toxic behaviour on the bug
tracker, closing bugs as "bogus" despite being serious and
reproducible, usually because he didn't understand them technically.
Even with other maintainers, I had to fight several times to keep
serious bugs open. I had no illusions that they would ever be fixed, I
just wanted them to be open for my reference and for the benefit of
anyone hitting the same issue. I filed bugs as "documentation issues",
requesting that undesired behaviour be documented in the manual, since
they were more likely to stay open that way.

My interactions with Derick Rethans were quite unpleasant, he would
not even consider accepting the code I wrote to fix a DoS
vulnerability which was affecting us constantly. He wouldn't provide a
code review, he just rejected it on principle. Instead he wrote his
own version of it a couple of years later. He seemed to think that
every line of code in the date module should be attributable to him.

Design discussions are apparently concentrated on the internals
mailing list, where there is an incredible amount of negativity for
any new idea. Developers really need a lot of energy to keep answering
negative comments, over and over for a period of months, in order to
get their RFCs accepted. Some language features which eventually made
it into PHP, such as short arrays, were shot down many times on the
mailing list before they found a champion who was sufficiently brave
and influential.

Their code review practices were quite archaic, I don't know if
they've improved. Their coding style is also dated.

Stas was great, a bright spot in a dismal field, which was why I was
so keen to hire him.

So I'm not looking forward to returning to PHP. But at least we know
what we are getting ourselves in for. Being community-driven means it
has inertia, change is slow. Facebook have been inconsistent with
HHVM, and have made it clear that they don't intend to cater to our needs.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Sep 19, 2017 9:45 PM, "Tim Starling" <tstarling@wikimedia.org> wrote:

Facebook have been inconsistent with
HHVM, and have made it clear that they don't intend to cater to our needs.


I'm curious: is this conclusion based on your recent meeting with them, or
on past behavior? Their recent announcement had a lot of "we know we
haven't been great, but we promise to change" stuff in it ("reinvest in
open source") and I'm curious to know if they enumerated concrete steps
they planned to take, or whether even in your most recent meeting with them
they failed to show actual interest.
--scott
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 20/09/17 12:19, C. Scott Ananian wrote:
> On Sep 19, 2017 9:45 PM, "Tim Starling" <tstarling@wikimedia.org> wrote:
>
> Facebook have been inconsistent with
> HHVM, and have made it clear that they don't intend to cater to our needs.
>
>
> I'm curious: is this conclusion based on your recent meeting with them, or
> on past behavior? Their recent announcement had a lot of "we know we
> haven't been great, but we promise to change" stuff in it ("reinvest in
> open source") and I'm curious to know if they enumerated concrete steps
> they planned to take, or whether even in your most recent meeting with them
> they failed to show actual interest.

"Have been inconsistent" refers to their past behaviour. "Don't intend
to cater" refers to the meeting and announcement.

According to people on the ops team who have worked with them
recently, they stopped working on the open source product altogether.
They stopped responding to bug reports. By "reinvest in open source"
they are apologising for that and promising to start reading their bug
mail again. This was discussed in the meeting.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 8:37 PM, Tim Starling <tstarling@wikimedia.org>
wrote:

> According to people on the ops team who have worked with them
> recently, they stopped working on the open source product altogether.
> They stopped responding to bug reports.


This makes it sound as if your estimation of one year to migrate off HHVM
is too long and we need to run for it even sooner.


--
Best regards,
Max Semenik ([[User:MaxSem]])
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

?????? 19 ????? 2017 22:52,? "Kevin Smith" <ksmith@wikimedia.org> ???:

On Tue, Sep 19, 2017 at 11:41 AM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Think of this like
> the traditional [[devil's advocate]] process for canonizing a saint. I'm
> just challenging us to take a hard look at our reasoning and make sure it
> is well-founded technically.
>

?I for one appreciate your engagement here, even if it has not been
popular.



+1

Rejecting HHVM and Hack may seem obvious to many people, but not to
everybody.
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> It was a big contrast to my interactions with the PHP community, which
> were so often negative. For example, Jani's toxic behaviour on the bug
> tracker, closing bugs as "bogus" despite being serious and
> reproducible, usually because he didn't understand them technically.
> Even with other maintainers, I had to fight several times to keep
> serious bugs open. I had no illusions that they would ever be fixed, I
> just wanted them to be open for my reference and for the benefit of
> anyone hitting the same issue. I filed bugs as "documentation issues",
> requesting that undesired behaviour be documented in the manual, since
> they were more likely to stay open that way.

By your mention of Jani, I derive it was a long time ago :) Quite a lot
changed since then, even though the internals list is still not the most
friendly place on the nets. But the processes got better, I think, and
the level of maturity of both discussion and participants increased.

Also, thank you for the kind words, and I'll be glad to help with what I
can if we need something done in PHP core.
--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 19/09/17 10:13, Tim Starling wrote:
> I'll run a benchmark

I upgraded the test wiki container on my laptop from Ubuntu 14.04 to
16.04, which also necessitated a platform switch from schroot to
systemd-nspawn. The benchmark is thus approximately native performance
on a Core i5 4210U @ 1.7 GHz. I killed the usual CPU hogs so that
everything was quiet in top. Then I ran benchmarkParse.php on a copy
of the [[Australia]] article, including templates, with --loops 3.

The results were

PHP 7.0: 1.59 seconds
HHVM 3.21: 1.75 seconds

So PHP 7 was significantly faster on this test.

Note that I ran HHVM with JIT enabled; total wall clock time including
compilation and warmup was about 75 seconds, compared to 13 seconds
for PHP 7.

The test wiki has Scribunto with LuaStandalone. Debug logging was
disabled for this test.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

About third-parties.

* Regardless of whether PHP 7 (or 8, or 9) is more similar to PHP 5 than
Hack is.
* Regardless of whether upgrading from PHP 5 to 7 is more or less difficult
than upgrading to Hack.
* Regardless of whether Zend PHP is going to make big
backwards-incompatible changes in the future (like HHVM is doing now in
Hack, e.g. what if Zend PHP 8 or 9 were to also drop references and
destructors?).

Regardless of these "our" costs, a site-admin must work with the options
available. We can expect current PHP 5 deployments to also (soon) have PHP
7. Shared hosting, dedicated hosting (with shell but limited packages), and
platform-as-a-service (PaaS), and other environments have, or will soon,
provide an option/package/image for PHP 7.

But having them consider a new run-time (e.g. Node, Go, Perl, Hack) was
always a good argument against "Let's rewrite MediaWiki in X".

On the other hand, I don't think we should consider it as something that
can't happen. We shouldn't decide on the account of every single hosting
provider. Some are probably still running PHP 4 and customers will have
decided to switch to a more caring provider. If no providers add Hack, the
loss is ours (leaving site-admins with no similar-cost options). If enough
providers switch, the loss is the provider's (customer leaves or provider
follows suit). I don't think a PHP-only provider is necessarily cheaper
than a more diverse provider. They're just differently focussed.

The marketshare of sites runnable on pure "LAMP" seems to be shrinking with
the other run-times gaining popularity, and new run-times being introduced.
I'd expect a hosting service providing only Zend PHP to have a shrinking
customer base in the long run. We might not be looking at "Will they add
Hack as the first non-PHP run-time option", but rather "Will they add Hack
among the other run-times". How do we feel about that? Does this seem like
something that could happen, as well as something we could *help* make
happen?

-- Timo


On Mon, Sep 18, 2017 at 9:58 PM, Max Semenik <maxsem.wiki@gmail.com> wrote:

> Today, the HHVM developers made an announcement[1] that they have plans of
> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> instead.
>
> While this does not mean that we need to take an action immediately,
> eventually we will have to decide something. As I see it, our options are:
>
> 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> PHP. This, however, is a dead end and will make things progressively harder
> as the implementations will diverge and various Composer libraries we use
> will start requiring some Zend-specific features.
>
> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
>
> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.
>
> I personally think that 3) is the only viable option in the long run. What
> do you think?
>
> ----
> [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
>
> --
> Best regards,
> Max Semenik ([[User:MaxSem]])
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Mon, Sep 18, 2017 at 1:58 PM, Max Semenik <maxsem.wiki@gmail.com> wrote:

> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
>

Hack has a couple interesting points, especially its async system, but
isn't _hugely_ compelling at this point.

If they're going to be dropping destructors and references we'd have to
retool our RAII patterns and our hook 'out-param' patterns to work with
future-Hack, but that's possible if we *want* to make such a change.

However I don't see a lot of interest in such a change in this discussion
or among TechCom, and buying into a single-vendor system is a risk both for
us (if they change the language in ways we don't like or drop it) and
others (harder to set up if few HHVM providers).


>
> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.
>

Migrating WMF's implementation to PHP 7 is probably the way to go. I leave
it up to ops to figure out how to make the change. :)

-- brion
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Wed, Sep 20, 2017 at 10:56 PM, Brion Vibber <bvibber@wikimedia.org>
wrote:

> On Mon, Sep 18, 2017 at 1:58 PM, Max Semenik <maxsem.wiki@gmail.com>
> wrote:
>
> >
> > 3) Revert WMF to Zend and forget about HHVM. This will result in
> > performance degradation, however it will not be that dramatic: when we
> > upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
> while
> > 5.6 and 7 provided nice performance improvements.
> >
>
> Migrating WMF's implementation to PHP 7 is probably the way to go. I leave
> it up to ops to figure out how to make the change. :)
>


I think this is the more viable option too, mostly not to cause huge issues
to non-Wikimedia users. The ease of installation and operation of PHP on
most platforms compared to HHVM is incomparable. Just to make an example, I
don't remember having to check the code of the VM to understand what an ini
setting does with PHP (or even PHP-fpm), while with HHVM that has been a
recurring pain, as options come and go without any warning between
versions, not to mention the online docs that can even be plainly
misleading. At the moment, there is no doubt PHP is a much friendlier
environment for any third-party wiki.


But I think there is other value in going the PHP7 way; I had actually
pitched the idea of switching to PHP7 for some time now, for various
reasons, namely:
- We use HHVM differently than what Facebook does. For instance, we use the
fcgi server mode, and not the pure http one, and we don't run in repoAuth
mode. This has brought us new bugs to solve every time we upgrade, as there
is no battle testing in production for those code patterns at scale before
we use it.
- The recent, prolonged difficulty of interaction with the FLOSS community,
although acknowledged by the HHVM team as something they're willing to fix,
is worrying in itself.
- PHP7 has shown performance comparable to HHVM for most PHP shops that
migrated. So the single most compelling reason for which we migrated
(performance) might not be a factor anymore. Using a runtime readily
available (and security-patched) by the upstream distribution would make
the ops team lives easier as well.

As for the actual migration:

I don't think there is any need to panic or rush to a decision, but the
timeline is pretty set: by the end of 2018, when official support for HHVM
3.24 will end, any migration should be well underway within the WMF
infrastructure. I expect a migration from HHVM to PHP 7 to be a less
formidable undertaking than the switch from PHP 5.3 to HHVM - we did repay
a good deal t of the tech debt in the Wikimedia Foundation installation
back then, and we won't have to change radically the way we serve
MediaWiki, as PHP 7 works as a FastCGI server as well. Still, it will take
time and resources, and it needs to be planned in advance.

One important consequence of the announcement for Wikimedia is that we
won't be able to use any version of MediaWiki not compatible with PHP 5.x
until we transition to PHP7 (unless we decide to support both HHVM and
PHP7). This might be important in steering the timing of the change in
MediaWiki itself.

Cheers,

Giuseppe

--
Giuseppe Lavagetto, Ph.d.
Senior Technical Operations Engineer, Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 19/09/17 20:56, Gilles Dubuc wrote:
> Should we have a TechComm-driven meeting about this ASAP?

We had an IRC meeting about this yesterday. Here is the log:

<https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.2017-09-27-21.03.log.html>

We mostly talked about the migration plan for production, including
the likely timeline.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l