Mailing List Archive

> On Sep 19, 2017, at 6:56 AM, Gilles Dubuc <gilles@wikimedia.org> wrote:
>
> Should we have a TechComm-driven meeting about this ASAP?
>
> Like others, I don't expect that there will be disagreement about the way
> to go, but there is a lot to discuss about what needs to be done,
> resourcing, etc.
>
> It would be nice to have Ori around for it, to pick his brains about any
> undocumented or little-known knowledge about the HHVM migration that could
> bite us when migrating to PHP 7.x if we don't know about it.
>
> On Tue, Sep 19, 2017 at 9:07 AM, Moritz Muehlenhoff <
> mmuhlenhoff@wikimedia.org> wrote:
>
>> On Tue, Sep 19, 2017 at 10:13:47AM +1000, Tim Starling wrote:
>>> On 19/09/17 06:58, Max Semenik wrote:
>>>> Today, the HHVM developers made an announcement[1] that they have
>> plans of
>>>> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
>>>> instead.
>>>
>>> The HHVM team did tell us privately that they were planning on
>>> changing their strategy, basically as you describe it above. The
>>> surprising things for me in this announcement were:
>>>
>>> * The plan to also drop PHP 5 compatibility, on a short timeline (1
>> year).
>>> * Rather than "drifting away" from PHP, their top priority plans
>>> include removing core language features like references and destructors.
>>>
>>>> While this does not mean that we need to take an action immediately,
>>>> eventually we will have to decide something.
>>>
>>> Actually, I think a year is a pretty short time for ops to switch to
>>> PHP 7. I think we need to decide on this pretty much immediately.
>>
>> The next step would be the upgrade of the mw* fleet to Debian stretch
>> while still using HHVM 3.18 (to minimise disruption since we've stabilised
>> 3.18 in it's current build). That work is tracked at T174431. 3.18 will
>> be supported by upstream for at least another six months (and if the
>> migration drags
>> further I can roll custom 3.18 security backports from later LTS releases)
>>
>> Debian stretch ships PHP7, so that'd be a good stepstone to migrate
>> back to Zend.
>>
>> Cheers,
>> Moritz
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 11:33 AM, bawolff <bawolff+wn@gmail.com> wrote:

> I disagree. I don't think its useful or possible to try to forecast
> technical trends 15 years out. 15 years from now, it is just as likely
> that facebook will be as relevant as myspace is today, as it is that
> facebook will go full cyberpunk dystopia on us and rule the world. I
> don't think we can realistically predict anything 15 years out.
>
>
That's why I argued we should consider the code quality of both, and
determine which we'd feel most comfortable supporting on our own in the
case our crystal balls are wrong and we end up holding the bag for the
runtime. (This is just one factor to consider; my argument is that we
should carefully enumerate the various factors before making an evaluation,
not just dismiss any of the options out of hand.)
--scott

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 11:34 AM, Bryan Davis <bd808@wikimedia.org> wrote:

> On Tue, Sep 19, 2017 at 9:21 AM, C. Scott Ananian
> <cananian@wikimedia.org> wrote:
> > There are other big users of HHVM -- do we know what other members of the
> > larger community are doing? We've heard that Phabricator intends to
> follow
> > PHP 7. Etsy also shifted to HHVM, do we know what their plans are?
>
> Etsy 'experimented' with HHVM [0] and then eventually switched to PHP7
> as their primary runtime. The blog posts about this are a little
> scattered, but Rasmus spoke about it [1] and Etsy started the phan
> project [2].
>

I got confirmation on twitter:
https://twitter.com/jazzdan/status/910162545805336576


> For what it's worth, my opinion is that PHP is an actual FLOSS
> software project with years of history and core contributions from
> Zend who make their living with PHP. HHVM is a well funded internal
> project from Facebook that has experimented with FLOSS but ultimately
> is controlled by the internal needs of Facebook. For me the choice
> here is obviously to back the community driven FLOSS project and help
> them continue to thrive.
>

Fair enough. My point is just that we should stop and reflect that this is
a major inflection point. Language choices are sticky, so this decision
will have significant long-term implications. We should at least stop to
evaluate PHP7 vs Hack and determine which is a better fit for our codebase,
and do due diligence on both sides (count how many engineers, how many open
source contributors, commit rates, etc). HHVM has been flirting with a
LLVM backend, and LLVM itself has quite a large and active community. The
PHP community has had issues with proper handling of security patches in
the past. I'm suggesting to proceed cautiously and have a proper
discussion of all the factors involved instead of over-simplifying this to
"community" vs "facebook".

For example, the top-line github stats are:
hhvm: 504 contributors (24,192 commits)
php-src: 496 contributors (104,566 commits)

HHVM seems to have a larger community of contributors despite a much
shorter active life. But note that the PHP github mirror has been broken
since Jul 29 (!). In the past 6 days I count 8 distinct contributors to
php-src, and 10 distinct contributors in the past *two days* to hhvm (one
of whom contributed an OCAML frontend(!)). These are just hand-wavy
figures; ideally we should try to determine how many of the recent
contributors to each project are employed by Facebook and/or Zend.

I think there's room for a reasonable debate.
--scott
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 9:40 AM C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Fair enough. My point is just that we should stop and reflect that this is
> a major inflection point. Language choices are sticky, so this decision
> will have significant long-term implications. We should at least stop to
> evaluate PHP7 vs Hack and determine which is a better fit for our codebase,
> and do due diligence on both sides (count how many engineers, how many open
> source contributors, commit rates, etc). HHVM has been flirting with a
> LLVM backend, and LLVM itself has quite a large and active community. The
> PHP community has had issues with proper handling of security patches in
> the past. I'm suggesting to proceed cautiously and have a proper
> discussion of all the factors involved instead of over-simplifying this to
> "community" vs "facebook".
>
>
I'm not trying to simplify this into community vs. facebook. And let's also
be clear: we never chose HHVM for Hack. We don't use Hack. The one
experiment I had at trying Hack never panned out. MediaWiki is in PHP, not
Hack.

The *only* reason we're having a language discussion is because HHVM has
announced that they're abandoning PHP in favor of Hack. If someone had some
to the list last week and said "Hey let's abandon PHP for XYZLang" they
would've been rightly laughed off.

The debate here is between runtimes for PHP, and on the long enough
timescale there's only one option. PHP has a long-standing history of being
a viable runtime. HHVM does not.

I don't see this as an A/B choice at all, but rather a clear path forward.
So sure: let's have an RfC/TechComm meeting to work out the details, but
let's not pretend that option #2 is even remotely viable. It is not.

-Chad
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 1:17 PM, Chad <innocentkiller@gmail.com> wrote:

> be clear: we never chose HHVM for Hack. We don't use Hack. The one
> experiment I had at trying Hack never panned out. MediaWiki is in PHP, not
> Hack.
>

To be super clear: MediaWiki is in *PHP5*. The choices are:

1) MediaWiki will always be in PHP5.
2) MediaWiki will eventually migrate to PHP7, or
3) MediaWiki will eventually migrate to Hack.

Is anyone arguing for #1?

So we've got two backwards-incompatible choices to make, eventually.
--scott

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 10:28 AM C. Scott Ananian <cananian@wikimedia.org>
wrote:

> On Tue, Sep 19, 2017 at 1:17 PM, Chad <innocentkiller@gmail.com> wrote:
>
> > be clear: we never chose HHVM for Hack. We don't use Hack. The one
> > experiment I had at trying Hack never panned out. MediaWiki is in PHP,
> not
> > Hack.
> >
>
> To be super clear: MediaWiki is in *PHP5*. The choices are:
>
> 1) MediaWiki will always be in PHP5.
> 2) MediaWiki will eventually migrate to PHP7, or
> 3) MediaWiki will eventually migrate to Hack.
>
> Is anyone arguing for #1?
>
> So we've got two backwards-incompatible choices to make, eventually.
>

I see everyone saying #1 for now and #2 down the road. You're the only one
here who seems to think #3 is even worth considering. The rest of us
have--rightly--dismissed it already.

-Chad
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 19 Sep 2017 5:40 pm, "C. Scott Ananian" <cananian@wikimedia.org> wrote:

I'm suggesting to proceed cautiously and have a proper
discussion of all the factors involved instead of over-simplifying this to
"community" vs "facebook".

For example, the top-line github stats are:
hhvm: 504 contributors (24,192 commits)
php-src: 496 contributors (104,566 commits)

HHVM seems to have a larger community of contributors despite a much
shorter active life.

By a difference of 8 contributors?

But note that the PHP github mirror has been broken since Jul 29 (!).

I'm not convinced an exclamation mark in brackets is required here.
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tuesday, September 19, 2017, C. Scott Ananian <cananian@wikimedia.org>
wrote:
> On Tue, Sep 19, 2017 at 1:17 PM, Chad <innocentkiller@gmail.com> wrote:
>
>> be clear: we never chose HHVM for Hack. We don't use Hack. The one
>> experiment I had at trying Hack never panned out. MediaWiki is in PHP,
not
>> Hack.
>>
>
> To be super clear: MediaWiki is in *PHP5*. The choices are:
>
> 1) MediaWiki will always be in PHP5.
> 2) MediaWiki will eventually migrate to PHP7, or
> 3) MediaWiki will eventually migrate to Hack.
>
> Is anyone arguing for #1?
>
> So we've got two backwards-incompatible choices to make, eventually.
> --scott
>
> --
> (http://cscott.net)
>

No, MediaWiki is written in a subset of php5 which is forwards compatible
with php7. The backwards incompatible things are by in large things no sane
code would rely on (http://php.net/manual/en/migration70.incompatible.php)
There are people who are using mediawiki with php7 right now without
complaint. Presumably eventually we will move to needing php7, just like
once we moved from php4 to php5, but thats really just a version
requirement increment.


Sure there may be backwards incompatible things that come up, but thats
nothing new. E.g. We used to have a class named Namespace. Then php 5.3
came along and its now MWNamespace. All signs point to php7
incompatibilities to be of this form.
--
Brian
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Chad: the more you argue that it's not even worth considering, the more I'm
going to push back and ask for actual numbers and facts. ;)

Alex: I admit 8 contributors is not significant, but 24,192 commits vs
104,566 commits is. The conclusion I was suggesting was "same number of
contributors over a much shorter time frame".

Re: github mirror broken: note that http://php.net/build-setup.php mentions
*only* the github mirror. And the github mirror has been broken for almost
two months now. There's no mention of the "real" repo. That merits an
exclamation mark, I think.
--scott
?
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi,

On 09/19/2017 10:27 AM, C. Scott Ananian wrote:
> To be super clear: MediaWiki is in *PHP5*. The choices are:

Well, kind of. MediaWiki has full support for PHP 7 while still
maintaining PHP 5 support.

> 1) MediaWiki will always be in PHP5.
> 2) MediaWiki will eventually migrate to PHP7, or
> 3) MediaWiki will eventually migrate to Hack.
>
> Is anyone arguing for #1?

I don't think anyone has argued for anything besides #2.

> So we've got two backwards-incompatible choices to make, eventually.

There's already an open RfC[1] about bumping the minimum PHP requirement
to 7.x.

[1] https://phabricator.wikimedia.org/T172165

-- Legoktm

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi,

On 09/19/2017 10:50 AM, C. Scott Ananian wrote:
> Chad: the more you argue that it's not even worth considering, the more I'm
> going to push back and ask for actual numbers and facts. ;)

Migrating to Hack simply isn't feasible for most users of MediaWiki.
Most distros don't have packages for HHVM, and it's not straightforward
to build AIUI.

Once you've figured that out, then you're going to quickly realize that
you need a cron job to regularly restart HHVM because it'll
OOM/crash/etc. So either you have downtime or multiple HHVM processes.

On the MediaWiki development side, we've been forced for fork all of the
external libraries we depend upon to support Hack. This includes basic
tooling like composer, phpunit, codesniffer, and so on.

There are probably more reasons that could be listed, but I think you
get the idea of why Chad considers it to be a non-starter.

-- Legoktm

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 10:50 AM C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Chad: the more you argue that it's not even worth considering, the more I'm
> going to push back and ask for actual numbers and facts. ;)
>
>
As I said in my prior message: it's been considered, and discarded rather
quickly. It doesn't need much further introspection than that. A couple of
major points:

* Brian W's point is correct: there's not much migration cost at all moving
to PHP7. Moving between PHP versions has always been pretty easy for us in
MW--we write very conservative code as it is.
* Various (there's lots from lots of people) benchmarks routinely show HHVM
and PHP7--especially 7.1+--performance to be roughly on par, depending on
workloads. We can get some firmer numbers here.
** A bunch of major projects have already dropped HHVM for PHP7+. Etsy,
Symphony, Wordpress (they no longer test against it)
* Swapping to HHVM/Hack would abandon many/most of our downstream users --
the stats Stas pointed to way far back shows world install base as well
under 0.5% of all PHP runtimes.
* Speaking of downstream users: HHVM has always been a second-class citizen
on non-Linux OSes. It's always been wonky on OSX (I would know, I was the
first person to ever get it built), and I don't think anyone's ever got it
working on Windows outside of Cygwin--please correct me if I'm wrong?
* It greatly complicates setup and development work for both WMF and
volunteers.
** Did I mention almost nobody has up to date packages for this?
* PHP has a long-documented history of working as a proper FLOSS project.
Facebook's track record here has been less than stellar--it comes in fits
and starts and there's a lot of "throwing code over the wall"
** We have a core PHP contributor (Stas) on this list. Other than
occasional patches we've shipped upstream to HHVM, I doubt anyone would
claim themselves a core HHVM contributor around here [maybe Tim early on,
heh]
* Choices HHVM is going to make upstream (references, destuctors) are a
/huge/ issue for us. The former is basically a requirement for our Hooks
system and the latter is used all over the dang place for profiling and
other fun scope-based tricks.
* Libraries we use -- composer, phpunit, etc can not be depended on to
support Hack in the long term, and there's no tooling in the HHVM world for
this (promises notwithstanding).

-Chad
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> I agree the short timeline seems to push us toward reverting to Zend. But
> it is worth having a meaningful discussion about the long-term outlook.
> Which VM is likely to be better supported in 15 years' time? Which VM

15 years is a lot to predict. 15 years ago Facebook, Twitter, Reddit and
Linkedin did not exist and Slashdot, Livejournal, etc. were all the
rage. We don't even know if Facebook as such would exist in 15 years or
would have budget to support its own language.

> would we rather adopt and maintain indefinitely ourselves, if needed --
> since in a 15 yr timeframe it's entirely possible that (a) Facebook could
> abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining

While (b) could happen, PHP project is not very dependent on Zend for
its existence. Zend owns none of the infrastructure or processes, and
while a lot of performance work on PHP 7 was conducted by Zend team (and
they are still working on improvements AFAIK), there are plenty of
community members that do not work for Zend and do not depend on Zend in
any way. Of course, it is possible that the whole community would
implode, but here we have many more stakeholders than in Hack case,
where the stakeholder is mostly a single - albeit large and currently
very successful - company.

> speaking, it's not really a choice between "lock-in" and "no lock in" -- we
> have to choose to align our futures with either Zend Technologies Ltd or
> Facebook. One of these is *much* better funded than the other. It is

Again, I do not think this is the right statement to make. The control
of Zend Tech as a company over the future of PHP is much less than
Facebook's one over Hack (which is pretty much absolute). PHP is guided
by the community, decisions are taken by using community processes in
which Zend does not have any special role, and PHP project could survive
reasonably survive without Zend, even if with less resources. Most PHP
infrastructure - Composer, debugging, IDEs, profiling, code quality,
frameworks, etc. - are completely independent of Zend (which also has a
number of tools, but it is not the only provider). So I do not think it
is an adequate comparison.

I am not sure if Hack has an open-source community outside Facebook (if
anybody has pointers to that, please share - commit numbers certainly
don't tell much) - but it is pretty clear to me that Facebook is in
absolute control over this platform. This is not the case with Zend and PHP.

--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Zend was testing against hhvm at one point, then decided they would stop testing against it. See https://www.phoronix.com/scan.php?page=news_item&px=PTS-PHP-7.1-Benchmarks
On Tuesday, 19 September 2017, 19:24:49 BST, Stas Malyshev <smalyshev@wikimedia.org> wrote:

Hi!

> I agree the short timeline seems to push us toward reverting to Zend.  But
> it is worth having a meaningful discussion about the long-term outlook.
> Which VM is likely to be better supported in 15 years' time?  Which VM

15 years is a lot to predict. 15 years ago Facebook, Twitter, Reddit and
Linkedin did not exist and Slashdot, Livejournal, etc. were all the
rage. We don't even know if Facebook as such would exist in 15 years or
would have budget to support its own language.

> would we rather adopt and maintain indefinitely ourselves, if needed --
> since in a 15 yr timeframe it's entirely possible that (a) Facebook could
> abandon Hack/HHVM, or (b) the PHP Zend team could implode.  Maintaining

While (b) could happen, PHP project is not very dependent on Zend for
its existence. Zend owns none of the infrastructure or processes, and
while a lot of performance work on PHP 7 was conducted by Zend team (and
they are still working on improvements AFAIK), there are plenty of
community members that do not work for Zend and do not depend on Zend in
any way. Of course, it is possible that the whole community would
implode, but here we have many more stakeholders than in Hack case,
where the stakeholder is mostly a single - albeit large and currently
very successful - company.

> speaking, it's not really a choice between "lock-in" and "no lock in" -- we
> have to choose to align our futures with either Zend Technologies Ltd or
> Facebook.  One of these is *much* better funded than the other.  It is

Again, I do not think this is the right statement to make. The control
of Zend Tech as a company over the future of PHP is much less than
Facebook's one over Hack (which is pretty much absolute). PHP is guided
by the community, decisions are taken by using community processes in
which Zend does not have any special role, and PHP project could survive
reasonably survive without Zend, even if with less resources. Most PHP
infrastructure - Composer, debugging, IDEs, profiling, code quality,
frameworks, etc. - are completely independent of Zend (which also has a
number of tools, but it is not the only provider). So I do not think it
is an adequate comparison.

I am not sure if Hack has an open-source community outside Facebook (if
anybody has pointers to that, please share - commit numbers certainly
don't tell much) - but it is pretty clear to me that Facebook is in
absolute control over this platform. This is not the case with Zend and PHP.

--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> Incidentally, how much work has been done on incorporating HHVM's
> improvements back into Zend?

Depends on which ones you're talking about. Syntax ones may or may not
find its way into PHP, but performance ones would probably be completely
different from HHVM - i.e. the resulting performance may or may not be
on par or better, but reusing most of the performance work on HHVM in
PHP would not be possible due to completely different engine internals.

So pretty much all that can be taken from HHVM into PHP would be "this
syntax looks like a good idea, let's reimplement it".
--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 09/19/2017 10:34 AM, Bryan Davis wrote:

> For what it's worth, my opinion is that PHP is an actual FLOSS
> software project with years of history and core contributions from
> Zend who make their living with PHP. HHVM is a well funded internal
> project from Facebook that has experimented with FLOSS but ultimately
> is controlled by the internal needs of Facebook. For me the choice
> here is obviously to back the community driven FLOSS project and help
> them continue to thrive.

+1.

As a practical choice, given the wide usage of PHP on the web, it makes
a lot more sense to throw our weight behind Zend development since it is
more likely to survive than HHVM which is more subject to the whims and
fancies of FB's internal choices which we are not privy to and cannot
influence in any meaningful way. I mean how big of a user besides
Mediawiki serving Wikipedia sites does it have to be for FB to consider
other user's needs?

Subbu.

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Chad: thanks for writing up all of those in one place. I'm not actually
trying to argue *for* Hack here. My argument is just that we should take
the time to lay out the arguments carefully on both sides, since this is a
decision that is likely to have long-lasting effects. Think of this like
the traditional [[devil's advocate]] process for canonizing a saint. I'm
just challenging us to take a hard look at our reasoning and make sure it
is well-founded technically.

In my ideal world we'd take your excellent point-by-point argument and try
to rigorously quantify each, just to satisfy ourselves that we've done due
diligence. You say "there's not much migration cost moving to PHP7" --
well, it would be nice to assign someone to go through the details in a
little more detail to double-check that. "Various benchmarks dhow HHVM and
PHP7 performance to be roughly on par" -- firmer numbers would be nice, as
you say. I think Tim was already going to do this? "A bunch of major
projects have already dropped HHVM" -- again, can we take the devil's
advocate position and ask the HHVM team who *hasn't* dropped HHVM ;) and
ask for their reasons? You also raise some points about packaging and
portability that I could probably counter by pointing out that HHVM has an
ARM and PowerPC JIT now (
http://hhvm.com/blog/2017/03/09/how-the-cyber-elephant-got-his-arm.html),
while Zend's JIT-in-progress only supports x86. We could also poke at the
HHVM folks and ask about their packaging plans, since they (according to
their announcement) are going to "reinvest in open source". You also
mentioned PHP's long history of FLOSS without also mentioning their long
history at sucking at security. And we could try to quantify how dependent
PHP is on Zend contributions vs how dependent HHVM is on facebook
contributions. We can get numbers on this, we don't have to assume things.

The HHVM announcement specifically mentioned that they will maintain
compatibility with composer and phpunit, so that seems to be a wash.

The references/destructors thing is actually really interesting as a
technical discussion. The reason given for removing these is all about
performance, in particular the performance hit you get from ref-counting as
opposed to pure GC. There's been a lot of really good GC work done in the
past two decades; see https://news.ycombinator.com/item?id=11759762
although I like to cite https://www.cs.princeton.edu/~appel/papers/45.ps as
foundational. It may be a good opportunity to take a hard look at our
Hooks system and figure out if its design is future-proof.
--scott

On Tue, Sep 19, 2017 at 2:04 PM, Chad <innocentkiller@gmail.com> wrote:

> On Tue, Sep 19, 2017 at 10:50 AM C. Scott Ananian <cananian@wikimedia.org>
> wrote:
>
> > Chad: the more you argue that it's not even worth considering, the more
> I'm
> > going to push back and ask for actual numbers and facts. ;)
> >
> >
> As I said in my prior message: it's been considered, and discarded rather
> quickly. It doesn't need much further introspection than that. A couple of
> major points:
>
> * Brian W's point is correct: there's not much migration cost at all moving
> to PHP7. Moving between PHP versions has always been pretty easy for us in
> MW--we write very conservative code as it is.
> * Various (there's lots from lots of people) benchmarks routinely show HHVM
> and PHP7--especially 7.1+--performance to be roughly on par, depending on
> workloads. We can get some firmer numbers here.
> ** A bunch of major projects have already dropped HHVM for PHP7+. Etsy,
> Symphony, Wordpress (they no longer test against it)
> * Swapping to HHVM/Hack would abandon many/most of our downstream users --
> the stats Stas pointed to way far back shows world install base as well
> under 0.5% of all PHP runtimes.
> * Speaking of downstream users: HHVM has always been a second-class citizen
> on non-Linux OSes. It's always been wonky on OSX (I would know, I was the
> first person to ever get it built), and I don't think anyone's ever got it
> working on Windows outside of Cygwin--please correct me if I'm wrong?
> * It greatly complicates setup and development work for both WMF and
> volunteers.
> ** Did I mention almost nobody has up to date packages for this?
> * PHP has a long-documented history of working as a proper FLOSS project.
> Facebook's track record here has been less than stellar--it comes in fits
> and starts and there's a lot of "throwing code over the wall"
> ** We have a core PHP contributor (Stas) on this list. Other than
> occasional patches we've shipped upstream to HHVM, I doubt anyone would
> claim themselves a core HHVM contributor around here [maybe Tim early on,
> heh]
> * Choices HHVM is going to make upstream (references, destuctors) are a
> /huge/ issue for us. The former is basically a requirement for our Hooks
> system and the latter is used all over the dang place for profiling and
> other fun scope-based tricks.
> * Libraries we use -- composer, phpunit, etc can not be depended on to
> support Hack in the long term, and there's no tooling in the HHVM world for
> this (promises notwithstanding).
>
> -Chad
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 2:41 PM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> source". You also mentioned PHP's long history of FLOSS without also
> mentioning their long history at sucking at security.
>

Whoops, I should have toned that down a bit before hitting send. To be
clear, I'm mostly talking about the ~2007 time frame where there was a lot
of tension between the PHP core team and various folks who wanted to make
PHP more secure in different ways. I don't actually know what the
present-day status is -- suhosin seems to be still around, but (for
instance) https://sektioneins.de/en/categories/php.html hasn't had any
particular complaints since 2015.

So to be super clear: I'm just pointing out that there used to be issues
here; sometimes the community's interests do not exactly align. Consider
me in the devil's advocate role again: I'd be interested to hear an
insider's opinion (Stas?) on how security issues are handled these days and
what the future outlook is,
https://www.cvedetails.com/vulnerability-list/vendor_id-74/product_id-128/PHP-PHP.html
doesn't look as nice as
http://www.cvedetails.com/vulnerability-list/vendor_id-7758/product_id-35896/Facebook-Hhvm.html
but
maybe the latter is misleading; older vulnerabilities seem to be at
http://www.cvedetails.com/vulnerability-list/vendor_id-7758/product_id-30684/Facebook-Hiphop-Virtual-Machine.html
for instance.
--scott

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 9:40 AM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Fair enough. My point is just that we should stop and reflect that this is
> a major inflection point. Language choices are sticky, so this decision
> will have significant long-term implications. We should at least stop to
> evaluate PHP7 vs Hack and determine which is a better fit for our codebase,
> and do due diligence on both sides (count how many engineers, how many open
> source contributors, commit rates, etc). HHVM has been flirting with a
> LLVM backend, and LLVM itself has quite a large and active community. The
> PHP community has had issues with proper handling of security patches in
> the past. I'm suggesting to proceed cautiously and have a proper
> discussion of all the factors involved instead of over-simplifying this to
> "community" vs "facebook".
>

PHP is an open-source language with mature tooling and major community
buy-in. Facebook has *promised* to turn Hack into an open-source language
with mature tooling and community buy-in; almost none of that exists
currently. Once it already does, a worthwhile discussion might be had about
switching to Hack. Right now it would be incredibly irresponsible.

Also, making PHP a viable language for third parties is the core business
model of Zend. Making Hack a viable language for third parties has
absolutely nothing to do with the business model of Facebook. At any point
they might decide it is a distraction they don't need. Comparing commit
numbers is not really meaningful without knowing what fraction of those
committers can disappear overnight if Facebook reconsiders its priorities.

IMO the more interesting discussion to be had is how little we invest into
the technology our whole platform is based on. You'd think the largest
production user of PHP would pay at least one part-time PHP developer, or
try to represent itself in standards and roadmap discussions, but we do
not. Is that normal?
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 11:41 AM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Think of this like
> the traditional [[devil's advocate]] process for canonizing a saint. I'm
> just challenging us to take a hard look at our reasoning and make sure it
> is well-founded technically.
>

?I for one appreciate your engagement here, even if it has not been
popular.

It is easy for a group to make a snap decision, or to end up in an echo
chamber, where decisions seem "obvious" to everyone who is participating.
Yes, sometimes there are easy or obvious decisions. But with the stakes so
high on this one, it seems prudent to think it through, enumerate the
arguments, and invite discussion.

If the decision is really so clearcut, the analysis and discussion should
be quick and painless. Hopefully it can also be civil and respectful,
especially toward those arguing the minority view (whether because they
believe it, or because they are just trying to help reach the best possible
outcome).

For what it's worth, I have added this topic to the TechCom internal
meeting agenda for this week. They can decide whether or not to schedule an
IRC meeting, or to take other steps.

Kevin
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> So to be super clear: I'm just pointing out that there used to be issues
> here; sometimes the community's interests do not exactly align. Consider
> me in the devil's advocate role again: I'd be interested to hear an
> insider's opinion (Stas?) on how security issues are handled these days and
> what the future outlook is,
> https://www.cvedetails.com/vulnerability-list/vendor_id-74/product_id-128/PHP-PHP.html

OK, so it's a big topic but I can do some quick survey and we can get
deeper if you want to. So:

Despite what you see there, there are not a lot of genuine security
issues in PHP. Unfortunately, CVE issuance process is such that it does
not involve any consultation with the vendor about classifying the issue
and assigning severity. You can just create CVE ID for anything and how
they assign severity is kinda mystery to me, but one thing is clear -
PHP core team is not consulted about it, at least not in any way I've
ever noticed. It doesn't mean those are all wrong, but some probably
are, and a lot are misclassified.

Now, for the substance of the issues. Many of them are unserialize()
issues. For this, without getting too much into the woods, I can only
say this: there seems to be no way to make unserialize() robust against
untrusted data, and it has to do with how references, object
construction, object destruction (notice the similarity with what Hack
intends to drop? It is not a coincidence) and serialization support
works in PHP. There is too much internal structure exposed to make it
work with untrusted data now. Maybe if we redesigned the whole thing
from scratch, it _could_ be possible, but even then I kinda doubt it, at
least not without sacrificing some feature support. For now, the best
approach to this is consider using any unserialize() on untrusted data
inherently insecure. It's just too low-level to be sure no corner case
ever does anything strange.

Some of those are genuine security problems, which now mostly
concentrate in several extensions, which has been either under-served or
have very wide exposed areas. Namely, wddx - obscure format that suffers
from issues similar to unserialize() and in addition lack
maintainership, phar - which wasn't really designed to deal with
untrusted scripts but seem to kinda be moving that direction, and gd -
which mostly relies on libgd and every issue there is also automatically
extends into PHP. The latest two would probably benefit from a good
fuzzing testing, but nobody took on it yet. I suspect if hhvm uses the
same or derived code - and it likely does, I don't think they
reimplemented libgd from scratch? - many of those would also be present
in hhvm if hhvm supports these (no idea).

There are also some long-standing debates about how randomness is
handled (basically, there are many ways to get randoms, and most of them
are not suitable for security-related randomness) and some DoS issues in
PHP hash tables - the latter are mostly resolved, but in kinda temporary
way, so there's more work to be done.

Some of these issues - like https://bugs.php.net/bug.php?id=74310 - are
definitely not security issues at all. Yes, you can create bad code that
hits some corner case and produces segfault. That shouldn't happen, but
this being complex C code which is 20 years old, it does. This has
absolutely nothing to do with security, and whoever decided to issue CVE
to it and assign 7.5 severity
(https://www.cvedetails.com/cve/CVE-2017-9119/) has done a very sloppy
job. As I said, this is done with zero communication with actual PHP
team as far as I know, which is very sad, but this is the state of
affairs. Some of those are, I am sorry to say, complete baloney, e.g.
https://www.cvedetails.com/cve/CVE-2017-8923/ says:

<<The zend_string_extend function in Zend/zend_string.h in PHP through
7.1.5 does not prevent changes to string objects that result in a
negative length, which allows remote attackers to cause a denial of
service (application crash) or possibly have unspecified other impact by
leveraging a script's use of .= with a long string.>>

This is nonsense - unless you run with no memory limit at all (nobody
sane does that) and specifically allow your code not only accept
infinite untrusted data, but have it fed specifically to certain
functions arranged into a specific code pattern, no remote attacker can
do it. There are many issues that have similar claims, none of them are
actual security issues.

Some are also assigned to PHP despite them being application issues,
e.g. https://www.cvedetails.com/cve/CVE-2017-9067/. To add insult to
injury, this is 2017 CVE about a version that was EOLed in 2014. The
data quality seems to be very sad there. I tried to figure out how to
make it better a while ago but pretty much gave up on it because I
couldn't find anybody responsible or at least concerned about this sad
state of things.

OK, this came out super-long and kinda ranty (sorry!), so I will stop
for now, but if you have any questions about it, please feel free to
ping me, and I will be glad to discuss it.
--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> IMO the more interesting discussion to be had is how little we invest into
> the technology our whole platform is based on. You'd think the largest
> production user of PHP would pay at least one part-time PHP developer, or
> try to represent itself in standards and roadmap discussions, but we do
> not. Is that normal?

I think this is a very good point. There are other ways to support PHP
too. Like assisting in regular testing of upcoming versions. Helping
writing the docs. Contributing to RFC discussions (we have a large
codebase, heavily visited site, and a lot of experience dealing with it,
surely there could be a thing or two we could contribute). Triaging the
bugs (one of the most necessary, thankless, and under-appreciated jobs
in an open-source project). Probably more...
--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 2:41 PM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> You say "there's not much migration cost moving to PHP7" --
> well, it would be nice to assign someone to go through the details in a
> little more detail to double-check that.


What migration costs there were for PHP7 have probably already been paid,
since as has already been noted several developers are already running on
PHP 7 (myself included). There's nothing much open on the NewPHP
workboard,[1] and what is still open seems to be false positives (T120336,
T173850, T173849, T120694), mostly done already (T153505), deprecations of
stuff we only still have for backwards compatibility (T120333, T143788),
irrelevant (T174199), or tracking backports (T174262).

[1]: https://phabricator.wikimedia.org/project/board/346/


> The HHVM announcement specifically mentioned that they will maintain
> compatibility with composer and phpunit, so that seems to be a wash.
>

They specifically mentioned they'd maintain compatibility with *current
versions* of composer and phpunit *until replacements exist*. No specific
criteria for whether a replacement is good enough have been supplied.

They also imply that they may not support full use of those tools, versus
only features of the tools required for whatever use cases they decide to
support.


> [... much discussion of garbage collection ...] It may be a good
> opportunity to take a hard look at our
> Hooks system and figure out if its design is future-proof.
>

I note our hook system has nothing to do with garbage collection or
destructors. It does rely on references, since that's how PHP handles
output parameters.[1] And in particular, explicit references are needed to
handle output parameters combined with call_user_func_array().

Garbage collection and destructors do make a major difference to the use of
RAII patterns[2] such as ScopedCallback and our database classes.

[1]: https://en.wikipedia.org/wiki/Output_parameter
[2]: https://en.wikipedia.org/wiki/RAII


--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 20/09/17 02:40, C. Scott Ananian wrote:
> For example, the top-line github stats are:
> hhvm: 504 contributors (24,192 commits)
> php-src: 496 contributors (104,566 commits)

There's a reason I've contributed loads of code to HHVM and hardly any
to PHP. Although the HHVM folks were sometimes tied up in internal
work and not available, when they were available, my interactions with
them were very pleasant. They were enthusiastic about my
contributions, and were happy to have long design discussions on IRC.
Code review sometimes had a lot of back and forth, but at least they
were positive and engaged. The people I dealt with in code review
usually had it as their job to accept community contributions. My bug
reports were respectfully treated.

It was a big contrast to my interactions with the PHP community, which
were so often negative. For example, Jani's toxic behaviour on the bug
tracker, closing bugs as "bogus" despite being serious and
reproducible, usually because he didn't understand them technically.
Even with other maintainers, I had to fight several times to keep
serious bugs open. I had no illusions that they would ever be fixed, I
just wanted them to be open for my reference and for the benefit of
anyone hitting the same issue. I filed bugs as "documentation issues",
requesting that undesired behaviour be documented in the manual, since
they were more likely to stay open that way.

My interactions with Derick Rethans were quite unpleasant, he would
not even consider accepting the code I wrote to fix a DoS
vulnerability which was affecting us constantly. He wouldn't provide a
code review, he just rejected it on principle. Instead he wrote his
own version of it a couple of years later. He seemed to think that
every line of code in the date module should be attributable to him.

Design discussions are apparently concentrated on the internals
mailing list, where there is an incredible amount of negativity for
any new idea. Developers really need a lot of energy to keep answering
negative comments, over and over for a period of months, in order to
get their RFCs accepted. Some language features which eventually made
it into PHP, such as short arrays, were shot down many times on the
mailing list before they found a champion who was sufficiently brave
and influential.

Their code review practices were quite archaic, I don't know if
they've improved. Their coding style is also dated.

Stas was great, a bright spot in a dismal field, which was why I was
so keen to hire him.

So I'm not looking forward to returning to PHP. But at least we know
what we are getting ourselves in for. Being community-driven means it
has inertia, change is slow. Facebook have been inconsistent with
HHVM, and have made it clear that they don't intend to cater to our needs.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Sep 19, 2017 9:45 PM, "Tim Starling" <tstarling@wikimedia.org> wrote:

Facebook have been inconsistent with
HHVM, and have made it clear that they don't intend to cater to our needs.


I'm curious: is this conclusion based on your recent meeting with them, or
on past behavior? Their recent announcement had a lot of "we know we
haven't been great, but we promise to change" stuff in it ("reinvest in
open source") and I'm curious to know if they enumerated concrete steps
they planned to take, or whether even in your most recent meeting with them
they failed to show actual interest.
--scott
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l