Mailing List Archive

On 20/09/17 12:19, C. Scott Ananian wrote:
> On Sep 19, 2017 9:45 PM, "Tim Starling" <tstarling@wikimedia.org> wrote:
>
> Facebook have been inconsistent with
> HHVM, and have made it clear that they don't intend to cater to our needs.
>
>
> I'm curious: is this conclusion based on your recent meeting with them, or
> on past behavior? Their recent announcement had a lot of "we know we
> haven't been great, but we promise to change" stuff in it ("reinvest in
> open source") and I'm curious to know if they enumerated concrete steps
> they planned to take, or whether even in your most recent meeting with them
> they failed to show actual interest.

"Have been inconsistent" refers to their past behaviour. "Don't intend
to cater" refers to the meeting and announcement.

According to people on the ops team who have worked with them
recently, they stopped working on the open source product altogether.
They stopped responding to bug reports. By "reinvest in open source"
they are apologising for that and promising to start reading their bug
mail again. This was discussed in the meeting.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Sep 19, 2017 at 8:37 PM, Tim Starling <tstarling@wikimedia.org>
wrote:

> According to people on the ops team who have worked with them
> recently, they stopped working on the open source product altogether.
> They stopped responding to bug reports.


This makes it sound as if your estimation of one year to migrate off HHVM
is too long and we need to run for it even sooner.


--
Best regards,
Max Semenik ([[User:MaxSem]])
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

?????? 19 ????? 2017 22:52,? "Kevin Smith" <ksmith@wikimedia.org> ???:

On Tue, Sep 19, 2017 at 11:41 AM, C. Scott Ananian <cananian@wikimedia.org>
wrote:

> Think of this like
> the traditional [[devil's advocate]] process for canonizing a saint. I'm
> just challenging us to take a hard look at our reasoning and make sure it
> is well-founded technically.
>

?I for one appreciate your engagement here, even if it has not been
popular.



+1

Rejecting HHVM and Hack may seem obvious to many people, but not to
everybody.
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi!

> It was a big contrast to my interactions with the PHP community, which
> were so often negative. For example, Jani's toxic behaviour on the bug
> tracker, closing bugs as "bogus" despite being serious and
> reproducible, usually because he didn't understand them technically.
> Even with other maintainers, I had to fight several times to keep
> serious bugs open. I had no illusions that they would ever be fixed, I
> just wanted them to be open for my reference and for the benefit of
> anyone hitting the same issue. I filed bugs as "documentation issues",
> requesting that undesired behaviour be documented in the manual, since
> they were more likely to stay open that way.

By your mention of Jani, I derive it was a long time ago :) Quite a lot
changed since then, even though the internals list is still not the most
friendly place on the nets. But the processes got better, I think, and
the level of maturity of both discussion and participants increased.

Also, thank you for the kind words, and I'll be glad to help with what I
can if we need something done in PHP core.
--
Stas Malyshev
smalyshev@wikimedia.org

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 19/09/17 10:13, Tim Starling wrote:
> I'll run a benchmark

I upgraded the test wiki container on my laptop from Ubuntu 14.04 to
16.04, which also necessitated a platform switch from schroot to
systemd-nspawn. The benchmark is thus approximately native performance
on a Core i5 4210U @ 1.7 GHz. I killed the usual CPU hogs so that
everything was quiet in top. Then I ran benchmarkParse.php on a copy
of the [[Australia]] article, including templates, with --loops 3.

The results were

PHP 7.0: 1.59 seconds
HHVM 3.21: 1.75 seconds

So PHP 7 was significantly faster on this test.

Note that I ran HHVM with JIT enabled; total wall clock time including
compilation and warmup was about 75 seconds, compared to 13 seconds
for PHP 7.

The test wiki has Scribunto with LuaStandalone. Debug logging was
disabled for this test.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

About third-parties.

* Regardless of whether PHP 7 (or 8, or 9) is more similar to PHP 5 than
Hack is.
* Regardless of whether upgrading from PHP 5 to 7 is more or less difficult
than upgrading to Hack.
* Regardless of whether Zend PHP is going to make big
backwards-incompatible changes in the future (like HHVM is doing now in
Hack, e.g. what if Zend PHP 8 or 9 were to also drop references and
destructors?).

Regardless of these "our" costs, a site-admin must work with the options
available. We can expect current PHP 5 deployments to also (soon) have PHP
7. Shared hosting, dedicated hosting (with shell but limited packages), and
platform-as-a-service (PaaS), and other environments have, or will soon,
provide an option/package/image for PHP 7.

But having them consider a new run-time (e.g. Node, Go, Perl, Hack) was
always a good argument against "Let's rewrite MediaWiki in X".

On the other hand, I don't think we should consider it as something that
can't happen. We shouldn't decide on the account of every single hosting
provider. Some are probably still running PHP 4 and customers will have
decided to switch to a more caring provider. If no providers add Hack, the
loss is ours (leaving site-admins with no similar-cost options). If enough
providers switch, the loss is the provider's (customer leaves or provider
follows suit). I don't think a PHP-only provider is necessarily cheaper
than a more diverse provider. They're just differently focussed.

The marketshare of sites runnable on pure "LAMP" seems to be shrinking with
the other run-times gaining popularity, and new run-times being introduced.
I'd expect a hosting service providing only Zend PHP to have a shrinking
customer base in the long run. We might not be looking at "Will they add
Hack as the first non-PHP run-time option", but rather "Will they add Hack
among the other run-times". How do we feel about that? Does this seem like
something that could happen, as well as something we could *help* make
happen?

-- Timo


On Mon, Sep 18, 2017 at 9:58 PM, Max Semenik <maxsem.wiki@gmail.com> wrote:

> Today, the HHVM developers made an announcement[1] that they have plans of
> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> instead.
>
> While this does not mean that we need to take an action immediately,
> eventually we will have to decide something. As I see it, our options are:
>
> 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> PHP. This, however, is a dead end and will make things progressively harder
> as the implementations will diverge and various Composer libraries we use
> will start requiring some Zend-specific features.
>
> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
>
> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.
>
> I personally think that 3) is the only viable option in the long run. What
> do you think?
>
> ----
> [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
>
> --
> Best regards,
> Max Semenik ([[User:MaxSem]])
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Mon, Sep 18, 2017 at 1:58 PM, Max Semenik <maxsem.wiki@gmail.com> wrote:

> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
>

Hack has a couple interesting points, especially its async system, but
isn't _hugely_ compelling at this point.

If they're going to be dropping destructors and references we'd have to
retool our RAII patterns and our hook 'out-param' patterns to work with
future-Hack, but that's possible if we *want* to make such a change.

However I don't see a lot of interest in such a change in this discussion
or among TechCom, and buying into a single-vendor system is a risk both for
us (if they change the language in ways we don't like or drop it) and
others (harder to set up if few HHVM providers).


>
> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.
>

Migrating WMF's implementation to PHP 7 is probably the way to go. I leave
it up to ops to figure out how to make the change. :)

-- brion
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Wed, Sep 20, 2017 at 10:56 PM, Brion Vibber <bvibber@wikimedia.org>
wrote:

> On Mon, Sep 18, 2017 at 1:58 PM, Max Semenik <maxsem.wiki@gmail.com>
> wrote:
>
> >
> > 3) Revert WMF to Zend and forget about HHVM. This will result in
> > performance degradation, however it will not be that dramatic: when we
> > upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
> while
> > 5.6 and 7 provided nice performance improvements.
> >
>
> Migrating WMF's implementation to PHP 7 is probably the way to go. I leave
> it up to ops to figure out how to make the change. :)
>


I think this is the more viable option too, mostly not to cause huge issues
to non-Wikimedia users. The ease of installation and operation of PHP on
most platforms compared to HHVM is incomparable. Just to make an example, I
don't remember having to check the code of the VM to understand what an ini
setting does with PHP (or even PHP-fpm), while with HHVM that has been a
recurring pain, as options come and go without any warning between
versions, not to mention the online docs that can even be plainly
misleading. At the moment, there is no doubt PHP is a much friendlier
environment for any third-party wiki.


But I think there is other value in going the PHP7 way; I had actually
pitched the idea of switching to PHP7 for some time now, for various
reasons, namely:
- We use HHVM differently than what Facebook does. For instance, we use the
fcgi server mode, and not the pure http one, and we don't run in repoAuth
mode. This has brought us new bugs to solve every time we upgrade, as there
is no battle testing in production for those code patterns at scale before
we use it.
- The recent, prolonged difficulty of interaction with the FLOSS community,
although acknowledged by the HHVM team as something they're willing to fix,
is worrying in itself.
- PHP7 has shown performance comparable to HHVM for most PHP shops that
migrated. So the single most compelling reason for which we migrated
(performance) might not be a factor anymore. Using a runtime readily
available (and security-patched) by the upstream distribution would make
the ops team lives easier as well.

As for the actual migration:

I don't think there is any need to panic or rush to a decision, but the
timeline is pretty set: by the end of 2018, when official support for HHVM
3.24 will end, any migration should be well underway within the WMF
infrastructure. I expect a migration from HHVM to PHP 7 to be a less
formidable undertaking than the switch from PHP 5.3 to HHVM - we did repay
a good deal t of the tech debt in the Wikimedia Foundation installation
back then, and we won't have to change radically the way we serve
MediaWiki, as PHP 7 works as a FastCGI server as well. Still, it will take
time and resources, and it needs to be planned in advance.

One important consequence of the announcement for Wikimedia is that we
won't be able to use any version of MediaWiki not compatible with PHP 5.x
until we transition to PHP7 (unless we decide to support both HHVM and
PHP7). This might be important in steering the timing of the change in
MediaWiki itself.

Cheers,

Giuseppe

--
Giuseppe Lavagetto, Ph.d.
Senior Technical Operations Engineer, Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 19/09/17 20:56, Gilles Dubuc wrote:
> Should we have a TechComm-driven meeting about this ASAP?

We had an IRC meeting about this yesterday. Here is the log:

<https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.2017-09-27-21.03.log.html>

We mostly talked about the migration plan for production, including
the likely timeline.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l