In YDEVS we develop fairly large applications for our customers, and performance
is something we must always keep in mind. Not only performance on the final
product, but on the systems we use for development as well.

There’s nothing more frustrating than a sluggish development environment and
that’s what we found ourselves with in one of our latest big projects.

Some Background

This project was build around our main stack: PHP 7.2 with the Symfony 4
framework and the API Platform components to build our data endpoints and
connect a frontend written in a mix of server-rendered twig files and rich
React components.

API Platform is an amazing system: it allows you to provide a fully featured
API (both REST and GraphQL!) from a simple set of annotations on your
Doctrine entities. It’s very customizable and allows pluggin into any of the
serialization and processing steps through all it’s built-in events. On
production, all this metadata and annotation information is compiled and cached
resulting in a very performant system. In development, however, it’s not!

While this is usually not a problem with most normal-sized applications, the
complexity of our project meant that the Symfony bootstrap time was nearing the
500ms mark, which made working with complex views performing tens of requests
very annoying. Some lists with complex information and rich widgets that would
take just a few undreds of milliseconds to load in production would take over
ten seconds in development.

The Solution

Our initial approach was to try to optimize how the different queries were
dispatched, bundle together results and in general be more economic on the
number of different requests the Symfony application had to handle. It quickly
became apparent that, while this approach would not hurt the project, was not
the right solution. We just needed to make the system faster.

One idea we played with was enabling full caching on development mode. While
this could have worked, it meant keeping in mind you had to flush this caches
every time a change could affect how the API worked. It would speed up the
system in development mode, but would also slow down development: not a great
tradeoff.

Finally we came across PHP-PM, the PHP
Process Manager.

What’s PHP-PM?

The PHP Process Manager acts as a process manager and load balancer for PHP.
It’s based on ReactPHP and works, simply put, by spawning a set of workers up to
the point where Symfony’s HTTPKernel Request object would be created from
the request globals. When a request comes in, it’s handled by PHP-PM and given
to one of the waiting processes, that quickly produces a result, having done all
the usual framework and container initialization beforehand.

The idea seemed right what we needed. The expensive loading of annotation
metadata would all be done and ready for the request to come in, making
development no much faster, and probably helping in production too. And so, we
went for it!

Running PHP-PM

Our current setup was based on Docker and included the usual suspects: Nginx
as a web frontend and PHP-FPM handling the code. The first step we needed
was to install PHP-PM globally. I’ve removed all the Docker details, so this
steps should be good for any other setup.

We can install PHP-PM through composer. In our case, we install it under the
/tools directory.

# php composer.phar global require -d /tools php-pm/php-pm php-pm/httpkernel-adapter

Once installed, we run it to produce a configuration file

/tools/vendor/bin/ppm config --workers=$((`nproc` + 1)) --host=127.0.0.1 --port=9000 --static-directory /var/www/public -c /tools/ppm.json

This command creates a /tools/ppm.json file that contains the passed
configuration details. We have set PHP-PM to run bound to localhost in the port
9000, handling static files in /var/www/public and spawning nproc + 1
workers, that is, one more than the number of available processors.

PHP-PM is able to handle static files by itself. However we use nginx for
some of its other capabilities so we decided to keep in in the stack. We just
had to change the existing PHP-FPM-ready configuration:

server {
    [...]

    location / {
        try_files $uri @ppm;
    }

    location @ppm {
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://127.0.0.1:9000;
    }
}

Finally in the server (inside a supervisord job in our case) we lauch the PHP-PM process with:

/tools/vendor/bin/ppm start --config=/tools/ppm.json --app-env=prod --debug=0 /var/www/

With this we are telling PHP-PM that our application lives on /var/www, that
we want to use the previously generated /tools/ppm.json configuration and that
we want to run our Symfony application in prod environment with debug set to
false.

Results

We didn’t even need to benchmark it. The difference was simply astounding:
requests taking before a dozen seconds would finish in less that a quarter of a
second. Everyone was happy, sanity was restored and life was good. Or was it?

Gotchas

There’s an unavoidable reality in programming: nothing works well on the first
try
. So it didn’t. Here’s a few of the issues we had, and I’m sure we will find
more.

Long lived processes and MySQL connections

You should know by now that PHP is meant to die.
That is: PHP was and is not designed for long running processes, and most
frameworks and libraries expect to work around a very short timespan where
everything gets built and then cleared and deleted at the end of each request.

With PHP-PM we have already a good deal of nice hacks and tweaks to work around
this, like manually flushing the Symfony toolbar and debug collectors, but there
was one point where we still had problems: the database connection.

MySQL by default has a timeout set after which, if the client hasn’t done
anything, the server disconnects. This results in the dreaded “MySQL server
has gone away”
error which we very much found when we would access the
production server after a few night hours of idle time. Apparently the framework
will open the connection during the boostrap process, leaving it open until the
request comes in.

The solution to this was pretty straight-forward: installing the
DoctrineMySQLComeBack
library allowed us to force reconnecting to the server whenever this happened.

An important detail to remember if you find yourself in this situation: if you
are using a connection string, you must provide it schemaless or this
library won’t work. That is, this:

mysql://root:root@db:3306/database

becomes this:

//root:root@db:3306/database

The Process Manager and Symfony’s Debug component

One of the nicest things about developing Symfony applications are the great
error pages when doing anything wrong, from throwing an exception somewhere to
forgetting a semicolon or typing something wrong. This second bit, the trickies
one, is handled by Symfony’s Debug component. It works by installing an error
handler
in the PHP interpreter: when an error bubbles up to the top of the
execution stack (which means the interpreter would crash and dump the trace)
it’s instead captured by Symfony and transformed into an Exception, that gets
thrown in the context and finally catched on the ->handle() method of the
HTTPKernel.

Here, if we are in development mode, a nice error screen is produced. Nice!

The issue here: as PHP-PM spawns all processes, keeps them around and hands them
the requests. This means that our errors never bubble up to the top of the
interpreter, instead reaching the PHP-PM code that, unable to tell what’s going
on, understands it as an unrecoverable exception, kills the worker process and
returns a cryptic “worker has disconnected” error.

After trying to bolt in an ad-hoc handling code by reimplementing Symfony’s
HTTPKernel in a way that would be aware of the Debug component in itself
without relying on the global error handled, we decided that the proper
solution was to just live with it: syntax errors and similar issues will no
longer appear as nice error messages and instead we just check them on the
Docker logs. Not ideal, but a tolerable tradeoff!

Hot code reload and twig files

One of the nicest (and totally critical, to be honest) features of PHP-PM is
that, in development mode, will automatically kill and restart all worker
process when detecting a PHP file change. This is incredibly important as,
otherwise, with every change the process manager would need to be restarted,
negating any advantage in development.

The issue here is that while this feature is built in, it does not do the same
for other files, like our .twig templates.

The solution was to explicitly tell PHP-PM to look for them. We do this by
adding a few lines of code to our Kernel.php class:

public function boot()
{
    if ($this->debug && \function_exists('PHPPM\register_file')) {
        $finder = new Finder();
        $baseDir = $this->getKernelParameters()['kernel.project_dir'].'/templates/';
        $files = $finder
            ->files()
            ->in($baseDir)
            ->name('*.twig');
        foreach ($files as $file) {
            \call_user_func('PHPPM\register_file', $baseDir.$file->getRelativePathname());
        }
    }

    return parent::boot();
}

The internal PHPPM\register_file function tells the Process Manager to also
keep an eye for changes in the passed file, which in our case solves this issue.

Conclusions

So, was it worth it? The short answer is clear: yes.

The long answer would
be that, while the development performance has improved massively, and we expect
to have a nice speedup on production too, we have also made our stack
significantly more complex and unstable: PHP-PM is not a mature and time-tested
projet and, while the current release is stable, finding good documentation is
still very tricky.

However, we would definitely do it again and we will take this approach the next
time we encounter a project with similar performance needs. This is a tradeoff
we do very consciously and we expect things to improve over time until this
becomes a non-issue and an obvious choice to improve massively the performance
of any PHP application.

Have you decided to integrate PHP-PM in your stack? You want to know more? You can reach me at @adlpz, leave a comment or send us an email at contact@ydevs.com.

Este sitio web utiliza cookies para que usted tenga la mejor experiencia de usuario. Si continúa navegando está dando su consentimiento para la aceptación de las mencionadas cookies y la aceptación de nuestra política de cookies, pinche el enlace para mayor información.plugin cookies

ACEPTAR
Aviso de cookies