In YDEVS we develop fairly large applications for our customers, and performance is something we must always keep in mind. Not only performance on the final product, but on the systems we use for development as well.
There’s nothing more frustrating than a sluggish development environment and that’s what we found ourselves with in one of our latest big projects.
Tabla de contenidos
Some Background
This project was build around our main stack: PHP 7.2 with the Symfony 4 framework and the API Platform components to build our data endpoints and connect a frontend written in a mix of server-rendered twig files and rich React components.
API Platform is an amazing system: it allows you to provide a fully featured . API (both REST and GraphQL!) from a simple set of annotations on your Doctrine entities. It’s very customizable and allows pluggin into any of the serialization and processing steps through all it’s built-in events. On production, all this metadata and annotation information is compiled and cached resulting in a very performant system. In development, however, it’s not!
While this is usually not a problem with most normal-sized applications, the complexity of our project meant that the Symfony bootstrap time was nearing the 500ms mark, which made working with complex views performing tens of requests very annoying. Some lists with complex information and rich widgets that would take just a few undreds of milliseconds to load in production would take over ten seconds in development.
The Solution
Our initial approach was to try to optimize how the different queries were dispatched, bundle together results and in general be more economic on the number of different requests the Symfony application had to handle. It quickly became apparent that, while this approach would not hurt the project, was not the right solution. We just needed to make the system faster.
One idea we played with was enabling full caching on development mode. While this could have worked, it meant keeping in mind you had to flush this caches every time a change could affect how the API worked. It would speed up the system in development mode, but would also slow down development: not a great tradeoff.
Finally we came across PHP-PM, the PHP Process Manager.
What’s PHP-PM?
The PHP Process Manager acts as a process manager and load balancer for PHP. It’s based on ReactPHP and works, simply put, by spawning a set of workers up to the point where Symfony’s HTTPKernel Request object would be created from the request globals. When a request comes in, it’s handled by PHP-PM and given to one of the waiting processes, that quickly produces a result, having done all the usual framework and container initialization beforehand.
The idea seemed right what we needed. The expensive loading of annotation metadata would all be done and ready for the request to come in, making development no much faster, and probably helping in production too. And so, we went for it!
Running PHP-PM
Our current setup was based on Docker and included the usual suspects: Nginx as a web frontend and PHP-FPM handling the code. The first step we needed was to install PHP-PM globally. I’ve removed all the Docker details, so this steps should be good for any other setup.
We can install PHP-PM through composer. In our case, we install it under the /tools directory.
# php composer.phar global require -d /tools php-pm/php-pm php-pm/httpkernel-adapterOnce installed, we run it to produce a configuration file
/tools/vendor/bin/ppm config --workers=$((`nproc` + 1)) --host=127.0.0.1 --port=9000 --static-directory /var/www/public -c /tools/ppm.jsonThis command creates a /tools/ppm.json file that contains the passed configuration details. We have set PHP-PM to run bound to localhost in the port 9000, handling static files in /var/www/public and spawning nproc + 1 workers, that is, one more than the number of available processors.
PHP-PM is able to handle static files by itself. However we use nginx for some of its other capabilities so we decided to keep in in the stack. We just had to change the existing PHP-FPM-ready configuration:
server {
[...]
location / {
try_files $uri @ppm;
}
location @ppm {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:9000;
}
}Finally in the server (inside a supervisord job in our case) we lauch the PHP-PM process with:
/tools/vendor/bin/ppm start --config=/tools/ppm.json --app-env=prod --debug=0 /var/www/With this we are telling PHP-PM that our application lives on /var/www, that we want to use the previously generated /tools/ppm.json configuration and that we want to run our Symfony application in prod environment with debug set tofalse.
Results
We didn’t even need to benchmark it. The difference was simply astounding: requests taking before a dozen seconds would finish in less that a quarter of a second. Everyone was happy, sanity was restored and life was good. Or was it?
Gotchas
There’s an unavoidable reality in programming: nothing works well on the first try. So it didn’t. Here’s a few of the issues we had, and I’m sure we will find more.
Long lived processes and MySQL connections
You should know by now that PHP is meant to die. That is: PHP was and is not designed for long running processes, and most frameworks and libraries expect to work around a very short timespan where everything gets built and then cleared and deleted at the end of each request.
With PHP-PM we have already a good deal of nice hacks and tweaks to work around this, like manually flushing the Symfony toolbar and debug collectors, but there was one point where we still had problems: the database connection.
MySQL by default has a timeout set after which, if the client hasn’t done anything, the server disconnects. This results in the dreaded «MySQL server has gone away» error which we very much found when we would access the production server after a few night hours of idle time. Apparently the framework will open the connection during the boostrap process, leaving it open until the request comes in.
The solution to this was pretty straight-forward: installing the DoctrineMySQLComeBack library allowed us to force reconnecting to the server whenever this happened.
An important detail to remember if you find yourself in this situation: if you are using a connection string, you must provide it schemaless or this library won’t work. That is, this:
mysql://root:root@db:3306/databasebecomes this:
//root:root@db:3306/databaseThe Process Manager and Symfony’s Debug component
One of the nicest things about developing Symfony applications are the great error pages when doing anything wrong, from throwing an exception somewhere to forgetting a semicolon or typing something wrong. This second bit, the trickies one, is handled by Symfony’s Debug component. It works by installing an error handler in the PHP interpreter: when an error bubbles up to the top of the execution stack (which means the interpreter would crash and dump the trace) it’s instead captured by Symfony and transformed into an Exception, that gets thrown in the context and finally catched on the ->handle() method of the HTTPKernel.
Here, if we are in development mode, a nice error screen is produced. Nice!
The issue here: as PHP-PM spawns all processes, keeps them around and hands them the requests. This means that our errors never bubble up to the top of the interpreter, instead reaching the PHP-PM code that, unable to tell what’s going on, understands it as an unrecoverable exception, kills the worker process and returns a cryptic «worker has disconnected» error.
After trying to bolt in an ad-hoc handling code by reimplementing Symfony’s HTTPKernel in a way that would be aware of the Debug component in itself without relying on the global error handled, we decided that the proper solution was to just live with it: syntax errors and similar issues will no longer appear as nice error messages and instead we just check them on the Docker logs. Not ideal, but a tolerable tradeoff!
Hot code reload and twig files
One of the nicest (and totally critical, to be honest) features of PHP-PM is that, in development mode, will automatically kill and restart all worker process when detecting a PHP file change. This is incredibly important as, otherwise, with every change the process manager would need to be restarted, negating any advantage in development.
The issue here is that while this feature is built in, it does not do the same for other files, like our .twig templates.
The solution was to explicitly tell PHP-PM to look for them. We do this by adding a few lines of code to our Kernel.php class:
public function boot()
{
if ($this->debug && \function_exists('PHPPM\register_file')) {
$finder = new Finder();
$baseDir = $this->getKernelParameters()['kernel.project_dir'].'/templates/';
$files = $finder
->files()
->in($baseDir)
->name('*.twig');
foreach ($files as $file) {
\call_user_func('PHPPM\register_file', $baseDir.$file->getRelativePathname());
}
}
return parent::boot();
}The internal PHPPM\register_file function tells the Process Manager to also keep an eye for changes in the passed file, which in our case solves this issue.
Conclusions
So, was it worth it? The short answer is clear: yes.
The long answer would be that, while the development performance has improved massively, and we expect to have a nice speedup on production too, we have also made our stack significantly more complex and unstable: PHP-PM is not a mature and time-tested project and, while the current release is stable, finding good documentation is still very tricky.
However, we would definitely do it again and we will take this approach the next time we encounter a project with similar performance needs. This is a tradeoff we do very consciously and we expect things to improve over time until this becomes a non-issue and an obvious choice to improve massively the performance of any PHP application.
Have you decided to integrate PHP-PM in your stack? You want to know more? You can reach me at @adlpz, leave a comment or send us an email at contact@ydevs.com.
Bienvenido a YDEVS!