Benchmarks: DotNet vs PHP vs Go vs NodeJS

  • DotNet Core with Bogus for generating data
  • PHP “Micro”: this is my own framework made of ThePHPLeague Router, PHPDI container, Faker for generating data.
  • PHP Symfony 5: the latest and the greatest
  • NodeJS Fastify: I was told this is the top performer in the ecosystem, with Faker for generating data
  • Go Echo: with its own Faker library
  • 3 tests: one with the APIs running on my local machine with full access to resources (i7, 32 Gb RAM), one in my local machine but in a Docker environment (4 cores allocated, 4 Gb of RAM), one in a Kubernetes cluster where each API has 2 running pods behind Nginx Ingress running in Azure in compute-optimized instances.
  • where applicable, logging was disabled
  • minimal setup for PHP (I used the official Apache Docker container for one set of tests, as well as another run using the RoadRunner platform which is written in Go).
  • otherwise, as out of the box as possible. Endpoints served by closures where possible.
  • using Autocannon (autocannon -c 100 -d 40 -p 10 -t 120) from my local machine (1Gbps available broadband)
  • Basically, it’s a 40 seconds test with concurrency of 100 and 40 requests in pipeline.
  1. Directly on local machine: https://imgur.com/Yzhv6E2
  2. Local machine in Docker: https://imgur.com/72N9y72
  3. In K8s cluster: https://imgur.com/zRFGRO5
  • I know the test isn’t really fair for interpreted languages (like PHP). I will add a more “real” scenario, perhaps fetching something from a DB
  • In my local environment I wanted to also use Python using Stilette framework with GUnicorn. I was shocked to see how poorly it did on the dynamic endpoint with data generation, perhaps I did something wrong as I’m not much of a Python expert.
  • For the first test, I didn’t setup Apache with PHP locally, I only performed tests with RoadRunner for both PHP apps (Symfony and ‘frameworkless’).
  • The results aren’t any average, I took an actual result set that was slightly above-average. In each case the tests were ran at least 10 times each for each of the endpoints.
  • The Docker on my local machine test used the exact same containers as those that ran in K8s. For each build the code/binary was copied to the container.
  • I’m puzzled by Fastify. I haven no explanation why when using Docker with my local machine it only managed 104 (always ~100) requests on the dynamic endpoint. In all other tests it did very much close to my expectations
  • On the PHP side, the ‘regular’ setup (aka, with Apache) saw the ‘frameworkless’ API usually under Symfony, though a few months ago I did a similar test with Symfony 4 and I have to say that Symfony 5 is way better
  • Roadrunner is a MAJOR boost for PHP. It’s the power of Go and the popularity of PHP all-in-one. It feels like PHP can compete in some cases!
  • Roadrunner also levelled the playing field between the ‘frameworkless’ setup and Symfony 5. Honestly, in this case, I can’t see why I’d choose Symfony and all the yml nonsense over defining my dependencies in plain PHP with PHPDI!
  • Go’s performance just explodes when it has more system threads to abuse!
  • I don’t know what happened to DotNet when used in Docker container (ran under the official AspNetCore image) but it was insane on the static endpoint! I ran that test maybe 20 times and I can say that 700k requests in container over 40s wasn’t the top result (it also achieve ~850k one time). It felt strange that it outperformed Go.
  • Fastify also outperformed Go in the same scenario. I didn’t expect that, since in the Docker environment it still had 2 vCPUs available and the GOMAXPROCS was set to 50 for good measure (I tried with settings between 2 and 50, such as 5, 10, 20, 25 and 35 after landing on 50, with little practical difference)
  • Node and DotNet seem to fare better with a single but better container environment, whereas Go made better use of the limited resources in K8s with some horizontal scaling in the mix (each pod ran a single container with 250m vCPU and up to 512Mb RAM)
  • With RoadRunner, the PHP ‘frameworkless’ (micro) also resulted in some errors on the RoadRunner side — so I made a bug report for RoadRunner.
  • I do find the test relevant, despite it being not fair to interpreted languages (life is not fair, the tool needs to do the job). Generating random data tests the computational capabilities of the platform. It’s a heavy task.
  • I looked into platforms in a way that should be similar to how developers would approach a task. Sure, you may want to build things from scratch but generally you’d pick a library / router or a framework depending on how much manual work you think it’s worth doing.
  • In Go world: I looked into lightweight things as well. The wonder library fasthttp + phi router is a great performer (in local test it did constant > 4 million requests for static endpoint with average of 1ms latency, but no difference for dynamic endpoint). But you need json encoder and logging to match (please, no logrus or stdlib json marshalling!)
  • Today I wouldn’t use PHP without Roadrunner. IMHO, PHP has too many moving parts and tools to configure in order to a) be production and b) provide a high performance production environment. Apache/fpm, phpunit, xdebug, etc — a wide array of third party tools *required* to deploy/develop something. A platform should provide them!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store