Luther Monson, Staff Software Engineer at SUSE

Luther Monson

Go by way of PHP · Containers · VMs · Windows · Kubernetes

Calling Rust From PHP Without a Socket, Part 2: Making Existing Code Use It

Thirteen Composer packages let existing Redis and database code in WordPress, Laravel and Symfony use ePHPm's built-in versions. This post covers what's in them and the bugs they turned up.

8-Minute Read

The APIs PHP apps already call, mapped through ePHPm wrapper packages onto ephpm_kv_* and ephpm_db_*

Part 1 covered the plumbing. ePHPm registers ephpm_kv_* and ephpm_db_* functions that call straight into Rust. A cache read through one costs about a tenth of a microsecond instead of a loopback round trip. That’s nice and also useless on its own. Nobody is going to rewrite their WordPress object cache or their Laravel cache store around a function called ephpm_kv_set(). Applications talk to existing APIs instead:

  • Redis through phpredis or Predis
  • the cache through PSR-6/16 or wp_cache_*
  • the database through wpdb, mysqli, Eloquent or DBAL

So there’s a set of Composer packages in the ephpm org whose whole job is to sit under those APIs and route them to the native functions. This post covers what’s in them and the rule that decides when they switch on. It also covers the bugs that only showed up once real applications ran on top.

The packages

There are 13 of them, seven on the KV side and six on the database side:

Package What it stands in for
phpredis-shim a userland \Redis class (the phpredis API)
predis-connection a Predis Connection, so Predis\Client works with no socket
cache-wordpress wp-content/object-cache.php, the WP_Object_Cache drop-in
cache PSR-16 SimpleCache and PSR-6 pools
cache-laravel a Laravel cache store, auto-discovered
cache-symfony a Symfony Cache adapter
session-handler a userland SessionHandlerInterface
db the base typed wrapper, exception hierarchy and IDE stubs for ephpm_db_*
db-wordpress wp-content/db.php, a wpdb with no mysqli
mysqli-shim the mysqli API, procedural and OO
db-laravel a Laravel database driver ('driver' => 'ephpm')
db-doctrine a Doctrine DBAL 4 driver
db-codeigniter a CodeIgniter 4 database driver

They aren’t on Packagist. You add the GitHub repo as a Composer vcs repository and require it. The Predis one is the easiest to show because it’s the smallest change to an existing app:

use Ephpm\Predis\KvConnection;
use Predis\Client;

$redis = new Client('ephpm:', [
    'connections' => ['ephpm' => KvConnection::class],
]);

$redis->incrby('hits:home', 1);
$visits = $redis->get('hits:home');

Everything else in the app keeps calling Predis. KvConnection takes the command objects Predis builds. Instead of serializing them to RESP, it dispatches on the command ID and calls ephpm_kv_*.

When a shim switches on

Every package that impersonates a global API follows the same rule: alias the global name only when the real extension isn’t loaded. The namespaced class always works. Here’s the entire activation file for the phpredis shim, loaded through Composer’s autoload.files:

use Ephpm\Phpredis\Redis;
use Ephpm\Phpredis\RedisException;

if (\extension_loaded('redis')) {
    return;
}

if (!\class_exists('Redis', false)) {
    \class_alias(Redis::class, 'Redis');
}

if (!\class_exists('RedisException', false)) {
    \class_alias(RedisException::class, 'RedisException');
}

A userland class can’t replace a loaded extension’s class anyway. Anything cleverer would just fatal on a box that has phpredis. The same rule has a funny consequence for the mysqli shim. The ePHPm SDK compiles mysqli in, so under ePHPm the global mysqli is always the real extension. Only the namespaced shim API is active. The global surface is for plain PHP builds without ext-mysqli. For WordPress that doesn’t matter, because the db.php drop-in replaces $wpdb instead of replacing mysqli.

The WordPress drop-ins also have to survive being copied to a host that isn’t ePHPm. db.php checks for ephpm_db_query() first. If it’s missing, it logs a notice and returns without setting $wpdb. WordPress then builds the stock mysqli wpdb exactly as if the drop-in weren’t there. object-cache.php does the same when it can’t load its classes. A drop-in that fatals your site when you move hosts is worse than no drop-in.

Throwing beats a wrong answer

The KV store behind ephpm_kv_* models strings, counters and TTLs. It doesn’t have pipelines, MULTI, Lua, pub/sub or key iteration. Through the native functions it doesn’t have hashes, lists and sets either. The obvious way to write a compatibility shim is to fake those. Run a “pipeline” as a sequence of calls, implement SCAN by returning nothing, and so on. Every one of those fakes gives callers an answer that’s wrong in a way they can’t detect.

So the phpredis shim throws instead:

public function multi(int $mode = self::MULTI): never
{
    throw NotImplementedException::for('MULTI/transactions', 'the KV store has no transaction/pipeline state');
}

public function scan(mixed &$iterator, mixed $pattern = null, int $count = 0, ?string $type = null): never
{
    throw NotImplementedException::for('SCAN', 'the KV store exposes no key iteration');
}

NotImplementedException extends BadMethodCallException, deliberately not RedisException. Plenty of application code wraps Redis calls in catch (\RedisException $e) and treats it as “Redis is down, carry on without cache”. Calling a command the shim doesn’t implement is a bug in your setup, not a transient outage. It shouldn’t get swallowed by that catch block. The Predis connection does the same with SET ... XX, GET, KEEPTTL and friends.

The WordPress object cache had the same question with wp_cache_flush_group(). With no key scan there is no way to delete every key in a persistent group. So flush_group() clears the in-request copy and returns false for persistent groups. wp_cache_supports('flush_group') says false. Plugins that check support fall back to their own invalidation.

The semantics fixes

These are the ones I’d point at if someone asked “why can’t I just write a quick adapter”.

wp_cache_add() is a mutex. The first version of add() in cache-wordpress checked whether the key existed and then set it. WordPress code uses wp_cache_add() as a cheap mutex. Only the first caller’s add succeeds, so whoever gets true does the work. With check-then-set, two concurrent requests could both see “absent” and both win. 19226c2 routes the persistent path through ephpm_kv_setnx. That’s one insert-if-absent under the store’s per-shard lock:

$inserted = $this->ops->setnx(
    $this->build_key($key, $group),
    $this->serialize($data),
    \max(0, $ttl),
);
if (!$inserted) {
    return false;
}

(int) false is zero. ephpm_kv_incr_by() returns false when the stored value isn’t an integer. Five of the packages had return (int) \ephpm_kv_incr_by($key, $delta);, which turns that error into 0. A counter that silently resets to zero is worse than one that throws. The same one-line bug got the same fix in each of them in the September audit:

$result = \ephpm_kv_incr_by($key, $delta);
if ($result === false) {
    throw new \RuntimeException(
        "value at key '{$key}' is not an integer and cannot be incremented"
    );
}
return (int) $result;

'O\'Reilly' broke the installer. wpdb without mysqli has to escape strings itself. The first version did what mysql_real_escape_string() does and backslash-escaped the single quote. litewire’s tenant path rejected \' as malformed SQL while accepting every other backslash escape. So any content with an apostrophe failed. The first place that showed up was the twentytwentyfive theme’s block-pattern transient, which 500’d the installer during its last step. 35db6a8 doubles the quote (''), which both MySQL and SQLite accept. It keeps doubling backslashes so a string still can’t be broken out of. Doctrine’s quote() and the mysqli shim’s real_escape_string() got the same change later.

Native ints printed “0 responses”. The bridge hands back real PHP ints and floats. mysqli doesn’t. Unless MYSQLI_OPT_INT_AND_FLOAT_NATIVE is set, every scalar column comes back as a string, and wpdb never sets it. WordPress core depends on that. The comments-title block does '0' === get_comments_number(). With native ints every post with no comments rendered a “0 responses” heading. I found it by rendering the same pages under stock wpdb and under the drop-in, then diffing the HTML byte for byte. ad746e9 stringifies non-NULL scalars during row hydration. NULL stays null, like mysqli.

Stop guessing from the first keyword. Every adapter that implements a single query() entry point had to pick between ephpm_db_query() (rows) and ephpm_db_execute() (affected rows). They all did it by looking at the first word of the SQL. That breaks on WITH ... INSERT, INSERT ... RETURNING and anything else that doesn’t fit. #357 added ephpm_db_run(). It reports has_rowset from the statement that executed. Doctrine deleted its QueryClassifier class outright (e991c20). db-wordpress and the mysqli shim moved over too.

Known limitations

I found two things while writing this up. Both come from the boundary rather than the wrappers’ own logic.

Keys go through the native functions as C strings and must be valid UTF-8, as covered in part 1. In the Predis connection, a plain SET that returns false is reported as Redis’ OOM error. The reasoning is that an overwrite can only fail for memory. But a key that isn’t valid UTF-8 also gets false from the bridge. It surfaces as OOM command not allowed when used memory > maxmemory., which will send you looking in the wrong place.

The WordPress object cache stores integers raw so the native incr keeps working on them. It applies that to integer-looking strings too:

private function serialize(mixed $value): string
{
    if (\is_int($value)) {
        return (string) $value;
    }
    if (\is_string($value) && \preg_match('/^-?\d+$/', $value) === 1) {
        return $value;
    }
    return \serialize($value);
}

private function unserialize(string $value): mixed
{
    if (\preg_match('/^-?\d+$/', $value) === 1) {
        return (int) $value;
    }
    return \unserialize($value);
}

On the next request (once it’s past the in-request cache), a string "02134" comes back as the int 2134. "5" comes back as 5. For most WordPress caching nobody notices. A ZIP code or a zero-padded ID cached as a string does notice. Distinguishing the two needs a type tag in the stored value, which is a format change. For now it’s a documented quirk rather than a quick patch.

The Laravel lock acquires atomically with setnx. But it releases by reading the owner and then deleting. That’s the same non-atomic release Laravel’s own Memcached lock has. It was written the day before the store grew a compare-and-delete, so it says “no CAS in the SAPI” in its docblock. That one’s a story of its own.

Trying it

The wordpress-sample repo puts most of this together:

  • WordPress on the embedded database through db-wordpress
  • the object cache on the KV store through cache-wordpress
  • a WebSocket broadcast from PHP through ephpm_ws_broadcast

It runs WooCommerce and Elementor, which between them exercise more MySQL-isms than WordPress core does.

The theme across all thirteen packages is the same as part 1. The boundary is the easy part. The work goes into matching what callers already depend on: string typing, atomic add(), errors that are errors.

Recent Posts

Categories

About

Luther is a 20+ year software veteran specializing in backend APIs and architecting products. Employed at SUSE as the Team Lead for the Containers on Windows Team working on Rancher + Kubernetes