QORL 4B and the Fine Print Behind '81% Faster'

Key takeaways

• Generating a plan faster and executing a query faster are different improvements.
• An 81% reduction in runtime is much larger than an 81% increase in execution speed.
• A useful comparison needs a fair PostgreSQL baseline and queries unseen during training.
• Model inference and plan validation can outweigh execution savings.

An “81% faster” claim will get a database engineer’s attention. QORL 4B’s claim about query plans raises an immediate practical question: how much sooner would your application actually get its answer?

The opportunity is in the optimizer’s choices

SQL describes the data you want. PostgreSQL still has to decide how to retrieve it.

Should it scan a whole table or use an index? When joining several tables, which should it process first? Those choices belong to the query optimizer, and they can make the same SQL statement take very different amounts of time.

The optimizer compares possible plans using data statistics and cost estimates. Those estimates can miss. If values cluster unevenly, or filtering conditions are correlated, PostgreSQL may misjudge how many rows an operation will return. A plan that looks cheap on paper can become expensive in execution.

That creates an opening for AI. Patterns learned from data and previous executions might help an optimizer choose better.

But “AI optimization” leaves a lot unspecified. A model could estimate row counts, rank candidate plans, or propose a plan directly. Each approach needs different validation. Before assessing QORL 4B’s performance, we need to know which decision it is improving.

“81% faster” needs a denominator

Start with the clock being measured.

Planning time is the time spent deciding how to run a query. Execution time is the time spent running it. Reducing one does not establish an improvement in the other, and the distinction matters to anyone waiting for a response.

Then there is the percentage itself. Suppose a query takes 10 seconds:

• An 81% reduction in execution time brings it down to 1.9 seconds.
• An 81% increase in execution speed, for the same work, brings it down to roughly 5.5 seconds.
• Finding faster plans for 81% of queries tells us nothing about the size of those individual gains.

These are hypothetical calculations illustrating different meanings of the phrase, not QORL benchmark results.

The average needs scrutiny, too. Averaging each query’s relative improvement can produce a different picture from comparing total runtime across the workload. Dramatically accelerating one enormous query also has different operational value from shaving a little time off nearly every request.

The percentage is doing a lot of work here. Without the metric and calculation, it cannot tell you how much improvement to expect.

A fair benchmark includes the queries that lose

PostgreSQL’s side of the comparison deserves as much attention as the model.

Which version and settings were used? Were indexes comparable? Were statistics current? If the model gets access to a useful index while the baseline does not, the comparison cannot cleanly isolate the model’s contribution. The same problem arises if PostgreSQL is making decisions from stale statistics.

Both approaches also need the same data and hardware, with cache conditions accounted for. Running one after the other can give the second approach an advantage if the first has already loaded the relevant data into memory. Both must return the same correct results.

Next comes generalization. Strong results on queries resembling the training examples do not establish that a model will handle unfamiliar queries well. Larger tables and changing data distributions can also alter which plan is best.

Finally, look beyond the average. A workload can improve overall while an important order lookup becomes painfully slow. The slow end of the latency distribution and the worst regressions belong alongside the headline gain.

I would look first at how many queries got slower, and by how much. Those are the results an on-call engineer may end up living with.

The model’s time belongs on the bill

Choosing a better plan takes time, too.

Consider another hypothetical example. A query normally executes in 100 milliseconds. An AI-selected plan reduces execution to 70 milliseconds, but selecting and validating that plan adds 50 milliseconds. The total becomes 120 milliseconds.

The execution improvement is real. So is the longer wait.

That calculation changes for expensive analytical queries run repeatedly. If a plan can be selected in advance and reused, its selection cost can be spread across multiple executions. But reuse comes with a condition: the plan must remain useful as the data changes.

This makes expensive, recurring queries a reasonable starting point for evaluation. A replica or isolated test environment can establish correctness and compare runtimes before the approach handles live requests. A production rollout also needs a time limit on plan selection and a fallback to PostgreSQL’s default optimizer when necessary.

QORL 4B’s claim becomes useful when the metric is clear, the comparison is fair, and the gains survive the added overhead. Whether those gains matter most for a few costly queries or for consistent latency across every request depends on your workload. The number to care about is the improvement your application can deliver repeatedly.