We have a security update for LiteSpeed’s QUIC and HTTP/3 Library (LSQUIC), and all three LiteSpeed server products. Recently we were made aware of a vulnerability affecting LSQUIC. We patched this vulnerability in v4.3.1.
Please update to the latest versions of LSQUIC and all LiteSpeed server products.
UDP packets, crafted in a particular way and sent to the HTTP/QUIC service port, can cause an unbounded memory leak. This has the potential to cause the process or the server to run out of memory, eventually leading to a Denial of Service.
This vulnerability in the LSQUIC Library affects all server products and may be easily exploited.
Actions
We strongly recommend that those using the QUIC and HTTP/3 library upgrade to LSQUIC version 4.3.1 or higher to patch this vulnerability.
Additionally, those who are using LiteSpeed server products, should upgrade to the following versions of these products immediately:
LiteSpeed Web Server (LSWS) v6.3.4 or higher
LiteSpeed Web ADC (LSADC) v3.3.1 or higher
OpenLiteSpeed (OLS) v1.8.4 or higher
If you cannot upgrade your server at this time, you can disable HTTP/3 to avoid this vulnerability.
Timeline
July 15, 2025: We were alerted to the issue.
July 18, 2025: Patch was added to our internal repo to be included in all subsequent builds of our commercial server products
August 1, 2025: Released LSWS v6.3.4 and OLS v1.8.4
August 4, 2025: Released LSADC v3.3.1
August 13, 2025: Released LSQUIC v4.3.1 to the GitHub repository
Conclusion
We thank Imperva Offensive Team for bringing this issue to our attention. This vulnerability has been patched, so if you are keeping your LSQUIC library or your LiteSpeed server products up-to-date, there is nothing you need to do. If you have not updated in a while, please do so today.
We have a security update for LiteSpeed’s QUIC and HTTP/3 Library (LSQUIC). Recently we were made aware of a vulnerability affecting several QUIC implementations, including LSQUIC. We patched this vulnerability in v4.2.0.
Please update to the latest version of LSQUIC.
This Hash-based Denial-of-Service vulnerability, reported by Paul Bottinelli, Cryptography and Security Consultant at NCC Group, has been assigned CVE-2025-24947.
Impact
Several QUIC implementations, including LSQUIC, use a hash table to store Secure Connection IDs (or, SCIDs) as indices to connection data. If the hash function is weak, an attacker can trigger a denial-of-service attack by initiating connections with colliding SCIDs, causing significant slow downs.
LSQUIC, while technically vulnerable to this type of attack, has a rate limiting feature. If there are too many INIT packets for new connections, a Retry packet will be triggered. This means, LSQUIC doesn’t put the client-generated SCID in the hash. As such, in a real-world deployment, this vulnerability is not easy to exploit with LSQUIC.
Previously, we used XXH32(), which is prone to hash collisions, to generate a 32-bit hash key. With v4.2.0, LSQUIC has switched to rapidhash, which allows us to generate a stronger random seed, and produces a 64-bit hash key.
Our test shows that it addresses the hash collision issue well. Here are some examples:
without seed
211C6C858BB29CDD408F1EBDAA43A980CE016B71 -> F3C569EBDE612455
211CEC85DBB29CDE418F1EBDAA43A980CF214B71 -> F3C569EBDE612455
with seed
211C6C858BB29CDD408F1EBDAA43A980CE016B71 -> 3173D06C62EB64E2
211CEC85DBB29CDE418F1EBDAA43A980CF214B71 -> EF06763606005C05
without seed
211C6C858BB29CDD408F1EBDAA43A980CE016B71 -> 56B71185739E534A
211CEC85DBB29CDE418F1EBDAA43A980CF214B71 -> 56B71185739E534A
with seed
211C6C858BB29CDD408F1EBDAA43A980CE016B71 -> D18323CAAFD14A5A
211CEC85DBB29CDE418F1EBDAA43A980CF214B71 -> 190C4F2674528BE1
without seed
211C6C858BB29CDD408F1EBDAA43A980CE016B71 -> 9183B84F3AE05847
211CEC85DBB29CDE418F1EBDAA43A980CF214B71 -> 9183B84F3AE05847
with seed
211C6C858BB29CDD408F1EBDAA43A980CE016B71 -> D05A92D1069111F3
211CEC85DBB29CDE418F1EBDAA43A980CF214B71 -> 3150BF22FFAA28E4
Actions
We recommend that those using the QUIC and HTTP/3 library upgrade to LSQUIC version 4.2.0 or higher to patch this vulnerability.
Those who are using LiteSpeed server products, including LiteSpeed Web Server (LSWS), LiteSpeed Web ADC (LSADC), and OpenLiteSpeed (OLS), should upgrade to the latest versions of these products.
Timeline
January 10, 2025: We were alerted to the issue.
January 15, 2025: Patch was added to our internal repo to be included in all subsequent builds of our commercial server products
February 18, 2025: Released LSWS v 6.3.2, LSADC v3.3.0, and OLS v1.8.3
February 18, 2025: Released LSQUIC v4.2.0 to the GitHub repository
Conclusion
We thank Paul Bottinelli for bringing this issue to our attention. This vulnerability has been patched, so if you are keeping your LSQUIC library or your LiteSpeed server products up-to-date, there is nothing you need to do. If you have not updated in a while, please do so today.
LiteSpeed Web Server and LiteSpeed Web ADC both got significant updates at the end of 2023. Let’s look under the hood to see how these new updates can make your LiteSpeed experience even better!
RE2 in ModSecurity
LiteSpeed’s sleek and performant ModSecurity engine now supports the RE2 regular expression library. You can stick with the PCRE library that Apache, nginx, and others currently use, or you can switch to RE2, which comes with a resource-management advantage.
PCRE commonly faces infinite-looping and thread-blocking issues when processing large input with poorly written patterns. This is a fatal problem for a high-performance, event-driven server. LiteSpeed minimizes the impact of thread blocking by using a thread pool that is separate from the main loop. But despite these measures, as PCRE’s resource usage grows exponentially, it can still cause a server to slow down.
Computational resource consumption by RE2, on the other hand, grows only linearly and is more predictable.
RE2 offers a slightly smaller number of features than PCRE does. However, RE2 is faster than PCRE when using certain operators like Boolean OR (|).
Bottom line: if PCRE is working for you, there is no need to switch. But if you are seeing high CPU utilization, and you think ModSecurity could be to blame, try using RE2, and see if that helps. Navigate to WebAdmin Console > Server > Security > Web Application Firewall (WAF) and select Use RE2 regex engine.
QUIC v2
In March of 2023, we updated our LSQUIC HTTP/3 Library to use QUIC v2. Since then, we’ve taken our time integrating QUIC v2 into our server products. This has allowed for maximum testing time and confidence before rolling it out.
As of LSWS v6.2 and LSADC v3.2, QUIC v2 is in production.
Request Header Validation
Request header validation is now more strict. As of these new versions, our validator checks for NULL bytes in the header value, and Transfer-Encoding value manipulation. If either of these is detected, a 400 bad request is returned.
LSWS Apache Compatibility
We have three compatibility improvements to share.
First up, Litespeed now supports the new flags recently added to Apache’s rewrite engine:
BNP
backrefnoplus
BCTLS
BNE
Second, we now support RewriteCond expr '<expression>'.
And third, we squashed a bug that was preventing us from supporting dynamic regular expression matching. The issue was with variables in the pattern, as seen in SSI code like this:
<!--#if expr="$referrer = /$host/" –>
LiteSpeed could support fixed regular expression patterns in Apache expressions, but failed when variables were used. This issue is now solved, and LiteSpeed Web Server v6.2 fully supports dynamic regular expression matching.
Web ADC HEAD Caching
LiteSpeed Web ADC now supports HEAD caching. Separate cache entries will be created for HEAD requests, to avoid unnecessary hits to the back end, and to improve the cache hit rate.
Conclusion
Have you updated LiteSpeed Web Server or LiteSpeed Web ADC yet? The new versions are stable and ready to serve!
If you’re new to LiteSpeed and want to take one of these products out for a spin, sign up for a 15-day trial, today:
After about 2 years of strong stability and performance in production, LiteSpeed’s QUIC and HTTP/3 library gets its first major feature enhancement release: lsquic v4.0. Notable enhancements include:
Support for QUIC version 2
Version negotiation
Packet retry for address validation
Handshake improvements under high packet loss
Greatly improved QUIC interop results.
Below is the detailed information about the improvements.
QUIC version 2
QUIC version 2 is in draft 10 as of this writing, and will be published as RFC9369 soon. QUIC version 2 is not intended to deprecate version 1. Instead, it is meant to mitigate ossification concerns and exercise the version negotiation mechanisms.
There are a few QUIC implementations that already support QUIC v2 including ngtcp2, Kwik, picoquic, and HAProxy. lsquic might have been a little late to the party, but we are here now with v4.
We also plan to release QUIC v2 into the production environment with our commercial server products very soon.
Version Negotiation
Version Negotiation is pretty much in the same boat as QUIC v2. It is in draft 14, and soon will be published as RFC9368. It updates RFC8999 by defining version negotiation mechanisms that leverage the Version Negotiation packet.
Quote from draft 14:
It is beneficial to avoid additional round trips whenever possible, especially given that most incremental versions are broadly similar to the previous version. This specification also defines a simple version negotiation mechanism which leverages similarities between versions and can negotiate between “compatible” versions without additional round trips.
Version negotiation is a required feature in order to support QUIC v2.
Retry Packet for Address Validation
This feature is defined in RFC9000 section-8.1.2. It is not something new. It has been used in our commercial WebADC product from the very beginning, however this feature was not previously available in the open source lsquic code base. The related code is now published in v4.
Handshake under high packet loss or corruption
In QUIC interop test cases, handshakeloss and handshakecorruption have been two of the toughest tests to pass in our course of improving lsquic interoperability.
The network quality simulated in the test cases is so poor, it is definitely not something a user will come across everyday. However, to make successful handshakes under such conditions, makes a QUIC implementation super resilient to bad network conditions. In the process of testing, many beneficial tweaks have been made, like minimizing server hello return packets, and ACK/PING fine tuning.
You can see how much lsquic has been improved in the next section.
QUIC interop improvements
lsquic was in the first batch of open source QUIC implementations. It has been running in production environments for many years, and is both stable and performant. It has not only been used in LiteSpeed products, but also by other third-parties in large scale deployments.
However, lsquic’s interoperability results have not always been the best. This does not mean that lsquic is not as good as the implementations that have better results. It is just that we do not focus on the interop tests. Some tests were not turned on, and some needed minor tweaks.
Since we are making big improvements to lsquic, it is a good time to take this opportunity to improve interoperability with other implementations. Here is the comparison of before and after.
Green is passed, gray is not supported by the client, and red is failure. As you can see, there are only a few red ones left. Those failures mostly are due to client side problems. lsquic now joins ngtcp2 and picoquic in the “passes all interop tests” club.
The lscquic v4.0 release is a big leap forward in terms of the latest development of the QUIC protocol. The LiteSpeed team is committed to bringing the best QUIC implementation to the public. Stay tuned for what is coming next.
The year 2020 was the fourth year that the LiteSpeed QUIC and HTTP/3 library, lsquic, has been available on GitHub under a liberal open-source license. The following list of numbers offers a snapshot review of 2020:
By the Numbers
10,400
Approximate number of lines of code by which the library grew (library proper in src/liblsquic)
90
New issues, of which
71
Have been resolved
65
Total number of lsquic releases
51
Minor releases
20
New pull requests
14
Major releases
8
New QUIC versions supported: IETF QUIC Internet-Draft versions 25 through 32
7
Old QUIC versions no longer supported: Q039 (that’s Google QUIC), IETF QUIC Internet-Draft versions 23, 24, 25, 26, 30, 31
It finally looks like QUIC and HTTP/3 will become bona fide IETF standards this year! Our outlook for both our lsquic library and the Internet standards is therefore bullish.
Goal: Determine whether any performance gain can be achieved with XDP UDP versus kernel UDP
HTTP/3 uses QUIC which in turn uses UDP (User Datagram Protocol) rather than TCP/IP for communications. XDP (eXpress Data Path) has been added recently to several operating systems to provide User Space applications performance comparable to direct kernel applications.
The goal of this blog is to determine if QUIC can gain significant enough performance to justify implementing XDP where it is available. Understanding performance gains usually requires detail so besides a simple test, a profiler will be used to see exactly where performance gains occur.
XDP can be run with hardware support or can be run entirely in software. The goal is to have an XDP program be usable in the widest possible range of computing environments. This is best accomplished by targeting virtual machines.
The Test Environment
LSQUIC
The QUIC engine used was the popular open-source LSQUIC library available from LiteSpeed Technologies. Only small modifications were made to the base library to support XDP. The XDP functionality was provided by a separate module, which implemented LSQUIC callbacks to send, receive, and allocate packets. Testing was performed using the http_server and http_client programs provided by LSQUIC, modified as necessary.
The client machine ran a released version of the http_client program from the LSQUIC library and was always the receiver. It was specified to use the Q046 QUIC protocol version.
The server machine ran the modified version of the http_server program from the LSQUIC library and was run with command line parameters to use either the native UDP or the XDP UDP. It ran in Interop mode, which is a mode of the program where data is transmitted without having to be read from disk.
Simple volume tests of 2GB were executed to show performance differences between native UDP and XDP UDP. A profiler was used to examine the effect of XDP UDP versus native UDP in the program to help identify specific bottlenecks.
Hardware and Software
Server machine:
OS: Lubuntu v19.04
CPU: Xeon E7 2.4 Ghz
Host: Proxmox 6.1
Host NIC: Intel X540-AT2 10Gb dual port NIC.
VM type: KVM
VM Memory: 8 GB
VM NIC: virtio_net driver (it has specific performance improvements for XDP built in)
Client machine:
OS: Lubuntu v19.04
CPU: Xeon E5 a 3.6 Ghz
VM type: None.
Memory: 8 GB
NIC: Intel X540-AT2 10Gb dual port NIC.
NOTE: A non-VM machine was chosen with a high speed data path so as to best simulate access to the wider internet in a controlled environment. Other works showing performance include this one, which shows runs simply tossing away or moving data without any real involvement. In contrast, we believe our comparison is the first where a real application could be used.
The Tests and Results
3 runs in a row were performed by the client machine pointing to the server machine which was configured to run in native UDP mode. Results reported were:
Test 1: 111,036,768 bytes/sec
Test 2: 98,336,240 bytes/sec
Test 3: 96,860,121 bytes/sec
Resulting in an average of 102,077,709 bytes per second.
The server was restarted to run in XDP UDP mode and the client tests were repeated:
Test 1: 152,644,777 bytes/sec
Test 2: 147,426,653 bytes/sec
Test 3: 138,277,563 bytes/sec
Resulting in an average of 146,116,331 bytes per second.
XDP Gets the Win
This is a 43% performance improvement by changing from native UDP sends to XDP UDP sends.
By examining the top program on all runs we could see that the server program was running at 100% or nearly so. On the client machine the CPU differences were significant. When running in XDP UDP mode the CPU utilization was close to or exceeding 70%. When running in native UDP mode the CPU utilization was less than 60%. This implies that the limit was in the CPU on the server system, which is where the UDP transmissions were occurring.
Flame Graph from perf Profiling
Since there’s such a significant difference in performance a profiler was used to see exactly where the program was spending its time. In particular the profiler perf was used as it provides great detail.
Flame Graph for native UDP
Processing the perf output with FlameGraph produces:
In a flamegraph the widest (horizontal) bars take the most CPU. As you go up the graph, you go deeper into the call stack. At the bottom is http_server. As you go up, you go into the program. The bar stays very wide until send_packets_out, an LSQUIC function that has two expensive children, gquic_encrypt_packet and send_batch, the latter being the more expensive of the two. Following send_batch it continues staying very wide into __libc_sendmsg where it leaves the program entirely and is routed into the kernel. From this we can determine that the kernel overhead is by far the most CPU intensive cost.
Flame Graph for XDP
Running the same perf run and FlameGraph with XDP results in:
This flamegraph has a much faster separation into narrower functions indicating that there are more contributors to CPU cost, in other words, not a single bottleneck. As above the widest row stays wide going into the program until send_packets out where it breaks into gquic_encrypt_packet and send_batch, where send_batch is the slightly wider function. The entry point into the kernel is __libc_sendto where it is barely the widest row at that depth indicating that CPU overhead of encryption is nearly as expensive as transmission.
This gives us an indication that the kernel overhead is substantially reduced in XDP and explains the performance improvements.
Conclusions
The goal of this blog post was to determine where the bottlenecks in QUIC and HTTP/3 are and to see if the XDP UDP interface could help in improving the overall performance. The results of the tests show quite convincingly that XDP reduces the kernel penalty of native UDP.
While XDP UDP is not a perfect solution for many environments as it is Linux specific it appears that for Linux web servers implementing XDP will result in higher overall performance.
The QUIC Interop event precedes the QUIC Interim. The Interop is meant to find bugs in different implementations — and in the specifications themselves! This year, these two events take place in Zurich, Switzerland, where Google is graciously hosting us.
Enter the Dragon
Kasernenstrasse 97
On Monday, February 3rd, I arrived at Kasernenstrasse 97 at 9 o’clock in the morning. The Zurich Google office is located in Sihlpost — a massive building on the left bank of the river Sihl. Being so massive, the building possesses two entrances, about 200 feet apart, and two addresses: Kasernenstrasse 97 and Kasernenstrasse 95. The former is the regular Google employee entrance; the latter is where the Google reception is located.
Visitors like me are to use the reception entrance where they are given a temporary guest badge. This I learned later. On that morning, having ignored the sign imploring the visitors to use the other entrance (“95? That must be on the next block!”), I tailgated my way to the fourth floor. There, having run into yet another badged entrance, I waited. Thankfully, a few seconds later, a Google employee emerged. I told her in my best German that I am here for the QUIC conference. “Oh! You should have used the other entrance. That’s OK, I will lead you to the reception.” Like the magical Sesame, gates fell open before us and, in as few as 30 seconds, I was at the reception. Thank you, my kind nameless guide!
Boys are Back in Town
I was one of the first people in the room. There, Brian Trammell, our host, greeted me and I met Stanislav Slusny (Akamai). Later, many familiar faces filled the room: Eric Kinnear (Apple) and his crew; Lars Eggert (NetApp); Lucas Pardue, the new QUIC Working Group co-chair, and Alessandro Ghedini (Cloudflare); Martin Duke (F5); David Schinazi and Ted Hardie (Google); Jana Iyengar (Fastly); Martin Thomson and Eric Rescorla (aka EKR) of Mozilla; and others. It was like the good old times again! We picked up where we left off: each running his QUIC client against the others’ servers.
The “Handshake Done” Bug
Christian Huitema, who attended the Interop remotely, and I had been debugging performance issues in lsquic/picoquic (Christian’s implementation) interactions for a few days. On Monday, I fetched the latest version of picoquic and saw the performance much improved. On the other hand, I also observed that some connections would fail during the handshake. After some digging and arguments, I had to admit that I was in the wrong and that lsquic server sent the “handshake done” confirmation too soon when TLS session was being resumed. I have a tentative bug fix that will be included in the next release of lsquic.
Later it turned out that F5 and Cloudflare QUIC implementations have also been afflicted with the same bug, which made me feel slightly less silly. This is the great value of interops: we are bound to find bugs in implementations of newly added protocol features (of which the HANDSHAKE_DONE frame is one).
Bug 2: Garbage Padding
Another new feature of the QUIC protocol is allowing UDP frames carrying Initial packets to be padded with garbage. An implementation is supposed to discard this padding. This was not happening in lsquic, which is what Andy Grover (Mozilla) discovered using the neqo client. I was able to fix this lsquic issue quickly and Andy successfully tested that it works. Thanks for a nice corner case, Andy!
GSO vs XDP
What’s better: GSO or XDP? This was one of the questions that Igor Lubashev (Akamai), Matt Joras (Facebook), Stanislav Slusny, and I discussed. (That is to say, I mostly listened.) This was one of those spontaneous moments that one can experience only when attending a QUIC meeting in person, not remotely. I can’t wait to apply these shared bits of wisdom in lsquic.
Delayed ACKs
Jana Iyengar and Ian Swett (Google), erstwhile coworkers and two of the principal movers of QUIC, teamed up to propose a protocol extension to control acknowledgement delay and other ACK generation parameters dynamically. This has the potential to reduce CPU overhead significantly by sending and processing fewer ACK packets. Getting this right is tricky, however. lsquic, picoquic, and quiche have experimental support for this feature and we played with it, looking for performance gains (or losses). lsquic’s mechanism for ACK control using this extension is quite bare-bones and improves CPU utilization in some cases, while in other cases it may cause significant throughput degradation. More work is required here.
Loss Bits
The loss bits extension (by Igor Lubashev, Alexandre Ferrieux (Orange), and your humble author) is meant to help operators troubleshoot network problems. In particular, inclusion of the loss bits into QUIC packets allows one to determine whether packet loss occurs upstream or downstream of a probe. To date, this extension has proved to be very controversial. The spin bit debates were epic — and they were over just a single bit. Now, we want to add two more bits — Q and L bits — to the unencrypted part of the QUIC packet header!
At LiteSpeed, we believe in being good citizens. This includes allowing network operators to do their job. From the practical side, availability of network tools should improve the chances that QUIC gets adopted. For these reasons, we had supported the spin bit when it was proposed. We now also support the loss bits extension.
Besides lsquic, one other implementation supports the loss bits extension — Christian Huitema’s picoquic. We had already achieved interoperability prior to this week’s event. On Tuesday, Igor and I ran several tests against picoquic and lsquic servers while introducing packet loss. These tests are ongoing.
And the Winner Is…
Just kidding — there is no winner! The Interop is a collaboration between many organizations, large and small, and individual developers. We proved once again that the protocol works. We found bugs. lsquic benefitted from this exercise and, in turn, helped others uncover bugs in their implementations.
The Interim meeting is tomorrow. This is the year that we ship the final spec. Let’s get it done!
On Wednesday, November 27th, I was a guest on the FLOSS Weekly show on TWiT to discuss LSQUIC, our QUIC and HTTP/3 library.
Background
I first learned about the FLOSS Weekly podcast a year or two ago. The idea to appear as a guest on the show did not occur to me until much later. Immediately, it made sense: our QUIC and HTTP/3 library is a full-featured, well-tested, and robust implementation of an increasingly important Internet protocol. And — it’s open source!
Because I have been involved with QUIC and HTTP/3 development at the IETF for some time, I would also be able to shed some light on certain aspects of these protocols.
I emailed Randal Schwartz, the host of the show, and he slotted me in for November 27th.
Preparation
The date of the show was drawing near and, as I still had not received any details about the setup, I began to worry. Did Randal forget about me? Finally, on the 26th, I emailed Randal basically to ask him this. He replied, somewhat incredulously, “did you not receive my previous emails?!” It turns out, Gmail had been helpfully discarding Randal’s emails as SPAM. Sabotage?*
The preparation for the show consisted mainly of testing my audio and video equipment. While the laptop’s built-in webcam was OK, TWiT deems any built-in laptop microphone as inadequate. I also learned about positioning of the webcam (eye-level) and lighting (in front, not behind). These simple adjustments make a large difference.
Making The Show
I admit it: I was nervous. While I have spoken in front of large audiences before, a live TV show would be a new experience for me.
I made a Skype connection with a TWiT Live Line user a few minutes before the show. We checked the audio and video feeds and I said hello to Randal and Jonathan Bennett, that week’s co-host. And the show was on!
Within a few minutes — after the introduction and an advertisement from the show’s sponsors — I was brought on and chatted away happily with Randal and Jonathan about QUIC, HTTP/3, and LSQUIC. The interviewers prepared well and we discussed the evolution of HTTP/3, the good and the bad of basing a transport protocol on top of UDP, LSQUIC performance and history, and many other topics. There were a few questions from the live “studio audience,” which is just a dedicated IRC channel. Some levity was had as well: no, there is no HTTP/4… yet!
After Recording
Rather than providing a condensed transcript, I suggest you click the link above and watch the YouTube video or grab it from your favorite podcast platform. The discussion lasted for about 50 minutes — or almost the whole show — after which it was time to wrap up.
The edited version of the show was posted online in three or four hours. They did not take anything out; rather, the show was rearranged by topics, as we jumped around a bit, and for a better flow. It was a good editing job, which is not surprising: ours was FLOSS Weekly show number 557.
I loved being on the FLOSS Weekly show and I recommend it to everyone. If you have an open-source project that you love, you should consider sharing it with the rest of the world on TWiT with Randal Schwartz and his brigade of co-hosts.
It was a special thrill for me to be interviewed by Randal Schwartz, the co-author of Programming Perl and Just another Perl hacker. That book I had on my bed stand and I carried it with me on my commute to college, hiding it under my jacket from the rain. Perl was my first true love.
