After a long time in the development "oven", OpenLI 1.1.0 has now been released!
A brief summary of the changes in this release can be found in the OpenLI ChangeLog. More details on the bigger changes will be provided later in this post.
Packages are available for download from our Cloudsmith repository for the following Linux distributions:
Debian (buster, bullseye, and bookworm)
Ubuntu (18.04, 20.04, and 22.04)
CentOS 7
RockyLinux 8 and 9
AlmaLinux 8 and 9
Fedora 37 and 38
Instructions for downloading and installing the OpenLI packages are available on the OpenLI wiki.
You can also, of course, install OpenLI from source by cloning the OpenLI git repository.
Email Interception
The most obvious addition in the 1.1.0 release is the support for email interception (i.e. the ETSI TS 102 232-2 standard).
OpenLI supports email interception for the SMTP, POP3 and IMAP protocols, using the email address as a target identifier. You may specify multiple email addresses for an intercept target to cover aliases or people with multiple mailboxes.
The provisioner REST API has been updated to support email intercepts and the web provisioner UI will also soon be upgraded accordingly.
In addition to being able to intercept email through packet capture, OpenLI collectors can also ingest email application layer messages that are sent to an HTTP(S) service running on the collector. This allows operators to use custom plugins on their mail services (such as dovecot or postfix plugins) to forward the email sessions to the collector directly from the mail servers, rather than having to mirror and intercept mail traffic at the network layer.
More details on the email ingestion feature, including the required format for messages sent to the ingestion service and how to configure a collector to support email ingestion can be found on the OpenLI wiki.
Mediator Enhancements
The previous design of the mediator code had some performance limitations and was not able to reliably retain intercept records if an agency handover was unavailable.
As a result, the internal structure of the mediator has undergone a major re-design that can be summarised in two changes.
Firstly, each collector which connects to a mediator, and each law enforcement agency that the mediator is connected to, are handled by a separate dedicated thread. Previously, we used an event loop to "juggle" servicing each of these entities, which meant that a particularly busy collector or agency could potentially starve the others of processing time. This change allows a mediator to make better use of multiple cores on the host hardware and achieve better scalability as the number of collectors and agencies grows.
Second, each mediator now uses RabbitMQ internally as a message broker to transfer intercept records from the collector-handling thread that receives them to the agency handover thread that will be forwarding them. This is a separate use of RabbitMQ in addition to the existing optional capability to have RabbitMQ broker records between the collector and the mediator.
Intercept records that have been received by a mediator but not yet been passed on their respective agency will be buffered on disk by RabbitMQ during that intermediate period. This means that if the mediator is restarted, those records will not be lost as they would have been in previous OpenLI versions which only buffered those records in memory.
The internal use of RabbitMQ by the mediator is transparent and should require no additional configuration for OpenLI users who use a packaged install. Most users will not notice this change but if you do run into problems with your mediator after upgrading, please reach out to us through our support channels.
Bug and Compliance Fixes
As our user base grows, we learn more about specific requirements in countries that OpenLI does not quite meet for one reason or another and, of course, we also get more bug reports. This is great, because it allows us to keep improving the software and make it useful in a wider variety of situations.
Here are some of the compliance issues that we've fixed:
OpenLI now supports configuring intercepts to emit only IRIs, or only CCs.
HI1 Operation messages now include the target-Information field, which is required in some jurisdictions.
IP Multimedia (VoIP) IRIs now no longer use the IRI type IRI-Report instead of the IRI-Continue type. This was a bug, but was being highlighted by our users as an incompatibility when testing with their local authorities.
The major bugs that we've fixed were:
a REST API bug that prevented DELETE requests from working for the coreserver/ endpoint.
a REST API bug where the end time for a VoIP intercept could not be modified.
a VOIP intercept bug where SIP sessions that include a 183 message that did not contain any RTP or IP information were not producing CC records.
a formatting bug that caused GTP intercepts to not be accepted by the mediator.
