Imply launches Lumi LogLake for searchable security data

Imply has made Lumi LogLake generally available, expanding its Lumi data platform for security teams.

The product is designed to let organisations search security telemetry stored in data lakes and object storage without moving the data into a separate search environment. It works with data where it sits, including in S3 and in formats such as Apache Iceberg, Delta Lake, Parquet, JSON and GZIP.

The launch comes as security teams face growing volumes of machine data while trying to manage storage costs and maintain access to older records. Companies are under pressure to decide what telemetry to retain, how long to keep it and how to make historical data usable for investigations.

Imply argues that AI has increased the value of older security data because larger historical datasets can provide more context for investigations and automated analysis. At the same time, object storage and data lakes have emerged as a cheaper place to keep those records than traditional security data systems, although search and access have often remained difficult.

Search in place

Lumi LogLake enables users to query unstructured machine data directly in object storage without predefined schemas, data catalogues, rehydration or duplicate data pipelines, according to Imply. That allows security teams to search both current and historical telemetry while continuing to use existing query languages, dashboards and detection processes.

The product also separates compute from storage, allowing teams to use permanent compute resources for live monitoring and call on extra compute only when they need to run historical or investigative searches. This approach is intended to help organisations manage costs while preserving access to larger stores of telemetry.

Another element of the launch is a single access layer for security data that can be used by both security tools and AI applications. In practice, the aim is to give analysts and automated systems access to the same underlying logs rather than maintain separate stores for different workloads.

Security teams have often had to prepare, rehydrate or transfer archived data before they can search it, particularly when information is held in lower-cost object storage. That can slow investigations and add technical complexity, especially when analysts need to move between real-time systems and older archives.

Imply's pitch is that organisations should be able to retain larger volumes of telemetry in cheaper storage while still making it immediately searchable. The company is targeting a common tension in cybersecurity operations, where the data most useful for long-running investigations or pattern analysis may also be the most expensive to keep in readily searchable form.

Imply is best known as the company behind Lumi and was founded by the original creators of Apache Druid. It focuses on software for searching and analysing large volumes of machine and event-driven data.

Eric Tschetter, Chief Architect at Imply, said the product was built to remove the trade-off between retention and usability. "Security teams shouldn't have to choose between keeping the data they need and being able to use it," Tschetter said. "That's the problem we built LogLake to solve. Teams can keep more history in object storage and search it when they need it, without changing how their analysts work."