Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
InfluxData has launched an open source InfluxDB IOx project that it said will become the foundation for the next generation of the company’s time-series databases.
At the same time, the company has made InfluxDB Open Source 2.0 generally available. This version of the company’s database comes in the form of a single binary that makes it easier to secure.
InfluxDB Open Source 2.0 also provides access to Data Explorer visualization tools as well as a lighter-weight Flux query language purpose-built for time series data.
InfluxData CTO Paul Dix said while improvements are still being made to the existing database offering, the time has come to start working toward addressing a number of issues via a new project anchored around a storage engine that will make it possible to query petabytes of data across thousands of servers. The InfluxDB IOx project will address that issue by eliminating current restrictions on cardinality, data size and cluster size that are inherent in the current platform, he said.
The InfluxDB IOx project is written in the Rust programming language and incorporates Apache Arrow, an open source in-memory analytics engine that incorporates a columnar data store that is being advanced under the auspices of the Apache Software Foundation (ASF). That approach will also make it possible for InfluxDB to natively support SQL along with InfluxQL and Flux query languages.
The current plan is to make InfluxDB IOx an optional storage backend in a future point release. By the latter half of next year, InfluxData plans that InfluxDB Enterprise customers will have a commercially supported version of InfluxDB IOx and InfluxDB Enterprise. InfluxDB IOx will have its own builds and can be run separately. In effect, Dix noted, InfluxDB IOx will make it easier to separate compute and storage requirements.
Dix said that while InfluxDB has served as the foundation that enables a wide range of monitoring tools to collect data in real-time, current cardinality limits that impact the number of tables that can be joined requires a new approach to add support for distributed tracing capabilities required to observe microservices-based applications. The IOx version will also employ object storage platforms to reduce the total cost of ownership while making it possible to address the requirements of distributed computing environments that now include a wide range of edge computing platforms, he noted.
The decision to embrace Apache Arrow also opens the platform to end users who prefer to work with SQL and other scripting languages, he added.
It may be a while before observability tools that leverage InfluxDB IOx find their way into DevOps tools. However, it’s clear that time-series databases will evolve to meet the requirements of cloud-native applications that will be based on large numbers of microservices spanning distributed computing environments. The only issue is to what degree the current rate of deployment of these applications might outpace the current ability of InfluxDB to keep pace.
