Get the latest tech news
OpenTelemetry protocol with Apache Arrow
We are excited to announce the next phase of the OpenTelemetry Protocol with Apache Arrow project (OTel-Arrow). We began this project several years ago with the goal of bridging between OpenTelemetry data and the Apache Arrow ecosystem. Apache Arrow is a framework designed for zero-copy exchange of structured data between column-oriented data producers and consumers. We believe that having OpenTelemetry data accessible to external systems through Apache Arrow will lead to powerful integrations, with the potential for new telemetry systems and applications to emerge. For large streams of telemetry, we know that column-oriented data handling is substantially more efficient, with improved data compression and performance.
With the help of the OpenTelemetry Governance Committee, we’ve defined a project scope that entails studying the potential for Rust-based OpenTelemtry pipelines without “being” a Collector. We couldn’t help but notice: the Rust ecosystem around Apache Arrow is large and vibrant, and we think it would be a missed opportunity not to explore this space. In the first phase of the project, we developed the protocol through a Golang adapter library and a matching pair of Exporter and Receiver components in the OpenTelemetry Collector Contrib repository.
Or read this on Hacker News