GH-49382: [Python] Enable OpenTelemetry on PyArrow wheels (#49383)
### Rationale for this change
Currently we are not building wheels with OpenTelemetry.
### What changes are included in this PR?
- Adding a new feature on our `vcpkg.json` to install required opentelemetry-cpp
- Enabling the opentelemetry feature when downloading dependencies via vcpkg for our wheels.
- Setting `ARROW_WITH_OPENTELEMETRY=ON` on the wheel build scripts
- Minor fix on checking exposed symbols (opentelemetry symbols are exposed)
- Minor improvement to `python/examples/flight/middleware.py` to be able to easily enable opentelemetry middleware tracing for testing.
- Fix on `server_tracing_middleware.cc` to retrieve tracer from `arrow::internal::tracing::GetTracer();` instead of `otel::trace::Provider::GetTracerProvider()->GetTracer("arrow");`
- Expose missing symbols to enable Windows builds
- Fix Windows compatibility adding windows compat header.
- Add `is_opentelemetry_enabled` utility function in pyarrow to validate whether OpenTelemetry was enabled or not when building Arrow C++. Expose this information on `pa.show_info()`.
### Are these changes tested?
Yes, wheels are being successfully built and tested locally with a built wheel that traces are being generated when properly configured.
Added test to validate traces appear on stdout and I've also validated Acero traces are being correctly generated by running the test suite and seeing the opentelemetry traces on Jaeger UI.
### Are there any user-facing changes?
Yes, PyArrow wheel users will be able to enable OpenTelemetry traces.
* GitHub Issue: #49382
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>Apache Arrow is a universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics. It contains a set of technologies that enable data systems to efficiently store, process, and move data.
Major components of the project include:
↗: Arrow-powered API, drivers, and libraries for access to databases and query engines↗↗↗↗↗↗↗The ↗ icon denotes that this component of the project is maintained in a separate repository.
Arrow is an Apache Software Foundation project. Learn more at arrow.apache.org.
The reference Arrow libraries contain many distinct software components:
The official Arrow libraries in this repository are in different stages of implementing the Arrow format and related features. See our current feature matrix on git main.
Please read our latest project contribution guide.
Even if you do not plan to contribute to Apache Arrow itself or Arrow integrations in other projects, we'd be happy to have you involved:
We use runs-on for managing the project self-hosted runners. We use AWS for some of the required infrastructure for the project.