A dive into opentelemetry with OpenObserve a single binary that provides full stack observability.
This was a quick exploritory dive into this world, so things are a little less polished here, just wanted to get up and testing as quick as possible.
Using K8s/k3s for this, just because it can be quick.
Stand up the services:
# MySQL 8 (I used mariadb here because I had trouble connecting from the other composer image we'll use in a minute, this works fine for rn)
kubectl run mysql8 --image=mariadb --env="MYSQL_ROOT_PASSWORD=password" --port=3306
# Expose MySQL 8 to the cluster
kubectl create svc clusterip mysql8 --tcp=3306:3306
# Standup a container to build wordpress in (in hindsight, this would probably be easier to use the wp docker and add composer)
kubectl run --image=composer:2 -- wordpress tail -f /dev/null
# Expose that
kubectl create svc clusterip wordpress --tcp=80:80
# Create ingress
kubectl create ingress observe-wordpress --rule=owp.kube.reaz.io/\*=wordpress:80 # Note: (you want to edit the "pods/wordpress" to add a app= label [again, #mvp])
# Now spin up open observe
kubectl create namespace openobserve
kubectl apply -f https://raw.githubusercontent.com/zinclabs/openobserve/main/deploy/k8s/statefulset.yaml
# Open ingress to that
k create ingress observe --rule=observe.kube.reaz.io/\*=openobserve:5080 # Note: ( you want to edit the "pods/observe-0" to add a app= label)
Next we want opentelemetry to start logging on wordpress, so we're going set it up and then have it start automatically collect traces
First, open a shell into the box
kubectl exec -it pods/wordpress -- bash
Then continue to setup the wordpress box
# We're going to compile some php extensions so we'll need these tools
apk add autoconf alpine-sdk tmux
# Next the php extensions
docker-php-ext-install pdo pod_mysql mysqli
# Next install opentelemetry php -> c extension from pecl
pecl install opentelemetry-1.0.0RC1
# Enable it
echo "extension=opentelemetry.so" > /usr/local/etc/php/conf.d/opentel.ini
# Now setup wordpress
wget wordpress.org/latest.zip
unzip latest.zip
cd wordpress
# Now setup composer with a composer.json like so
cat composer.json
{
"name": "root/wordpress",
"require": {
"open-telemetry/opentelemetry-auto-wordpress": "dev-main",
"open-telemetry/sdk": "^1.0@dev",
"open-telemetry/exporter-otlp": "^1.0@dev",
"php-http/guzzle7-adapter": "1.x-dev"
},
"minimum-stability": "dev",
"config": {
"allow-plugins": {
"php-http/discovery": false
}
}
}
# Install composer dependencies
composer install
# Instruct php to always autoload the opentelemetry
echo "auto_prepend_file=/app/wordpress/vendor/autoload.php" >> /usr/local/etc/php/conf.d/opentel.ini
# Then open a tmux session and start a temporary web server
tmux
env OTEL_PHP_AUTOLOAD_ENABLED=true \
OTEL_SERVICE_NAME=otelwp \ # Replace w/ your service's preferred name
OTEL_TRACES_EXPORTER=otlp \
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://observe.kube.reaz.io/api/default/traces \
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic __YOUR_AUTH_HERE__" \ # update this auth token
OTEL_PROPAGATORS=baggage,tracecontext php -S 0.0.0.0:80
That's it for step one! You're up and running. Click around the wp site and teh wp-admin, you should start to see traces in the OpenObserve UI, with things like a GET request -> wp_connect() -> wp_query(), wp_request() like from this screenshot.
- Trouble installing the php packages from composer? Opentel just released RC, so its stability is a bit degraded, for now you might need to run the following if you didn't copy and paste the
composer.json
setup from above.
composer config minimum-stability dev
-
Sometimes the php server sticks and locks up or HTTPS doesn't work well
- Yeah, the
php -S
is great for what it is, but it doesn't handle these situations that well. Next guide will use a better WordPress docker image/php server.
- Yeah, the
-
Watch your php logs from the
php -S 0.0.0.0:80
command when clicking around wordpress pages, you should see errors. And/or ensure the observe logs are getting a200
when traces are being posted by tailing the logs there with
kubectl logs pods/observe -f
- Run a local non-http reporting server that will output traces in the console
# Log to console
env OTEL_PHP_AUTOLOAD_ENABLED=true \
OTEL_TRACES_EXPORTER=console \
OTEL_METRICS_EXPORTER=none \
OTEL_LOGS_EXPORTER=console \
php -S 0.0.0.0:80
- Create a test file to run at ad-hoc
<?php
declare(strict_types=1);
namespace OpenTelemetry\Example;
use OpenTelemetry\API\Logs\EventLogger;
use OpenTelemetry\API\Logs\LogRecord;
# Uncomment or play around with as needed
#putenv('OTEL_PHP_AUTOLOAD_ENABLED=true');
#putenv('OTEL_TRACES_EXPORTER=otlp');
#putenv('OTEL_METRICS_EXPORTER=otlp');
#putenv('OTEL_LOGS_EXPORTER=otlp');
#putenv('OTEL_EXPORTER_OTLP_PROTOCOL=http');
#putenv('OTEL_EXPORTER_OTLP_ENDPOINT=https://observe.kube.reaz.io/api/default/default/');
#putenv('OTEL_PROPAGATORS=b3,baggage,tracecontext');
echo 'autoloading SDK example starting...' . PHP_EOL;
// Composer autoloader will execute SDK/_autoload.php which will register global instrumentation from environment configuration
require dirname(__DIR__) . '/wordpress/vendor/autoload.php';
$instrumentation = new \OpenTelemetry\API\Instrumentation\CachedInstrumentation('demo');
$instrumentation->tracer()->spanBuilder('root')->startSpan()->end();
$instrumentation->meter()->createCounter('cnt')->add(1);
$eventLogger = new EventLogger($instrumentation->logger(), 'my-domain');
$eventLogger->logEvent('foo', new LogRecord('hello, otel'));
echo 'Finished!' . PHP_EOL;
Then run php test.php
- Opentelemetry (otel) php getting started guide https://opentelemetry.io/docs/instrumentation/php/getting-started/
- Otel auto wordpress package https://packagist.org/packages/open-telemetry/opentelemetry-auto-wordpress (Github)
- OpenObserve docs https://openobserve.ai/docs
- baggage HTTP header format for things like auth header (https://github.com/w3c/baggage/blob/main/baggage/HTTP_HEADER_FORMAT.md)
- OTEL env variables https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/