---

Introduction

Bithon is a word combining binocular together with python.

It targets application metrics, logging, distributed tracing, alert and application risk governance under microservice environment.

Architecture

The above pic illustrates the main components of this project, including:

• Agent, which collects metrics/tracing logs automatically from client application without any code modification at the application side
• Collector, which provides various interfaces (including OpenTelemetry GRPC interface) to receive metrics/tracing logs from clients
• Pipeline, which provides a flexible and robust way to hande small data scale to a very huge data scale for incoming metrics or tracing logs
• Storage, which provides an abstraction to underlying storages like H2, MySQL or Clickhouse
• Alerting, which allows us to set up alerts by using MetricSQL style expression on existing metrics or tracing logs
• Web, which provides a simple web portal for metrics/tracing log visualization

Highlights

• Around 200 built-in metrics for various JDK or various Java middlewares like Apache Http Components
• Open Telemetry Tracing standard support and integration
• Built-in debugging diagnosis commands, including JMX bean realtime monitoring for target application
• Flexible deployment to adapt small data scale and huge data scale use cases
• Fast queries and very low storage cost benefit from ClickHouse
• PromQL style alerting expression support

Reference:

• White Paper
• How does the agent work?
• What's the difference between Jaeger and Bithon?
• What's the difference between OpenTelemetry and Bithon?

Preview

You can use the docker images in the Docker Hub to get a preview.

1. Start the backend service

   docker run --network=host -e bithon_application_env=local -e JAVA_OPTS="-Dspring.profiles.active=all-in-one" -it bithon/server:latest

   This starts the backend services with all-in-one profile, which includes all the services like tracing, event, metric, ctrl and alerting, using H2 database as back end storage. And also the backend services is monitored by itself, you can see the metrics and tracing logs of the backend services itself.

2. Start the front-end service

   docker run --network=host -e NEXT_PUBLIC_BITHON_API_SERVER_URL=http://localhost:9897 -itd bithon/web-app:latest

   Change the environment variable NEXT_PUBLIC_BITHON_API_SERVER_URL to point to your own backend server if needed.

After the above steps, visit http://localhost:3000 to experience.

Demo

A demo is provided by this demo repo with a docker-compose file. You can follow the README on that demo repo to start the demo within just 3 steps.

Build

1. Clone source code

After cloning this project along with all submodules by following commands

git clone https://github.com/FrankChen021/bithon.git
cd bithon && git submodule update --init

2. Configure JDK

Since backend services are built upon SpringBoot 3.0, a JDK 17 or higher (like 21) is required to build this project. If you have multiple JDKs on your machine, use export JAVA_HOME={YOUR_JDK_HOME} command to set correct JDK. For example

export JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk-17.jdk/Contents/Home

3. Build the project

For the first time to build this project, use the following command to build dependencies first:

mvn clean install --activate-profiles shaded,jooq -T 1C

and then execute the following command to build the project.

mvn clean install -DskipTests -T 1C

After the first build, we don't need to build the dependencies anymore unless there are changes in these dependencies.

Run

Once the project has been built, you could run the project in a standalone mode to evaluate this project.

1. Launch the backend services all in one

To launch server in evaluation mode, execute the following command:

java -Dspring.profiles.active=all-in-one -jar server/server-starter/target/server-starter.jar

By default, the application opens and listens on following ports at local

Function	Port
tracing	9895
event	9896
metric	9898
ctrl	9899
web	9897

Note: -Dspring.profiles.include parameter here is just for demo.

You can make changes to server/server-starter/src/main/resources/application.yml to reflect your own settings.

You can also use enable Alibaba Nacos as your configuration storage center.

2. Attach agent to your target Java applications

Attach the agent to your java application so that your application can be managed the agent. Add the the following VM arguments to your target Java application.

-javaagent:<YOUR_PROJECT_DIRECTORY>/agent/agent-distribution/target/agent-distribution/agent-main.jar -Dbithon.application.name=<YOUR_APPLICATION_NAME> -Dbithon.application.env=<YOUR_APPLICATION_ENV>

Variable	Description
YOUR_PROJECT_DIRECTORY	the directory where this project saves
YOUR_APPLICATION_NAME	the name of your application. It could be any string
YOUR_APPLICATION_ENV	the name of your environment to label your application. It could be any string. Usually it could be dev, test, prd

By default, the agent connects collector running at local(127.0.0.1). Collector address could be changed in file agent/agent-main/src/main/resources/agent.yml. Make sure to re-build the project after changing the configuration file above.

JDKs Compatibility

Even the project is built by JDK 17 and above, the agent is compatible with JDK 1.8+. The following matrix lists the JDKs that are compatible with the agent on macOS. And in theory, this matrix works both for Windows and Linux.

JDK	Supported
JDK 1.8.0_291	✓
JDK 9.0.4	✓
JDK 10.0.2	✓
JDK 11.0.12	✓
JDK 12.0.2	✓
JDK 13.0.2	✓
JDK 14.0.2	✓
JDK 15.0.2	✓
JDK 16.02	✓
JDK 17	✓
JDK 21	✓

JDK 11 and above

If the target application runs under JDK 11 and above, the following arguments should be added to JVM command to allow the agent to use Java Reflection on corresponding packages.

--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED --add-exports=java.base/sun.net.www=ALL-UNNAMED

Supported Components

Component	Min Version	Max Version	Metrics	Tracing
JVM	1.8		✓
JDK - Thread Pool	1.8		✓	✓
JDK - HTTP Client	1.8		✓	✓
Apache Druid(1)	0.16	31.0		✓
Apache Kafka(2)	0.10.0.0	3.9.0	✓	✓
Apache OZone	1.3.0			✓
Apache ZooKeeper Client	3.5	3.9	✓
Eclipse Glassfish	2.34			✓
GRPC	1.57.0		✓	✓
Google Guice	4.1.0			✓
HTTP Client - Apache	4.5.2	5.x	✓	✓
HTTP Client - Jetty	9.4.6		✓	✓
HTTP Client - Netty	3.10.6	< 4.0	✓	✓
HTTP Client - okhttp3	3.2	4.9	✓	✓
HTTP Client - reactor-netty	1.0.11		✓	✓
Jersey	1.19.4			✓
JDBC - Alibaba Druid	1.0.28		✓	✓
JDBC - Apache Derby	10.14.2		✓	✓
JDBC - H2	2.2.224		✓	✓
JDBC - MySQL	5.x	8.x	✓	✓
JDBC - PostgreSQL	42.4.3		✓	✓
MongoDB	3.4.2		✓
Open Feign	10.8			✓
Quartz	2.x		✓	✓
Redis - Jedis	2.9	5.x	✓	✓
Redis - Lettuce(3)	5.1.2	6.x	✓	✓
Redis - Redisson	3.19.0		✓	✓
Spring Boot	1.5	3.0+		✓
Spring Bean	4.3.12			✓
Spring Open Feign	10.8			✓
Spring Rest Template	4.3.12			✓
Spring Scheduling	4.3.12			✓
Spring Gateway	3.0.0		✓	✓
HTTP Server - Jetty	9.4.41		✓	✓
HTTP Server - Netty	2.0.0			✓
HTTP Server - Tomcat	8.5.20		✓	✓
HTTP Server - Undertow	1.4.12		✓	✓
xxl-job	2.3.0			✓

Restrictions

1. For Apache Druid, the Jersey plugin is required to be enabled to collect query information.
2. From Apache Kafka clients 3.7, the consumer metrics only works when the group.protocol is configured as classic which is the default configuration of the consumer client.
3. For Lettuce, the tracing support is only available when it's used with Spring Data Redis API.

User Doc

1. Metrics
2. Tracing
3. Logging
4. Diagnosis
5. Configuration
6. SDK
   1. Metrics
   2. Tracing

Contribution

To develop for this project, intellij is recommended.

A code style template file(dev/bithon_intellij_code_style) must be imported into intellij for coding.

For more information, check the development doc.

License

Apache License, Version 2.0