Performance/memory optimizations, stability/bug fixes, and small enhancements (#183) * Reduce query stuttering effect during range topology change * Simplify client range router patching mechanism * Refactor BatchPubCall and related classes to improve fan-out handling and reduce memory overhead * Don't report redundant ServerBusy event * Reduce loading time by making DistWorkerCoProc/InboxStoreCoProc reset process async * Reduce memory overhead by doing contract to root after remove * Refactor base-kv to support report leader change to CoProc * Only report tenant metrics from leader co-proc * Optimize retry logic for pushing qos1/2 messages and adjust related events * New event for matching retained messages successfully and related tests * Add createdAt field to InboxMetadata and implement drop event reporting on delete * Add an api to retrieve inbox state * Reduce memory overhead during resetting * fixed the bug causing DistWorkerCleaner stop running * Adjust default values of gc params * Conditional clear batch call state during reset to avoid task leaking * Prevent concurrent update non-thread safe result proto builder * Improved child removal logic and add benchmarking for performance evaluation * Add set session type to ClientInfo before making attach request * Enhance DistWorkerCleaner with heuristic interval and step * Implemented dynamic sending window for confirmable messages * Improve gossiping: * Do not early confirm when single node * Reduce message complexity * Optimize child branch detachment and compression logic in TopicLevelTrie * Enhance TopicIndex to support custom value equality strategy * Fix potential consistency issue in TenantRouteCache and TopicIndex * Update tenant stats on inbox clearance and add integration test for session deletion * Refactor InboxMetaCache to reduce refresh/seek operation * Correct range lookup key for InboxCheckSubScheduler * Ensure strict fifo order for check permission call * Try drain staging buffer immediately after buffering message * Schedule an on-demand hint/confirm timeout to ensure fetch loop non-stop * Add @JsonMerge annotation to service configuration classes to reduce config file complexity further * Reduce IO overhead for InboxStoreCoProc
BifroMQ is a high-performance, distributed MQTT broker that natively supports multi-tenancy. It is designed to enable the building of large-scale IoT device connectivity and messaging systems.
You can access the documentation on the official website. Additionally, contributions to the documentation are welcome in the GitHub repository.
docker run -d -m <MEM_LIMIT> -e MEM_LIMIT='<MEM_LIMIT_IN_BYTES>' --name bifromq -p 1883:1883 bifromq/bifromq:latest
Substitute <MEM_LIMIT> and <MEM_LIMIT_IN_BYTES> with the actual memory allocation for the Docker process, for example, 2G for <MEM_LIMIT> and 2147483648 for <MEM_LIMIT_IN_BYTES>. If not specified, BifroMQ defaults to using the hosting server‘s physical memory for determining JVM parameters. This can result in the Docker process being terminated by the host’s Out-of-Memory (OOM) Killer. Refer to here for more information.
You can build a BifroMQ cluster using Docker Compose on a single host for development and testing. Suppose you want to create a cluster with three nodes: node1, node2, and node3. The directory structure should be as follows:
|- docker-compose.yml |- node1 |- node2 |- node3
Each node should have a configuration file, it is defined as follows:
clusterConfig: env: "Test" host: bifromq-node1 # Change this to bifromq-node2 for node2 and bifromq-node3 for node3 port: 8899 seedEndpoints: "bifromq-node1:8899,bifromq-node2:8899,bifromq-node3:8899"
The docker-compose.yml file defines the services for the three nodes:
services: bifromq-node1: image: bifromq/bifromq:latest container_name: bifromq-node1 volumes: - ./node1/standalone.yml:/home/bifromq/conf/standalone.yml ports: - "1883:1883" environment: - MEM_LIMIT=2147483648 # Adjust the value according to the actual host configuration. networks: - bifromq-net bifromq-node2: image: bifromq/bifromq:latest container_name: bifromq-node2 volumes: - ./node2/standalone.yml:/home/bifromq/conf/standalone.yml ports: - "1884:1883" environment: - MEM_LIMIT=2147483648 networks: - bifromq-net bifromq-node3: image: bifromq/bifromq:latest container_name: bifromq-node3 volumes: - ./node3/standalone.yml:/home/bifromq/conf/standalone.yml ports: - "1885:1883" environment: - MEM_LIMIT=2147483648 networks: - bifromq-net networks: bifromq-net: driver: bridge
To launch the cluster, run the following command:
docker compose up -d
Clone the repository to your local workspace:
cd <YOUR_WORKSPACE> git clone https://github.com/apache/bifromq bifromq
Navigate to the project root folder and execute the following commands to build the entire project:
cd bifromq mvn wrapper:wrapper ./mvnw -U clean package
The build output consists of several archive files located under /target/output
bifromq-<VERSION>.tar.gzbifromq-<VERSION>-windows.zipExecute the following command in the project root folder to run all test cases, including unit tests and integration tests. Note: The tests may take some time to finish
mvn test
To quickly set up a BifroMQ server, extract the bifromq-<VERSION>.tar.gz file into a directory. You will see the following directory structure:
|- bin |- conf |- lib |- plugins
To start or stop the server, execute the respective command in the bin directory:
To start the server, run:
./standalone.sh start // This starts the server process in the background.
To stop the server, run:
./standalone.sh stop
The configuration file, standalone.yml, can be found in the conf directory. The settings within this file are named in a self-explanatory manner. By default, the standalone server stores persistent data in the data directory.
To jump start your BifroMQ plugin development, execute the following Maven command:
mvn archetype:generate \
-DarchetypeGroupId=org.apache.bifromq \
-DarchetypeArtifactId=bifromq-plugin-archetype \
-DarchetypeVersion=<BIFROMQ_VERSION> \
-DgroupId=<YOUR_GROUP_ID> \
-DartifactId=<YOUR_ARTIFACT_ID> \
-Dversion=<YOUR_PROJECT_VERSION> \
-DpluginName=<YOUR_PLUGIN_CLASS_NAME> \
-DpluginContextName=<YOUR_PLUGIN_CONTEXT_CLASS_NAME> \
-DbifromqVersion=<BIFROMQ_VERSION> \
-DinteractiveMode=false
Replace <YOUR_GROUP_ID>, <YOUR_ARTIFACT_ID>, <YOUR_PROJECT_VERSION>, <YOUR_PLUGIN_CLASS_NAME>, and < YOUR_PLUGIN_CONTEXT_CLASS_NAME> with your specific details. This command generates a ready-to-build multi-module project structured for BifroMQ plugin development.
Important Note: The archetype version should be 3.2.0 or higher as the archetype is compatible starting from version 3.2.0. Ensure that <BIFROMQ_VERSION> is set accordingly.
BifroMQ has two cluster deployment modes: Standard Cluster, Independent-Workload Cluster
The standard cluster deployment mode is suitable for small to medium-sized production environments that require reliability and scalability. It comprises several fully functional standalone nodes working together as a logical MQTT broker instance, ensuring high availability. You can also scale up the concurrent mqtt connection workload by adding more nodes, while some types of messaging related workload are not horizontal scalable in this mode.
The Independent Workload Cluster deployment mode is designed for building large-scale, multi-tenant serverless clusters. In this mode, the cluster consists of several specialized sub-clusters, each focusing on a particular ‘independent type’ of workload. These sub-clusters work together coherently to form a logical MQTT broker instance.
We welcome you to connect with the BifroMQ community:
Apache BifroMQ™ is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.