Apache Cloudberry is not fully compatible with PostgreSQL, and some features and syntax are Apache Cloudberry-specific. If your business already relies on Apache Cloudberry and you want to use the Apache Cloudberry-specific syntax and features on a single node to avoid compatibility issues with PostgreSQL, you can consider deploying Apache Cloudberry free of segments.
Apache Cloudberry provides the single-computing-node deployment mode. This mode runs under the utility gp_role, with only one coordinator (QD) node and one coordinator standby node, without a segment node or data distribution. You can directly connect to the coordinator and run queries as if you were connecting to a regular multi-node cluster. Note that some SQL statements are not effective in this mode because data distribution does not exist, and some SQL statements are not supported. See user behavior changes for details.
Log into each host as the root user, and modify the settings of each node server in the order of the following sections.
gpadmin admin userFollow the example below to create a user group and username gpadmin. Set the user group and username identifier to 520. Create and specify the gpadmin home directory /home/gpadmin.
groupadd -g 520 gpadmin # Adds user group gpadmin. useradd -g 520 -u 520 -m -d /home/gpadmin/ -s /bin/bash gpadmin # Adds username gpadmin and creates the home directory of gpadmin. passwd gpadmin # Sets a password for gpadmin; after executing, follow the prompts to input the password.
Run systemctl status firewalld to view the firewall status. If the firewall is on, you need to turn it off by setting the SELINUX parameter to disabled in the /etc/selinux/config file.
SELINUX=disabled
You can also disable the firewall using the following commands:
systemctl stop firewalld.service systemctl disable firewalld.service
Add relevant system parameters in the /etc/sysctl.conf configuration file, and run the sysctl -p command to make the configuration file effective.
When setting the configuration parameters, you can take the following example as a reference and set them according to your needs. Details of some of these parameters and recommended settings are provided below.
kernel.shmall = _PHYS_PAGES / 2 kernel.shmall = 197951838 kernel.shmmax = kernel.shmall * PAGE_SIZE kernel.shmmax = 810810728448 kernel.shmmni = 4096 vm.overcommit_memory = 2 vm.overcommit_ratio = 95 net.ipv4.ip_local_port_range = 10000 65535 kernel.sem = 250 2048000 200 8192 kernel.sysrq = 1 kernel.core_uses_pid = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.msgmni = 2048 net.ipv4.tcp_syncookies = 1 net.ipv4.conf.default.accept_source_route = 0 net.ipv4.tcp_max_syn_backlog = 4096 net.ipv4.conf.all.arp_filter = 1 net.ipv4.ipfrag_high_thresh = 41943040 net.ipv4.ipfrag_low_thresh = 31457280 net.ipv4.ipfrag_time = 60 net.core.netdev_max_backlog = 10000 net.core.rmem_max = 2097152 net.core.wmem_max = 2097152 vm.swappiness = 10 vm.zone_reclaim_mode = 0 vm.dirty_expire_centisecs = 500 vm.dirty_writeback_centisecs = 100 vm.dirty_background_ratio = 0 vm.dirty_ratio = 0 vm.dirty_background_bytes = 1610612736 vm.dirty_bytes = 4294967296
In the /etc/sysctl.conf configuration file, kernel.shmall represents the total amount of available shared memory, in pages. kernel.shmmax represents the maximum size of a single shared memory segment, in bytes.
You can define these 2 values using the operating system's _PHYS_PAGES and PAGE_SIZE parameters:
kernel.shmall = ( _PHYS_PAGES / 2) kernel.shmmax = ( _PHYS_PAGES / 2) * PAGE_SIZE
To get the values of these 2 operating system parameters, you can use getconf, for example:
$echo $(expr $(getconf _PHYS_PAGES)/2) $echo $(expr $(getconf _PHYS_PAGES)/2 \*$(getconf PAGE_SIZE))
vm.overcommit_memory indicates the overcommit handling modes for memory. Available options are:
0: Heuristic overcommit handling1: Always overcommit2: Don't overcommitSet the value of this parameter to 2 to refuse overcommit.
vm.overcommit_ratio is a kernel parameter and is the percentage of RAM occupied by the application process. The default value on CentOS is 50. vm.overcommit_ratio is calculated as follows:
vm.overcommit_ratio = (RAM - 0.026 * gp_vmem) / RAM
The calculation method of gp_vmem is as follows:
# If the system memory is less than 256 GB, use the following formula to calculate: gp_vmem = ((SWAP + RAM) – (7.5GB + 0.05 * RAM)) / 1.7 # If the system memory is greater than or equal to 256 GB, use the following formula to calculate: gp_vmem = ((SWAP + RAM) – (7.5GB + 0.05 * RAM)) / 1.17 # In the above formulas, SWAP is the swap space on the host, in GB. # RAM is the size of the memory installed on the host, in GB.
When the Apache Cloudberry uses the UDP protocol for internal connection, the network card controls the fragmentation and reassembly of IP packets. If the size of a UDP message is larger than the maximum size of network transmission unit (MTU), the IP layer fragments the message.
net.ipv4.ipfrag_high_thresh: When the total size of IP fragments exceeds this threshold, the kernel will attempt to reorganize IP fragments. If the fragments exceed this threshold but all fragments have not arrived within the specified time, the kernel will not reorganize the fragments. This threshold is typically used to control whether larger shards are reorganized. The default value is 4194304 bytes (4 MB).net.ipv4.ipfrag_low_thresh: Indicates that when the total size of IP fragments is below this threshold, the kernel will wait as long as possible for more fragments to arrive, to allow for larger reorganizations. This threshold is used to minimize unfinished reorganization operations and improve system performance. The default value is 3145728 bytes (3 MB).net.ipv4.ipfrag_time is a kernel parameter that controls the IP fragment reassembly timeout. The default value is 30.It is recommended to set the above parameters to the following values:
net.ipv4.ipfrag_high_thresh = 41943040 net.ipv4.ipfrag_low_thresh = 31457280 net.ipv4.ipfrag_time = 60
If the server memory exceeds 64 GB, the following parameters are recommended in the /etc/sysctl.conf configuration file:
vm.dirty_background_ratio = 0 vm.dirty_ratio = 0 vm.dirty_background_bytes = 1610612736 # 1.5GB vm.dirty_bytes = 4294967296 # 4GB
If the server memory is less than 64 GB, you do not need to set vm.dirty_background_bytes or vm.dirty_bytes. It is recommended to set the following parameters in the /etc/sysctl.conf configuration file:
vm.dirty_background_ratio = 3 vm.dirty_ratio = 10
To deal with emergency situations when the system is under memory pressure, it is recommended to add the vm.min_free_kbytes parameter to the /etc/sysctl.conf configuration file to control the amount of available memory reserved by the system. It is recommended to set vm.min_free_kbytes to 3% of the system's physical memory, with the following command:
awk 'BEGIN {OFMT = "%.0f";} /MemTotal/ {print "vm.min_free_kbytes =", $2 * .03;}' /proc/meminfo /etc/sysctl.conf
The setting of vm.min_free_kbytes is not recommended to exceed 5% of the system's physical memory.
Edit the /etc/security/limits.conf file and add the following content, which will limit the amount of hardware and software resources.
*soft nofile 524288 *hard nofile 524288 *soft nproc 131072 *hard nproc 131072
Add the following parameter to the /etc/sysctl.conf configuration file:
kernel.core_pattern=/var/core/core.%h.%t
Run the following command to make the configuration effective:
sysctl -p
Add the following parameter to /etc/security/limits.conf:
* soft core unlimited
XFS is the file system for the data directory of Apache Cloudberry. XFS has the following mount options:
rw,nodev,noatime,inode64
You can set up XFS file mounting in the /etc/fstab file. See the following commands. You need to choose the file path according to the actual situation:
mkdir -p /data0/ mkfs.xfs -f /dev/vdc echo "/dev/vdc /data0 xfs rw,nodev,noatime,inode64 0 0" /etc/fstab mount /data0 chown -R gpadmin:gpadmin /data0/
:::note The nobarrier option is not supported on RHEL 8/9 or Ubuntu systems. Use only the options rw,nodev,noatime,inode64. :::
Run the following command to check whether the mounting is successful:
df -h
The blockdev value for each disk file should be 16384. To verify the blockdev value of a disk device, use the following command:
sudo/sbin/blockdev --getra<devname>
For example, to verify the blockdev value of the example server disk:
sudo/sbin/blockdev --getra /dev/vdc
To modify the blockdev value of a device file, use the following command:
sudo/sbin/blockdev --setra<bytes> <devname>
For example, to modify the file blockdev value of the hard disk of the example server:
sudo/sbin/blockdev --setra16384/dev/vdc
The disk type, operating system, and scheduling policies of Apache Cloudberry are as follows:
Refer to the following command to modify the scheduling policy. Note that this command is only a temporary modification, and the modification becomes invalid after the server is restarted.
echo schedulername>/sys/block/<devname>/queue/scheduler
For example, temporarily modify the disk I/O scheduling policy of the example server:
echo deadline>/sys/block/vdc/queue/scheduler
To permanently modify the scheduling policy, use the system utility grubby. After using grubby, the modification takes effect immediately after you restart the server. The sample command is as follows:
grubby --update-kernel=ALL --args="elevator=deadline"
To view the kernel parameter settings, use the following command:
grubby --info=ALL
You need to disable Transparent Huge Pages (THP), because it reduces database performance. The command is as follows:
grubby --update-kernel=ALL --args="transparent_hugepage=never"
Check the status of THP:
cat /sys/kernel/mm/*transparent_hugepage/enabled
Disable IPC object deletion by setting the value of RemoveIPC to no. You can set this parameter in the /etc/systemd/logind.conf file of Apache Cloudberry.
RemoveIPC=no
After disabling it, run the following command to restart the server to make the disabling setting effective:
service systemd-logind restart
To set the SSH connection threshold, you need to modify the MaxStartups and MaxSessions parameters in the /etc/ssh/sshd_config configuration file. Both of the following writing methods are acceptable.
MaxStartups 200 MaxSessions 200
MaxStartups 10:30:200 MaxSessions 200
Run the following command to restart the server to make the setting take effect:
service sshd restart
Apache Cloudberry requires the clock synchronization to be configured for all hosts, and the clock synchronization service should be started when the host starts. You can choose one of the following synchronization methods:
The example in this document uses an external clock source for synchronization, that is, adding the following configuration to the /etc/chrony.conf configuration file:
# Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). server 0.centos.pool.ntp.org iburst
After setting, you can run the following command to check the clock synchronization status:
systemctl status chronyd
:::info Starting from Apache Cloudberry 2.1, RPM and DEB packages are officially provided. RPM packages support Rocky Linux 8/9, RHEL 8/9, and compatible distributions. DEB packages support Ubuntu 22.04. :::
Download the package to the home directory of gpadmin.
Install the package in the /home/gpadmin directory.
When running the following command, you need to replace <package path> with the actual package path, as the root user. During the installation, the directory /usr/local/cloudberry-db/ is automatically created.
cd /home/gpadmin # For RPM (Rocky Linux, RHEL, etc.) dnf install <RPM package path> # Or for older systems: yum install <RPM package path> # For DEB (Ubuntu) apt install <DEB package path> # Or alternatively: dpkg -i <DEB package path> && apt-get install -f
Grant the gpadmin user the permission to access the /usr/local/cloudberry-db/ directory.
chown -R gpadmin:gpadmin /usr/local/cloudberry*
Configure local SSH connection for the node. As the gpadmin user, perform the following operations:
ssh-keygen ssh-copy-id localhost ssh `hostname` # Makes sure that the local SSH connection works well.
Use the scripting tool gpdemo to quickly deploy Apache Cloudberry. gpdemo is included in the RPM package and will be installed in the GPHOME/bin directory along with the configuration scripts (gpinitsystem, gpstart, and gpstop). gpdemo supports quickly deploying Apache Cloudberry with a single computing node.
In the above setting mount options for the XFS file system, the XFS file system's data directory is mounted on /data0. The following commands deploy a single-computing-node cluster in this data directory:
cd /data0 NUM_PRIMARY_MIRROR_PAIRS=0 gpdemo # Uses gpdemo
When gpdemo is running, a warning will be output [WARNING]: -SinglenodeMode has been enabled, no segment will be created., which indicates that Apache Cloudberry is currently being deployed in the single-computing-node mode.
Perform the following steps to confirm the deployment mode of the current Apache Cloudberry cluster:
Connect to the coordinator node.
Execute SHOW gp_role; to view the operating mode of the cluster.
If utility is returned, it indicates that the cluster is in utility mode (maintenance mode), where only the coordinator node is available.
Continue to run the SHOW gp_internal_is_singlenode; command to see whether the cluster is in the single-computing-node mode.
on is returned, the current cluster is in the single-computing-node mode.off is returned, the current cluster is in utility maintenance mode.If dispatch is returned, it means that the current cluster is a regular cluster containing segment nodes. You can further check the segment count, status, port, data directory, and other information of the cluster by running SELECT * FROM n;.
gpdemo automatically creates a data directory in the current path ($PWD). For the single-computing-node deployment:
./datadirs/singlenodedir../datadirs/standby.When you are deploying Apache Cloudberry in the single-computing-node mode, the deployment script gpdemo writes gp_internal_is_singlenode = true to the configuration file postgresql.conf and starts a coordinator and a coordinator standby node with the gp_role = utility parameter setting. All data is written locally without a segment or data distribution.
In the single-computing-node mode, the product behavior of Apache Cloudberry has the following changes. You should pay attention to these changes before performing related operations:
CREATE TABLE to create a table, the DISTRIBUTED BY clause no longer takes effect. A warning is output: WARNING: DISTRIBUTED BY clause has no effect in singlenode mode.SCATTER BY clause of the SELECT statement is no longer effective. A warning is output: WARNING: SCATTER BY clause has no effect in singlenode mode.ALTER TABLE SET DISTRIBUTED BY) are declined with an error.UPDATE and DELETE statements will be reduced from ExclusiveLock to RowExclusiveLock to provide better concurrency performance, because there is only a single node without global transactions or global deadlocks. This behavior is consistent with PostgreSQL.