This guide provides a streamlined, copy-paste-friendly procedure for experienced administrators. This guide shows how to deploy a 5-node Apache Cloudberry cluster:
cdw)scdw)sdw1, sdw2, sdw3), each with 2 primary and 2 mirror instances:::info For detailed explanations of each step, refer to the full deployment guides: Configuring Your Systems, Installing Using RPM/DEB Package, Creating the Data Storage Areas, and Initializing Apache Cloudberry. :::
| Host | Role | IP (example) |
|---|---|---|
| cdw | Coordinator | 192.168.1.1 |
| scdw | Standby Coordinator | 192.168.1.2 |
| sdw1 | Segment 1 | 192.168.1.3 |
| sdw2 | Segment 2 | 192.168.1.4 |
| sdw3 | Segment 3 | 192.168.1.5 |
Data directories used:
/data/coordinator/data/primary (2 instances per host)/data/mirror (2 instances per host):::note This step applies to RHEL/Oracle/Rocky Linux only. On Ubuntu, SELinux is not installed by default and this step can be skipped. :::
# Check current status sestatus # If not disabled, deactivate it sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config # Reboot is required to take effect
For RHEL/Rocky Linux (firewalld):
# Check current status systemctl status firewalld # If not disabled, deactivate it systemctl stop firewalld.service systemctl disable firewalld.service
For Ubuntu (ufw):
# Check current status (disabled by default) ufw status # If active, disable it ufw disable
Set the hostname on each host (takes effect immediately, no reboot required):
# Run on each host respectively hostnamectl set-hostname cdw # on coordinator hostnamectl set-hostname scdw # on standby coordinator hostnamectl set-hostname sdw1 # on segment 1 hostnamectl set-hostname sdw2 # on segment 2 hostnamectl set-hostname sdw3 # on segment 3 # Verify hostname # Refresh the shell prompt to show the new hostname exec bash
Edit /etc/hosts on all hosts:
cat >> /etc/hosts <<EOF 192.168.1.1 cdw 192.168.1.2 scdw 192.168.1.3 sdw1 192.168.1.4 sdw2 192.168.1.5 sdw3 EOF
cat >> /etc/sysctl.conf <<EOF kernel.shmall = 197951838 kernel.shmmax = 810810728448 kernel.shmmni = 4096 vm.overcommit_memory = 2 vm.overcommit_ratio = 95 net.ipv4.ip_local_port_range = 10000 65535 kernel.sem = 250 2048000 200 8192 kernel.sysrq = 1 kernel.core_uses_pid = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.msgmni = 2048 net.ipv4.tcp_syncookies = 1 net.ipv4.conf.default.accept_source_route = 0 net.ipv4.tcp_max_syn_backlog = 4096 net.ipv4.conf.all.arp_filter = 1 net.ipv4.ipfrag_high_thresh = 41943040 net.ipv4.ipfrag_low_thresh = 31457280 net.ipv4.ipfrag_time = 60 net.core.netdev_max_backlog = 10000 net.core.rmem_max = 2097152 net.core.wmem_max = 2097152 vm.swappiness = 10 vm.zone_reclaim_mode = 0 vm.dirty_expire_centisecs = 500 vm.dirty_writeback_centisecs = 100 vm.dirty_background_ratio = 0 vm.dirty_ratio = 0 vm.dirty_background_bytes = 1610612736 vm.dirty_bytes = 4294967296 kernel.core_pattern=/var/core/core.%h.%t EOF mkdir -p /var/core sysctl -p
:::note The kernel.shmall and kernel.shmmax values above are examples for a host with ~1583 GB memory. Calculate the correct values for your system:
echo $(expr $(getconf _PHYS_PAGES) / 2) # kernel.shmall echo $(expr $(getconf _PHYS_PAGES) / 2 \* $(getconf PAGE_SIZE)) # kernel.shmmax
:::
cat >> /etc/security/limits.conf <<EOF * soft nofile 524288 * hard nofile 524288 * soft nproc 131072 * hard nproc 131072 * soft core unlimited EOF
nofile — maximum number of open file descriptorsnproc — maximum number of processescore — core dump file size (unlimited enables core dumps for debugging):::note If you are using cloud VMs with only a single system disk (no additional data disk), you can skip the disk formatting and mounting steps below. Simply create the data directory on the existing filesystem:
mkdir -p /data
The XFS formatting steps below apply when you have a dedicated data disk available. :::
mkdir -p /data mkfs.xfs -f /dev/sdb # adjust device name as needed # Use device name directly echo "/dev/sdb /data xfs rw,nodev,noatime,inode64 0 0" >> /etc/fstab # Or use UUID for stable identification across reboots (recommended for cloud/VM environments) # echo "UUID=$(blkid -s UUID -o value /dev/vdb1) /data xfs rw,nodev,noatime,inode64 0 0" >> /etc/fstab mount /data
# Set read-ahead value /sbin/blockdev --setra 16384 /dev/sdb # Persist read-ahead across reboots echo '/sbin/blockdev --setra 16384 /dev/sdb' >> /etc/rc.d/rc.local chmod +x /etc/rc.d/rc.local # Set I/O scheduler permanently (for non-NVMe/SSD disks) # For RHEL/Rocky Linux: grubby --update-kernel=ALL --args="elevator=mq-deadline" # For Ubuntu: sed -i 's/GRUB_CMDLINE_LINUX="\(.*\)"/GRUB_CMDLINE_LINUX="\1 elevator=mq-deadline"/' /etc/default/grub && update-grub
# Check current THP status cat /sys/kernel/mm/*transparent_hugepage/enabled # If not [never], disable it permanently # For RHEL/Rocky Linux: grubby --update-kernel=ALL --args="transparent_hugepage=never" # For Ubuntu: sed -i 's/GRUB_CMDLINE_LINUX="\(.*\)"/GRUB_CMDLINE_LINUX="\1 transparent_hugepage=never"/' /etc/default/grub && update-grub # Reboot is required to take effect
sed -i 's/^#RemoveIPC=.*/RemoveIPC=no/' /etc/systemd/logind.conf service systemd-logind restart
sed -i 's/^#MaxStartups.*/MaxStartups 10:30:200/' /etc/ssh/sshd_config sed -i 's/^#MaxSessions.*/MaxSessions 200/' /etc/ssh/sshd_config service sshd restart
# For RHEL/Rocky Linux systemctl enable chronyd systemctl restart chronyd chronyc tracking # For Ubuntu (enable must use 'chrony', restart works with either name) systemctl enable chrony systemctl restart chronyd chronyc tracking
groupadd gpadmin useradd gpadmin -r -m -g gpadmin passwd gpadmin # Grant passwordless sudo # For RHEL/Rocky Linux: uncomment the %wheel NOPASSWD line in sudoers # Or use visudo to uncomment: %wheel ALL=(ALL) NOPASSWD: ALL sed -i 's/^# %wheel\tALL=(ALL)\tNOPASSWD: ALL/%wheel\tALL=(ALL)\tNOPASSWD: ALL/' /etc/sudoers usermod -aG wheel gpadmin # For Ubuntu: add gpadmin to sudo group and create a sudoers drop-in file usermod -aG sudo gpadmin echo "gpadmin ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/gpadmin chmod 440 /etc/sudoers.d/gpadmin # Set data directory ownership to gpadmin chown -R gpadmin:gpadmin /data
:::note Make sure the gpadmin user has the same UID/GID across all hosts. Use id gpadmin to verify, or pass -g <gid> to groupadd and -u <uid> to useradd to set fixed values. :::
Download the package from Apache Cloudberry Releases to /home/gpadmin/ on every host, then install:
# For RPM (Rocky Linux, RHEL, etc.) dnf install /home/gpadmin/apache-cloudberry-db-incubating-*.rpm # For DEB (Ubuntu) # apt install --fix-broken /home/gpadmin/apache-cloudberry-db-incubating-*.deb # Set ownership chown -R gpadmin:gpadmin /usr/local/cloudberry*
su - gpadmin source /usr/local/cloudberry-db/cloudberry-env.sh # Generate SSH key (press Enter at passphrase prompts) ssh-keygen -t rsa -b 4096 -N "" -f ~/.ssh/id_rsa # Enable 1-n passwordless SSH (will prompt for gpadmin password) ssh-copy-id cdw ssh-copy-id scdw ssh-copy-id sdw1 ssh-copy-id sdw2 ssh-copy-id sdw3 # Create the host file for gpssh-exkeys cat > ~/hostfile_exkeys <<EOF cdw scdw sdw1 sdw2 sdw3 EOF # Enable n-n passwordless SSH gpssh-exkeys -f ~/hostfile_exkeys # Verify gpssh -f ~/hostfile_exkeys -e 'hostname'
gpssh -f ~/hostfile_exkeys -e 'ls -l /usr/local/cloudberry-db-*'
source /usr/local/cloudberry-db/cloudberry-env.sh # Coordinator data directory sudo mkdir -p /data/coordinator sudo chown gpadmin:gpadmin /data/coordinator # Standby coordinator data directory gpssh -h scdw -e 'sudo mkdir -p /data/coordinator' gpssh -h scdw -e 'sudo chown gpadmin:gpadmin /data/coordinator' # Segment host file cat > ~/hostfile_gpinitsystem <<EOF sdw1 sdw2 sdw3 EOF # Segment primary and mirror directories gpssh -f ~/hostfile_gpinitsystem -e 'sudo mkdir -p /data/primary' gpssh -f ~/hostfile_gpinitsystem -e 'sudo mkdir -p /data/mirror' gpssh -f ~/hostfile_gpinitsystem -e 'sudo chown -R gpadmin:gpadmin /data'
# Already created in step 4 as ~/hostfile_gpinitsystem cat ~/hostfile_gpinitsystem
mkdir -p ~/gpconfigs cat > ~/gpconfigs/gpinitsystem_config <<EOF SEG_PREFIX=gpseg PORT_BASE=6000 declare -a DATA_DIRECTORY=(/data/primary /data/primary) COORDINATOR_HOSTNAME=cdw COORDINATOR_DIRECTORY=/data/coordinator COORDINATOR_PORT=5432 TRUSTED_SHELL=ssh CHECK_POINT_SEGMENTS=8 ENCODING=UNICODE MIRROR_PORT_BASE=7000 declare -a MIRROR_DATA_DIRECTORY=(/data/mirror /data/mirror) DATABASE_NAME=warehouse EOF
gpinitsystem -c ~/gpconfigs/gpinitsystem_config -h ~/hostfile_gpinitsystem \ -s scdw --mirror-mode=spread
When prompted, type y to continue. On success you should see:
=> Apache Cloudberry instance successfully created.
:::tip The -s scdw option initializes the standby coordinator during cluster creation. If you omit -s during gpinitsystem, you can initialize the standby separately afterwards:
gpinitstandby -s scdw
Verify the standby is synchronized:
gpstate -f
:::
gpconfig -c TimeZone -v 'US/Pacific' gpstop -ra
cat >> ~/.bashrc <<EOF source /usr/local/cloudberry-db/cloudberry-env.sh export COORDINATOR_DATA_DIRECTORY=/data/coordinator/gpseg-1 export PGPORT=5432 export PGUSER=gpadmin export PGDATABASE=warehouse # (Optional) RHEL/Rocky Linux: ensure ps works under cloudberry-env.sh export LD_PRELOAD=/lib64/libz.so.1 ps EOF source ~/.bashrc # Copy to standby scp ~/.bashrc scdw:~/
psql -d warehouse
Check cluster status:
warehouse=# SELECT * FROM gp_segment_configuration ORDER BY content, role;
# Check cluster status gpstate -s # Stop the cluster gpstop -a # Start the cluster gpstart -a # Restart the cluster gpstop -ra