Pages

Automating Multi-Host Active Directory Join on Linux

Managing Linux servers in an Active Directory (AD) environment can be tedious, especially when you have multiple hosts that need to join the domain. Doing this manually is time-consuming, error-prone, and inconsistent.
In this post, I’ll share a fully automated, secure, and parallelized Bash script that allows you to join multiple Linux hosts to an AD domain, configure SSSD, home directories, and sudo permissions for AD groups — all in one go.

Why Automation Matters
  • Manually joining Linux hosts to AD can involve:
  • Installing the required packages (realmd, sssd, adcli, etc.)
  • Editing /etc/resolv.conf and /etc/hosts
  • Configuring Kerberos encryption
  • Joining the AD domain using credentials
  • Setting up SSSD and home directory creation
  • Granting sudo privileges to AD groups
Doing this across 10, 50, or 100 hosts is not scalable. With this script, it becomes:
Secure – prompts once for your AD password and never stores it in plain text.
Parallel – all hosts are processed simultaneously for faster deployment.
Robust – logs are kept per host, so troubleshooting is easier.

The Script: Multi-Host AD Join
Below is the script. Save it as ad_join_multi_pw.sh, make it executable with chmod +x ad_join_multi_hosts.sh, and run it with your hostnames as arguments:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
#!/bin/bash
#
# Author : adminCtrlX
# Description : Multi-host AD join with secure password prompt, parallel execution, logs
# Usage : ./ad_join_multi_hosts.sh host1 host2 host3
#
set -euo pipefail

### CHECK ARGUMENTS ###
if [ $# -lt 1 ]; then
echo "Usage: $0 <hostname1> [hostname2 ...]"
exit 1
fi

HOSTS=("$@")
AD_ADMIN="administrator"
AD_SERVER_FQDN="inddcpads01.ppc.com"
DOMAIN="ppc.com"
REALM="PPC.COM"
AD_GROUP="unix_admin"

# Prompt once for the AD password
read -s -p "Enter AD password for $AD_ADMIN: " AD_PASSWORD
echo
echo "===== Starting AD Integration for ${#HOSTS[@]} host(s) ====="

for TARGET_HOST in "${HOSTS[@]}"; do
(
LOG_FILE="/tmp/ad_join_${TARGET_HOST}.log"
echo "===== Starting AD join on $TARGET_HOST =====" | tee -a "$LOG_FILE"

ssh -q root@"$TARGET_HOST" bash -s "$DOMAIN" "$REALM" "$AD_SERVER_FQDN" "$AD_GROUP" "$AD_ADMIN" "$AD_PASSWORD" >> "$LOG_FILE" 2>&1 <<'REMOTE_EOF'

set -euo pipefail

# Assign positional parameters
DOMAIN="$1"
REALM="$2"
AD_SERVER_FQDN="$3"
AD_GROUP="$4"
AD_ADMIN="$5"
AD_PASSWORD="$6"
RESOLV_CONF="/etc/resolv.conf"
HOSTS_FILE="/etc/hosts"
SSSD_CONF="/etc/sssd/sssd.conf"
KRB_CRYPTO="/etc/krb5.conf.d/crypto-policies"
SUDOERS_FILE="/etc/sudoers.d/unix_admin"

echo "===== Running AD Integration on $(hostname) ====="

# Install required packages
dnf install -y realmd sssd oddjob oddjob-mkhomedir adcli samba-common-tools krb5-workstation

# Configure DNS (append safely)
grep -q "$DOMAIN" "$RESOLV_CONF" || echo "search $DOMAIN" >> "$RESOLV_CONF"
grep -q "192.168.10.100" "$RESOLV_CONF" || echo "nameserver 192.168.10.100" >> "$RESOLV_CONF"
grep -q "192.168.20.100" "$RESOLV_CONF" || echo "nameserver 192.168.20.100" >> "$RESOLV_CONF"

# Update /etc/hosts if missing
grep -q "$AD_SERVER_FQDN" "$HOSTS_FILE" || echo "192.168.10.100 $AD_SERVER_FQDN inddcpads01" >> "$HOSTS_FILE"

# Configure Kerberos crypto policies
mkdir -p "$(dirname "$KRB_CRYPTO")"
cat <<KRB > "$KRB_CRYPTO"
[libdefaults]
permitted_enctypes = aes256-cts-hmac-sha1-96 aes256-cts-hmac-sha384-192 camellia256-cts-cmac aes128-cts-hmac-sha1-96 aes128-cts-hmac-sha256-128 camellia128-cts-cmac rc4-hmac
KRB

# Join AD domain using password
echo "$AD_PASSWORD" | realm join -v -U "$AD_ADMIN" "$AD_SERVER_FQDN"

# Configure SSSD
cat <<SSSD > "$SSSD_CONF"
[sssd]
domains = $DOMAIN
services = nss, pam
[domain/$DOMAIN]
id_provider = ad
ad_domain = $DOMAIN
krb5_realm = $REALM
cache_credentials = True
use_fully_qualified_names = False
fallback_homedir = /home/%u
default_shell = /bin/bash
access_provider = simple
simple_allow_groups = $AD_GROUP
SSSD

chmod 600 "$SSSD_CONF"
systemctl enable --now sssd

# Ensure oddjobd for home directories
systemctl enable --now oddjobd.service
authselect select sssd with-mkhomedir --force

# Configure sudoers for AD group
echo "%$AD_GROUP ALL=(ALL) NOPASSWD: ALL" > "$SUDOERS_FILE"
chmod 440 "$SUDOERS_FILE"
visudo -cf "$SUDOERS_FILE"

# Permit AD group
realm permit -g "$AD_GROUP"
echo "===== AD Integration Completed on $(hostname) ====="
REMOTE_EOF
echo "===== Finished AD join on $TARGET_HOST. Log: $LOG_FILE ====="
) &
done

# Wait for all parallel jobs to finish
wait
unset AD_PASSWORD
echo "===== All AD joins completed ====="
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
How It Works
The script prompts for the AD administrator password once and never stores it.
Each host is joined to AD in parallel, logging progress to /tmp/ad_join_<hostname>.log.
DNS and /etc/hosts are updated safely.
Kerberos encryption is configured to meet AD standards.
SSSD is configured for AD authentication, with automatic home directory creation (oddjobd).
The specified AD group (unix_admin) is granted sudo privileges.

Usage Example
chmod +x ad_join_multi_hosts.sh
./ad_join_multi_hosts.sh host1 host2 host3
Check logs for each host in /tmp/ad_join_<hostname>.log.
After completion, all hosts will be AD-joined, with SSSD running and AD users able to log in with home directories and sudo privileges.

Benefits
Fast – parallel execution saves time.
Secure – password handled safely.
Reproducible – consistent configuration across all hosts.
Auditable – per-host logs help track exactly what happened.

This script is perfect for Linux administrators managing multiple AD-integrated servers in enterprise environments.

Automating Linux Active Directory Integration with a Shell Script

Integrating Linux servers with Microsoft Active Directory (AD) is a standard requirement in enterprise environments. While manual configuration works, it becomes inefficient and error-prone when managing multiple servers.
This post explains how to automate Linux–AD integration using a single Bash script, covering installation, configuration, domain joining, access control, and sudo permissions.

Why Use a Script?
Automating AD integration provides:
  • Consistent configuration across servers
  • Faster provisioning
  • Reduced human error
  • Easy reuse for new environments
  • Improved operational reliability
What the Script Accomplishes
The script performs the following:
  • Installs required AD integration packages
  • Configures DNS resolution
  • Updates /etc/hosts
  • Configures Kerberos encryption policies
  • Enables SSSD and automatic home directory creation
  • Joins the Linux system to Active Directory
  • Restricts login access to a specific AD group
  • Configures SSSD securely
  • Grants sudo access to an AD group
  • Restarts required services
Prerequisites
Before running the script, ensure:
  • You are logged in as root
  • The system can reach the AD Domain Controller
  • Correct DNS and domain information is available
  • The AD group unix_admin exists
  • You have AD administrator credentials
Complete AD Integration Script
Save the following script as ad_join.sh:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
#!/bin/bash
#  Author        : adminCtrlX
#  Description : Automates Linux Active Directory integration on server.
#                     - Installs required packages
#                     - Configures DNS, Kerberos, and SSSD
#                     - Joins the system to Active Directory
#                     - Restricts access to an AD group
#                     - Configures sudo using /etc/sudoers
#  Usage          : ./ad_join.sh

set -e

### VARIABLES ###
DOMAIN="ppc.com"
REALM="PPC.COM"
AD_SERVER_FQDN="inddcpads01.ppc.com"
AD_SERVER_IP1="192.168.10.100"
AD_SERVER_IP2="192.168.20.100"
AD_GROUP="unix_admin"
AD_ADMIN="administrator"
RESOLV_CONF="/etc/resolv.conf"
HOSTS_FILE="/etc/hosts"
SSSD_CONF="/etc/sssd/sssd.conf"
KRB_CRYPTO="/etc/krb5.conf.d/crypto-policies"

echo "===== Starting AD Integration Setup ====="

### STEP 1: Install Required Packages ###
echo "Installing required packages..."
dnf install -y realmd sssd oddjob oddjob-mkhomedir adcli samba-common-tools krb5-workstation

### STEP 2: Configure DNS ###
echo "Configuring DNS..."
cp $RESOLV_CONF ${RESOLV_CONF}.bak
cat <<EOF > $RESOLV_CONF
search $DOMAIN
nameserver $AD_SERVER_IP1
nameserver $AD_SERVER_IP2
EOF

### STEP 3: Update /etc/hosts ###
echo "Updating /etc/hosts..."
cp $HOSTS_FILE ${HOSTS_FILE}.bak
if ! grep -q "$AD_SERVER_FQDN" $HOSTS_FILE; then
echo "$AD_SERVER_IP1 $AD_SERVER_FQDN inddcpads01" >> $HOSTS_FILE
fi

### STEP 4: Configure Kerberos Encryption Types ###
echo "Configuring Kerberos crypto policies..."
cp $KRB_CRYPTO ${KRB_CRYPTO}.bak || true
cat <<EOF > $KRB_CRYPTO
[libdefaults]
permitted_enctypes = aes256-cts-hmac-sha1-96 aes256-cts-hmac-sha384-192 camellia256-cts-cmac aes128-cts-hmac-sha1-96 aes128-cts-hmac-sha256-128 camellia128-cts-cmac rc4-hmac
EOF

### STEP 5: Configure Authselect and Oddjob ###
echo "Configuring authselect and oddjob..."
mkdir -p /etc/authselect
authselect select sssd with-mkhomedir --force
systemctl enable --now oddjobd.service

### STEP 6: Join AD Domain ###
echo "Joining AD domain..."
realm join -v -U $AD_ADMIN $AD_SERVER_FQDN

### STEP 7: Verify Domain Join ###
echo "Verifying domain join..."
realm list

### STEP 8: Permit AD Group Login ###
echo "Permitting AD group access..."
realm permit -g $AD_GROUP

### STEP 9: Configure SSSD ###
echo "Configuring SSSD..."
cp $SSSD_CONF ${SSSD_CONF}.bak || true
cat <<EOF > $SSSD_CONF
[sssd]
domains = $DOMAIN
services = nss, pam
[domain/$DOMAIN]
ad_server = $AD_SERVER_FQDN
ad_domain = $DOMAIN
krb5_realm = $REALM
realmd_tags = manages-system joined-with-adcli
cache_credentials = True
id_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = True
use_fully_qualified_names = False
fallback_homedir = /home/%u
access_provider = simple
simple_allow_groups = $AD_GROUP
EOF

chmod 600 $SSSD_CONF

### STEP 10: Restart SSSD ###
echo "Restarting SSSD..."
systemctl restart sssd

### STEP 11: AD Group/User Creation ###
echo "NOTE: Ensure AD group '$AD_GROUP' exists and users are added on the AD server."

### STEP 12: Configure Sudo Access ###
echo "Configuring sudo access..."
if ! grep -q "%$AD_GROUP" /etc/sudoers; then
echo "%$AD_GROUP ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
fi

### STEP 13: Final Test Instructions ###
echo "===== Setup Complete ====="
echo "Test with:"
echo " ssh sysadm@$(hostname)"
echo " sudo su -"
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
How to Run the Script
chmod +x ad_join.sh
./ad_join.sh
You will be prompted for the Active Directory administrator password during the domain join process.

Testing the Configuration
Log in using an AD user:
ssh sysadm@hostname
Verify sudo/root access:
sudo su -
If successful, the system is fully integrated with Active Directory.

Best Practices & Improvements
  • Use /etc/sudoers.d/unix_admin instead of editing /etc/sudoers
  • Configure DNS via nmcli on NetworkManager systems
  • Use keytabs for non-interactive domain joins
  • Convert this script into an Ansible role
  • Add logging and rollback mechanisms
Conclusion
This script provides a clean, repeatable, and enterprise-ready solution for integrating Linux systems with Active Directory. It is ideal for system administrators, DevOps teams, and automated provisioning workflows.

Join a RHEL Linux Server to Active Directory Using SSSD

This guide explains how to join a Linux server to an Active Directory (AD) domain, configure authentication using SSSD, allow AD groups to log in, and grant sudo privileges to a specific AD group.

Prerequisites
  • RHEL Linux Servers 8/9/10
  • Proper DNS connectivity to the AD server
  • AD administrator credentials
  • Root or sudo access on the Linux server
Step 1: Install Required Packages
Install all necessary packages for AD integration.
# dnf install -y realmd sssd oddjob oddjob-mkhomedir adcli samba-common-tools krb5-workstation

Step 2: Configure DNS Resolution
Edit /etc/resolv.conf
# vi /etc/resolv.conf
Add the following entries:
search ppc.com
nameserver 192.168.10.100
nameserver 192.168.20.100
Save and exit:
Press Esc
Type :wq!
Press Enter

Step 3: Update /etc/hosts

# vi /etc/hosts
Add the AD server entry:
192.168.10.100 inddcpads01.ppc.com inddcpads01
Save and exit using :wq!.

Step 4: Configure Kerberos Encryption Types
Edit the crypto policy file:
# vi /etc/krb5.conf.d/crypto-policies
Add the following:
[libdefaults]
permitted_enctypes = aes256-cts-hmac-sha1-96 aes256-cts-hmac-sha384-192 camellia256-cts-cmac aes128-cts-hmac-sha1-96 aes128-cts-hmac-sha256-128 camellia128-cts-cmac rc4-hmac
Save and exit.

Step 5: Configure Authselect and Oddjob
If you see the message:
Directory [/etc/authselect] does not exist, please create it!
Run the following commands:
# mkdir /etc/authselect
# authselect select sssd with-mkhomedir --force
# systemctl enable --now oddjobd.service
This enables automatic home directory creation for AD users.

Step 6: Join the Linux Server to the AD Domain
Join the system to the domain using an AD administrator account:
# realm join -v -U administrator inddcpads01.ppc.com
Enter the AD Administrator password when prompted.

Step 7: Verify Domain Join Status
Check whether the system successfully joined the domain:
# realm list
Confirm that:
The domain is listed
permitted-groups includes the intended AD group

Step 8: Permit an AD Group to Log In
Allow the AD group unix_admin to access the Linux server:
# realm permit -g unix_admin

Step 9: Configure SSSD
Edit the SSSD configuration file:
# vi /etc/sssd/sssd.conf
Update or add the following configuration:
[domain/ppc.com]
ad_server = inddcpads01.ppc.com
ad_domain = ppc.com
krb5_realm = ppc.com
realmd_tags = manages-system joined-with-adcli
cache_credentials = True
id_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = True
use_fully_qualified_names = False
fallback_homedir = /home/%u
access_provider = simple
simple_allow_groups = unix_admin

Save and exit.
Important: Ensure correct file permissions:
# chmod 600 /etc/sssd/sssd.conf

Step 10: Restart SSSD Service
# systemctl restart sssd

Step 11: Create AD Group and Add Users (On AD Server)
Create the AD group: unix_admin
Add required AD users (e.g., sysadm) to this group
Allow time for AD replication if needed

Step 12: Configure Sudo Access for AD Group
Edit the sudoers file safely:
# visudo
Add the following line:
%unix_admin ALL=(ALL) NOPASSWD: ALL
This grants passwordless sudo access to all members of the unix_admin group.

Step 13: Test Login and Root Access

Log in using an AD user (example: sysadm)
Verify sudo access:
$ sudo su -
If successful, the user now has root privileges.

Conclusion
You have successfully:
  • Joined  Linux server to Active Directory
  • Configured SSSD authentication
  • Enabled automatic home directory creation
  • Restricted access to a specific AD group
  • Granted sudo/root privileges to AD users
This setup provides centralized authentication, improved security, and easier user management across Linux servers.

Grafana Enterprise Installation

This document provides step-by-step guidance for installing Grafana Enterprise on RHEL/CentOS, configuring NGINX as a reverse proxy with HTTPS (Let’s Encrypt), and enabling high availability (HA) for Grafana, Prometheus, Alertmanager, and the load balancer.

1. Install Grafana Enterprise
1.1 Install RPM Package
# yum install -y https://dl.grafana.com/grafana-enterprise/release/12.3.1/grafana-enterprise_12.3.1_20271043721_linux_amd64.rpm
# OR
# dnf install -y https://dl.grafana.com/grafana-enterprise/release/12.3.1/grafana-enterprise_12.3.1_20271043721_linux_amd64.rpm
1.2 Enable and Start Grafana
# systemctl enable --now grafana-server

2. Configure Grafana Server
2.1 Update Grafana Configuration
# vi /etc/grafana/grafana.ini
Edit [server] section:
[server]
http_addr = localhost
http_port = 3000
domain = www.grafana.ppc.com
Binding Grafana to localhost ensures external access only via NGINX.
2.2 Restart Grafana
# systemctl restart grafana-server

3. Install and Configure NGINX Reverse Proxy
3.1 Install NGINX
# dnf install nginx -y
3.2 NGINX Configuration for Grafana

Create /etc/nginx/conf.d/grafana.conf:

# Proxy WebSocket connections for Grafana Live
map $http_upgrade $connection_upgrade {
  default upgrade;
  '' close;
}
server {
    listen 80;
    server_name grafana.example.io;
    rewrite ^ https://$server_name$request_uri? permanent;
}
server {
  listen 443 ssl http2;
  server_name grafana.example.io;

  root /usr/share/nginx/html;
  index index.html index.htm;

  ssl_certificate /etc/letsencrypt/live/grafana.example.io/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/grafana.example.io/privkey.pem;

  access_log /var/log/nginx/grafana-access.log;
  error_log /var/log/nginx/grafana-error.log;

  location / {
    proxy_pass https://localhost:3000/;
  }

  location /api/live {
    rewrite ^/(.*) /$1 break;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header Host $http_host;
    proxy_pass https://localhost:3000/;
  }
}
3.3 Validate and Start NGINX
# nginx -t
# systemctl enable --now nginx
# systemctl status nginx

4. Access Grafana
Open browser at:
https://www.grafana.ppc.com
Default credentials:
Username: admin
Password: admin (change on first login)

5. SELinux Configuration for Grafana + NGINX
5.1 Verify SELinux Status
# getenforce
Expected: Enforcing
5.2 Allow NGINX Network Connections
# setsebool -P httpd_can_network_connect 1
5.3 WebSocket Support
WebSockets for Grafana Live are allowed via the same boolean above. No additional SELinux policies required.
5.4 Optional: Verify Grafana Port Context
# semanage port -l | grep 3000
# semanage port -a -t http_port_t -p tcp 3000  # if required
5.5 Check SELinux Denials
# ausearch -m AVC -ts recent
# journalctl -t setroubleshoot

6. Grafana High Availability (HA)
6.1 HA Requirements
Component Requirement
Database PostgreSQL or MySQL (not SQLite)
Sessions Shared DB
Storage Local disks
Load Balancer NGINX or HAProxy
Grafana Version Enterprise / OSS
6.2 Shared Database Configuration
[database]
type = postgres
host = dbserver.example.io:5432
name = grafana
user = grafana
password = strongpassword
ssl_mode = disable
Restart Grafana on each node:
# systemctl restart grafana-server
6.3 Node Configuration
[server]
http_addr = 0.0.0.0
http_port = 3000
domain = grafana.example.io

[unified_alerting]
enabled = true

Each node listens locally; external access only via the load balancer.

7. Load Balancer Configuration
Option A: NGINX

Create /etc/nginx/conf.d/grafana-ha.conf:

upstream grafana_backend {
    least_conn;
    server indrxgraf01:3000;
    server indrxgraf02:3000;
}

server {
    listen 443 ssl http2;
    server_name grafana.example.io;

    ssl_certificate /etc/letsencrypt/live/grafana.example.io/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/grafana.example.io/privkey.pem;

    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://grafana_backend;
    }

    location /api/live {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_pass http://grafana_backend;
    }
}
# nginx -t
# systemctl reload nginx
Option B: HAProxy (Alternative)
frontend grafana_https
    bind *:443 ssl crt /etc/haproxy/certs/grafana.pem
    default_backend grafana_nodes

backend grafana_nodes
    balance roundrobin
    server graf01 indrxgraf01:3000 check
    server graf02 indrxgraf02:3000 check

8. Keepalived Load Balancer HA
8.1 Install Keepalived
# dnf install keepalived -y
# echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
# sysctl -p
8.2 Configure MASTER Node

Edit /etc/keepalived/keepalived.conf:

vrrp_script chk_nginx {
    script "/usr/bin/pidof nginx"
    interval 2
    weight -20
}

vrrp_instance VI_GRAFANA {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 101
    advert_int 1
    authentication { auth_type PASS auth_pass StrongPass }
    virtual_ipaddress { 192.168.20.50/24 }
    track_script { chk_nginx }
}
8.3 Configure BACKUP Node
Same file, change:
state BACKUP
priority 100
8.4 Enable Keepalived
# systemctl enable --now keepalived
# systemctl status keepalived
# ip a | grep 192.168.20.50
8.5 SELinux & Firewall
# setsebool -P keepalived_connect_any 1
# firewall-cmd --add-service=keepalived --permanent
# firewall-cmd --reload

9. Prometheus High Availability
9.1 Install Prometheus
# dnf install prometheus -y
# systemctl enable --now prometheus
9.2 Prometheus Configuration
Edit /etc/prometheus/prometheus.yml:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'grafana'
    static_configs:
      - targets:
          - indrxgraf01:3000
          - indrxgraf02:3000
Restart Prometheus:
# systemctl restart prometheus

10. Alertmanager High Availability
10.1 Install Alertmanager
# dnf install alertmanager -y
# systemctl enable --now alertmanager
10.2 Configure Clustering
Node 1:
ALERTMANAGER_OPTS="--cluster.listen-address=0.0.0.0:9094 --cluster.peer=alert01.example.io:9094"

Node 2:
ALERTMANAGER_OPTS="--cluster.listen-address=0.0.0.0:9094 --cluster.peer=alert02.example.io:9094"
10.3 Prometheus Alertmanager Integration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
            - alert01.example.io:9093
            - alert02.example.io:9093
Restart Prometheus:
# systemctl restart prometheus

11. Grafana Integration
Add Prometheus HA endpoints as data sources
Add Alertmanager HA endpoints for alerting
Grafana handles failover automatically

12. Result
Secure Grafana Enterprise installation with HTTPS
NGINX handles SSL termination and WebSocket connections
Active-active HA for Grafana, Prometheus, Alertmanager, and load balancer
SELinux remains enforcing
No single point of failure

DB2 GPFS Cluster LUN Setup Using iSCSI and LVM on RHEL/CentOS

End-to-End High Availability Storage & Database Architecture

IBM DB2 high availability deployments require consistent shared storage, predictable fencing, and cluster-aware filesystems. When SAN infrastructure is unavailable or cost-prohibitive, iSCSI + GPFS (IBM Spectrum Scale) provides a fully supported, enterprise-grade alternative.

This guide covers the complete stack:
  • iSCSI shared LUNs (targetcli + LVM)
  • Multipath-secured access
  • GPFS cluster installation and quorum design
  • DB2 installation on GPFS
  • Performance tuning and HA validation
Reference Architecture
┌───────────────────────────────┐
│  iSCSI Target Server          │
│  RHEL/CentOS                  │
│  LVM VG: datavg               │
│  ├─ db2disk01 (3G)  → Data    │
│  ├─ db2disk02 (7G)  → Indexes │
│  ├─ db2disk03 (3G)  → Logs    │
│  └─ db2disk04 (7G)  → Temp    │
│  targetcli / LIO              │
└──────────────┬────────────────┘
               │ iSCSI (TCP 3260, MTU 9000)
 ┌─────────────┴─────────────┐
 │                           │
┌────────────────────┐   ┌────────────────────┐
│ DB2 Node 1         │   │ DB2 Node 2         │
indrxldb201        │   │ indrxldb202        │
│ GPFS Node          │   │ GPFS Node          │
│ DB2 Instance       │   │ DB2 Instance       │
└────────────────────┘   └────────────────────┘

Why GPFS for DB2?
RequirementGPFS Capability
Concurrent accessDistributed lock manager
HA fencingQuorum + tiebreaker disks
Write orderingJournaled metadata
Fast failoverSub-30s remount
IBM supportFully certified

ext4/XFS are NOT supported for DB2 HA shared-disk clusters

Environment Summary
ComponentValue
OSRHEL 8/9 or CentOS Stream
GPFSIBM Spectrum Scale 5.1+
DB211.5.x
Network10Gb iSCSI VLAN (MTU 9000)
Cluster Nodesindrxldb201, indrxldb202
QuorumTiebreaker disk required

Part 1 – GPFS Installation (Both Nodes)

1.1 Prerequisites
# dnf install -y \
  kernel-devel \
  gcc \
  cpp \
  elfutils-libelf-devel \
  numactl \
  chrony \
  net-tools
Ensure time synchronization:
# systemctl enable --now chronyd

1.2 Install IBM Spectrum Scale Packages
Upload Spectrum Scale RPMs to both nodes.
# rpm -ivh gpfs.base*.rpm gpfs.gpl*.rpm gpfs.msg*.rpm
Build kernel module:
/usr/lpp/mmfs/bin/mmbuildgpl
Verify:
# lsmod | grep mmfs
# echo "export PATH=$PATH:/usr/lpp/mmfs/bin" >> /root/.bash_profile

1.3 Create GPFS Cluster
Run once from primary node:
# mmcrcluster -N indrxldb201:manager-quorum,indrxldb202:manager-quorum -p indrxldb201 -s indrxldb202 -r /usr/bin/ssh -R /usr/bin/scp -C GPFS_DB2

# /usr/lpp/mmfs/bin/mmstartup -N indrxldb201
# /usr/lpp/mmfs/bin/mmstartup -N indrxldb201

Start GPFS:
# mmstartup -a
Verify:
# mmgetstate -a

1.4 Configure Quorum & Tiebreaker
For 2-node clusters, quorum is mandatory.
# mmadddisk tb -F /dev/mapper/mpatha
# mmchconfig tiebreakerDisks=mpatha
# mmchconfig quorum=4

Vote SourceCount
Node 11
Node 21
Data disk1
Tiebreaker1

Part 2 – GPFS Filesystem Design for DB2
FilesystemPurpose
gpfs_db2dataTablespaces
gpfs_db2indexIndexes
gpfs_db2logsTransaction logs
gpfs_db2tempTemp tables
Create File /tmp/nsdlist
# vi /tmp/nsdlist
%nsd:
device=/dev/sdb
nsd=nsd_01
servers=indrxldb201,indrxldb202
usage=dataAndMetadata 
failureGroup=-1 
pool=system 

Create NSDs
# mmcrnsd -F /tmp/nsdlist
Create Filesystems
# mmcrfs gpfs_db2data  -F /dev/mapper/mpathb -A yes -Q yes
# mmcrfs gpfs_db2index -F /dev/mapper/mpathc -A yes -Q yes
# mmcrfs gpfs_db2logs  -F /dev/mapper/mpathd -A yes -Q no
Mount:
# mmmount all -a

Part 3 – DB2 Installation (Both Nodes)

3.1 OS Kernel Tuning
# sysctl -w kernel.shmmni=8192
# sysctl -w kernel.shmmax=$(($(getconf _PHYS_PAGES) * 4096))
# sysctl -w kernel.shmall=$(($(getconf _PHYS_PAGES) * 4096 / 4096))
# sysctl -w vm.swappiness=1
Persist in /etc/sysctl.conf.

3.2 Create DB2 User & Groups
# groupadd -g 1001 db2iadm1
# groupadd -g 1002 db2fadm1
# useradd -u 1001 -g db2iadm1 -G db2fadm1 db2inst1
# passwd db2inst1

3.3 Install DB2 Software
# tar -xvf DB2_Svr_11.5_Linux_x86-64.tar.gz
# cd server_dec
# ./db2_install
Select:
Server Edition
Typical install

3.4 Create DB2 Instance
# /opt/ibm/db2/V11.5/instance/db2icrt \
  -u db2fadm1 db2inst1

Part 4 – DB2 Database on GPFS
Directory Structure
# mkdir -p /gpfs_db2data/db2inst1
# mkdir -p /gpfs_db2logs/db2inst1
# chown -R db2inst1:db2iadm1 /gpfs*

Create Database
# su - db2inst1
db2 create database PRODDB \
  on /gpfs_db2data \
  dbpath on /gpfs_db2data \
  using codeset UTF-8 territory us

Move logs:
# db2 update db cfg for PRODDB using NEWLOGPATH /gpfs_db2logs/db2inst1

Part 5 – High Availability Behavior
EventResult
Node crashGPFS remounts
Network splitQuorum prevents corruption
iSCSI path lossMultipath reroutes
Storage restartDB2 recovers logs

Typical DB2 recovery time: 20–45 seconds

Performance Tuning
GPFS
# mmchconfig pagepool=80%RAM
# mmchconfig maxFilesToCache=1000000
DB2
# db2 update db cfg using LOGFILSIZ 8192
# db2 update db cfg using LOGPRIMARY 20
# db2 update db cfg using LOGSECOND 100
iSCSI
# echo 256 > /sys/block/sdX/queue/nr_requests

Validation Checklist
  • iSCSI sessions persistent
  • Multipath active
  • GPFS quorum healthy
  • DB2 database starts on either node
  • Forced node failure recovers cleanly
  • No GPFS fencing events
Operational Commands Cheat Sheet
# mmgetstate -a
# mmlscluster
# mmlsdisk gpfs_db2data
# db2pd -db PRODDB -logs
# iscsiadm -m session -P 3
# multipath -ll

Final Thoughts
This architecture delivers:
  • IBM-supported DB2 HA
  • SAN-like behavior using software-defined storage
  • Strong fencing and split-brain prevention
  • Predictable performance at scale
The success factors are discipline and testing:
  • Quorum
  • Multipath
  • Dedicated network
  • Regular failover drills