Friday, September 8, 2023

Multipath setup in Linux

A multipath setup in Linux is a configuration that allows multiple physical paths (usually represented by multiple physical storage devices or network connections) to be used to access a single logical device or storage target. The primary goal of multipath is to enhance redundancy and fault tolerance while providing load balancing and improved performance. Multipath is commonly used in storage area networks (SANs) and environments where high availability and reliability are critical.

Here are the key components and concepts of a multipath setup in Linux:

Multipath Devices (Multipath): In a multipath setup, there is a logical device known as a multipath device (often referred to as a multipath or mpath). This logical device represents a single storage target, such as a disk or LUN (Logical Unit Number), even though it is accessible through multiple physical paths.

Physical Paths: Physical paths are the actual connections or channels through which the storage target is accessible. These paths can be physical SCSI buses, Fibre Channel links, iSCSI connections, or any other transport mechanism. Each path is associated with a unique identifier, typically called a World Wide Name (WWN), device name, or other similar identifiers.

Path Management: The multipath software in Linux (such as multipathd and multipath-tools) manages the physical paths and ensures that they are utilized effectively. It monitors the status of the paths and makes decisions about which path to use for I/O operations. It can also detect and respond to path failures or changes in path availability.

Load Balancing: Multipath configurations often include load balancing mechanisms that distribute I/O requests across the available paths. This helps improve performance by distributing the workload and preventing one path from becoming a bottleneck.

Redundancy and Failover: Multipath setups provide redundancy and failover capabilities. If one path fails due to hardware or network issues, the system can automatically switch to an alternate path without interrupting I/O operations. This enhances system reliability and availability.

Device Mapper (DM-Multipath): In Linux, the Device Mapper subsystem is commonly used to manage multipath devices. DM-Multipath is a kernel component that works with the multipath software to create and manage multipath devices. It presents a single device to the operating system, which is actually a combination of the multiple physical paths.

Configuration Files: To set up multipath in Linux, administrators configure multipath settings using configuration files. The main configuration file is typically located at /etc/multipath.conf (or a similar location) and defines the behavior of the multipath devices.

Multipath Tools: The multipath tools package (multipath-tools or similar) includes utilities such as multipath and multipathd that are used to manage and configure multipath devices. These tools help monitor path status, configure load balancing policies, and perform other administrative tasks related to multipathing.

For Example : This system is using multipath and LVM for storage management to provide redundancy and flexibility in managing storage devices. Both sda and sdb are part of the multipath configuration, and their partitions are managed using LVM. This setup is commonly used in enterprise environments for high availability and fault tolerance

There are three partitions on the multipath device mpatha (which represents both sda and sdb due to multipathing). 

The lsblk command is used to list block devices on this system, displaying information about disks, partitions, and their relationships. Let's break down the lsblk output:

NAME                            MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda                               8:0    0   80G  0 disk
└─mpatha                        253:0    0   80G  0 mpath
  ├─mpatha1                     253:1    0    4M  0 part
  ├─mpatha2                     253:2    0    1G  0 part  /boot
  └─mpatha3                     253:3    0   79G  0 part
    ├─rhel_myhost-root 253:4    0 47.7G  0 lvm   /
    ├─rhel_myhost-swap 253:5    0    8G  0 lvm   [SWAP]
    └─rhel_myhost-home 253:6    0 23.3G  0 lvm   /home
sdb                               8:16   0   80G  0 disk
└─mpatha                        253:0    0   80G  0 mpath
  ├─mpatha1                     253:1    0    4M  0 part
  ├─mpatha2                     253:2    0    1G  0 part  /boot
  └─mpatha3                     253:3    0   79G  0 part
    ├─rhel_myhost-root 253:4    0 47.7G  0 lvm   /
    ├─rhel_myhost-swap 253:5    0    8G  0 lvm   [SWAP]
    └─rhel_myhost-home 253:6    0 23.3G  0 lvm   /home

Here's a breakdown of the information in each column:

NAME: This column shows the name of the block device.
MAJ:MIN: Major and minor device numbers that uniquely identify the device to the operating system.
RM: This indicates whether the device is removable (1 for yes, 0 for no).
SIZE: The size of the device, often in gigabytes (G) or megabytes (M).
RO: This indicates whether the device is read-only (1 for yes, 0 for no).
TYPE: The type of device, which can be "disk" for physical disks or "part" for partitions. In this case, you also see "mpath" and "lvm," which are related to storage management.
MOUNTPOINT: The mount point where the device is currently mounted. If it's not mounted, this field will be empty.

Now, let's interpret the information based on the provided output:

There are two disk devices: sda and sdb.
Both sda and sdb are part of a multipath configuration, as indicated by the "mpath" type.
Each disk (sda and sdb) has three partitions (mpatha1, mpatha2, and mpatha3), and each of these partitions is used in an LVM (Logical Volume Management) setup.
The /boot partition (mpatha2) is mounted on both sda and sdb, and it contains the boot files.
The root (/) partition (rhel_myhost-root) is mounted on both sda and sdb, and it is the root filesystem.
The swap partition (rhel_myhost-swap) is also mounted on both sda and sdb and is used for swap space.
The /home partition (rhel_myhost-home) is mounted on both sda and sdb and is used for user home directories.

Here's what each of these partitions is typically used for:

mpatha1: This partition appears to be very small (only 4MB), and it is often used for storing bootloader-related files. Specifically, it might contain the GRUB bootloader's core files or other boot-related data. It's a common practice to allocate a small partition for bootloader files to ensure that they are easily accessible and less likely to be affected by changes or issues in the rest of the filesystem. A small partition like this is often sufficient for storing the essential boot files.

mpatha2: This partition is mounted as /boot, and it contains the kernel and initial ramdisk files needed for booting the system. /boot typically holds the Linux kernel, GRUB configuration files, and other boot-related data. Having a separate /boot partition is a common practice, especially in systems that use LVM or other complex storage configurations. It ensures that essential boot files are easily accessible and are less prone to issues that might affect other partitions.

mpatha3: This partition appears to be the largest and is not directly mounted as part of the root filesystem (/). Instead, it seems to be part of an LVM (Logical Volume Management) setup. It is divided into multiple logical volumes (rhel_myhost-root, rhel_myhost-swap, and rhel_myhost-home) that are used for various purposes:

rhel_myhost-root: This is the root filesystem (/) where most of the operating system and software are installed.

rhel_myhost-swap: This logical volume is used as swap space, which is used for virtual memory and can help improve system performance.

rhel_myhost-home: This logical volume is typically used for user home directories. User data and files are stored in the /home directory, which is often mounted on a separate filesystem to isolate user data from the root filesystem.

So, mpatha1 is likely used for bootloader files, mpatha2 is the /boot partition containing boot-related files, and mpatha3 represents an LVM setup with separate logical volumes for the root filesystem, swap space, and user home directories. This kind of partitioning and storage management allows for flexibility, scalability, and better system maintenance

You can map the UUID specified in the /etc/fstab file to the corresponding device name, such as /dev/sda1, /dev/sdb1, or /dev/mpatha1. The UUID (Universally Unique Identifier) is a unique identifier assigned to each filesystem or partition and is a more reliable way to identify devices than device names, which can change if hardware configurations are altered.

To map a UUID to the corresponding device name, you can use the blkid command. 

[root@myhost ~]# blkid
/dev/mapper/rhel_myhost-root: UUID="XXXXXXXXXXXXXXX" BLOCK_SIZE="512" TYPE="xfs"
/dev/mapper/mpatha3: UUID="XXXXXXXXX" TYPE="LVM2_member" PARTUUID="XXXXXXXX"
/dev/sda: PTUUID="XXXXXXXX" PTTYPE="dos"
/dev/mapper/mpatha: PTUUID="abc" PTTYPE="dos"
/dev/sdb: PTUUID="XXXXXXX" PTTYPE="dos"
/dev/mapper/mpatha1: PARTUUID="abc-01"
/dev/mapper/rhel_myhost-swap: UUID="abc123c" TYPE="swap"
/dev/mapper/rhel_myhost-home: UUID="xyz123" BLOCK_SIZE="512" TYPE="xfs"
[root@myhost ~]#

[root@myhost ~]# cat /boot/grub2/
# this device map was generated by anaconda
(hd0)      /dev/mapper/mpatha
[root@myhost ~]#


[root@myhost ~]# cat /etc/fstab

# /etc/fstab
/dev/mapper/rhel_myhost-root /                       xfs     defaults        0 0
UUID=XXXXXXXXXXXXXXXXXXXXXXX /boot                   xfs     defaults        0 0
/dev/mapper/rhel_myhost-home /home                   xfs     defaults        0 0
/dev/mapper/rhel_myhost-swap none                    swap    defaults        0 0
[root@myhost ~]#

A multipath setup in Linux provides redundancy, load balancing, and fault tolerance for storage devices, ensuring that data remains accessible even if one path or connection fails. This technology is crucial in enterprise environments where continuous access to data is essential for operations.


The grub2-install command is a utility used in Linux to install the GRUB (Grand Unified Bootloader) bootloader onto a device, typically a hard disk or a partition.

The primary purpose of the grub2-install command is to install the GRUB bootloader on a specific device. You specify the target device as an argument to the command. 

For example :  grub2-install /dev/sda

In this example, the GRUB bootloader is installed on the MBR (Master Boot Record) of /dev/sda, which is typically the primary boot device.

Boot Device Configuration:
When GRUB is installed on a device, it configures the bootloader to locate and load the kernel and initial ramdisk (initrd) from the designated boot device or partition. It also stores configuration information, such as the location of the kernel and the root filesystem.

Device Map Configuration:
GRUB maintains a device map that associates BIOS drive numbers (e.g., (hd0), (hd1)) with actual device names (e.g., /dev/sda, /dev/sdb). The grub2-install command updates or creates this device map, ensuring that GRUB can correctly identify the boot device.

Bootloader Configuration File:
GRUB bootloader configurations are specified in the /boot/grub2/grub.cfg (or similar) file. This configuration file is automatically generated by GRUB utilities and scripts based on the system's configuration, such as the kernel and initrd locations, boot options, and menu entries.

Boot Menu:
GRUB provides a boot menu during system startup, allowing users to select from available kernels and boot options. The grub2-install command ensures that the necessary components for this boot menu are installed and configured correctly.

Updating GRUB Configuration:
In addition to installing GRUB, the grub2-install command also updates the bootloader's configuration to reflect changes in the system's disk layout or partitioning scheme. This includes updating device names and paths if necessary.

EFI and UEFI Support:
The behavior of grub2-install can differ depending on whether the system uses BIOS or UEFI (Unified Extensible Firmware Interface) for booting. For UEFI systems, the grub2-install command installs the UEFI version of GRUB and configures it accordingly.

Additional Options:
The grub2-install command supports various options to specify installation details, such as the target architecture, firmware type (BIOS or UEFI), and more. You can use the --target, --boot-directory, and other options to customize the installation.


The PReP boot partition is a specialized partition used on PowerPC systems that follow the PReP boot standard to store firmware-specific bootloader and boot-related files. On the other hand, the /boot partition is a common convention on Linux systems, including PowerPC systems, to store kernel, initramfs, and bootloader configuration files, but it is not tied to any specific firmware standard and is used across various hardware architectures.

PReP Boot Partition:
The PReP boot partition is a specific partition type used in the context of the PReP boot standard, which is a firmware standard for booting PowerPC-based systems.
Its primary purpose is to store the bootloader and boot-related information required to initiate the boot process on PowerPC systems adhering to the PReP standard.
It typically contains essential firmware boot files, such as Open Firmware or IEEE 1275-compliant firmware, which are necessary to start the system.

/boot Partition:
The /boot partition is a common convention used on various Linux distributions, including those running on PowerPC systems.
Its purpose is to store the kernel, initramfs (initial RAM disk), bootloader configuration files, and other files required for the early stages of the boot process.
The /boot partition is part of the Linux filesystem structure and is used by the Linux bootloader (e.g., GRUB) to locate and load the kernel and initramfs during the boot process.
Firmware Dependency:

PReP Boot Partition:
The PReP boot partition's usage is closely tied to the firmware standard it follows, such as Open Firmware or IEEE 1275-compliant firmware. The firmware is responsible for loading the bootloader from this partition.
It may also contain firmware-specific files and configurations.

/boot Partition:
The /boot partition is not firmware-dependent and is part of the Linux filesystem. It is managed by the Linux bootloader (e.g., GRUB) and the operating system itself.
The bootloader reads the kernel and initramfs from the /boot partition during the boot process, and this partition is independent of the system's firmware.
Common Usage:

PReP Boot Partition:
Commonly used on older PowerPC-based systems that adhere to the PReP standard.
It's specific to the boot process defined by the firmware standard used on these systems

/boot Partition:
Widely used on various Linux distributions, including those on PowerPC systems.
It's part of the standard Linux filesystem structure and is used on many different hardware platforms.

No comments:

Post a Comment