IBM Power Systems is a family of high-end servers made by IBM. Think of it as a very powerful computer — but instead of running just one operating system like your laptop does, it can run hundreds of separate computers simultaneously inside one physical box. Each of those "virtual computers" is called an LPAR (we will explain this shortly).
Imagine a large apartment building (the IBM Power server). Each apartment (LPAR) has its own door, its own electricity meter, its own tenants (applications). The building manager (HMC) controls who gets how many rooms. The building's wiring and plumbing (VIOS) supplies electricity and water to each apartment. Tenants never see the wiring behind the walls — they just plug things in.
LPAR — Logical Partition
An LPAR is a virtual computer. It behaves exactly like a real physical server — it has its own CPU, RAM, storage, and network connection. It runs its own Linux operating system. From the perspective of Linux, it thinks it is a real server. It has no idea it's sharing the same physical box with 50 other LPARs.
An LPAR is like a virtual machine on your laptop (VirtualBox, VMware Workstation). You create a VM, give it 4 CPUs and 8 GB RAM, install Linux, and it runs as if it were a real computer. An LPAR is the same concept — just running on IBM's enterprise-grade hardware with far more performance, reliability, and scale.
| LPAR Setting | What it means in plain English | Example value |
|---|---|---|
| vCPU | How many virtual CPU cores this LPAR gets to use | 4 vCPU |
| Memory | How much RAM this LPAR gets | 16 GB |
| Shared vs Dedicated CPU | Shared = borrows CPUs from a pool (like a shared office desk). Dedicated = has its own CPUs always reserved (like your own private desk). | Shared capped |
| LPAR name | The hostname of this virtual server | db-prod-lpar01 |
| Operating system | What Linux version runs inside it | RHEL 9.4 ppc64le |
| vNIC | Virtual network card — how the LPAR connects to the network | eth0 |
| vFC | Virtual Fibre Channel — how the LPAR connects to storage | /dev/sda, /dev/sdb |
Hypervisor — The Invisible Referee
The hypervisor is a piece of software (actually firmware on IBM Power) that runs directly on the hardware — beneath everything else. Its job is to create and manage LPARs, share the physical CPU and memory fairly between them, and make sure they can never interfere with each other.
Imagine a sports referee at a football match. The referee does not play the game (it's not an operating system running apps). It just makes sure every player follows the rules — no cheating, no crossing into someone else's area. The hypervisor is that referee for LPARs.
| Type | Runs on | Example | IBM Power equivalent |
|---|---|---|---|
| Type-1 Hypervisor | Directly on bare metal hardware — no OS below it | VMware ESXi, Microsoft Hyper-V | PowerVM PHYP ✓ |
| Type-2 Hypervisor | On top of a host OS (Windows, macOS) | VirtualBox, VMware Workstation | Not used on Power servers |
HMC — Hardware Management Console
The HMC is the control panel for your IBM Power server. It is a separate computer (or virtual appliance) that connects to the Power server over the network. From the HMC you can create new LPARs, start and stop them, change how much CPU or memory they have, and move them to another server — all without touching the server physically.
Think of the HMC as the building management system in a large office tower. It controls the electricity to each floor (LPAR), allocates office space, and lets the building manager add a new floor or remove one — all from a central console. Tenants (applications) in each office never interact with it directly.
VIOS — Virtual I/O Server
VIOS is a special LPAR whose only job is to own the physical network cards and storage cables on the server, and then share them with all the other LPARs. A normal Linux LPAR never talks to a physical network card directly — it always goes through the VIOS.
VIOS is like a shared printer room in an office building. No individual office (LPAR) has its own printer plugged directly into the wall. Instead, there is one printer room (VIOS) with all the printers. Each office sends print jobs to the printer room, which then uses the real printers. If you need a second printer room for backup, you install VIOS 2.
veth, vNIC, and vFC — What Is the Difference?
When a Linux LPAR needs to talk to the network or to storage, it uses virtual adapters. These are not real hardware cards — they are software-defined connections that the hypervisor creates and hands to the LPAR. There are three types, and new joiners often confuse them.
The standard virtual network card every Linux LPAR gets. When you run ip addr inside an LPAR you see eth0 — that is a veth. It connects to the PowerVM virtual switch inside the server, then travels through the VIOS Shared Ethernet Adapter (SEA) to reach the real network cable. Used by the vast majority of LPARs.
A more advanced network adapter that bypasses the VIOS entirely and connects the LPAR almost directly to a physical NIC port via hardware partitioning. The result is much lower latency and higher throughput. Used for demanding workloads like OpenShift worker nodes or high-frequency trading. Most standard LPARs use veth — not vNIC SR-IOV.
The virtual storage adapter. While veth carries network packets, vFC carries storage commands to the SAN (disk array). Every LPAR that needs persistent storage gets a vFC adapter. Linux sees it as a SCSI disk — /dev/sda, /dev/sdb. The VIOS maps it to a physical HBA card via NPIV. Network and storage travel on completely separate paths.
| Adapter | What it carries | Goes through VIOS? | Linux kernel driver | What you see in Linux |
|---|---|---|---|---|
| veth | Network (Ethernet / IP) | Yes — via SEA | ibmveth | eth0, eth1 (ip addr) |
| vNIC (SR-IOV) | Network (Ethernet / IP) | No — hardware bypass | ibmvnic or vendor NIC driver | eth0 (higher throughput) |
| vFC | Storage (Fibre Channel / SCSI) | Yes — via NPIV HBA | ibmvfc | /dev/sda, /dev/sdb (lsblk) |
SAN and WWPN — Where Your Disk Actually Lives
A Linux LPAR running on IBM Power usually does not have a local hard drive inside the server. Its storage lives in a SAN — Storage Area Network. Think of it as a huge, fast, dedicated network whose only job is to carry disk data. The LPAR connects to the SAN through its virtual FC adapter and the VIOS.
Your home internet connection carries web pages, emails, and video calls all mixed together on one cable. A SAN is different — it is a dedicated private road that only carries storage traffic at very high speed. Nothing else is allowed on it. The protocol it uses is called Fibre Channel (FC), and it is much faster and more reliable than regular Ethernet for disk access.
A dedicated high-speed private network whose only purpose is carrying disk data between servers and storage arrays. It uses Fibre Channel cables and switches — separate from your regular Ethernet network. The LPAR's disk physically lives on a disk array connected to the SAN, not inside the server box.
A LUN is a chunk of storage that the SAN array "exports" to a specific LPAR. It is like a virtual hard drive. The storage admin creates a LUN of say 500 GB on the FlashSystem and maps it to your LPAR's WWPN. Inside Linux you then see it as /dev/sda. You then format it and use it like any other disk.
A unique address for a storage port — like a MAC address is to a network card, but for Fibre Channel. Every vFC adapter an LPAR gets is assigned a unique WWPN pair (one for each path). The storage admin uses this WWPN to say: "this LUN is allowed to be accessed by WWPN 50:05:07:68:XX:XX:XX:XX" — this is called zoning.
Zoning is the SAN's equivalent of a firewall rule. It tells the FC switch: "only let LPAR-1's WWPN talk to Storage-Port-A's WWPN — everyone else is blocked." Without zoning, any LPAR could try to access any storage — a serious security problem. Zoning is configured on the FC switch (Brocade or Cisco MDS) by the storage admin.
A clever trick that lets one physical HBA port in the VIOS pretend to be many separate ports — one per LPAR. Each LPAR gets its own virtual port with its own WWPN. The FC switch and the storage array see each LPAR as a separate device. NPIV is how 10 LPARs can share 2 physical HBA ports in the VIOS while each having its own isolated storage access.
The physical card plugged into the IBM Power server's PCIe slot that has the Fibre Channel connectors. It is to storage what a NIC is to networking. The VIOS owns the HBA and uses NPIV to give each LPAR a virtual port on it. A production server typically has 2–4 HBA ports, split across VIOS 1 and VIOS 2 for redundancy.
SEA — How Network Traffic Leaves the Server
When a Linux LPAR sends a network packet (say, connecting to a website or another server), it travels through its veth adapter → the PowerVM virtual switch → VIOS → SEA → physical NIC → network switch. The SEA (Shared Ethernet Adapter) is the bridge that VIOS builds between the internal virtual world and the external real network.
A bridge created inside VIOS that connects the internal PowerVM virtual switch to the external physical NIC. It carries all VLAN tags from the real network through to the LPARs. It also bonds two or more physical NICs together (LACP) to give more bandwidth and automatic failover if one NIC cable is unplugged.
A standard that lets you bundle two or more physical network cables together so they behave as one. If you have two 25 Gb cables bonded with LACP you get 50 Gb total bandwidth and if one cable fails the other keeps traffic flowing — automatically, with no manual intervention. LACP negotiation happens between the VIOS SEA and the ToR switch.
A software switch inside the IBM Power server that routes traffic between LPARs on the same physical box — without sending anything out of the server at all. It handles VLAN tagging so different LPARs can be on different networks. It does not connect to the outside world directly — that is the SEA's job.
A way to logically separate network traffic on the same physical cables. For example, LPAR 1 might be on VLAN 100 (production network) and LPAR 2 on VLAN 200 (test network) — but both share the same physical cables between server and switch. VLANs keep them isolated. The SEA passes all VLANs to the correct LPARs automatically.
RTAS — The Firmware's Phone Line to Linux
RTAS stands for Run-Time Abstraction Services. It is a set of services built directly into the IBM Power server's firmware. When Linux needs to do something that requires hardware-level privilege — like adding a new CPU while the server is running, recovering from a failed PCIe card, or preparing for a live migration — it makes an RTAS call. Think of it as Linux picking up a special hotline to the firmware.
Imagine you rent an office (LPAR) in a building. You cannot change the building's electrical wiring yourself — you call the building management team (RTAS) on a special internal phone line. You say "I need a new power socket in room 4" (add a CPU). The building team does the work behind the scenes, and when they are done, the new socket just appears in your office — no need to move out (reboot).
| RTAS Service | Plain English meaning | When it is triggered |
|---|---|---|
| ibm,configure-connector | Add or remove a CPU/memory/adapter to a running LPAR | When HMC or PowerVC does a DLPAR operation |
| ibm,read-slot-reset-state2 | Check if a PCIe card has faulted; reset it if needed | When a vFC or NIC adapter glitches (EEH recovery) |
| ibm,suspend-me | Ask all CPUs to pause so the LPAR can be moved live to another server | During Live Partition Mobility (LPM) |
| get-sensor-state | Read hardware temperature, voltage, and fan speed | Monitoring agents, IBM Electronic Service Agent |
| check-exception | Deliver hardware error events to the OS | When hardware detects a fault — for logging and alerting |
lscpu on a Power LPAR and see CPUs appearing or disappearing — that is RTAS at work. When your disk path disappears and comes back in 10 seconds without a reboot — that is RTAS EEH recovery. When a running LPAR silently moves to another server during maintenance — that is RTAS suspend/resume. You will not configure RTAS directly; the system uses it behind the scenes, but understanding it helps you diagnose unexpected resource changes.LPM — Moving a Running LPAR With No Downtime
LPM stands for Live Partition Mobility. It lets a system administrator take a running Linux LPAR — with an active application, open network connections, and live disk I/O — and move it completely to a different physical Power server, without stopping it. Your application does not even notice.
Imagine you are on a phone call while riding on a bus. The bus reaches a stop and you need to get off and onto a different bus — but the phone call must not be interrupted. A crew of people pick you up and carry you, mid-conversation, from one bus to the other in under a second. That is LPM. The "phone call" is your running application. The buses are the two Power servers.
IBM Power Server Models — Scale-Out vs Scale-Up
IBM sells Power servers in two broad categories. Scale-out servers (S-series) are like a fleet of delivery vans — many medium-sized units, great for consolidating lots of workloads. Scale-up servers (E-series) are like a single enormous articulated lorry — one giant machine with massive memory for mission-critical databases.
Glossary — Every Term on One Page
A cheat-sheet for everything covered in this guide. Bookmark this page.
| Term | Full name | One-line plain English definition |
|---|---|---|
| LPAR | Logical Partition | A virtual computer running inside a physical IBM Power server. Has its own CPU, RAM, network, and storage. |
| HMC | Hardware Management Console | The remote control panel for managing LPARs, starting/stopping them, and allocating resources on the Power server. |
| PowerVM | Power Virtual Machine (Hypervisor) | IBM's hypervisor firmware that creates and isolates LPARs on the bare metal — like a referee between hardware and virtual computers. |
| PHYP | Power Hypervisor | The technical name for the PowerVM firmware layer. Type-1 — runs directly on hardware with no OS beneath it. |
| VIOS | Virtual I/O Server | A special service LPAR that owns the real NIC and HBA cards and shares them with all other LPARs on the server. |
| veth / ibmveth | Virtual Ethernet Adapter | The virtual network card an LPAR uses for IP/Ethernet traffic. Seen in Linux as eth0. Travels through VIOS SEA. |
| vNIC (SR-IOV) | Virtual Network Interface Card | A high-performance virtual network card that bypasses VIOS and connects almost directly to a physical NIC port. |
| vFC / ibmvfc | Virtual Fibre Channel Adapter | The virtual storage adapter an LPAR uses to access SAN disk. Seen in Linux as /dev/sda. Goes through VIOS NPIV. |
| SEA | Shared Ethernet Adapter | A bridge created inside VIOS that connects the internal PowerVM vSwitch to the external physical NIC, with LACP bonding. |
| LACP | Link Aggregation Control Protocol | Bonds two or more physical NIC cables together for more speed and automatic failover if a cable is unplugged. |
| SAN | Storage Area Network | A dedicated high-speed private network used only for disk storage traffic between servers and storage arrays (Fibre Channel). |
| LUN | Logical Unit Number | A chunk of storage (like a virtual hard drive) carved out of the SAN array and given to a specific LPAR. |
| HBA | Host Bus Adapter | The physical PCIe card in the server with Fibre Channel connectors. VIOS owns it and shares it via NPIV. |
| WWPN | World Wide Port Name | A unique address for a Fibre Channel port — like a MAC address but for storage. Used for zoning on the FC switch. |
| NPIV | N_Port ID Virtualisation | Lets one physical HBA port pretend to be many — giving each LPAR its own virtual FC port and WWPN. |
| Zoning | FC Switch Access Control | A firewall rule on the FC switch that allows only specific LPARs (by WWPN) to talk to specific storage ports. |
| dm-multipath | Device Mapper Multipath | Linux software that manages two or more separate paths to the same disk — if one fails, the other takes over silently. |
| LVM2 | Logical Volume Manager 2 | Linux disk management software that turns raw LUNs into flexible, resizable disk partitions (PV → VG → LV → filesystem). |
| DLPAR | Dynamic Logical Partitioning | Adding or removing CPU, RAM, or adapters from a running LPAR without rebooting it. Uses RTAS under the hood. |
| RTAS | Run-Time Abstraction Services | A set of firmware services Linux calls to request hardware changes (DLPAR, EEH recovery, LPM suspend) without direct hardware access. |
| EEH | Enhanced Error Handling | RTAS-powered ability to recover from a failed PCIe card (NIC or HBA) without rebooting the LPAR or server. |
| LPM | Live Partition Mobility | Moving a running LPAR from one Power server to another with no downtime. Requires shared SAN storage. |
| PowerVC | Power Virtualisation Centre | IBM's OpenStack-based self-service portal and API for deploying and managing LPARs — like a private cloud for Power servers. |
| ppc64le | Power PC 64-bit Little Endian | The CPU architecture of IBM Power servers. When you download RHEL or SLES for Power, you choose the ppc64le build. |
| CoD | Capacity on Demand | Pre-installed but locked CPU/memory that can be activated instantly by buying a licence — no engineer visit needed. |
| SMT | Simultaneous Multithreading | Each physical CPU core on Power can run 8 threads simultaneously (SMT8). One core = 8 "virtual CPUs" visible to Linux. |
Putting It All Together — A Day in the Life of a Linux LPAR
Let us trace what happens from the moment your team's Linux LPAR is created to a user making a database query — using every term from this guide.
- 1Admin opens the HMCThe admin logs into the Hardware Management Console and creates a new LPAR. They name it
db-prod-01, give it 8 vCPU and 64 GB RAM from the shared processor pool, and assign two vFC adapters (one per VIOS) and two veth adapters for network. - 2PowerVM PHYP creates the partitionThe hypervisor firmware carves out the requested CPU entitlement and memory from the physical server's resources and registers a new isolated partition. The LPAR does not exist to any other LPAR — they cannot see it.
- 3VIOS maps the vFC adapters via NPIVEach vFC adapter gets a unique WWPN pair. VIOS 1 maps the first vFC to a virtual port on its physical HBA (NPIV). VIOS 2 maps the second vFC to its own separate HBA. The storage admin zones both WWPNs to the LUN on the FlashSystem.
- 4RHEL boots via RTAS device treeThe LPAR boots from network (PXE) or a captured image. The Linux kernel reads the RTAS device tree and discovers: 8 vCPUs, 64 GB RAM, 2 × vFC adapters (ibmvfc driver), 2 × veth adapters (ibmveth driver). It initialises them all automatically.
- 5dm-multipath discovers two paths to the LUNLinux sees
/dev/sda(path via VIOS 1 HBA) and/dev/sdb(path via VIOS 2 HBA) — both pointing to the same SAN LUN. dm-multipath combines them into/dev/mapper/mpathaand uses both paths simultaneously (round-robin I/O). - 6LVM2 formats the storage and mounts the filesystemThe admin runs
pvcreate /dev/mapper/mpatha,vgcreate vg_db /dev/mapper/mpatha,lvcreate -L 400G -n lv_data vg_db, then formats it as XFS and mounts it at/oracle. The database is installed there. - 7Network goes live through SEAThe veth adapter
eth0is configured with an IP address on VLAN 100. Traffic flows: LPAR eth0 → PowerVM vSwitch → VIOS SEA (802.3ad LACP bond across two 25 GbE NICs) → ToR switch → enterprise network. The LPAR gets a hostname in DNS and is reachable. - 8Six months later — live migration with zero downtimeThe physical server needs a firmware update. The admin triggers LPM from the HMC. The running LPAR is moved to a spare Power server in 90 seconds. The database query that was running during the migration completes normally. Users notice nothing. Afterwards, the firmware is updated and the LPAR migrated back.
No comments:
Post a Comment