Mainframe Virtualization

The CMS strategy for Mainframe Consolidation and Virtualization requires a highly centralized and highly consolidated environment that employs server virtualization technology. This strategy will achieve a large reduction in infrastructure and support costs by improving resource utilization and consolidating management functions. The mainframe virtualization figure below provides an overview of the basic components that comprise mainframe server virtualization solutions.

Figure Mainframe Virtualization Figure depicts IBM zOS Mainframe Virtualization. The figure shows the logical relationship between the zOS operating system, LPARs, zLinux and z/VM instances, memory and processor pools, and the I/O subsystem and peripherals. The mainframe is depicted as a large rectangle containing all of the cited components with the exception of the peripherals. The zOS operating system is depicted at the reader's far left by a vertical rectangle containing the RACF, APF, and LDAP functionality. The remainder of the Mainframe rectangle is subdivided into three vertical rectangles representing three LPARs (LPAR 1, 2, and 3). At the top of each LPAR, four zLinux instances are depicted as vertical bars resting on a horizontal rectangle representing the z/VM instance in each LPAR. Under the z/VM instances is a larger horizontal rectangle that overlaps the zOS rectangle and spans the three LPARs within the Mainframe. This larger horizontal rectangle represents the PR/SM LPAR hypervisor. Within the PR/SM rectangle are three pairs of memory and processor pools represented as squares, and each pair is associated with one of the three LPARs. Below the PR/SM is a narrower horizontal rectangle that represents the I/O subsystem. This horizontal rectangle also cross cuts the three LPARs and the zOS rectangle. A series of boxes appear from the reader's left to the right within the I/O subsystem. These boxes, labeled sequentially with hexadecimal CHIPIDs, represent the Channel Paths. These boxes are connected by lines to peripherals that appear below and outside the Mainframe, such as a DASD Controller, a GigE switch blade, and a FICON switch connected to a SAN controller.
Mainframe Virtualization

Business Rules for Mainframe Virtualization

CMS derived the following mainframe virtualization (MV)-specific BRs from various sources, including the CMS ARS and CMS TRA – Foundation. Not all of the Server Virtualization Business Rules apply to the mainframe environment; therefore, only those Business Virtualization rules that do apply appear below.

BR-MV-1: z/VM Sole Configuration

z/VM must be the sole configuration for all logical partitions (hereafter referred to as an LPAR-z/VM).

BR-MV-2: z/Linux as Sole Guest for z/VM

z/Linux (hereafter referred to as a Virtual Machine) must be the sole guest operating system for all z/VMs.

BR-MV-3: No Nested z/VM Instances

Nested z/VM instances must not run on z/VMs.

BR-MV-4: Multiple Zones for a Single Application

Applications are not permitted to run n-zones (web front end / business logic / database) virtualized on a single LPAR-z/VM Production environment.

Mainframe Virtualization-Specific Terminology

The Mainframe Virtualization-Specific Terms table defines the following terms that are specific to virtualization of Mainframe server implementations hosting CMS applications.

Table - Mainframe Virtualization-Specific Terms

Term

Definition

Authorized Program Facility (APF)

Selectively permits or denies program access to sensitive system functions.

Channel

A communication path from the mainframe to an external device such as disk or tape. I/O devices are attached to the channel subsystem through the control unit.

tChannel Path

The connection between the channel subsystem and an I/O control unit.

Channels Path Identifiers (CHPID)

Uniquely identify channel paths and can be assigned to one or more LPARs.

Control Blocks

A data structure that serves as a vehicle for communication in z/OS.

Direct Access Storage Device (DASD)

A storage device (e.g., a disk drive) that is directly connected to and controlled by the Mainframe. Disk I/O functions are controlled by the Mainframe operating system.

Logical Partition (LPAR)

A set of processor, memory and I/O interface pools on a mainframe that acts as a virtual hardware platform for operating systems or VMs.

LPAR Hypervisor

A native hypervisor that runs directly on the mainframe hardware as part of the Processor Resource/Systems Manager.

Multi-Level Security (MLS)

A security policy that allows the classification of data and users on a system of hierarchical security levels combined with a system of non-hierarchical security categories.

Processor Resource / Systems Manager (PR/SM)

The feature that allows the processor to use several z/OS images simultaneously and provides logical partitioning capability.

Resource Access Control Facility (RACF)

An IBM Security Center application that protects resources by granting access only to authorized users of the protected resources.

Security Category (SC)

A non-hierarchical category that corresponds to some grouping within an organization that has similar security access authorization within a security level.

Security Label

A combination of a hierarchical level of classification (security level) and a set of zero or more non-hierarchical categories (security category).

Security Level

A positive integer that defines the hierarchical degree of sensitivity of the data. Higher numbers correspond to higher sensitivity. z/OS System 10 supports 256 levels.

Storage Area Network

A Storage Area Network is a network specifically dedicated to the task of transporting data for storage and retrieval. The SAN provides an architecture to attach remote computer storage devices to servers such that the devices appear as locally attached to the operating system. The remote system accesses storage as though it were a local disk.

z/VM

A software hypervisor that runs as an OS instance in an LPAR.

zLinux

Linux operating system compiled to run on zOS LPARs or as VMs in a z/VM, simply denoted VM.

Mainframe Server Implementation and Security

The following describes the design and implementation rules for consistency with the CMS TRA.

Mainframe Virtualization Implementation Rules

The general rules apply to all environments and zones within an environment. In addition, rules for each environment and each zone within an environment are provided when required.

General Mainframe Virtualization Implementation Rules

The following implementation rules apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in all environments and zones:

  • All management access to VMs (e.g., administrator login), must be limited to a single physical management interface or management interface pool that is not accessible by user processes in a VM.

  • All network management access to z/VM instances must be from the Management Zone via a CMS-approved secure access method.

  • All network management access to the LPAR hypervisor must be from the Management Zone via a CMS-approved secure access method, e.g., via VPN into attached consoles.

  • Management accounts for LPAR-z/VM management access must be different from those used to access VMs.

  • Multiple environments must not share LPAR-z/VMs.

  • Management interfaces on LPARs must be configured to be in a unique LPAR Management VLAN accessible only from the Management Zone .

  • Management interfaces for z/VM instances must be configured to be in a unique z/VM Management VLAN accessible only from the Management Zone .

  • Management interfaces serving the Security segment must be physically separate from those serving all other Management segments (e.g., LPAR-z/VM management segments).

Non-ATO(ed) Environment Mainframe Virtualization Implementation Rules

The following implementation requirements apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in Non-ATO(ed) environments, i.e., Test, Development, and Implementation environments, and their zones. The general rules apply to all zones within the environments. In addition, rules for each zone within the environment are provided when required.

  • When multiple development projects share the same LPAR-z/VM, each project must reside in a different VM.

Data Zone Rules

The following implementation rules apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in non-ATO(ed) environments and their Data Zones:

  • Non-Production Data Zone servers must reside in unique VMs.

  • Non-Production Data Zone VMs must use VLAN virtual network interfaces mapped to individual or pooled NICs.

Application and Presentation Zone Rules

The following implementation rules apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in non-ATO(ed) environments and their Application and Presentation Zones:

  • Non-Production Application and Presentation Zone servers may share VMs.

  • Non-Production Application and Presentation Zone VMs may use shared VLAN virtual network interfaces.

ATO(ed) Environment Mainframe Virtualization Implementation Rules

The following implementation rules apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in ATO(ed) environments. The general rules apply to all zones within the environment. In addition, rules for each zone within the environment are provided when required.

  • Production LPAR-z/VMs must not be shared between zones.

  • Each Production application server must reside in its own dedicated VM.

  • All Production VMs must use VLAN-dedicated physical or pooled network interfaces.

  • Management interfaces serving the Security segment must be physically separate from those serving all other Management segments, e.g., Backup segments.

Mainframe Virtualization Security Rules

CMS derived the following Mainframe-specific security rules from security requirements presented in CMS TRA Network Services and the CMS ARS. The general rules apply to all environments and zones within an environment. In addition, rules for each environment and each zone within an environment are provided when required.

General Mainframe Virtualization Security Rules

The following security rules apply to LPAR- z/VMs and VMs running on LPAR-z/VMs in all environments and zones:

  • Management interfaces must use dedicated interfaces or interface pools with VLAN separation.

  • Management traffic must only use management interfaces and VLANs.

  • Management traffic for VMs on the same LPAR-z/VM may share the same VLAN.

  • Management traffic for LPAR-z/VMs must use a separate, dedicated VLAN, distinct from that used for VMs.

  • Security traffic must use a separate physical interface from other management functions, e.g., backup.

  • All management traffic must be encrypted and must only use SSH version 2 or Transport Layer Security (TLS), never telnet. Authentication must be provided by a Role-based Security (RBS) with authentication, authorization, and accounting (AAA) functions provided by a common AAA server. Encryption must be accomplished using Federal Information Processing Standards (FIPS) 140-2 certified modules (please refer to CMS ARS Security Control SC-13).

  • All management traffic must originate or terminate in the Management Zone .

  • VMs must have individual access rules and roles for authentication through an RBS.

  • All accesses and configurations must be in accordance with the CMS ARS.

  • Management accounts with administrative privileges in VMs must not have access privileges to LPAR-z/VMs.

  • All VMs and LPAR-z/VMs must be assigned unique IP addresses.

  • All VMs on a single LPAR-z/VM must be assigned IP addresses from a subnet unique to the LPAR-z/VM.

Non-ATO(ed) Environment Mainframe Virtualization Security Rules

The following security rules apply to LPAR-z/VMs instances and VMs running on LPAR-z/VMs in Non-ATO(ed) environments and their zones. The general rules apply to all zones within an environment. In addition, rules for each zone within an environment are provided when required.

  • Information associated with different projects must use project- and user-unique keys for all information protected by cryptographic hashes, rather than those inherited from VMs on which the business applications run.

  • Access to file systems associated with a project must be controlled through project-unique security categories to prevent direct access by members of other projects. These security categories must map to UNIX Group Identifiers (GID), with the User Identifiers (UID) of group members associated with the unique GID.

Data Zone Rules

The following security rules apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in non-ATO(ed) environments and their Data Zones:

  • Non-Production Data Zone application servers must use separate VMs.

  • When Non-Production Data Zone VMs reside on the same LPAR-z/VM, RBS AAA functions must be provided by a common AAA server to ensure that management and user roles are unique for each VM.

  • Non-Production Data Zone VMs on the same LPAR-z/VM may route internally between each other on their z/VM instance’s subnet.

  • Non-Production Data Zone LPAR-z/VMs must be configured to filter all traffic at the IP subnet level.

Application and Presentation Zone Rules

The following security rules apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in non-ATO(ed) environments and their Application and Presentation Zones:

When Non-Production Application and Presentation Zone application servers share VMs,

  1. They must use separate file systems; and

  2. They must use different UIDs and GIDs with

    1. RBS AAA functions provided by a common AAA server to ensure that management and user roles are common throughout a zone to prevent accidental or unauthorized access; or

    2. Application account passwords that are unique to the VM instance, e.g., apache account, to enforce separation of roles.

ATO(ed) Environment Mainframe Virtualization Security Rules

The following security rules apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in ATO(ed) environments and their zones. The general rules apply to all zones within an environment. In addition, requirements for each zone within an environment are provided when required.

Production VMs must use:

  • RBS AAA functions provided by a common AAA server to ensure that management and user roles are common throughout a zone to prevent accidental or unauthorized access; or

  • Application account passwords that are unique to the VM instance, e.g., apache account, must be used to enforce separation of roles; and

  • Unique GIDs and UIDs across all file systems to prevent inter-VM access.

  • Crash dumps residing on a common log server must be protected to prevent accidental destruction or unauthorized access.

  • Each VM must implement the CMS ARS Security Controls as required by the CMS Security Level of the resident application (please refer to Risk Management Handbook [RMH] Volume II, Procedure 2.3, Categorizing an Information System, located at: Information Security Library).

  • Resident applications with different CMS Security Levels or information types may not share a VM.

  • When required by CMS ARS requirements on the resident application, each VM will implement a HIPS.

  • All LPAR-z/VM access, and management traffic to VMs, must be encrypted using FIPS 140-2 certified modules (please refer to CMS ARS Security Control SC-13).

  • When required by CMS ARS requirements on the resident application, each VM will implement a firewall configured to allow communication only with authorized VMs in other zones over approved ports, protocols, and services.

  • Access to file systems associated with a project must be controlled through project-unique security categories to prevent direct access by members of other projects. These security categories must map to UNIX GIDs, with the UIDs of group members associated with the unique GID.

  • Production LPAR-z/VMs must run at a higher security level than Non-Production environment LPARs.

  • Production and Implementation LPAR-z/VMs must not be accessible from Test or Development LPAR-z/VMs.

  • Firewall rules must be implemented to deny intra-zone, inter-VM network traffic by default and must allow only ports, protocols, and services authorized for a source and destination IP address pair to pass.

Data Zone Rules

The following security rules apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in ATO(ed) environments and their Data Zones:

  • Production Data Zone LPAR-z/VMs must authenticate file systems before remount after VM crash.

  • GID and UIDs must be unique to each VM in a Production Data Zone.

  • Production Data Zone LPAR-z/VMs must run at a higher security level than all other LPAR-z/VMs on a Mainframe, except when Management Zone LPAR-z/VMs are present.

  • Production Data Zone application servers must reside on unique VMs.

  • Production Data Zone data sets, e.g., databases, must have unique security categories.

  • Processes associated with user applications must not be allowed to use APF.

  • Production Data Zone applications accessing Personally Identifiable Information (PII) or Protected Health Information (PHI) must run with a higher security category than those not requiring access to PII or PHI.

Application Zone Rules

The following security rules apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in ATO(ed) environments and their Application Zones:

  • Production Application Zone LPAR-z/VMs must authenticate file systems before remount after VM crash.

  • Production Application VMs must employ RBS AAA functions that must be provided by a common AAA server to ensure that management and user roles are common throughout a zone to prevent accidental or unauthorized access.

  • An application account’s passwords for Production Application VMs must be unique for each VM instance, e.g., an apache account must be used to enforce separation of roles.

  • Production Application VMs’ GID and UIDs must be unique across all file systems to prevent inter-VM access.

Presentation Zone Rules

The following security rules apply to LPAR-z/VMs and VMs running on LPAR-z/VMs in ATO(ed) environments and their Presentation Zones:

  • Production Presentation Zone LPAR-z/VMs must authenticate file systems before remount after VM crash.

  • Production Presentation VMs’ RBS AAA functions must be provided by a common AAA server to ensure that management and user roles are unique for each VM.

  • Production Presentation VMs’ GID and UIDs must be unique across all file systems to prevent inter-VM access.

  • Production Presentation Zone VMs providing intranet and extranet access must not reside on the same LPAR-z/VM.

  • Production Presentation Zone LPAR-z/VMs providing extranet access can run at a lower security level than all other LPAR-z/VMs on the mainframe except those LPAR-z/VMs supporting publicly available Internet web sites.

  • Production Presentation Zone LPAR-z/VMs supporting a publicly available Internet web site can run at a lower security level than an extranet.