I have a multipath config that was working but now shows a 'faulty' path:

• Map In Use Multipath Windows 10

At the same time I'm seeing these three lines over and over in /var/log/messages

And this line shows up fairly often too

Multipath propagation causes multipath interference, including constructive and destructive interference, and phase shifting of the signal; destructive interference causes fading. Where the magnitudes of the signals arriving by the various paths have a distribution known as the Rayleigh distribution, this is known as Rayleigh fading.

One thing I don't understand is why its using the readsector0 checking method when my /etc/multipath.conf file say to use tur

[root@nas ~]# tail -n15 /etc/multipath.conf

Looking at the upstream documentation here this paragraph seems relevant:http://christophe.varoqui.free.fr/usage.html

Its been well over 24 hours for me so its not temporary.

So with all that as background my questions are
- how can I determine the root cause here?
- how can I manually/command-line perform whatever check its doing
- why is it ignoring my multipath.conf (did I do it wrong?)

Thanks in advance for any ideas, if there's anything else I can provide for info let me know in a comment and I'll edit it into the post.

1 Answer

There's a subtle bug in your multipath.conf, vendor and product are matching at the regexp level, that you've added a series of leading spaces is causing multipathd to fail to match your configuration with the actual devices on the system. If you were to examine the output of echo 'show config' multipathd -k you would find two device sections for your SAN, one that matches all the extra spaces you added, and the default config (should it exist) provided by internal database.

Adjust your multipath.conf to look like this:

SCSI Inquiry expects a vendor field that is no greater than 8 characters terminated by an ASCII Zero, if you don't use all 8 you must pad the field withspaces to reach 8 characters. Multipathd is interpreting the spec to the letter of the law, you could have also done 'XIOTECH.*' if you really want to be sure.

Once you make these changes, stop multipathd using your initscripts, multipath -F which will flush your config and then start multipathd again. Your config file should be honored now. If you still have problems, reboot.

If there's ever a doubt that your config file isn't being honored, always examine the running config using the echo incantation and compare what's loaded in the database to your config file.

Not the answer you're looking for? Browse other questions tagged centosstorage-area-networkrhel5multipath or ask your own question.

DM-Multipathing (DM-MPIO) provides input-output (I/O) fail-over and load-balancing by using multipath I/O within Linux for block devices.^[1][2][3] By utilizing device-mapper, the multipathd daemon provides the host-side logic to use multiple paths of a redundant network to provide continuous availability and higher-bandwidth connectivity between the host server and the block-level device.^[4] DM-MPIO handles the rerouting of block I/O to an alternate path in the event of a path failure. DM-MPIO can also balance the I/O load across all of the available paths that are typically utilized in Fibre Channel (FC) and iSCSISAN environments.^[5]DM-MPIO is based on the device mapper, which provides the basic framework that maps one block device onto another.

• 3Components

Considerations[edit]

When utilizing Linux DM-MPIO in a datacenter that has other operating systems and multipath solutions, key components of path management must be considered.

• Load balancing — The workload is distributed across the available hardware components. Goal: Reduce I/O completion time, maximize throughput, and optimize resources
• Path failover and recovery — Utilizes redundant I/O channels to redirect application reads and writes when one or more paths are no longer available.

History[edit]

DM-MPIO started as a patch set created by Joe Thornber, and was later maintained by Alasdair G Kergon at Red Hat. It was included in mainline Linux with kernel version 2.6.12, which was released on June 17, 2005.^[6]

Components[edit]

Map In Use Multipath Windows 10

DM-MPIO in Linux consists of kernel components and user-space components.

• Kernel – device-mapper – block subsystem that provides layering mechanism for block devices.
  • dm-multipath – kernel module implementing the multipath device-mapper target.
• User-space – multipath-tools – provides the tools to manage multipathed devices by instructing the device-mapper multipath module what to do. The tools consist of:
  • Multipath: scans the system for multipathed devices, assembles them, updates the device-mapper's map.^[5]
  • Multipathd: daemon that waits for maps events, and then executes multipath and monitors the paths. Marks a path as failed when the path becomes faulty. Depending on the failback policy, it can reactivate the path.^[5]
  • Devmap-name: provides a meaningful device-name to udev for devmaps.^[5]
  • Kpartx: maps linear devmaps to device partitions to make multipath maps partitionable.^[5]
  • Multipath.conf: configuration file for the multipath daemon. Used to overwrite the built-in configuration table of multipathd.

Configuration file[edit]

The configuration file /etc/multipath.conf makes many of the DM-MPIO features user-configurable. The multipath command and the kernel daemon multipathd use information found in this file. The file is only consulted during the configuration of the multipath devices. Changes must be made prior to running the multipath command. Changes to the file afterwards will require multipath to be executed again.

The multipath.conf has five sections:^[7]

1. System level defaults (defaults): User can override system level defaults.
2. Blacklisted devices (blacklist): User specifies the list of devices that is not to be under the control of DM-MPIO.
3. Blacklist exceptions (blacklist_exceptions): Specific devices to be treated as multipath devices even if listed in the blacklist.
4. Storage controller specific settings (devices): User specified configuration settings will be applied to devices with specified 'Vendor' and 'Product' information.
5. Device specific settings (multipaths): Fine tune the configuration settings for individual LUNs.

Terminology[edit]

• HBA: Host bus adapters provide the physical interface between the input/output (I/O) host bus of Fibre Channel devices and the underlying Fibre Channel network.^[8]
• Path: Connection from the server through the HBA to a specific LUN.
• DM Path States: The device mapper's view of the path condition. Only two conditions are possible:
  • Active: The last I/O operation sent through this path successfully completed. Analogous to ready path state.
  • Failed: The last I/O operation sent through this path did not successfully complete. Analogous to faulty path state.
• Failover: When a path is determined to be in a failed state, a path that is in ready state will be made active.^[9]
• Failback: When a failed path is determined to be active again, multipathd may choose to failback to the path as determined by the failback policy.^[10]
• Failback Policy: Four options as set in the multipath.conf configuration file.
  • Immediate: Immediately failback to the highest priority path.
  • Manual: The failed path is not monitored, requires user intervention to failback.
  • Followover(for clusters): Only perform automatic failback when the first path of a pathgroup becomes active. This keeps a node from automatically failing back when another node requested the failover.
  • Number of seconds: Wait for a specified number of seconds to allow the I/O to stabilize, then failback to the highest priority path.
• Active/Active: In a system that has two storage controllers, each controller can process I/O.^[11]
• Active/Passive: In a system that has two storage controllers, only one controller at a time is able to process I/O, the other (passive) is in a standby mode.^[11]
• LUN: SCSILogical Unit Number
• WWID: Worldwide Identifier is an identifier for the multipath device that is guaranteed to be globally unique and unchanging.

Further reading[edit]

• Michael, T., Kabir, R., Giles, J. & Hull, J. (2006.) Configuring Linux to Enable Multipath I/O. Retrieved from http://www.dell.com/downloads/global/power/ps3q06-20060189-Michael.pdf
• Goggin, E., Kergon, A., Varoqui, C., & Olien, D. (2005) Proceedings of the Linux Symposium – Linux Multipathing. Retrieved from http://www.linuxinsight.com/files/ols2005/goggin-reprint.pdf
• Red Hat Documentation. (n.d.) Red Hat Enterprise Linux 6, DM Multipath. Retrieved from https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/DM_Multipath/
• Varoqui, C. (2010.) The Linux multipath implementation. Retrieved from http://christophe.varoqui.free.fr/refbook.html

References[edit]

1. ^Goggin, Edward; Kergon, Alasdair; Varoqui, Christophe; Olien, David. 'Linux Multipathing'(PDF).
2. ^Varoqui, Christophe. 'The Linux multipath implementation'.
3. ^Oberg, Michael. 'Exploration of Parallel Storage Architectures for a Blue Gene/L on the TeraGrid'(PDF).
4. ^van Vugt, Sander. A Practical Guide to XEN High Availability.
5. ^ ^abcdeSUSE. 'Storage Administration Guide, SUSE Linux Enterprise Server 11 SP1'(PDF). SLES11 Documentation, pg. 49.
6. ^Linux 2.6.12 kernelnewbies.org
7. ^RedHat. 'Using Device-Mapper Multipath'. Using Device-Mapper Multipath.
8. ^Gupta, Meeta (2002). Storage Area Network Fundamentals. Indianapolis, IN: Cisco Press. p. 81. ISBN1-58705-065-X.
9. ^Anderson, Michael. 'SCSI Mid-Level Multipath'(PDF).
10. ^'Storage Administration Guide, SLES11 Documentation'(PDF). SUSE. p. 73.
11. ^ ^abCentos. 'Overview of DM-Multipath'. Using Device-Mapper Multipath.

External links[edit]

• multipath-tools, homepage of the upstream project used to drive the Device Mapper multipathing driver.