[illumos-Developer] Changing sd_io_time to 8?

[Ceri Davies]

On Fri, Apr 29, 2011 at 01:26:57PM -0700, Richard Elling wrote:
> I'd like to change sd_io_time from 60 (!) to 8.
> 
> For those who are unfamiliar, sd_io_time is the default time before the sd driver
> will timeout a command. By default, it will retry 3 or 5 times before failing the I/O.
> 
> >From http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/sys/scsi/targets/sddef.h#1704
> 
>     /*
>     * 60 seconds is a *very* reasonable amount of time for most slow CD
>     * operations.
>     */
>     #define	SD_IO_TIME			60
> 
> Many RAID cards give up at around 8 seconds. Many HDDs guarantee some sort of response
> in under 8 seconds (enterprise or RAID version).  If a consumer-grade drive can't get it's act 
> together in 8 seconds, not much chance it will get any better in 60 seconds. Nobody cares about
> 1x SCSI CD-ROMS circa 1989, except perhaps the SPARC folks :-) 

One thing worth noting is that VMware ESX{i,} will suspend I/O to VMs
for up to 60 seconds if a FC path failover occurs [1]. Note that this is
intentionally done in order to allow the SAN topology to "settle" and
isn't related to how long it actually takes for the path failover to
occur.

This often leads to Linux VMs timing out enough commands to degrade
filesystems to read-only, and the "official" workaround is setting
whatever tunable /sys/block/sda/device/timeout controls to 180; this is
done automatically when VMware Tools is installed on Linux VMs.

It's my understanding that changing the sd_io_time parameter would
therefore have a detrimental effect in VMs and I'd ask you to
reconsider on that basis.  If I've misunderstood, so be it.

Ceri

[1] See page 21 of the VMware "Fibre Channel SAN Configuration Guide" at
     http://www.vmware.com/pdf/vsphere4/r41/vsp_41_san_cfg.pdf
-- 
Haffely, Gaffely, Gaffely, Gonward.