[illumos-Developer] Changing sd_io_time to 8?

[Garrett D'Amore]

Richard, lets work together on this.

I've already got at least one trivial change queued to eliminate disk
sort for SSDs, and a couple of others for enhanced mpxio support, I
think there can be others.

I *despise* the current tuning interface in sd.conf... I can't imagine
anything more byzantine, and revamping this would be a good idea.

At the end of the idea, the cruftiness in sd, and in SCSA generally, is
one reason why I have moved away from SCSA (and strongly encourage
others to do the same) when dealing with things that are not
intrinsically SCSI -- blkdev is a good alternate.

Ultimately, I'd love to break sd apart -- there is no real reason that
pure SATA, or cdroms, should be included as part of the SCSI disk driver
-- and there are a lot of crazy edge cases that we have to deal with
that add a lot of complexity to sd as a result.  Sadly, its probably too
late to undo those decisions....

	- Garrett


On Sat, 2011-04-30 at 08:36 -0700, Richard Elling wrote:
> On Apr 30, 2011, at 6:59 AM, Mike La Spina wrote:
> > Hi Richard,
> >  
> > This would be a very beneficial change since a single failing sd
> > device in a raidz1 seg would cause a 60 second delay in all IO for
> > that seg.  I can see some potential for a negative impact in the VM
> > world but if it’s tunable that’s really a non issue.
> > e.g
> >  
> > if VENDOR = VMware
> >  then
> > sd_io_time = 120  
> 
> 
> IIRC, I wrote an RFE for that about 10 years ago... don't recall the
> CR number, though.
> As you can see, for high availability systems (I was working with Sun
> Cluster at the time)
> the timeouts are critical. Similarly, for upper layers, like ZFS, that
> rely on the lower layers
> to handle errors, waiting 5 minutes to declare an HDD dead is
> unacceptable.
> 
> 
> Judging from some of the responses here, and a depressingly huge
> amount of misinformation
> on the 'net, the reset logic and control structure could use a good
> dose of exposure. I had done
> that analysis 10 years ago, and can redo today, but please give me a
> few weeks to pull it together.
> 
> 
> There are many things about sd that are crufty or difficult to
> administer (eg sd.conf). There are also
> many compiled-in specific-case tunings that are, IMHO, better managed
> as a white list. I believe 
> that we can more easily manage the special cases of million dollar
> arrays and lightning fast SSDs
> by tuning via a white list. [While we're at it, we can fix the whole
> sector size fiasco]
>  -- richard
> 
> > 
> >  
> > +1
> >  
> > From: Brendan Gregg [mailto:brendan.gregg at joyent.com] 
> > Sent: Friday, April 29, 2011 3:33 PM
> > To: Richard Elling
> > Cc: developer at lists.illumos.org
> > Subject: Re: [illumos-Developer] Changing sd_io_time to 8?
> >  
> > G'Day Richard,
> >  
> > Yes, this has been a nuisance and would be good to tune.  I'm trying
> > to remember if it could break anything.  Could this be made an int
> > so it can be mdb -kw tuned back to 60 if things go bad for someone?
> >  Of course, we don't want to create yet-another tunable if we can
> > avoid it, so this would be temporary and eventually switched back to
> > a #define.
> >  
> > Brendan 
> >  
> > On Fri, Apr 29, 2011 at 1:26 PM, Richard Elling
> > <richard.elling at richardelling.com> wrote:
> > I'd like to change sd_io_time from 60 (!) to 8.
> > 
> > For those who are unfamiliar, sd_io_time is the default time before
> > the sd driver
> > will timeout a command. By default, it will retry 3 or 5 times
> > before failing the I/O.
> > 
> > From http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/sys/scsi/targets/sddef.h#1704
> > 
> >    /*
> >    * 60 seconds is a *very* reasonable amount of time for most slow
> > CD
> >    * operations.
> >    */
> >    #define     SD_IO_TIME                      60
> > 
> > Many RAID cards give up at around 8 seconds. Many HDDs guarantee
> > some sort of response
> > in under 8 seconds (enterprise or RAID version).  If a
> > consumer-grade drive can't get it's act
> > together in 8 seconds, not much chance it will get any better in 60
> > seconds. Nobody cares about
> > 1x SCSI CD-ROMS circa 1989, except perhaps the SPARC folks :-)
> > 
> > Thoughts?
> >  -- richard
> > 
> > 
> > _______________________________________________
> > Developer mailing list
> > Developer at lists.illumos.org
> > http://lists.illumos.org/m/listinfo/developer
> > 
> > 
> > 
> > -- 
> > Brendan Gregg, Joyent
> >  http://dtrace.org/blogs/brendan
> 
> _______________________________________________
> Developer mailing list
> Developer at lists.illumos.org
> http://lists.illumos.org/m/listinfo/developer