[illumos-Discuss] Test/build setup?

[Garrett D'Amore]

On Thu, 2010-08-19 at 11:02 -0700, Dmitry Yusupov wrote:

> > Nexenta has a system called NABAT, but its highly specific to
> > debian/apt, so we need something more generic.
> 
> Upcoming NexentaCore 3.9 is going to be Illumos based and *is* going to 
> be nightly tested/released by NABAT. While I agree, we need a generic 
> framework I would suspect that we only need to focus on testing Illumos 
> components and not its permutations or compositions. Good example would 
> be ztest utility..
> 
> For instance, NABAT is also testing NFS/CIFS/iSCSI against tens of 
> clients such us Windows/Linux/ESX/Xen/Hyper-V/etc to ensure its 
> interoperability. And at distro level it is easier and Illumos is not a 
> distro.
> 

So Illumos is the beneficiary of this testing, which is awesome.

But it isn't sufficient IMO for a couple of reasons:

a) NCP replaces a number of userland components with bits from
GNU/Debian.  While this is fine for NCP, the Illumos community still
needs some basic verification of the ones that everyone else in the
Illumos community uses.

b) NABAT is owned by Nexenta.  While as a Nexenta employee I have no
concerns about Nexenta's interests, I think its important to try to make
sure that we use resources that can't be taken away later by Nexenta.
Not that I fear this will happen, but by eliminating even the
possibility we can prevent any such concerns from arising elsewhere.

c) IMO, there is great value in alternative testing.  If everyone always
runs the same test suites, and nothing else, then we don't get to find
out about bugs that are not covered by that specific test procedure in
the system during test. (Instead it bits us when someone is trying to
*use* the software, which is infinitely worse than finding the bug
during automated testing.)

Hopefully this makes sense.

	-- Garrett