[illumos-Discuss] Test/build setup?

[Dmitry Yusupov]

On 08/19/2010 11:09 AM, Garrett D'Amore wrote:
> On Thu, 2010-08-19 at 11:02 -0700, Dmitry Yusupov wrote:
>
>>> Nexenta has a system called NABAT, but its highly specific to
>>> debian/apt, so we need something more generic.
>>
>> Upcoming NexentaCore 3.9 is going to be Illumos based and *is* going to
>> be nightly tested/released by NABAT. While I agree, we need a generic
>> framework I would suspect that we only need to focus on testing Illumos
>> components and not its permutations or compositions. Good example would
>> be ztest utility..
>>
>> For instance, NABAT is also testing NFS/CIFS/iSCSI against tens of
>> clients such us Windows/Linux/ESX/Xen/Hyper-V/etc to ensure its
>> interoperability. And at distro level it is easier and Illumos is not a
>> distro.
>>
>
> So Illumos is the beneficiary of this testing, which is awesome.
>
> But it isn't sufficient IMO for a couple of reasons:
>
> a) NCP replaces a number of userland components with bits from
> GNU/Debian.  While this is fine for NCP, the Illumos community still
> needs some basic verification of the ones that everyone else in the
> Illumos community uses.
>
> b) NABAT is owned by Nexenta.  While as a Nexenta employee I have no
> concerns about Nexenta's interests, I think its important to try to make
> sure that we use resources that can't be taken away later by Nexenta.
> Not that I fear this will happen, but by eliminating even the
> possibility we can prevent any such concerns from arising elsewhere.
>
> c) IMO, there is great value in alternative testing.  If everyone always
> runs the same test suites, and nothing else, then we don't get to find
> out about bugs that are not covered by that specific test procedure in
> the system during test. (Instead it bits us when someone is trying to
> *use* the software, which is infinitely worse than finding the bug
> during automated testing.)
>
> Hopefully this makes sense.

Make sense! And for (c), I bet Nexenta would be happy to sponsor its 
integration into NABAT later on.