[illumos-Developer] RFE : replace digest with schily mdigest

[Dennis Clarke]

I tend to use sha256 hash data in most things inside Blastwave. The old
MD5 hash process has long since been cracked and it is already a ( nearly
) deterministic process to create two files with the same MD5 hash where
the files even have similar size. NIST is clear on the secure hash
algorithms to implement :


   http://csrc.nist.gov/groups/ST/toolkit/secure_hashing.html

   Approved Algorithms

   There are five (5) Approved algorithms for generating a
   condensed representation of a message (message digest):
   SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512.


However the digest binary in OpenSolaris was built with libs and such that
are closed as well as, even worse, non-portable assembly language bits.

I would propose that the schily tools release of mdigest be used as it is
very very fast and totally portable, open and it seems to work well
everywhere.

Here is a test run of openssl, my own static implementation of the sha256
algorithm as well as the schily mdigest :

# ptime /opt/csw/bin/openssl dgst -sha256 debian-503-powerpc-kde-CD-1.iso
SHA256(debian-503-powerpc-kde-CD-1.iso)=
7eae5d77d53b3768e6a5dacd11401e8ef3795a7281a35bc66d3c1906e6f588a8

real     1:14.087
user     1:02.171
sys        11.307


# file /opt/csw/libexec/common/sha256
/opt/csw/libexec/common/sha256: ELF 32-bit LSB executable 80386 Version 1,
statically linked, stripped

# ptime /opt/csw/libexec/common/sha256 debian-503-powerpc-kde-CD-1.iso
7eae5d77d53b3768e6a5dacd11401e8ef3795a7281a35bc66d3c1906e6f588a8

real     1:06.731
user       54.800
sys        11.351

# ptime /opt/schily/bin/mdigest -a sha256 debian-503-powerpc-kde-CD-1.iso
7eae5d77d53b3768e6a5dacd11401e8ef3795a7281a35bc66d3c1906e6f588a8 
debian-503-powerpc-kde-CD-1.iso

real     1:08.003
user       59.238
sys         8.282

I have tried to make a very fast sha256 implementation but assembly
language changes were required. Thus I stuck a copyright on the resultant
binary and made it a static part of the CSWcommon package from Blastwave.
Sorry but it was a successful experiment however mdigest is far better.


-- 
Dennis Clarke
dclarke at opensolaris.ca  <- Email related to the open source Solaris
dclarke at blastwave.org   <- Email related to open source for Solaris