[illumos-Developer] "od" i386 binary for benchmark

[Joerg Schilling]

????? ???????????? <olga.kryzhanovska at gmail.com> wrote:

> I uploaded the binary to
> http://www.nrubsig.org/people/fleyta/opensolaris/illumos/posixutilities/gisburn_od_20101020_i386.bz2
> as alternative location until the list has been fixed.
>
> Olga
>
> 2010/10/20 ????? ???????????? <olga.kryzhanovska at gmail.com>:
> > Garrett, could you please benchmark the binary attached against your
> > od? I tested it against your latest version and measures these times
> > on vmware:
> > timex ~/bin/ksh -c 'export LC_ALL=C ; for ((i=0 ; i < 40 ; i++ )) ; do
> > ./garrettod/od -c </usr/pub/UTF-8 >/dev/null ; done ; true'
> > real           8.50
> > user           8.04
> > sys            0.01
> >
> > timex ~/bin/ksh -c 'export LC_ALL=C ; for ((i=0 ; i < 40 ; i++ )) ; do
> > gisburn_od -c </usr/pub/UTF-8 >/dev/null ; done ; true'
> > real           4.86
> > user           4.33
> > sys            0.01

Olga,

speed is something to look at after the software is working correctly. The od
implementation you are advertisig above, has several deviations from the POSIX
standard and it implements a behavior that is in conflict with the documented 
behavior of the Closed Source Sun "od".

Given the fact that the AT&T implementation you are advertising heavily depends 
on functions that are undocumented on Solaris and that the way of implementation
is not obvious, I would guess that it is not trivial to fix the bugs and to add
the missing support for the Sun behavior.

You offered a 64 bit binary only, so I cannot verify whether the floating point
implementation would work correctly for a 32 bit application on x86 hardware, 
since there are different long double sizes to respect (12 bytes vs. 16 bytes).

The implementation you offer has definitely fewer problems than the 
implementation offered by Garret, but I still remember several problems from 
the tests I did. I did not write down everything, so there may be actually more 
problems than these:

-	Please look at the output from:

	LC_ALL=de_DE.UTF-8 /tmp/gisburn_od_20101020_i386 -C /usr/pub/UTF-8

	and compare it with the output from my "od" implementation (see
	ftp://ftp.berlios.de/pub/schily). There are intentional 
	incompatibilities with the POSIX standard and there are bugs that
	are probably a result from using a self made i18n implementation
	that does not work properly. I remember the following problems:

	-	incorrect print out for some 100% correct printable multi-byte
		characters (this seems to be a bug in your i18n implementaton)

	-	incorrect print out for byte sequences that are non-valid
		multi byte sequences

	-	"printing" byte sequences as char that are valid multi-byte
		sequences although they are marked non-printable

-	There are similar POSIX incompatibilities even with single byte
	locales. These incompatibilities are most likely intentional too.

-	Your od does not always behave correctly with respect to the EOF 
	condition and partially prints a wrong file/dump size (see address label
	at the last line of the output).

Note that my tests also unveiled several bugs in the current Sun Closed Source
od implementation. So I started to write my own od(1) implementation after I 
checked:

-	The current Solaris Closed Source od from /usr/bin/od and 
	/usr/xpg4/bin/od

-	/usr/gnu/bin/od

-	The od implementation from FreeBSD

-	/tmp/gisburn_od_20101020_i386

-	Garret's implementation.

When doing the introducing tests, my impression was that there has been no 
od(1) implementation that was sufficiently close to the POSIX standard as well
as to the traditional Solaris behavior.

        My "od" aims to implement 100% compatibility to the POSIX standard and
	to the traditional Solaris "od". My "od" can be installed as 
	/usr/bin/od and /usr/xpg4/bin/od as well as /usr/bin/hdump, note that
	this must be done as separate file copies or as hardlinks to make the
	interface detection work correctly. Note that this release does not 
	yet install as .../xpg4/bin/od, you need to create this hard link by 
	hand if you like to test support for xpg4 behavior.

        Please test this new program for compatibility with POSIX and the closed
        source Solaris implementation and for bugs as it is intended to act as a
        OSS replacemtent for the closed source "od" from Solaris on SchilliX.

        Note that my new OSS od(1) fixes several bugs found in the current
        closed source implementation on Solaris:

        -       My od implements UTF-8 char printing in a 100% correct way

        -       My od implementation honors the size of long double numbers
                and uses a block size of 12 or 24 in case it is compiled as
                32 bit application on Intel x86 or Motorola mc680x0 systems,
                where sizeof (long double) is 12.

        -       My od produdes multi-line output that is properly aligned, e.g.
		with "od -to1o2o4o8 file"

        -       My od correctly works even when called similar to:

                        "od unreadable - name"

                and "-" appears amongst other file names in the command line
                past an unreadable file.

        Note that the Sun od implementation supports to have multiple "-" 
        arguents in a command line referring to stdin multiple times. In this
        case the old Sun implementation uses the actual stdin file offset at 
        start time for the first occurence of "-" and rewinds stdin before
        using it another time. This results in:

        (read line; od -ta - - ) < /etc/passwd

        to first dump the passwd file starting at line #2 and then to dump 
        the whole passwd file again starting at offset 0.

        My implementation of od currently keeps the file offset for stdin.
        This results in an EOF condition in case "-" is used a second time.
        The example above thus would result in printing the passwd file once
        starting with the second line. Please send me a message if you like
        this implementation to behave the way Solaris od(1) does in this case. 

My implementation of course is portable to many platforms and available in:

	ftp://ftp.berlios.de/pub/schily/schily-2010-11-10.tar.bz2

I do not expect it to be completely without bugs, so please test and report if
you find bugs.

Jörg

-- 
 EMail:joerg at schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js at cs.tu-berlin.de                (uni)  
       joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily