[illumos-Developer] grep -Z etc? (decompress! was: webrev: 650 grep support for -q ...)

[Gordon Ross]

On Fri, May 6, 2011 at 9:59 AM, Kurt Lidl <kurt.lidl at cello.com> wrote:
[...]
(gwr)> Or it could just fork off a "gzip -dc" (or whatever) when it needs
>> to decompress something.  I took a stab at that (diff attached).
[...]
> Forking off a copy of 'gzip -dc', and then having to copy all the compressed
> data through a pipe to gzip, and then having to copy all the uncompressed
> data back through the pipe to grep is slow.

First, a clarification:  The compressed data does _not_ go through a pipe.
The uncompress program runs with the original _file_ on FD=0 (stdin), and
the pipe as its stdout.  The grep program reads from the other end of that
pipe (and never writes to the pipe).  So one pipe, used only one direction.

> Yes, I know, it's "common" to do this, but it really stinks,
> performance-wise.

You sure about that?

> Integrating direct usage of the libgz (or libz or libbz or liblzma when the
> time comes) is a huge performance win.

If that's true, then there may be something wrong with our pipes...

Gordon