[illumos-Advocates] RTI 992 towlower/towupper are broken

[Garrett D'Amore]

I would prefer to just continue forward with all of your changes.

I believe that there is no key requirement that the u8_* functions match what we have here, and I recognize that the requirements for case folding are different than those for case conversion.  Furthermore, having case conversion functions for character sets for which we have no data, seems wrong.

That said, one possible way to test this is to write a test program which iterates over all of utf8 space and identifies the cases where these functions have a non-identity mapping, and then check with the u8_*.   But I really think Gordon is being unduly cautious.

Fundamentally, we need to support the POSIX standards for localedef.  This code does that.  I am very disinterested in trying to special case the character set mappings in order to artifically try to share some code.   Such sharing would break our ability to support correct POSIX localedef, and specifically would not support non-UTF-8 data.

Gordon, what do you think, can we just let Yuri move ahead?  Certainly nobody could say his changes do anything except *improve* the current situation.

  -- Garrett D'Amore

On May 17, 2011, at 8:59 AM, Yuri Pankov <yuri.pankov at gmail.com> wrote:

> On Mon, May 16, 2011 at 04:09:47PM -0400, Gordon Ross wrote:
>> On Wed, May 11, 2011 at 7:34 PM, Yuri Pankov <yuri.pankov at gmail.com> wrote:
>>> Hi,
>>> 
>>> illumos:yuri:~/ws/992-localedef$ hg outgoing -v ssh://anonhg@hg.illumos.org/illumos-gate
>>> running ssh anonhg at hg.illumos.org "hg -R illumos-gate serve --stdio"
>>> remote: Not trusting file /export/illumos/hgrepos/illumos-gate/.hg/hgrc from untrusted user hg, group hg
>>> comparing with ssh://anonhg@hg.illumos.org/illumos-gate
>>> searching for changes
>>> 
>>> changeset:   13369:b913fe55a4c0
>>> tag:         tip
>>> user:        Yuri Pankov <yuri.pankov at gmail.com>
>>> date:        Thu May 12 03:21:34 2011 +0400
>>> 
>>> description:
>>>        992 towlower/towupper are broken
>>>        Reviewed by: Garrett D'Amore <garrett at damore.org>
>>> 
>>> modified:
>>>   usr/src/cmd/localedef/Makefile
>>>   usr/src/cmd/localedef/ctype.c
>>> added:
>>>   usr/src/cmd/localedef/data/ctype.sh
>>> 
>>> remote: Not trusting file /export/illumos/hgrepos/illumos-gate/.hg/hgrc from untrusted user hg, group hg
>>> 
>>> 
>>> Tested by using towlower/towupper functions for latin, cyrillic and
>>> greek characters in en_US.UTF-8 and ru_RU.UTF-8 locales - results are
>>> the same in both.
>> [...]
>> 
>> Hi Yuri,
>> 
>> Are you still working on this?
>> 
>> I'd like to see an answer to the functionality questions about this
>> before we integrate.  (How do we know if the fix is complete?)
>> 
>> I suggested one way you could verify your fix.  I'm sure you could
>> find many other ways as well.  Please choose a test method and
>> use it to demonstrate that your fix is complete.
> 
> Ok, let's make this just a fix for __maplower_ext excluding other
> changes as I can't comment on the best way to provide common ctype data,
> and, more so, on u8_* functions, which seem to be private (as well as
> non-standard) to me - I just thought getting ctype data from locales we
> actually support seems reasonable, but probably incorrect. I guess we
> should continue discussing the best way to do this in the thread Garrett
> started.
> 
> 
> Yuri
> 
> _______________________________________________
> Advocates mailing list
> Advocates at lists.illumos.org
> http://lists.illumos.org/m/listinfo/advocates