Topic: Defect Report: istreambuf_iterator should have an operator->()


Author: noreply@this.is.invalid (Niels Dekker - no return address)
Date: Wed, 28 Mar 2007 00:02:08 GMT
Raw View
Howard Hinnant wrote:
> I want a proposed resolution.

Thank you for providing a proposed resolution already  :-)  You wrote at
http://home.twcny.rr.com/hinnant/cpp_extensions/issues_preview/lwg-active.html#659
> Proposed resolution:
>   Add to the synopsis in 24.5.3 [istreambuf.iterator]:
>     [...]
>     charT* operator->() const;
>     [...]
>
>   Change 24.5.3 [istreambuf.iterator], p1:
>     [...]  operator-> returns 0.

Unfortunately this solution won't work if char_type is a class that
contains one or more members.  E.g.:

  class CharWrapper {
  public:
    char value;
  };

  namespace std {
    template<> struct char_traits<CharWrapper> {
      [...]
    };
  }

  void Func(std::istreambuf_iterator<CharWrapper> &i)
  {
    char c1 = (*i).value;  // This is just fine...
    char c2 = i->value;  // ... so this should also be supported!
  }

> Yeah, we've got a bit of a problem.  Normally you would want
> operator->() to just return &operator*().  But when operator*() is
> returning a scalar by value, you are not allowed to take the address of
> it.  So what do we return the address of? :-)

One solution would be to have the iterator internally store a copy of
the current character, as a member.  Apparently that's similar to what
James Dennett did, as he wrote:
> to make istreambuf_iterator work
> with concepts I added a char_type buffer inside the
> iterator type and returned a reference to that

The approach of using a proxy has been discussed in this thread as well,
and I'm glad to hear from Douglas Gregor that this should *not* clash
with the concepts library proposal.  So I guess one could use a proxy
like operator_arrow_proxy from
http://www.boost.org/boost/iterator/iterator_facade.hpp as follows:

  template<class charT, class traits>
  operator_arrow_proxy<charT>
  istreambuf_iterator<charT, traits>::operator->() const
  {
    charT val = **this;
    operator_arrow_proxy<charT> result(&val);
    return result;
  }

Please let me know if either of these two approaches is inappropriate,
when implementing istreambuf_iterator::operator->().


Thanks so far,

  Niels

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: noreply@this.is.invalid (Niels Dekker - no return address)
Date: Sun, 25 Mar 2007 23:40:57 GMT
Raw View
Okay, okay, I'm converted! Greg Herlihy has clearly demonstrated that a
user defined input iterator should have an operator->(), even if its
value type is a built-in type.  And as Howard Hinnant remarked that the
input iterator istreambuf_iterator doesn't have one, this must be a
defect!

Based on Greg's example, the following code demonstrates the issue:

  #include <iostream>
  #include <fstream>
  #include <streambuf>

  typedef char C;
  int main ()
  {
    std::ifstream s("filename", std::ios::in);
    std::istreambuf_iterator<char> i(s);

    (*i).~C();  // This is well-formed...
    i->~C();  // ... so this should be supported!
  }

--
Niels Dekker
http://www.xs4all.nl/~nd/dekkerware
C++ programmer at LKEB, Leiden University Medical Center

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: "James Kanze" <james.kanze@gmail.com>
Date: Mon, 26 Mar 2007 08:40:32 CST
Raw View
On Mar 26, 1:40 am, nore...@this.is.invalid (Niels Dekker - no return
address) wrote:
> Okay, okay, I'm converted! Greg Herlihy has clearly demonstrated that a
> user defined input iterator should have an operator->(), even if its
> value type is a built-in type.  And as Howard Hinnant remarked that the
> input iterator istreambuf_iterator doesn't have one, this must be a
> defect!

> Based on Greg's example, the following code demonstrates the issue:

>   #include <iostream>
>   #include <fstream>
>   #include <streambuf>

>   typedef char C;
>   int main ()
>   {
>     std::ifstream s("filename", std::ios::in);
>     std::istreambuf_iterator<char> i(s);

>     (*i).~C();  // This is well-formed...

Well formed, perhaps, but it looks like undefined behavior at
execution.  (Maybe not in the case of char; char's rather
special in this regard.  But definitily in the case of other
types.)

>     i->~C();  // ... so this should be supported!
>   }

I'm not convinced, at least not for istreambuf_iterator (which
is a bit special).

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: howard.hinnant@gmail.com (Howard Hinnant)
Date: Mon, 26 Mar 2007 14:13:50 GMT
Raw View
In article <4606E7F1.9137D31F@this.is.invalid>,
 noreply@this.is.invalid (Niels Dekker - no return address) wrote:

> Okay, okay, I'm converted! Greg Herlihy has clearly demonstrated that a
> user defined input iterator should have an operator->(), even if its
> value type is a built-in type.  And as Howard Hinnant remarked that the
> input iterator istreambuf_iterator doesn't have one, this must be a
> defect!
>
> Based on Greg's example, the following code demonstrates the issue:
>
>   #include <iostream>
>   #include <fstream>
>   #include <streambuf>
>
>   typedef char C;
>   int main ()
>   {
>     std::ifstream s("filename", std::ios::in);
>     std::istreambuf_iterator<char> i(s);
>
>     (*i).~C();  // This is well-formed...
>     i->~C();  // ... so this should be supported!
>   }


Ok, but I'm not letting you off the hook quite so easily. :-)

I want a proposed resolution.  Here's what I've got so far:

http://home.twcny.rr.com/hinnant/cpp_extensions/issues_preview/lwg-active
.html#659

Was that a gasp I heard? ;-)

Yeah, we've got a bit of a problem.  Normally you would want
operator->() to just return &operator*().  But when operator*() is
returning a scalar by value, you are not allowed to take the address of
it.  So what do we return the address of? :-)

operator*() is going to be calling sgetc() on the internal streambuf.
But sgetc() returns an int_type, not a char_type.  So operator*() is
going to be making a cast, generating a temporary char_type, and
returning that by value.  So even inside of operator*() there is no
lvalue (with sufficient lifetime) we can point to.  Perhaps we could
return the address of an unspecified char_type?  Are we fighting an
uphill battle?

<disclaimer>
You are not required to provide a proposed resolution with your defect
report.  It is perfectly fine for you to leave the problem solving to
the committee.  I can leave the proposed resolution area blank.  I'm
just trying to up the odds of this being processed in a way that would
be agreeable to you.
</disclaimer>

-Howard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: tasjaevan@gmail.com
Date: Mon, 26 Mar 2007 11:56:35 CST
Raw View
On Mar 26, 3:13 pm, howard.hinn...@gmail.com (Howard Hinnant) wrote:
>
> operator*() is going to be calling sgetc() on the internal streambuf.
> But sgetc() returns an int_type, not a char_type.  So operator*() is
> going to be making a cast, generating a temporary char_type, and
> returning that by value.  So even inside of operator*() there is no
> lvalue (with sufficient lifetime) we can point to.  Perhaps we could
> return the address of an unspecified char_type?  Are we fighting an
> uphill battle?
>

I think it's an uphill battle without Boost's improved iterator
concepts, which were dropped from Standardisation.

  http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/papers/2003/n1477.html

In that terminology, istreambuf_iterator is only 'Readable', not
'Readable Lvalue'.

If we were switching to those concepts, I would have suggested
operator-> return a proxy (much like boost::iterator_facade does for
non-lvalue iterators).

As it is, am I right in thinking that returning a proxy would clash
with the concepts library proposal?

James


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: "Greg Herlihy" <greghe@pacbell.net>
Date: Mon, 26 Mar 2007 11:52:47 CST
Raw View
On Mar 26, 7:40 am, "James Kanze" <james.ka...@gmail.com> wrote:
> On Mar 26, 1:40 am, nore...@this.is.invalid (Niels Dekker - no return
> address) wrote:
> > Based on Greg's example, the following code demonstrates the issue:
> >   #include <iostream>
> >   #include <fstream>
> >   #include <streambuf>
> >   typedef char C;
> >   int main ()
> >   {
> >     std::ifstream s("filename", std::ios::in);
> >     std::istreambuf_iterator<char> i(s);
> >     (*i).~C();  // This is well-formed...
>
> Well formed, perhaps, but it looks like undefined behavior at
> execution.  (Maybe not in the case of char; char's rather
> special in this regard.  But definitily in the case of other
> types.)

There would no point in allowing pseudo-destructor calls in C++ unless
such calls had well-defined behavior. Otherwise, the Standard would
simply be trading a compilation error for undefined behavior - which
would not be a sensible trade. So we can be assured that a psuedo-
destructor call for any scalar type is well-defined - and, in fact, it
is:

"The use of a pseudo-destructor-name after a dot . or arrow ->
operator represents the destructor for the non-class type named by
type-name. The result shall only be used as the operand for the
function call operator (), and the result of such a call has type
void. The only effect is the evaluation of the postfix-expression
before the dot or arrow." [   5.2.4/2]

Greg


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: James Dennett <jdennett@acm.org>
Date: Tue, 27 Mar 2007 09:49:50 CST
Raw View
tasjaevan@gmail.com wrote:
> On Mar 26, 3:13 pm, howard.hinn...@gmail.com (Howard Hinnant) wrote:
>> operator*() is going to be calling sgetc() on the internal streambuf.
>> But sgetc() returns an int_type, not a char_type.  So operator*() is
>> going to be making a cast, generating a temporary char_type, and
>> returning that by value.  So even inside of operator*() there is no
>> lvalue (with sufficient lifetime) we can point to.  Perhaps we could
>> return the address of an unspecified char_type?  Are we fighting an
>> uphill battle?
>>
>
> I think it's an uphill battle without Boost's improved iterator
> concepts, which were dropped from Standardisation.
>
>   http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/papers/2003/n1477.html
>
> In that terminology, istreambuf_iterator is only 'Readable', not
> 'Readable Lvalue'.
>
> If we were switching to those concepts, I would have suggested
> operator-> return a proxy (much like boost::iterator_facade does for
> non-lvalue iterators).
>
> As it is, am I right in thinking that returning a proxy would clash
> with the concepts library proposal?

I believe it would; to make istreambuf_iterator work
with concepts I added a char_type buffer inside the
iterator type and returned a reference to that --
though I'd have to check whether the lifetime of
the object returned from operator* does not clash
with that.  An improved classification of iterators
might well improve on that situation.

-- James

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: "James Kanze" <james.kanze@gmail.com>
Date: Tue, 27 Mar 2007 12:18:30 CST
Raw View
On Mar 26, 7:52 pm, "Greg Herlihy" <gre...@pacbell.net> wrote:
> On Mar 26, 7:40 am, "James Kanze" <james.ka...@gmail.com> wrote:

> > On Mar 26, 1:40 am, nore...@this.is.invalid (Niels Dekker - no return
> > address) wrote:
> > > Based on Greg's example, the following code demonstrates the issue:
> > >   #include <iostream>
> > >   #include <fstream>
> > >   #include <streambuf>
> > >   typedef char C;
> > >   int main ()
> > >   {
> > >     std::ifstream s("filename", std::ios::in);
> > >     std::istreambuf_iterator<char> i(s);
> > >     (*i).~C();  // This is well-formed...

> > Well formed, perhaps, but it looks like undefined behavior at
> > execution.  (Maybe not in the case of char; char's rather
> > special in this regard.  But definitily in the case of other
> > types.)

> There would no point in allowing pseudo-destructor calls in C++ unless
> such calls had well-defined behavior.

The call itself may have well-defined behavior, but you can't
use it just anywhere.  In the case of a streambuf_iterator, the
streambuf_iterator is managing the life of the object itself,
and any destruction of the object from outside the
streambuf_iterator will fatally cause problems with that
management.

> Otherwise, the Standard would
> simply be trading a compilation error for undefined behavior - which
> would not be a sensible trade. So we can be assured that a psuedo-
> destructor call for any scalar type is well-defined

No one is doubting that.  The problem is that what remains isn't
an object.  And that if what remains remains somewhere where
someone can use it, and probable destruct it themselves, the
results will be undefined behavior.  You wouldn't expect to be
allowed to do iter->~Toto() on an iterator from
std::vector<Toto>, would you?  What makes streambuf_iterator
different?

--
James Kanze (GABI Software)            mailto:james.kanze@gmail.com
Conseils en informatique orient?e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S?mard, 78210 St.-Cyr-l'?cole, France, +33 (0)1 30 23 00 34

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: "Douglas Gregor" <doug.gregor@gmail.com>
Date: Tue, 27 Mar 2007 14:53:51 CST
Raw View
On Mar 26, 1:56 pm, tasjae...@gmail.com wrote:
> On Mar 26, 3:13 pm, howard.hinn...@gmail.com (Howard Hinnant) wrote:
> > operator*() is going to be calling sgetc() on the internal streambuf.
> > But sgetc() returns an int_type, not a char_type.  So operator*() is
> > going to be making a cast, generating a temporary char_type, and
> > returning that by value.  So even inside of operator*() there is no
> > lvalue (with sufficient lifetime) we can point to.  Perhaps we could
> > return the address of an unspecified char_type?  Are we fighting an
> > uphill battle?
>
> I think it's an uphill battle without Boost's improved iterator
> concepts, which were dropped from Standardisation.
>
>  http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/papers/2003/n1477.html
>
> In that terminology, istreambuf_iterator is only 'Readable', not
> 'Readable Lvalue'.

The iterator concept refactoring in N1477 is still a good idea, even
though it wasn't the best fit for TR1.
We're planning to bring those iterator concepts into C++0x (providing
backward compatibility for C++98 iterators the same way we do now,
with concept maps).

> If we were switching to those concepts, I would have suggested
> operator-> return a proxy (much like boost::iterator_facade does for
> non-lvalue iterators).

One can still have proxies with Input Iterators.

> As it is, am I right in thinking that returning a proxy would clash
> with the concepts library proposal?

They should not clash; if they do, it's an error in our specification
of the concepts (or in ConceptGCC's handling of those concepts). The
"pointer" and "reference" associated types of an Input Iterator are
permitted to be proxies.

  Cheers,
  Doug

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]