I agree that it's a problem.  This comes up whenever
A allocates B, and B keeps a backpointer to A.
For a "managed object", A should pass an encapsulated
pointer to B.  But there's the problem that with auto_ptr
and many smart pointer systems, there's no way to get
from "this" to the owning smart pointer.  That's a problem
of the smart pointer system, not a language-level problem.
Smart pointer implementions can be written that let you
get a smart pointer to the object you're in.  For
auto_ptr objects, of course, it's a violation of the
auto_ptr single owner concept to create an object
with a backpointer.

    One approach, as I mentioned, would be to provide
a way to designate classes as "managed".  Microsoft
uses the declaration attribute  "__gc" in their
dialect of C++.  That's ugly, but it's a precedent.

    "managed objects" seem to need the following
restrictions:

-- raw pointers to them are limited to restricted-scope
   copying, as mentioned in the previous "pointer const"
   discussion.
-- "this" gets the same restrictive treatment.
-- "delete" is not allowed on "pointer const" pointers.

Is this enough to make auto_ptr airtight?  And general
enough to make smart pointers work?  Does anybody care
about safety in C++, or did all those people move to
other languages?

   John Nagle
   Animats

Fergus Henderson wrote:
> metadata@sbcglobal.net (Matt Seitz) writes:
>
>
>>Fergus Henderson wrote:
>>
>>>There is no guarantee that the pointer is discarded.  If operator->
>>>is used to call a member function, the pointer is passed to the member
>>>function as the "this" parameter, and the called function may easily
>>>store it in global variables, return it, etc.
>>
>>That's a good point.  Are there examples of such member functions in
>>production code, or is it just a hypothetical problem?
>
>
> I think there are examples in production code.
> For example I think patterns such as
>
>  class Foo() {
>  public:
>   register() { registry.register(this); }
>   ...
>  };
>
> are not uncommon.  Here "registry" is a global collection of objects,
> and "registry.register(this)" stores the "this" pointer in that collection
> so that other code can later iterate over all registered objects.
>

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

In article <40285caa$1@news.unimelb.edu.au>, Fergus Henderson
<fjh@cs.mu.OZ.AU> writes
>I think there are examples in production code.
>For example I think patterns such as
>
>       class Foo() {
>       public:
>               register() { registry.register(this); }
>               ...
>       };
>
>are not uncommon.  Here "registry" is a global collection of objects,
>and "registry.register(this)" stores the "this" pointer in that collection
>so that other code can later iterate over all registered objects.

I understand what you mean but, of course, no such code as you wrote
will compile (register is a reserved word:)


--
Francis Glassborow      ACCU
Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
For project ideas and contributions: http://www.spellen.org/youcandoit/projects

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

John Nagle wrote:
>  The underlying language problem is that any pointer encapsulation must
> at some point return a language-level
> pointer or reference.  That pointer or reference can then
> be easily misused in ways that break the encapsulation.

Only if there is an operation that requires a true pointer.  If one could allow
the operations through the encapsulation's interface, then the encapsulation
could be maintined.

This problem sounds similar to ones faced by std::vector and std::string:  they
allow access to a pointer because there are some functions that currently
require a pointer.  That allows the user to bypass the encapsulation.  Now, if
those operations could be rewritten to take a std::vector or std::string, then
they could use the class's member functions and encapsulation could be
maintained.  The use of the naked pointer could be reduced and eventually
eliminated.

So, perhaps the solution is not to change how pointers work, but instead to
rewrite the operations that require a pointer to instead use auto_ptr member
functions.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Matt Seitz wrote:
> John Nagle wrote:
>
>>  The underlying language problem is that any pointer encapsulation
>> must at some point return a language-level
>> pointer or reference.  That pointer or reference can then
>> be easily misused in ways that break the encapsulation.
>
>
> Only if there is an operation that requires a true pointer.  If one
> could allow the operations through the encapsulation's interface, then
> the encapsulation could be maintined.

    auto_ptr, and smart pointers generally, don't know anything about
the objects whose pointers they encapsulate.  And the objects whose
pointers are being encapsulated don't know that they're being
managed by auto_ptr or a smart pointer.  That's why some language
support is needed.

    The most common operation that requires a true pointer is field access,
or "operator."  Trying to implement field access by overloading
"operator." isn't allowed.  (Overloading "->" is supposedly allowed,
but doesn't help implement field access.)

> This problem sounds similar to ones faced by std::vector and
> std::string:  they allow access to a pointer because there are some
> functions that currently require a pointer.

   "operator[]" of "std::vector<T>" would then return what?

> So, perhaps the solution is not to change how pointers work, but instead
> to rewrite the operations that require a pointer to instead use auto_ptr
> member functions.

   If that was possible, the people trying to fix auto_ptr would have
fixed it by now.  The "ref const" and "pointer const" semantics I'm
proposing may seem radical, but all attempts to fix this via templates
have failed.

    John Nagle

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

John Nagle wrote:
> Matt Seitz wrote:
>
>> John Nagle wrote:
>
>> This problem sounds similar to ones faced by std::vector and
>> std::string:  they allow access to a pointer because there are some
>> functions that currently require a pointer.
>
>
>   "operator[]" of "std::vector<T>" would then return what?

The same thing it does now.  My suggestion is to change the functions that take
pointer arguments to instead either take vector arguments, or at least take
iterator arguments.

>
>> So, perhaps the solution is not to change how pointers work, but
>> instead to rewrite the operations that require a pointer to instead
>> use auto_ptr member functions.
>
>
>   If that was possible, the people trying to fix auto_ptr would have
> fixed it by now.

The fact that something hasn't been done doesn't mean something can't be done.
The fact that many standard library functions, such as ofstream::ofstream, still
  require a char * instead of accepting a std::string argument or a generic
iterator argument doesn't mean it couldn't be done.

> The "ref const" and "pointer const" semantics I'm
> proposing may seem radical, but all attempts to fix this via templates
> have failed.

I'm not suggesting fixing this via templates.  I'm suggesting that instead of
changing the way pointers are defined we should try to reduce the need for pointers.

Perhaps I'm not understanding the problem that is trying to be solved.  I
thought the problem this proposal addressed was:
a) functions expect pointers
b) allowing access to the encapsulated pointer, either through functions like
auto_ptr::get or by some automatic conversion operation like operator T*, breaks
the encapsulation.

This proposal seems to be attacking part b), by allowing access to a pointer but
still maintaing some of the properties given by encapsulation.  I'm suggesting
instead taking the approach of attacking problem a), trying to reduce the need
to use pointers.

Perhaps an example of code that demonstrates the problem would help me
understand the problem better?

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Matt Seitz wrote:
> John Nagle wrote:
>
>> Matt Seitz wrote:
>>
>>> John Nagle wrote:
>>
>>
>>> This problem sounds similar to ones faced by std::vector and
>>> std::string:  they allow access to a pointer because there are some
>>> functions that currently require a pointer.
>>
>>
>>
>>   "operator[]" of "std::vector<T>" would then return what?
>
>
> The same thing it does now.  My suggestion is to change the functions
> that take pointer arguments to instead either take vector arguments, or
> at least take iterator arguments.

   The vector is the entire collection.  At some point, you need
to be able to talk about collection elements.

   There's something to be said for using iterators more and pointers
less.  Iterators have better-defined semantics.  But it doesn't help
deal with the ownership issues auto_ptr and smart pointers address.

>>   If that was possible, the people trying to fix auto_ptr would have
>> fixed it by now.
>
> The fact that something hasn't been done doesn't mean something can't be
> done.

   Many smart people have tried to make auto_ptr work.  We're now on
the fourth major revision, the first three having been found wanting.
We don't have a formal proof that it's impossible (although it may
be possible to generate one), but by now it is clearly recognized
as a very hard problem.

   P. J. Plauger wrote recently:

 "auto_ptr was *invented* to take advantage of a loophole in Standard
 C++. It has caused various degrees of pain for *all* compilers over
 the years, and hardly anybody on the C++ committee is happy with it
 in its current form. It is most prudent to avoid corner cases, at
 least in practical code. If you're looking for trouble, however,
 there's plenty to find.."

> Perhaps I'm not understanding the problem that is trying to be solved.
> I thought the problem this proposal addressed was:
> a) functions expect pointers
> b) allowing access to the encapsulated pointer, either through functions
> like auto_ptr::get or by some automatic conversion operation like
> operator T*, breaks the encapsulation.

   The problem is that functions NEED pointers, or references, or
iterators.  At some point, you need field access.  Encapsulation
doesn't allow field access, unless the encapsulation is handwritten
for each type encapsulated.  Template based encapsulation does not
work here.

> Perhaps an example of code that demonstrates the problem would help me
> understand the problem better?

   Reading the previous discussions of auto_ptr will provide
the necessary background.  This is well-trodden ground.

    John Nagle
    Animats

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

John Nagle wrote:

>     The problem is that functions NEED pointers, or references, or
> iterators.  At some point, you need field access.  Encapsulation
> doesn't allow field access, unless the encapsulation is handwritten
> for each type encapsulated.

Is operator-> not a solution here?  While it technically does return a copy of a
pointer, that copy is simply immediately derefrenced and then discarded.  One
could abuse that idiiom by calling operator->() explicitly and using the naked
pointer.  But that seems more like deliberate abuse than the type of accidental
misuse that it sounds like this proposal is trying to address.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

   "operator->" won't help.  It has drastically different
semantics than the builtin "->".

   "operator->" does not take a field name as an operand
Only the builtin "->" and "." take field names as a second
operand.

   Also, there's no "operator." in C++.

   This problem is not solveable by encapsulation. It's
a real language problem.

    John Nagle
    Animats

Matt Seitz wrote:
> John Nagle wrote:
>
>>     The problem is that functions NEED pointers, or references, or
>> iterators.  At some point, you need field access.  Encapsulation
>> doesn't allow field access, unless the encapsulation is handwritten
>> for each type encapsulated.
>
>
> Is operator-> not a solution here?  While it technically does return a
> copy of a pointer, that copy is simply immediately derefrenced and then
> discarded.  One could abuse that idiiom by calling operator->()
> explicitly and using the naked pointer.  But that seems more like
> deliberate abuse than the type of accidental misuse that it sounds like
> this proposal is trying to address.
>
> ---
> [ comp.std.c++ is moderated.  To submit articles, try just posting with ]
> [ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
> [              --- Please see the FAQ before posting. ---               ]
> [ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]
>

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

metadata@sbcglobal.net (Matt Seitz) writes:

>John Nagle wrote:
>
>>     The problem is that functions NEED pointers, or references, or
>> iterators.  At some point, you need field access.  Encapsulation
>> doesn't allow field access, unless the encapsulation is handwritten
>> for each type encapsulated.
>
>Is operator-> not a solution here?  While it technically does return a copy
>of a pointer, that copy is simply immediately derefrenced and then discarded.

There is no guarantee that the pointer is discarded.  If operator->
is used to call a member function, the pointer is passed to the member
function as the "this" parameter, and the called function may easily
store it in global variables, return it, etc.

--
Fergus Henderson <fjh@cs.mu.oz.au>  |  "I have always known that the pursuit
The University of Melbourne         |  of excellence is a lethal habit"
WWW: <http://www.cs.mu.oz.au/~fjh>  |     -- the last words of T. S. Garp.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

John Nagle wrote:
>   "operator->" won't help.  It has drastically different
> semantics than the builtin "->".

How are the semantics drastically different?  Both are used in the same way:

X * x1 = new X
auto_ptr<X> x2 = new X;

x1->field =0; //Uses built-in ->
x2->field =0; //Uses auto_ptr::operator->

>   "operator->" does not take a field name as an operand

Technically, that is true.  However, when operator-> is normally used (as
above), the built in -> operator is automatically called.  The built in ->
operator does take a field name, allowing field access through operator->.

>   Also, there's no "operator." in C++.

You can't use the built-in . operator on a pointer in any case.  So why would
you need one on an encapsulated pointer type?

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

    Fergus raises a relevant issue.  There's no way
to declare a "this" pointer a "pointer const".
You can declare an object as const, but not the "this"
pointer itself.  There's no syntax.  So the "pointer
const" concept hits a snag there.  Fixing this problem
within the existing syntax may not be possible.

    What we're running into here is the problem that
caused Microsoft to put "managed objects" into their
dialect of C++ (".NET").  Microsoft uses the keyword
"__gc" to indicate a managed object.  Microsoft's approach is
heavily tied to Microsoft's libraries and garbage
collector, but it gives us something to look at.
Microsoft had to go to a whole new language (C#) to
make the "managed object" concept work safely.
>From a C++ perspective, the question to ask is whether
there is some backward-compatible set of changes that
can make auto_ptr and smart pointers (which are C++'s
"managed objects") work safely.

     From a user perspective, the cleanest
implementation is one where, to experience the
pleasure of managed objects, you somehow tag an
object as "managed" in one place, never have
to worry about storage management anywhere else,
and can't inadvertently break the storage management.
Just about every widely used language today, other
than C and C++, has this.

     That's the goal.  Can we get there?

   John Nagle
   Animats

Fergus Henderson wrote:
> metadata@sbcglobal.net (Matt Seitz) writes:
>
>
>>John Nagle wrote:
>>
>>
>>>    The problem is that functions NEED pointers, or references, or
>>>iterators.  At some point, you need field access.  Encapsulation
>>>doesn't allow field access, unless the encapsulation is handwritten
>>>for each type encapsulated.
>>
>>Is operator-> not a solution here?  While it technically does return a copy
>>of a pointer, that copy is simply immediately derefrenced and then discarded.
>
>
> There is no guarantee that the pointer is discarded.  If operator->
> is used to call a member function, the pointer is passed to the member
> function as the "this" parameter, and the called function may easily
> store it in global variables, return it, etc.


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Fergus Henderson wrote:
> metadata@sbcglobal.net (Matt Seitz) writes:
>
>
>>John Nagle wrote:
>>
>>
>>>    The problem is that functions NEED pointers, or references, or
>>>iterators.  At some point, you need field access.  Encapsulation
>>>doesn't allow field access, unless the encapsulation is handwritten
>>>for each type encapsulated.
>>
>>Is operator-> not a solution here?  While it technically does return a copy
>>of a pointer, that copy is simply immediately derefrenced and then discarded.
>
>
> There is no guarantee that the pointer is discarded.  If operator->
> is used to call a member function, the pointer is passed to the member
> function as the "this" parameter, and the called function may easily
> store it in global variables, return it, etc.

   Very true. "this" exposes a raw pointer to the object.
Applications can thus break smart pointers or auto_ptr.

   "this" could be made a "pointer const", as
previously discussed, but too much existing code would break
if that feature were on all the time.

   It's not just assignment to
"this" that would break.  Child objects with backpointers
violate the scope lifetime rules.  They're notorious for
causing allocation and deallocation order problems,
so this is an area worth addressing, but it's hard
to do right.

   But not impossible.  Doing backpointers right in the
presence of destructors requires
both reference counts and weak pointers in the Perl sense
(not the Java sense).  If A allocates and owns B, A has a strong
pointer to B, and B has a weak pointer back to A.
Dereferencing a weak pointer to an
object in construction or destruction must throw an exception.  With
this, it all works right.  Perl programmers are barely aware that
their allocation problems are solved that way, and it works.
But it's not the way C++ went.

   What we're headed for here is "managed objects".
Is that a useful direction?  Is there a feeling that
pointers in C++ should be made safe?  Or should that be
left to the newer languages like Java and C#?

    John Nagle


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Fergus Henderson wrote:
> There is no guarantee that the pointer is discarded.  If operator->
> is used to call a member function, the pointer is passed to the member
> function as the "this" parameter, and the called function may easily
> store it in global variables, return it, etc.

That's a good point.  Are there examples of such member functions is production
code, or is it just a hypothetical problem?

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

metadata@sbcglobal.net (Matt Seitz) writes:

>Fergus Henderson wrote:
>> There is no guarantee that the pointer is discarded.  If operator->
>> is used to call a member function, the pointer is passed to the member
>> function as the "this" parameter, and the called function may easily
>> store it in global variables, return it, etc.
>
>That's a good point.  Are there examples of such member functions in
>production code, or is it just a hypothetical problem?

I think there are examples in production code.
For example I think patterns such as

 class Foo() {
 public:
  register() { registry.register(this); }
  ...
 };

are not uncommon.  Here "registry" is a global collection of objects,
and "registry.register(this)" stores the "this" pointer in that collection
so that other code can later iterate over all registered objects.

--
Fergus Henderson <fjh@cs.mu.oz.au>  |  "I have always known that the pursuit
The University of Melbourne         |  of excellence is a lethal habit"
WWW: <http://www.cs.mu.oz.au/~fjh>  |     -- the last words of T. S. Garp.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

  The ongoing struggle to make auto_ptr work makes it clear
that the language just doesn't have the capabilities needed
to do this right.  The same problem comes up with most
smart pointer implementations.

  The underlying language problem is that any pointer
encapsulation must at some point return a language-level
pointer or reference.  That pointer or reference can then
be easily misused in ways that break the encapsulation.

   What's needed is a pointer or reference with some
additional restrictions attached to it.  There
are three issues:

   1) What should the restrictions be?
   2) What syntax should be used to express the restrictions?
   3) What is the smallest change that can be made to the
 language to fix the problem.

   The restriction needed to make auto_ptr safe is one that
insures the returned reference or pointer can't outlive the
auto_ptr.  Therefore, the returned reference or pointer
should be of limited lifetime, i.e. "auto".  More than
that, it should not be possible to save that pointer
in a variable with longer lifetime than its source.
So copying such a pointer or reference to an outer scope
should be prohibited.

   This is the key concept - enforcement of pointer lifetime
by scope.

   As for syntax, I suggest using the existing "pointer const"
notation, as in

 T obj;  // object of type T
 T& const p = obj; // reference form
 T* const p = obj; // pointer form

This changes, slightly, the semantics of a rarely used bit of syntax.
Both forms are currently legal, although the "ref const" form is
obsolete.

   In practice, this would be used as follows.

 class T {  // a type
 public:
  int field1; // with a public field
 };
 T* const p0 = 0; // for use later
 void somefunct(T* const p); // declare a function


 T obj1;  // local object of type T
 auto_ptr<T> obj2(new T); // heap object of type T as auto_ptr
 ...
 T* const p1 = &obj1; // OK, LHS has shorter life
 T* const p2 = obj2; // OK, LHS has shorter life
 T* const p3 = p2; // OK, LHS has shorter life
 T& const r1 = &p2; // OK, LHS has shorter life
 r1.field1 = 100; // OK, ref is const, not obj
 T* p4 = p2; // ERROR, loses qualifiers
 somefunct(p3); // OK, param has shorter life
 static T* const p2a = obj2; // ERROR: LHS has longer life
 p0 = obj2; // ERROR: LHS has longer life


 ...
 void somefunct(T* const p)
 {    int n = p->field1; // OK
      p->field1 = 1; // OK, pointer is const, not obj
      T* q const = q; // OK, LHS has shorter life
      delete(p); // ERROR: can't allow deletion
      p++; // ERROR, pointer is const
 }


This works even better for "smart pointers", where the overhead
of reference counting is typically a problem.  A "smart pointer"
can safely return a "ref const" or "pointer const" for local,
low-overhead use.  It's not necessary to use the smart pointer
object, with its reference count updates, every time.

Issues for discussion:
-- Should STL collection classes return "ref const"?
     vector <T> tab;
     T& const item = tab[i]; // OK
     T& item = tab[i];  // should this be disallowed?

   It's probably too late to retrofit this to the STL,
   even though it's the right thing to do.

-- Should library function arguments that could be
  "ptr const" or "ref const" be changed?

   It's the right thing to do, but it's work.

-- Is a cast needed to make the "ptr const" restriction go
   away?

   Is there a way to do this now, short of old-style casts?


Does anyone have a better alternative?  This issue has been
unsolved for years, and this might be a way out.

    John Nagle
    Animats


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]