David Abrahams wrote:
> nagle@animats.com (John Nagle) writes:
>
>
>>David Abrahams wrote:
>>
>>>nagle@animats.com (John Nagle) writes:
>>>
>>>and it's pure syntax with no semantic difference from
>>>using operator->().
>>
>>   One could make the same argument against references generally.
>
>
> Nobody's proposing to *add* references at this moment.
>
> And no, you can't make the same argument because there are semantic
> differences between references and pointers.
>
>
>>>It doesn't solve any safety problems, does it?
>>
>>   It removes another temptation to use raw pointers.
>
>
> I don't see how.

    I don't have time to explain this today.  Could someone
else write an explaination, please?

    John Nagle

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

> OK, you are using the event-based model. You ::destroy B, ~B notifies
> everyone holding a reference to B so they can set their references to
> NULL. ~B also drops its reference to A. However, this doesn't destroy
> A, because other parts of the program still keep references to it. So
> the scheme doesn't work in general.

It works fine if every object has at least one owned_ptr to it, that
will explicitly destroy it when the object holder that owned_ptr is
destroyed (or someone calls explicitly calls ::destroy, which is the
same thing). Everything else is just recursion, and making sure no
dangling pointer check fails along the way.

> > In this way, all objects in our app are tied together, and if I want to
> > destroy one, I just shared_ptr::destroy it. It works beautifully, often
> > without thinking much about it. Building the same with
> > shared_ptr/weak_ptrs would give me the permanent headache of either
> > having overlooked some newly introduced circular reference, thereby not
> > freeing memory at all, or having created some nulled weak_ptr that will
> > haunt me later.
>
> Have you tried?

I admit, no. I had to pick a scheme at the onset of my project, and I
picked the one I deemed safest. Neither have you tried my model, but
yours is in TR1 already:-(

> It's really not that different from your model, except that a weak_ptr
> nulls itself automatically, without waiting for a notification from the
> target. This reduces dependencies. It means that you have to lock() and
> test it on every access, but if your notification scheme nulls a
> shared_ptr to an object that is destroyed, you have to test it, too.

The crucial difference in my model is that you _have the option_ to do
the test at the time of object destruction, not at the time of access.
I do not want to abolish weak_ptr, I support the same concept in my
implementation.

First,

A) pointer is nullable without harm, its holder stays functional and
B) pointer is part of circular reference chain

are _not_ the same thing. In fact, they are completely orthogonal. In
particular, the case "not A but possibly B" should be expressable in
the code. Upholding the not-nulled invariant should be checked by the
machine, not the programmer. And it should not require some fancy
contraption that only C++ experts can come up with, and should be as
efficient as possible, in case you want to keep it in Release builds.

Second, the check whether an object is being destroyed should also be
automated. If the programmer expects it to happen at a certain time,
why should not he be able to say so, to guard against memory leaks?
Currently, their is no good way for an object to check for its own
non-existence:

In your concept, you would typically implement a method ::go_away for
each object, which notifies all objects holding shared_ptrs to release
them. After you did this, you need to check that the destruction in
fact happened (because you may have forgotten to release some
shared_ptr). But there is no efficient way (again, Release build) to do
it. You could use a local variable weak_ptr to yourself and check for
it to be nulled, but you need a shared_ptr to initialize it. This
requires each object to carry around a shared_ptr member to itself, or
being passed one as a parameter of ::go_away. Plus, this approach
requires artificially splitting code between ::go_away and the dtor,
although this code presumably always runs in sequence.

Before we home in on a solution to the two issues, do you accept that
these two things above are good functionality?

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Arno wrote:
> > OK, you are using the event-based model. You ::destroy B, ~B notifies
> > everyone holding a reference to B so they can set their references to
> > NULL. ~B also drops its reference to A. However, this doesn't destroy
> > A, because other parts of the program still keep references to it. So
> > the scheme doesn't work in general.
>
> It works fine if every object has at least one owned_ptr to it, that
> will explicitly destroy it when the object holder that owned_ptr is
> destroyed (or someone calls explicitly calls ::destroy, which is the
> same thing). Everything else is just recursion, and making sure no
> dangling pointer check fails along the way.

I'm afraid that I still don't get it. Given two interconnected objects
A and B and a program that holds references to both A and B, I don't
see how ::destroy would enable me to destroy A and B safely if I only
know about A, or only know about B.

> In your concept, you would typically implement a method ::go_away for
> each object, which notifies all objects holding shared_ptrs to release
> them.

No, not really.

In a typical reference-counted/shared ownership model, an object stays
alive as long as there are shared_ptrs to it. It doesn't have a
'go_away' method; it can't go away if there are owners. (I can't call
it "my concept" because I haven't invented it; it's been known for at
least a decade.)

Non-owners hold weak_ptrs to the object, not shared_ptrs, and the
object doesn't need to know about them or notify them. (This is one of
the relatively novel parts of shared_ptr.)

Or, to put it in possibly clearer terms, objects that depend on our
object A's existence hold shared_ptrs to A. Objects that may outlive A
hold weak_ptrs to A.

Even if we assume an event-based model and a 'go_away' method (which
your scheme also needs to have - but you probably would put it in ~A),
there's still no problem. The part of the program that wants to execute
the 'go_away' action has to hold a shared_ptr to A (if it didn't, it
wouldn't have had access to its go_away member function). It can use
the temporary weak_ptr trick to verify that its shared_ptr is the only
remaining instance after go_away has notified everyone else to drop
their references.

Children of A (objects whose lifetime is bounded by the lifetime of A)
that want to refer to it usually hold an ordinary raw pointer under a
typical shared_ptr-based design. This can create dangling pointers, but
in practice tends not to. I've described in my first post to this
thread how one can create a "safe" backpointer that detects dangling
backpointer errors in such situations.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

> I'm afraid that I still don't get it. Given two interconnected objects
> A and B and a program that holds references to both A and B, I don't
> see how ::destroy would enable me to destroy A and B safely if I only
> know about A, or only know about B.

As long as A knows B, and B knows A, why should some C wanting to
destroy A need to know B? A can take care of B by itself.

What is "safety" for you? Is garbage collection "safe"? It avoids all
dangling pointers. Non-dangling pointers may be safe in the sense of
avoiding memory corruption and GPFs, but if they are pointing to zombie
objects that are in a semantically invalid state, what good is that for
the correctness of the program?

Noone can guarantee "safety". Or rather, correctness. Said Turing. But
you can create tools that permit writing into your program what you
consider correct. owned_ptrs encode the lifetime implication "this
object must not outlive me", and shared_ptrs encode "this object must
be there as long as I am here, but it may outlive me." And correctness
which has been encoded in such a way can be efficiently checked at
runtime.

> In a typical reference-counted/shared ownership model, an object stays
> alive as long as there are shared_ptrs to it. It doesn't have a
> 'go_away' method; it can't go away if there are owners. (I can't call
> it "my concept" because I haven't invented it; it's been known for at
> least a decade.)

I understand this concept, it is indeed widely known. My point is that
there are often multiple such 'owners', any one of which must trigger
destruction, and going from one (for which shared_ptr is sufficient) to
multiple owners requires introduction of a go_away method. And you (or
your new hire) must realize in the first place that such a second owner
has come into existence, and it is time to write go_away.

> Or, to put it in possibly clearer terms, objects that depend on our
> object A's existence hold shared_ptrs to A. Objects that may outlive A
> hold weak_ptrs to A.

Exactly. But this is not how shared_ptr/weak_ptr are used! In a typical
parent/child relationship, the parent has a shared_ptr to the child,
and the child a weak_ptr to the parent.

Now compare this to what you said. The parent may outlive the child and
thus should hold a weak_ptr, but the child may not outlive the parent,
and thus should have a shared_ptr. It is all backwards.

>
> Even if we assume an event-based model and a 'go_away' method (which
> your scheme also needs to have - but you probably would put it in ~A),

Yes, together with all other destruction code, not split it between
go_away and the dtor.

> there's still no problem. The part of the program that wants to execute
> the 'go_away' action has to hold a shared_ptr to A (if it didn't, it
> wouldn't have had access to its go_away member function). It can use
> the temporary weak_ptr trick to verify that its shared_ptr is the only
> remaining instance after go_away has notified everyone else to drop
> their references.

This check should be in 'go_away', not scattered in all of 'go_away's
callers.

>
> Children of A (objects whose lifetime is bounded by the lifetime of A)
> that want to refer to it usually hold an ordinary raw pointer under a
> typical shared_ptr-based design.

Compare this with your lifetime statement above.

> This can create dangling pointers, but
> in practice tends not to.

It did in my practice. Especially when there are two owners, one owner
dies, but its child does not because of the other owner. The child
still exists, but now has a dangling backpointer.

> I've described in my first post to this
> thread how one can create a "safe" backpointer that detects dangling
> backpointer errors in such situations.

Which required extra infrastructure (including a heap-allocated
tracker). I do not want to have extra heap allocations in my Release
builds; the heap frequently is a performance bottleneck.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Arno wrote:
> > I'm afraid that I still don't get it. Given two interconnected objects
> > A and B and a program that holds references to both A and B, I don't
> > see how ::destroy would enable me to destroy A and B safely if I only
> > know about A, or only know about B.
>
> As long as A knows B, and B knows A, why should some C wanting to
> destroy A need to know B? A can take care of B by itself.

A knows B. B knows A. C holds a reference to A. D holds a reference to
B. C wants to destroy A. What happens?

> > Children of A (objects whose lifetime is bounded by the lifetime of A)
> > that want to refer to it usually hold an ordinary raw pointer under a
> > typical shared_ptr-based design.
>
> Compare this with your lifetime statement above.
>
> > This can create dangling pointers, but
> > in practice tends not to.
>
> It did in my practice. Especially when there are two owners, one owner
> dies, but its child does not because of the other owner. The child
> still exists, but now has a dangling backpointer.

I think I'm starting to understand you. :-)

You have a parent A and several children B1..Bn that hold backpointers.

You have several owners C1..Cm that collectively own A.

You have several objects D1..Dk that hold references to Bi.

When A goes away, you want to catch any dangling references to Bi.

The root of the problem is that you've let shared_ptr<B>'s to escape
into the world, because shared_ptr isn't the appropriate tool to
describe the relationship between D and B. But I see why in some
situations you'd have no other choice.

The possible solutions are:

1. weak_ptr backpointers. You don't like them because in your design A
destruction in the above situation is a programming error and weak
backpointers catch it too late.

2. Raw backpointers and a check in ~A (and wherever else in A a child
is being destroyed) using the weak_ptr temp trick that this B has
really been destroyed.

3. Raw backpointers and giving out shared_ptr<B> with a custom deleter
that actually holds A alive. (The equivalent of garbage collection in
this situation; A will outlive all Ci and Dj.) Unfortunately a
weak_ptr<B> obtained from this shared_ptr won't work correctly.

How does your proposed solution handle the situation? Which links are
owned_ptr and which are shared_ptr?

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

> > As long as A knows B, and B knows A, why should some C wanting to
> > destroy A need to know B? A can take care of B by itself.
>
> A knows B. B knows A. C holds a reference to A. D holds a reference to
> B. C wants to destroy A. What happens?

We assume A needs B, and B needs A, and neither should outlive the
other. So A triggers B's destruction. B needs to do whatever is needed
to cleanly destruct (no matter who triggered its destruction), so it is
responsible for notifying D to release its reference.

> > It did in my practice. Especially when there are two owners, one owner
> > dies, but its child does not because of the other owner. The child
> > still exists, but now has a dangling backpointer.
>
> I think I'm starting to understand you. :-)
>
> You have a parent A and several children B1..Bn that hold backpointers.
>
> You have several owners C1..Cm that collectively own A.
>
> You have several objects D1..Dk that hold references to Bi.
>
> When A goes away, you want to catch any dangling references to Bi.
>
> The root of the problem is that you've let shared_ptr<B>'s to escape
> into the world, because shared_ptr isn't the appropriate tool to
> describe the relationship between D and B. But I see why in some
> situations you'd have no other choice.

Regarding "appropriate tools", shouldn't we first agree on plausible
lifetime dependency scenarios, before we decide which tools are
appropriate to cover them? I see three main ones:

- notification holder (for example, in a view-document framework, a UI
control needs to know when the screen resolution changes)
- ownership, possibly from multiple objects (for example, in the same
view-document framework, the UI control exists if both the document
object that is being controlled exists, and the view showing the UI
control exists)
- necessary backpointer (for example, each UI control needs to know
both its view and its document)

Do these scenarios seem plausible to you?

> The possible solutions are:
>
> 1. weak_ptr backpointers. You don't like them because in your design A
> destruction in the above situation is a programming error and weak
> backpointers catch it too late.

You mean weak backpointers in D1..Dk to B and in B to A? Yes, they
should be protected from being set to NULL.

> 2. Raw backpointers and a check in ~A (and wherever else in A a child
> is being destroyed) using the weak_ptr temp trick that this B has
> really been destroyed.

A "destruction-guaranteeing shard_ptr" could indeed look like this:

weak_ptr p = last_shared_ptr
last_shared_ptr.release
assert( p==NULL ) // sorry about the syntax, you know what I mean

This does solve the problem that B is checked to die when A does, and
it fails when there are still other holders of B. If these other
holders exist, there is no canonical place to notify them, though. And
I do not see how this affects the weak_ptr problem described in 1. Raw
pointers would dangle instead of being NULL.

> 3. Raw backpointers and giving out shared_ptr<B> with a custom deleter
> that actually holds A

do you mean B?

> alive. (The equivalent of garbage collection in
> this situation; A will outlive all Ci and Dj.) Unfortunately a
> weak_ptr<B> obtained from this shared_ptr won't work correctly.

I think I do not understand this solution.

> How does your proposed solution handle the situation? Which links are
> owned_ptr and which are shared_ptr?

Rule:

- If X may outlive Y, but Y may not outlive X, (typical parent X -
child Y relationship) use owned_ptr for X->Y and shared_ptr for Y->X.
- If X and Y live and die in unison, use owned_ptr for both X->Y and
Y->X.

So, if C1..Cn jointly own A and A has a backpointer to C1..Cn, C1..Cn
should have owned_ptrs to A, A shared_ptrs to C1..Cn. If anyone of
C1..Cn dies, A will die as well.

If A and B live and die together (my original idea), both hold
owned_ptrs to each other.

If D1..m must be notified if B is destroyed, they hold shared_ptrs to
B. B must also hold some pointers to D for the notifying call. If B
needs no notification if D1..m die, and someone else holds D1..m alive,
these could be weak_ptrs.

All these decisions can be made locally, without looking at the "bigger
picture".

When C1 gets destroyed, C1's owned_ptr destroys A, ~A tells C2..Cn, A's
owned_ptr destroys B, ~B tells D1..Dm and B's owned_ptr tries to
destroy A. The "inside dtor" check prevents ~A to run again.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Arno wrote:
>>>As long as A knows B, and B knows A, why should some C wanting to
>>>destroy A need to know B? A can take care of B by itself.
>>
>>A knows B. B knows A. C holds a reference to A. D holds a reference to
>>B. C wants to destroy A. What happens?
>
>
> We assume A needs B, and B needs A, and neither should outlive the
> other. So A triggers B's destruction. B needs to do whatever is needed
> to cleanly destruct (no matter who triggered its destruction), so it is
> responsible for notifying D to release its reference.
>
>
>>>It did in my practice. Especially when there are two owners, one owner
>>>dies, but its child does not because of the other owner. The child
>>>still exists, but now has a dangling backpointer.

    That's exactly what weak pointers are for.

     See this tutorial on how to use Perl weak references to
solve this problem.

  http://www.simplicidade.org/notes/archives/2005/07/weaken_referenc.html

     For a more formal view, see

http://perl.enstimac.fr/perl5.8.5/site_perl/5.8.5/sun4-solaris/WeakRef.html

by Tuomas Lukka, the designer of Perl's weak references. As he puts it

"A weak reference is just like an ordinary Perl reference except that it
isn't included in the reference count of the thing referred to. This
means that once all references to a particular piece of data are weak,
the piece of data is freed and all the weak references are set to undef.
This is particularly useful for implementing circular data structures
without memory leaks or caches of objects."

     That's how it's done.

    John Nagle
    Animats

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Arno wrote:
> > and support for multiple owning pointer copies can be added with:
> >
> > weak_ptr<X> tmp( object_ptr_ );
> > weak_ptr<void> tmp2( tracker_ );
> > object_ptr_.reset();
> > tracker_.reset();
> > assert( !tmp.expired() || tmp2.expired() );
>
> Sorry I did not read this carefully earlier. If I understand, it checks
> that object_ptr_ NULL -> tracker NULL, thus guarantees no nulled
> backpointers. But when having multiple owning pointers, releasing one
> won't trigger object destruction, both must be released.

Yes; the scenario is that the parent A has an owning pointer to the
child B, B has a backpointer to A, and other parts of the program
sometimes temporarily retain copies of the owning pointer.

Something like this could happen when A holds a std::vector<> of owning
pointers and this vector reallocates. The temporary copies of the
owning pointers should not trigger the assertion.

> The example I gave of two interlinked objects has two owning pointers,
> any one of which triggers destruction.
>
> Basically, it is all about maintaining relationships like this one, and
> not wasting too much memory in the meantime:
>
> If Object A exists, then Object B must exist; or Object A -> Object B.
>
> Maybe you would call A a child of B. shared_ptr maintains this
> relationship from A's side. As long as A is there, B is guaranteed to
> stay. When A goes, B goes, as long as there is noone else requiring B
> to stay.
>
> But what about B's side? When B goes, A must go, or the relationship
> breaks. But there is no simple construct for B to tell A. Why not?

I'm not sure how something like this can work. How can you initiate
destruction from B's side? B doesn't know which parts of the program
reference the parent A. If A references B, B references A and the
program freely references A and B, only a garbage collector can safely
destroy A and B.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Peter Dimov wrote:

> Arno wrote:

> I'm not sure how something like this can work. How can you initiate
> destruction from B's side? B doesn't know which parts of the program
> reference the parent A. If A references B, B references A and the
> program freely references A and B, only a garbage collector can safely
> destroy A and B.

     Very carefully.

     Which, of course, is the problem.  It's not airtight and it's
complicated, which, in memory allocation, is trouble.

     Actually, we're slowly converging on a good answer, which has
three parts, and an optional fourth part.

 1.  shared_ptr or something like it.
 2.  weak_ptr or something like it.
 3.  overloading of "operator.()"

All of these have been proposed multiple times.  But they need to be
put into one coherent proposal, because each of them is more useful
if you have all of them.  The normal way to access a shared_ptr's object
should be via "operator.()", which is safe, convenient, and if properly
implemented, cost-free.  Obtaining a raw pointer from a shared_ptr
should be possible but only via some explicit mechanism
("extract_unsafe_raw_pointer" perhaps) that can be easily searched for.
That mechanism would be used primarily in library shim files providing
overloads to call existing libraries.

 4.  scoped references.

Several people, including me, have proposed "scoped references",
which can only refer to something that can outlive the reference.
It's not essential to have scoped references, but they provide
something to pass around which is a simple reference and is guaranteed
to be a valid one.

Scoped references have foundered on keyword issues (too many different
proposals repurpose "auto"), but that's a side issue.  The concept
is sound.  If you need a "raw pointer", it really should be a scoped
reference, which is safe.  New libraries should have parameters which
are scoped references.  These would accept ordinary references, for
backwards compatibility, so just adding the "scoped" (or "auto", or
whatever) attribute in the library declarations would upgrade the
library to the new form.  This provides a low-cost way to upgrade
libraries, without having to make them use shared_ptr.  This gives
us a smooth migration path to safe memory allocation.

    John Nagle
    Animats

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

nagle@animats.com (John Nagle) writes:

>     Actually, we're slowly converging on a good answer, which has
> three parts, and an optional fourth part.
>
>  1.  shared_ptr or something like it.
>  2.  weak_ptr or something like it.
>  3.  overloading of "operator.()"
>
> All of these have been proposed multiple times.  But they need to be
> put into one coherent proposal, because each of them is more useful
> if you have all of them.  The normal way to access a shared_ptr's object
> should be via "operator.()", which is safe, convenient, and if properly
> implemented, cost-free.

How does operator.() help anything?  It doesn't get you a true proxy
(for example, free functions taking T arguments don't work on the
shared_ptr<T>), and it's pure syntax with no semantic difference from
using operator->().  It doesn't solve any safety problems, does it?

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

>      Very carefully.
>
>      Which, of course, is the problem.  It's not airtight and it's
> complicated, which, in memory allocation, is trouble.

I think memory allocation should really be treated in the concept of
general object lifecycle management, which is what I aim to do. It is
just the last thing you do when an object dies, but most of the time,
by no means the only thing. And that object B's destruction requires
object A to be destroyed as well may not be pretty or safe under all
circumstances, but is often a fact of life, which needs to be
implemented one way or another. Memory deallocation is then just a
side-note.

> Scoped references have foundered on keyword issues (too many different
> proposals repurpose "auto"), but that's a side issue.  The concept
> is sound.  If you need a "raw pointer", it really should be a scoped
> reference, which is safe.  New libraries should have parameters which
> are scoped references.  These would accept ordinary references, for
> backwards compatibility, so just adding the "scoped" (or "auto", or
> whatever) attribute in the library declarations would upgrade the
> library to the new form.  This provides a low-cost way to upgrade
> libraries, without having to make them use shared_ptr.  This gives
> us a smooth migration path to safe memory allocation.

I absolutely agree. With my proposal, I want to discuss useful
semantics of such a reference, since shared_ptr already comes close.
Different syntax and access rules are a different, and more
far-reaching matter, though, and to find something everyone is happy
with is going to be difficult, so I do not want to trigger a discussion
in this direction.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

> Yes; the scenario is that the parent A has an owning pointer to the
> child B, B has a backpointer to A, and other parts of the program
> sometimes temporarily retain copies of the owning pointer.
>
> Something like this could happen when A holds a std::vector<> of owning
> pointers and this vector reallocates. The temporary copies of the
> owning pointers should not trigger the assertion.

I hoped I could sweep this little issue under the rug in the discussion
here. Did not work;-( In our project, we implemented owned_ptr
containers which do the right thing, which I do not dare to propose for
the standard. An adaptor for using regular containers would probably be
better, or just a simple shared_ptr::destroy for starters. I suggest
discussing the merits of the concept first, before worrying about
implementational details.

> I'm not sure how something like this can work. How can you initiate
> destruction from B's side? B doesn't know which parts of the program
> reference the parent A. If A references B, B references A and the
> program freely references A and B, only a garbage collector can safely
> destroy A and B.

Every object is responsible for cleaning up after itself in its dtor,
so everyone referencing it knows it will be gone soon and release any
references.

You may find it tedious to have to deregister with everyone, but often,
there are other things to do when an object deregisters beyond
resetting the pointer. And you discipline yourself to think about what
to do, when you explicitly reset the pointer. If you forget to reset
something, the dangling pointer warning goes off.

In this way, all objects in our app are tied together, and if I want to
destroy one, I just shared_ptr::destroy it. It works beautifully, often
without thinking much about it. Building the same with
shared_ptr/weak_ptrs would give me the permanent headache of either
having overlooked some newly introduced circular reference, thereby not
freeing memory at all, or having created some nulled weak_ptr that will
haunt me later.

Since you mention it, IMO, garbage collection is a programmer-soothing
piece of crap. It takes care of the very last aspect of an object's
life, freeing the object's memory. So all references are always valid.
But what good is that really? The object behind this reference may
already be in a state of utter destruction, and you simply forgot this
last lone reference holder. The memory is still there, but all
semantics have long ago gone down the drain. And the best of it all,
you even lose dtor semantics, and get some funny finalize() that may or
may not ever get called.

You simply cannot automate object lifecycles, and there is no use
pretending. GC certainly has its purpose, I just have not discovered it
yet.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

David Abrahams wrote:
> nagle@animats.com (John Nagle) writes:
>
>
>>    Actually, we're slowly converging on a good answer, which has
>>three parts, and an optional fourth part.
>>
>> 1.  shared_ptr or something like it.
>> 2.  weak_ptr or something like it.
>> 3.  overloading of "operator.()"
>>
>>All of these have been proposed multiple times.  But they need to be
>>put into one coherent proposal, because each of them is more useful
>>if you have all of them.  The normal way to access a shared_ptr's object
>>should be via "operator.()", which is safe, convenient, and if properly
>>implemented, cost-free.
>
>
> How does operator.() help anything?  It doesn't get you a true proxy
> (for example, free functions taking T arguments don't work on the
> shared_ptr<T>),

    No, it doesn't.

    Of course, if we had scoped references, it would be
possible to have a conversion from "shared_ptr<T>" to "scoped T&".
Then we could have functions that accepted T or shared_ptr<T>.
Until then, overloads.


 > and it's pure syntax with no semantic difference from
> using operator->().

    One could make the same argument against references generally.

 > It doesn't solve any safety problems, does it?

    It removes another temptation to use raw pointers.

    John Nagle

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

nagle@animats.com (John Nagle) writes:

> David Abrahams wrote:
>> nagle@animats.com (John Nagle) writes:
>>
>> and it's pure syntax with no semantic difference from
>> using operator->().
>
>    One could make the same argument against references generally.

Nobody's proposing to *add* references at this moment.

And no, you can't make the same argument because there are semantic
differences between references and pointers.

>> It doesn't solve any safety problems, does it?
>
>    It removes another temptation to use raw pointers.

I don't see how.

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Arno wrote:
> > I'm not sure how something like this can work. How can you initiate
> > destruction from B's side? B doesn't know which parts of the program
> > reference the parent A. If A references B, B references A and the
> > program freely references A and B, only a garbage collector can safely
> > destroy A and B.
>
> Every object is responsible for cleaning up after itself in its dtor,
> so everyone referencing it knows it will be gone soon and release any
> references.
>
> You may find it tedious to have to deregister with everyone, but often,
> there are other things to do when an object deregisters beyond
> resetting the pointer. And you discipline yourself to think about what
> to do, when you explicitly reset the pointer. If you forget to reset
> something, the dangling pointer warning goes off.

OK, you are using the event-based model. You ::destroy B, ~B notifies
everyone holding a reference to B so they can set their references to
NULL. ~B also drops its reference to A. However, this doesn't destroy
A, because other parts of the program still keep references to it. So
the scheme doesn't work in general.

> In this way, all objects in our app are tied together, and if I want to
> destroy one, I just shared_ptr::destroy it. It works beautifully, often
> without thinking much about it. Building the same with
> shared_ptr/weak_ptrs would give me the permanent headache of either
> having overlooked some newly introduced circular reference, thereby not
> freeing memory at all, or having created some nulled weak_ptr that will
> haunt me later.

Have you tried?

It's really not that different from your model, except that a weak_ptr
nulls itself automatically, without waiting for a notification from the
target. This reduces dependencies. It means that you have to lock() and
test it on every access, but if your notification scheme nulls a
shared_ptr to an object that is destroyed, you have to test it, too.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Hello,

In TR1, when all shared_ptrs have released their references, remaining
weak_ptrs will be set to null, becoming essentially dangling pointers.
Thus any dereferencing of a weak_ptr must always guard against possible
invalidity.

I would like to solicit comments on the following addition to TR1's
shared_ptr/weak_ptr concept: all pointers are shared_ptrs, even
backpointers. To solve the problem of circular references, we
introduced an explicit shared_ptr<T>::destroy, that releases the
pointer and regardless of the reference count calls the object's
destructor. Upon termination of the destructor, it checks whether the
ref-count has fallen to zero. If not, it throws a fatal exception -
something is wrong. For convenience, we added a owned_ptr<T> that calls
destroy in its destructor.

This concept has two advantages: after destroy has succeeded, it is
assured that a) there are no dangling pointers left that may surprise
us later, and b) that no circular references cause objects to stay
alive which we deem already dead.

We have written our complete application (www.think-cell.com) this way,
with much success. Object lifecycle bugs are caught very quickly. We do
also have the equivalent of weak_ptrs, but use them only in exceptional
circumstances.

I am looking forward to comments.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

On 06/29/2005 06:16 PM, Arno wrote:
[snip]
> I would like to solicit comments on the following addition to TR1's
> shared_ptr/weak_ptr concept: all pointers are shared_ptrs, even
> backpointers. To solve the problem of circular references, we
> introduced an explicit shared_ptr<T>::destroy, that releases the
> pointer and regardless of the reference count calls the object's
> destructor. Upon termination of the destructor, it checks whether the
> ref-count has fallen to zero. If not, it throws a fatal exception -
> something is wrong. For convenience, we added a owned_ptr<T> that calls
> destroy in its destructor.
IIUC, the programmer decides which shared_ptr is to call the destroy.
Why not just make this the single owned_ptr, and all other's weak_ptr's
which check the strong refcount to assure it's positive before
returing the T*.  If it's zero, *then* throw the fatal exception.  IOW,
there's each object has a single owned_ptr and all the owned_ptr's form
a forest; hence, there's no cycles or even dags.  OOPS, why couldn't
there be a cycle of owned_ptr's?  Anyway, this is the way polaris':

   http://polaris.cs.uiuc.edu/polaris/polaris_developer/node42.html
   http://polaris.cs.uiuc.edu/polaris/polaris_developer/node40.html

Live and Ref wrappers (owned_ptr and weak_ptr) work.
>
> This concept has two advantages: after destroy has succeeded, it is
> assured that a) there are no dangling pointers left that may surprise
> us later, and b) that no circular references cause objects to stay
> alive which we deem already dead.

I'm not so sure.  Couldn't there be a cycle of shared_ptr's created?
Then, if the programmer forgets (or doesn't know) to call destroy on the
last access to this cycle from anywhere outside the cycle, the cycle
leaks.  IOW, the programmer has to know where cycles occur and guard
against that; yet, this is the same weakness that shared_ptr has, IIUC.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

"Arno" <aschoedl@think-cell.com> wrote in message
news:1120082180.044619.10230@o13g2000cwo.googlegroups.com...
> Hello,
>
> In TR1, when all shared_ptrs have released their references, remaining
> weak_ptrs will be set to null, becoming essentially dangling pointers.
> Thus any dereferencing of a weak_ptr must always guard against possible
> invalidity.

  I disagree. There are two major differences between an expired weak_ptr
and a dangling pointer.

   1) It is possible to test whether a weak_ptr has expired.  With a true
dangling pointer, for example q after the following sequence:
    int * p = new int(0);
     int * q = p;
    delete p;
    int i = *q; //dangling pointer error
  it is impossible to tell that the pointer is dangling until you
dereference it and get a seg fault.
   2) It is very difficult to dereference a weak_ptr and forget to check it
for validity.  weak_ptr does not have an operator*(), operator->(), or get()
member function.  Thus, if you want to dereference an object wp of type
weak_ptr<Foo>, you have to have code like
    shared_ptr<Foo> sp = wp.lock();
This is unusual enough that it is pretty easy to get into the habit of
writing it like
    if (shared_ptr<Foo> sp = wp.lock()) { //Work with valid shared_ptr<Foo>

Joe Gottman

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

In article <1120082180.044619.10230@o13g2000cwo.googlegroups.com>, Arno
<aschoedl@think-cell.com> wrote:

> Hello,
>
> In TR1, when all shared_ptrs have released their references, remaining
> weak_ptrs will be set to null, becoming essentially dangling pointers.
> Thus any dereferencing of a weak_ptr must always guard against possible
> invalidity.
>
   A weak_ptr is a semi-dumb pointer that can detect if it is still a
valid pointer even if the underlying data is removed under the foot ,so
to speak of the weak_ptr.  This can be used for anything that can
ignore
invalid pointers in a relatively infrequent usuage, such as notifying a
removed object via an observer pattern.  This observer only has to
check if the weak_ptr is valid, if not don't notify. Nothing has to
happen to the observer pattern itself when the object is deallocated,
only when the deallocated data would be used, if it were valid.

  A weak_ptr<T> is a fairly recent addition to the C++ library and as
such does not need protection from legacy code, to save beginners:)

> I would like to solicit comments on the following addition to TR1's
> shared_ptr/weak_ptr concept: all pointers are shared_ptrs, even
> backpointers. To solve the problem of circular references, we
> introduced an explicit shared_ptr<T>::destroy, that releases the
> pointer and regardless of the reference count calls the object's
> destructor. Upon termination of the destructor, it checks whether the
> ref-count has fallen to zero. If not, it throws a fatal exception -
> something is wrong. For convenience, we added a owned_ptr<T> that calls
> destroy in its destructor.
>
   The advantages of shared_ptr is that it is non-intrusive and adding
intrusiveness to shared_ptr/weak_ptr is a weakening of shared_ptr and
not an improvement.

> This concept has two advantages: after destroy has succeeded, it is
> assured that a) there are no dangling pointers left that may surprise
> us later, and b) that no circular references cause objects to stay
> alive which we deem already dead.
>
  good luck finding all dead references in a general case, I surely
don't want that executed everytime a shared_ptr<>'s dtor is called.

> We have written our complete application (www.think-cell.com) this way,
> with much success. Object lifecycle bugs are caught very quickly. We do
> also have the equivalent of weak_ptrs, but use them only in exceptional
> circumstances.
>
   If you think this is useful write a proposal for another smart ptr
class, but modifying shared_ptr<T> this way will reduce its value, not
enhance it.
   All comments are mine and not those of any organization what so
ever:)

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Larry Evans wrote:
> On 06/29/2005 06:16 PM, Arno wrote:
> [snip]
> IIUC, the programmer decides which shared_ptr is to call the destroy.
> Why not just make this the single owned_ptr, and all other's weak_ptr's
> which check the strong refcount to assure it's positive before
> returing the T*.  If it's zero, *then* throw the fatal exception.  IOW,
> there's each object has a single owned_ptr and all the owned_ptr's form
> a forest; hence, there's no cycles or even dags.  OOPS, why couldn't
> there be a cycle of owned_ptr's?  Anyway, this is the way polaris':
>
>    http://polaris.cs.uiuc.edu/polaris/polaris_developer/node42.html
>    http://polaris.cs.uiuc.edu/polaris/polaris_developer/node40.html
>

Yes, there can be leaks with cycles of shared_ptrs, as with all non-GC
schemes, but when using the concept you can err on the side of
destroying eagerly, because you notified whenever you created a
dangling reference. Catching the dangling reference at the time of
object destruction, _not_ at the time of using the dangling reference,
is key, because it cuts down the number of places errors can occur and
thus makes testing much easier.

Cycles of owned_ptrs are common, but in fact not a problem, as long as
other owned_ptrs recognize that the destructor is already running
(through adding an offset to the refcount, for example, when the
destructor is started). owned_ptr is a bit of a misnomer: if an object
A has a pointer to object B, and if A is gone, B should be gone as
well, use a owned_ptr. If there are multiple A1, A2, that are all
prerequisites to Bs existance, use owned_ptrs in all A1, A2...


> Live and Ref wrappers (owned_ptr and weak_ptr) work.
> >
> > This concept has two advantages: after destroy has succeeded, it is
> > assured that a) there are no dangling pointers left that may surprise
> > us later, and b) that no circular references cause objects to stay
> > alive which we deem already dead.
>
> I'm not so sure.  Couldn't there be a cycle of shared_ptr's created?
> Then, if the programmer forgets (or doesn't know) to call destroy on the
> last access to this cycle from anywhere outside the cycle, the cycle
> leaks.  IOW, the programmer has to know where cycles occur and guard
> against that; yet, this is the same weakness that shared_ptr has, IIUC.

Calling destroy (or using owned_ptr) at the right places vs. using
weak_ptrs in the right places is approximately equal burden on the
programmer, I think. Using destroy/owned_ptr has the advantage of early
error checking. BTW, I hate garbage collection. It is nice and
automatic, but you lose destructor semantics, which I hold dear.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Arno wrote:

> In TR1, when all shared_ptrs have released their references,
> remaining weak_ptrs will be set to null, becoming essentially
> dangling pointers.

Just the opposite.  A null pointer can never dangle.

> Thus any dereferencing of a weak_ptr must always guard against
> possible invalidity.

If the TR1 proposition corresponds to the boost::shared_ptr,
then you cannot dereference a weak_ptr.  You can only get a
shared_ptr from it, and dereference that.  And if the weak_ptr
is null, trying to get a shread_ptr from it fails.

--
James Kanze                                           GABI Software
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

I understand that weak_ptrs know when they are dangling, regular
pointers do not. But this is not the point. It is much safer to check
upon object destruction, whether dangling pointers were created, rather
than checking and handling a failure upon access of such a pointer.
Some pointers may be in rarely used code paths, and often, invalid
weak_ptrs should simply not occur if everything goes as intended by the
programmer and children are destroyed together with parents.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Arno wrote:
> Hello,
>
> In TR1, when all shared_ptrs have released their references, remaining
> weak_ptrs will be set to null, becoming essentially dangling pointers.
> Thus any dereferencing of a weak_ptr must always guard against possible
> invalidity.

See Joe Gottman's post. :-)

> I would like to solicit comments on the following addition to TR1's
> shared_ptr/weak_ptr concept: all pointers are shared_ptrs, even
> backpointers. To solve the problem of circular references, we
> introduced an explicit shared_ptr<T>::destroy, that releases the
> pointer and regardless of the reference count calls the object's
> destructor. Upon termination of the destructor, it checks whether the
> ref-count has fallen to zero. If not, it throws a fatal exception -
> something is wrong. For convenience, we added a owned_ptr<T> that calls
> destroy in its destructor.

Interesting. Let's see if I understand correctly.

You can do that with an ordinary shared_ptr. An owning pointer p is:

shared_ptr<X> object_ptr_;
shared_ptr<void> tracker_;

where tracker_ is arbitrarily initialized to (int*)0, for example.

A backpointer q is

X * object_ptr_;
shared_ptr<void> tracker_;

and is initialized to ( p.object_ptr_.get(), p.tracker_ ).

When an owning pointer is destroyed, it checks that its tracker's
reference count has dropped to one. (*)

You can #ifdef out the tracker in NDEBUG builds, because it's only good
for finding coding errors; throwing an exception from the destructor is
usually non-recoverable. This solution also doesn't create any cycles.

(*) The MT-safe version is slightly more convoluted:

weak_ptr<void> tmp( tracker_ );
object_ptr_.reset();
tracker_.reset();
assert( tmp.expired() );

and support for multiple owning pointer copies can be added with:

weak_ptr<X> tmp( object_ptr_ );
weak_ptr<void> tmp2( tracker_ );
object_ptr_.reset();
tracker_.reset();
assert( !tmp.expired() || tmp2.expired() );

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

"Arno" <aschoedl@think-cell.com> writes:

> Hello,
>
> In TR1, when all shared_ptrs have released their references, remaining
> weak_ptrs will be set to null, becoming essentially dangling
> pointers.

A null pointer is one of two opposites of a dangling pointer.  The
other opposite is a pointer to a live object.  A dangling pointer
holds the address of an object that has been destroyed.

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

>    A weak_ptr is a semi-dumb pointer that can detect if it is still a
> valid pointer even if the underlying data is removed under the foot ,so
> to speak of the weak_ptr.  This can be used for anything that can
> ignore
> invalid pointers in a relatively infrequent usuage, such as notifying a
> removed object via an observer pattern.  This observer only has to
> check if the weak_ptr is valid, if not don't notify. Nothing has to
> happen to the observer pattern itself when the object is deallocated,
> only when the deallocated data would be used, if it were valid.

This is clear to me. If objects can live without the reference being
valid, as you suggested for the case of an observer pattern, weak_ptrs
are nice.

But often, backpointers are a crucial part of an object, and the object
is unusable without them. In this case, using weak_ptrs as backpointers
is bad because they should always be valid; the object should be
destroyed before them becoming invalid.

>    The advantages of shared_ptr is that it is non-intrusive and adding
> intrusiveness to shared_ptr/weak_ptr is a weakening of shared_ptr and
> not an improvement.

If I am not mistaken, the same mechanism that makes shared_ptr
non-intrusive could be used for owned_ptr/shared_ptr::destroy, no need
for intrusiveness.

>
> > This concept has two advantages: after destroy has succeeded, it is
> > assured that a) there are no dangling pointers left that may surprise
> > us later, and b) that no circular references cause objects to stay
> > alive which we deem already dead.
> >
>   good luck finding all dead references in a general case, I surely
> don't want that executed everytime a shared_ptr<>'s dtor is called.

You do the check only upon destruction of the object, not upon
destruction of every shared_ptr. In fact, you do not need to explicitly
identify dead references at all, simply checking for refcount==0 after
the object's dtor is enough. In debug builds, you can keep all
references in a double-linked list, so when a destroy fails, you can
find all dangling references.

The whole scheme is no guarantee against leaking memory, circular
shared_ptr references will still be leaked. But eager use of
shared_ptr::destroy or owned_ptr is encouraged, because any mistakes
will be reported upon calling destroy, not later, when being surprised
by an invalid weak_ptr that you did not expect to be invalid.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Arno wrote:
> Hello,
>
> In TR1, when all shared_ptrs have released their references, remaining
> weak_ptrs will be set to null, becoming essentially dangling pointers.
> Thus any dereferencing of a weak_ptr must always guard against possible
> invalidity.

     Where is this in TR1?  That's very good. I'd like to
read the proposal.

> I would like to solicit comments on the following addition to TR1's
> shared_ptr/weak_ptr concept: all pointers are shared_ptrs, even
> backpointers.

     If all you have are shared_ptrs, without weak pointers,
order of destructor execution between objects that point to
each other becomes tricky.  If all pointers are shared
pointers, deleting a tree is complicated.
To delete a tree with backpointers, for example, you must
first walk the tree and clear all the backpointers, then
delete the root.

      Perl used to have shared pointers only, but they
had to add weak pointers to deal neatly with such cases.

    John Nagle

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

John Nagle wrote:
> Arno wrote:
> > Hello,
> >
> > In TR1, when all shared_ptrs have released their references, remaining
> > weak_ptrs will be set to null, becoming essentially dangling pointers.
> > Thus any dereferencing of a weak_ptr must always guard against possible
> > invalidity.
>
>      Where is this in TR1?  That's very good. I'd like to
> read the proposal.

TR1:

http://open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1836.pdf

The original shared_ptr paper:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2003/n1450.html

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

>      If all you have are shared_ptrs, without weak pointers,
> order of destructor execution between objects that point to
> each other becomes tricky.  If all pointers are shared
> pointers, deleting a tree is complicated.
> To delete a tree with backpointers, for example, you must
> first walk the tree and clear all the backpointers, then
> delete the root.
>
>       Perl used to have shared pointers only, but they
> had to add weak pointers to deal neatly with such cases.

My proposal deals with such cases by providing shared_ptr::destroy (or
owned_ptr, which calls destroy in its destructor). It triggers the
object destructor and checks afterwards that there are no references
left. In your example, you would destroy the root, which in its
destructor destroys its children, and so on, cleaning up back-pointers
along the way. Upon completion of each destructor, it is checked that
no invalid references have been left over.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

aschoedl@think-cell.com (Arno) wrote (abridged):
> I would like to solicit comments on the following addition to TR1's
> shared_ptr/weak_ptr concept: all pointers are shared_ptrs, even
> backpointers. To solve the problem of circular references, we
> introduced an explicit shared_ptr<T>::destroy, that releases the
> pointer and regardless of the reference count calls the object's
> destructor. Upon termination of the destructor, it checks whether the
> ref-count has fallen to zero. If not, it throws a fatal exception -
> something is wrong. For convenience, we added a owned_ptr<T> that calls
> destroy in its destructor.

It seems to me that deciding when to destroy() is just as difficult as
deciding when to delete. It's a non-local decision. The pointer is no
longer managing lifetimes for you, you still have to do that for yourself:
it's job is now to tell you when you've got it wrong.

Your destroy() is scary. As an alternative, consider avoiding cycles by
using a mixture of shared_ptr and weak_ptr. Then write something like:

    template <typename T>
    void destroy( shared_ptr<T> &p ) {
        weak_ptr<T> weak( p );
        p = 0;
        assert( weak.expired() );
        assert( weak.unique() ); // New.
    }

which you invoke on the last shared pointer. The first assert verifies
that there are indeed no other outstanding shared_ptrs. The second assert
should verify that there are no outstanding weak_ptrs either. I think
weak_ptr::unique() requires an addition to the proposal - there doesn't
seem to be any way to query an object's weak_count. Given that, it would
guarantee that you have no weak pointers getting nulled unexpectedly.

-- Dave Harris, Nottingham, UK.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

> It seems to me that deciding when to destroy() is just as difficult as
> deciding when to delete. It's a non-local decision. The pointer is no
> longer managing lifetimes for you, you still have to do that for yourself:
> it's job is now to tell you when you've got it wrong.
>
> Your destroy() is scary. As an alternative, consider avoiding cycles by
> using a mixture of shared_ptr and weak_ptr. Then write something like:
>
>     template <typename T>
>     void destroy( shared_ptr<T> &p ) {
>         weak_ptr<T> weak( p );
>         p = 0;
>         assert( weak.expired() );
>         assert( weak.unique() ); // New.
>     }

This is not quite the same semantics. It works if you have a single
shared_ptr, not if you have two and trigger destruction from one of
them. In earlier posts, I described having two objects that need each
other. Then both can destroy the other in their destructor.

But you certainly have a point. The scheme I propose is a mix of two
concepts which have little to do with each other, but just happen to
require very little extra overhead, if you have one, to have the other
as well.

The first concept is reference counting.

The second is plain old T* new/delete semantics, with a dangling
pointer check after the delete. shared_ptr::destroy just calls the
dtor, no matter what other pointers are there, which is what a simple
delete does.

Given that dangling pointer are hard to find, we wanted to keep the
dangling pointer check even for Release builds. Reporting errors
w/stack traces over the internet is becoming increasingly popular, so
it is very useful to know about a bug even it occurs in the Release
build.

An efficient way to know about the presence of dangling pointers is
reference counting, but the current shared_ptr cannot be used in this
way because weak_ptrs do not increase the count. shared_ptrs do, but
you currently cannot force a destroy on them.

You could come up with the following hack, similar in spirit to the one
you proposed:

     template <typename T>
     void destroy( shared_ptr<T> &p ) {
         delete &(*p); // force dtor and memory deallocation; legal?
         // check that p is the last holder of this reference
         assert( p.use_count()==1 ); // o.k. because shared_ptr is
non-intrusive, so dtor did not interfere with reference count
         p.detach(); // non-standard, detach p without triggering
destructor again
     }

I think this simulates new/delete semantics with a dangling pointer
check. Yes, it is not pretty, and shared_ptr does not have a detach().

But worse, it does not work (just like plain new/delete does not) in
the case of two objects A and B, where, if one is destroyed, it should
destroy the other with it. If both in their destructors call destroy,
and one of the objects gets destroyed from the outside, infinite
recursion occurs: A dtor runs -> destroys reference to B -> B dtor runs
-> destroys reference to A -> ...

We solve this problem by using a flag (in fact the highest ref-count
bit) that indicates whether the destructor has started running. In this
case, a destroy is interpreted as a simple release, lowering the
reference count but not triggering another dtor run.

Given that we need the reference counting infrastructure anyway as the
cheapest way to do the dangling pointer check, why not use regular
reference counting in addition to explicit destroy? If there are no
circular references, shared_ptr can be used, for example for the
classic case of maintaining external resources.

If there are circular references, shared_ptr::destroy (or owned_ptr,
which calls shared_ptr::destroy in its dtor) can be used to break the
cycle. In fact, we put in sensible shared_ptr::destroys everywhere,
just in case a circular reference is added later.

I hope the rationale of my proposal is a little clearer now. It is no
magic, you need to know where to put in destroy just as you have to
know where to use weak_ptr instead of shared_ptr in the existing
scheme. But it works really well, and I found it easy to teach to my
developers.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Arno wrote:

>>     If all you have are shared_ptrs, without weak pointers,
>>order of destructor execution between objects that point to
>>each other becomes tricky.  If all pointers are shared
>>pointers, deleting a tree is complicated.
>>To delete a tree with backpointers, for example, you must
>>first walk the tree and clear all the backpointers, then
>>delete the root.
>>
>>      Perl used to have shared pointers only, but they
>>had to add weak pointers to deal neatly with such cases.
>
>
> My proposal deals with such cases by providing shared_ptr::destroy (or
> owned_ptr, which calls destroy in its destructor). It triggers the
> object destructor and checks afterwards that there are no references
> left. In your example, you would destroy the root, which in its
> destructor destroys its children, and so on, cleaning up back-pointers
> along the way. Upon completion of each destructor, it is checked that
> no invalid references have been left over.
>
> Arno

     If the destructor of a tree element has to explicitly destroy
its children, the whole point of reference-counted pointers has
been lost.  The idea is to make storage management automatic
and implicit.

    John Nagle

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

aschoedl@think-cell.com (Arno) wrote (abridged):
> > Your destroy() is scary. As an alternative, consider avoiding cycles
> > by using a mixture of shared_ptr and weak_ptr. Then write
> > something like:
> >
> >     template <typename T>
> >     void destroy( shared_ptr<T> &p ) {
> >         weak_ptr<T> weak( p );
> >         p = 0;
> >         assert( weak.expired() );
> >         assert( weak.unique() ); // New.
> >     }
>
> This is not quite the same semantics. It works if you have a single
> shared_ptr, not if you have two and trigger destruction from one of
> them.

Yes. That's why I mentioned using weak_ptrs. My argument is that if you
know enough about the global state of the pointer network to call
destroy(), you probably know which pointer you want to call it on, and you
can make that pointer strong and the others weak.

In the extreme case, you could give the object a strong pointer to itself
and make all the others weak. You destroy the object by asking it to null
its strong pointer. Then you don't have to worry about loops at all.
(Admittedly this is not what boost::weak_ptr is designed for, and it would
be a pain to have to lock it whenever you want to dereference it.)


> shared_ptr does not have a detach().

No, but you can create it with a null_deleter and then just set it to 0.
But I think it is better to use weak pointers.


> [...] in the case of two objects A and B, where, if one is destroyed,
> it should destroy the other with it. If both in their destructors
> call destroy, and one of the objects gets destroyed from the outside,
> infinite recursion occurs: A dtor runs -> destroys reference to B ->
> B dtor runs -> destroys reference to A -> ...

Using my scheme it would be like this:

    class B;

    class A {
    public:
        void destroy() {
            m_this = NULL;
        }
        // ...
    private:
        ~A() {
            if (shared_ptr<B> p = m_b.lock())
                p->destroy();
        }
        shared_ptr<A> m_this;
        weak_ptr<B> m_b;
    };

with B being similar. If A is destroyed first, the sequence should be:

(1) A::destroy() entered.
(2) Set A::m_this to null. A's strong count hits zero.
(3) A is disposed. Weak pointers to A are effectively nulled.
(4) A::~A() entered.
(5) ~A() invokes destroy on A::m_b.
(6) B::destroy() entered.
(7) B:m_this set to null. B's strong count hits zero.
(8) B is disposed. Weak pointers to B are effectively nulled.
(9) B::~B() entered.
(10) ~B tries to lock B::m_a, but this weak pointer has already been
nulled by step (3) so the "if" fails. Nothing more happens.
(11) B::~B() runs to completion.
(12) B::destroy() resumes and runs to completion.
(13) A::~A() resumes and runs to completion.
(14) A::destroy() resumes and runs to completion.

That works. The only problem is that we cannot put the assertions in
B::destroy(), because they should not be tested until both A and B are
destroyed. That is, at step (12) B cannot assert there are no weak
pointers to itself because one remains in A (albeit nulled). We can put
the assertions in a non-member function as before.


> I hope the rationale of my proposal is a little clearer now. It is no
> magic, you need to know where to put in destroy just as you have to
> know where to use weak_ptr instead of shared_ptr in the existing
> scheme.

I believe I already understood it. A point in its favour is that destroy()
can be added to shared_ptr<T> without affecting the rest of the API.
People who don't use it wouldn't be harmed.

Never-the-less I am not comfortable with it. Destroy() circumvents the
guarantees that shared_ptr was designed to ensure. I would be worried that
other programmers would call destroy() when they shouldn't. So I'd rather
destroy() not be added to the shared_ptr interface. It really doesn't
belong there. I would rather see a whole new smart pointer class which
clearly signals it is meant to be used in the way you propose.

-- Dave Harris, Nottingham, UK.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Dave Harris wrote:

[...]

>     template <typename T>
>     void destroy( shared_ptr<T> &p ) {
>         weak_ptr<T> weak( p );
>         p = 0;
>         assert( weak.expired() );
>         assert( weak.unique() ); // New.
>     }
>
> which you invoke on the last shared pointer. The first assert verifies
> that there are indeed no other outstanding shared_ptrs. The second assert
> should verify that there are no outstanding weak_ptrs either.

The second assert is unnecessary. The whole point of weak_ptr is to
detect when its target goes away. You are prohibiting this legitimate
use for no benefit.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Arno wrote:

> Given that dangling pointer are hard to find, we wanted to keep the
> dangling pointer check even for Release builds. Reporting errors
> w/stack traces over the internet is becoming increasingly popular, so
> it is very useful to know about a bug even it occurs in the Release
> build.

This is reasonable, but throwing an exception is not a safe way to
report the error. When your destroy() fails, it leaves dangling
shared_ptr instances behind. The unwinding caused by the exception may
cause them to be destroyed.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

> Yes. That's why I mentioned using weak_ptrs. My argument is that if you
> know enough about the global state of the pointer network to call
> destroy(), you probably know which pointer you want to call it on, and you
> can make that pointer strong and the others weak.

True. In practice, it turns out that destruction order is sometimes
important, so you want to explicitly "destroy". Of course, you can
implement this by explicitly nulling the shared pointer.

<stuff deleted>
> (1) A::destroy() entered.
> (2) Set A::m_this to null. A's strong count hits zero.
> (3) A is disposed. Weak pointers to A are effectively nulled.
> (4) A::~A() entered.
> (5) ~A() invokes destroy on A::m_b.
> (6) B::destroy() entered.
> (7) B:m_this set to null. B's strong count hits zero.
> (8) B is disposed. Weak pointers to B are effectively nulled.
> (9) B::~B() entered.
> (10) ~B tries to lock B::m_a, but this weak pointer has already been
> nulled by step (3) so the "if" fails. Nothing more happens.
> (11) B::~B() runs to completion.
> (12) B::destroy() resumes and runs to completion.
> (13) A::~A() resumes and runs to completion.
> (14) A::destroy() resumes and runs to completion.
>
> That works. The only problem is that we cannot put the assertions in
> B::destroy(), because they should not be tested until both A and B are
> destroyed. That is, at step (12) B cannot assert there are no weak
> pointers to itself because one remains in A (albeit nulled). We can put
> the assertions in a non-member function as before.

All true. This final assert though is the crucial point, you do not
want to have anyone test for it manually.

> Never-the-less I am not comfortable with it. Destroy() circumvents the
> guarantees that shared_ptr was designed to ensure. I would be worried that
> other programmers would call destroy() when they shouldn't. So I'd rather
> destroy() not be added to the shared_ptr interface. It really doesn't
> belong there. I would rather see a whole new smart pointer class which
> clearly signals it is meant to be used in the way you propose.

Ok, I accept that. I think that given the popularity of
garbage-collected languages like Java or C#, C++ should try harder to
become more convenient in the object lifecycle/memory management
domain. Coming up with a single smart pointer type that can handle the
object framework of the application, and that could be taught as _the_
smart pointer to use when creating objects, would be a good start. C++
is great because you can always tailor something for yourself that does
what you want, which is what we did. But the value of language does lie
to a large extent in what comes out of the box, not least because any
C++ programmer you hire will (or at least should) then know the
concept.

If you can accept this premise, TR1 now offers shared_ptr as the first
pretty universal smart pointer, which probably aspires to be the smart
pointer concept that everyone is taught and knows. So I see it as being
more than a reference-counted pointer, it is _the_ standard object
pointer. And for that, it is currently not quite applicable, because of
the nulled weak_ptr problem that the programmer cannot easily check
for.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Peter Dimov wrote:
> Arno wrote:
>
> > Given that dangling pointer are hard to find, we wanted to keep the
> > dangling pointer check even for Release builds. Reporting errors
> > w/stack traces over the internet is becoming increasingly popular, so
> > it is very useful to know about a bug even it occurs in the Release
> > build.
>
> This is reasonable, but throwing an exception is not a safe way to
> report the error. When your destroy() fails, it leaves dangling
> shared_ptr instances behind. The unwinding caused by the exception may
> cause them to be destroyed.

Ok, I see that. We just leave the pointers dangling, but for the
standard, more elaborate approaches are probably more appropriate. All
outstanding pointers could automatically zero themselves before the
exception is thrown. For non-intrusive implementations using
heap-allocated proxies, this would be quite low overhead. Is that the
way it is done for weak_ptrs in boost?

Maybe take a look at my previous answer. I see shared_ptr/weak_ptr, if
adopted as the standard, to be not just a reference-counted pointer,
but the _standard_ object pointer, the C++ equivalent to Java
garbage-collected references, the memory management concept that is
taught to C++ student everywhere. So it should cover the important use
cases.

The standard could offer a separate dangle-checked delete along with an
appropriate pointer class, which should also prevent double destructor
invocation. But then we have two pointer classes, and with it the
problems of assigning between them. And for almost no extra overhead,
this functionality can be added to shared_ptr/weak_ptr.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

pdimov@gmail.com (Peter Dimov) wrote (abridged):
> The second assert is unnecessary. The whole point of weak_ptr is to
> detect when its target goes away. You are prohibiting this legitimate
> use for no benefit.

First, I am not prohibiting it. Second, there is a benefit.

The original poster wants to support an idiom where-by he can ensure that
pointers do not unexpectedly become null or dangling. We're looking at
low-impact ways of supporting that idiom within the shared_ptr framework.
He obviously thinks the idiom has benefit for him, and if you disagree,
you can just not use it.

As I've argued elsewhere, I don't think shared_ptr is the right place, but
that's another matter. Adding unique() (or destroy(), for that matter) to
the API does not affect or constrain programs which don't use them.

-- Dave Harris, Nottingham, UK.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Dave Harris wrote:
> pdimov@gmail.com (Peter Dimov) wrote (abridged):
> > The second assert is unnecessary. The whole point of weak_ptr is to
> > detect when its target goes away. You are prohibiting this legitimate
> > use for no benefit.
>
> First, I am not prohibiting it. Second, there is a benefit.

I think that you are wrong on both counts, but I may have
misinterpreted your code.

> The original poster wants to support an idiom where-by he can ensure that
> pointers do not unexpectedly become null or dangling. We're looking at
> low-impact ways of supporting that idiom within the shared_ptr framework.

Right. The particular idiom under discussion is an object that has
several children, which keep backpointers to the parent. The goal is to
ensure that no backpointers are left dangling.

The original proposed solution uses shared_ptr for backpointers. Since
this creates cycles, an unconditional shared_ptr::destroy method is
proposed. This method checks that the reference count has dropped to
zero in order to detect dangling shared_ptr instances.

My proposed solution uses ordinary pointers as backpointers and a
separate mechanism that keeps track of the backpointers to ensure that
they are destroyed along with the object.

Using weak pointers as backpointers has the following drawbacks:

1. Weak pointers are intentionally hard to dereference;
2. If you use weak pointers as backpointers (in this specific sense),
you'll no longer be able to use them as weak pointers, because it is
impossible to distinguish between the two.

Your assert( unique ) disallows any weak pointers used as weak
pointers. Their defining characteristic is that they can outlive their
target and communicate that condition.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

> and support for multiple owning pointer copies can be added with:
>
> weak_ptr<X> tmp( object_ptr_ );
> weak_ptr<void> tmp2( tracker_ );
> object_ptr_.reset();
> tracker_.reset();
> assert( !tmp.expired() || tmp2.expired() );

Sorry I did not read this carefully earlier. If I understand, it checks
that object_ptr_ NULL -> tracker NULL, thus guarantees no nulled
backpointers. But when having multiple owning pointers, releasing one
won't trigger object destruction, both must be released. The example I
gave of two interlinked objects has two owning pointers, any one of
which triggers destruction.

Basically, it is all about maintaining relationships like this one, and
not wasting too much memory in the meantime:

If Object A exists, then Object B must exist; or Object A -> Object B.

Maybe you would call A a child of B. shared_ptr maintains this
relationship from A's side. As long as A is there, B is guaranteed to
stay. When A goes, B goes, as long as there is noone else requiring B
to stay.

But what about B's side? When B goes, A must go, or the relationship
breaks. But there is no simple construct for B to tell A. Why not?

Looking at it this way, constructs like

Object C exists <- Object A exists -> Object B exists

are quite natural. And both C's and B's destruction must trigger A's
destruction.

So both B and C have owned_ptrs to A, and A has shared_ptrs to B and C.
This view of things suddenly makes owned_ptr a quite natural companion
to shared_ptr.

Arno

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]