On 12/13/2004 09:28 PM, John Nagle wrote:
> Thorsten Ottosen wrote:
>
>> Actually I said that because I thought I read that the boehm collector
>> could not detect cycles.
[snip]
> Destructors are called long after the object is
> no longer needed, so if they're supposed to release
> some resource like a window or a socket, the resource
> is released late, if ever.  Destructors may be called
> by an unexpected thread at an unexpected time, so they
> have to have locking.  Such locking may deadlock the
> garbage collector.

Couldn't the garbage collector 1st break any cycles
by arbitrarily chosing some link in the cycle?  Admittedly
the designer of the destructor may not have intended this;
however, he obviously didn't intend cycles either.

As far as deadlocks are concerned, I remember from  OS
course in school that deadlocks are caused by cycles
in the locker and thread.  If the gc could find this
cycle, could it somehow also break the cycle?

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Hans Boehm wrote:
> I would argue that's backwards.  Destructors triggered by reference
> counting may be called by an unexpected thread at an unexpected time.
> An assignment somewhere in your may (?) recursively cause deletion of and
> destructor invocations on objects that should be opaque to you.  The
> destructor may access a global data structure (e.g. to remove the
> object from a table) which is currently being accessed by one of your
> callers, and is hence locked leading to self-deadlock (bad) or in an
> inconsistent state (worse).

    A program vulnerable to that problem from reference counting is
also vulnerable to it from garbage collection.  But in a system
that runs garbage collection at random times, the problem will
generate unrepeatable race conditions.

> Garbage-collector triggered finalizers

    In C++, there are no finalizers.  There are destructors.
That's the problem.  Finalizers and destructors have quite
different semantics.  Java has garbage collection and finalizers,
and they play well together because they were designed together.

    It's possible to have good garbage-collected languages,
but retrofitting garbage collection onto C++ results in
major changes to the language semantics.  There's the illusion
that because you don't have to change the syntax to add garbage
collection, it's a low-impact change.  But it's not.

    Not that reference counting is an easy retrofit, either.

    But we can't ignore the problem any more.  It's starting
to attract attention at the higher levels of the Department
of Homeland Security.  See Amit Yoran's speech on the
"19 programming flaws" that cause most security vulnerabilities.

    John Nagle

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Hans Boehm wrote:

> John Nagle wrote:
>
>>Hans Boehm wrote:

> Finalizers triggered by a GC need to lock global data structures they
> access.  Since they are run in their own thread, and can thus safely
> block, they are essentially a form of concurrency and require threads.
> But it is just as easy or difficult to avoid deadlock with (properly
> implemented) finalizers as it is with any other form of concurrency.

    True.

    There's something to be said for a highly concurrent style of
programming, where there are free-running threads all over the
place and everything is locked when necessary.

    But that's an uncommon style for C++.  It's much more common
for C++ code to assume purely sequential execution and not lock much.
The language, after all, provides no support for threads or locking.

    In particular, seldom is C++ GUI code written for a highly concurrent
model.  Arguably, it should be, but for historical reasons, most
GUI systems are single-thread.  Threads came late to both UNIX and
Windows, and we still feel the weight of the problems created by
that.

     Java has a different history and different conventions.
The garbage collection/finalizer model works well for Java.
But what works for Java is tough to retrofit to C++.

    John Nagle
    Animats

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

X-Replace-Address:yes
John Max Skaller wrote:
> The problem with the newstyle cast system is that
> it was never properly thought out. For example this
> doesn't work on g++:
>
>  struct X {};
>  struct Y {};
>  Y f(); // hack to correct g++ parsing bug
>  reinterpret_cast<X&>( f() };
>
> All I want to do is 'view' the Y as an X,
> which reinterpret cast should allow. I get a hard error.

Normal: That is the behaviour specified in     5.2.10/10:
Reference reinterpret_cast works for an lvalue argument only
and yields an lvalue.

> At least g++ isn't stupid enough to prevent me addressing
> an rvalue (I only get a warning)

The way the language is specified today (    5.3.1/2), taking
address works on lvalues only (barring classes with overloaded
operator &) and g++ is right in issuing a diagnostic. If it
lets you continue and if the generated code does what you
expected, you are just lucky...

> which is needed to do
> it like this:
>
>  *(X*)(void*)& f()

Which could be rewritten as
   *reinterpret_cast<X*>(&f());
which is just as good (or rather: just as bad) since you are in
Undefined Behaviour Land anyway once you have taken the address
of an rvalue...
As long as you don't expect the temporary object to live longer
than the end of the full expression in which it occurs, there
is a chance that it actually works but I wouldn't bet on it...

> Reinterpret cast SHOULD allow reinterpretation
> of an rvalue (one way or another :)
>
> This need actually arises systematically in the Felix
> code generator.

Really? What problem does it solve that wouldn't be solved by
doing
   Y f_result(f());
   reinterpret_cast<X&>(f_result);
? Do you have some example code? Just curious...

Another solution would be to let f() return a reference to Y
(provided it refers to an object that lives long enough), in
which case
   reinterpret_cast<X&>(f());
will work fine.

If you really want to improve the Standard you rather should think
about how to fix that whole lvalue / rvalue mess...

> Reinterpret cast also won't handle const casting,
> which is utterly stupid: it isn't just a matter of
> having to use multiple casts .. how about
>
>  Y const * -->  X*
>
> You can't do this with any combination of const and reinterpret
> casts, for example: the only way to do a pure cast from
> char const * to unsigned char * is an old style cast:
> you can't cast the base type with reinterpret cast
> (because of the const) and you can't cast the const away
> with a const cast (because of the reinterpretation).

Don't understand the problem you are facing...
Both
   X* px1 = reinterpret_cast<X*>(const_cast<Y*>(pointer_to_const_Y));
and
   X* px2 = const_cast<X*>(reinterpret_cast<X const*>(pointer_to_const_Y));

work fine and that is as it ought to be! Each cast does one and *only*
one thing: const_cast removes the const qualifier but doesn't touch the
type the pointer points to, reinterpret_cast allows you to treat the
representation of an object in memory as if it were of another type but
it leaves alone its const-ness.

Falk

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

John Max Skaller wrote:

> On Mon, 13 Dec 2004 14:17:33 -0600, John Nagle wrote:
>
>
>>    Should it be an error to cast to a non-POD object?
>>Maybe it's time to require a "reinterpret_cast" to cast
>>to a non-POD object, and disallow the use of old-style C
>>casts to non-POD objects.
>
>
> No!! Gak!!
>
> The problem with the newstyle cast system is that
> it was never properly thought out. For example this
> doesn't work on g++:
>
>  struct X {};
>  struct Y {};
>  Y f(); // hack to correct g++ parsing bug
>  reinterpret_cast<X&>( f() };
>
> All I want to do is 'view' the Y as an X,
> which reinterpret cast should allow. I get a hard error.
> At least g++ isn't stupid enough to prevent me addressing
> an rvalue (I only get a warning) which is needed to do
> it like this:
>
>  *(X*)(void*)& f()
>
> Reinterpret cast SHOULD allow reinterpretation
> of an rvalue (one way or another :)

     Why?  C++ correctly doesn't you to take the
address of a temporary.  Try compiling such code on a
machine that stores most temporaries on registers,
like a SPARC, and see what happens.

> This need actually arises systematically in the Felix
> code generator.

     Sounds like a bug in Felix.

    John Nagle

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Hi,

> The reason is that C++ compilers are generally brain dead
> and can't be trusted to optimise the SSA form, since
> it would require data flow analysis .. whereas
> expressions are easy to do data flow analysis with .. :)

I'd even call all compilers brain dead (but OTOH I'm still bitter about
not winning the laptop because I trusted Intel's compiler to be able to
do loop unrolling).

>>>Reinterpret cast also won't handle const casting,
>>>which is utterly stupid

> Reinterpret cast discards the type of its argument,
> and retypes it to some other type. This can cause
> all sorts of problems -- it's intrinsically unsafe.

True, but I can imagine there may be cases in template programming where
it is necessary to leave typesafety for some reason, but const
correctness may still be desired. These cases would need additional
template classes that switch on the constness of their argument types.

I'd consider const correctness and the rest of the type system to be
mostly orthogonal, so I can apply one without the other.

    Simon

(All C++ code is equally ugly. The great thing about C++ is that the
ugliness distributes over the entire codebase, and by designing
particularly ugly helper classes, you can make the main program really
beautiful.)

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

   In the languages with manual storage allocation (C, C++,
Pascal, etc.) programmers must obsess on storage management.
It dominates program design and is a constant source of bugs.

   In languages with safe garbage collection (LISP, Java), garbage
collection requires some programmer attention.  Speed,
concurrency, stalls, virtual memory thrashing,
and finalizer semantics all require attention.  Side effects
of finalizers are especially troublesome.

   In languages with safe reference counts (Perl, Python, Visual
Basic), programmers seem to barely be aware of storage
management.  Even finalization tends to be a non-problem.
Visual Basic destroys GUI widgets during finalization,
yet this doesn't seem to require much programmer
attention.

    This bears thinking about.

       John Nagle
    Animats

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

John Nagle wrote:
>   In the languages with manual storage allocation (C, C++,
> Pascal, etc.) programmers must obsess on storage management.
> It dominates program design and is a constant source of bugs.
>
>   In languages with safe garbage collection (LISP, Java), garbage
> collection requires some programmer attention.  Speed,
> concurrency, stalls, virtual memory thrashing,
> and finalizer semantics all require attention.  Side effects
> of finalizers are especially troublesome.
>
>   In languages with safe reference counts (Perl, Python, Visual
> Basic), programmers seem to barely be aware of storage
> management.  Even finalization tends to be a non-problem.
> Visual Basic destroys GUI widgets during finalization,
> yet this doesn't seem to require much programmer
> attention.
>
>    This bears thinking about.
>
No, it doesn't.

I'm going to ignore your comments about how storage management
"dominates program design" and is a "constant source of bugs" any more
than any other part of programming. This has previously been discussed
more times than I care to count, and I doubt either of us could add
anything new to the discussion.

You distguish between "safe reference counts" and "safe garbage
collection" languages (without I notice defining either term, and safe
reference counts aren't common enough to appear on google. try to be
more careful in future.)

I personally see no reason why LISP and java have to use their current
garbage collecting system, or perl, python or visual basic have to use
theirs. On which is more efficent, from my experience (with LISP, java
and python. Not used perl or visual basic much) I would say that neither
has a great advantage over the other. There is a tendancy in Java in
particular to create huge numbers of objects, but this is a language
issue not a garbage collection issue. Also garbage collection can be
more noticable in Java than Python or Perl, as (in my experience) more
interactive programs are written in Java.

I don't want to sound like I'm insulting you, but unless you have some
interesting observation or comment to add to a very, very long running
discussion, and perferably numbers to back it up, then there isn't
really much else to say.

Chris

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

pjp@plauger.com wrote:
> [pjp] Funny, I find that I have to devote about the same amount of
> careful attention to such issues no matter how hard the underlying
> language endeavors to protect me from myself (and/or writing efficient
> code). YMMV, of course.
>
 Yes, sophisticated problems require sophisticated solutions. By definition.

 We often see the statements like these:
  A: GC is good, it solves problems.
  B: GC is bad, it doesn't solve problems.
 that seem to contradict to each other and involve a lot of flames. But what
is actually meant is:
  A: GC is good, it solves [my simple] problems.
  B: GC is bad, it doesn't solve [my complex] problems.
 i.e. no contradiction involved.

 IMHO simple optional GC _without_ finalization facilities is worth to be
considered. It doesn't nullify destructors[*] but simplifies their code: no
need to explicitly free memory. That's all.

[*] "Object lifetime" and "unreachable memory block" are orthogonal concepts.
In particular, Value Objects can easily be "destructed" by GC but Entity
Objects (say, an interface to DBMS transaction) must be destructed manually.
--
         With all respect, Sergey.               http://ders.angen.net/
         mailto : ders at skeptik.net

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

On 12/09/2004 10:24 PM, Thorsten Ottosen wrote:
[snip]
>
> yes, we really need to support optional garbage collection.
>
> | Regretfully, there are a few things in the current standard that would
> | make it non conforming.
>
> If I understand the situation correctly, then present day C++ garbage
> collectors
> cannot remove circular references. If that is true, then we should consider
I assume you mean precise collectors, because the Boehm conservative
can certainly handle cycles in the pointer graph. OTOH, if by
circular references you mean circular reference counting, then
shared_ptr has a conservative method for collecting cycles
which is available as a prototype.  In addition, there's a
precise version for collecting cycles in:

http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sandbox/boost/managed_ptr/

That code is being cleaned up and refactored into a part,

   field_visitor

which can precisely locate any member variable with
field_selector<a_field_visitor> as a super_type.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Thorsten Ottosen wrote:

> If I understand the situation correctly, then present day C++ garbage
> collectors
> cannot remove circular references. If that is true, then we should consider
> adding
> some kind of strict mode which allows garbage collection to work without
> problems.
>
> -Thorsten

    That's quite possible.  The restrictions are painful to retrofit.
But it may not be necessary to solve that problem.

    The key to avoiding circular references is to impose some kind of
hierarchy on objects, and insisting that only pointers down the
hierarchy can be "strong".  For a tree structure, the pointers
to leaves are strong, but backpointers are weak.  This won't
result in circular references.

    One could enforce the hierarchy requirement by requiring
each node to have a "depth", its distance from the root.
Strong pointers are allowed only to objects of greater
depth.  It's possible to write smart pointer classes which
do this.

    If you need true peer to peer pointers, then they should
be weak, and the objects involved owned by some collection.
When you clear or delete the collection, everything it
owns is released.

    Perl storage allocation is often done that way, now that
Perl has weak pointers.  Perl, which is reference-counted,
doesn't enforce this, but circular structures aren't a major
problem in Perl.  In a way, thats's suprising.

    The major reason circular structures aren't a major problem
is that, unlike dangling pointers, they rarely happen by
accident.   Either they're designed into a data structure,
in which case you tend to get memory leaks which show up early
in debugging, or they don't happen at all.  So they tend not
to result in subtle defects or intermittent failures.

    John Nagle
    Animats

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

kanze@gabi-soft.fr wrote:
> The problem has nothing to do with circular references.  The problem is
> that the C++ standard currently allows "hiding" a pointer in ways that
> a
> garbage collector cannot possibly find it.  And when the garbage
> collector cannot find any pointers to an object, it collects the
> object.  If the code then later recovers the pointer, and dereferences
> it, there is a problem.

    C++ iterators provide the beginnings of a way out of that problem.
Iterators have stricter semantics than pointers, even when implemented
as pointers.

    There's a need for a clear distinction between a "pointer to an
object" (this ought to be called a "reference", but the word is
in use), and an iterator.  A pointer to an object can only point
to the beginning of an object.  An iterator is bound to both a
collection and a position within that collection (STLport actually
implements iterators that way in debug mode.)

    Iterators tend to be underutilized, partly because the declaration
syntax is so bulky.  I'd still like to see "let", so that one could
write
 for (let p = tab.begin(), p != tab.end(); p++)
 {}

(Some people want to use "auto" in place of "let", which retrofits
better but is less understandable.)

    The issue of security vulnerabilities due to language design
problems is heating up.  Amit Yoran's parting speech when he
left Homeland Security's US-CERT said quite a bit about this.
It's not clear if his successor will continue the push.

        John Nagle
    Animats

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Dave Harris wrote:
 > kanze@gabi-soft.fr () wrote (abridged):

 >>The problem is that the C++ standard currently allows "hiding" a
 >>pointer in ways that a garbage collector cannot possibly find it.

 > I don't agree. It's my understanding that code like:

 >     void test() {
 >         int i =3D 0;
 >         int *p1 =3D &i;
 >         long x =3D reinterpret_cast<long>( p1 );
 >         int *p2 =3D reinterpret_cast<int *>( x );
 >         ++*p;
 >     }

 > has never been portable. The first cast works only if the target type =
is
 > large enough, and the standard does not require that any sufficiently
 > large integral type be provided. (Adding (void *) to the mix doesn't
 > affect this.)

That's not portable, but that's not my example.  My example uses memcpy
to put the pointer into an unsigned char[sizeof(int*)], and memcpy to
get it back again.  (See =A73.9/2.)  Already, that allows the only
remaining representation of a pointer to be misaligned (which means that
the Boehm collector won't see it).  But of course, you can also
manipulate the buffer in various ways: encrypt it, then decrypte it;
write it to disk, then read it back, etc.  As long as the original bit
pattern is restored to the pointer, you're guaranteed that the pointer
will work.  (Note that this only holds for PODs.  But a pointer is a
POD.)

 > My current understanding is that the real problem is std::less<> for
 > pointer types. If the GC can move objects around in memory, it's hard =
to
 > implement std::less<> efficiently.

Practically speaking, I don't think you can use a copying collector with
C++.  Theoretically, you can't use any collector, because of "pointer
hiding", as described above, but I think that this would actually be a
problem with very few, if any programs.  The problem with less will hit
a lot more real programs.

     [...]

 >>I think that 1) garbage collection must be optional,

 > I am not sure what that means in practice.

Me neither.  At least not in terms of the standard.

 > In my view, GC should not
 > invoke destructors, and I don't even think it should invoke finalisers
 > either. So an implementation with GC should be indistinguishable from =
an
 > implementation with unbounded memory. Since unbounded memory is alread=
y
 > permitted, I don't see what would change if we allowed GC as well.

Sort of:-).  The problem is with "hidden pointers", above.  If GC is
used, you can't allow them.

Other than that, I think what we mean is that mandatory garbage
collection must be optional.  Basically, if I write my program to use
GC, I need GC.  If it is optional, I need some way of telling the
compiler or the system that I need it, and that I want to exercise the
option.

 > I believe the controversial issues are (a) some people still seem to=20
want
 > finalisers and/or destructors to be invoked by the GC; (b) some people
 > rely on the non-portable behaviour described above, and will avoid
 > implementations which don't support it; and (c) we need better
 > reassurances that the std::less<> problem can be solved than the
 > handwaving I give above.

The simplest solution for the std::less<> problem is not to use a
copying collector:-).

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

James Kanze wrote:
> Dave Harris wrote:
>  > kanze@gabi-soft.fr () wrote (abridged):
>
>  >>The problem is that the C++ standard currently allows "hiding" a
>  >>pointer in ways that a garbage collector cannot possibly find it.

    Is it undefined behavior to use "memcpy", etc. on a non-POD
object?

    Should it be an error to cast to a non-POD object?
Maybe it's time to require a "reinterpret_cast" to cast
to a non-POD object, and disallow the use of old-style C
casts to non-POD objects.

    The concept of "reinterpret_cast" is that it's the proper way
to express a deliberate violation of the type rules.
The problem is that legacy C casts can also do this, but
far less visibly.  This is an ongoing source of bugs.

    A program which doesn't contain "reintepret_cast" ought not
to have such type conversions.

> Practically speaking, I don't think you can use a copying collector with
> C++.

    That's realistic.

     John Nagle
     Animats

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Thorsten Ottosen wrote:
> Actually I said that because I thought I read that the boehm collector could
> not detect cycles.

    A bit of background.

    Garbage collectors usually detect cycles just fine.
The troubles with garbage collectors are 1) stalls during
garbage collection, 2) destructor semantics, and 3)
virtual memory thrashing during garbage collection.

1) and 3) are performance problems, and there's been
considerable progress on both.  2) remains a tough
problem for a language with destructors, like C++.
Destructors are called long after the object is
no longer needed, so if they're supposed to release
some resource like a window or a socket, the resource
is released late, if ever.  Destructors may be called
by an unexpected thread at an unexpected time, so they
have to have locking.  Such locking may deadlock the
garbage collector.

Destructor semantics changes drastically with garbage
collection.  That's a tough retrofit for C++.
Microsoft tried with "Managed C++", but the semantics
of destructors there are quite wierd.  Among other things,
they can be called more than once.

   John Nagle
   Animats

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]