(Apologies for the late response.)

On 2006-11-09, Howard Hinnant <howard.hinnant@gmail.com> wrote:
> In article
><45530096$0$3036$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
>  gi3nospam@mariani.ws (Gianni Mariani) wrote:
<snip>
>> T1 - Readlock A
>> T2 - Readlock B
>> T1 - Upgrade A to Writelock (must wait until T2 releases Readlock)
>> T2 - Upgrade A to Writelock (must waut until T1 releases Readlock)
>>
>> oops - neither T1 or T2 will release readlock A!
>
> The proposed upgradable_mutex does not have this problem.  Indeed, this
> problem is the motivating use case for upgradable_mutex.  The
> upgradable_mutex as proposed has unique ownership semantics with respect
> to other upgradable_mutexes.  It can only share with other
> sharable_mutexes.
<snip>

The description of unlock_upgradable_and_lock() says:

"During this operation it is possible that other threads will obtain
sharable or upgradable ownership. However, it is guaranteed that no
other thread will obtain exclusive ownership before this thread does.
Therefore the thread may assume that what it has read under the
upgradable ownership is still valid."

So apparently the following can happen:

T1 - Upgradeable-lock A
T1 - Upgrade A to Write-lock (blocks)
T2 - Upgradeable-lock A
T2 - Upgrade A to Write-lock (blocks)

after which neither thread can be given an exclusive lock before the
other.  The upgrade is allowed to fail (and throw) "due to lack of
system resources", but that doesn't seem to cover this case.  If another
thread is allowed to gain upgradeable ownership while another is waiting
to upgrade, I don't see what use the uniqueness of an upgradeable
ownership provides.  Is there a measurable performance gain?

Ben.

--
Ben Hutchings
Reality is just a crutch for people who can't handle science fiction.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Ben Hutchings wrote:
> (Apologies for the late response.)
>
> On 2006-11-09, Howard Hinnant <howard.hinnant@gmail.com> wrote:
> > In article
> ><45530096$0$3036$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
> >  gi3nospam@mariani.ws (Gianni Mariani) wrote:
> <snip>
> >> T1 - Readlock A
> >> T2 - Readlock B
> >> T1 - Upgrade A to Writelock (must wait until T2 releases Readlock)
> >> T2 - Upgrade A to Writelock (must waut until T1 releases Readlock)
> >>
> >> oops - neither T1 or T2 will release readlock A!
> >
> > The proposed upgradable_mutex does not have this problem.  Indeed, this
> > problem is the motivating use case for upgradable_mutex.  The
> > upgradable_mutex as proposed has unique ownership semantics with respect
> > to other upgradable_mutexes.  It can only share with other
> > sharable_mutexes.
> <snip>
>
> The description of unlock_upgradable_and_lock() says:
>
> "During this operation it is possible that other threads will obtain
> sharable or upgradable ownership. However, it is guaranteed that no
> other thread will obtain exclusive ownership before this thread does.
> Therefore the thread may assume that what it has read under the
> upgradable ownership is still valid."
>
> So apparently the following can happen:
>
> T1 - Upgradeable-lock A
> T1 - Upgrade A to Write-lock (blocks)
> T2 - Upgradeable-lock A
> T2 - Upgrade A to Write-lock (blocks)

No, I interpret the quoted paragraph as merely summarizing the obvious:
that is, once a thread graduates from upgradeable to exclusive
ownership, other threads may at the same time also move up one rung on
the ownership ladder - each advancing thread assuming the ownership
that the thread at the next higher rung just surrendered.

In other words, when the call to unlock_upgradable_and_lock() returns -
not only will the calling thread find itself with exclusive ownership
of the mutex - but it may also find that the upgradeable ownership that
it used to have - now belongs to another thread ... and so on down the
line.

Greg

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

In article <slrner8dfb.5id.ben-public-nospam@decadent.org.uk>,
 Ben Hutchings <ben-public-nospam@decadentplace.org.uk> wrote:

> (Apologies for the late response.)
>
> On 2006-11-09, Howard Hinnant <howard.hinnant@gmail.com> wrote:
> > In article
> ><45530096$0$3036$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
> >  gi3nospam@mariani.ws (Gianni Mariani) wrote:
> <snip>
> >> T1 - Readlock A
> >> T2 - Readlock B
> >> T1 - Upgrade A to Writelock (must wait until T2 releases Readlock)
> >> T2 - Upgrade A to Writelock (must waut until T1 releases Readlock)
> >>
> >> oops - neither T1 or T2 will release readlock A!
> >
> > The proposed upgradable_mutex does not have this problem.  Indeed, this
> > problem is the motivating use case for upgradable_mutex.  The
> > upgradable_mutex as proposed has unique ownership semantics with respect
> > to other upgradable_mutexes.  It can only share with other
> > sharable_mutexes.
> <snip>
>
> The description of unlock_upgradable_and_lock() says:
>
> "During this operation it is possible that other threads will obtain
> sharable or upgradable ownership. However, it is guaranteed that no
> other thread will obtain exclusive ownership before this thread does.
> Therefore the thread may assume that what it has read under the
> upgradable ownership is still valid."
>
> So apparently the following can happen:
>
> T1 - Upgradeable-lock A
> T1 - Upgrade A to Write-lock (blocks)
> T2 - Upgradeable-lock A
> T2 - Upgrade A to Write-lock (blocks)
>
> after which neither thread can be given an exclusive lock before the
> other.  The upgrade is allowed to fail (and throw) "due to lack of
> system resources", but that doesn't seem to cover this case.  If another
> thread is allowed to gain upgradeable ownership while another is waiting
> to upgrade, I don't see what use the uniqueness of an upgradeable
> ownership provides.  Is there a measurable performance gain?

I believe you've pointed out a valid defect in the description of
unlock_upgradable_and_lock().  I believe the "or upgradable" should be
deleted from the description.  This edit is consistent with the
reference implementation I prototyped the proposal with.  Thank you for
pointing it out.

-Howard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

James Kanze wrote:
> Gianni Mariani wrote:
.
>
>> I vote for an overridable but default "cancel" on destruct and join and
>> leave the exception handling to the application.
>
> Cancel is advisory under Unix, and from what I hear, doesn't
> even exist under Windows.  So using cancel under Unix can still
> result in an endless hang in a destructor, and your suggestion
> might not even be implementable under Windows.

An endless hang in a destructor is a bug !  A thread object must have a
well defined shutdown procedure and if some sort of signal needs to be
given (pthread_cancel) or otherwise, there is no problem mandating this
in the standard thread interface.

>
> There are probably cases where you can safely do a join in the
> destructor, and the user should probably be allowed to authorize
> it, but I don't like it as a general policy.

I don't mind making it optional, but I do this in practice and I have
run into very few problems.

>
>> In fact, why not make the delivered exception a "message" to a message
>> handler ?  That way you remove the exception on join issue since it is
>> unlikely that the joining thread is always the thread that can handle
>> the exception anyway and mandating that it like that is a bad idea.
>
> I don't quite get what you are trying to say.  The problem here
> isn't an exception on join; it's calling join as the result of
> an exception (which invokes the destructors).  You surely aren't
> suggesting that exceptions shouldn't do stack walkback.

What I normally see on an unhanded exception is a the stack intact and
either a core dump or debugger showing the PC at the point where
exception was thrown.  I don't want to change that (even if the standard
does not specify it).

Now, if an exception was thown and "managed", it may be interesting to
propagate that exception into another thread's stack or possibly just
handled as an event in a different thread.  In other words, I would
prefer a message passed to a "handler" rather than needing a try/catch
block - I'm not particularly wed to this idea, I just don't see the
reason why an exception in one thread should propagate as an exception
in another thread unless the application (future<> or whatever
synchronization object as "application" in this sense) chose to perform
the propagation.  In other words, the propagation should be handled by
choice and not mandated by the thread interface.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Gianni Mariani wrote:
> James Kanze wrote:
> > Gianni Mariani wrote:

> >> I vote for an overridable but default "cancel" on destruct and join and
> >> leave the exception handling to the application.

> > Cancel is advisory under Unix, and from what I hear, doesn't
> > even exist under Windows.  So using cancel under Unix can still
> > result in an endless hang in a destructor, and your suggestion
> > might not even be implementable under Windows.

> An endless hang in a destructor is a bug !

Which is why we don't want to unconditionally join in a
destructor.  A join may potentially hang endlessly, and there is
no way, at least under Posix, to prevent this.

> A thread object must have a
> well defined shutdown procedure and if some sort of signal needs to be
> given (pthread_cancel) or otherwise, there is no problem mandating this
> in the standard thread interface.

Except that it isn't implementable under Posix (nor, I think,
under Unix).  Threads must collaborate, but there is no way in
standardese to define what "collaboration" means.

    [...]
> >> In fact, why not make the delivered exception a "message" to a message
> >> handler ?  That way you remove the exception on join issue since it is
> >> unlikely that the joining thread is always the thread that can handle
> >> the exception anyway and mandating that it like that is a bad idea.

> > I don't quite get what you are trying to say.  The problem here
> > isn't an exception on join; it's calling join as the result of
> > an exception (which invokes the destructors).  You surely aren't
> > suggesting that exceptions shouldn't do stack walkback.

> What I normally see on an unhanded exception is a the stack intact and
> either a core dump or debugger showing the PC at the point where
> exception was thrown.  I don't want to change that (even if the standard
> does not specify it).

I wasn't so much thinking of a totally unhandled exception
(which leads to termination of the process), but one which
wasn't expected in the function which created the thread (and
expected to join with it), and which propagates up beyond the
function.

> Now, if an exception was thown and "managed", it may be interesting to
> propagate that exception into another thread's stack or possibly just
> handled as an event in a different thread.

Agreed.

At present, there's no way to "clone" an exception at all,
unless you catch it with its exact type.  That problem has to be
addressed (and is being addressed).  Beyond that, I'd say that
it's largely up to the user; he can catch the exception, make a
copy of it, and then do with it what he wants.

> In other words, I would
> prefer a message passed to a "handler" rather than needing a try/catch
> block - I'm not particularly wed to this idea, I just don't see the
> reason why an exception in one thread should propagate as an exception
> in another thread unless the application (future<> or whatever
> synchronization object as "application" in this sense) chose to perform
> the propagation.

> In other words, the propagation should be handled by
> choice and not mandated by the thread interface.

I agree.  An uncaught exception is a fatal error, regardless of
the thread in which it occurs.  If the user catches the
exception at the top level of the thread, he should be able to
do with it what he wants: propagate it back to the joining
thread, send it to some other message queue, or even log it and
forget it.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

James Kanze wrote:
> Gianni Mariani wrote:
.
>> An endless hang in a destructor is a bug !
>
> Which is why we don't want to unconditionally join in a
> destructor.  A join may potentially hang endlessly, and there is
> no way, at least under Posix, to prevent this.

What I tried to imply is that you can't prevent programmers from writing
erroneous code.  This kind of bug is usually very repetitive and so it's
easy to track down and fix.  What can be done from a standard
perspective is provide a mandated "shutdown" procedure.  It might simply
be a flag that means 'stop doing stuff already' to the thread being
destroyed.  Whatever it is, it's going to need co-operation from the
programmer.

>
>> A thread object must have a
>> well defined shutdown procedure and if some sort of signal needs to be
>> given (pthread_cancel) or otherwise, there is no problem mandating this
>> in the standard thread interface.
>
> Except that it isn't implementable under Posix (nor, I think,
> under Unix).  Threads must collaborate, but there is no way in
> standardese to define what "collaboration" means.

Other than to perhaps mandate an interface.
.
>
> I agree.  An uncaught exception is a fatal error, regardless of
> the thread in which it occurs.  If the user catches the
> exception at the top level of the thread, he should be able to
> do with it what he wants: propagate it back to the joining
> thread, send it to some other message queue, or even log it and
> forget it.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

In article <1163152319.119671.250100@h48g2000cwc.googlegroups.com>,
 "James Kanze" <james.kanze@gmail.com> wrote:

> Howard Hinnant wrote:
>
> > I've seen lots of use cases for either desire:  join on
> > destruct, detach on destruct.  It is nice to easily allow
> > either behavior.
>
> Or neither: I don't like anything that essential being delegated
> to a destructor (which can be called as a result of an uncaught
> exception).  In general, I would consider calling the destructor
> on a joinable (but unjoined) thread should be illegal, and both
> joining and detaching should be explicit behavior on the part of
> the user, and not because someone forgot to catch an exception.

What behavior would you suggest for ~thread() where someone forgot to
explicitly join or detach?  Error not detected?  Throw an exception?
Terminate?

> The issue concerning joinable depends somewhat on how and if
> futures are supported, but I don't think a destructor should
> ever perform an action that can wait for a potentially infinite
> amount of time.

<nod> That is one disadvantage to join on destruct.

> Note that it is arguable that threads should never be detached,
> since it is necessary to wait until all threads stop in order to
> do a clean shutdown.

What is the difference between waiting for a potentially infinite amount
of time for an object to destruct, and waiting for a potentially
infinite amount of time to return from main?

> This is probably an overly extreme position;

<nod>

> But I think it safe to say that in many cases where the
> application is using detached threads, it really should be
> transfering the threads to some sort of reaper object, which on
> shutdown will signal each thread that it should terminate, and
> then join on each of the threads, possibly with a time out, just
> in case.

Sounds like a good use case for a thread pool, or thread group.

> The problem, of course, is that calling exit (either directly,
> or by returning from main) will generally result in undefined
> behavior if any other threads are still running.

Are you saying we should outlaw the following behavior?

#include <thread>
#include <cstdio>
#include <cstdlib>

bool stop_now = false;
std::mutex mut;

extern "C" void bye()
{
    std::exclusive_lock<std::mutex> lk(mut);
    std::printf("stopping\n");
    stop_now = true;
}

void f()
{
    while (true)
    {
        std::exclusive_lock<std::mutex> lk(mut);
        if (stop_now)
            return;
        std::printf("f running\n");
    }
}

int main()
{
    (std::thread(f));  // detaches f on destruct
    std::atexit(bye);
}

..
f running
f running
f running
stopping

The program above detaches a thread, lets it continue running after main
returns, but arranges for it to cleanly shut down via other
communication sometime during the atexit chain.

Valid program or not?

-Howard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

James Kanze wrote:
> Howard Hinnant wrote:
>
>> I've seen lots of use cases for either desire:  join on
>> destruct, detach on destruct.  It is nice to easily allow
>> either behavior.
>
.
>
> The problem, of course, is that calling exit (either directly,
> or by returning from main) will generally result in undefined
> behavior if any other threads are still running.

Exactly.

Given the last paragraph, I don't see how you can support your current
position.

The "undefined behaviour" you refer to here results in a very hard to
track down bug on today's implementations.  This is exactly one of the
reasons that I do use join on destruct semantics.  Yep, sure, if the
application code has a bug and does not cancel/cleanup/terminate then
that's an application BUG. This type of bug is usually very easy to
track down and fix since on inspection it's quite clear what is going on.

Given this, I don't understand your issue with "forgot to catch".  It
seems like you're trying to outlaw well defined behaviour.  I argue that
if something undesirable happens (like an exception initialed destructor
hanging), it is an application issue.  If the application writer wishes
to abort, it's a behaviour that should be explicitly requested not a
mandated default.

I suppose all I'm arguing is "what is the default".  Signal(cancel)+join
on destruct seems like a far more desirable alternative.

As for thread cancel mechanisms, I just don't understand how that can
work properly in the vast majority of threaded applications.  I agree
that there should be a mandated way to cause a thread wrap up quickly
and "return" which I also don't understand why that mechanism is not
something that has a standard interface that is simply overridden by the
  client.

I see the whole pthread_cancel mechanism to be far more complex than it
needs to be, especially for C++ code which already has a well defined
cleanup mechanism.

My experience shows this whole area of managing thread lifetime is where
most programmers get unstuck.

As for other places where programmers get unstuck, it is the whole
asynchronous event mechanism.  The "future" concept is a great start but
I think that it addresses a very narrow slice of real world cases.  I'll
need to write up a few cases to make the point.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
> In article <1163152319.119671.250100@h48g2000cwc.googlegroups.com>,
>  "James Kanze" <james.kanze@gmail.com> wrote:
.
>
> The program above detaches a thread, lets it continue running after main
> returns, but arranges for it to cleanly shut down via other
> communication sometime during the atexit chain.
>
> Valid program or not?

No.  bye() must join with the thread.

The thing is that once a program starts destroying globals, or the
system starts unloading DLL's/.so's everything past returning from main
is undefined.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

In article
<4554d356$0$3015$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
 gi3nospam@mariani.ws (Gianni Mariani) wrote:

> Howard Hinnant wrote:
> > In article <1163152319.119671.250100@h48g2000cwc.googlegroups.com>,
> >  "James Kanze" <james.kanze@gmail.com> wrote:
> .
> >
> > The program above detaches a thread, lets it continue running after main
> > returns, but arranges for it to cleanly shut down via other
> > communication sometime during the atexit chain.
> >
> > Valid program or not?
>
> No.  bye() must join with the thread.
>
> The thing is that once a program starts destroying globals, or the
> system starts unloading DLL's/.so's everything past returning from main
> is undefined.

Are you saying that today's single threaded atexit-registered functions
can not access *any* globals?

That's not the way I read C++03, 3.6.3p3:

> -3- If a function is registered with atexit (see <cstdlib>,
> lib.support.start.term) then following the call to exit, any objects with
> static storage duration initialized prior to the registration of that
> function shall not be destroyed until the registered function is called from
> the termination process and has completed. For an object with static storage
> duration constructed after a function is registered with atexit, then
> following the call to exit, the registered function is not called until the
> execution of the object's destructor has completed. If atexit is called
> during the construction of an object, the complete object to which it belongs
> shall be destroyed before the registered function is called.

-Howard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
.
> Are you saying that today's single threaded atexit-registered functions
> can not access *any* globals?

It seems the thread running f() can run after all atexit functions are
called. Your code does not synchronize the exit of the f() method, does it ?

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

In article
<4554fc75$0$3015$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
 Gianni Mariani <gi3nospam@mariani.ws> wrote:

> Howard Hinnant wrote:
> .
> > Are you saying that today's single threaded atexit-registered functions
> > can not access *any* globals?
>
> It seems the thread running f() can run after all atexit functions are
> called. Your code does not synchronize the exit of the f() method, does it ?

That is correct.  f() could return after all atexit functions are done
running and whatever else the main thread has to do to shut down the
process (which would include killing f()).  All my example program
showed was that f() and the main() thread might continue to communicate
via a global while running the atexit chain (and there could well be a
bug still there in my example, just the concept of multithreading at
atexit time is what I'm exploring).

-Howard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
> In article
> <4554fc75$0$3015$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
>  Gianni Mariani <gi3nospam@mariani.ws> wrote:
>
>> Howard Hinnant wrote:
>> .
>>> Are you saying that today's single threaded atexit-registered functions
>>> can not access *any* globals?
>> It seems the thread running f() can run after all atexit functions are
>> called. Your code does not synchronize the exit of the f() method, does it ?
>
> That is correct.  f() could return after all atexit functions are done
> running and whatever else the main thread has to do to shut down the
> process (which would include killing f()).  All my example program
> showed was that f() and the main() thread might continue to communicate
> via a global while running the atexit chain (and there could well be a
> bug still there in my example, just the concept of multithreading at
> atexit time is what I'm exploring).

Ah, then the call to std::printf is at risk of exhibiting undefined
behaviour.  It sounds like you don't believe me, but having experienced
it for real in many situations, I now firmly believe that the only way
to eliminate these kinds of race conditions is to provide a well defined
shut-down procedure that joins every thread that was created before
returning from main.

I also have a policy to never invoke a new thread before main() (through
global object initializers).  To that end, I have a "main() initializer"
system that is used to "initialize" any objects that require threads or
in general initialization after main and shut them down before main()
exits in the Austria C++ library.

The C++ standard should strongly encourage development of simple well
defined use cases for threads and make programmers go out of their way
to create complex and potentially nonsensical/unreliable ones.  The code
posted earlier is never going to work reliably in real life without
imposition of unjustifiable constraints in the standard.

I have seen so many very unnecessarily complex thread management
strategies and there is a chance here to "get it right" and result in
more reliable code from the start.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Gianni Mariani wrote:
> James Kanze wrote:
> > Howard Hinnant wrote:

> >> I've seen lots of use cases for either desire:  join on
> >> destruct, detach on destruct.  It is nice to easily allow
> >> either behavior.

> > The problem, of course, is that calling exit (either directly,
> > or by returning from main) will generally result in undefined
> > behavior if any other threads are still running.

> Exactly.

> Given the last paragraph, I don't see how you can support your current
> position.

What in my current position is in disagreement with this?
(Except, maybe, that I haven't outright said that we shouldn't
support detached threads.)

> The "undefined behaviour" you refer to here results in a very
> hard to track down bug on today's implementations.  This is
> exactly one of the reasons that I do use join on destruct
> semantics.  Yep, sure, if the application code has a bug and
> does not cancel/cleanup/terminate then that's an application
> BUG. This type of bug is usually very easy to track down and
> fix since on inspection it's quite clear what is going on.

Crashing the program makes it even easier to track down and fix.
Can you give an application scenario where join on destruct is
useful in correct code?

> Given this, I don't understand your issue with "forgot to catch".  It
> seems like you're trying to outlaw well defined behaviour.  I argue that
> if something undesirable happens (like an exception initialed destructor
> hanging), it is an application issue.  If the application writer wishes
> to abort, it's a behaviour that should be explicitly requested not a
> mandated default.

> I suppose all I'm arguing is "what is the default".  Signal(cancel)+join
> on destruct seems like a far more desirable alternative.

It's certainly preferrable to simply detaching.  If there are
reasonable scenarios as to where it is useful in a correct
application, I'll consider it.  Until then, it's a fatal error,
along the same lines as not catching an exception.  (After all,
it indirectly means that you cannot catch the exception.)

> As for thread cancel mechanisms, I just don't understand how that can
> work properly in the vast majority of threaded applications.

It depends on what you mean by "thread cancel mechanisms".  The
Posix mechanism which goes by that name is advisory, and not
forcing.  But any given application does need some means for all
running threads to be notified of a requested shutdown.

> I agree
> that there should be a mandated way to cause a thread wrap up quickly
> and "return" which I also don't understand why that mechanism is not
> something that has a standard interface that is simply overridden by the
> client.

> I see the whole pthread_cancel mechanism to be far more complex than it
> needs to be, especially for C++ code which already has a well defined
> cleanup mechanism.

The pthread_cancel mechanism has the advantage of being
implemented in kernel code, and thus being able to interrupt
blocking system requests.

--
James Kanze (Gabi Software)            email: james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
> In article <1163152319.119671.250100@h48g2000cwc.googlegroups.com>,
>  "James Kanze" <james.kanze@gmail.com> wrote:

> > Howard Hinnant wrote:

> > > I've seen lots of use cases for either desire:  join on
> > > destruct, detach on destruct.  It is nice to easily allow
> > > either behavior.

> > Or neither: I don't like anything that essential being delegated
> > to a destructor (which can be called as a result of an uncaught
> > exception).  In general, I would consider calling the destructor
> > on a joinable (but unjoined) thread should be illegal, and both
> > joining and detaching should be explicit behavior on the part of
> > the user, and not because someone forgot to catch an exception.

> What behavior would you suggest for ~thread() where someone forgot to
> explicitly join or detach?  Error not detected?  Throw an exception?
> Terminate?

Fatal error.  The true C/C++ tradition would be "undefined
behavior".  But I think we both agree that that's not a
particularly good tradition, if we can avoid it.  As for
throwing an exception... I expect that most of the time the
problem occurs, it will be because we are unwinding the stack as
a result of an exception, so throwing an exception becomes just
another case of terminate.

Note that it's not really a killer critera, since I can always
write my own wrapper which does whatever is necessary (except
that I don't know how to detach without calling the destructor
of the thread); that's what I do with boost::threads currently.
On the other hand, the current situation will lead to a plethora
of subtly incompatible wrapper classes.  Not an ideal situation,
but since we're not talking about a class which typically
appears at the interface level between sub-systems, probably not
a serious problem in practice.

> > The issue concerning joinable depends somewhat on how and if
> > futures are supported, but I don't think a destructor should
> > ever perform an action that can wait for a potentially infinite
> > amount of time.

> <nod> That is one disadvantage to join on destruct.

A very big one, IMHO.  While I don't expect thread unwinding due
to an exception to be blazingly fast, I don't normally expect it
to sit there for hours in one destructor, either.

> > Note that it is arguable that threads should never be detached,
> > since it is necessary to wait until all threads stop in order to
> > do a clean shutdown.

> What is the difference between waiting for a potentially
> infinite amount of time for an object to destruct, and waiting
> for a potentially infinite amount of time to return from main?

Not enough:-).  There is a difference: presumably, if I catch an
exception, I'm prepared to recover and continue, where as in
main, I"m probably trying to shut down.  But it is a problem.  A
very big one: we know that in practice, there is no reliable way
to shut down a non-collaborating thread.

> > This is probably an overly extreme position;

> <nod>

> > But I think it safe to say that in many cases where the
> > application is using detached threads, it really should be
> > transfering the threads to some sort of reaper object, which on
> > shutdown will signal each thread that it should terminate, and
> > then join on each of the threads, possibly with a time out, just
> > in case.

> Sounds like a good use case for a thread pool, or thread group.

Possibly.  I've generally used thread groups, when I've used
them, at a lower level of granularity.  Most of the thread
classes I've used (and designed) have also contained a static
registry with *all* threads, and a static function shutdown,
which signaled all running threads, and waited for them to
finish.  I'm not at all sure that my solutions, however, would
be appropriate in the standard; they almost always involved some
very application specific aspects.  (And they've all counted on
collaborating threads, which explicitly checked from time to
time whether shutdown had been requested or not.)

> > The problem, of course, is that calling exit (either directly,
> > or by returning from main) will generally result in undefined
> > behavior if any other threads are still running.

> Are you saying we should outlaw the following behavior?

> #include <thread>
> #include <cstdio>
> #include <cstdlib>

> bool stop_now = false;
> std::mutex mut;

> extern "C" void bye()
> {
>     std::exclusive_lock<std::mutex> lk(mut);
>     std::printf("stopping\n");
>     stop_now = true;
> }

> void f()
> {
>     while (true)
>     {
>         std::exclusive_lock<std::mutex> lk(mut);
>         if (stop_now)
>             return;
>         std::printf("f running\n");
>     }
> }

> int main()
> {
>     (std::thread(f));  // detaches f on destruct
>     std::atexit(bye);
> }

> ..
> f running
> f running
> f running
> stopping

> The program above detaches a thread, lets it continue running
> after main returns, but arranges for it to cleanly shut down
> via other communication sometime during the atexit chain.

Definitly.  If the thread uses any global resources, this is
irremediably broken.

Waiting in bye() until all of the threads have actually finished
(which can be determined by other means than join) would make it
almost acceptable, but there can still be problems involving
global resources freed in an earlier function invoked by atexit.

Putting the signalling and wait code at the end of main makes it
OK.

> Valid program or not?

I wouldn't like to have to formulate the guarantees necessary
for it to be valid.

Note that making it undefined behavior doesn't necessarily mean
that it can't work anywhere.  In this case, it may simply mean
that the limitations necessary for it to work will be negotiated
between the compiler implementor and his customers.  In
practice, I don't think you can give any reasonably usable
guarantees under Posix, but perhaps other systems (and some
Posix compliant systems give more guarantees than just Posix as
well).

--
James Kanze (Gabi Software)            email: james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

James Kanze wrote:
> Gianni Mariani wrote:
>> James Kanze wrote:
>>> Howard Hinnant wrote:
>
>>>> I've seen lots of use cases for either desire:  join on
>>>> destruct, detach on destruct.  It is nice to easily allow
>>>> either behavior.
>
>>> The problem, of course, is that calling exit (either directly,
>>> or by returning from main) will generally result in undefined
>>> behavior if any other threads are still running.
>
>> Exactly.
>
>> Given the last paragraph, I don't see how you can support your current
>> position.
>
> What in my current position is in disagreement with this?
> (Except, maybe, that I haven't outright said that we shouldn't
> support detached threads.)

Fine, I must have misinterpreted your post.  My bad.  So then we agree
that detached threads are a bad thing and the standard must strongly
discourage the use of detached threads.

>
>> The "undefined behaviour" you refer to here results in a very
>> hard to track down bug on today's implementations.  This is
>> exactly one of the reasons that I do use join on destruct
>> semantics.  Yep, sure, if the application code has a bug and
>> does not cancel/cleanup/terminate then that's an application
>> BUG. This type of bug is usually very easy to track down and
>> fix since on inspection it's quite clear what is going on.
>
> Crashing the program makes it even easier to track down and fix.
> Can you give an application scenario where join on destruct is
> useful in correct code?

We'll just have to disagree on this.  I think that attaching a debugger
to a process and being able to observe the nature of the code makes it
easier to track down.

>
>> Given this, I don't understand your issue with "forgot to catch".  It
>> seems like you're trying to outlaw well defined behaviour.  I argue that
>> if something undesirable happens (like an exception initialed destructor
>> hanging), it is an application issue.  If the application writer wishes
>> to abort, it's a behaviour that should be explicitly requested not a
>> mandated default.
>
>> I suppose all I'm arguing is "what is the default".  Signal(cancel)+join
>> on destruct seems like a far more desirable alternative.
>
> It's certainly preferrable to simply detaching.  If there are
> reasonable scenarios as to where it is useful in a correct
> application, I'll consider it.  Until then, it's a fatal error,
> along the same lines as not catching an exception.  (After all,
> it indirectly means that you cannot catch the exception.)

OK, I hadn't thought through the whole exception thing since this whole
idea of exception propagation is kind of new to me.

My preferred way of handling exceptions in a thread is to have them
handled by the application otherwise just do the regular "uncaught
exception thing" (crash and burn).  I find this far more preferable
since at the point of the exception being thrown I usually have a core
dump or a hung process I can go and debug easily.  Once the stack is
unwound, it becomes much harder to diagnose.

I certainly don't want to see the current "uncaught exception" in a
thread behaviour be modified, I would like to see an uncaught exception
at the stack frame that caused it.  I would be able to live with a child
thread hanging and awaiting for it's reaper to come along but that would
almost certainly mean deadlock in an otherwise "well formed" application.

It appears that for exception propagation to work, you must "store and
forward" the exception because the stack must be unwound (releasing
possibly held mutexes) otherwise there will be deadlocks.

I really really do prefer the uncaught exception behaviour to the store
and forward behaviour.

>
>> As for thread cancel mechanisms, I just don't understand how that can
>> work properly in the vast majority of threaded applications.
>
> It depends on what you mean by "thread cancel mechanisms".  The
> Posix mechanism which goes by that name is advisory, and not
> forcing.  But any given application does need some means for all
> running threads to be notified of a requested shutdown.

Yes, it does.  I have used application mechanisms in the past, I'm not
opposed to having a "proper" standard mechanism.

>
>> I agree
>> that there should be a mandated way to cause a thread wrap up quickly
>> and "return" which I also don't understand why that mechanism is not
>> something that has a standard interface that is simply overridden by the
>> client.
>
>> I see the whole pthread_cancel mechanism to be far more complex than it
>> needs to be, especially for C++ code which already has a well defined
>> cleanup mechanism.
>
> The pthread_cancel mechanism has the advantage of being
> implemented in kernel code, and thus being able to interrupt
> blocking system requests.

I assume that an application calling pthread_testcancel (or the
equivalent in the proposed standard) will have the stack rolled back
similar to when an exception is thrown ?

G

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

In article <1163341583.146827.43850@f16g2000cwb.googlegroups.com>,
 "James Kanze" <james.kanze@gmail.com> wrote:

> > What behavior would you suggest for ~thread() where someone forgot to
> > explicitly join or detach?  Error not detected?  Throw an exception?
> > Terminate?
>
> Fatal error.  The true C/C++ tradition would be "undefined
> behavior".  But I think we both agree that that's not a
> particularly good tradition, if we can avoid it.

I'm a little concerned about having an object on the stack that you
can't just destruct at any moment without risking a fatal error.  I'm
not sure there exists any other object in the std::lib whose use is that
finicky:

template <class T>
void foo(T t)
{
    // If we don't make the right call to t, foo() ends in terminate()
}

> As for
> throwing an exception... I expect that most of the time the
> problem occurs, it will be because we are unwinding the stack as
> a result of an exception, so throwing an exception becomes just
> another case of terminate.

Actually I expect most calls to ~thread() will execute in a
non-exceptional context.  Having ~thread() throw would pretty much mean
that std::container<thread> doesn't behave well.

>
> Note that it's not really a killer critera, since I can always
> write my own wrapper which does whatever is necessary (except
> that I don't know how to detach without calling the destructor
> of the thread); that's what I do with boost::threads currently.

Assuming std::thread looked something like boost::thread, I think adding
a detach() member to it would be a fine idea.  I'm a little disconcerted
that this would leave the client with a handle to a thread which can no
longer reliably do things like fetch thread identity.  Otoh, and again
assuming boost::thread semantics plus move semantics, it is trivial to
get the detach() functionality anyway, so might as well standardize it:

// Without a member detach()

inline void detach_thread(std::thread) {}

void foo()
{
    std::thread(f) t;
    // ...
    detach_thread(std::move(t));
    // t detached here
}

-------------------------

// With a member detach()

void foo()
{
    std::thread(f) t;
    // ...
    t.detach();
    // t detached here
}

<shrug>  Making boost::thread movable will really change the way people
use it (much for the better).  It will put a lot more power in the
client's hands.  E.g.

std::vector<boost::thread> net_io_thread_group;
net_io_thread_group.push_back(boost::thread(f));  // ok!

> > > The issue concerning joinable depends somewhat on how and if
> > > futures are supported, but I don't think a destructor should
> > > ever perform an action that can wait for a potentially infinite
> > > amount of time.
>
> > <nod> That is one disadvantage to join on destruct.
>
> A very big one, IMHO.  While I don't expect thread unwinding due
> to an exception to be blazingly fast, I don't normally expect it
> to sit there for hours in one destructor, either.

The thought of cancel-on-destruct just crossed my mind.  I haven't yet
decided whether I like that idea or hate it.  I'm currently leaning
towards the latter... :-)

> > Sounds like a good use case for a thread pool, or thread group.
>
> Possibly.  I've generally used thread groups, when I've used
> them, at a lower level of granularity.  Most of the thread
> classes I've used (and designed) have also contained a static
> registry with *all* threads, and a static function shutdown,
> which signaled all running threads, and waited for them to
> finish.  I'm not at all sure that my solutions, however, would
> be appropriate in the standard; they almost always involved some
> very application specific aspects.  (And they've all counted on
> collaborating threads, which explicitly checked from time to
> time whether shutdown had been requested or not.)

Agreed.  I would like to make it easy for user's to create such
registries if they want to (multiple registries even).  But I'm loath to
mandate them.

> > Are you saying we should outlaw the following behavior?
>
> > #include <thread>
> > #include <cstdio>
> > #include <cstdlib>
>
> > bool stop_now = false;
> > std::mutex mut;
>
> > extern "C" void bye()
> > {
> >     std::exclusive_lock<std::mutex> lk(mut);
> >     std::printf("stopping\n");
> >     stop_now = true;
> > }
>
> > void f()
> > {
> >     while (true)
> >     {
> >         std::exclusive_lock<std::mutex> lk(mut);
> >         if (stop_now)
> >             return;
> >         std::printf("f running\n");
> >     }
> > }
>
> > int main()
> > {
> >     (std::thread(f));  // detaches f on destruct
> >     std::atexit(bye);
> > }
>
> > ..
> > f running
> > f running
> > f running
> > stopping
>
> > The program above detaches a thread, lets it continue running
> > after main returns, but arranges for it to cleanly shut down
> > via other communication sometime during the atexit chain.
>
> Definitly.  If the thread uses any global resources, this is
> irremediably broken.

Not if the global resource is not yet destructed.  The current atexit /
destructor sequence is well defined.  Today globals can safely be
manipulated at atexit time if the global construction is properly
coordinated with the atexit registration.

> > Valid program or not?
>
> I wouldn't like to have to formulate the guarantees necessary
> for it to be valid.

I think that work has already been done:

3.6.3p3:  Interleave static destructors with atexit registered functions.

-Howard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
> In article
> <4554d356$0$3015$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
>  gi3nospam@mariani.ws (Gianni Mariani) wrote:

> > Howard Hinnant wrote:
> > > In article <1163152319.119671.250100@h48g2000cwc.googlegroups.com>,
> > >  "James Kanze" <james.kanze@gmail.com> wrote:

> > > The program above detaches a thread, lets it continue running after main
> > > returns, but arranges for it to cleanly shut down via other
> > > communication sometime during the atexit chain.

> > > Valid program or not?

> > No.  bye() must join with the thread.

> > The thing is that once a program starts destroying globals, or the
> > system starts unloading DLL's/.so's everything past returning from main
> > is undefined.

> Are you saying that today's single threaded atexit-registered functions
> can not access *any* globals?

I don't think that that's the point.  Suppose you have several
threads running.  You execute bye().  No problem.  You continue
execution in the main thread, executing further destructors.
For example, the destructor for the mutex.  What happens to the
thread waiting on the mutex?

The order of functions registered with atexit is defined.  The
order things happen in a thread relative to them isn't.
Presumably, the main thread could actually finish executing all
of the destructors, and start cleaning up global memory before
one of the waiting threads woke up.

> That's not the way I read C++03, 3.6.3p3:

Agreed.  But you'll also agree that this only talks of single
threaded behavior.

--
James Kanze (Gabi Software)            email: james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

In article <1163342424.043179.87490@f16g2000cwb.googlegroups.com>,
 "James Kanze" <james.kanze@gmail.com> wrote:

> > That's not the way I read C++03, 3.6.3p3:
>
> Agreed.  But you'll also agree that this only talks of single
> threaded behavior.

Sure.  But imho 3.6.3p3 shouldn't say one thing for single threaded and
something completely different for multithreaded.  Rather the model for
multithreading should be a natural extension of what it says for single
threaded.

> Suppose you have several
> threads running.  You execute bye().  No problem.  You continue
> execution in the main thread, executing further destructors.
> For example, the destructor for the mutex.  What happens to the
> thread waiting on the mutex?
>
> The order of functions registered with atexit is defined.  The
> order things happen in a thread relative to them isn't.
> Presumably, the main thread could actually finish executing all
> of the destructors, and start cleaning up global memory before
> one of the waiting threads woke up.

You're right.  I had a bug.  Here I've attempted to correct it.

Well defined?

#include <thread>
#include <cstdio>
#include <cstdlib>

bool stop_now = false;
std::mutex mut;
std::condition<std::exclusive_lock<std::mutex> > cv;

extern "C" void bye()
{
    std::exclusive_lock<std::mutex> lk(mut);
    std::printf("stopping f\n");
    stop_now = true;
    while (stop_now)
        cv.wait(lk);
    std::printf("f stopped\n");
}

void f()
{
    while (true)
    {
        std::exclusive_lock<std::mutex> lk(mut);
        if (stop_now)
        {
            stop_now = false;
            cv.notify_one();
            return;
        }
        std::printf("f running\n");
    }
}

int main()
{
    (std::thread(f));
    std::atexit(bye);
}

Above bye() is guaranteed to start *and* finish executing before mut and
cv destruct.  The "f" thread is detached, and continues running after
main exits.  The "f" thread is guaranteed to shut down after bye()
starts executing and before bye() finishes executing (or at least
guaranteed not to touch any global state after bye() returns).

Do we have sufficient motivation to call this program ill-formed just
because it lets a detached thread run for awhile after main returns?

I don't think we do.

I think the most we can say is similar to what we already say in 3.6.3p2:

> If a function contains a local object of static storage duration that has
> been destroyed and the function is called during the destruction of an object
> with static storage duration, the program has undefined behavior if the flow
> of control passes through the definition of the previously destroyed local
> object.

(needs to be expanded to all objects with static storage duration, not
just function-locals)

Oh, wait a sec, I think namespace scope objects are covered in 3.8p5.

-Howard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
> In article <1163341583.146827.43850@f16g2000cwb.googlegroups.com>,
>  "James Kanze" <james.kanze@gmail.com> wrote:

> > > What behavior would you suggest for ~thread() where
> > > someone forgot to explicitly join or detach?  Error not
> > > detected?  Throw an exception?  Terminate?

> > Fatal error.  The true C/C++ tradition would be "undefined
> > behavior".  But I think we both agree that that's not a
> > particularly good tradition, if we can avoid it.

> I'm a little concerned about having an object on the stack
> that you can't just destruct at any moment without risking a
> fatal error.

I'm not particularly happy about it myself.  It just seems less
bad than the alternatives.  If we admit that it can only happen
as a result of a program error.  Perhaps we need some sort of a
user choice.

More generally, it depends somewhat on what we do with futures.
I've not really made up my mind about futures, but if we do go
that route, it changes a lot of things.  In particular, we could
consider a thread conceptually non-joinable until a future was
acquired for it.  And there would be no problem with destroying
it.  (At the implementation level, of course, it would actually
be detached in the destructor, if and only if no future had been
acquired for it.)  Of course, part of the problem with futures
is what happens if the last one for a given thread is destructed
before a join has occurred.

> I'm not sure there exists any other object in the std::lib
> whose use is that finicky:

> template <class T>
> void foo(T t)
> {
>     // If we don't make the right call to t, foo() ends in terminate()
> }

I don't know, but I'm not sure the small example you give is
relevant.  Unless it has changed from boost, a thread isn't
copiable, and so cannot be passed by value.  (And of course, if
passed by reference, the destructor won't be called.)

> > As for
> > throwing an exception... I expect that most of the time the
> > problem occurs, it will be because we are unwinding the stack as
> > a result of an exception, so throwing an exception becomes just
> > another case of terminate.

> Actually I expect most calls to ~thread() will execute in a
> non-exceptional context.  Having ~thread() throw would pretty
> much mean that std::container<thread> doesn't behave well.

Agreed, but since it isn't copiable anyway, it cannot be used in
a container today.

> > Note that it's not really a killer critera, since I can always
> > write my own wrapper which does whatever is necessary (except
> > that I don't know how to detach without calling the destructor
> > of the thread); that's what I do with boost::threads currently.

> Assuming std::thread looked something like boost::thread, I
> think adding a detach() member to it would be a fine idea.

I would say that this would remove the main objection I have to
boost threads.  I could live with either an implicit join of an
unjoined, undetached thread in the destructor, or a fatal error
in this case.  But IMHO, something as distinctive as detach
shouldn't be an "accidental" side effect of the destructor.

Another "objection" I have to boost threads is that it seems to
coalesce several distinct concepts in one type---sleep and
yield, for example, really don't belong there.  But since I
don't have any alternative to propose, much less one with which
people have any pratical experience, I'll let it go.  And while
I'm not happy about the facilities for returning values, out
parameters, and especially propagating exceptions, I think that
they are problems which should probably be solved outside the
context of just boost::threads---means for finding out the
actual type of an exception, cloning it, and throwing an object
based on its dynamic type could be generally useful, and
handling return values and out parameters is probably more a
question of finding a good design pattern than anything
requiring a modification in the specification.

    [...]
> <shrug>  Making boost::thread movable will really change the way people
> use it (much for the better).  It will put a lot more power in the
> client's hands.  E.g.

> std::vector<boost::thread> net_io_thread_group;
> net_io_thread_group.push_back(boost::thread(f));  // ok!

Which would, in turn, remove some of the motivation for (as yet
untried?) futures.

> > > > The issue concerning joinable depends somewhat on how and if
> > > > futures are supported, but I don't think a destructor should
> > > > ever perform an action that can wait for a potentially infinite
> > > > amount of time.

> > > <nod> That is one disadvantage to join on destruct.

> > A very big one, IMHO.  While I don't expect thread unwinding due
> > to an exception to be blazingly fast, I don't normally expect it
> > to sit there for hours in one destructor, either.

> The thought of cancel-on-destruct just crossed my mind.  I haven't yet
> decided whether I like that idea or hate it.  I'm currently leaning
> towards the latter... :-)

I don't think it solves the problem.  Under Posix, at least,
cancel has no effect on joinability---either a join or a detach
is still necessary.  And of course, cancel is only advisory
anyway; the target thread isn't actually required to respect it.

> > > Sounds like a good use case for a thread pool, or thread group.

> > Possibly.  I've generally used thread groups, when I've used
> > them, at a lower level of granularity.  Most of the thread
> > classes I've used (and designed) have also contained a static
> > registry with *all* threads, and a static function shutdown,
> > which signaled all running threads, and waited for them to
> > finish.  I'm not at all sure that my solutions, however, would
> > be appropriate in the standard; they almost always involved some
> > very application specific aspects.  (And they've all counted on
> > collaborating threads, which explicitly checked from time to
> > time whether shutdown had been requested or not.)

> Agreed.  I would like to make it easy for user's to create such
> registries if they want to (multiple registries even).  But I'm loath to
> mandate them.

Understood.  (In general, C++ has never mandated what I would
consider good programming.  Unlike the other languages I know,
however, it's never actively prevented it.  Which is, at least,
something.)

> > > Are you saying we should outlaw the following behavior?

> > > #include <thread>
> > > #include <cstdio>
> > > #include <cstdlib>

> > > bool stop_now = false;
> > > std::mutex mut;

> > > extern "C" void bye()
> > > {
> > >     std::exclusive_lock<std::mutex> lk(mut);
> > >     std::printf("stopping\n");
> > >     stop_now = true;
> > > }

> > > void f()
> > > {
> > >     while (true)
> > >     {
> > >         std::exclusive_lock<std::mutex> lk(mut);
> > >         if (stop_now)
> > >             return;
> > >         std::printf("f running\n");
> > >     }
> > > }

> > > int main()
> > > {
> > >     (std::thread(f));  // detaches f on destruct
> > >     std::atexit(bye);
> > > }

> > > ..
> > > f running
> > > f running
> > > f running
> > > stopping

> > > The program above detaches a thread, lets it continue running
> > > after main returns, but arranges for it to cleanly shut down
> > > via other communication sometime during the atexit chain.

> > Definitly.  If the thread uses any global resources, this is
> > irremediably broken.

> Not if the global resource is not yet destructed.  The current atexit /
> destructor sequence is well defined.  Today globals can safely be
> manipulated at atexit time if the global construction is properly
> coordinated with the atexit registration.

FWIW: I've looked at the implementation of boost::threads, and
the thread kick-off routine (which calls f()) uses a
boost::function, which, from what I understand, uses global
resources (operator's new and delete).  So you have to ensure,
somehow, that the thread actually returns from the boost
kick-off routine before you finish running the last destructor
of a static object (after which, the system will start shutting
down the global resources that it manages, like dynamic memory).

I don't know how to do that other than with join.

> > > Valid program or not?

> > I wouldn't like to have to formulate the guarantees necessary
> > for it to be valid.

> I think that work has already been done:

> 3.6.3p3:  Interleave static destructors with atexit registered functions.

I don't think so.  The describes the requirements on sequencing
a single thread.  We have a problem with multiple threads in
that destructors of auto objects can be called *after* all of
the system shutdown code has run, and those destructors can
require the presence of resources (e.g. dynamic memory
management) which have already been shut down.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Gianni Mariani wrote:
> James Kanze wrote:

    [...]
> >> I agree
> >> that there should be a mandated way to cause a thread wrap up quickly
> >> and "return" which I also don't understand why that mechanism is not
> >> something that has a standard interface that is simply overridden by the
> >> client.

> >> I see the whole pthread_cancel mechanism to be far more complex than it
> >> needs to be, especially for C++ code which already has a well defined
> >> cleanup mechanism.

> > The pthread_cancel mechanism has the advantage of being
> > implemented in kernel code, and thus being able to interrupt
> > blocking system requests.

> I assume that an application calling pthread_testcancel (or the
> equivalent in the proposed standard) will have the stack rolled back
> similar to when an exception is thrown ?

I would assume that IF some sort of cancel is adopted, it would
have semantics similar to those of an exception.  Or hope, at
least---at present, calling std::exit() (which is effectively a
"cancel" of the sole thread in a single thread environment)
doesn't.

Current compilers do NOT behave this way.  Sun CC will unwind
the stack and call destructors, but ignores catch blocks (don't
ask me why).  G++ basically does the equivalent of a longjmp,
and doesn't even call destructors.  The result is that
pthread_cancel cannot really be used in a C++ program, and you
end up having to put time-outs on all your system calls in order
to poll a global variable.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
.
>
> You're right.  I had a bug.  Here I've attempted to correct it.
>
> Well defined?
>
. code snipped

Nice try, no, not well defined.

The constraint you're placing on cv.notify_one() is that the notifying
thread cannot touch the mutex object once the mutex is released.  Also,
assuming we really want this nonsense of spurious wakeup from cv, then
your cv and mutex might get nuked before the "f" thread even calls notify.

One remedy is to leak the mutex (never destroy it), or reference count
it and make sure that whatever thread does post the notify is never to
touch global variables (apart from ones on it's own stack) then sure.

The question is, why put yourself through that pain ?  It's error prone
and for most people with mediocre MT experience, plainly it's a trap. I
assume you have plenty of experience with threads and this still nabs
you.  Imagine the thousands of less experienced or less smart
programmers cursing at the thread committee for letting them get
themselves in such a mess.

We're talking A NEW part of the standard, fresh, with no blunders in it
yet.  We're talking few programmers who can really understand the
complexity of threads.  I think this situation demands a standard that
has utility yet limits the scope of stupid errors.  Detached threads are
high on my biggest source of errors from experience.  Even seemingly
common libraries from big reputable companies get this one wrong.  The
advantage of having detachable threads does not justify the
"maintenance" cost.

>
> Above bye() is guaranteed to start *and* finish executing before mut and
> cv destruct.  The "f" thread is detached, and continues running after
> main exits.  The "f" thread is guaranteed to shut down after bye()
> starts executing and before bye() finishes executing (or at least
> guaranteed not to touch any global state after bye() returns).
>
> Do we have sufficient motivation to call this program ill-formed just
> because it lets a detached thread run for awhile after main returns?

It depends on what constraints you can put on the compiler vendors.  I
don't the the constraint justifies the gains.

>
> I don't think we do.
>
> I think the most we can say is similar to what we already say in 3.6.3p2:

In general I'd agree with you, in this specific case I don't think it
makes sense.

>
>> If a function contains a local object of static storage duration that has
>> been destroyed and the function is called during the destruction of an object
>> with static storage duration, the program has undefined behavior if the flow
>> of control passes through the definition of the previously destroyed local
>> object.
>
> (needs to be expanded to all objects with static storage duration, not
> just function-locals)
>
> Oh, wait a sec, I think namespace scope objects are covered in 3.8p5.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
> In article <1163342424.043179.87490@f16g2000cwb.googlegroups.com>,
>  "James Kanze" <james.kanze@gmail.com> wrote:

> > > That's not the way I read C++03, 3.6.3p3:

> > Agreed.  But you'll also agree that this only talks of single
> > threaded behavior.

> Sure.  But imho 3.6.3p3 shouldn't say one thing for single
> threaded and something completely different for multithreaded.
> Rather the model for multithreading should be a natural
> extension of what it says for single threaded.

I don't think a "natural extension" works.  That's why Hans is
working so hard to give us a working memory model.

But that's not the issue here.     3.6.3/3 describes a sequence of
actions which occur at a specific time during the execution of a
program.  If other threads can still be running, then we have
additional questions concerning the synchronization with those
other threads.  Obviously, this issue isn't considered in the
current wording.

I'm not really saying that defining such things would be
impossible.  I am saying that it would be incredibly difficult,
and would lead us into areas which are largely unknown, and
where we'd have to define things without really knowing what all
of the implications are.  IMHO, the safest solution (and the
only one which will allow the next version of the standard in a
reasonable timespan) is to declare it undefined behavior.

Note that I'm talking here from a purely standards point of
view.  If I accidentally did leave a thread running, and that
thread did nothing, and the system actually did reformat my hard
disk, I'd be very pissed off.  But I think that at present,
where we are today, this is best left as a quality of
implementation issue.  Because obviously, the threads which are
still running can't do everything that they normally would, and
trying to specify what they can and cannot do is a hornets nest,
better left undisturbed until we have more experience with the
issues.

> > Suppose you have several
> > threads running.  You execute bye().  No problem.  You continue
> > execution in the main thread, executing further destructors.
> > For example, the destructor for the mutex.  What happens to the
> > thread waiting on the mutex?

> > The order of functions registered with atexit is defined.  The
> > order things happen in a thread relative to them isn't.
> > Presumably, the main thread could actually finish executing all
> > of the destructors, and start cleaning up global memory before
> > one of the waiting threads woke up.

> You're right.  I had a bug.  Here I've attempted to correct it.

> Well defined?

That's precisely the issue.  At present, I really don't have any
concrete critera by which to judge.  By chance, you're initial
code contained an "obvious" error.  But what is really
guaranteed?  What can I count on, and what not?

> #include <thread>
> #include <cstdio>
> #include <cstdlib>

> bool stop_now = false;
> std::mutex mut;
> std::condition<std::exclusive_lock<std::mutex> > cv;

> extern "C" void bye()
> {
>     std::exclusive_lock<std::mutex> lk(mut);
>     std::printf("stopping f\n");
>     stop_now = true;
>     while (stop_now)
>         cv.wait(lk);
>     std::printf("f stopped\n");
> }

> void f()
> {
>     while (true)
>     {
>         std::exclusive_lock<std::mutex> lk(mut);
>         if (stop_now)
>         {
>             stop_now = false;
>             cv.notify_one();
>             return;
>         }
>         std::printf("f running\n");
>     }
> }

> int main()
> {
>     (std::thread(f));
>     std::atexit(bye);
> }

> Above bye() is guaranteed to start *and* finish executing
> before mut and cv destruct.  The "f" thread is detached, and
> continues running after main exits.  The "f" thread is
> guaranteed to shut down after bye() starts executing and
> before bye() finishes executing (or at least guaranteed not to
> touch any global state after bye() returns).

I don't think so.  Once f() executes cv.notify_one(), bye() can
continue executing, and return.  And the return from
cv.notify_one() uses global state (e.g. the mutex is reaquired).

Dynamically allocating the mutex and the destructor (without any
smart pointers!) so that they will never be destructed will
improve things a lot, but can you guarantee that the stack for
f() will not somehow have been freed once you return from bye()
(but before you return from f())?  I can't (although I know that
in practice, that isn't a problem under Posix, unless I make it
one).

The problem is that no matter what you do, unless you've
actually joined, or the thread has really, completely
terminated, you have to define exactly what it can do.  I'd be
all in favor of it, but it's a lot of definition, and we've
really relatively little experience with the issues.  So what
I'm arguing for, really, is that we punt.  If it turns out later
that we can do better, and that it is useful, there's no problem
defining what was previously undefined behavior.  If we actually
do define something, and find out later that it is the wrong
thing, changing defined behavior is another matter.

In short, don't overspecify until we really know what should be
specified.

> Do we have sufficient motivation to call this program
> ill-formed just because it lets a detached thread run for
> awhile after main returns?

> I don't think we do.

I'm afraid you do.  I think you would agree with me that it
can't do just anything---once we've started running destructors
on static objects, it can't use those objects, obviously.  More
importantly, once we've finished running those destructors, it's
hard to say what it can legally do.

I would like it if it could at least return, up to the point
where there is no more thread (much like I can return from a
member function after having done "delete this"), but I'm not
sure that we can guarantee even this.

> I think the most we can say is similar to what we already say
> in 3.6.3p2:

> > If a function contains a local object of static storage
> > duration that has been destroyed and the function is called
> > during the destruction of an object with static storage
> > duration, the program has undefined behavior if the flow of
> > control passes through the definition of the previously
> > destroyed local object.

> (needs to be expanded to all objects with static storage
> duration, not just function-locals)

> Oh, wait a sec, I think namespace scope objects are covered in
> 3.8p5.

Maybe.  Of course, part of the problem is that the system may
treat the thread's stack exactly like any other static object.
And there is no way to ensure that the thread will not run when
it's too late.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

In article <1163502349.257866.52580@f16g2000cwb.googlegroups.com>,
 "James Kanze" <james.kanze@gmail.com> wrote:

> > #include <thread>
> > #include <cstdio>
> > #include <cstdlib>
>
> > bool stop_now = false;
> > std::mutex mut;
> > std::condition<std::exclusive_lock<std::mutex> > cv;
>
> > extern "C" void bye()
> > {
> >     std::exclusive_lock<std::mutex> lk(mut);
> >     std::printf("stopping f\n");
> >     stop_now = true;
> >     while (stop_now)
> >         cv.wait(lk);
> >     std::printf("f stopped\n");
> > }
>
> > void f()
> > {
> >     while (true)
> >     {
> >         std::exclusive_lock<std::mutex> lk(mut);
> >         if (stop_now)
> >         {
> >             stop_now = false;
> >             cv.notify_one();
> >             return;
> >         }
> >         std::printf("f running\n");
> >     }
> > }
>
> > int main()
> > {
> >     (std::thread(f));
> >     std::atexit(bye);
> > }
>
> > Above bye() is guaranteed to start *and* finish executing
> > before mut and cv destruct.  The "f" thread is detached, and
> > continues running after main exits.  The "f" thread is
> > guaranteed to shut down after bye() starts executing and
> > before bye() finishes executing (or at least guaranteed not to
> > touch any global state after bye() returns).
>
> I don't think so.  Once f() executes cv.notify_one(), bye() can
> continue executing, and return.  And the return from
> cv.notify_one() uses global state (e.g. the mutex is reaquired).

In article
<4559a00d$0$28221$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
 gi3nospam@mariani.ws (Gianni Mariani) wrote:

> Nice try, no, not well defined.
>
> The constraint you're placing on cv.notify_one() is that the notifying
> thread cannot touch the mutex object once the mutex is released.  Also,
> assuming we really want this nonsense of spurious wakeup from cv, then
> your cv and mutex might get nuked before the "f" thread even calls notify.

During the call to cv.notify_one(), f() never relinquishes the lock on
mut.  If bye() gets a spurious (or even legitimate) wake up while f() is
executing, bye() can still not return from cv.wait() until it reacquires
mut.

f() maintains ownership of mut until it returns.  Destructing lk, and
thus unlocking mut is the last thing f() does.  Afterwards it touches no
global state.  bye() can not continue until this point.

-Howard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

James Kanze wrote:
.
> Current compilers do NOT behave this way.  Sun CC will unwind
> the stack and call destructors, but ignores catch blocks (don't
> ask me why).  G++ basically does the equivalent of a longjmp,
> and doesn't even call destructors.  The result is that
> pthread_cancel cannot really be used in a C++ program, and you
> end up having to put time-outs on all your system calls in order
> to poll a global variable.
>

Just at a glance, it would seem that a "std::thread_cancel" exception
caught in the topmost stack frame would be a simple C++ clean way of
doing this.  The only question is how to hook into the pthread cancel
mechanism which at a glance seems seriously nontivial if the application
is to call a C library which uses
pthread_cleanup_push/pthread_cleanup_pop as the C mechanism for cleanup.

This raises the interesting question.  Should a C++ standard thread
cancel mechanism mandate that the pthread_cleanup_push/pop also be
managed.  I think it does.  If this does, it gives a well defines stack
rollback procedure for mixed C/C++ stack frames.  Quite frankly, this
has always been a problem in real life applications and one of the
reasons exceptions are shunned. But then, does it mean that all the
pthread_cleanup_push/pop frames should also be invoked on a regular
exception stack unwind ?

Seems like this is the hard question.

A quick google code search revealed very little code that uses
pthread_cleanup_push/pop in applications so it's probably safe to tweak
the semantics.

I don't know how this would apply to other platforms (win32).

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
> In article <1163502349.257866.52580@f16g2000cwb.googlegroups.com>,
>  "James Kanze" <james.kanze@gmail.com> wrote:
.
>> Nice try, no, not well defined.
>>
>> The constraint you're placing on cv.notify_one() is that the notifying
>> thread cannot touch the mutex object once the mutex is released.  Also,
>> assuming we really want this nonsense of spurious wakeup from cv, then
>> your cv and mutex might get nuked before the "f" thread even calls notify.
>
> During the call to cv.notify_one(), f() never relinquishes the lock on
> mut.  If bye() gets a spurious (or even legitimate) wake up while f() is
> executing, bye() can still not return from cv.wait() until it reacquires
> mut.
>
> f() maintains ownership of mut until it returns.  Destructing lk, and
> thus unlocking mut is the last thing f() does.  Afterwards it touches no
> global state.  bye() can not continue until this point.

OK, you're right.

While in f() however, the lk mutex is unlocked and potentially
immediately destroyed at then whatever global handlers of the thread are
called, you may or may not be undefined.  Too hard.

The only guarenteed reliable way is to have an indication that the
thread is dead through a join before proceeding with destruction.

I'd be happy if the vendor provides guarantees above this but I
certainly don't want to see them mandated.


G

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Gianni Mariani wrote:
> James Kanze wrote:

> > Current compilers do NOT behave this way.  Sun CC will unwind
> > the stack and call destructors, but ignores catch blocks (don't
> > ask me why).  G++ basically does the equivalent of a longjmp,
> > and doesn't even call destructors.  The result is that
> > pthread_cancel cannot really be used in a C++ program, and you
> > end up having to put time-outs on all your system calls in order
> > to poll a global variable.

> Just at a glance, it would seem that a "std::thread_cancel" exception
> caught in the topmost stack frame would be a simple C++ clean way of
> doing this.  The only question is how to hook into the pthread cancel
> mechanism which at a glance seems seriously nontivial if the application
> is to call a C library which uses
> pthread_cleanup_push/pthread_cleanup_pop as the C mechanism for cleanup.

Exactly.  How to hook into the pthread cleanup and cancel
mechanism is seriously nontrival.  I'm sure that most of the
compiler developers know what is needed; the problem is that
they don't necessarily know how to best provide it.  Calling
destructors can probably be done by generating code similar to
what is in pthread_cleanup_push/pthread_cleanup_pop, but this is
likely to have non-negligeable runtime overhead (depending on
the implementation).  And of course, handling catch blocks is
far more difficult.  (On compromise would be to give the
exception an unnamable type, so it couldn't be explicitly
caught.  catch(...) could be handled as a destructor, and none
of the other catch blocks could be relevant.  You'd still need
some special handling for throw, when the exception is rethrown,
but it should be doable.)

> This raises the interesting question.  Should a C++ standard thread
> cancel mechanism mandate that the pthread_cleanup_push/pop also be
> managed.  I think it does.  If this does, it gives a well defines stack
> rollback procedure for mixed C/C++ stack frames.  Quite frankly, this
> has always been a problem in real life applications and one of the
> reasons exceptions are shunned. But then, does it mean that all the
> pthread_cleanup_push/pop frames should also be invoked on a regular
> exception stack unwind ?

> Seems like this is the hard question.

One of them, anyway:-).

> A quick google code search revealed very little code that uses
> pthread_cleanup_push/pop in applications so it's probably safe to tweak
> the semantics.

I don't know.  I've never seen a threaded application which
didn't use it.  Not necessarily very often, but usually, for
something very critical.

> I don't know how this would apply to other platforms (win32).

Me neither.  Does Win32 even have a thread cancellation mechanism?

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
> In article <1163502349.257866.52580@f16g2000cwb.googlegroups.com>,
>  "James Kanze" <james.kanze@gmail.com> wrote:

> > > #include <thread>
> > > #include <cstdio>
> > > #include <cstdlib>

> > > bool stop_now = false;
> > > std::mutex mut;
> > > std::condition<std::exclusive_lock<std::mutex> > cv;

> > > extern "C" void bye()
> > > {
> > >     std::exclusive_lock<std::mutex> lk(mut);
> > >     std::printf("stopping f\n");
> > >     stop_now = true;
> > >     while (stop_now)
> > >         cv.wait(lk);
> > >     std::printf("f stopped\n");
> > > }

> > > void f()
> > > {
> > >     while (true)
> > >     {
> > >         std::exclusive_lock<std::mutex> lk(mut);
> > >         if (stop_now)
> > >         {
> > >             stop_now = false;
> > >             cv.notify_one();
> > >             return;
> > >         }
> > >         std::printf("f running\n");
> > >     }
> > > }

> > > int main()
> > > {
> > >     (std::thread(f));
> > >     std::atexit(bye);
> > > }

> > > Above bye() is guaranteed to start *and* finish executing
> > > before mut and cv destruct.  The "f" thread is detached, and
> > > continues running after main exits.  The "f" thread is
> > > guaranteed to shut down after bye() starts executing and
> > > before bye() finishes executing (or at least guaranteed not to
> > > touch any global state after bye() returns).

> > I don't think so.  Once f() executes cv.notify_one(), bye() can
> > continue executing, and return.  And the return from
> > cv.notify_one() uses global state (e.g. the mutex is reaquired).

> In article
> <4559a00d$0$28221$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
>  gi3nospam@mariani.ws (Gianni Mariani) wrote:

> > Nice try, no, not well defined.

> > The constraint you're placing on cv.notify_one() is that the
> > notifying thread cannot touch the mutex object once the
> > mutex is released.  Also, assuming we really want this
> > nonsense of spurious wakeup from cv, then your cv and mutex
> > might get nuked before the "f" thread even calls notify.

> During the call to cv.notify_one(), f() never relinquishes the
> lock on mut.  If bye() gets a spurious (or even legitimate)
> wake up while f() is executing, bye() can still not return
> from cv.wait() until it reacquires mut.

> f() maintains ownership of mut until it returns.  Destructing
> lk, and thus unlocking mut is the last thing f() does.
> Afterwards it touches no global state.  bye() can not continue
> until this point.

Yes.  After posting, I realized that I was thinking of a problem
I had had with an earlier (pre-posix) system, where notify
automatically released the lock.  (In fact, it didn't use a
separate lock.)

You're still not really safe, of course, unless you can ensure
that the thread makes no use of any global resources after
freeing the lock.  Global resources like, for example, the
memory manager.  From what I understand, the current
implementation of boost::thread actually does make use of
dynamic memory in its thread kick-off routine (indirectly, via
the function object).  You'd have to ban this is you wanted to
(formally) allow threads running after calling exit.

Note that I'm not saying that detached threads don't have any
utility.  In my servers, I use them all the time---practically
speaking, there is no other solution.  I do so knowing that I'm
flirting with undefined behavior, however, counting on:

 -- the probability of a real problem is fairly low,

 -- the actual behavior is limited---the undefined behavior will
    not reformat my hard disk or anything like that, and

 -- everything that the application had to do has been done, all
    client and data base connections are closed, all files have
    been sync'ed, etc.

In fact, I'm confident that the worse thing that can really
happen is an extra core dump somewhere, and possibly (although
not in my current application) some unfreed system resources,
like a non-deleted temporary file.  I don't reach the point of
possible undefined behavior until all client connections are
shut down, all persistent data is safely written to disk, etc.
So it's a risk I can live with, and costs less that the cost of
trying to join all threads (which would be doable, but would
require some extra gymnastics).

I suspect that this is the case for most uses of detached
threads.  And I'm pretty sure that trying to rigorously define
exactly what you can legally do in such cases is an enormous
hornets nest, that we don't want to stir up.  In practice, we
can live with the undefined behavior, and there are a lot more
important things that need doing first.  It's not as if the
committee had time on its hands, with nothing else to do, so it
could spend time on what is fundamentally a cosmetic problem,
and which will in practice not cause problems for anyone.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:
> In article <1163341583.146827.43850@f16g2000cwb.googlegroups.com>,
>  "James Kanze" <james.kanze@gmail.com> wrote:

> > Note that it's not really a killer critera, since I can always
> > write my own wrapper which does whatever is necessary (except
> > that I don't know how to detach without calling the destructor
> > of the thread); that's what I do with boost::threads currently.

> Assuming std::thread looked something like boost::thread, I
> think adding a detach() member to it would be a fine idea.
> I'm a little disconcerted that this would leave the client
> with a handle to a thread which can no longer reliably do
> things like fetch thread identity.

I'm not sure what you mean here.  Thread identity should be
independant of joinability---at least if we also support cancel,
there are other things which can be done with a thread as well.

The more I think about it, the more I think that the simplest
solution is two different types: joinable_thread and
detached_thread.  I'm not sure how universal my experience is,
but I've never created a thread without knowing in advance
whether I wanted to join with it or detach it.  This still
leaves open the question about what to do in the destructor of a
joinable_thread which hasn't been joined, but it does solve the
problem about accidentally joining a detached thread, or
whatever.  (This is, in fact, what I currently do: I derive two
classes from boost threads, one for joinable threads, and the
other for detached threads, and handle the appropriate case in
the destructor of the derived class.)

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

James Kanze wrote:
> Howard Hinnant wrote:
>> In article <1163341583.146827.43850@f16g2000cwb.googlegroups.com>,
>>  "James Kanze" <james.kanze@gmail.com> wrote:
>
>>> Note that it's not really a killer critera, since I can always
>>> write my own wrapper which does whatever is necessary (except
>>> that I don't know how to detach without calling the destructor
>>> of the thread); that's what I do with boost::threads currently.
>
>> Assuming std::thread looked something like boost::thread, I
>> think adding a detach() member to it would be a fine idea.
>> I'm a little disconcerted that this would leave the client
>> with a handle to a thread which can no longer reliably do
>> things like fetch thread identity.
>
> I'm not sure what you mean here.  Thread identity should be
> independant of joinability---at least if we also support cancel,
> there are other things which can be done with a thread as well.
>
> The more I think about it, the more I think that the simplest
> solution is two different types: joinable_thread and
> detached_thread.  I'm not sure how universal my experience is,
> but I've never created a thread without knowing in advance
> whether I wanted to join with it or detach it.  This still
> leaves open the question about what to do in the destructor of a
> joinable_thread which hasn't been joined, but it does solve the
> problem about accidentally joining a detached thread, or
> whatever.  (This is, in fact, what I currently do: I derive two
> classes from boost threads, one for joinable threads, and the
> other for detached threads, and handle the appropriate case in
> the destructor of the derived class.)

I vote for an overridable but default "cancel" on destruct and join and
leave the exception handling to the application.

In fact, why not make the delivered exception a "message" to a message
handler ?  That way you remove the exception on join issue since it is
unlikely that the joining thread is always the thread that can handle
the exception anyway and mandating that it like that is a bad idea.

Also, make it so that the thread may or may not accept exceptions, i.e.
the exception "transfer" functionality is not mandated but available as
an option so that anyone who would rather handle all exceptions gets a
predictable std::terminate if they didn't handle the exception rather
than having the stack rolled back on them.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Gianni Mariani wrote:
> James Kanze wrote:
> > Howard Hinnant wrote:
> >> In article <1163341583.146827.43850@f16g2000cwb.googlegroups.com>,
> >>  "James Kanze" <james.kanze@gmail.com> wrote:

> >>> Note that it's not really a killer critera, since I can always
> >>> write my own wrapper which does whatever is necessary (except
> >>> that I don't know how to detach without calling the destructor
> >>> of the thread); that's what I do with boost::threads currently.

> >> Assuming std::thread looked something like boost::thread, I
> >> think adding a detach() member to it would be a fine idea.
> >> I'm a little disconcerted that this would leave the client
> >> with a handle to a thread which can no longer reliably do
> >> things like fetch thread identity.

> > I'm not sure what you mean here.  Thread identity should be
> > independant of joinability---at least if we also support cancel,
> > there are other things which can be done with a thread as well.

> > The more I think about it, the more I think that the simplest
> > solution is two different types: joinable_thread and
> > detached_thread.  I'm not sure how universal my experience is,
> > but I've never created a thread without knowing in advance
> > whether I wanted to join with it or detach it.  This still
> > leaves open the question about what to do in the destructor of a
> > joinable_thread which hasn't been joined, but it does solve the
> > problem about accidentally joining a detached thread, or
> > whatever.  (This is, in fact, what I currently do: I derive two
> > classes from boost threads, one for joinable threads, and the
> > other for detached threads, and handle the appropriate case in
> > the destructor of the derived class.)

> I vote for an overridable but default "cancel" on destruct and join and
> leave the exception handling to the application.

Cancel is advisory under Unix, and from what I hear, doesn't
even exist under Windows.  So using cancel under Unix can still
result in an endless hang in a destructor, and your suggestion
might not even be implementable under Windows.

There are probably cases where you can safely do a join in the
destructor, and the user should probably be allowed to authorize
it, but I don't like it as a general policy.

> In fact, why not make the delivered exception a "message" to a message
> handler ?  That way you remove the exception on join issue since it is
> unlikely that the joining thread is always the thread that can handle
> the exception anyway and mandating that it like that is a bad idea.

I don't quite get what you are trying to say.  The problem here
isn't an exception on join; it's calling join as the result of
an exception (which invokes the destructors).  You surely aren't
suggesting that exceptions shouldn't do stack walkback.

--
James Kanze (Gabi Software)            email: james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

I just became aware of this article by Eric Niebler

http://www.artima.com/cppsource/threads_meeting.html

I just want to add one thought on the "upgradable_mutex".  I remember
implementing one of these over 10 years ago and quickly reverted it
because it's next to useless in practice.  I don't know the exact
semantics of the one proposed by Lawrence Crowl, but I found that the
semantics I had would almost always cause a deadlock.  The thing is,
unless you give up your read lock and then request a write lock, you're
almost always going to cause a deadlock.

Simple sequence.

TN - Thread N

T1 - Readlock A
T2 - Readlock B
T1 - Upgrade A to Writelock (must wait until T2 releases Readlock)
T2 - Upgrade A to Writelock (must waut until T1 releases Readlock)

oops - neither T1 or T2 will release readlock A!

So, if the semantics of the upgrade is to release the readlock first I
would argue that it is an "unexpected" thing for most programmers that a
conversion causes a prior release of the lock.  In fact, simply
requiring the the read lock is released before quiring the write locks
is a less complex, easier to understand interface.

Also, can anyone point me to the CoreX api ?  I have yet to see it.

The yet to be released version (early draft is available here:
http://netcabletv.org/public_releases/) contains a number of new
features. (warning - it's big - 100meg - contains some prebuilt binaries)

As some of you may know I have been developing the Austria C++ api which
also has a threading interface.  It's a very simple API.  There is also
a "thread pool" type API called ActivityList that allows a hybrid
callback/event model which gives you all the benefits of event
processing with a callback style interface.

There is also a thread safe smart pointer.  i.e. two threads can
read/write to the same smart pointer (i.e. atomic smart pointer thing)
which is a specialization of the regular austria smart pointer.

I also was under the impression that the terminology for a
"reader/writer" mutex was an "access" mutex.  "Access" in the sense that
it controls the type of access.

One of the other things that I don't see handled very well in real-life
multithreaded code is the notion of thread life (joining) and object
life.  At least in Austria C++, a thread is an object and the lifetime
of the thread does not exceed the lifetime of the object (i.e. deleting
the object WILL incur a join).  There are too many API's I have come
across where it is impossible to synchronize object deletion and thread
join reliably - (win32's wininet is one).

Perhaps the only niggling repetitious error in clients using the Austria
API is that it requires the client developer to place waits (joins) in
the most derived destructor.

The other interesting feature in the Austria API is the "TwinMT" system.
  This allows a class designer to define a reliable "coupling" system.
For example, the Austria C++ Timer API uses this.  A client can simply
delete itself and the deregistration of the timer happens automatically
(No fuss).

The concept of object coupling is a very powerful one and makes writing
client code much easier to write.

Now, I don't think I really care which API ends up in the standard as
long as it is not intellectutual property encumbered in any way and
provides all the primitives so I can remove all the cross platform hacks
from my Austria C++ code.

The Austria C++ low level API's from the latest rev) are:

a) Atomic (integer and pointer types)
 increment, decrement, add, exchange, compare/exchange
b) Mutex, Condition Variable (mutex/condition), Exclusion lock

c) Thread, Thread::wait (join), ThreadID, Thread exit status

d) scheduler control (yield)

The rest of the Austria C++ Threading API is built upon these and include:

Lock (RAII mutex aquire release)
Unlock (like a Lock but does a release and then an aquire on destruct)
Swaplock ( "swaps" locks )
Trylock (Like an RAII lock but does not block)
smart pointer traits to make atomic smart pointers
thread pools + single threaded pools

While I am at it, I should describe a really really interesting use of
Austria thread pools.  The austira C++ thread pools are lists of
"activities" with a special "consumer".  The run-o-the-mill consumer
type is a "thread pool" where you get to nominate the maximum number
of threads that are used to consume enqueued activities.  To make the
API really simple, the activity list is nominated at the time of
construction of an activity so an activity simply needs to be
"enqueued".  Trying to enqueue and already enqueued activity is
basically a nop which means that an activity can only be called by one
thread at any time.  Replace the regular thread pool consumer with an
activity and now you have an activity list which will execute activities
one at a time.  That means that I can create an object with N activities
and be guarenteed that only one will execute which results low cost
inbuilt synchronization.  It turns out the paradigm produces a very
powerful paradigm to build MT systems.  It's kind of the COM single
threaded apartment model.

Anyhow, I have enjoyed writing MT code using Austria C++.  If you stick
to the API, it avoids many of the classic MT pitfalls and it's quite robust.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

In article
<45530096$0$3036$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
 gi3nospam@mariani.ws (Gianni Mariani) wrote:

> I just became aware of this article by Eric Niebler
>
> http://www.artima.com/cppsource/threads_meeting.html
>
> I just want to add one thought on the "upgradable_mutex".  I remember
> implementing one of these over 10 years ago and quickly reverted it
> because it's next to useless in practice.  I don't know the exact
> semantics of the one proposed by Lawrence Crowl,

Fwiw, you can blame the upgradable_mutex proposal on me:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2094.html#mutexs

> but I found that the
> semantics I had would almost always cause a deadlock.  The thing is,
> unless you give up your read lock and then request a write lock, you're
> almost always going to cause a deadlock.
>
> Simple sequence.
>
> TN - Thread N
>
> T1 - Readlock A
> T2 - Readlock B
> T1 - Upgrade A to Writelock (must wait until T2 releases Readlock)
> T2 - Upgrade A to Writelock (must waut until T1 releases Readlock)
>
> oops - neither T1 or T2 will release readlock A!

The proposed upgradable_mutex does not have this problem.  Indeed, this
problem is the motivating use case for upgradable_mutex.  The
upgradable_mutex as proposed has unique ownership semantics with respect
to other upgradable_mutexes.  It can only share with other
sharable_mutexes.

> So, if the semantics of the upgrade is to release the readlock first I
> would argue that it is an "unexpected" thing for most programmers that a
> conversion causes a prior release of the lock.  In fact, simply
> requiring the the read lock is released before quiring the write locks
> is a less complex, easier to understand interface.

The proposal also allows for that use case.

> One of the other things that I don't see handled very well in real-life
> multithreaded code is the notion of thread life (joining) and object
> life.  At least in Austria C++, a thread is an object and the lifetime
> of the thread does not exceed the lifetime of the object (i.e. deleting
> the object WILL incur a join).  There are too many API's I have come
> across where it is impossible to synchronize object deletion and thread
> join reliably - (win32's wininet is one).

In all of the proposals I've seen to date, this is easily accomplished
by creating a thin wrapper, say thread_barrier, and putting the join
operation in ~thread_barrier().

> Perhaps the only niggling repetitious error in clients using the Austria
> API is that it requires the client developer to place waits (joins) in
> the most derived destructor.

I've seen lots of use cases for either desire:  join on destruct, detach
on destruct.  It is nice to easily allow either behavior.

If you have detach on destruct, it is easy and efficient to turn it into
join on destruct with a thin wrapper.  If you have join on destruct and
you want to turn it into detach on destruct, you have to put the handle
on the heap.  The latter is much more costly and inconvenient.

> The Austria C++ low level API's from the latest rev) are:
>
> a) Atomic (integer and pointer types)
>  increment, decrement, add, exchange, compare/exchange
> b) Mutex, Condition Variable (mutex/condition), Exclusion lock
>
> c) Thread, Thread::wait (join), ThreadID, Thread exit status
>
> d) scheduler control (yield)

<nod> Sounds good.

> The rest of the Austria C++ Threading API is built upon these and include:
>
> Lock (RAII mutex aquire release)

Yup, gotta have it.

> Unlock (like a Lock but does a release and then an aquire on destruct)

This hasn't been proposed, but I've privately toyed with a generic mutex
adaptor class that does this.

> Swaplock ( "swaps" locks )
> Trylock (Like an RAII lock but does not block)

N2094 integrates these with Lock.

> smart pointer traits to make atomic smart pointers
> thread pools + single threaded pools
>
> While I am at it, I should describe a really really interesting use of
> Austria thread pools.  The austira C++ thread pools are lists of
> "activities" with a special "consumer".  The run-o-the-mill consumer
> type is a "thread pool" where you get to nominate the maximum number
> of threads that are used to consume enqueued activities.  To make the
> API really simple, the activity list is nominated at the time of
> construction of an activity so an activity simply needs to be
> "enqueued".  Trying to enqueue and already enqueued activity is
> basically a nop which means that an activity can only be called by one
> thread at any time.  Replace the regular thread pool consumer with an
> activity and now you have an activity list which will execute activities
> one at a time.  That means that I can create an object with N activities
> and be guarenteed that only one will execute which results low cost
> inbuilt synchronization.  It turns out the paradigm produces a very
> powerful paradigm to build MT systems.  It's kind of the COM single
> threaded apartment model.
>
> Anyhow, I have enjoyed writing MT code using Austria C++.  If you stick
> to the API, it avoids many of the classic MT pitfalls and it's quite robust.

Thanks for the feedback.

-Howard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

Howard Hinnant wrote:

> I've seen lots of use cases for either desire:  join on
> destruct, detach on destruct.  It is nice to easily allow
> either behavior.

Or neither: I don't like anything that essential being delegated
to a destructor (which can be called as a result of an uncaught
exception).  In general, I would consider calling the destructor
on a joinable (but unjoined) thread should be illegal, and both
joining and detaching should be explicit behavior on the part of
the user, and not because someone forgot to catch an exception.
The issue concerning joinable depends somewhat on how and if
futures are supported, but I don't think a destructor should
ever perform an action that can wait for a potentially infinite
amount of time.

Note that it is arguable that threads should never be detached,
since it is necessary to wait until all threads stop in order to
do a clean shutdown.  This is probably an overly extreme
position; there are often threads that don't necessarily need a
clean shutdown, and of course, globally, the "application" (i.e.
set of processes on one or more machines) can't count on a clean
shutdown, since the process could also crash for other reasons.
But I think it safe to say that in many cases where the
application is using detached threads, it really should be
transfering the threads to some sort of reaper object, which on
shutdown will signal each thread that it should terminate, and
then join on each of the threads, possibly with a time out, just
in case.

Note that under Posix, pthreads_cancel handles the first part
very nicely, requesting that the thread terminate, and
interrupting any blocking system request in process.  Posix
doesn't support a time-out on pthread_join, however, and of
course, there's absolutely no guarantee that the other thread
will respect the pthread_cancel request.  Globally, the only
solution I've found has been to associate a termination
condition with the thread (in the thread object), use
pthread_cond_timedwait, and signal the condition at the end of
the thread start-off routine (generally in a pthread_cleanup
routine, although RAII can be used with Sun CC under Solaris).

The problem, of course, is that calling exit (either directly,
or by returning from main) will generally result in undefined
behavior if any other threads are still running.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S   mard, 78210 St.-Cyr-l'   cole, France, +33 (0)1 30 23 00 34


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]