On 8/22/07 8:41 AM, in article
1187789200.788467.181630@50g2000hsm.googlegroups.com, "Richard Smith"
<richard@ex-parrot.com> wrote:
>
>   // Declared in "serialization.h"
>   namespace serialization {
>     template <class T>
>     void serialize( ostream& os, T const& obj );
>   }
>
>   // Declared in "dump.h"
>   namespace serialization {
>     template <class T> void dump() {
>       T obj = ... ;
>       serialize( std::cout, obj );
>     }
>   }
>
>   // Declared in "my_type.h"
>   class my_type {};
>   namespace serialization {
>     void serialize( ostream& os, my_type const& obj );
>   }
>

> If the three headers in the example were included in the order listed,
> the code will break when dump<my_type>() is instantiated because the
> my_type overload of serialize can only be found via ordinary lookup
> from the template instantiation context (#4 in the list above), and in
> a conforming compiler, that doesn't occur.  But reorder the headers so
> that "my_type.h" is included before "dump.h" and everything will work
> fine.  In general, it's not reasonable to argue that "dump.h" should
> include "my_type.h", and requiring a particular order of #includes is
> very fragile, especially if an incorrect order does not result in a
> compile-time error.

But it is reasonable to require that a function be declared before it is
called. So the only "serialize()" functions that can match the call in
dump() are the ones declared at the point that dump() is defined in the
translation unit. After all, the programmer who wrote dump() could not have
intended to call a serialize() routine that is not declared by the point of
the call to serialize(); since there would be no assurance in that case that
this yet undeclared serialize() routine would ever be declared in the
translation unit - and therefore no assurance that the intended serialize()
would be the one called in dump(). So the author of dump() must have have
made sure that the declaration for the serialize() intended to be invoked -
had already been included in the translation unit before dump() called it.

More importantly, this rule prevents some other serialize() routine whose
declaration is included only after dump()'s definition (and a routine that
the dump()'s author may know nothing about) is mistakenly considered for a
match for the serialize() function call. Indeed, without this rule, there
would be nothing that dump()'s author could do that would prevent the wrong
serialize() function from being invoked (except perhaps by choosing an
obscure function name - like serialize23() - that would be unlikely to be
used by anyone else).

A program can easily avoid these sorts of problems by adhering to a simple
convention: place all functions declarations before all function definitions
in a translation unit. In this way, every function defined in a translation
unit has a declaration that is visible to every other function defined
within the same translation unit. So in this case, one solution would be to
create an additional header file, say, "user_specializations.h" that would
forwardly-declare all of the program's overloaded specialize() routines -
and do so in one place:

    // user_specializations.h

    class my_type;

    namespace specialization {
        void serialize( ostream& os, const my_type& obj);
        ...
   }

So if "specialization.h" includes "user_specializations.h" then the relative
order in which "dump.h" and "my_type.h" are included by a source file - will
not matter a bit.

Greg


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

On 25 Aug, 17:44, gre...@pacbell.net (Greg Herlihy) wrote:
> On 8/22/07 8:41 AM, in article
> 1187789200.788467.181...@50g2000hsm.googlegroups.com, "Richard Smith"
>
>
>
> <rich...@ex-parrot.com> wrote:
>
> >   // Declared in "serialization.h"
> >   namespace serialization {
> >     template <class T>
> >     void serialize( ostream& os, T const& obj );
> >   }
>
> >   // Declared in "dump.h"
> >   namespace serialization {
> >     template <class T> void dump() {
> >       T obj = ... ;
> >       serialize( std::cout, obj );
> >     }
> >   }
>
> >   // Declared in "my_type.h"
> >   class my_type {};
> >   namespace serialization {
> >     void serialize( ostream& os, my_type const& obj );
> >   }
>
> > If the three headers in the example were included in the order listed,
> > the code will break when dump<my_type>() is instantiated because the
> > my_type overload of serialize can only be found via ordinary lookup
> > from the template instantiation context (#4 in the list above), and in
> > a conforming compiler, that doesn't occur.  But reorder the headers so
> > that "my_type.h" is included before "dump.h" and everything will work
> > fine.  In general, it's not reasonable to argue that "dump.h" should
> > include "my_type.h", and requiring a particular order of #includes is
> > very fragile, especially if an incorrect order does not result in a
> > compile-time error.
>
> But it is reasonable to require that a function be declared before it is
> called.  So the only "serialize()" functions that can match the call in
> dump() are the ones declared at the point that dump() is defined in the
> translation unit.

But this isn't always the case.  ADL from the context of the point of
instantiation will find functions that are declared after the function
call.  That's really my whole point: if you move  my_type in the code
above into the serialization namespace, then that becomes an
associated namespace and you get ADL into it.  That completely changes
things and the second function (the one declared after the function
call) is selected.

> After all, the programmer who wrote dump() could not have
> intended to call a serialize() routine that is not declared by the point of
> the call to serialize(); since there would be no assurance in that case that
> this yet undeclared serialize() routine would ever be declared in the
> translation unit - and therefore no assurance that the intended serialize()
> would be the one called in dump(). So the author of dump() must have have
> made sure that the declaration for the serialize() intended to be invoked -
> had already been included in the translation unit before dump() called it.

But this isn't standard practice either!  Consider the common idiom:

  template <class Iter>
  void my_iter_swap( Iter a, Iter b ) {
    using namespace std;
    swap( *a, *b );
  }

In this case, it's clear that the function's author is intending a
std::swap to be used as a fallback if there is no specific overload of
swap in Iter::value_type's namespace.  And in this case, it doesn't
matter if the overload of swap is only declared after the definition
of my_iter_swap.

> More importantly, this rule prevents some other serialize() routine whose
> declaration is included only after dump()'s definition (and a routine that
> the dump()'s author may know nothing about) is mistakenly considered for a
> match for the serialize() function call. Indeed, without this rule, there
> would be nothing that dump()'s author could do that would prevent the wrong
> serialize() function from being invoked (except perhaps by choosing an
> obscure function name - like serialize23() - that would be unlikely to be
> used by anyone else).

No, that problem already exists because of ADL.  (Though, to be
honest, the partial ordering rules mean that you'd be very unlucky to
find a function with a signature suitable to make it a better match
than the generic serialize function without it being an intended
replacement.  Even if you had two similar serialization libraries, I
think you'd still be unlucky to be bitten by it.)

If you want to get around it, you need to (namespace) qualify the call
to serialize and you'll restrict the overload set to those found in
the declaration context -- i.e. those that have been declared before
the function call.

The problem already exists for a second (more mundane) reason too: if
the only thing preventing the wrong overload being selected is that it
hasn't yet been declared, then you're relying on the dump function (to
stick with this example) being included before the unrelated serialize
function.  Surely that's very fragile?

> A program can easily avoid these sorts of problems by adhering to a simple
> convention: place all functions declarations before all function definitions
> in a translation unit. In this way, every function defined in a translation
> unit has a declaration that is visible to every other function defined
> within the same translation unit. So in this case, one solution would be to
> create an additional header file, say, "user_specializations.h" that would
> forwardly-declare all of the program's overloaded specialize() routines -
> and do so in one place:
>
>     // user_specializations.h
>
>     class my_type;
>
>     namespace specialization {
>         void serialize( ostream& os, const my_type& obj);
>         ...
>    }
>
> So if "specialization.h" includes "user_specializations.h" then the relative
> order in which "dump.h" and "my_type.h" are included by a source file - will
> not matter a bit.

That's a dreadful solution!  "serialization.h" is presumably a low-
level general-purpose library (I'm using the term loosely, not
necessarily implying it's a DSO / DLL); perhaps it's used in several
projects; perhaps it's from a third-party.   If it includes
"user_serializations.h", that becomes part of the low-level library
too.  Is that then to list all classes that might possibly get
serialized in any of the projects it's used in?

I guess you could insist on "user_serializations.h" being a high-level
component mandatorily supplied by the program.  But that has it's own
problems.  Should it really have to list all types that get serialized
anywhere?  And what about implementation-detail types that are
declared in an anoymous namespace in a source file?

--
Richard Smith

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]

GCC has recently (in version 4.1) fixed its implementation of two
phase lookup so that when looking up a unqualified dependent name, the
candidate function set is (correctly) composed of functions found by:

  1. Ordinary lookup in the template definition context;
  2. ADL in the template definition context;
  3. ADL in the template instantiation context.

In previous versions, it also (incorrectly) found names a fourth way:

  4.  Ordinary lookup in the template instantiation context.

Paragraph 14.6.4.2/1 makes it very clear that this was incorrect and
that GCC were right to fix this.

However, I've recently been porting a large code base to using a fixed
version of GCC and have been encountered a large number of subtle bugs
resulting from this.  A typical example is:

  // Declared in "serialization.h"
  namespace serialization {
    template <class T>
    void serialize( ostream& os, T const& obj );
  }

  // Declared in "dump.h"
  namespace serialization {
    template <class T> void dump() {
      T obj = ... ;
      serialize( std::cout, obj );
    }
  }

  // Declared in "my_type.h"
  class my_type {};
  namespace serialization {
    void serialize( ostream& os, my_type const& obj );
  }

Now I appreciate that this isn't necessarily an example of best
practice, which would probably either have my_type's serialize
function in my_type's namespace, or use specialization rather than
overloading.  However, this technique does seem to be quite common
and, in simple cases, works fine; but in slightly more complicated
examples (e.g. where serialize is called from a template as above),
however, it is very fragile.

If the three headers in the example were included in the order listed,
the code will break when dump<my_type>() is instantiated because the
my_type overload of serialize can only be found via ordinary lookup
from the template instantiation context (#4 in the list above), and in
a conforming compiler, that doesn't occur.  But reorder the headers so
that "my_type.h" is included before "dump.h" and everything will work
fine.  In general, it's not reasonable to argue that "dump.h" should
include "my_type.h", and requiring a particular order of #includes is
very fragile, especially if an incorrect order does not result in a
compile-time error.


I'm aware of lots of ways of fixing the code -- that's not why I'm
posting this -- and I accept that a decent unit test should catch such
problems.  However, I'm interested in why the standard doesn't specify
that 2nd phase lookup (i.e. lookup in the context of the point of the
instantiation) should include ordinary lookup.  My experiences of
using a compiler that does this extra 2nd phase ordinary lookup (e.g.
gcc 4.0.x) compared to a more conformant compiler that doesn't (e.g.
gcc 4.1.x) is that the extra ordinary lookup step makes the result of
unqualified dependent type lookup more intuitive and less fragile.
And I can't recall having been bitten by this extra lookup finding
something silly.

Anyway, I would be interested if anyone can shed any light on why the
standard is as it is in this regard.

Thanks
Richard Smith

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]