Pete Becker wrote:
>
> You're too deep in implementation details to see it.

I forgot to mention that this is a common ailment. <g> Everyone who
works with shared libraries delves into the details, and it's hard to
pull up from there for a high level view. Sort of like newcomers always
wanting to discuss the behavior of virtual functions in terms of what's
in the vtable. For standardization it's important to come up with a
workable abstraction, so in 1496 I tried pretty hard to avoid saying
anything about particular implementations except in the notes. Of
courese, that doesn't mean that implementation details are irrelevant --
whatever we say has to be implementable and reasonably efficient.

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

On Thu, 20 May 2004 02:55:16 +0000 (UTC), Pete Becker <petebecker@acm.org>
wrote:

>> You're too deep in implementation details to see it.
>
> I forgot to mention that this is a common ailment. <g> Everyone who
> works with shared libraries delves into the details, and it's hard to
> pull up from there for a high level view. Sort of like newcomers always
> wanting to discuss the behavior of virtual functions in terms of what's
> in the vtable. For standardization it's important to come up with a
> workable abstraction, so in 1496 I tried pretty hard to avoid saying
> anything about particular implementations except in the notes. Of
> courese, that doesn't mean that implementation details are irrelevant --
> whatever we say has to be implementable and reasonably efficient.

Aww, I'd just written an aggressive post about how I viewed the lack of
implementational depth when drafting the standard had led to my top five
"most annoying things about C++". I'm originally an assembler programmer -
I still look at C++ as pseudonyms for assembler instructions and I doubt
that will ever change. I keep away from groups like this because I just
end up annoying experts as poor Marshall Cline once found out, god bless
him!

Anyway, what I will say is that shared libraries is somewhat of a special
case for ISO C++ standardisation - on nearly 98% of non-embedded computers
in use today, only two systems are in use. Therefore any C++ amendments
must choose mostly one or the other or face the feature going the way of
the "export" keyword. Furthermore, we have an opportunity to not repeat
the mistakes fifteen years of dynamic library experience can teach us
rather than wimping out and declaring where systems differ as
"implementation defined". As I mentioned before, I take an autocratic view
that if the correct method of implementation is known, no tolerance for
difference past those of legacy interoperability should be permitted.

I'm going to bow out very shortly - since no one has seemed to find fault
with my proposed GCC visibility patch (and pending a copyright assignment,
it's been accepted for GCC v3.5 mainline) my primary task has finished.

Cheers,
Niall

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

On Thu, 20 May 2004 02:51:37 +0000 (UTC), Pete Becker <petebecker@acm.org>
wrote:

>> The virtual function table must be unaffected by sharedness except being
>> itself shared if its type is.
>
> Objects (including virtual function tables, if they exist at all) are
> never shared. Names of objects are shared.

You're probably talking from the source point of view whereas I'm talking
 from the linker's point of view. Again, I'm coming totally from an
implementational viewpoint but I think this is useful to you as
C++ compilers show a remarkable similarity of implementation.

> I don't know what it means for a type to be shared, except to say that
> since a type isn't an object, it can't be shared in the sense that we've
> just been talking about. Which means that I can't say whether sharing a
> type requires sharing its vtable.

If you mark any member function of a class as available outside its DLL,
the compiler must also mark the vtable of the whole class (including
pointers to non-shared virtual member functions) as available outside its
DLL. Vtables normally come with typeinfo too, so that'll also need to be
available outside its DLL.

Therefore there is no such thing as a shared class. There is a shared
function and there is also a shared member function. But no more. When
specifying "shared class Foo" one is no more declaring a shared class than
"shared { extern int foo(); }".

Cheers,
Niall

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Niall Douglas wrote:
>
> On Thu, 20 May 2004 02:51:37 +0000 (UTC), Pete Becker <petebecker@acm.org>
> wrote:
>
> >> The virtual function table must be unaffected by sharedness except being
> >> itself shared if its type is.
> >
> > Objects (including virtual function tables, if they exist at all) are
> > never shared. Names of objects are shared.
>
> You're probably talking from the source point of view whereas I'm talking
>  from the linker's point of view. Again, I'm coming totally from an
> implementational viewpoint but I think this is useful to you as
> C++ compilers show a remarkable similarity of implementation.

I'm talking about what the C++ standard should say, since that's what's
on topic here. Implementation details are unimportant, except when they
demonstrate that something can't be implemented efficiently.

>
> > I don't know what it means for a type to be shared, except to say that
> > since a type isn't an object, it can't be shared in the sense that we've
> > just been talking about. Which means that I can't say whether sharing a
> > type requires sharing its vtable.
>
> If you mark any member function of a class as available outside its DLL,
> the compiler must also mark the vtable of the whole class (including
> pointers to non-shared virtual member functions) as available outside its
> DLL. Vtables normally come with typeinfo too, so that'll also need to be
> available outside its DLL.

vtables (if they exist) do not have to be available outside a dynamic
library. There are some object implementation techniques that do require
it, but that's an implementation decision. It's not inherent in C++.

With a compiler that uses vtables (as all do today), you need the vtable
pointer in two (sort of) places: constructors and destructors. If all
the constructors and destructors are in the dynamic library then the
vtable doesn't have to be available outside of it. If they're in
multiple dynamic libraries (which can easily happen if a constructor is
inline, or if you rely on default constructors) the compiler can put one
vtable in each dynamic library. That's how Borland's implementation
works.

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

===================================== MODERATOR'S COMMENT:

The moderators debated whether to approve this post.  We eventually
decided to accept it because standardization of dynamic libraries is an
important subject; people who are interested in this subject might want
to read the standards committee document N1496.  While this post is
relevant to standardization of dynamic libraries, much of it deals with
issues that are relevant to one specific platform (ELF) and to
implementation techniques for one specific compiler (gcc).  Followups
to this post should address the standardization issues: "what should a
future C++ standard say", as opposed to "how should gcc behave".
Gcc-specific and ELF-specific discussion belongs elsewhere.


===================================== END OF MODERATOR'S COMMENT
Hi,

Dave Abrahams suggested I post here - it seems a little too GCC orientated
for this group, but similarly if GCC implements it then other
C++ compilers will surely be forced to eventually follow. Also, it's hard
to find such a concentration of experience elsewhere! :)

Firstly, the functionality has already been implemented - I'm looking for
comments, support and testing but also I'd like to raise awareness that
not only this is possible, it should become the de facto mechanism for all
new code and existing projects should be converted to support it. Here's
the patch:

http://www.nedprod.com/programs/gccvisibility.html
http://gcc.gnu.org/ml/gcc-patches/2004-05/msg00571.html

A rather full discussion of its development is at:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=15000

Ok what is it I'm proposing? At the moment on systems using ELF shared
objects, symbols are by default "default" (ie; public) unless specified
otherwise. I'm proposing that they go like Win32 PE DLL's and be by
default "hidden" unless specified otherwise. The patch above *optionally*
enables this without affecting behaviour if the new command line option
-fvisibility is not used (-fvisibility was chosen as Intel's C++ compiler
has the same switch - and I believe the Solaris may have something similar
too). All GCC regression tests pass.

Why is this a good thing? Because template & namespace heavy C++ generates
lots of very long symbol names, cutting out most of them can significantly
reduce link & times and cut several megabytes off the DSO. An extreme
example is my library TnFOX's python bindings based on Boost.Python going
 from a six minute load time down to only eight seconds with 21Mb removed
 from the binary! Furthermore as previous commentators like Ulrich Drepper
have pointed out (at http://www.nedprod.com/programs/dsohowto.pdf), costly
PLT indirections and load-time relocations can be avoided. The historical
sluggishness of loading applications and disjoint of code to produced
binary on Linux/BSD versus Windows could be eliminated.

The syntax is broadly compatible with MSVC, allowing the usual macro
system choosing between dllexport and dllimport to be hijacked and
conversion of such projects to be performed in a few minutes. It would
take longer however to mark DSO-internal external symbols as private in
order to maximise the benefits, but this was a seldomly-selected option
which should have been done for previous GCC versions anyway.

The other option is more controversial:- fvisibility-inlines-hidden makes
all inlineable code of hidden visibility thus causing with one fell swoop
a major reduction in export table size with no source modification
required. With a typical C++ library, you would see a 40% reduction with
this option as compared to a 60% reduction with fully implemented
-fvisibility=hidden. It could cause problems however, and I'd like
comments so I can make the docs better.

While I personally think that each API must be manually marked as hidden
when its enclosing class scope is different (it's part of the interface
contract IMHO), Brian Ryner who did the hard work in the patch disagrees.
To cover all views there is a #pragma GCC visibility which lets you change
the default visibility for the enclosing code. With the patch enabled my
library's symbol export table approaches MSVC's output in size (9435
versus 9038, the difference mostly arising from implementational
differences in RTTI).

Lastly, here are some concerns which have already been aired but deserve
further comment from you guys:

1. No warning is currently issued for base classes having a different
visibility to the current class. A simple implementation of this could be
added but it would be annoying for the many times when one is inheriting
off an instantiation of a template which will be almost always hidden (but
that's okay as it's also inlined). Therefore to be useful, a complex
traversal of base classes and their members would have to be made and if
and only if all are inlined does any non-default visibility become
unimportant. This warning isn't vital however because if you do make this
mistake you'll get a link error albeit at the point of the very last link
so really it's a convenience measure.

2. More seriously, this feature breaks the ISO C++ and ANSI C spec
requirement of being able to compare function pointers for equivalence
across a DSO boundary. It works fine if the function is default but not if
there are two separate copies residing in each binary. I think this is an
acceptable cost as if the programmer does want to compare them they can
make the relevent API non-hidden.

3. The semantics of __declspec(dllexport) and
__attribute__((visibility("default"))) are not the same. For example when
applied to function pointers one can run into trouble because under this
patch there is no such thing as a private pointer to public visibility
code or vice versa - the attribute always applies to the thing being
declared/defined, not what it points to. This can cause spurious warnings
at best but at worst misoperation due to symbol collision for example. As
these cases are isolated and usually cause a warning, I think it requires
a source change using some preprocessor test.

Any comments, test results or opinions welcomed. Most importantly of all
I'd love to see you guys using this facility for the code you write and in
examples in CUJ for example so that this "better practice" spreads rapidly.

(Note that due to receiving over 1000 spams per day my 33k dialup
connection simply can't cope and I've had to use a mangled email address
above).

Cheers,
Niall

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Niall Douglas wrote:
>
> ===================================== MODERATOR'S COMMENT:
>
> The moderators debated whether to approve this post.  We eventually
> decided to accept it because standardization of dynamic libraries is an
> important subject; people who are interested in this subject might want
> to read the standards committee document N1496.

Another paper that might be useful is N1418, which summarizes a
technical session that I presented on this topic. It's more in the form
of background, and tries to summarize the state of the art and to
provide a common base of technical terms and concepts.

www.open-std.org/jtc1/sc22/wg21/docs/papers/2002/n1418.html

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

On Tue, 18 May 2004 16:57:09 +0000 (UTC), Niall Douglas
<s_googlegroups@nedprod.com> wrote:

> MODERATOR'S COMMENT: The moderators debated whether to approve this
> post.  We eventually decided to accept it because standardization of
> dynamic libraries is an important subject; people who are interested in
> this subject might want to read the standards committee document N1496.
> While this post is relevant to standardization of dynamic libraries,
> much of it deals with issues that are relevant to one specific platform
> (ELF) and to implementation techniques for one specific compiler (gcc).
> Followups to this post should address the standardization issues: "what
> should a future C++ standard say", as opposed to "how should gcc
> behave".  Gcc-specific and ELF-specific discussion belongs elsewhere.

I was vaguely aware that N1496 existed and having read it now for the
first time, I make the following comments:

1. From experience I can tell you that a method of not sharing something
is needed. Therefore the "unshared" keyword should also be applicable to
individual members of a class even if the class itself is something else
eg;

shared class Foo
{
    void method();
    unshared void method2();
};

MSVC currently doesn't support setting individual members of a class when
the class itself is set - mine & Brian's patch to GCC does permit this. I
think here MSVC should be upgraded - often I find myself needing to use
private methods for internal use only which I personally prefix with
"int_" to avoid accidents with friendly use outside the DLL. If one could
override on a per-member basis I wouldn't have to do that (though I
probably still would to prevent accidental intra-DLL usage).

2. AFAICS N1496 doesn't address legacy code issues - what happens when a
header file is included which does declare a shared library API but
doesn't use the new "shared" keyword. I had this issue with the GCC patch
which I solved by "if it's declared not defined it's assumed shared".
Unfortunately that brings in inefficiency as an unnecessary PLT
indirection is required even when the definition resides in the local DSO
though possibly specifying ELF protected visibility might solve this. As
things currently stand with the ELF and Unix shared library system, N1496
has a problem.

3. N1496 speaks of "final resolution" of a shared linkage symbol. ELF
currently doesn't do this, it always indirects via the GOT which changes
as loaded shared libraries change. Therefore currently there is not final
resolution on ELF/GCC.

4. N1496 could do with an example of the difference between "shared void
(*foo)()" and "void (shared *foo)()".

4. N1496 doesn't speak of loadable libraries - however I don't like how on
Unix/ELF/GCC you have one choice of either using RTLD_GLOBAL or not - this
is too much or not enough because symbols from it can easily screw up your
linkage table but without RTLD_GLOBAL means any shared libraries the
library itself uses cause a new copy to be loaded ignoring the current one
(which breaks loads of code, see the python mailing list for why
RTLD_GLOBAL must be used for loading extension modules). It also opens
security risks and generally makes people's lives more complicated. If
N1496 doesn't do something, it mustn't do this.

5. Speaking of doing things, N1496 should specify a default set of search
locations for any loaded library eg; next to the calling executable and
paths in a certain environment variable. For libraries the loaded library
loads, it should search the calling executable's location and the loaded
library's location and so on recursively. Both Unix/ELF/GCC and MSVC
diverge radically here and it causes issues - I have lots of manual search
code forcing a standard search mechanism for both platforms and quite
frankly I shouldn't need to as it's obvious to anyone sane.

In general, N1496 has chosen to follow the Windows shared library
mechanism rather than the Unix/ELF/GCC one. I will apologise now if this
offends anyone, but I have always found the Unix/ELF/GCC mechanism to be
ill conceived and even with my patch to GCC it still comes with costs &
overheads which better design (ie; the Windows mechanism) automatically
avoids. Therefore I agree with the author of N1496's conclusion that the
Windows mechanism is preferable.

There is some isomorphism between my patch's syntax and MSVC's/N1496's. I
suggest that this patch to GCC should be viewed as a temporary stepping
stone to bridge existing Unix libraries onto an intermediate still ELF
based system but with a view to the eventual move to a MSVC/N1496-like
system in a future ISO C++ standard. Immediately moving from what things
are now to N1496 would break almost all Unix/ELF/GCC code (see point 2
above) but at least with a system inbetween it could be modified to
support N1496 with little cost, so long as N1496 allows "class shared Foo"
ie; the shared to come after class and "extern shared int foo()" etc. (ie;
let the "shared" go anywhere).

Comments and thoughts?

Cheers,
Niall

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

s_googlegroups@nedprod.com (Niall Douglas) writes:

> On Tue, 18 May 2004 16:57:09 +0000 (UTC), Niall Douglas
> <s_googlegroups@nedprod.com> wrote:
[snip]
> 1. From experience I can tell you that a method of not sharing
>    something is needed. Therefore the "unshared" keyword should also
>    be applicable to individual members of a class even if the class
>    itself is something else eg;
>
> shared class Foo
> {
>     void method();
>     unshared void method2();

'not shared' is IMO as clear as 'unshared'. So C++ can avoid a keyword,
    if people feel that is important.

> };
[snip]
> 2. AFAICS N1496 doesn't address legacy code issues - what happens when
>    a header file is included which does declare a shared library API
>    but doesn't use the new "shared" keyword. I had this issue with the
>    GCC patch which I solved by "if it's declared not defined it's
>    assumed shared". Unfortunately that brings in inefficiency as an
>    unnecessary PLT indirection is required even when the definition
>    resides in the local DSO though possibly specifying ELF protected
>    visibility might solve this. As things currently stand with the ELF
>    and Unix shared library system, N1496 has a problem.
[snip]

I believe this is only true of the first of the 3 alternatives.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

s_googlegroups@nedprod.com (Niall Douglas) wrote in message news:<opr77mjyuoy9klrv@news.iol.ie>...
> On Tue, 18 May 2004 16:57:09 +0000 (UTC), Niall Douglas
> <s_googlegroups@nedprod.com> wrote:
> I was vaguely aware that N1496 existed and having read it now for the
> first time, I make the following comments:
>
> 1. From experience I can tell you that a method of not sharing something
> is needed. Therefore the "unshared" keyword should also be applicable to
> individual members of a class even if the class itself is something else
> eg;
>
> shared class Foo
> {
>     void method();
>     unshared void method2();
> };
>
> MSVC currently doesn't support setting individual members of a class when
> the class itself is set - mine & Brian's patch to GCC does permit this. I
> think here MSVC should be upgraded - often I find myself needing to use
> private methods for internal use only which I personally prefix with
> "int_" to avoid accidents with friendly use outside the DLL. If one could
> override on a per-member basis I wouldn't have to do that (though I
> probably still would to prevent accidental intra-DLL usage).

This may indeed be useful, but before such wording is accepted into
the standard we should make sure this is reasonably implementable.
I assume most if not all compilers/linkers treat methods as normal
functions with a first parameter (classname* this), distinguished at
most by their mangled name.
The only effect I do worry about is the combination of virtual/RTTI
and unshared. I'd imagine that a class which is shared will have
identical type_info, same virtual behavior etc across linkage unit
boundaries. A class which is unshared in effect amounts to one
class per linakge unit, so the type_info's would be different. A
shared class with one unshared method is somewhat in the middle,
but I'm not sure what that would mean for type_info.

> 2. AFAICS N1496 doesn't address legacy code issues - what happens when a
> header file is included which does declare a shared library API but
> doesn't use the new "shared" keyword.  [snip]

The standard doesn't have to address legacy issues, especially if they're
limited to one platform. E.g. <iostream.h> is not covered, yet many
compilers include such headers for legacy code issues.

> 3. N1496 speaks of "final resolution" of a shared linkage symbol. ELF
> currently doesn't do this, it always indirects via the GOT which changes
> as loaded shared libraries change. Therefore currently there is not final
> resolution on ELF/GCC.

This doesn't matter, I think. The proposed model doesn't support unload,
and adding multiple definitions of the same shared symbol would violate
the ODR. To detect the behavior you mention, a new library would have to
change an entry in the GOT, whih in turns implies an ODR violation,
which implies UB. This may include "behavior typical for the platform",
so the behavior typical for ELF is allowed.
In other words, "final" means only "until you have undefined behavior".

> 4. N1496 could do with an example of the difference between "shared void
> (*foo)()" and "void (shared *foo)()".

Well, perhaps you could write such an example ? You probably know more
about this subject than the majority of the committee members.

> 4. N1496 doesn't speak of loadable libraries - however I don't like how on
> Unix/ELF/GCC you have one choice of either using RTLD_GLOBAL or not - this
> is too much or not enough because symbols from it can easily screw up your
> linkage table but without RTLD_GLOBAL means any shared libraries the
> library itself uses cause a new copy to be loaded ignoring the current one
> (which breaks loads of code, see the python mailing list for why
> RTLD_GLOBAL must be used for loading extension modules). It also opens
> security risks and generally makes people's lives more complicated. If
> N1496 doesn't do something, it mustn't do this.

I'm not sure about what this means?

> 5. Speaking of doing things, N1496 should specify a default set of search
> locations for any loaded library eg; next to the calling executable and
> paths in a certain environment variable. For libraries the loaded library
> loads, it should search the calling executable's location and the loaded
> library's location and so on recursively. Both Unix/ELF/GCC and MSVC
> diverge radically here and it causes issues - I have lots of manual search
> code forcing a standard search mechanism for both platforms and quite
> frankly I shouldn't need to as it's obvious to anyone sane.

This is definitely outside the scope of the C++ standard, just as the
default set of search locations for the executable itelf currently is
not specified. I would accept an 'implementation-defined' method,
but 'unspecified' is an alternative.
This is precisely because the techniques currently used differ so
much. We're not trying to change that, we're trying to create a
portable method of using the common subset.

> There is some isomorphism between my patch's syntax and MSVC's/N1496's. I
> suggest that this patch to GCC should be viewed as a temporary stepping
> stone to bridge existing Unix libraries onto an intermediate still ELF
> based system but with a view to the eventual move to a MSVC/N1496-like
> system in a future ISO C++ standard. Immediately moving from what things
> are now to N1496 would break almost all Unix/ELF/GCC code (see point 2
> above) but at least with a system inbetween it could be modified to
> support N1496 with little cost, so long as N1496 allows "class shared Foo"
> ie; the shared to come after class and "extern shared int foo()" etc. (ie;
> let the "shared" go anywhere).

While it would be nice to have a low-cost transition path from non-standard
C++ programs to standard C++ programs, this is IMO not a primary concern for
the proposal. Legacy platform issues are best solved by the maintainers
of those platforms, and GCC has a good track record (e.g. for-scoping).

In general, I think this development shows that N1496 is a viable direction,
and adding the support required can be done in current compilers. It also
shows how much more work it is to deal with not just standard code, but
also with legacy code. If we have quick wins here (e.g. by being
flexible about the position of shared in a declaration) we should take
them, but let's not try too hard. Systems differ, we just define a
single interface to the intersection of functionality.

Regards,
Michiel Salters

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

On Wed, 19 May 2004 17:14:53 +0000 (UTC), Michiel Salters
<Michiel.Salters@logicacmg.com> wrote:

>> 1. From experience I can tell you that a method of not sharing something
>> is needed. Therefore the "unshared" keyword should also be applicable to
>> individual members of a class even if the class itself is something else
>> eg;
>>
>> shared class Foo
>> {
>>     void method();
>>     unshared void method2();
>> };
> [snip]
>
> This may indeed be useful, but before such wording is accepted into
> the standard we should make sure this is reasonably implementable.
> I assume most if not all compilers/linkers treat methods as normal
> functions with a first parameter (classname* this), distinguished at
> most by their mangled name.

Does that matter? The "shared" and "unshared" attribute merely affects
whether the definition of a particular symbol is present in the export
table or not.

> The only effect I do worry about is the combination of virtual/RTTI
> and unshared. I'd imagine that a class which is shared will have
> identical type_info, same virtual behavior etc across linkage unit
> boundaries. A class which is unshared in effect amounts to one
> class per linakge unit, so the type_info's would be different. A
> shared class with one unshared method is somewhat in the middle,
> but I'm not sure what that would mean for type_info.

Very little surely. type_info for a class tells us about its type, not
what methods it has apart from indirectly what virtual functions it has.

The virtual function table must be unaffected by sharedness except being
itself shared if its type is. Interestingly a virtual function may be
unavailable via the export table (direct reference) but available by being
present in the virtual function table constructed within the same DLL as
the definition of that non-shared virtual function. If the constructor is
inlined, this should cause a link error unless it's redefined by a
subclass.

>> 2. AFAICS N1496 doesn't address legacy code issues - what happens when a
>> header file is included which does declare a shared library API but
>> doesn't use the new "shared" keyword.  [snip]
>
> The standard doesn't have to address legacy issues, especially if they're
> limited to one platform. E.g. <iostream.h> is not covered, yet many
> compilers include such headers for legacy code issues.

I hope I'm not overstepping my position, but if legacy code issues weren't
very important to C++ they'd have broken much more with C and indeed
legacy AT&T C++.

The longer the next ISO C++ standard doesn't touch shared library support,
the more chance for divergence between proprietary support. I had hell
with convincing some GCC developers that any ground whatsoever should be
given towards syntaxical compatibility with MSVC despite the millions of
lines out there already with the MSVC mechanism supported :(

I personally view shared libraries as as intrinsic to good program design
as class layout. Therefore they are important to the language.

>> 3. N1496 speaks of "final resolution" of a shared linkage symbol. ELF
>> currently doesn't do this, it always indirects via the GOT which changes
>> as loaded shared libraries change. Therefore currently there is not
>> final
>> resolution on ELF/GCC.
>
> This doesn't matter, I think. The proposed model doesn't support unload,
> and adding multiple definitions of the same shared symbol would violate
> the ODR. To detect the behavior you mention, a new library would have to
> change an entry in the GOT, whih in turns implies an ODR violation,
> which implies UB. This may include "behavior typical for the platform",
> so the behavior typical for ELF is allowed.
> In other words, "final" means only "until you have undefined behavior".

I generally take the rather un-Unix viewpoint that there is one correct
way of doing something and choice where the correct way is known is a bad
idea.

Unix's shared library model is ill conceived at best and I suppose I would
hope that the next ISO C++ standard could be used to force Unix to dump
the system it has now in favour of the correct one. Of course, I am
probably being too optimistic :(

>> 4. N1496 could do with an example of the difference between "shared void
>> (*foo)()" and "void (shared *foo)()".
>
> Well, perhaps you could write such an example ? You probably know more
> about this subject than the majority of the committee members.

I'd doubt that. Of course, if someone would like to pay for me to attend
C++ conferences I'd be happy to contribute a wide range of probably very
divisive opinions (I'm not a supporter of OO for example :). However as it
stands I'm leaving software programming as I've been unemployed for two
years now.

shared void (*foo)() is a pointer to a function where the pointer itself
is shared.
void (shared *foo)() is a pointer to a shared function
shared void (shared *foo)() is a shared pointer to a shared function.

>> 4. N1496 doesn't speak of loadable libraries - however I don't like how
>> on
>> Unix/ELF/GCC you have one choice of either using RTLD_GLOBAL or not -
>> this
>> is too much or not enough because symbols from it can easily screw up
>> your
>> linkage table but without RTLD_GLOBAL means any shared libraries the
>> library itself uses cause a new copy to be loaded ignoring the current
>> one
>> (which breaks loads of code, see the python mailing list for why
>> RTLD_GLOBAL must be used for loading extension modules). It also opens
>> security risks and generally makes people's lives more complicated. If
>> N1496 doesn't do something, it mustn't do this.
>
> I'm not sure about what this means?

On Unix/ELF/GCC you have a choice for resolving external symbols when you
load a shared library - either use the process' existing set of symbol
mappings or create a new set by loading new copies of each library it
needs. Unfortunately the process is two-way - if the library being loaded
reuses the process' existing set for the symbols it needs, it also can
rewrite symbols it itself defines to point at itself despite previously
pointing elsewhere. This is the dreaded "symbol collision" problem which
previously you used namespaces to avoid (which significantly slow load &
link times by sheer mangled name length volume).

What we actually want is two flags - one to cause reuse of existing loaded
libraries rather than loading new copies and another to allow symbols
being loaded to overwrite the current ones. The latter is most amenable to
individual symbol setting like sharedness.

Actually, N1496 could add a parameter eg; shared(not), shared(override) to
specify a symbol as overriding any already present. I'd like to see the
ISO C++ spec support setting of symbol weakness officially too - maybe
some unified mechanism of setting symbol attributes should be used though
shared(weak) sounds funny.

> This is precisely because the techniques currently used differ so
> much. We're not trying to change that, we're trying to create a
> portable method of using the common subset.

Again, C++'s popularity means it can throw its weight around to force more
consistent behaviour across platforms. Of course, certain members of the
ISO committee will oppose this as it makes writing portable C++ easier and
they are extending & embracing C++ right now to help people choose their
proprietary system instead.

> In general, I think this development shows that N1496 is a viable
> direction,
> and adding the support required can be done in current compilers. It also
> shows how much more work it is to deal with not just standard code, but
> also with legacy code. If we have quick wins here (e.g. by being
> flexible about the position of shared in a declaration) we should take
> them, but let's not try too hard. Systems differ, we just define a
> single interface to the intersection of functionality.

Well if GCC goes then ICC will surely follow quickly. That with MSVC,
Borland and DMC++ then covers a good proportion of the overall
C++ compilers out there.

Cheers,
Niall

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Niall Douglas wrote:
>
> MSVC currently doesn't support setting individual members of a class when
> the class itself is set - mine & Brian's patch to GCC does permit this. I
> think here MSVC should be upgraded - often I find myself needing to use
> private methods for internal use only which I personally prefix with
> "int_" to avoid accidents with friendly use outside the DLL.

Back in the bad old days with Windows compilers you could mark
individual functions as shared, which gave you the power to select which
member functions would be shared. Nobody used it.

> 2. AFAICS N1496 doesn't address legacy code issues - what happens when a
> header file is included which does declare a shared library API but
> doesn't use the new "shared" keyword.

It's not intended to.

> 3. N1496 speaks of "final resolution" of a shared linkage symbol. ELF
> currently doesn't do this, it always indirects via the GOT which changes
> as loaded shared libraries change. Therefore currently there is not final
> resolution on ELF/GCC.

Sure there is. You're too deep in implementation details to see it. 1496
presents an abstraction. How it's implemented under the covers doesn't
matter. Final resolution simply means that when you call a function
that's defined in a shared library you get code from the current version
of that shared library.

>
> 4. N1496 could do with an example of the difference between "shared void
> (*foo)()" and "void (shared *foo)()".

Not until there's been quite a bit more discussion of syntax and
semantics.

>
> 4. N1496 doesn't speak of loadable libraries

It does: it says that it doesn't address them yet.

> - however I don't like how on
> Unix/ELF/GCC you have one choice of either using RTLD_GLOBAL or not - this
> is too much or not enough because symbols from it can easily screw up your
> linkage table but without RTLD_GLOBAL means any shared libraries the
> library itself uses cause a new copy to be loaded ignoring the current one
> (which breaks loads of code, see the python mailing list for why
> RTLD_GLOBAL must be used for loading extension modules). It also opens
> security risks and generally makes people's lives more complicated. If
> N1496 doesn't do something, it mustn't do this.

But again you're deep in implementation details. The goal of the paper
is to get to an abstraction that's reasonably implementable. How that
maps to ELF, PE, or whatever else you may be dealing with is up to the
compiler writer.

>
> 5. Speaking of doing things, N1496 should specify a default set of search
> locations for any loaded library

No. Again, too deep in details. The standard doesn't say anything about
paths anywhere: not for include files, object files, libraries,
executables.

> There is some isomorphism between my patch's syntax and MSVC's/N1496's.

1496 does not yet propose any syntax.

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Niall Douglas wrote:
>
> 2. AFAICS N1496 doesn't address legacy code issues - what happens when a
> header file is included which does declare a shared library API but
> doesn't use the new "shared" keyword.

Nor should it, in my view. No two platforms, let alone compilers,
address symbol visibility in shared libraries in the same way. Should
the standard list all known source-code syntax extensions and bless
them? I don't think that makes any sense. The standard should specify
one valid syntax. New compilers could choose to accept their legacy
syntax in addition to the new standard syntax.

As part of developing a standard way to deal with this issue, we
should of course look at existing practice, and try to come up with a
scheme that allows easy adaption of as much existing code as possible.

Case study:

Sun C++ in its current release (Sun Studio 8, C++ 5.5) allows
specification of symbol visibility in shared libraries by means of new
keywords in source code (__global, __symbolic, __hidden), plus
command-line options to control default visibility.

One goal of our approach was to allow code written for MS VC++ to be
easy to port. We developed macros to transform the VC++ __declspec
syntax to our keywords. When porting code from Windows to Solaris, you
can define those macros before including the headers developed for
VC++. (Sun C++ doesn't recognize all VC++ extensions, however.)

We also wanted to accomodate similar g++ syntax, but we found that
different versions of g++ used significantly different syntax. One
version of the syntax could be transformed easily via macros as above,
but another could not. We elected to wait until g++ syntax stabilized
before making any committment to compatibility.

I'm not holding up the Sun C++ implementation as one that necessarily
should be adopted as the standard, although of course I would not
object. :-)  I provide this case study as an example of trying to
accomodate existing practice.

As it happened, some of our customers were not happy with the macro
solution. Since our syntax was only another local extension, we wound
up also accepting the VC++ syntax directly.

But I still think the C++ standard would not serve the programming
community well by specifying multiple ways to accomplish the same
task. With due consideration to existing practice, the standard should
define a single way to deal with symbol visibility.

---
Steve Clamage, stephen.clamage@sun.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]

Niall Douglas wrote:
>
> The virtual function table must be unaffected by sharedness except being
> itself shared if its type is.

Objects (including virtual function tables, if they exist at all) are
never shared. Names of objects are shared.

I don't know what it means for a type to be shared, except to say that
since a type isn't an object, it can't be shared in the sense that we've
just been talking about. Which means that I can't say whether sharing a
type requires sharing its vtable.

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html                       ]