In article <8lssbl$4b3$1@nnrp1.deja.com>,
  wmm@fastdial.net wrote:
> In article <3981C62B.1835C1F1@aspi.net>,
>   "Trevor L. Jackson, III" <fullmoon@aspi.net> wrote:
> >
> > [snip]
> >
> > Note that the argument regarding existing code bases in which
> T::operator
> > new(size_t) ignores its size parameter could [should?] have been met
> with
> > T::operator new(/*void*/).  The idea of preserving such a lethal
> recessive and at
> > the cost of a new special method, T::operator new[](size_t), is
> unpersuasive.
>
> Unpersuasive to you, and in this particular context.  We
> faced dozens of decisions about whether to break existing
> code or not.  We evaluated each one not only on its own
> merits but also on the cumulative pain adoption of the new
> Standard would inflict on the user community and avoided
> breaking code unless it was well-nigh unavoidable.
>
> In this particular case, the issue of breaking code was
> only a minor buttress to the decision we made; I don't
> think anyone on the Committee seriously championed the
> idea that array allocations should use T::operator new,
> for exactly the reasons that I quoted from the ARM:
>
> 1) An array of T is not a T, so it doesn't make sense to
> use an allocator of T to allocate an array of them.
>
> 2) An allocator of single objects is almost always simpler
> than an allocator of arbitrary-sized chunks of memory,
> particularly for the kinds of things they are most
> frequently used for (managing a free list of reusable
> objects or allocating a large chunk of memory and
> parceling it out one piece at a time), because you can
> assume that all the objects are the same size (in the
> absence of inheritance/polymorphism, which is often the
> case for typical applications of this capability).

In the absense of inheritance, yes...

I have occasionally considered implementing operator new
using a pool-based allocator specialized for fixed-size
objects. I've been dissuaded because my allocator would
break the minute someone derived from my class.

I like Trevor's suggestion of an operator new(/*void*/).
Such an allocator would not be inherited, and would be
explicitly permitted to assume a fixed allocation size.
It would even be fairly easy to implement a generic
template-based pool allocator and reuse it wherever
advantageous:

class MyClass {
public:
...
    void* operator new(void)
    {
        return PoolAllocator<MyClass>::alloc();
    }
};

If this is indeed the typical application for overloading
operator new, surely it would be nice if the language
provided a way of doing it safely.


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

On Thu,  3 Aug 2000 18:13:19 GMT, niklasb@my-deja.com wrote:
> I have occasionally considered implementing operator new
> using a pool-based allocator specialized for fixed-size
> objects. I've been dissuaded because my allocator would
> break the minute someone derived from my class.

Not if you include what I think should be a standard test at the top of
class-specific operator news:

  void* MyClass::operator new(size_t sz)
  {
    if (sz != sizeof(MyClass) return ::operator new(sz);

    // custom allocation stuff goes here
  }

This test also handles the unlikely, but legal, possibility that sz is 0.
For a fuller discusson, see Effective C++, Item 8.

Scott

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

In article <MPG.13f3855eec51c281989737@news.supernews.com>,
  smeyers@aristeia.com (Scott Meyers) wrote:
> On Thu,  3 Aug 2000 18:13:19 GMT, niklasb@my-deja.com wrote:
> > I have occasionally considered implementing operator new
> > using a pool-based allocator specialized for fixed-size
> > objects. I've been dissuaded because my allocator would
> > break the minute someone derived from my class.
>
> Not if you include what I think should be a standard test at the top
of
> class-specific operator news:
>
>   void* MyClass::operator new(size_t sz)
>   {
>     if (sz != sizeof(MyClass) return ::operator new(sz);
>
>     // custom allocation stuff goes here
>   }
>
> This test also handles the unlikely, but legal, possibility that sz
is 0.
> For a fuller discusson, see Effective C++, Item 8.

This had occurred to me, but I always wondered how to do the same
test in operator delete. I had thought my allocator would have to
be able to determine -- given only a pointer -- whether or not it
allocated the object. I have since learned that there are two
possible signatures for a class-member operator delete, the second
of which takes a size argument.


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

James Kuyper wrote:

> "Trevor L. Jackson, III" wrote:
> >
> > wmm@fastdial.net wrote:
> ....
> > > However, some people felt that it should be possible to
> > > control allocation of arrays of class objects specially rather
> > > than invoking the global ::operator new.  The operator new[]
> > > was invented to satisfy this desire.
> >
> > There's a terminology issue here.  I believe the rationale you described
> > applies to the new[] T operator rather than the function T::operator new[]().
> > AFAIK the former does not require the latter.
>
> You're correct; if T::operator new[]() does not exist, new[] will look
> for some other allocation function in the scope of T, and if it finds
> none it will use ::operator new[]().
> However, what was described is the reason for T::operator new[](); the
> new[] operator just provided a convenient syntax for invoking it (if
> present).

I think this is overstating the case.  Changing array allocation's default from
::operator new() to T:~somthing~() did not require the existence of T:operator
new[]().  Clearly the existence is optional, so it could not have been required.
T::operator new(size_t) provides all of the functionality and design freedom
necessary.  [Occam hereby invoked]

Note that the argument regarding existing code bases in which T::operator
new(size_t) ignores its size parameter could [should?] have been met with
T::operator new(/*void*/).  The idea of preserving such a lethal recessive and at
the cost of a new special method, T::operator new[](size_t), is unpersuasive.
<absurd mode>After all, what's to stop someone from ignoring the size provided by
T::operator new[](size_t)?  After all, the programmer knows how big the arrays he's
going to allocate are going to be... </absurd mode> [pause to wipe drool from
chin]  This is not the kind of software C++ should be protecting from good design
principles.

<rant>Condoning such practices is an egregious violation of good design
principles.  N.B., principles are weird -- you only know they are helping you when
you feel the pain of the constraints they impose.  IMHO the change to class
allocated arrays did not cause enough bloodshed to legacy code that should have
been stillborn. </rant>

>
>
> ....
> > (2) The distinction between operator new() and operator new[]() allows for
> > optimized allocation strategies based on small and large allocation
> > expectations respectively.  This eliminates a single conditional within
> > operator new(), but operator new[]() still needs the same conditional due to
> > the possibility of new[1] T.
>
> No it doesn't; you could let the inefficient algorithm be used
> regardless of the size, counting on the general tendency of operator
> new[]() calls to be for bigger sizes than operator new() calls to give
> you the extra efficiency, rather than spending time doing exactly the
> correct test. I'm not saying that it's worthwhile, but it would
> certainly be possible.

So why is it not possible to tolerate exactly the same inefficiency in T::operator
new(size_t) given that T::operator new[](size_t) did not exist?  In the few cases
where it matters I can put the same optimized algorithms I would have used in the
separate methods in a single method partitioned by a single if.

>
>
> ....
> > I'll get better software out of a journeyman programmer by restricting him to
> > use only realloc() than by encouraging him to use the variants of new/delete.
> > Certainly I can improve code quality by forbidding the use of new T
> > completely, replacing all such expressions with new[1] T.
>
> I certainly wouldn't recommend passing code which needs to use variants
> of new/delete to a journeyman programmer. The situations calling for
> anything other than the implementation-provided default ::operator new()
> are rare and tricky; you should call in one of your more experienced
> people to handle them.

The primary reason it is tricky is the convoluted design of the primitives.  The
purpose of my original question, and thus part of the purpose of this thread is to
identify the rationale that's makes C++'s heap management make sense.  The
responses I'm getting are convincing me that it does not make sense, but "jest
growed".

>
>
> In any event, realloc() isn't sufficient to cover the issues that arise
> in those situations, and can be more easily misused than operator
> new[](), because of the greater type safety of the new[] / operator
> new[]() interface.

I did not make the trade off explicit enough.  I'd be willing to sacrifice type
safety (automatic ctor/dtor invocation) in order to get a heap management interface
that has some kind of interface rationale that I can both comprehend and
propagate.  This is not a heap management issue.  It is a heap interface issue.

That's a fairly radical position.  It's based on an aesthetic criteria I suppose.
But the goal, robust software, is not one I'm willing to sacrifice to the vagaries
of historical usage or current expediency.

>
> T::operator new[]() has no need to know anything about the type of the
> objects it is allocating (they needn't be of type T, though they usually
> will be). The new[] operator, on the other hand, doesn't need to know
> anything about how to allocate raw memory. It's a good division of
> labor.

The division of labor between the new operators (new and new[]) and the underlying
heap management functions (operator new() and operator new[]) is a good division of
labor.  But notice that if you eliminate operator new[] (both global and local) you
get a system with the same division of labor.  So the quality of the division of
labor is not an issue.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

In article <3981C62B.1835C1F1@aspi.net>,
  "Trevor L. Jackson, III" <fullmoon@aspi.net> wrote:
> James Kuyper wrote:
>
> > "Trevor L. Jackson, III" wrote:
> > >
> > > wmm@fastdial.net wrote:
> > ....
> > > > However, some people felt that it should be possible to
> > > > control allocation of arrays of class objects specially rather
> > > > than invoking the global ::operator new.  The operator new[]
> > > > was invented to satisfy this desire.
> > >
> > > There's a terminology issue here.  I believe the rationale you
described
> > > applies to the new[] T operator rather than the function
T::operator new[]().
> > > AFAIK the former does not require the latter.

No, this has nothing to do with the "new[] T" operator (there
isn't any such operator, BTW; just the "new" operator applied
to an array type, either one that was already an array type
via a typedef or one created in situ using the
direct-new-declarator syntax).  I'm giving the rationale for
the existence of "T::operator new[]".  It was specifically
to allow per-class control of the storage allocation for
arrays of class objects.  "T::operator new", which already
existed, was not suitable (see below), so "T::operator new[]"
was invented.

> I think this is overstating the case.  Changing array allocation's
default from
> ::operator new() to T:~somthing~() did not require the existence of
T:operator
> new[]().

No, but since "T::operator new" was not a suitable candidate
for "T::~something~", some new member function needed to be
invented.  We did discuss a number of different names, but
"operator new[]" eventually won out.

> Clearly the existence is optional, so it could not have been required.
> T::operator new(size_t) provides all of the functionality and design
freedom
> necessary.

That was not the opinion of the Committee.  I'll quote from
the ARM, which dates from before the invention of operator
new[]: "The reason that an array of elements of class X isn't
allocated by an X::operator new() is simply that such an
array isn't an X.  This has the important side effect that
X::operator new() really can be specialized to be an
allocator for objects of class X and classes derived from X.
Had X::operator new() been called for arrays, it would have
had to be an almost general allocator with all the overheads
this implies."

You may disagree, but you asked the reason and this is the
reason.

> Note that the argument regarding existing code bases in which
T::operator
> new(size_t) ignores its size parameter could [should?] have been met
with
> T::operator new(/*void*/).  The idea of preserving such a lethal
recessive and at
> the cost of a new special method, T::operator new[](size_t), is
unpersuasive.

Unpersuasive to you, and in this particular context.  We
faced dozens of decisions about whether to break existing
code or not.  We evaluated each one not only on its own
merits but also on the cumulative pain adoption of the new
Standard would inflict on the user community and avoided
breaking code unless it was well-nigh unavoidable.

In this particular case, the issue of breaking code was
only a minor buttress to the decision we made; I don't
think anyone on the Committee seriously championed the
idea that array allocations should use T::operator new,
for exactly the reasons that I quoted from the ARM:

1) An array of T is not a T, so it doesn't make sense to
use an allocator of T to allocate an array of them.

2) An allocator of single objects is almost always simpler
than an allocator of arbitrary-sized chunks of memory,
particularly for the kinds of things they are most
frequently used for (managing a free list of reusable
objects or allocating a large chunk of memory and
parceling it out one piece at a time), because you can
assume that all the objects are the same size (in the
absence of inheritance/polymorphism, which is often the
case for typical applications of this capability).

There's another point here as well: although we
recognized that per-class allocation strategies for
arrays of class objects could make sense, there was (and
probably still is) much less demand for that capability.
It seemed reasonable to create an extra optional
facility for that support rather than changing the
simpler semantics already in use for the majority case.

> > > I'll get better software out of a journeyman programmer by
restricting him to
> > > use only realloc() than by encouraging him to use the variants of
new/delete.
> > > Certainly I can improve code quality by forbidding the use of new
T
> > > completely, replacing all such expressions with new[1] T.
> >
> > I certainly wouldn't recommend passing code which needs to use
variants
> > of new/delete to a journeyman programmer. The situations calling for
> > anything other than the implementation-provided default ::operator
new()
> > are rare and tricky; you should call in one of your more experienced
> > people to handle them.
>
> The primary reason it is tricky is the convoluted design of the
primitives.  The
> purpose of my original question, and thus part of the purpose of this
thread is to
> identify the rationale that's makes C++'s heap management make
sense.  The
> responses I'm getting are convincing me that it does not make sense,
but "jest
> growed".

That's not how I would characterize the process.  We had what
we considered to be valid reasons for the decisions we made;
it wasn't done thoughtlessly or arbitrarily.

--
William M. Miller, wmm@fastdial.net
Vignette Corporation (www.vignette.com)


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

James Kuyper  <kuyper@wizard.net> wrote:
+ Phil Edwards wrote:
+ >  -  The requirements on allocators ([20.1.5], table 32) specify that
+ >     allocate() and deallocate() take a parameter N as the number of
+ >     /objects/.
+ >
+ >  -  The argument to the various forms of new and delete is required to be
+ >     a number of /bytes/.
[snip]
+ > I was hoping for my allocators' array functions to be more efficient than
+ > "Take the given parameter, divide it to find the number of objects, add
+ > 2 or 3 to it just to be safe because there's no tellin' how many objects
+ > we're /really/ dealing with, and allocate space for that many."  :-)
+
+ You know precisely how many bytes must be allocated - the function's
+ argument tells you. It necessarily includes the extra memory.

We're in violent agreement there.  But this doesn't address my problem:
how are allocators supposed to be used here?  What's the /point/ in having
allocate() take a "number of objects" parameter if we can't determine what
the number is supposed to be, when it isn't 1?


Phil

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

"Trevor L. Jackson, III" wrote:
>
> James Kuyper wrote:
>
> > Phil Edwards wrote:
> > >
> > > I'm really hoping somebody will point out what I'm missing.
> > >
> > >  -  The requirements on allocators ([20.1.5], table 32) specify that
> > >     allocate() and deallocate() take a parameter N as the number of
> > >     /objects/.
> > >
> > >  -  The argument to the various forms of new and delete is required to be
> > >     a number of /bytes/.
> > >
> > > Okay, fine, for a class "X" I can write my single-object form like this:
> > >
> > >     static void* operator new (std::size_t)
> > >     {
> > >         return A.allocate(1);
> > >     }
> > >
> > > assuming a static class member "A" is a conforming allocator templatized
> > > on X.  Or I could have named the size_t parameter, divided it by sizeof(X),
> > > and passed that in, if I wanted an expensive method of calculating 1.
> > >
> > > But for arrays, this
> > >
> > >     static void* operator new[] (std::size_t numbytes)
> > >     {
> > >         return A.allocate(numbytes/sizeof(X));
> > >     }
> > >
> > > will break running under at least one compiler, because that compiler
> > > requests an extra 8 bytes of memory.  That is, for N objects, numbytes ==
> > > (N*sizeof(X) + 8).  The compiler stores N in that first 8 bytes, then
> > > uses the rest of the storage as the array.  You can see how this is going
> > > to break.
> > >
> > > Okay, grabbing extra "leading" space for bookkepping is a common
> > > memory-management technique and all, but how am I supposed to know how many
> > > objects to allocate?  Is this permitted behavior?  Surely the folks on the
> >
> > You don't. You're not supposed to allocate a certain number of objects,
> > you're supposed to allocate a certain amount of raw memory. How it's
> > parsed up into objects is an issue for the compiler to resolve, not for
> > operator new[]().
> >
> > > relevent portion of the committee thought about this; what's the rationale?
> > > Or rather, what's the technique we're all supposed to be using?
> >
> > malloc(N) returns space suitably aligned for any object that can be fit
> > in N bytes. new char[N] makes a similar guarantee (5.3.4p10). Whether
> > Allocator<T>::allocate() makes such a guarantee is up to the definer of
> > the Allocator class; the standard only requires it to produce results
> > suitably aligned for objects of type T. Therefore, im general an
> > Allocator<T> can't be used to allocate the space provided by a
> > user-defined operator new[].
> >
> > Any operator new[]() you provide must either call a member of the
> > malloc() family, use new char[], or manually meet the alignment
> > guarantee, which usually can't be done in a portable fashion. It
> > requires non-portable awareness of the alignment restrictions of the
> > implementation you're running under.
> >
> > > I was hoping for my allocators' array functions to be more efficient than
> > > "Take the given parameter, divide it to find the number of objects, add
> > > 2 or 3 to it just to be safe because there's no tellin' how many objects
> > > we're /really/ dealing with, and allocate space for that many."  :-)
> >
> > You know precisely how many bytes must be allocated - the function's
> > argument tells you. It necessarily includes the extra memory.
>
> OK, than why does operator new[]() exist?  Why don't the new and new[] operators
> both call operator new()?

I guess calls to operator new[] tend to have larger sizes, so
it could use an algorithm better suited for large sizes, while
operator new uses an algorithm optimized for small sizes.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

"Trevor L. Jackson, III" wrote:
>
>
> OK, than why does operator new[]() exist?  Why don't the new and new[] operators
> both call operator new()?
>

By default they do.

--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
Contributing Editor, C/C++ Users Journal (http://www.cuj.com)

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

In article <397E2124.DDD2A06C@aspi.net>,
  "Trevor L. Jackson, III" <fullmoon@aspi.net> wrote:
> OK, than why does operator new[]() exist?  Why don't the new and new
[] operators
> both call operator new()?

Prior to the invention of operator new[], there were two kinds
of allocation functions: T::operator new and ::operator new.
If you allocated an array of T, ::operator new was called, not
T::operator new.  Some people found that surprising, but the
rationale was straightforward: an array of T is not a T, and
the allocation function T::operator new ought to be permitted
to make assumptions about the circumstances of its invocation
that would be invalid for an array of T.

However, some people felt that it should be possible to
control allocation of arrays of class objects specially rather
than invoking the global ::operator new.  The operator new[]
was invented to satisfy this desire.

--
William M. Miller, wmm@fastdial.net
Vignette Corporation (www.vignette.com)


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Phil Edwards wrote:
....
> We're in violent agreement there.  But this doesn't address my problem:
> how are allocators supposed to be used here?  What's the /point/ in having
> allocate() take a "number of objects" parameter if we can't determine what
> the number is supposed to be, when it isn't 1?

You use allocate() for some purpose other than providing the
implementation of operator new[](), for some case when you do know what
the number of object is supposed to be. For instance, allocate() is very
useful when implementing containers.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Phil Edwards wrote:
>
> James Kuyper  <kuyper@wizard.net> wrote:
> + Phil Edwards wrote:
> + >  -  The requirements on allocators ([20.1.5], table 32) specify that
> + >     allocate() and deallocate() take a parameter N as the number of
> + >     /objects/.
> + >
> + >  -  The argument to the various forms of new and delete is required to be
> + >     a number of /bytes/.
> [snip]
> + > I was hoping for my allocators' array functions to be more efficient than
> + > "Take the given parameter, divide it to find the number of objects, add
> + > 2 or 3 to it just to be safe because there's no tellin' how many objects
> + > we're /really/ dealing with, and allocate space for that many."  :-)
> +
> + You know precisely how many bytes must be allocated - the function's
> + argument tells you. It necessarily includes the extra memory.
>
> We're in violent agreement there.  But this doesn't address my problem:
> how are allocators supposed to be used here?  What's the /point/ in having
> allocate() take a "number of objects" parameter if we can't determine what
> the number is supposed to be, when it isn't 1?

I guess allocators are _not_ to be used here. Instead I'd expect
operator new to be used in allocators.
Allocators are expected to be used in containers (and possibly
other classes which have to do memory management).

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Christopher Eltschka wrote:

> "Trevor L. Jackson, III" wrote:
> >
> > James Kuyper wrote:
> >
> > > Phil Edwards wrote:
> > > >

[snip historical context]

>
> > >
> > > > I was hoping for my allocators' array functions to be more efficient than
> > > > "Take the given parameter, divide it to find the number of objects, add
> > > > 2 or 3 to it just to be safe because there's no tellin' how many objects
> > > > we're /really/ dealing with, and allocate space for that many."  :-)
> > >
> > > You know precisely how many bytes must be allocated - the function's
> > > argument tells you. It necessarily includes the extra memory.
> >
> > OK, than why does operator new[]() exist?  Why don't the new and new[] operators
> > both call operator new()?
>
> I guess calls to operator new[] tend to have larger sizes, so
> it could use an algorithm better suited for large sizes, while
> operator new uses an algorithm optimized for small sizes.

So the language design optimizes away an if statement at the beginning of operator
new()?  Spare us from such optimizations! (C.f., a segmented memory architecture)

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

wmm@fastdial.net wrote:

> In article <397E2124.DDD2A06C@aspi.net>,
>   "Trevor L. Jackson, III" <fullmoon@aspi.net> wrote:
> > OK, than why does operator new[]() exist?  Why don't the new and new
> [] operators
> > both call operator new()?
>
> Prior to the invention of operator new[], there were two kinds
> of allocation functions: T::operator new and ::operator new.
> If you allocated an array of T, ::operator new was called, not
> T::operator new.  Some people found that surprising, but the
> rationale was straightforward: an array of T is not a T, and
> the allocation function T::operator new ought to be permitted
> to make assumptions about the circumstances of its invocation
> that would be invalid for an array of T.
>
> However, some people felt that it should be possible to
> control allocation of arrays of class objects specially rather
> than invoking the global ::operator new.  The operator new[]
> was invented to satisfy this desire.

There's a terminology issue here.  I believe the rationale you described
applies to the new[] T operator rather than the function T::operator new[]().
AFAIK the former does not require the latter.

So far I've heard three explanations for the distinction between operator
new() and operator new[]().

(1)  The function operator new() can deal with a fixed-size allocation
efficiently (faster and less fragmentation), whereas operator new[]() has to
deal with variable sizes.
  I find this explanation unconvincing because a derived class might inherit
operator new(), and there goes your efficiency.

(2) The distinction between operator new() and operator new[]() allows for
optimized allocation strategies based on small and large allocation
expectations respectively.  This eliminates a single conditional within
operator new(), but operator new[]() still needs the same conditional due to
the possibility of new[1] T.

(3) (Paraphrasing the above message) The existence of the new T and new[] T
distinction naturally implies a parallel distinction between operator new()
and operator new[]().

So, I still do not understand the rationale for the distinction between
operator new() and operator new[]().  The purpose of revamping the heap
management as part of the C to C++ evolution was to improve the reliability of
software by making it harder to make mistakes.  It appears to me that all of
the new/new[] rationales are driven by optimization concerns rather than
robustness concerns.

I'll get better software out of a journeyman programmer by restricting him to
use only realloc() than by encouraging him to use the variants of new/delete.
Certainly I can improve code quality by forbidding the use of new T
completely, replacing all such expressions with new[1] T.


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

"Trevor L. Jackson, III" wrote:
>
> wmm@fastdial.net wrote:
....
> > However, some people felt that it should be possible to
> > control allocation of arrays of class objects specially rather
> > than invoking the global ::operator new.  The operator new[]
> > was invented to satisfy this desire.
>
> There's a terminology issue here.  I believe the rationale you described
> applies to the new[] T operator rather than the function T::operator new[]().
> AFAIK the former does not require the latter.

You're correct; if T::operator new[]() does not exist, new[] will look
for some other allocation function in the scope of T, and if it finds
none it will use ::operator new[]().
However, what was described is the reason for T::operator new[](); the
new[] operator just provided a convenient syntax for invoking it (if
present).

....
> (2) The distinction between operator new() and operator new[]() allows for
> optimized allocation strategies based on small and large allocation
> expectations respectively.  This eliminates a single conditional within
> operator new(), but operator new[]() still needs the same conditional due to
> the possibility of new[1] T.

No it doesn't; you could let the inefficient algorithm be used
regardless of the size, counting on the general tendency of operator
new[]() calls to be for bigger sizes than operator new() calls to give
you the extra efficiency, rather than spending time doing exactly the
correct test. I'm not saying that it's worthwhile, but it would
certainly be possible.

....
> I'll get better software out of a journeyman programmer by restricting him to
> use only realloc() than by encouraging him to use the variants of new/delete.
> Certainly I can improve code quality by forbidding the use of new T
> completely, replacing all such expressions with new[1] T.

I certainly wouldn't recommend passing code which needs to use variants
of new/delete to a journeyman programmer. The situations calling for
anything other than the implementation-provided default ::operator new()
are rare and tricky; you should call in one of your more experienced
people to handle them.

In any event, realloc() isn't sufficient to cover the issues that arise
in those situations, and can be more easily misused than operator
new[](), because of the greater type safety of the new[] / operator
new[]() interface.
T::operator new[]() has no need to know anything about the type of the
objects it is allocating (they needn't be of type T, though they usually
will be). The new[] operator, on the other hand, doesn't need to know
anything about how to allocate raw memory. It's a good division of
labor.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Matthew Austern  <austern@research.att.com> wrote:
+ pedwards@dmapub.dma.org (Phil Edwards) writes:
+ > Surely the folks on the
+ > relevent portion of the committee thought about this; what's the rationale?
+ > Or rather, what's the technique we're all supposed to be using?
+
+ The rationale is that you aren't supposed to be using allocators with
+ array new-exprssions.  You're supposed to be using them to implement
+ STL containers, and there's no reason for an STL container to use
+ array operator new.

Ahhh... I see.  Thanks!  I desperately need that upcoming "Rationale"
book to be published.  Or I need to join the committee, just to listen to
this kind of thing being discussed.


Phil

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

William M. Miller <wmm@fastdial.net> wrote:
> 2) An allocator of single objects is almost always simpler
> than an allocator of arbitrary-sized chunks of memory,
> particularly for the kinds of things they are most
> frequently used for (managing a free list of reusable
> objects or allocating a large chunk of memory and
> parceling it out one piece at a time), because you can
> assume that all the objects are the same size (in the
> absence of inheritance/polymorphism, which is often the
> case for typical applications of this capability).

But the problem of inheritance, and to a lesser extent the ability
to call operator new directly, does compromise the safety of this
strategy.  I think the problem traces back to C++ insisting that
all uninitialized memory has the same type.

When the time comes to update the standard, I intend to agitate
for an extension which would allow an allocation function to
return storage_for<T>*, rather than void*, indicating that it
only allocates memory suitable for the exact type T.

                                             --Lisa Lippincott

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

I'm really hoping somebody will point out what I'm missing.


 -  The requirements on allocators ([20.1.5], table 32) specify that
    allocate() and deallocate() take a parameter N as the number of
    /objects/.

 -  The argument to the various forms of new and delete is required to be
    a number of /bytes/.

Okay, fine, for a class "X" I can write my single-object form like this:

    static void* operator new (std::size_t)
    {
        return A.allocate(1);
    }

assuming a static class member "A" is a conforming allocator templatized
on X.  Or I could have named the size_t parameter, divided it by sizeof(X),
and passed that in, if I wanted an expensive method of calculating 1.

But for arrays, this

    static void* operator new[] (std::size_t numbytes)
    {
        return A.allocate(numbytes/sizeof(X));
    }

will break running under at least one compiler, because that compiler
requests an extra 8 bytes of memory.  That is, for N objects, numbytes ==
(N*sizeof(X) + 8).  The compiler stores N in that first 8 bytes, then
uses the rest of the storage as the array.  You can see how this is going
to break.

Okay, grabbing extra "leading" space for bookkepping is a common
memory-management technique and all, but how am I supposed to know how many
objects to allocate?  Is this permitted behavior?  Surely the folks on the
relevent portion of the committee thought about this; what's the rationale?
Or rather, what's the technique we're all supposed to be using?

I was hoping for my allocators' array functions to be more efficient than
"Take the given parameter, divide it to find the number of objects, add
2 or 3 to it just to be safe because there's no tellin' how many objects
we're /really/ dealing with, and allocate space for that many."  :-)


Much thanks for clearing this up,
Phil

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Phil Edwards wrote:
>
> I'm really hoping somebody will point out what I'm missing.
>
>  -  The requirements on allocators ([20.1.5], table 32) specify that
>     allocate() and deallocate() take a parameter N as the number of
>     /objects/.
>
>  -  The argument to the various forms of new and delete is required to be
>     a number of /bytes/.
>
> Okay, fine, for a class "X" I can write my single-object form like this:
>
>     static void* operator new (std::size_t)
>     {
>         return A.allocate(1);
>     }
>
> assuming a static class member "A" is a conforming allocator templatized
> on X.  Or I could have named the size_t parameter, divided it by sizeof(X),
> and passed that in, if I wanted an expensive method of calculating 1.
>
> But for arrays, this
>
>     static void* operator new[] (std::size_t numbytes)
>     {
>         return A.allocate(numbytes/sizeof(X));
>     }
>
> will break running under at least one compiler, because that compiler
> requests an extra 8 bytes of memory.  That is, for N objects, numbytes ==
> (N*sizeof(X) + 8).  The compiler stores N in that first 8 bytes, then
> uses the rest of the storage as the array.  You can see how this is going
> to break.
>
> Okay, grabbing extra "leading" space for bookkepping is a common
> memory-management technique and all, but how am I supposed to know how many
> objects to allocate?  Is this permitted behavior?  Surely the folks on the

You don't. You're not supposed to allocate a certain number of objects,
you're supposed to allocate a certain amount of raw memory. How it's
parsed up into objects is an issue for the compiler to resolve, not for
operator new[]().

> relevent portion of the committee thought about this; what's the rationale?
> Or rather, what's the technique we're all supposed to be using?

malloc(N) returns space suitably aligned for any object that can be fit
in N bytes. new char[N] makes a similar guarantee (5.3.4p10). Whether
Allocator<T>::allocate() makes such a guarantee is up to the definer of
the Allocator class; the standard only requires it to produce results
suitably aligned for objects of type T. Therefore, im general an
Allocator<T> can't be used to allocate the space provided by a
user-defined operator new[].

Any operator new[]() you provide must either call a member of the
malloc() family, use new char[], or manually meet the alignment
guarantee, which usually can't be done in a portable fashion. It
requires non-portable awareness of the alignment restrictions of the
implementation you're running under.

> I was hoping for my allocators' array functions to be more efficient than
> "Take the given parameter, divide it to find the number of objects, add
> 2 or 3 to it just to be safe because there's no tellin' how many objects
> we're /really/ dealing with, and allocate space for that many."  :-)

You know precisely how many bytes must be allocated - the function's
argument tells you. It necessarily includes the extra memory.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

James Kuyper wrote:

> Phil Edwards wrote:
> >
> > I'm really hoping somebody will point out what I'm missing.
> >
> >  -  The requirements on allocators ([20.1.5], table 32) specify that
> >     allocate() and deallocate() take a parameter N as the number of
> >     /objects/.
> >
> >  -  The argument to the various forms of new and delete is required to be
> >     a number of /bytes/.
> >
> > Okay, fine, for a class "X" I can write my single-object form like this:
> >
> >     static void* operator new (std::size_t)
> >     {
> >         return A.allocate(1);
> >     }
> >
> > assuming a static class member "A" is a conforming allocator templatized
> > on X.  Or I could have named the size_t parameter, divided it by sizeof(X),
> > and passed that in, if I wanted an expensive method of calculating 1.
> >
> > But for arrays, this
> >
> >     static void* operator new[] (std::size_t numbytes)
> >     {
> >         return A.allocate(numbytes/sizeof(X));
> >     }
> >
> > will break running under at least one compiler, because that compiler
> > requests an extra 8 bytes of memory.  That is, for N objects, numbytes ==
> > (N*sizeof(X) + 8).  The compiler stores N in that first 8 bytes, then
> > uses the rest of the storage as the array.  You can see how this is going
> > to break.
> >
> > Okay, grabbing extra "leading" space for bookkepping is a common
> > memory-management technique and all, but how am I supposed to know how many
> > objects to allocate?  Is this permitted behavior?  Surely the folks on the
>
> You don't. You're not supposed to allocate a certain number of objects,
> you're supposed to allocate a certain amount of raw memory. How it's
> parsed up into objects is an issue for the compiler to resolve, not for
> operator new[]().
>
> > relevent portion of the committee thought about this; what's the rationale?
> > Or rather, what's the technique we're all supposed to be using?
>
> malloc(N) returns space suitably aligned for any object that can be fit
> in N bytes. new char[N] makes a similar guarantee (5.3.4p10). Whether
> Allocator<T>::allocate() makes such a guarantee is up to the definer of
> the Allocator class; the standard only requires it to produce results
> suitably aligned for objects of type T. Therefore, im general an
> Allocator<T> can't be used to allocate the space provided by a
> user-defined operator new[].
>
> Any operator new[]() you provide must either call a member of the
> malloc() family, use new char[], or manually meet the alignment
> guarantee, which usually can't be done in a portable fashion. It
> requires non-portable awareness of the alignment restrictions of the
> implementation you're running under.
>
> > I was hoping for my allocators' array functions to be more efficient than
> > "Take the given parameter, divide it to find the number of objects, add
> > 2 or 3 to it just to be safe because there's no tellin' how many objects
> > we're /really/ dealing with, and allocate space for that many."  :-)
>
> You know precisely how many bytes must be allocated - the function's
> argument tells you. It necessarily includes the extra memory.

OK, than why does operator new[]() exist?  Why don't the new and new[] operators
both call operator new()?


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]