"Steve Clamage" <stephen.clamage@sun.com> wrote

> A specific implemenation might use an operator new[] that assured that
> the block size was available at (for example) a fixed offset from
> the returned pointer, and that any extra amount allocated was somehow
> (how?) guaranteed to be smaller than sizeof(object).

If the reported "block size" is the originally requested size rather than
the actual amount of memory, you don't have to worry about how much "extra"
was allocated.  This assumes that the deallocator will be able to compute
the "extra" size when given the requested size.

> I'd like to know how systems that use this method continue to work
> in the presence of a user-supplied operator new[] that provides only
> what the standard requires.

I would assume that any such implementation would have to fall back to
another strategy when the user supplies new().

Since the implementation will have to handle the case where the allocator
doesn't "remember" the size, I would think that there would usually be very
little benefit in using a built-in allocator that did remember (and provide
access to) the size.

A special case might be where the implementation provides garbage collection
(without destructor invocation) for blocks that are allocated by the
built-in allocator.  For this type of GC it would be convenient to be able
to compute the allocated size without knowing the object type.  In this case
the default behavior for  'new T[N]' can be to store (N*sizeof(T)+overhead)
in the prefix bytes (overhead measures how big the prefix is, not how many
"spare" suffix bytes were included in the block). delete[] can easily
compute N, and the garbage collector can presumably compute the size of the
actual heap block.



---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Bill Wade <bill.wade@stoner.com> wrote in message
news:8n119h$du3@library1.airnews.net...

>In this case
> the default behavior for  'new T[N]' can be to store
(N*sizeof(T)+overhead)
> in the prefix bytes (overhead measures how big the prefix is, not how many
> "spare" suffix bytes were included in the block). delete[] can easily
> compute N, and the garbage collector can presumably compute the size of
the
> actual heap block.

Note, I assume that the prefix not only has to big enough to store the count
of elements (if it is implemented that way)  but also some padding bytes so
that start address of the array of objects is properly aligned.

Suppose for sake of argument we have sizeof(double) is 8 and sizeof(int) is
4, and that double's should be aligned on an address that is divisible by 8.
An implementation's new[] might do malloc(N*sizeof(double)+sizeof(int)),
storing the element count and now assuming malloc() returned an address that
aligned, the new[]'ed array of double's is misaligned.

Stephen Howe


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

"Stephen Howe" <SPAMGUARDstephen.howe@dial.pipex.co.uk> wrote

> Note, I assume that the prefix not only has to big enough to store the
count
> of elements (if it is implemented that way)  but also some padding bytes
so
> that start address of the array of objects is properly aligned.

Correct.  The address returned by operator new[]() points at the beginning
of the prefix, and must be suitably aligned for any type that fits in the
returned space.  The address returned by the new expression (the address of
the first element of the array) must be suitably aligned for any object that
could fit in the array.  In practice that means the size of the prefix will
usually be some multiple of the strictest alignment requirement used on the
platform.



---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

David R Tribble wrote:
>
> Stephen Clamage wrote:
> > ...
> > I know of two methods in common use for keeping track of the number of
> > elements in an allocated array.
> > ...
> > 2. For any array allocation, the new-expression requests a little
> > extra memory from the allocator, stores the count in that space, and
> > adjusts the returned value to point past the count. An array delete-
> > expression decrements the pointer value to point to the start of the
> > extra memory, retrieves the element count, calls destructors, and
> > passes the adjusted address to the deallocator function (operator
> > delete[]).
> >
>
> As has been pointed out in this newsgroup many months ago, it is not
> strictly necessary to allocate extra space for the array count;
> dividing the total size of the allocated block (known at runtime) by
> the size of the element type (known at compile time) will result in
> the proper array element count.  (I believe that many systems
> actually do it this way.)

That method assumes that

1. the compiler can generate code to find out the actual size of the
block that was allocated, and

2. the size allocated for N objects is smaller than (N+1)*sizeof(object).

Operator new, which is replaceable by users, does not provide a way
to find the actual size allocated.  In a typical implementation,
operator new[] just calls malloc, which does not provide the information
either. Without this information, the allocation process must store
the allocation size somewhere, which I don't see as simpler or better
than storing the count. After all, what you really want is the count.

Some allocators round up allocation requests to a multiple of some
interesting size, which might be larger that the size of the
allocated object. In that case, size/sizeof(object) provides too
large a value for N.

A specific implemenation might use an operator new[] that assured that
the block size was available at (for example) a fixed offset from
the returned pointer, and that any extra amount allocated was somehow
(how?) guaranteed to be smaller than sizeof(object).

I'd like to know how systems that use this method continue to work
in the presence of a user-supplied operator new[] that provides only
what the standard requires.

--
Steve Clamage, stephen.clamage@sun.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Anders Pytte <anders@milkweed.com> wrote:
>> I read in other posts, however, that operator new[] was not really
>> needed by the runtime to distinguish, and was only added so that
>> users could define different allocation strategies for arrays. I'm
>> not sure how the runtime would keep track of the array size without
>> the differentiation, unless it stored a count for non-array
>> allocations as well.

Stephen Clamage wrote:
> ...
> I know of two methods in common use for keeping track of the number of
> elements in an allocated array.
> ...
> 2. For any array allocation, the new-expression requests a little
> extra memory from the allocator, stores the count in that space, and
> adjusts the returned value to point past the count. An array delete-
> expression decrements the pointer value to point to the start of the
> extra memory, retrieves the element count, calls destructors, and
> passes the adjusted address to the deallocator function (operator
> delete[]).
>
> ...
> The second method is straightforward and a little easier to implement.
> Although there is no "wasted" space for the associative array, systems
> with large alignment requirements will use up more than the space
> needed for the count, in order to retain alignment; the extra space is
> always allocated, even for types that don't have destructors to run.

As has been pointed out in this newsgroup many months ago, it is not
strictly necessary to allocate extra space for the array count;
dividing the total size of the allocated block (known at runtime) by
the size of the element type (known at compile time) will result in
the proper array element count.  (I believe that many systems
actually do it this way.)

The only trouble that can occur is if the delete[] is coded with
the wrong type information:

    class Base { ... };
    class Der: public Base { ... };

    static Der * getIt(int n)
    {
        return (new Der[n]);    // Allocate array of Der objects
    }

    static void freeIt(Base *a)
    {
        delete[] a;             // Delete array of Base objects
    }

    int main()
    {
        freeIt(getIt(100));     // Oops
        return 0;
    }

But this is bad programming anyway, since it treats an array of one
type (Der) as an array of a different type (Base), which is never
correct.

--
David R. Tribble, mailto:david@tribble.com, http://david.tribble.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

<wmm@fastdial.net> wrote in message
news:8m4pjo$ghf$1@nnrp1.deja.com...
[about delete actually misteriously knowing the size of the object
being deleted]
> Right -- I tend to forget about that because I've never
> needed it, but the capability is there.

The only good use I found for it is exactly in special-purpose
allocators: because you know the size of the chunk being deallocated,
you can use a fast algorithm instead of a slow one.

Interestingly enough, implementing this knowledge is quite hard for
compiler writers. Basically they must count on a virtual destructor
and have that destructor return the actual size of the object.

Which kind of explains why the destructor must be virtual if you want
to delete objects of derived types; otherwise, operator delete doesn't
know the size to deallocate. It all comes nicely together.


Andrei


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Anders Pytte wrote:

> in article 397F4783.7524D636@aspi.net, Trevor L. Jackson, III at
> fullmoon@aspi.net wrote on 7/28/00 1:31 AM:
>
> > Anders Pytte wrote:
> >
> >> in article 8lld0r$ieu$1@nnrp1.deja.com, Marcus Barnes at
> >> marcus_aurelius@my-deja.com wrote on 7/27/00 5:06 AM:
> >>
> >>> C++ decouples allocation from initialization such that allocation is
> >>> very constrained. Class static operator new and delete seem to share
> >>> similar limitations.
> >>>
> >>> I've oftened wondered why a class can have freestore operators that deal
> >>> only in bytes? Classes are descriptions of objects not raw memory.
> >>
> >> Because initialization of storage is the responsibility of constructors?
> >>
> >> I think the purpose of operator new and delete is to manage the raw storage.
> >
> > Then why do we care about operator new() versus operator new[]()?  The
> > distinction has to do with object boundaries not raw storage.
>
> I think this is because in the array case, the runtime system must remember
> the number of elements, in order to call the constructors for all of the
> elements.
>
> I don't know how this is typically implemented - I wish I did.
>
> The [] is thus a hint to the runtime system (and it also makes sure the
> correct version of delete is called, in case there is a difference - the
> default version of delete[] just calls delete).
>
> The operator itself has nothing to do with destruction, however.
>
> At least, I think this is how things work.

My understanding of the process matches yours, but I believe it to be partitioned
differently.  The function operator new[]() has nothing to do with calling
constructors or destructors.  The compiler arranges that magically.  The compiler
also manages the multiplication of the number of array elements by the size of each
element in order to come up with the subtotal of memory required.  It is free to
add to this subtotal whatever it needs in order to provide space for the
destruction count.  It can store pointers to dtors or delete functions in addition
to the count.  None of that affects the function operator new[]().

Note that operator new[]()'s interface is identical to operator new()'s interface.
The only difference is the name.  Now one of the thumb rules of good interface
design is that one does not provide extra methods merely to provide aliases.  The
function operator new[]() is an alias for the operator new() function in almost
every conceivable circumstance.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

In article <3981C7F3.C3C11DF0@aspi.net>,
  "Trevor L. Jackson, III" <fullmoon@aspi.net> wrote:
> My understanding of the process matches yours, but I believe it to be
partitioned
> differently.  The function operator new[]() has nothing to do with
calling
> constructors or destructors.  The compiler arranges that magically.
The compiler
> also manages the multiplication of the number of array elements by
the size of each
> element in order to come up with the subtotal of memory required.  It
is free to
> add to this subtotal whatever it needs in order to provide space for
the
> destruction count.  It can store pointers to dtors or delete
functions in addition
> to the count.  None of that affects the function operator new[]().

This is correct.  The allocation functions, operator new and
operator new[], are strictly for allocating storage.

> Note that operator new[]()'s interface is identical to operator new
()'s interface.
> The only difference is the name.  Now one of the thumb rules of good
interface
> design is that one does not provide extra methods merely to provide
aliases.  The
> function operator new[]() is an alias for the operator new() function
in almost
> every conceivable circumstance.

Here I disagree.  I think probably the majority of member
function operator new/operator delete are for doing things
like managing a pool of freed objects to avoid storage
allocation/freeing overhead in cases where there are a
large number of new/free cycles but only a small number of
objects in use at any given time.  This is clearly not a
suitable application for operator new[].  If for some
reason you also wanted to allocate an array of these
objects, you would obviously _not_ want to alias
operator new[] onto operator new.

--
William M. Miller, wmm@fastdial.net
Vignette Corporation (www.vignette.com)


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

"Trevor L. Jackson, III" wrote:
>
> Anders Pytte wrote:
>
> > in article 8lld0r$ieu$1@nnrp1.deja.com, Marcus Barnes at
> > marcus_aurelius@my-deja.com wrote on 7/27/00 5:06 AM:
> >
> > > C++ decouples allocation from initialization such that allocation is
> > > very constrained. Class static operator new and delete seem to share
> > > similar limitations.
> > >
> > > I've oftened wondered why a class can have freestore operators that deal
> > > only in bytes? Classes are descriptions of objects not raw memory.
> >
> > Because initialization of storage is the responsibility of constructors?
> >
> > I think the purpose of operator new and delete is to manage the raw storage.
>
> Then why do we care about operator new() versus operator new[]()?  The
> distinction has to do with object boundaries not raw storage.

As you said originally, allocation is distinct from initialization.
If you don't want to use the default allocator, you can write your
own. A class can have its own allocator for objects of its type.

The semantics of a new-expression are
1. allocate space;
2. if the allocation succeeds and there is an initializer, initialize.

A constructor applies to all objects of the class type, whether
allocated on the heap via a new-expression, or allocated elsewhere via
a variable definition.

An allocator has one responsibility: provide storage space if possible.
A constructor has one responsibilty: initialize the object for which
space has been allocated.

This othogonality keeps everything simpler. Imagine having to keep
track of which constructor to use depending on the kind of object.
And then you'd need a way to call the matching destructor.

Operator new() and operator new[]() became differentiated at the
request of C++ users who wanted the capability to specify different
memory pools for arrays, presumed to be large, and single objects,
presumed to be smaller.

As a related issue, you must match delete with new, and delete[]
with new[], because new and new[] might be unrelated. You must
return memory to the pool from which it was obtained.

--
Steve Clamage, stephen.clamage@sun.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

in article 3980CB78.6323906F@wizard.net, James Kuyper at kuyper@wizard.net
wrote on 7/29/00 5:11 AM:

> Anders Pytte wrote:

>> I think this is because in the array case, the runtime system must remember
>> the number of elements, in order to call the constructors for all of the
>
> I assume you mean destructors?

Indeed I did.

>> elements.
>>
>> I don't know how this is typically implemented - I wish I did.
>
> The typical implementation is to allocate slightly more space than is
> actually needed, put the number of elements in that extra space at the
> beginning of the allocated memory, and then to offset the value returned
> by operator new[]() by the amount of that extra space. This means that
> the amount of extra space must be an integer multiple of the alignment
> requirement.
> Operator delete then subtracts that offset from the pointer passed to
> it, and retrieves the count from that location. It uses that count to
> determine how many dtor calls are needed. After that, it calls operator
> delete() to actually deallocate the memory.

Ahah! I suspected something like that.

I read in other posts, however, that operator new[] was not really needed by
the runtime to distinguish, and was only added so that users could define
different allocation strategies for arrays. I'm not sure how the runtime
would keep track of the array size without the differentiation, unless it
stored a count for non-array allocations as well.

Can you clue me in?

Thanks,

Anders.

--
Anders Pytte                                   Milkweed Software
PO Box 32                                  voice: (802) 586-2545
Craftsbury, VT 05826                  email: anders@milkweed.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

wmm@fastdial.net wrote:

> In article <3981C7F3.C3C11DF0@aspi.net>,
>   "Trevor L. Jackson, III" <fullmoon@aspi.net> wrote:
> > Note that operator new[]()'s interface is identical to operator new
> ()'s interface.
> > The only difference is the name.  Now one of the thumb rules of good
> interface
> > design is that one does not provide extra methods merely to provide
> aliases.  The
> > function operator new[]() is an alias for the operator new() function
> in almost
> > every conceivable circumstance.
>
> Here I disagree.  I think probably the majority of member
> function operator new/operator delete are for doing things
> like managing a pool of freed objects to avoid storage
> allocation/freeing overhead in cases where there are a
> large number of new/free cycles but only a small number of
> objects in use at any given time.

This is a reasonable supposition, but it does not relieve the author of
operator new(size_t) from handling an arbitrary allocation.  I.e., it is not
reasonable to assume the size is predefined unless the class is private, which
is quite rare AFAIK.

>  This is clearly not a
> suitable application for operator new[].  If for some
> reason you also wanted to allocate an array of these
> objects, you would obviously _not_ want to alias
> operator new[] onto operator new.

OK I'll bite.  _Why_ would you not want to use the same function for operator
new() as operator new[]()?


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Steve Clamage wrote:

> "Trevor L. Jackson, III" wrote:
> >
> > Anders Pytte wrote:
> >
> > > in article 8lld0r$ieu$1@nnrp1.deja.com, Marcus Barnes at
> > > marcus_aurelius@my-deja.com wrote on 7/27/00 5:06 AM:
> > >
> > > > C++ decouples allocation from initialization such that allocation is
> > > > very constrained. Class static operator new and delete seem to share
> > > > similar limitations.
> > > >
> > > > I've oftened wondered why a class can have freestore operators that deal
> > > > only in bytes? Classes are descriptions of objects not raw memory.
> > >
> > > Because initialization of storage is the responsibility of constructors?
> > >
> > > I think the purpose of operator new and delete is to manage the raw storage.
> >
> > Then why do we care about operator new() versus operator new[]()?  The
> > distinction has to do with object boundaries not raw storage.
>
> As you said originally, allocation is distinct from initialization.
> If you don't want to use the default allocator, you can write your
> own. A class can have its own allocator for objects of its type.
>
> The semantics of a new-expression are
> 1. allocate space;
> 2. if the allocation succeeds and there is an initializer, initialize.
>
> A constructor applies to all objects of the class type, whether
> allocated on the heap via a new-expression, or allocated elsewhere via
> a variable definition.
>
> An allocator has one responsibility: provide storage space if possible.
> A constructor has one responsibilty: initialize the object for which
> space has been allocated.
>
> This othogonality keeps everything simpler. Imagine having to keep
> track of which constructor to use depending on the kind of object.
> And then you'd need a way to call the matching destructor.

All of the above makes sense, but is irrelevant.

> Operator new() and operator new[]() became differentiated at the
> request of C++ users who wanted the capability to specify different
> memory pools for arrays, presumed to be large, and single objects,
> presumed to be smaller.

This consideration would justify the presence of an if at the start of the
allocation function.  Why does it justify the proliferation of heap interfaces?
AFAICS it is only a trivial optimization, but creates serious correctness issues by
creating opportunities for errors.  This kind of thing should be designed _out_ of
the language rather than enshrined within it.

>
>
> As a related issue, you must match delete with new, and delete[]
> with new[], because new and new[] might be unrelated. You must
> return memory to the pool from which it was obtained.

Which is a good reason to avoid proliferation of types of memory allocation.  It is
_not_ a sane reason to differentiate otherwise identical methods.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

In article <398239FF.8577644E@aspi.net>,
  "Trevor L. Jackson, III" <fullmoon@aspi.net> wrote:
> wmm@fastdial.net wrote:
>
> > In article <3981C7F3.C3C11DF0@aspi.net>,
> >   "Trevor L. Jackson, III" <fullmoon@aspi.net> wrote:
> > > Note that operator new[]()'s interface is identical to operator
new
> > ()'s interface.
> > > The only difference is the name.  Now one of the thumb rules of
good
> > interface
> > > design is that one does not provide extra methods merely to
provide
> > aliases.  The
> > > function operator new[]() is an alias for the operator new()
function
> > in almost
> > > every conceivable circumstance.
> >
> > Here I disagree.  I think probably the majority of member
> > function operator new/operator delete are for doing things
> > like managing a pool of freed objects to avoid storage
> > allocation/freeing overhead in cases where there are a
> > large number of new/free cycles but only a small number of
> > objects in use at any given time.
>
> This is a reasonable supposition, but it does not relieve the author
of
> operator new(size_t) from handling an arbitrary allocation.  I.e., it
is not
> reasonable to assume the size is predefined unless the class is
private, which
> is quite rare AFAIK.
>
> >  This is clearly not a
> > suitable application for operator new[].  If for some
> > reason you also wanted to allocate an array of these
> > objects, you would obviously _not_ want to alias
> > operator new[] onto operator new.
>
> OK I'll bite.  _Why_ would you not want to use the same function for
operator
> new() as operator new[]()?

Your assumptions are not my assumptions.  You (appear to)
assume that every (non-private) class is a candidate for
derivation.  I don't see the need to assume that, even
though C++ doesn't have a "final" qualifier like Java.
I find lots of applications for simple little data
structures that are intended to be standalone, not the
root of an inheritance hierarchy.  Often these little
data structures are allocated and freed very frequently,
in which case it can make a big difference in performance
to keep a free pool and reuse them once allocated rather
than using the system storage management routines.

The typical implementation of this is to have the class's
operator delete chain the freed objects onto a free list;
the class's operator new first looks on the free list and
returns one of the free objects if there are any and only
allocates a new object if the free list is empty.

The class's operator delete cannot tell the size of the
object being freed; it can only put whatever address it
is given onto the free list.  Similarly, operator new
can't tell the size of something on the free list, it
must assume that it's big enough to hold the object being
created.  Happily, there are lots of applications where
these constraints are no problem at all: the object
can't be a derived object because by design there's no
inheritance from the class, and the language guarantees
that array allocations, if ever one were performed, will
be handled elsewhere.

There's no room in this kind of arrangement for "handling
an arbitrary allocation," and it would be needless
over-generalization to put support for it into the
operator new code, especially since the corresponding
operator delete can't deal with the result.

It may be that your environment is such that every class
you write must be fully general and available for
derivation.  If so, you're right -- any operator new you
wrote would need to be prepared for arbitrary sizes.
And if you have no need to distinguish between array
allocations and non-array allocations, fine -- it's
trivial to write an inline operator new[] that simply
forwards to operator new (or vice versa).

Not all code has those characteristics.

--
William M. Miller, wmm@fastdial.net
Vignette Corporation (www.vignette.com)


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

<wmm@fastdial.net> wrote in message
news:8m4719$126$1@nnrp1.deja.com...
> The class's operator delete cannot tell the size of the
> object being freed;

Yes it does. Just add a second size_t parameter to class' operator
delete:

class Something
{
    ...
    void operator delete(void* data, size_t dataSize)
    {
        ...
    }
};


Andrei


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

In article <sobjr4ra9ft76@news.supernews.com>,
  "Andrei Alexandrescu" <andrewalex@hotmail.com> wrote:
> <wmm@fastdial.net> wrote in message
> news:8m4719$126$1@nnrp1.deja.com...
> > The class's operator delete cannot tell the size of the
> > object being freed;
>
> Yes it does. Just add a second size_t parameter to class' operator
> delete:
>
> class Something
> {
>     ...
>     void operator delete(void* data, size_t dataSize)
>     {
>         ...
>     }
> };

Right -- I tend to forget about that because I've never
needed it, but the capability is there.  Thanks for the
reminder.

--
William M. Miller, wmm@fastdial.net
Vignette Corporation (www.vignette.com)


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

C++ decouples allocation from initialization such that allocation is
very constrained. Class static operator new and delete seem to share
similar limitations.

I've oftened wondered why a class can have freestore operators that deal
only in bytes? Classes are descriptions of objects not raw memory.

--
Regards, Marcus


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

in article 8lld0r$ieu$1@nnrp1.deja.com, Marcus Barnes at
marcus_aurelius@my-deja.com wrote on 7/27/00 5:06 AM:

> C++ decouples allocation from initialization such that allocation is
> very constrained. Class static operator new and delete seem to share
> similar limitations.
>
> I've oftened wondered why a class can have freestore operators that deal
> only in bytes? Classes are descriptions of objects not raw memory.

Because initialization of storage is the responsibility of constructors?

I think the purpose of operator new and delete is to manage the raw storage.

Anders.


--
Anders Pytte                                   Milkweed Software
PO Box 32                                  voice: (802) 586-2545
Craftsbury, VT 05826                  email: anders@milkweed.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Anders Pytte wrote:

> in article 8lld0r$ieu$1@nnrp1.deja.com, Marcus Barnes at
> marcus_aurelius@my-deja.com wrote on 7/27/00 5:06 AM:
>
> > C++ decouples allocation from initialization such that allocation is
> > very constrained. Class static operator new and delete seem to share
> > similar limitations.
> >
> > I've oftened wondered why a class can have freestore operators that deal
> > only in bytes? Classes are descriptions of objects not raw memory.
>
> Because initialization of storage is the responsibility of constructors?
>
> I think the purpose of operator new and delete is to manage the raw storage.

Then why do we care about operator new() versus operator new[]()?  The
distinction has to do with object boundaries not raw storage.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

in article 397F4783.7524D636@aspi.net, Trevor L. Jackson, III at
fullmoon@aspi.net wrote on 7/28/00 1:31 AM:

> Anders Pytte wrote:
>
>> in article 8lld0r$ieu$1@nnrp1.deja.com, Marcus Barnes at
>> marcus_aurelius@my-deja.com wrote on 7/27/00 5:06 AM:
>>
>>> C++ decouples allocation from initialization such that allocation is
>>> very constrained. Class static operator new and delete seem to share
>>> similar limitations.
>>>
>>> I've oftened wondered why a class can have freestore operators that deal
>>> only in bytes? Classes are descriptions of objects not raw memory.
>>
>> Because initialization of storage is the responsibility of constructors?
>>
>> I think the purpose of operator new and delete is to manage the raw storage.
>
> Then why do we care about operator new() versus operator new[]()?  The
> distinction has to do with object boundaries not raw storage.

I think this is because in the array case, the runtime system must remember
the number of elements, in order to call the constructors for all of the
elements.

I don't know how this is typically implemented - I wish I did.

The [] is thus a hint to the runtime system (and it also makes sure the
correct version of delete is called, in case there is a difference - the
default version of delete[] just calls delete).

The operator itself has nothing to do with destruction, however.

At least, I think this is how things work.

Anders.

--
Anders Pytte                                   Milkweed Software
PO Box 32                                  voice: (802) 586-2545
Craftsbury, VT 05826                  email: anders@milkweed.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

Anders Pytte wrote:
>
> in article 397F4783.7524D636@aspi.net, Trevor L. Jackson, III at
> fullmoon@aspi.net wrote on 7/28/00 1:31 AM:
....
> > Then why do we care about operator new() versus operator new[]()?  The
> > distinction has to do with object boundaries not raw storage.
>
> I think this is because in the array case, the runtime system must remember
> the number of elements, in order to call the constructors for all of the

I assume you mean destructors?

> elements.
>
> I don't know how this is typically implemented - I wish I did.

The typical implementation is to allocate slightly more space than is
actually needed, put the number of elements in that extra space at the
beginning of the allocated memory, and then to offset the value returned
by operator new[]() by the amount of that extra space. This means that
the amount of extra space must be an integer multiple of the alignment
requirement.
Operator delete then subtracts that offset from the pointer passed to
it, and retrieves the count from that location. It uses that count to
determine how many dtor calls are needed. After that, it calls operator
delete() to actually deallocate the memory.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

On Tue, 1 Aug 2000 00:24:42 CST, Anders Pytte <anders@milkweed.com>
wrote:

>I read in other posts, however, that operator new[] was not really needed by
>the runtime to distinguish, and was only added so that users could define
>different allocation strategies for arrays. I'm not sure how the runtime
>would keep track of the array size without the differentiation, unless it
>stored a count for non-array allocations as well.
>
>Can you clue me in?

The difficulty that recurs in these discussions is the confusion
between the language construct known as a new-expression, and
functions whose name is operator new or operator new[].

An array version of a new-expression, such as
 T* p = new T[10];
results in a call to the appropriate version of operator new[] to
acquire enough space for the array, followed by construction of the 10
objects of type T.  The runtime system is responsible for remembering
that 10 objects were constructed, so that an expression like
 delete [] p;
results in the 10 objects being destroyed.

The various versions of operator new[] are responsible only for
providing the amount of requested memory, or reporting that the memory
could not be allocated. These allocators need not know the type or
number of objects being allocated, and indeed in the general case
cannot find out the type or number; that information is not available
in the interface to the global allocators. Thus, the operator new[]
functions are not needed to implement the rules about creating and
destroying arrays, and did not even exist for many years.

I know of two methods in common use for keeping track of the number of
elements in an allocated array.

1. The run-time system needs to keep track of only of arrays allocated
on the heap by a new-expression, and then only for types with a
non-trivial destructor. For those cases, the run-time system keeps an
associative array of allocation addresses with element count. For each
array delete expression (but not single-object delete-expressions),
the address is looked up in the array, and if it is there, the
destructors are called. Doing so involves programming contortions not
ordinarily available, but these routines are written by the compiler
implementor according to the exact way the specific implementation
works.

2. For any array allocation, the new-expression requests a little
extra memory from the allocator, stores the count in that space, and
adjusts the returned value to point past the count. An array delete-
expression decrements the pointer value to point to the start of the
extra memory, retrieves the element count, calls destructors, and
passes the adjusted address to the deallocator function (operator
delete[]).

Each method has its advantages.

The first method can clear the associative array element on delete.
Attempting to delete via an incorrect address, or deleting twice, can
be detected for these kinds of arrays. Programming errors that write
beyond array bounds will not clobber the array count, although the
errors usually cause other serious problems anyway. Extra space is
allocated only for array types that need it. The extra time to search
the associative array is ordinarily not measurable, given that you
will call a bunch of destructors and return the allocated memory.

The second method is straightforward and a little easier to implement.
Although there is no "wasted" space for the associative array, systems
with large alignment requirements will use up more than the space
needed for the count, in order to retain alignment; the extra space is
always allocated, even for types that don't have destructors to run.
On some popular systems, 12 bytes per array allocation are wasted,
when a 4-byte count is stored in a 16-byte alignment unit.

---
Steve Clamage, stephen.clamage@sun.com

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

From: "Trevor L. Jackson, III" <fullmoon@aspi.net>
> It appears that the array allocation methods were created not to support
array
> allocation but to protect legacy (mis)use of the original allocation
methods
> by ignoring the size parameter as you described above.

As I said earlier in this thread, the legacy code issue would have
been significant if anyone had seriously proposed using
X::operator new to allocate arrays of X.  To my recollection, no
one on the Committee proposed that at the time.  As far as I recall,
everyone agreed with the position I quoted earlier from the ARM:
X::operator new is intended to allocate storage for objects of type
X (and types derived therefrom, if not hidden by a more-derived
operator new).  An array of X is not an object of type X.  It's
therefore inappropriate to use X::operator new to allocate storage
for an array of X.

This was the basis on which a separate allocation function for
arrays was created, not the legacy code issue, although the
weight of the legacy code that had been written relying on this
philosophical principle certainly would have exerted an influence
against changing that philosophy, had we been otherwise inclined to
do so.

> The question I posed was "why not use new() for everything".  Your answer
> appears to be "to support free lists and array allocation".  Does your
answer
> imply that you do not believe that simple free lists and array allocation
can
> be implemented safely with operator new()?

No.  I used the free list example to illustrate my contention that
there are cases where the allocation and freeing of individual
objects needs to be done differently from allocation and freeing
of arrays.  The fact that it could be handled in a single allocation
function does not, to my mind, constitute a persuasive argument
that every allocation function should be burdened by the language
to handle both kinds of allocation.

In the part of your reply I snipped out above, you wrote:

> > And if you have no need to distinguish between array
> > allocations and non-array allocations, fine -- it's
> > trivial to write an inline operator new[] that simply
> > forwards to operator new (or vice versa).
>
> I think this is a reply to an argument I'm not making.  It is not my claim
> that I do not need to distinguish between array allocations and nonarray
> allocations.  On that side topic it is my belief that no one needs
separate
> array allocation methods.  But that is not the central issue.

My point in the free list example was that I'm perfectly happy with
the default handling of array allocations; all I need to change is
the handling of allocating individual objects.  The separation of
scalar and array allocation may not be "needed," but it is quite
convenient.

However, the real reason not to use operator new for everything is
not primarily that I can write a simpler operator new but the
philosophical principle I mentioned above: an array of X is not an
X.

-- William M. Miller


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

"Stephen Clamage" <stephen.clamage@sun.com> wrote
[snip]
> 2. For any array allocation, the new-expression requests a little
> extra memory from the allocator, stores the count in that space, and
> adjusts the returned value to point past the count. An array delete-
> expression decrements the pointer value to point to the start of the
> extra memory, retrieves the element count, calls destructors, and
> passes the adjusted address to the deallocator function (operator
> delete[]).

[snip]

>  the extra space is
> always allocated, even for types that don't have destructors to run.

Is that strictly necessary?  If the destructor is trivial you don't care how
many objects there are.  If the eventual deallocation function doesn't take
a second size_t argument the implementation won't need to remember what size
was passed to new[].  At worst the arguments needed for the eventual
deallocation function are known when the program is linked (assuming a
conventional compile/link architecture).  If the deallocation function is a
member, its arguments* are known at compile time at the point where new[] is
called.

* Not the value of delete[]'s argument(s), but whether or not delete[] takes
a size_t argument.


---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]

wmm@fastdial.net wrote:

> In article <398239FF.8577644E@aspi.net>,
>   "Trevor L. Jackson, III" <fullmoon@aspi.net> wrote:
>
> > OK I'll bite.  _Why_ would you not want to use the same function for
> operator
> > new() as operator new[]()?
>
> Your assumptions are not my assumptions.  You (appear to)
> assume that every (non-private) class is a candidate for
> derivation.

Yes, within the life cycle of the software.  It appears to me to be a bad idea
to embed the contrary assumption within the design, especially given that the
alternative, "safe heap", is simple.  I believe Meyers calls this "programming
in the future tense".

>
> I don't see the need to assume that, even though C++ doesn't have a "final"
> qualifier like Java.
> I find lots of applications for simple little data
> structures that are intended to be standalone, not the
> root of an inheritance hierarchy.  Often these little
> data structures are allocated and freed very frequently,
> in which case it can make a big difference in performance
> to keep a free pool and reuse them once allocated rather
> than using the system storage management routines.
>
> The typical implementation of this is to have the class's
> operator delete chain the freed objects onto a free list;
> the class's operator new first looks on the free list and
> returns one of the free objects if there are any and only
> allocates a new object if the free list is empty.

Yes.  This is a very effective, and (should be) common technique.


> The class's operator delete cannot tell the size of the object being freed;
> it can only put whatever address it
> is given onto the free list.

I think this assertion is false.  Given that operator delete(void*,size_t) is
a proper superset of operator delete(void*), basing a design on the former is
unnecessary.
If T::operator new(void) existed it would be a sensible counterpart to
T::operator delete(void*) -- the compiler could guarantee that they were only
called with size ==sizeof(T).
But we don't have T::operator new(void), nor do we have the related guarantee.

>  Similarly, operator new
> can't tell the size of something on the free list, it
> must assume that it's big enough to hold the object being
> created.  Happily, there are lots of applications where
> these constraints are no problem at all: the object
> can't be a derived object because by design there's no
> inheritance from the class, and the language guarantees
> that array allocations, if ever one were performed, will
> be handled elsewhere.

The assertion that "by design there's no inheritance from the class" is a
reasonable one to make iff it can be enforced by the compiler.  AFAIK it
can't.  So it, too, is an unreasonable basis for a design.

Note also that there is no need to determine the size of objects on the free
list.  It should be comprised of objects of the same size.  Any object of a
differing size can simply be deleted when it is delete()'d. (sorry)

> There's no room in this kind of arrangement for "handling
> an arbitrary allocation," and it would be needless
> over-generalization to put support for it into the
> operator new code, especially since the corresponding
> operator delete can't deal with the result.

There would be "no room" for this if operator new(void) existed.  It does
not.  Treating operator new(size_t) as if it were operator new(void) is
another questionable basis for a design.  IMHO an obviously bad one.  Not one
that the language should encourage.

>
>
> It may be that your environment is such that every class
> you write must be fully general and available for
> derivation.

Of course not.  My experience re inheritance hierarchies is similar to yours.
The issue isn't usually singly rooted hierarchies versus forests, but the vast
numbers of metaphorical shrubs and grasses.

However, software evolves (devolves is probably a more accurate term) during
its lifetime.  The favored change is not a radical enhancement of fundamental
capabilities, but an elaboration of existing capabilities.  Derivation is
likely to be applied in the most unlikely places.  C.f., Murphy applied to
as-shipped design.

>
> If so, you're right -- any operator new youwrote would need to be prepared
> for arbitrary sizes.
> And if you have no need to distinguish between array
> allocations and non-array allocations, fine -- it's
> trivial to write an inline operator new[] that simply
> forwards to operator new (or vice versa).

I think this is a reply to an argument I'm not making.  It is not my claim
that I do not need to distinguish between array allocations and nonarray
allocations.  On that side topic it is my belief that no one needs separate
array allocation methods.  But that is not the central issue.

My argument is that the basic allocation and deallocation methods have
sufficient information to support efficient array allocation -- that special
array methods are unnecessary.  Consider an implementation as you described
above, but that also supports array allocation.  The allocation method for
such a class would resemble:

    static void * operator new(size_t size)
    {
    if ( size == sizeof( *this ) ) {
        /* allocate from free list */
    } else {
        /* allocate from heap */
    }

Now please consider the else clause.  Your argument above was that it is
unnecessary in most cases.  Stipulating that observation does not lead to the
conclusion that the else clause should be empty, but that it should complain.
Assert()s are cheap.

It appears that the array allocation methods were created not to support array
allocation but to protect legacy (mis)use of the original allocation methods
by ignoring the size parameter as you described above.

The question I posed was "why not use new() for everything".  Your answer
appears to be "to support free lists and array allocation".  Does your answer
imply that you do not believe that simple free lists and array allocation can
be implemented safely with operator new()?

> Not all code has those characteristics.

All of the code we write should have the characteristics that lead to
robustness.  Elaborating the language in stilted ways to protect and
_encourage_ bad implementations is not appropriate.



---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://reality.sgi.com/austern_mti/std-c++/faq.html              ]