Earlier on this group I had a topic on C++`s memory management
mechanism which was not approved due to my lazy fingers who try to get
rid of typing. I was trying to device reference counting in a manor
that it would not need twice dereferencing. Let me first describe a
generalized version of ref-counting that I saw in an old fashioned
book a few years ago when I was just stepping into the dream land of C+
+ programming:

template < class TData >
struct smart_ref{
 typedef unsigned long cnt_type;
 typedef TData Value_type;

 smart_ref(Tdata * pd):ptr(new node(pd)){if(!ptr)throw "insufficient
memory";};
 smart_ref(smart_ref& r):ptr(r.ptr){ptr->count++;};

 smart_ref& operator=(smart_ref& r){replace(r.ptr);ptr->count++;return
*this;};

 ~smart_ref(){replace(NULL);};

 TData& operator*()const{return *pointer();};
 TData* operator->()const{return pointer();};

 TData* pointer()const{return ptr->data;};
 cnt_type ref_count()const{return ptr->count;};

private:

 void replace(node *in){
  if(ptr=in) retrun;
  if(!ptr->count--) delete ptr;
  ptr=in;
 };//void replace(node *in);

 node * ptr;

 struct node{

  node(value_type * ptr):data(ptr),count(0){};
  ~node(){delete data;};

  value_type * data;
  cnt_type count;

 };//struct node;

};//template < class TData > struct smart_ref;

static smart_ref<int> si=new int(0);

I guess that underlying implementation of 'shared_ptr' more or less
contains something with similar structure as this 'smart_ptr'(it is
obviously much more complicated).in order to access '*si' ,according
to above code twice dereferencing is accomplished :I- extract 'si.ptr-
>data' and II- dereference 'data'.In order to eliminate this run time
overhead we can place  a 'value_type * val;' inside 'smart_ptr' which
is always supposed to equal 'ptr->data' ;but that adds a linear memory
overhead proportional to the number of smart pointers constructed.
Note that in order to initialize 'node::data' neither 'smart_ptr' nor
its 'node' does invoke 'new'.It is left to instantiation time by the
coder just as we see with 'si' because we want the interface to be
flexible and give the user the chance to choose between different
constructors for a any dynamic object . But aside from the overhead
for twice dereferencing there is always a risk of accidental transfer
of the address of a none dynamic object to 'smar_ptr' which could be
some sort of crash when 'delete'd .We can use something like the
following code for reference-counting(or any sort of smartly pointing
to heap) with just one level of dereferencing in an almost overhead-
free manor(as fine as built-in pointers!!)but we lose the capability
of selective construction:

template<typename TData>
struct fast_ref{
 typedef unsigned long cnt_type;
 typedef TData Value_type;

 fast_ref(const value_type& r):ptr(new Buffer(r)){if(!ptr)throw
"insufficient memory";};
 fast_ref(fast_ref& r):ptr(r.ptr){ptr->count++;};

 fast_ref& operator=(fast_ref& r){replace(r.ptr);ptr->count++;return
*this;};

 ~fast_ref(){replace(NULL);};

 TData& operator*()const{return reference();};
 TData* operator->()const{return &reference();};

 TData& reference()const{return ptr.data;};
 cnt_type ref_count()const{return ptr->count;};

private:

 void replace(Buffer *in){
  if(ptr=in) retrun;
  if(!ptr->count--) delete ptr;
  ptr=in;
 };//void replace(Buffer *in);

 Buffer * ptr;

 struct Buffer{

  Buffer(const value_type & ref):data(ref),count(0){};

  value_type  data;
  cnt_type count;

 };//struct Buffer;

};//template<typename TData> struct fast_ref;

Maybe I had to templatize 'fast_ref' and 'smart_ref' into one class
that takes an allocator ;since the only major difference between them
is in 'node' and 'buffer' that define the allocation mechanism.
However ,with 'fast_ref' and 'Buffer' we have to pay the expense of
twice construction overhead ;specifically if 'TData' is a really huge
type. That means we have to construct a temporary from which the
buffer copy(/move)-constructs the 'data'.This puts the unnecessary
limitation of copy/move-constructablity on the 'TData' as another
unpleasant side effect(we have the same problem with STL containers
too).

Another aproach; Using Variadic templates:

using varayadic templates (probably with refined forwarding and
hot(rvalue) references)we can decrease the use of 'new' keyword to the
default pointer placment form (used to initialize objects previously
allocated using custom algorithms):

template<typename TData>
struct quick_ref{

 typedef unsigned long cnt_type;
 typedef TData Value_type;

 quick_ref(quick_ref& r):ptr(r.ptr){ptr->count++;};

 template<typename... init_args>
  quick_ref(init_args&& vargs...):
   ptr(::new(my_alloc()) Buffer(vargs...))//void* new(size_t,void*)
  {if(!ptr)throw "insufficient memory";};

 quick_ref& operator=(quick_ref& r){replace(r.ptr);ptr->count++;return
*this;};

 ~quick_ref(){replace(NULL);};

 TData& operator*()const{return reference();};
 TData* operator->()const{return &reference();};

 TData& reference()const{return ptr.data;};
 cnt_type ref_count()const{return ptr->count;};

private:

 void * my_alloc(){return malloc(sizeof(Buffer));};

 void replace(Buffer *in){
  if(ptr=in) retrun;
  if(!ptr->count--) delete ptr;
  ptr=in;
 };//void replace(Buffer *in);

 Buffer * ptr;

 struct Buffer{

  template<typename... init_args>
  Buffer(init_args&& vargs...):data(vargs...),count(0){};

  value_type  data;
  cnt_type count;

 };//struct Buffer;

};//template<typename TData> struct quick_ref;

this solution looks semi-perfect .If concepts come into play we can
consider built-in types as exceptional cases .But both hot(rvalue)
references and variadic templates are new features which are under
discussion yet. Moreover we still need to push references onto the
stack when constructing 'fast_ptr' objects because references are
fully optimized only for inline functions since even using hot
(rvalue) references can not provide full forwarding for out of line
functions.

another solution would be embedding reference-counting(smart-pointing)
stuff in a placement form of 'new':

static const struct MEM_MANAGE{/*empty*/}mem_manage;

void *operator new(size_t sz,MEM_MANAGE/*unused*/){

 enum{My_Bytes=/*number of extra bytes needed for embeded smart-
pointing*/...};

 char * buf=(char *)malloc(sz + My_Bytes);

 return displace(buf,sz);/*initialize memory management bytes either
at the end or at the begining of 'buf' and return the point where data
stuff should be place*/

};


template<typename TData>
struct trick_ref{
 typedef unsigned long cnt_type;
 typedef TData Value_type;

 trick_ref(const value_type* r):ptr(static_cast<buffer*>(r)){if(!
ptr)throw "insufficient memory";};
 trick_ref(trick_ref& r):ptr(r.ptr){ptr->count++;};

 trick_ref& operator=(trick_ref& r){replace(r.ptr);ptr->count++;return
*this;};

 ~trick_ref(){replace(NULL);};

 TData& operator*()const{return reference();};
 TData* operator->()const{return &reference();};

 TData& reference()const{return ptr.data;};
 cnt_type ref_count()const{return ptr->count;};

private:

 void replace(Buffer *in){
  if(ptr=in) retrun;
  if(!ptr->count--) delete ptr;
  ptr=in;
 };//void replace(Buffer *in);

 Buffer * ptr;

 struct Buffer{

  value_type  data;
  cnt_type count;

 };//struct Buffer;

};//template<typename TData> struct trick_ref;

trick_ref<int> ti=new(mem_manage) int (0);

This approach can remove -if done exactly and carefully (forgetting
platform dependency of such code for a few secs for the sake of
simplification) - the overhead both for construction and dereferencing
but we are running on the edges!!. What if someone accidentally or by
mistake uses new without displacement ('mem_manage' parameter)? The
answer is disaster , crash , suicide!! What if he passes the result of
such a 'new' operator to something other than a 'trick_ref'? What if
he 'delete's the result of such a 'new' operator ?
would it not be better if we could specify a 'smart_ptr' as the return
type of 'operator new' in some way or other?
This was the basic idea of my older topic.My intention was that since
the 'new/delete' mechanism was added to C++ prior to GP paradigms, we
had better replace it with a new set of tools compatible with GP - at
least with the existing none-constraint templates.
Consider the following piece of code:

class A{
public:
 A(parameters){
  //constructor body
  . . .
       };
       //class stuff
       . . .
};

auto A a(arguments);//declare a as A and construct with arguments

The compiler replaces (symbolically) this with something like this:

A* A::ctor(void * void_this, parameters){
 A*const this=(A*)void_this;
 //constructor body
 . . .
 return this;
};

A& a = *A::ctor(auto_storage(),arguments);

Now consider this one:

A * aptr = new A(arguments);

The compiler will interpret this as follows:

A* aptr = A::ctor(adjust<A>(operator new(sizeof(A)
+reserved_bytes_number)),arguments);

Where:

template<T>
inline void* adjust(void *ptr){
 //puts some stuff at the reserved bytes in the location pointed to by
ptr
 return ptr + reserved_bytes_number;
};

The 'adjust' function and 'reserved_bytes_number' in the above context
are currently out of programmer control and platform dependent. The
programmer can only affect the 'operator new' and 'A::ctor'.
Providing a way for the programmer to handle the former two factors
can result in a much more flexible and elegant memory management
mechanism that can remove the unnecessary restriction of copy (/move)
constructability for the argument type of many generic classes
including STL containers.
IMHO the creation of a single dynamic object has the following four
steps:

I. Get a primary handle to a dynamic block of memory with sufficient
space for the object in the following steps(steps ii and iii can be
mixed):
 i. Determination of the number of bytes to be allocated based on the
type of object to be allocated.
 ii. Allocation of an appropriate block: This is the only step that
necessitates the use of low level codes (e.g. malloc function),
primary unsafe data (e.g. 'void*' pointers), and explicit type
castings and conversions (e.g. from 'void*' to other types).
 iii. Preparation of the block and placing run time stuff related to
clean up and garbage collection in it.
II. Extract the effective address of the object to be created from the
primary handle.
III. Construct the object at the effective address.
IV. Building a handle or smart pointer to the constructed object with
the following requirements:
 i. Access to the dynamic object.
 ii. Certain mechanism for destruction of the dynamic object it points
to.


Each of the above steps can be modeled as a function and each major
step can map to one of C++ constructs:
I. == allocation operator;
Primary handle is the bare handle or pointer to the allocated block.
II. == cast to (void*) operator,
III. == constructor of the allocated type,(ctor returns a normal
pointer to the type as described above but programmers are banned to
know this fact because CONSTRUCTOR SHOULD NOT DECLARE A RETURN
TYPE!!!)
IV. == constructor of the smart pointer that casts the primary
handle(result of I) to the smart pointer

The current style of using 'new' limits the programmer in many ways;
It forces the programmer to use result of step III but we already Know
that the result of step I is more valuable specially for the
optimization of memory usage. Among the stages of Step I currently the
programmer only controls I.ii which is really insufficient for a clean
and optimized -as well as type safe- memory management.
We can templatize 'operator new' so that it takes a compile-time type
parameter instead of an insufficient run-time 'size_t' one and allow
the programmer to manage the whole process of allocation and return a
templated version of the result of stage I to the programmer. This
approach gives us the ability to embed the reference-counting details
in the allocated block without generating any insecurity, or overhead
for selective construction of the object:

template <typename TData>
struct handle{
 //definition of memory block handle
 ...
private:
 operator void*();//This is necessary
friend handle operator new <TData>();//Let the memory management
mechanism cast handle to void*
};

template <typename TData>
struct block{
       //definition of a memory block
};

enum TAG{block_tag};

template <typename TData>
block<TData>* operator new(TAG/*unused*/);// block<TData>* is by
default castable to void*

template <typename TData>
handle<TData> operator new();//handle<TData> must be castable to void*

template <typename TData>
struct smart_ptr{
 smart_ptr(handle<TData>);
 smart_ptr(block<TData> *);
 TData & operator*();
 TData * operator->();
 //and so on
 ...
};

smart_ptr<int> sp_i=new int (0);//ok: call smart_ptr<int>(handle
<int>)
smart_ptr<int> sp_i=new(block_tag) int;//ok: smart_ptr<int>(block
<int> *)

int * iptr=new int;//ERROR: can not cast from 'handle<int>' to 'int*'
int * ibptr=new(block_tag) int;//ERROR: can not cast 'from
block<int>*' to 'int*'

template <typename TData>
struct wrong{
 //do not define 'operator void *()'
};

enum ERROR{error};

wrong <TData> operator new(ERROR /*unused*/);

wrong <int> ieptr1 = new(error) int;/* ok? :int does not need
constructors?? To be discussed.*/
wrong <int> ieptr2= new(error) int(0);/* ERROR: wrong<TData> is not
castable to void* and objects can not be constructed*/

For the case of array allocation (new[]) things are more complicated.
I want to extend the meaning of array from simple sequential vector to
any kind of container. The third major step(construction phase) of the
afformentioned proposal for different phases of memory management is
more complex here. We have to iterate through the container and
construct each object one by one. This article is going to be
compatible with sequential vector new operator that is we want to be
able to write:

struct A a1,a2,a3,a4;
new[] A {a1,a2,a3,a4};/*allocate an array of 4 'A's and copy/move-
construct them with a1...a4*/
new[10] A{a1,a2};/*allocate 10 'A's copy/move-constructing first and
second with a1 and a2 and default the rest*/
new[10] A(a1);/*allocate 10 'A's and copy/move-construct all with a1*/
new[10] A;/*allocate 10 'A's and default-construct all*/

So for array allocation the result of step I must be copy-move/
constructible incrementable and castable to void* and finishable:

template <typename TData>
struct array_handle{
private:
 operator void *();//REQUIRED: first element is constructed here
 array_handle operator++();//REQUIRED: next element is constructed
here
 bool finished();//REQUIRED to break the construction loop;

       //let memory management mechanism access REQUIRED fields:
       template <typename index>
  friend array_handle operator new<TData ,TIndex> (TIndex)[];
 //rest of the struct definition
 ...
};

template <typename TData>
smart_array{
 smart_array ( array_handle <TData>& );
 TData& operator[](unsigned);
 //and so on
 ...
};

template <typename TData ,typename TIndex>
array_handle <TData> operator new(TIndex size_parameter)[];

smart_array sa_i=new[10] int;

Note that the 'TIndex size_parameter' need not be integral and hence
can be templated. Its effect must be observed in the 'finished()'
member of 'array_handle' used to terminate the construction loop.
Making both ,the single and the vector form of 'new' behave like
existing dangerous strip-pointer-returning 'new' is easy; All that we
need ,is to replace the 'operator void*()' of 'handle' and
'array_handle' with 'operator TData *()'which also provides the
castability to void* (casting from strip pointers to void* is
implicit).
One other point is that for aggregates both the 'handle' and
'array_handle' can omit their REQUIRED fields as long as no
costruction parameters are prepared for the dynamic objects:

template<>struct handle<int>{
 //missing 'operator void*()'
 ...
};

template<>struct array_handle<int>{
 //missing everything
 ...
};

new int;//ok
new int[5];//ok

new int(0);//ERROR: missing 'operator void*()'
new int[](1);//ERROR: missing every thing

One can argue that we can achieve all this via variadic templates and
we need not change the semantics for 'new' keyword. I can just give
him a complete variadic library on how to do it but on the other hand
I can argue that we do not need 'new/delete' operators at all except
for the placement form of 'void *operator new(size_t,void*)' which
does not even need to be overloadable because I can simulate every
other feature of 'new/delete' using a combination of concepts and
vaiadic templates.
So if we are going to keep 'new' keyword as an operator in C++ we had
better have it in a more powerful form which can provide the compiler
with more flexibility.

Regards
FM

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]