Topic: defect report: vector reallocation and max_size are underspecified


Author: David Krauss <potswa@gmail.com>
Date: Fri, 27 Aug 2010 02:07:23 CST
Raw View
[ forwarded to the committee -- mod/jad ]

Discussion:

max_size in std::vector, like in all containers, is defined as "size()
of the largest possible container" (23.1, table 65), a requirement so
ambiguous as to often be flouted for most other containers. However,
unlike any other container in the Library, a vector's max_size can be
precisely defined by its allocator's max_size, and the value of
max_size actually has an effect: determining when std::length_error is
thrown from std::vector::reserve. This issue addresses the
inconsistency of that effect.

Explicitly using reserve to exceed a vector's max_size results in a
std::length_error exception (23.2.4.2/4). However, implicit
reallocations are not required to call reserve, and v.max_size() is
not explicitly required to match v.get_allocator().max_size().

As the size of an implicit reallocation is unspecified, populating a
vector with any number of objects may result in an exception thrown
from the allocator, despite the allocator providing information to
prevent any exception. As std::length_error is specified only for
explicit reallocation, the error condition of excessive requested
capacity may be signaled by either length_error or bad_alloc; it is
implementation-dependent. As the max_size values aren't explicitly
related, an implementation may avoid the requirement to use
length_error at all by returning std::numeric_limits<size_t>::max()
from max_size.

There is minimal performance cost to guaranteeing that only
length_error is thrown when the requested size or capacity exceeds
max_size, and max_size is not exceeded unnecessarily. As is,
vector::insert must catch exceptions from the copy constructor,
necessitating a try block. This try block may simply be extended to
include the attempt at allocation. The following catch block may
determine whether the allocator or the constructor failed by examining
the variable receiving the result of allocation. At this point, the
catch block can retry (loop) with max_size or clean up and/or rethrow.
Therefore, the runtime cost of the fix is disabling possible
optimization of the try block in the event that construction cannot
throw.

Proposed resolution:

Append the following to 23.2.4/1:

The contiguous storage may exceed the size of the sequence. Its size
shall be equal to the member capacity().

Insert the following as an additional paragraph in 23.2.4:

vector manages its storage by occasional <em>reallocation</em>. If and
only if capacity() must increase, new memory is allocated using
get_allocator(). If the capacity must increase above
get_allocator().max_size(), only a std::length_error exception may be
thrown. Otherwise an exception from the allocator may be propagated.
The sequence is then moved to the newly allocated memory by move
construction. If an exception is thrown other than by the move
constructor of a non-CopyConstructible type, there are no effects. The
previous storage is then destroyed and freed. Reallocation invalidates
all the references, pointers, and iterators referring to the elements
in the sequence.

Insert the following in 23.2.4.2:

size_type max_size() const;

Returns: get_allocator().max_size().

Finally, consider removing redundant language describing reallocation
from the descriptions of reserve and the modifiers.

--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@netlab.cs.rpi.edu]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: David Krauss <potswa@gmail.com>
Date: Fri, 27 Aug 2010 11:39:06 CST
Raw View
Also note: the exception behavior of reserve and the sequence
modifiers differ in the FCD, with reserve being much safer. I used
this stronger requirement above. The modifiers say "If an exception is
thrown other than by the copy constructor, move constructor,
assignment operator, or move assignment operator of T or by any
InputIterator operation there are no effects. If an exception is
thrown by the move constructor of a non-CopyConstructible T, the
effects are unspecified." The "no effects" guarantee would seem to
apply only to allocation exceptions. Although the result of a move
constructor exception is explicitly unspecified, the result of any of
the other exceptions not guaranteed to have no effects are implicitly
unspecified. Also, isn't "no effects" far stronger than saying the
sequence should end up as it started?

Finally, how can the existence of a copy constructor have an effect on
the handling of an exception during move construction? Does the FCD
require that reserve individually retry a move constructor failure by
copy construction? Yet almost no guarantees are made for insert,
emplace_back, etc?

See also http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html#985

--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use
mailto:std-c++@netlab.cs.rpi.edu<std-c%2B%2B@netlab.cs.rpi.edu>
]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: "Bo Persson" <bop@gmb.dk>
Date: Fri, 27 Aug 2010 13:29:06 CST
Raw View
David Krauss wrote:
>
> Finally, how can the existence of a copy constructor have an effect
> on the handling of an exception during move construction? Does the
> FCD require that reserve individually retry a move constructor
> failure by copy construction?

If the move constructor can throw, the implementation will likely not
use it, and fall back to copying instead. That makes it possible to
recover from an exception.

If the object is moveable but not CopyConstructible or CopyAssignable,
the container is forced to try to move anyway. If that fails from an
exception, the result is "unspecified" and probably not good.
Moved-from objects are likely to be lost.


Bo Persson





--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@netlab.cs.rpi.edu]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: David Krauss <potswa@gmail.com>
Date: Fri, 27 Aug 2010 18:10:53 CST
Raw View
On Aug 27, 2:29 pm, "Bo Persson" <b...@gmb.dk> wrote:
>
> If the move constructor can throw, the implementation will likely not
> use it, and fall back to copying instead. That makes it possible to
> recover from an exception.
>
> If the object is moveable but not CopyConstructible or CopyAssignable,
> the container is forced to try to move anyway. If that fails from an
> exception, the result is "unspecified" and probably not good.
> Moved-from objects are likely to be lost.

So it's important to remember noexcept or throw() on the move
constructor! Thanks.

It sounds like the requirements for reserve are correct, and the
modifier functions are specified wrong. It's a good thing that the
right idea is there. So long as the rest gets fixed.


--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use
mailto:std-c++@netlab.cs.rpi.edu<std-c%2B%2B@netlab.cs.rpi.edu>
]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]