Topic: P0709 Heap exhaustion and try_... for
Author: Florian Weimer <fw@deneb.enyo.de>
Date: Sat, 20 Oct 2018 11:20:31 +0200
Raw View
> The OOM handling as we have it today is indeed suboptimal for many many
> environmants. And making all those functions nothrow looks like progress in
> the good direction. The throwing approach is only good for those who
> actually handle it and keep the program going. My past experience on
> several platforms is that the program just never survived such an attempt,
> it failed in some OS call or just got killed.
It's a configuration matter. For example, under Linux, if you want
malloc to fail when there is no memory available, instead of
terminating random processes, you can set the vm.overcommit_memory=2
sysctl. (I believe that on most other systems, this behavior is
actually the default.)
There are also examples of run-times which impose their own memory
allocation limits (such as PostgreSQL). I have seen servers survive
out-of-memory situations reliably. We also tend to get angry bug
reports if we have bugs on memory allocation failure paths in glibc
(such as not checking for malloc failure and reporting it to the
caller, but crashing due to a null pointer dereference). Of course,
there are many counterexamples of systems which do not deal properly
with memory allocation failures, but once a basic system component
gives up on this matter, you can never add it back again at a higher
level, at least not reliably.
Now one could argue that all these are written in C and applications
where resource allocation can fail and need recovery (beyond mere
process termination) are not C++'s domain and should instead use C, or
at least not the C++ standard library. But this is not the feeling I
get from skimming the proposals, many of which imply that there should
not be a lower-level language than C++, and C++ should be up to these
tasks.
Furthermore, until the recent almost universal anti-exception stance,
C++ seemed to me a particularly good language for writing such systems
because due to RAII, you had a decent chance of returning to a
known-good state after an unexpected exception. C++ implementations
even tend to have emergency memory reservations so that they can throw
std::bad_alloc even if the regular heap is exhausted.
> And to help out the code that actually likes the current interface and
> wants to deal with catching bad_alloc, I'm positive we can think up several
> ways from specific interface additions (i.e vector does not really need to
> duplicate all the interface, just have a try_reserve) or some control flags
> at global/thread/object level to alter behavior. If the replacable
> new_handler is not enough.
Realistically, you will need to support libraries with do both forms
of error handling in the same process. Maybe they even have to
interchange std::string objects. I'm not sure if flags are going to
work for this, and certainly not global, process-wide properties.
I'm not even sure if changes are needed. Even today, people can get
process termination on std::bad_alloc if they declare the functions
they write as noexcept, I suppose. (This is the strength of unchecked
exceptions: If you don't want to deal with them, at least you get a
clean error, and you don't need to spend mental effort on ways to
sweep the error under the rug, such as returning made-up return values
because there are insufficient resources to compute the correct
result.)
--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/87o9bpht40.fsf%40mid.deneb.enyo.de.
.
Author: pasa@lib.hu
Date: Sat, 20 Oct 2018 05:18:05 -0700 (PDT)
Raw View
------=_Part_3902_1169589392.1540037885847
Content-Type: multipart/alternative;
boundary="----=_Part_3903_1091774779.1540037885848"
------=_Part_3903_1091774779.1540037885848
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
2018. okt=C3=B3ber 20., szombat 11:20:36 UTC+2 id=C5=91pontban Florian Weim=
er a=20
k=C3=B6vetkez=C5=91t =C3=ADrta:
>
> > The OOM handling as we have it today is indeed suboptimal for many many=
=20
> > environmants. And making all those functions nothrow looks like progres=
s=20
> in=20
> > the good direction. The throwing approach is only good for those who=20
> > actually handle it and keep the program going. My past experience on=20
> > several platforms is that the program just never survived such an=20
> attempt,=20
> > it failed in some OS call or just got killed.=20
>
> It's a configuration matter. For example, under Linux, if you want=20
> malloc to fail when there is no memory available, instead of=20
> terminating random processes, you can set the vm.overcommit_memory=3D2=20
> sysctl. (I believe that on most other systems, this behavior is=20
> actually the default.)=20
>
There were many articles around that in the 90s. Concluding that with the=
=20
default practice of existing linux-like system it is just impossible to=20
have a conforming C++ system. As with the memory overcommit thingy you get=
=20
the memory, no null return, no bac_alloc, no indication, then you just get=
=20
a crash accessing arr[42] -- when the system can not summon a memory=20
page. The other common variant was very funny too, that just killed=20
random processes on a whim, yours if you won the lottery.=20
C++ seemed to me a particularly good language for writing such systems=20
> because due to RAII, you had a decent chance of returning to a=20
> known-good state after an unexpected exception. C++ implementations=20
> even tend to have emergency memory reservations so that they can throw=20
> std::bad_alloc even if the regular heap is exhausted.=20
>
In theory it definitely is. In practice, not that much. My experience on=20
Windows with MFC: It has a fairly centralized system to grab memory. It=20
throws CMemoryException * on failure. The attached object is preallocated=
=20
so from throw point you're supposedly fine.=20
In practice you started the elephant (the memory stress app form win sdk)=
=20
and your app crashed. And so did winword and all the other programs you=20
had. O,O
If you managed to the exception handler and tried as much as a MessageBox,=
=20
it likely had missing buttons, missing text, block characters instead of=20
text or just crash. Pick your poison. I felt lucky if could just close my=
=20
files and exit let alone hope to start something else.=20
Meanwhile in other systems where memory was actually scarce using the=20
regular heap on the go was out of question. The components grabbed their=20
full memory on start and used that till exit.=20
--=20
You received this message because you are subscribed to the Google Groups "=
ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp=
..org/d/msgid/std-proposals/aa08220a-4bd1-452d-86b3-021355f15e29%40isocpp.or=
g.
------=_Part_3903_1091774779.1540037885848
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><br><br>2018. okt=C3=B3ber 20., szombat 11:20:36 UTC+2 id=
=C5=91pontban Florian Weimer a k=C3=B6vetkez=C5=91t =C3=ADrta:<blockquote c=
lass=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;border-left: 1px=
#ccc solid;padding-left: 1ex;">> The OOM handling as we have it today i=
s indeed suboptimal for many many=20
<br>> environmants. And making all those functions nothrow looks like pr=
ogress in=20
<br>> the good direction. The throwing approach is only good for those w=
ho=20
<br>> actually handle it and keep the program going. My past experience =
on=20
<br>> several platforms is that the program just never survived such an =
attempt,=20
<br>> it failed in some OS call or just got killed.
<br>
<br>It's a configuration matter. =C2=A0For example, under Linux, if you=
want
<br>malloc to fail when there is no memory available, instead of
<br>terminating random processes, you can set the vm.overcommit_memory=3D2
<br>sysctl. =C2=A0(I believe that on most other systems, this behavior is
<br>actually the default.)
<br></blockquote><div><br></div><div>There were many articles around that i=
n the 90s. Concluding that with the default practice of existing linux-like=
system it is just impossible to have a conforming C++ system. As with the =
memory overcommit thingy you get the memory, no null return, no bac_alloc, =
no indication, then you just get a crash accessing arr[42] -- when the syst=
em can not summon a memory page.=C2=A0=C2=A0=C2=A0 The other common variant=
was very funny too, that just killed random processes on a whim, yours if =
you won the lottery. <br></div><div></div><div><br></div><blockquote class=
=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;border-left: 1px #cc=
c solid;padding-left: 1ex;">C++ seemed to me a particularly good language f=
or writing such systems
<br>because due to RAII, you had a decent chance of returning to a
<br>known-good state after an unexpected exception. =C2=A0C++ implementatio=
ns
<br>even tend to have emergency memory reservations so that they can throw
<br>std::bad_alloc even if the regular heap is exhausted.
<br></blockquote><div><br></div><div>In theory it definitely is. In practic=
e, not that much. My experience on Windows with MFC: It has a fairly centra=
lized system to grab memory. It throws CMemoryException * on failure. The a=
ttached object is preallocated so from throw point you're supposedly fi=
ne. <br></div><div><br></div><div>In practice you started the elephant (the=
memory stress app form win sdk) and your app crashed. And so did winword a=
nd all the other programs you had. O,O</div><div>If you managed to the exce=
ption handler and tried as much as a MessageBox, it likely had missing butt=
ons, missing text, block characters instead of text or just crash. Pick you=
r poison. I felt lucky if could just close my files and exit let alone hope=
to start something else. <br></div><div></div><div><br></div><div>Meanwhil=
e in other systems where memory was actually scarce using the regular heap =
on the go was out of question. The components grabbed their full memory on =
start and used that till exit. <br></div><div></div><br></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;ISO C++ Standard - Future Proposals" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:std-proposals+unsubscribe@isocpp.org">std-proposa=
ls+unsubscribe@isocpp.org</a>.<br />
To post to this group, send email to <a href=3D"mailto:std-proposals@isocpp=
..org">std-proposals@isocpp.org</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/a/isocpp.org/d/msgid/std-proposals/aa08220a-4bd1-452d-86b3-021355f15e29%=
40isocpp.org?utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.=
com/a/isocpp.org/d/msgid/std-proposals/aa08220a-4bd1-452d-86b3-021355f15e29=
%40isocpp.org</a>.<br />
------=_Part_3903_1091774779.1540037885848--
------=_Part_3902_1169589392.1540037885847--
.