Paul J. Ste. Marie (pstemari@erinet.com) wrote:
: In article <3qv228$j51@galaxy.ucr.edu>, thp@cs.ucr.edu (Tom Payne)
: wrote:
: :[snip]
: :Specifically, signal handlers must be able to:
: :
: :  * read and write global variables (with the usual understandings
: :    that there you may get a stale value due to an outstanding
: :    copy in a register).

: Hmm...wouldn't this be better qualified as "read and write
: _volatile_ global variables"

:  --Paul J. Ste. Marie, pstemari@well.sf.ca.us, pstemari@erinet.com

: The Financial Crimes Enforcement Network claims that they capture every
: public posting that has their name ("FinCEN") in it.  I wish them good hunting.


Yes!!  That would certainly suffice and would be much more reasonable
(i.e., implementable and optimizable).  Unfortunately, per the current
C Standard, reading any global leads to undefined behavior.  The view
is the handlers shoud only set globals, which are then polled by the
underlying program.  Ugh!

Tom Payne

In article <3qv228$j51@galaxy.ucr.edu>, thp@cs.ucr.edu (Tom Payne)
wrote:
:[snip]
:Specifically, signal handlers must be able to:
:
:  * read and write global variables (with the usual understandings
:    that there you may get a stale value due to an outstanding
:    copy in a register).

Hmm...wouldn't this be better qualified as "read and write
_volatile_ global variables"

 --Paul J. Ste. Marie, pstemari@well.sf.ca.us, pstemari@erinet.com

The Financial Crimes Enforcement Network claims that they capture every
public posting that has their name ("FinCEN") in it.  I wish them good hunting.

: >Pre-emptive multithreading seems to present a more difficult problem.

:  No. Not really. The main problem is stupid library
: functions that are not re-entrant.

:  However, threads and tasking are outside the scope
: of the C++ Standard at present.

:  Coroutines are not.


Coroutines + signals + 1500 lines of C++ code == threads + monitors

The problem with the current C Standard for signal handlers is that
their behavior is undefined if they even attempt to read a global
variable.  This polling-only model for asynchrony is marginal even for
standard event-driven programming.

Specifically, signal handlers must be able to:

  * read and write global variables (with the usual understandings
    that there you may get a stale value due to an outstanding copy
    in a register).

  * call a coroutine (with the usual understanding that signals need
    to be blocked at critical times).

It would be convenient, but not necessary, for signals to be able to
throw exceptions, which appears to be compatible with the
implementation strategy for exceptions mentioned on page 397 of D&E.

The necessary norportables functions are the ability to create,
destroy, and call coroutines and to install a given function as a
signal handler.  It looks to me as through we can portably implement
signal blocking with global flags, and from signal blocking we can
get locks in the monoprocessor case.  Multiprocessor locking can
be done portably via the bakery algorithm, but some nonportable
assistance based on the underlying instruction set would improve
efficiency.

What I have in mind is a very simple hardware analog that views a
signal as an involuntary function call on behalf of the currently
running function invocation and gets serviced on the current stack.
(There are various ways to circumvent the old conundrum: "Then where
do you handle stack-overflow signals?")

Of course, for any implementation of threads & monitors to be of much
use, libraries need to be thread safe.  Overloading the global news
and deletes so that they use safe versions of malloc and free is not
difficult, but every class can have its local news and deletes, which
can lead to chaos.


Tom Payne

In article <3piugd$1od@galaxy.ucr.edu>, Tom Payne <thp@cs.ucr.edu> wrote:
>
>:  But it _isn't_ clear any "easy to port" implementation
>: in C++ is possible because of EH. C perhaps, since it has no EH.
>: There's no telling what an implementor might do to keep track of
>: what exception to throw and where to find handlers.
>
>No doubt there are ways to implement exceptions that make easy things
>difficult, but an implementation of exceptions can't survive a full
>saving and subsequent restoration of the working registers (including
>stack pointer and base pointer) is likely to have trouble with signals
>and longjmps as well.

 Nope. If you check the CD, setjmp/longjmp doesn't
have to work except in HIGHLY restricted contexts.
In particular, if there is ANY automatic variable on the stack
between the setjmp and longjmp, the results are undefined.

 This is a change from the C Standard.

>For full portability, how about augmenting the standard with two new
>functions the setjmp library?
>
>  * makejmp( jmp_buf, void (*)() )
>    would allocate a new stack (wherever appropriate) and initialize
>    jmp_buf with register settings for that stack and for the
>    parameterless void function as the base function of the resulting
>    thread (coroutine).
>
>  * killjmp( jmp_buf )
>    would deallocate the stack associated with the jmp_buf (presumably
>    not the program's initial stack).
>

 Yes. I agree.

>The obvious question is, "Why mess with setjmp and longjmp rather than
>adding an entirely new mechanism?"  IMHO:
>
>  *  Their semantics is almost exatly that of coroutines, and a full
>     implementation of threads can be built on them (plus locks).

 Yes.

>Pre-emptive multithreading seems to present a more difficult problem.

 No. Not really. The main problem is stupid library
functions that are not re-entrant.

 However, threads and tasking are outside the scope
of the C++ Standard at present.

 Coroutines are not.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

John Max Skaller (maxtal@Physics.usyd.edu.au) wrote:
: In article <3oufem$apf@galaxy.ucr.edu>, Tom Payne <thp@cs.ucr.edu> wrote:
: >John Max Skaller (maxtal@Physics.usyd.edu.au) wrote:
: >: In article <3olo3n$54@galaxy.ucr.edu>, Tom Payne <thp@cs.ucr.edu> wrote:
: >: >Matthew Austern (matt@dogbert.lbl.gov) wrote:
: >: >
: >: >It's not terribly difficult to implement threads, monitors, and
: >: >conditions (along the lines of Hoare's Simula-inspired CACM paper,
: >: >v17, #10) as a simple C++ library.
: >
: >:  A _portable_ implementation?
: >: Which works with Exception Handling?
: >
: >Not totally _portable_ but _easy_to_port_.
: >

:  The question is whether such an implementation would
: work correctly with exceptions. In fact I've just ported
: John English's excellent Borland 3.1 coroutine package to
: Metaware High C/C++ -- and I can report that with some caveats about throwing
: exceptions out of a coroutine -- or destroying the coroutine
: with an exception by unwinding it -- it seems to work.

:  But it _isn't_ clear any "easy to port" implementation
: in C++ is possible because of EH. C perhaps, since it has no EH.
: There's no telling what an implementor might do to keep track of
: what exception to throw and where to find handlers.

No doubt there are ways to implement exceptions that make easy things
difficult, but an implementation of exceptions can't survive a full
saving and subsequent restoration of the working registers (including
stack pointer and base pointer) is likely to have trouble with signals
and longjmps as well.

For full portability, how about augmenting the standard with two new
functions the setjmp library?

  * makejmp( jmp_buf, void (*)() )
    would allocate a new stack (wherever appropriate) and initialize
    jmp_buf with register settings for that stack and for the
    parameterless void function as the base function of the resulting
    thread (coroutine).

  * killjmp( jmp_buf )
    would deallocate the stack associated with the jmp_buf (presumably
    not the program's initial stack).

Wording would have to be added to the standard specifying that idiom

    if( ! setjmp(thread1.context) ) longjmp(thread2.context);

indeed transfers the stream of execution (CPU) from one thread (jmp_buf)
to the other in such a way that exceptions and signals work correctly
as long as certain protocols are observed, e.g.:
  *  no thread may throw an exception back past its creation (makejmp).
  *  no signal handler or any function it calls (even indirectly) may
     throw an exception past its invocation.
And so on.

The obvious question is, "Why mess with setjmp and longjmp rather than
adding an entirely new mechanism?"  IMHO:

  *  Their semantics is almost exatly that of coroutines, and a full
     implementation of threads can be built on them (plus locks).

  *  They already seem to work tolerably well for multithreading in
     many implementations.  (My experiece is restricted to g++ under
     SunOS, Solaris, and Linux, but I've heard rumors of success with
     other systems).

  *  Their original rationale has been pre-empted by exception handling,
     so they currently have no real mission.

  *  Recycling them for this new purpose, via tightened specification,
     would be in keeping with C's economy of concepts (stinginess with
     keywords).

Pre-emptive multithreading seems to present a more difficult problem.
Pre-emption must be initiated by invocations of a signal handler,
e.g., the timer signal for timeslicing.  If the handler directly
transfers the CPU on behalf the interrupted thread, that thread sleeps
on an uncompleted handler invocation, whereas some implementers and
designers of signal protocols have in mind that the last invocation of
a handler will complete before the next one is honored.

The alternatives get quite messy.  The handler must modify the
environment in such a way that after it (the handler) has returned the
interrupted thread will spontaneously transfer execution to another
thread and, upon reawakening from that transfer, will return to the
point of interruption.  For instance one might imagine building a
couple of activation records on the stack on which the handler itself
is executing -- not an easy trick.

Tom Payne

In article <3oufem$apf@galaxy.ucr.edu>, Tom Payne <thp@cs.ucr.edu> wrote:
>John Max Skaller (maxtal@Physics.usyd.edu.au) wrote:
>: In article <3olo3n$54@galaxy.ucr.edu>, Tom Payne <thp@cs.ucr.edu> wrote:
>: >Matthew Austern (matt@dogbert.lbl.gov) wrote:
>: >
>: >It's not terribly difficult to implement threads, monitors, and
>: >conditions (along the lines of Hoare's Simula-inspired CACM paper,
>: >v17, #10) as a simple C++ library.
>
>:  A _portable_ implementation?
>: Which works with Exception Handling?
>
>Not totally _portable_ but _easy_to_port_.
>

 The question is whether such an implementation would
work correctly with exceptions. In fact I've just ported
John English's excellent Borland 3.1 coroutine package to
Metaware High C/C++ -- and I can report that with some caveats about throwing
exceptions out of a coroutine -- or destroying the coroutine
with an exception by unwinding it -- it seems to work.

 But it _isn't_ clear any "easy to port" implementation
in C++ is possible because of EH. C perhaps, since it has no EH.
There's no telling what an implementor might do to keep track of
what exception to throw and where to find handlers.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <D8qLMK.IwB@ucc.su.OZ.AU>,
John Max Skaller <maxtal@Physics.usyd.edu.au> wrote:
>In article <3oufem$apf@galaxy.ucr.edu>, Tom Payne <thp@cs.ucr.edu> wrote:
>>John Max Skaller (maxtal@Physics.usyd.edu.au) wrote:
>>: In article <3olo3n$54@galaxy.ucr.edu>, Tom Payne <thp@cs.ucr.edu> wrote:
>>: >Matthew Austern (matt@dogbert.lbl.gov) wrote:
>>: >
>>: >It's not terribly difficult to implement threads, monitors, and
>>: >conditions (along the lines of Hoare's Simula-inspired CACM paper,
>>: >v17, #10) as a simple C++ library.
>>
>>:  A _portable_ implementation?
>>: Which works with Exception Handling?
>>
>>Not totally _portable_ but _easy_to_port_.
>>
>
> The question is whether such an implementation would
>work correctly with exceptions. In fact I've just ported
>John English's excellent Borland 3.1 coroutine package to
>Metaware High C/C++ -- and I can report that with some caveats about throwing
>exceptions out of a coroutine -- or destroying the coroutine
>with an exception by unwinding it -- it seems to work.
>
> But it _isn't_ clear any "easy to port" implementation
>in C++ is possible because of EH. C perhaps, since it has no EH.
>There's no telling what an implementor might do to keep track of
>what exception to throw and where to find handlers.

 In fact a bit more testing reveals the EH mechanism
stuffs up the coroutines. Leaving C++ without an important
control structure that C permits (although it doesn't provide).
--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

John Max Skaller (maxtal@Physics.usyd.edu.au) wrote:
: In article <3olo3n$54@galaxy.ucr.edu>, Tom Payne <thp@cs.ucr.edu> wrote:
: >Matthew Austern (matt@dogbert.lbl.gov) wrote:
: >
: >It's not terribly difficult to implement threads, monitors, and
: >conditions (along the lines of Hoare's Simula-inspired CACM paper,
: >v17, #10) as a simple C++ library.

:  A _portable_ implementation?
: Which works with Exception Handling?



Not totally _portable_ but _easy_to_port_.

Creation, suspension, and resumption of threads may require a bit of
assembly language to initialize, save, and restore contexts.  Often,
however, setjmp and longjmp can be used to transfer execution:

  if( ! setjmp( suspendingContext ) longjmp( resumingContext, 1 );

A few lines of non-portable C++ are still required to initialize the
SP and possibly other fields of a thread's context (jmp_buf).  Also,
if the architecture doesn't support heap-allocated stacks, the OS must
be invoked for the allocation of a thread's stack segment.

If a system already has a threads implementation, e.g., in the OS or a
C-based thread libraries (like Pthreads or Cthreads), an
object-oriented C++ threads library can use those facilities to
create, suspend, and resume threads.  Semaphores initialized to one
can be used as locks, and semaphores initialized to zero can be used
to implement conditions (per Hoare's paper).


I have yet to try threads in the presence of exception handling, but I
take seriously Stroustrup's comment (D&E p.385) that one of the ideals
for C++ exception handling was that it be "A mechanism that by
*default* will work correctly in a multi-threaded program."  [emphasis
added]

The most obvious problem, releasing the appropriate locks when an
exception occurs, can be handled by relegating all lock handling to
local objects (sentries) whose constructor acquires a specified lock
and blocks certain interrupts and signals, and whose destructor
releases the lock and restores signal and interrupt status.  There are
other obvious considerations (e.g., the base function of a thread must
be capable of handling all exceptions that the thread might throw),
and I'd appreciate information about yet others.


Tom Payne

In article <MATT.95May8103436@dogbert.lbl.gov>,
Matthew Austern <matt@physics.berkeley.edu> wrote:
>In article <D873A8.7tp@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John Max Skaller) writes:
>
>>  This IS provided by coroutines/threads and channels.
>> Unfortunately, yet again, C++ fails to provide language
>> support for a completely fundamental idiom.
>
>That's an interesting point.  The obvious followup question, then:
>what do you think that language support for threads would look like?

 First you do coroutines. They are much simpler because
control exchange is synchronous.

 In a dumb implementation you need to create a coroutine
with a given stack size, you need a "function" to  begin
executing and a way of exchanging control. You also need to
kill the coroutine (if it doesn't suicide).

 A suitable interface is a few lines of code. It is very
hard to implement in C++ because of exception handling.
In C it is fairly easy -- have a look at John English package
for Borland 3.1 (DOS) called CCL110JE. Or Dag Bruck's
proposed (and then withdrawn) class :-(

 Threads are the same except control can _also_ transfer
asynchronously and so you need to have locking.

 Given that monitors can be implemented, a threadwise
monitor may provide blocking or unblocked transmissions.
There is a C variable called "Alef" which does this in the
core language -- very nice.

>The most obvious answer is just to provide library functions to create
>and kill threads; if you're using a multithreaded operating system
>(OS/2, NT, Solaris, and so on), then your compiler vendor has probably
>already provided a library function giving you access to that
>functionality.  But I imagine that real support for concurrency
>involves something more than that.

 Multi-processing in general is a larger field. Outside the
agreed on scope of the first C++ Standard.

 Coroutines are a fundamental control structure for
synchronous symmetric exchange of a _single_ thread of control.

 That is -- within the scope of the C++ Standard.
--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <3olo3n$54@galaxy.ucr.edu>, Tom Payne <thp@cs.ucr.edu> wrote:
>Matthew Austern (matt@dogbert.lbl.gov) wrote:
>
>It's not terribly difficult to implement threads, monitors, and
>conditions (along the lines of Hoare's Simula-inspired CACM paper,
>v17, #10) as a simple C++ library.

 A _portable_ implementation?
Which works with Exception Handling?

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

maxtal@Physics.usyd.edu.au (John Max Skaller) writes:

>Andrew Fitzgibbon <andrewfg@ed.ac.uk> wrote:
>>
>>Derived classes plus RTTI gives you
>>discriminated unions *but for* the size difference, which should only be an
>>issue if you're taking over storage management.
>
> No. Using RTTI gives you MUCH MORE than
>a discriminated union, which is exactly the problem.

Using RTTI to simulate discriminated unions is not as good
as using inheritance from an abstract base class.

>A proper discriminated unions offers RIGID guarrantees:
>
> a) a FINITE set of KNOWN and SPECIFIED types
> b) a compiler error if you access the wrong type
>    (provided you do not change the type inside
>    a case of another type)
> c) a compiler error is you _miss_ a type case
> d) the ability to compose ANY types
>
>With RTTI (ignoring the storage mamagement issues)
>
> a) there is no bound on the possible types
> b) you get a run time error on selecting the wrong type
> c) compile time diagnostics for incorrect use after selection
> d) no verification of completeness of case handling
> e) invasion: you CANNOT compose ANY type, only types
>    with a common base

With abstract base classes as suggested by James Kanze (and perhaps
downcasting with dynamic_cast, but not typeid()), you get

 (a) there can be a bound on the possible types, if you
     want (by making the abstract base class constructor
     private and listing all the possible types as friends
     of the abstract base class), but there doesn't have to be
 (b) you get a compile error on selecting the wrong type
 (c) you get either a compile error if you miss a type case
     (if you use virtual functions for the dispatch) or
     incorrect results at runtime (if you do partial matching
     via downcasting).

but it has the minor disadvantage

 e) invasion: you cannot compose any type, only types
    derived from the abstract base class (so you have to
    do a little extra work to create the derived classes, and
    conversions between a derived class and its data member
    are by default explicit, not implicit).

and the major disadvantage

 f) adding a new operation forces you to recompile everything

--
Fergus Henderson                       | I'll forgive even GNU emacs as
fjh@cs.mu.oz.au                        | long as gcc is available ;-)
http://www.cs.mu.oz.au/~fjh            |             - Linus Torvalds

In article <9512603.23750@mulga.cs.mu.OZ.AU>,
Fergus Henderson <fjh@munta.cs.mu.OZ.AU> wrote:
>
>Inheritence from an abstract base class does give you the same sort
>of effect as discriminated unions, but at the expense of having to
>structure your code inside-out (with potentially bad effects on
>recompilation time).

 To me that is a far lesser concern than the fact that
one must turn ones thinking inside out to understand such
perverted design.

 This is sort of like callbacks and event driven
programming -- you end up using objects to partition the
state space but lose the ability to use the natural stacklike
structure which structured programming emphasises. YOu end up
programming state machines manually in the objects. This
is a major step BACKWARDS in software engineering -- it is
equivalent to "flat" global programming like in C or COBOL
or FORTRAN -- but in smaller state space.

 There is a place for callbacks -- but there needs to
be a method of interfacing this control structure with the
_inverted_ structure obtained by polling.

 This IS provided by coroutines/threads and channels.
Unfortunately, yet again, C++ fails to provide language
support for a completely fundamental idiom.

 I call this particular variant of the problem
"control inversion". It is categorically similar to the
union problem above -- there is a time to place
methods inside the data, and a time to place the methods
outside the data.

 It is the mixing and interfacing of these
"inside/outside" problems which constitutes software
engineering and it is the failure of C++ to support
both "inside" and "outside" programming properly --
and the ability to interface them -- which is my
strongest criticism of C++ as a language.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <KANZE.95May3122008@slsvhdt.lts.sel.alcatel.de>,
James Kanze US/ESC 60/3/141 #40763 <kanze@lts.sel.alcatel.de> wrote:
>In article <D7zAzJ.EMo@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
>Max Skaller) writes:
>
>    [concerning: must `vector' be a template...]
>|>  Yes. Which is why I'm right and James is wrong.
>|> There's no requirement that any Standard Library entity I'm
>|> aware of actually be what it is presented as being. The Standard
>|> Library merely leverages off the core language to present a
>|> specification and then permits any mechanism of implementation
>|> at all which conforms to the required behaviour.
>
>|>  For example "memcpy" does not have to be a function.
>|> Some compilers generate code directly. The same applies to
>|> say a vector<T> where T is memcpy-able and it applies
>|> very much in particular <g> to vectors of bool where
>|> major advantages may be gained by directly generating
>|> code using machine instructions which manipulate bits.
>
>This may be just a disagreement concerning terms.

 Yes, but being pedantic is sometimes important.
See below.

>The standard does *not* say how the function must be implemented.  A
>compiler is free to use "magic" to recognise the special case, and
>inline the function if it wants.
>
>Similarly, the C++ standard defines `vector' as a template.

 Yes. But there the similarity ends. Templates
have to have definitions because they are _extremely_ sensitive
to context. It may make a BIG difference what the actual body
of a template definition is. Because of implicit name binding
and instantiation issues, etc.

 Let me explain in more detail. The C++ CD mentions
a thing called the ODR -- One Definition Rule. But the ODR
is not properly specified in the CD.

 I have written a paper proposing a specific rule
for the ODR; I believe it has some support. The wording
of the rule requires certain definitions to be equivalent,
and states exactly what "equivalent" means.

 Equivalence is defined by starting off with a rule

 TE1) The sequence of tokens of the definition or
 expression shall be the same.

So for example the following code BREAKS the ODR:

 // file 1
 inline extern void f(){}

 // file 2
 extern inline void f(){}

because the definitions are NOT equivalent because the tokens "extern"
"inline" are swapped around.

There are more rules, but the point is that one CANNOT SPEAK
ABOUT THE EQUIVALENCE OF GENERATED DEFINITIONS.

Do you see? A definition is a kind of declaration which
IS A SEQUENCE OF TOKENS. Written by the programmer.

The ODR in the form I wrote it CANNOT be applied to template
instances (it CAN be applied to the templates themselves).

It also CANNOT be applied to the Standard Library because
the Standard Library entities DO NOT NECESSARILY HAVE DEFINITIONS.

So when the ODR says "A function shall have exactly one
definition ... a function may have more than one definition"

... it is refering to FUNCTIONS. Which are exclusively things
coded by users. No entity of the Standard Library is
necessarily a function because no entity of the Standard
Library can be required to even HAVE a definition.
Because a definition is a specific sequence of tokens
written by the programmer.

So if the words above are accepted, only users can code functions.
Otherwise the words need to say "user function" (non-generated
non-Standard Library function).

The ODR does not apply to generated definitions and it does
not apply to the Standard Library.

Note that there is no problem except with templates.

A separate TODR (Template ODR) is needed to handle templates.
It will have somehow to work for the "templates" of the
Standard Library, an extra complication.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <D7y986.K4@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
Max Skaller) writes:

|> In article <KANZE.95Apr27145146@slsvhdt.lts.sel.alcatel.de>,
|> James Kanze US/ESC 60/3/141 #40763 <kanze@lts.sel.alcatel.de> wrote:
|> >
|> >|>  Stl containers in the Standard Library are not templates
|> >|> because they do not "quack" like templates. Real templates
|> >|> "quack" because they have definitions consisting of examinable
|> >|> source code.
|> >
|> >Really.  I have the Rogue Wave components, complete with templates,
|> >delivered with my C++ compiler.  But there is no examinable source
|> >code.
|> >
|> >What makes you state that examinable source code is necessary?

|>  Simple. The WP says so.

Where?  A quick glance at chapter 14 showed no such text.  In fact, if
it did, we would have a problem: the notion of `examinable source
code' is nowhere defined.  (Examinable by whom, for example?  By me,
or only by the compiler?  Consider the Sun distribution of the Rogue
Wave components.  The source code is encrypted.  The compiler can read
it correctly.  I can even examine it.  But what I see when I examine
it will look like a binary file, and not C++ sources.  Does this count
as `examinable source code'?)

|> Specifically, a template is
|> a sequence of tokens contained in a translation unit parsed
|> as a template according to the rules of the grammar and
|> associated constraints and semantic rules.

Strictly speaking, the language doesn't have templates, it has
template declarations and template definitions.  Even if your
interpretation of the above is correct, it simply means that some part
of the implementation, at some time, must have seen the corresponding
sequence of tokens.  It doesn't mean that they are in anyway made
available to me.  It doesn't even mean that they are available on my
machine in source form.  (They could be preloaded directly in internal
formaat into the compiler, for example.  The implementation would have
seen them when the compiler itself was being compiled.)

This (with the exception of the example in parenthese) is valid for
*all* templates, even user defined ones.  The Rogue Wave classes
provided with the Sun compiler are basically an externally written
class library; any class library could be provided in this format
(except that Sun doesn't make the encryption tools available).

Suppose that Sun simply did this automatically.  When you compiled a
template definition, it simply did the required syntax checks, and put
an encrypted copy into the object file.  Are you claiming that this is
illegal, according to the standard.

In the case of the standard library, of course, there is (probably)
even more freedom.  I would contend that the standard doesn't
(shouldn't) even require the library to have ever existed as clear
text.  The implementation should be free to use any desired
meta-magic; the standard defines what *you* can do with the library,
and not how the compiler goes about implementing this.

|>  The behaviour of this entity called a template
|> is (or should be) defined in the CD.

Agreed.  And *only* the behavior.

When I write:

 #include <vector>

 vector< double >    vd ;

in my program, this has a defined meaning.  How the compiler goes
about implementing this meaning is none of the standards business.

|>  Any component not in this form -- or which was not
|> at some stage in this form -- is catgorically NOT a
|> function, template, or any other such thing described
|> in the CD. [Unless it is a library function,
|> which is not a function]

Why are library functions not functions?  Why must library templates
be templates?  IMHO, library functions are functions.  But there is no
requirement that either a function or a template must have, at some
stage, been in the form of ASCII (or any other recognized code set).

I'm not even 100% convinced that the use of meta-magic is forbidden
for functions/classes *not* defined in the standard.  The compiler
need simply provide some way of turning them off.  (Example: most
compilers search for include files in an ordered list of places, with
the place where the "system" include files reside at the end of the
list.  Suppose that when looking for the include file 'hashtbl', if my
compiler finds it in the place where the system include's normally
reside, *and* the file simply contains some meta-magical string, which
is not legal C++, I simply `unhide' a preloaded definition.  This
should be legal.  For the system include files, at least in C, it is
explicitly legal.)

|>  The behviour of a C++ function is determined by
|> its definition which is a sequence of tokens.

Correction: C++ provides one portable way of defining (the behavior
of) a function: inputting a sequence of tokens into the implementation
(compiler or interpreter).  All implementations have been extended in
some way or another to provide other possibilities, at the very least,
to link with programs written in assembler for example.

On my system, I have a header file called unistd.h, which declares a
function called read.  Are you saying that read is not a function,
because it was not written as a sequence of C/C++ tokens?  (In fact,
there is *no* way to write a function in pure C/C++ with the
functionality of read.  It requires an extension, either in the form
of linking with assembler code, inline assembler code, or special
compiler extensions like _input.)

Read is a very real function.  In C or C++.  It was not defined as a
sequence of (ISO standard) C/C++ tokens.

|>  A Standard Library entity that quacks like a function
|> is NOT a function.

|>  Vendors have the freedom to supply things that
|> quack like functions behaviourly but are not in fact functions,
|> in the Standard Library.

|>  Same applies to templates.

OK, so we differ concerning a definition.  But I would argue that not
only are vendors free to supply such things, they also have the
freedom to make extensions which allow the users to supply them.  All
vendors that I know of *do* have such extensions.

|> >|>  Standard library STL containers are not templates
|> >|> because they do not have to have a definition, so there
|> >|> is no way to do the deduction.
|> >
|> >I beg to disagree.  According to the description I read, they *are*
|> >templates.  The working papers define them clearly as templates, e.g.:
|> >section 23.2.8 Template class vector.

|>  No, it doesn't. It _describes_ them as if they
|> were templates for convinence, and if it says anything
|> else it is wrong and must be fixed.

And I think we are just arguing terminology (which *is* important in a
standard).  If you claim that 1) there is no requirement that the
things describes as templates in the working papers have ever existed
as actual template source code, and 2) there is no requirement that
the actual source code for any template be accessible to the
implementation at the moment I present my source code (which uses said
template) to the implementation (or at any time later), then we are
agreed on what I consider the essential.  I would also argue that
there is nothing which prevents an implementation from offering (as an
extension) the possibility for a user to implement something
additional in the same way as 1).

So why not just call it a template.  Since there is *no* practical way
within the implementation that you can distinguish it from a classical
template.  (Being able as a human being to examine the source code is
definitely outside the implementation, unless you consider yourself as
part of the implementation.)
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

In article <D7y164.KEH@aisb.ed.ac.uk> andrewfg@dai.ed.ac.uk (Andrew
Fitzgibbon) writes:

|> In article <D6ytsv.HJC@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
|> Max Skaller) writes:

|>     [You need discriminated unions for...]
|> |>  2) A grammar (or parse tree).

|> |>  Each node of a grammar is either a terminal or non-terminal.
|> |>  You can't process the grammar without knowing which is which.
|> |>  You have to build a tree of nodes. The node has to be
|> |>  a discrimniated union of the two kinds.

|> Hold on.  This can't be right.  Derived classes plus RTTI gives you
|> discriminated unions *but for* the size difference, which should only be an
|> issue if you're taking over storage management.

Actually, you don't need RTTI, and derived classes can theoretically
require less memory than discriminated unions.  (A union,
discriminated or not, must be as big as its biggest member.  Each
derived class can be only as big as necessary.  Of course, except in
the unlikely case where one node type is significantly larger than all
of the others, this is unlikely to make a significant difference in
practice.)

I state this from experience.  I have written at least two grammars
using these techniques.  In neither did I ever feel the lack of RTTI
(nor did I implement it myself, except insofar as the `dump' function
returned the a symbolic representation of the node type, with
additional data).
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

andrewfg@dai.ed.ac.uk (Andrew Fitzgibbon) writes:

>In article <D6ytsv.HJC@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
>Max Skaller) writes:
>
>    [You need discriminated unions for...]
>|>  2) A grammar (or parse tree).
>
>|>  Each node of a grammar is either a terminal or non-terminal.
>|>  You can't process the grammar without knowing which is which.
>|>  You have to build a tree of nodes. The node has to be
>|>  a discrimniated union of the two kinds.
>
>Hold on.  This can't be right.  Derived classes plus RTTI gives you
>discriminated unions *but for* the size difference, which should only be an
>issue if you're taking over storage management.

No, derived classes plus RTTI (typeid or dynamic_cast) does not
give you the equivalent of discriminated unions, because the compiler
will not tell you if you have missed a case.

Inheritence from an abstract base class does give you the same sort
of effect as discriminated unions, but at the expense of having to
structure your code inside-out (with potentially bad effects on
recompilation time).

--
Fergus Henderson                       | I'll forgive even GNU emacs as
fjh@cs.mu.oz.au                        | long as gcc is available ;-)
http://www.cs.mu.oz.au/~fjh            |             - Linus Torvalds

kanze@lts.sel.alcatel.de (James Kanze US/ESC 60/3/141 #40763) writes:

[Regarding discriminated unions:]
>Actually, you don't need RTTI, and derived classes can theoretically
>require less memory than discriminated unions.  (A union,
>discriminated or not, must be as big as its biggest member.  Each
>derived class can be only as big as necessary.  Of course, except in
>the unlikely case where one node type is significantly larger than all
>of the others, this is unlikely to make a significant difference in
>practice.)

I would say that discriminated unions are a mathematical construct, not
an implementation technique.  They do not have to be implemented
naively.  The compiler I'm currently working on (for Mercury) usually
implements discriminated unions as tagged pointers to variable-sized
storage allocated on the heap.

Using inheritence from an abstract base class is not such a bad way
of implementing discriminated unions in C++, but it does fundamentally
structure the program around the different data types, rather than
around the different algorithms; this is not always the best way
of structuring programs, and forces the entire program to be recompiled
whenever you add a new algorithm.

--
Fergus Henderson                       | I'll forgive even GNU emacs as
fjh@cs.mu.oz.au                        | long as gcc is available ;-)
http://www.cs.mu.oz.au/~fjh            |             - Linus Torvalds

rridge@calum.csclub.uwaterloo.ca (Ross Ridge) writes:

>So what in the standard can be enforced?

Most of it.

>What exactly does
>"enforced" mean anyways?  The question I'm interested in is "is
>it conformant?" and the answer to that can only be "no" or "maybe".

If you can get a "no" answer, then the rule is enforcable.  But the
point is that with some of the complexity requirements (e.g. those that
talk about something being "linear") it is not possible to show that an
implementation does not conform.

--
Fergus Henderson                       | I'll forgive even GNU emacs as
fjh@cs.mu.oz.au                        | long as gcc is available ;-)
http://www.cs.mu.oz.au/~fjh            |             - Linus Torvalds

In article <D873A8.7tp@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John Max Skaller) writes:

>  This IS provided by coroutines/threads and channels.
> Unfortunately, yet again, C++ fails to provide language
> support for a completely fundamental idiom.

That's an interesting point.  The obvious followup question, then:
what do you think that language support for threads would look like?

The most obvious answer is just to provide library functions to create
and kill threads; if you're using a multithreaded operating system
(OS/2, NT, Solaris, and so on), then your compiler vendor has probably
already provided a library function giving you access to that
functionality.  But I imagine that real support for concurrency
involves something more than that.

I imagine the right answer would go something along the lines of
formalizing the difference between an active and an inactive object,
and allowing the number of active objects to be greater than one and
to change dynamically.  That's a big change conceptually, but I bet it
wouldn't involve adding very many new language constructs or changing
the semantics of very many existing ones.  I bet that an experimental
Concurrent C++ compiler, presumably based on gcc, wouldn't be all that
big a project.

I'm sure the Ada contingent will tell me that Ada has already solved
this problem...  One of these days I really ought to learn more about
that language.
--
Matt Austern          matt@physics.berkeley.edu
http://dogbert.lbl.gov/~matt

Matthew Austern (matt@dogbert.lbl.gov) wrote:
: In article <D873A8.7tp@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John Max Skaller) writes:

: >  This IS provided by coroutines/threads and channels.
: > Unfortunately, yet again, C++ fails to provide language
: > support for a completely fundamental idiom.

: That's an interesting point.  The obvious followup question, then:
: what do you think that language support for threads would look like?

: The most obvious answer is just to provide library functions to create
: and kill threads; if you're using a multithreaded operating system
[stuff deleted]
: I imagine the right answer would go something along the lines of
: formalizing the difference between an active and an inactive object,
: and allowing the number of active objects to be greater than one and
: to change dynamically.  That's a big change conceptually, but I bet it
[more stuff deleted]


It's not terribly difficult to implement threads, monitors, and
conditions (along the lines of Hoare's Simula-inspired CACM paper,
v17, #10) as a simple C++ library.  The problem seems to be producing
an implementation of the Standard C++ Library that is thread-safe
relative to a given definition of threads and thread coordination.
For that reason (among others), it would be best if there were an
official C++ standard for threads libraries.

Tom Payne

In article <DOUG.95Apr27094046@monet.ads.com>,
Doug Morgan <doug@monet.ads.com> wrote:
>In article <KANZE.95Apr27145146@slsvhdt.lts.sel.alcatel.de> kanze@lts.sel.alcatel.de (James Kanze US/ESC 60/3/141 #40763) writes:
>> In article <D7Iw22.HqA@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
>> Max Skaller) writes:
>>[...]
>> |>  Standard library STL containers are not templates
>> |> because they do not have to have a definition, so there
>> |> is no way to do the deduction.
>>
>> I beg to disagree.  According to the description I read, they *are*
>> templates.  The working papers define them clearly as templates, e.g.:
>> section 23.2.8 Template class vector.  They are thus templates by
>> definition. [...]
>
>That sounds mildly depressing.  By my reading of the orginal STL
>papers, containers were any objects with certain specified semantics.
>Objects of particular templates could be examples of containers, but
>were not all containers.  Under the definitions of the STL papers,
>there could be many other true STL vector containers other than the
>objects of any one particular class template that happens to be named
>vector.
>
>If the standardization process doesn't maintain this distinction in
>some form, then we will be reduced to descriptions like "this object
>presents the same external interface as an instantiation of the
>(one-and-only, language-provided) STL template class vector," rather
>than "this object is an STL vector."  We'll also lose the ability to
>say "this is a vendor-supplied STL vector and these are various kinds
>of user-supplied STL vectors."

 Yes. Which is why I'm right and James is wrong.
There's no requirement that any Standard Library entity I'm
aware of actually be what it is presented as being. The Standard
Library merely leverages off the core language to present a
specification and then permits any mechanism of implementation
at all which conforms to the required behaviour.

 For example "memcpy" does not have to be a function.
Some compilers generate code directly. The same applies to
say a vector<T> where T is memcpy-able and it applies
very much in particular <g> to vectors of bool where
major advantages may be gained by directly generating
code using machine instructions which manipulate bits.

 It isn't clear _any_ portable template implementation
in C++ could be as efficient as this and there is no intention
to prevent such an implementation. On the contrary a major
goal of placing components in the Standard Library is to
enable such optimisation.

 Another example is the numerical array components
which were specifically designed by a physicist (Kent Budge)
to allow parallel processing. (Vectorisation).

 There IS an intention to ensure that it is POSSIBLE to
write a portable Standard Library (or most of it).

 So lets be very clear. The Standard Library is NOT a library
in the sense of a library of user defined components.
It is an _intrinsic_ part of the C++ language which hopefully
CAN be implemented as a library of "user" defined components
(by the vendor) but does NOT have to be.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <D7y164.KEH@aisb.ed.ac.uk>,
Andrew Fitzgibbon <andrewfg@ed.ac.uk> wrote:
>In article <D6ytsv.HJC@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
>Max Skaller) writes:
>
>    [You need discriminated unions for...]
>|>  2) A grammar (or parse tree).
>
>|>  Each node of a grammar is either a terminal or non-terminal.
>|>  You can't process the grammar without knowing which is which.
>|>  You have to build a tree of nodes. The node has to be
>|>  a discrimniated union of the two kinds.
>
>Hold on.  This can't be right.  Derived classes plus RTTI gives you
>discriminated unions *but for* the size difference, which should only be an
>issue if you're taking over storage management.

 No. Using RTTI gives you MUCH MORE than
a discriminated union, which is exactly the problem.
A proper discriminated unions offers RIGID guarrantees:

 a) a FINITE set of KNOWN and SPECIFIED types
 b) a compiler error if you access the wrong type
    (provided you do not change the type inside
    a case of another type)
 c) a compiler error is you _miss_ a type case
 d) the ability to compose ANY types

With RTTI (ignoring the storage mamagement issues)

 a) there is no bound on the possible types
 b) you get a run time error on selecting the wrong type
 c) compile time diagnostics for incorrect use after selection
 d) no verification of completeness of case handling
 e) invasion: you CANNOT compose ANY type, only types
    with a common base

The importance of discriminated unions is that in MANY cases
the alternatives are finite -- a grammar is an example in
which there are exactly TWO types. A "binary tree" has exactly
four types (leaf, two branches, left branch only, right branch only)

The fact is that discriminated unions and structures are
of "equal" importance in programming and programming is not
possible without both any more than you can do logic with
"AND" but without "OR".

In fact, structures (composition) and unions (unification)
are the _fundamental_ constructions of imperative
programming as presented in the language of Category Theory,
in which these constructions correspond to things called
Products (cartesian product) and Sums (disjoint set union).

A finite sum turns out to be useful just like a finite
product: most structs and unions have finite members.

It is possible to build weak representations of
discriminated unions in C++ which do not fully enforce
the constraints. However this lack of enforcement and
the extreme clumbsiness of the constructions makes
the natural use of such dunions unattractive and perverts
coding styles to inappropriate use of inheritance.

It is particularly sad that in a language with type
inference (overloading, templates) there is no
coherent unification operation.
--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <D7zAzJ.EMo@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
Max Skaller) writes:

    [concerning: must `vector' be a template...]
|>  Yes. Which is why I'm right and James is wrong.
|> There's no requirement that any Standard Library entity I'm
|> aware of actually be what it is presented as being. The Standard
|> Library merely leverages off the core language to present a
|> specification and then permits any mechanism of implementation
|> at all which conforms to the required behaviour.

|>  For example "memcpy" does not have to be a function.
|> Some compilers generate code directly. The same applies to
|> say a vector<T> where T is memcpy-able and it applies
|> very much in particular <g> to vectors of bool where
|> major advantages may be gained by directly generating
|> code using machine instructions which manipulate bits.

This may be just a disagreement concerning terms.  The C standard
states explicitly that `memcpy' must be either a function or a macro;
the standard also defines how the function may be accessed in the case
where it is a macro (i.e.: the function must exist).

The standard does *not* say how the function must be implemented.  A
compiler is free to use "magic" to recognise the special case, and
inline the function if it wants.

Similarly, the C++ standard defines `vector' as a template.  In time,
I would imagine that most C++ compilers will use meta-magic to handle
this particular template in a special way.  Note that the name
`vector' *is* a reserved word.  In this sense, vector is different
from any template that a user can write, and the compiler `knows' the
semantics of the template, and is free to know this knowledge.  This
does not in any way stop it from being a template, however, anymore
than the right to use such knowledge concerning `memcpy' means that
`memcpy' is not a function.
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

In article <3o3jsi$44o@calum.csclub.uwaterloo.ca>,
Ross Ridge <rridge@calum.csclub.uwaterloo.ca> wrote:
>John Max Skaller <maxtal@Physics.usyd.edu.au> wrote:
>> As it stands, certain complexity requirements
>>cannot be enforced because they cannot be measured
>>or because they are limiting phenomena (which are inherently
>>untestable)
>
>So what in the standard can be enforced?  What exactly does
>"enforced" mean anyways?  The question I'm interested in is "is
>it conformant?" and the answer to that can only be "no" or "maybe".

 Normative rules of an ISO Standard are "enforceable"
in the sense that test methods exist which can determine
by experiment repeatable by different parties, that a particular
product is not conforming.

 Statements of the Standard fail to be normative
if they cannot be enforced because the Standard -- like any
standard -- is something against which measurements are taken.

 If it cannot be measured, it cannot be a normative
part of a Standard.

 Some Standards state what kinds of measurements are
permitted (e.g. C++) and other specify precisely minimal
test methods (like POSIX and many safety related Standards).

 In C++ what is measured is called "behaviour" --
of both the translator and the executing program.
Behaviour is defined as the sequence of calls to Standard
Library functions and accesses to volatile memory locations
(if I remember rightly).

 No permission is granted to measure performance or
memory utilisation.

 Here's a case in point: the ARM says something about
"not eliding global objects with side-effects even if they
are not refered to". No normative rule of the Standard
can require objects _without_ side effects being elided
because if there are no side effects there is no way to
tell if the object has been elided or not.

 Of course this matters for people building libraries.
The C++ Standard has nothing to say about libraries.
It speaks only of programs.

 Another example is the template instantiation request.
It actually has semantics and we do NOT want it to.
We want the normative semantics to be "none".
(Unfortunately where you instantiate a template _can_
make a difference)

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <D6ytsv.HJC@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
Max Skaller) writes:

    [You need discriminated unions for...]
|>  2) A grammar (or parse tree).

|>  Each node of a grammar is either a terminal or non-terminal.
|>  You can't process the grammar without knowing which is which.
|>  You have to build a tree of nodes. The node has to be
|>  a discrimniated union of the two kinds.

Hold on.  This can't be right.  Derived classes plus RTTI gives you
discriminated unions *but for* the size difference, which should only be an
issue if you're taking over storage management.

A.

--
Andrew Fitzgibbon (Research Associate),                     andrewfg@ed.ac.uk
Artificial Intelligence, Edinburgh University.               +44 031 650 4504
<a href=http://www.dai.ed.ac.uk/staff/personal_pages/andrewfg> Home Page </a>
                         "Never say there is no way" -- me.

In article <KANZE.95Apr27145146@slsvhdt.lts.sel.alcatel.de>,
James Kanze US/ESC 60/3/141 #40763 <kanze@lts.sel.alcatel.de> wrote:
>
>|>  Stl containers in the Standard Library are not templates
>|> because they do not "quack" like templates. Real templates
>|> "quack" because they have definitions consisting of examinable
>|> source code.
>
>Really.  I have the Rogue Wave components, complete with templates,
>delivered with my C++ compiler.  But there is no examinable source
>code.
>
>What makes you state that examinable source code is necessary?

 Simple. The WP says so. Specifically, a template is
a sequence of tokens contained in a translation unit parsed
as a template according to the rules of the grammar and
associated constraints and semantic rules.

 The behaviour of this entity called a template
is (or should be) defined in the CD.

 Any component not in this form -- or which was not
at some stage in this form -- is catgorically NOT a
function, template, or any other such thing described
in the CD. [Unless it is a library function,
which is not a function]

 The behviour of a C++ function is determined by
its definition which is a sequence of tokens.

 A Standard Library entity that quacks like a function
is NOT a function.

 Vendors have the freedom to supply things that
quack like functions behaviourly but are not in fact functions,
in the Standard Library.

 Same applies to templates.


>|>  Standard library STL containers are not templates
>|> because they do not have to have a definition, so there
>|> is no way to do the deduction.
>
>I beg to disagree.  According to the description I read, they *are*
>templates.  The working papers define them clearly as templates, e.g.:
>section 23.2.8 Template class vector.

 No, it doesn't. It _describes_ them as if they
were templates for convinence, and if it says anything
else it is wrong and must be fixed.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <DOUG.95Apr27094046@monet.ads.com> doug@monet.ads.com (Doug
Morgan) writes:

|> In article <KANZE.95Apr27145146@slsvhdt.lts.sel.alcatel.de> kanze@lts.sel.alcatel.de (James Kanze US/ESC 60/3/141 #40763) writes:
|> > In article <D7Iw22.HqA@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
|> > Max Skaller) writes:
|> >[...]
|> > |>  Standard library STL containers are not templates
|> > |> because they do not have to have a definition, so there
|> > |> is no way to do the deduction.
|> >
|> > I beg to disagree.  According to the description I read, they *are*
|> > templates.  The working papers define them clearly as templates, e.g.:
|> > section 23.2.8 Template class vector.  They are thus templates by
|> > definition. [...]

|> That sounds mildly depressing.  By my reading of the orginal STL
|> papers, containers were any objects with certain specified semantics.
|> Objects of particular templates could be examples of containers, but
|> were not all containers.  Under the definitions of the STL papers,
|> there could be many other true STL vector containers other than the
|> objects of any one particular class template that happens to be named
|> vector.

Sorry for the confusion.  I thought that John was talking about the
STL containers provided in the standard library.

There is no requirement, for example, that the containers passed to
the functions in the algorithmic section be templates.  Only that they
meet the requirements for the specific function.

There is a requirement that the vector type provided by the
implementation (that is, the type whose name is the reserved word
`vector') be a template.
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

In <ncmD7tHzr.17F@netcom.com>, ncm@netcom.com (Nathan Myers) writes:
>>In article <ncmD7DHFK.H8y@netcom.com>, Nathan Myers <ncm@netcom.com> wrote:
>>>If it quacks like a template, and instantiates like a template,
>>>it's a template.
>>>
>>>In particular, when I instantiate it,
>>>it had better use my operator<().  If it doesn't, that's
>>>not conforming; if it does I can count calls to it.
>
>In article <D7Iw22.HqA@ucc.su.oz.au>,
>John Max Skaller <maxtal@Physics.usyd.edu.au> wrote:
>> I agree. I modify my previous claim: I was not
>>thinking of counts of calls to user defined functions
>>but the more generic complexity requirements which
>>are not enforeable.
>>
>> What I mean is that requirements expressed in terms
>>of algorithmic limiting complexity cannot be enforced.
>>Requirements expressed as exact values or upper bounds
>>on calls to use defined functions can be.
>>
>> For example a requirement that "insert" be constant
>>time is not enforceable because it requires using
>>a clock to measure performance of an infinite sequences
>>of operations on increasingly large containers.
>
>Skaller's modified claim, I'm afraid, holds no more water than
>before.  Like Fergus Henderson's statement that "linear time
>complexity" is untestable, it's belied simply by counting
>all operations on the element type instantiated on.  If the
>number of accesses, comparisons, assignments, copy constructions,
>etc. exceeds linear behavior, the implementation is not conforming.
>
>None of this requires inspecting implementation code.  None
>of this requires "infinite" containers.  The reason is that
>it is impossible even in principle to show that an implementation
>conforms; all you can do, and all you need to do, is show that
>one does not.  For that I have already demonstrated that two
>small sequences are sufficient.
>
>Nathan Myers
>myersn@roguewave.com

Surely you do not mean to imply that measuring counts for two small test cases is
sufficient to establish non-linearity, as any two points will establish the constants
which determine the line.

Some curves appear linear within measurable precision over a signinficant span,
and yet diverge from the asymptote as they depart that span -- I fail to see that
a standard which requires "linear" behavior of a function, without specifying
test cases to exercise the entire range of the function, can be enforceable.


Tres Seaver                    tseaver@neosoft.com
MACRO Enterprises, Inc.        Vox: (713) 827-7273
Houston, Texas, USA            Fax: (&13) 827-7278

John Max Skaller <maxtal@Physics.usyd.edu.au> wrote:
> As it stands, certain complexity requirements
>cannot be enforced because they cannot be measured
>or because they are limiting phenomena (which are inherently
>untestable)

So what in the standard can be enforced?  What exactly does
"enforced" mean anyways?  The question I'm interested in is "is
it conformant?" and the answer to that can only be "no" or "maybe".

      Ross Ridge

--
 l/  //   Ross Ridge -- The Great HTMU, Ook                    +1 519 883 4329
[oo][oo]  rridge@csclub.uwaterloo.ca      http://csclub.uwaterloo.ca/u/rridge/
-()-/()/
 db  //

In article <KANZE.95Apr27145146@slsvhdt.lts.sel.alcatel.de> kanze@lts.sel.alcatel.de (James Kanze US/ESC 60/3/141 #40763) writes:
> In article <D7Iw22.HqA@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
> Max Skaller) writes:
>[...]
> |>  Standard library STL containers are not templates
> |> because they do not have to have a definition, so there
> |> is no way to do the deduction.
>
> I beg to disagree.  According to the description I read, they *are*
> templates.  The working papers define them clearly as templates, e.g.:
> section 23.2.8 Template class vector.  They are thus templates by
> definition. [...]

That sounds mildly depressing.  By my reading of the orginal STL
papers, containers were any objects with certain specified semantics.
Objects of particular templates could be examples of containers, but
were not all containers.  Under the definitions of the STL papers,
there could be many other true STL vector containers other than the
objects of any one particular class template that happens to be named
vector.

If the standardization process doesn't maintain this distinction in
some form, then we will be reduced to descriptions like "this object
presents the same external interface as an instantiation of the
(one-and-only, language-provided) STL template class vector," rather
than "this object is an STL vector."  We'll also lose the ability to
say "this is a vendor-supplied STL vector and these are various kinds
of user-supplied STL vectors."

Doug
----------
Doug Morgan, doug@ads.com
Booz-Allen & Hamilton
1500 Plymouth St.
Mountain View, CA 94043-1230
     (415) 960-7444
FAX: (415) 960-7500
----------

>In article <ncmD7DHFK.H8y@netcom.com>, Nathan Myers <ncm@netcom.com> wrote:
>>If it quacks like a template, and instantiates like a template,
>>it's a template.
>>
>>In particular, when I instantiate it,
>>it had better use my operator<().  If it doesn't, that's
>>not conforming; if it does I can count calls to it.

In article <D7Iw22.HqA@ucc.su.oz.au>,
John Max Skaller <maxtal@Physics.usyd.edu.au> wrote:
> I agree. I modify my previous claim: I was not
>thinking of counts of calls to user defined functions
>but the more generic complexity requirements which
>are not enforeable.
>
> What I mean is that requirements expressed in terms
>of algorithmic limiting complexity cannot be enforced.
>Requirements expressed as exact values or upper bounds
>on calls to use defined functions can be.
>
> For example a requirement that "insert" be constant
>time is not enforceable because it requires using
>a clock to measure performance of an infinite sequences
>of operations on increasingly large containers.

Skaller's modified claim, I'm afraid, holds no more water than
before.  Like Fergus Henderson's statement that "linear time
complexity" is untestable, it's belied simply by counting
all operations on the element type instantiated on.  If the
number of accesses, comparisons, assignments, copy constructions,
etc. exceeds linear behavior, the implementation is not conforming.

None of this requires inspecting implementation code.  None
of this requires "infinite" containers.  The reason is that
it is impossible even in principle to show that an implementation
conforms; all you can do, and all you need to do, is show that
one does not.  For that I have already demonstrated that two
small sequences are sufficient.

Nathan Myers
myersn@roguewave.com

In article <D7Iw22.HqA@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
Max Skaller) writes:

|> In article <ncmD7DHFK.H8y@netcom.com>, Nathan Myers <ncm@netcom.com> wrote:
|> >
|> >> They _would_ be were STL components required
|> >>to be actual templates <they're not> with sources
|> >>which vendors had to submit to conformance testers
|> >><which is not a requirement of the proposed Standard>.
|> >
|> >If it quacks like a template, and instantiates like a template,
|> >it's a template.

|>  Stl containers in the Standard Library are not templates
|> because they do not "quack" like templates. Real templates
|> "quack" because they have definitions consisting of examinable
|> source code.

Really.  I have the Rogue Wave components, complete with templates,
delivered with my C++ compiler.  But there is no examinable source
code.

What makes you state that examinable source code is necessary?

|>  Standard library STL containers are not templates
|> because they do not have to have a definition, so there
|> is no way to do the deduction.

I beg to disagree.  According to the description I read, they *are*
templates.  The working papers define them clearly as templates, e.g.:
section 23.2.8 Template class vector.  They are thus templates by
definition.  (I.e.: if the standards committee were to adapt a rule
requiring the sources of templates to be available and human-readable,
then automatically, a conforming implementation must make the sources
of vector available in a human-readable form.  In fact, the standards
committee has not adapted such a rule, and I hold it for highly
unlikely that they will adapt such a rule.)
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

In article <9511222.24198@mulga.cs.mu.OZ.AU>,
Fergus Henderson <fjh@munta.cs.mu.OZ.AU> wrote:
>>
>> I'm sorry Nathan but you are not right here.
>
>Actually I agree with Nathan!  At least with respect to those complexity
>requirements that are expressed in terms of counts.

 Agreed. I wasn't thinking in terms of "counts" of calls
to user supplied function arguments to algorithms. I adjust
my position accordingly. Such specification can be normative.

>(As I outlined
>in another post, I still disagree with respect to those complexity
>requirements that are stated as an operation being "linear", "constant
>time"", etc.)

 So it seems we agree.

>> There is no requirement that vendors supply
>>C++ source code for STL parts of the C++ Standard for the
>>inspection of conformance testers.
>
>Correct.
>
>> Experimental measurements cannot either confirm or
>>deny compliance.
>
>They can't confirm compliance, but they in some circumstances can deny
>it.  For example, when instantiating a standard library template with
>a user-defined type, you can put code in your copy constructors and
>comparison operators which counts the number of times they are invoked,
>and compare this with the number of times the standard specifies.

 It is not clear you are correct in the case of copy
constructors. I agree in the case of comparisons though.

[The issue of copy constructors relates to an issue raised
by you (Fergus) in the Australian National Body Comments on CDR
which should be followed up -- when exactly are temporaries
created and copy constructors called?]
--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <ncmD7DHFK.H8y@netcom.com>, Nathan Myers <ncm@netcom.com> wrote:
>
>> They _would_ be were STL components required
>>to be actual templates <they're not> with sources
>>which vendors had to submit to conformance testers
>><which is not a requirement of the proposed Standard>.
>
>If it quacks like a template, and instantiates like a template,
>it's a template.

 Stl containers in the Standard Library are not templates
because they do not "quack" like templates. Real templates
"quack" because they have definitions consisting of examinable
source code.

>In particular, when I instantiate it,
>it had better use my operator<().  If it doesn't, that's
>not conforming; if it does I can count calls to it.

 I agree. I modify my previous claim: I was not
thinking of counts of calls to user defined functions
but the more generic complexity requirements which
are not enforeable.

 What I mean is that requirements expressed in terms
of algorithmic limiting complexity cannot be enforced.
Requirements expressed as exact values or upper bounds
on calls to use defined functions can be.

 For example a requirement that "insert" be constant
time is not enforceable because it requires using
a clock to measure performance of an infinite sequences
of operations on increasingly large containers.

 Limits of sequences cannot be measured,
only deduced from the definition.

 Standard library STL containers are not templates
because they do not have to have a definition, so there
is no way to do the deduction.

 We could require that the STL containers actually
be templates and then such deduction would be possible.
This would be interesting in that it would be possible
to confirm (or deny) compliance unequivocably, rather than
the usual case where compliance can not be positively
confirmed, only denied.

 As it stands, certain complexity requirements
cannot be enforced because they cannot be measured
or because they are limiting phenomena (which are inherently
untestable)


--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

kanze@us-es.sel.de (James Kanze US/ESC 60/3/141 #40763) writes:

>In all of the grammars I've written, I've found polymorphism
>(derivation) to be the ideal solution here.  In fact, back when I
>still worked in C, my standard solution was to have each node type
>start with a pointer to a node descriptor.  Typically, the node
>descriptor contains a variety of information; for anything complex
>(say calculate the Sethi-Ulmann number), it contained a pointer to a
>function.  Given this structure in C, you can imagine how quickly I
>switched to C++.

Could you explain in a bit more detail?  In your C++ code, do you have
a class for each node type, and bunch of virtual functions for the
operations on nodes, or is it vice versa?  How do you organize your
source files - one file per operation, or one file per node?  When you
come to add a new operation, how many source files do you have to
modify?  How many do you have to recompile?  How about when adding a
new node?  Is the tree traversal code reused or duplicated?

(I'm genuinely curious.)

--
Fergus Henderson            | Tell you what: go write a 100x100 matrix multiply
fjh@cs.mu.oz.au             | of integers in both languages and then let's talk
http://www.cs.mu.oz.au/~fjh | about speed, ok? - Tom Christiansen.

maxtal@Physics.usyd.edu.au (John Max Skaller) writes:
>Nathan Myers <ncm@netcom.com> wrote:
>>Fergus Henderson <fjh@munta.cs.mu.OZ.AU> wrote:
>>>I think it's clear that complexity requirements are unenforceable.
>>
>>On the contrary.  The complexity specifications are in terms
>>of the number of operations -- calls to copy constructors
>>and comparison operators -- which can be counted.
>
> I'm sorry Nathan but you are not right here.

Actually I agree with Nathan!  At least with respect to those complexity
requirements that are expressed in terms of counts.  (As I outlined
in another post, I still disagree with respect to those complexity
requirements that are stated as an operation being "linear", "constant
time"", etc.)

> There is no requirement that vendors supply
>C++ source code for STL parts of the C++ Standard for the
>inspection of conformance testers.

Correct.

> Experimental measurements cannot either confirm or
>deny compliance.

They can't confirm compliance, but they in some circumstances can deny
it.  For example, when instantiating a standard library template with
a user-defined type, you can put code in your copy constructors and
comparison operators which counts the number of times they are invoked,
and compare this with the number of times the standard specifies.

--
Fergus Henderson            | Tell you what: go write a 100x100 matrix multiply
fjh@cs.mu.oz.au             | of integers in both languages and then let's talk
http://www.cs.mu.oz.au/~fjh | about speed, ok? - Tom Christiansen.

In article <DOUG.95Apr20094823@monet.ads.com> doug@monet.ads.com (Doug
Morgan) writes:

|> In article <D7BtJy.7EG@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John Max Skaller) writes:
|> > ...
|> >  In particular, this is kind of tricky to do and
|> > I suspect there is an "error" in the current WP:
|> > a dereferenced iterator to a container of T does NOT
|> > have to be an lvalue of type T (even if the WP says that).

|> This is certainly an error.  I remember the original STL documents as
|> being pretty clear that iterators had to dereference to something that
|> is convertible to a T.  There was a similar error (to the one
|> apparently now in the WP) in table 10 of the May 31, 1994 version of
|> the STL document.  It specified that sequence's a.front(), a.back(),
|> and a[n] return references.  Of course this was an oversight (and even
|> the example vector<bool> did not return references).  The next version
|> of the document was corrected.  Hopefully, the WP can be corrected
|> soon, too.

This is an important point, since it is a decernable difference.  The
"as if" rule cannot be used to allow the iterator to dereference to an
object which converts to an lvalue of type T, since this would
introduce a user defined conversion.
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

In article <19950419.194958.67@morpheus.demon.co.uk>
gustav@morpheus.demon.co.uk (Paul Moore) writes:

|> In message <KANZE.95Apr19150630@slsvhdt.us-es.sel.de> James Kanze US/ESC 60/3/141
|> #40763 wrote:

|> > In article <9510113.6681@mulga.cs.mu.OZ.AU> fjh@munta.cs.mu.OZ.AU
|> > (Fergus Henderson) writes:
|> >
|> > |> Yes.  I agree entirely that discriminated unions are the right way of
|> > |> handling this sort of thing.  The only problem is that C++ doesn't support
|> > |> discriminated unions very well.  If you follow this argument to it's
|> > |> logical conclusion, you have to say that even null pointers were a mistake.
|> >
|> > Aren't they?  I've stopped using them, in favor of `Fallible' (e.g.: a
|> > function will return `Fallible< char* >', rather than simply `char*',
|> > if it can fail).
|> >

|> I've never seen "Fallible". Can you point me to a description, and
|> possibly an implementation?

Barton and Nackmann.  I don't have my copy here, so I cannot give the
exact page number, but it is in the index.

The idea is really pretty simple.  The class `template< class T >
class Fallible' associates a bool defining the validity with a value
of type T.  The default constructor constructs an `invalid' Fallible,
a constructor from `T const&' constructs a valid Fallible.  Assigning
a T to a Fallible also makes it valid.

The class contains a conversion operator for T, which throws an
exception (or has an assertion failure) if the value is not valid.
There is also a function to test validity.

As a simple example, imagine strchr using Fallible:

 Fallible< char* >
 strchr( char* p , char c )
 {
     Fallible< char* >   result ;
     while ( ! result.isValid() && *p != '\0' )
         if ( *p == c )
             result = p ;
     return result ;
 }

In a simple case like the above, a null pointer does an acceptible
job, and will certainly be more efficient in terms of runtime.  But in
the more general case, Fallible is probably more understandable (since
it doesn't require a `unique' value), and the efficiency differences
will be almost negligible.

Even in the above case, Fallible has the advantage that attempting to
use the pointer returned by strchr in the case where the character was
not found will result in an exception, rather than undefined behavior.

The following code is off the top of my head, so some of the details
may not be right, but it should be enough to get the general idea:

 template< class T >
 class Fallible
 {
 public :
                         Fallible() ;
                         Fallible( T const& val ) ;
     Fallible< T >&      operator=( T const& val ) ;

     bool                isValid() const ;
                         operator T() const ;
 private :
     T                   value ;
     bool                valid ;
 } ;

 template< class T >
 Fallible< T >::Fallible()
     :   valid( false )
 {
 }

 template< class T >
 Fallible< T >::Fallible( T const& val )
     :   value( val )
     ,   valid( true )
 {
 }

 template< class T >
 Fallible< T >&
 Fallible< T >::operator=( T const& val )
 {
     value = val ;
     valid = true ;
 }

 template< class T >
 bool
 Fallible< T >::isValid() const
 {
     return valid ;
 }

 template< class T >
 Fallible< T >::operator T() const
 {
     assert( valid ) ;
     return value ;
 }
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

>>In article <9510023.17400@mulga.cs.mu.oz.au>,
>>Fergus Henderson <fjh@munta.cs.mu.OZ.AU> wrote:
>>>I think it's clear that complexity requirements are unenforceable.

>In article <ncmD6wpGv.4p1@netcom.com>, Nathan Myers <ncm@netcom.com> wrote:
>>On the contrary.  The complexity specifications are in terms
>>of the number of operations -- calls to copy constructors
>>and comparison operators -- which can be counted.

In article <D7BsGI.3KB@ucc.su.oz.au>,
John Max Skaller <maxtal@Physics.usyd.edu.au> wrote:
> I'm sorry Nathan but you are not right here.
>
> There is no requirement that vendors supply
>C++ source code for STL parts of the C++ Standard for the
>inspection of conformance testers.

Source code is irrelevant. The question is, does the
implementation behave as specified?

> Experimental measurements cannot either confirm or
>deny compliance.  Hence ...the requirements are not enforceable.

Of course they can.  If I run an experiment in which
sort() fails to sort elements, the implementation doesn't
conform.  If it calls operator<() too many times, it
doesn't conform.  I've already outlined what "too many
times" means, and it's the conventional definition.

Of course it's not possible to confirm compliance in any case,
as I've (also) already outlined.

> They _would_ be were STL components required
>to be actual templates <they're not> with sources
>which vendors had to submit to conformance testers
><which is not a requirement of the proposed Standard>.

If it quacks like a template, and instantiates like a template,
it's a template.  In particular, when I instantiate it,
it had better use my operator<().  If it doesn't, that's
not conforming; if it does I can count calls to it.

> Do you desire to change the conformance requirements
>of the proposed C++ Standard so that in fact the complexity
>requirements are normative (enforceable)?

No, I find the present form entirely satisfactory, thank you.

kanze@us-es.sel.de (James Kanze US/ESC 60/3/141 #40763) writes:

>> fjh@munta.cs.mu.OZ.AU (Fergus Henderson) writes:
>>
>> |> If you follow this argument to it's logical conclusion,
>> |> you have to say that even null pointers were a mistake.
>>
>> Aren't they?  I've stopped using them, in favor of `Fallible'

An optimized FalliblePtr (you could make it a template specialization
for Fallible<T*>, I guess) and conditional compilation could give you
optimal performance when you needed it and guaranteed error-checking
when you don't.  I think something like that is a good idea, although I
don't use it myself.  But there are some draw-backs: you probably run
into the "only one user-defined conversion" limit sooner; of course
compilation time is worse; and some C++ compilers don't do a good job of
inlining templates.  In C++, pointers are the way everyone does things.
I prefer not to rock the boat so much unless the benefit is large.

BTW, similar techniques have been around in other languages for
a long time.  There is a type in the ML standard library, I think
think it is called "maybe", which does a similar job to "Fallible".
Of course the details are different.  I think the ML definition
is one line of code rather than 50, although perhaps it doesn't
provide quite as much functionality.

--
Fergus Henderson            | Tell you what: go write a 100x100 matrix multiply
fjh@cs.mu.oz.au             | of integers in both languages and then let's talk
http://www.cs.mu.oz.au/~fjh | about speed, ok? - Tom Christiansen.

In article <ncmD6wpGv.4p1@netcom.com>, Nathan Myers <ncm@netcom.com> wrote:
>In article <9510023.17400@mulga.cs.mu.oz.au>,
>Fergus Henderson <fjh@munta.cs.mu.OZ.AU> wrote:
>>I think it's clear that complexity requirements are unenforceable.
>
>On the contrary.  The complexity specifications are in terms
>of the number of operations -- calls to copy constructors
>and comparison operators -- which can be counted.

 I'm sorry Nathan but you are not right here.

 There is no requirement that vendors supply
C++ source code for STL parts of the C++ Standard for the
inspection of conformance testers.

 Experimental measurements cannot either confirm or
deny compliance.

 Hence Fergus is right -- the requirements are not
enforceable.

 They _would_ be were STL components required
to be actual templates <they're not> with sources
which vendors had to submit to conformance testers
<which is not a requirement of the proposed Standard>.

 Do you desire to change the conformance requirements
of the proposed C++ Standard so that in fact the complexity
requirements are normative (enforceable)?

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <ncmD6wpGv.4p1@netcom.com>, Nathan Myers <ncm@netcom.com> wrote:
>In article <9510023.17400@mulga.cs.mu.oz.au>,
[enforcement of STL complexity requirements ..]

>This "unenforceable" myth keeps popping up, and I don't know why.

 The reason  is a large number of committee members
and non-members -- including if not especially those on the
Library Working Group -- do not seem to understand that the
C++ Standard Library is NOT a user defined library provided
as sources, compiled object modules, or whatever, but is
an intrinsic and inalienable part of the C++ core language.

 Just because such components are characterised in terms
of the C++ language itself "as if" they were user written
does not mean that they must be.

 That is, there is no requirement any library component
have, or ever did have, a declaration, definition, or any other
such thing associated with all user defined types and functions.

 This is very important for two reasons -- the first
is optimisation, compilers can do things with Standard Library
components that would be impossible with user defined entities.

 The second reason is that some library entities
cannot possibly be written as portable user defined entities,
and others -- such as malloc -- which can be often are not.

 The notion -- in the proposed C++ Standard  -- and
most other languages -- of a "standard library" is merely
a way of leveraging the description of the formal semantics
of the entities described off the rest of the "core" language.

 In fact, the STL description (of the original paper)
takes some pains to describe behaviour in terms of the syntax
used rather than the entities that such usages denotes.
For example an "iterator" is a piece of syntax with certain
properties.

 In particular, this is kind of tricky to do and
I suspect there is an "error" in the current WP:
a dereferenced iterator to a container of T does NOT
have to be an lvalue of type T (even if the WP says that).

 It is only necessary it look like one, it may
in fact be an object that converts to one on any use,
but is not in itself such an lvalue.

 The example in mind is an iterator to a vector
of bool specialised to use 1 bit per value. The dereferenced
iterator might well be an rvalue of some handle class
which will accept any valid syntax that might be applied
to a bool lvalue -- but actually isn't such an lvalue.

 Or, the iterator may just be notation the compiler
converts _directly_ into appropriate machine instructions.

 In fact, the WP description can be rescued by
noting that entities denoted by STL Standard Library
components do not have to be actual classes or even types:
all that is required is that the _look as if_ they are.

 This makes it important NOT to require
these components be actual classes or types -- and indeed
at present the conformance model cannot require that
even if we wanted it to or actually said it.

 [And all of this clearly excludes any possibility
of complexity requirements being normative :-)


--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

ncm@netcom.com (Nathan Myers) writes:

>In article <9510023.17400@mulga.cs.mu.oz.au>,
>Fergus Henderson <fjh@munta.cs.mu.OZ.AU> wrote:
>>I think it's clear that complexity requirements are unenforceable.
>
>On the contrary.  The complexity specifications are in terms
>of the number of operations -- calls to copy constructors
>and comparison operators -- which can be counted.

In the current draft, the complexity of the various STL algorithms
is specified in terms of counts, but the complexity of the iterator
and container operations is simply specified as "constant time (amortized)",
"linear", etc.

I agree with you in part, and must retract part of my claim.
For the algorithms, where the draft states "Complexity: Exactly <n>
applications of the corresponding predicate." or "Complexity: At most
<n> assignments" or something similar, the complexity requirements are
enforceable.  But for containers and iterators, where the draft just
states that the complexity is "linear" or "constant", etc., the complexity
requirements are not enforceable.

>The "constant
>factor" can be bounded simply by requiring (for instance) that
>a list of 2N elements take no more than twice as long to
>process as a list of N elements, for a linear algorithm; no
>more than four times as long, for a quadratic algoritm; and
>so on.

The standard _could_ require that, but unless I am missing something,
it currently doesn't.  To do so would be contrary to the usual meaning
of "linear" and "quadratic". Furthermore, such a requirement
would make the implementations that I have seen non-conforming!
Memory hierarchy effects mean that there is almost invariably
some N for which processing a list of 2N elements thrashes the
cache or main memory, causing it to take considerably more than
twice as long than for process a list of N elements, even though
the algorithm is linear.

>This "unenforceable" myth keeps popping up, and I don't know why.
>It's easy to show that a library doesn't meet such a requirement
>for some input [...]

That is only true if complexity requirements are stated in terms
of maximum number of invokations of some operation, which is true
of only some of the complexity requirements in the current draft.

Furthermore, the sort of complexity requirements that are enforceable
don't actually give you any guarantees about the actual time
requirements of the program.

[Note: all the above does not mean that I think complexity requirements
in the standard are not a good idea.  I think they are a great idea!
For all practical purposes, they achieve their aim.  But I don't think
we should be deluded into thinking that they prevent peversely low-quality
implementations from executing our programs as slowly as they want.]

--
Fergus Henderson            | As practiced by computer science, the study of
fjh@cs.mu.oz.au             | programming is an unholy mixture of mathematics,
http://www.cs.mu.oz.au/~fjh | literary criticism, and folklore. - B. A. Sheil

In article <D7BtJy.7EG@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John Max Skaller) writes:
> ...
>  In particular, this is kind of tricky to do and
> I suspect there is an "error" in the current WP:
> a dereferenced iterator to a container of T does NOT
> have to be an lvalue of type T (even if the WP says that).

This is certainly an error.  I remember the original STL documents as
being pretty clear that iterators had to dereference to something that
is convertible to a T.  There was a similar error (to the one
apparently now in the WP) in table 10 of the May 31, 1994 version of
the STL document.  It specified that sequence's a.front(), a.back(),
and a[n] return references.  Of course this was an oversight (and even
the example vector<bool> did not return references).  The next version
of the document was corrected.  Hopefully, the WP can be corrected
soon, too.

> ...
>  This makes it important NOT to require
> these components be actual classes or types -- and indeed
> at present the conformance model cannot require that
> even if we wanted it to or actually said it.

All correct.  I hope that STL as an abstract interface standard isn't
wiped out (with the best of misguided intentions) by committees that
don't understand what they are standardizing.

Doug
----------
Doug Morgan, doug@ads.com
Booz-Allen & Hamilton
1500 Plymouth St.
Mountain View, CA 94043-1230
     (415) 960-7444
FAX: (415) 960-7500
----------

In article <9510113.6681@mulga.cs.mu.OZ.AU> fjh@munta.cs.mu.OZ.AU
(Fergus Henderson) writes:

|> Yes.  I agree entirely that discriminated unions are the right way of
|> handling this sort of thing.  The only problem is that C++ doesn't support
|> discriminated unions very well.  If you follow this argument to it's
|> logical conclusion, you have to say that even null pointers were a mistake.

Aren't they?  I've stopped using them, in favor of `Fallible' (e.g.: a
function will return `Fallible< char* >', rather than simply `char*',
if it can fail).

The one advantage of null pointers in this regard is that most
compilers will return a single pointer in a register, whereas I know
of no compiler which currently optimizes class type return values into
a register when they fit.  (And I will be the first to admit that I
don't expect to see compilers returning a Fallible in registers,
although such an optimization is both possible and useful.)

Since this advantage is not valid for class types, I see *no* reason
to try and perpetuate this hack for iterators.  (Not to mention John's
point that you often need more than one error code.)
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

In article <3m55ss$fl2@eclipse.eng.sc.rolm.com>
eddy@clipper.robadome.com (eddy Gorsuch) writes:

|> OK, I think I understand what you are trying to do.
|> You want your iterator to serve two purposes:
|> 1. Be an iterator
|> 2. Be an indication that some routine failed miserably.

|> I agree with John Max Skaller that you should really be returning 2
|> different results. Since you don't want to use exceptions here, could you
|> change your Dicts::look_up_word() to return a
|> pair<error_indicator, Dicts::iterator> instead of just an iterator?

Agreed, 100%.  I would strongly recommend reading Barton and Nackmann.
In particular, look at their `Fallible' class.
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

In article <D6ytsv.HJC@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John
Max Skaller) writes:

    [Concerning discriminated unions...]
|>  Here are two examples where you need them:

|>  1) A transaction processing system.

|>  There are a finite set of transactions.
|>  No polymorphism is involved, you have to process each
|>  one differently according to fixed rules dependent on
|>  the type.

The type of the transaction, or the type of the object.  Generally, I
suspect both.  Which means multiple dispatch (not discriminate
unions).  Although C++ doesn't support multiple dispatch, there is
actually a fairly good work-around for the case where the number of
variants of one of the objects is closed, as is the case here.

|>  The effects of a transaction permeates the system wholistically,
|>  and cannot be encapsulated into a single notion.

|>  2) A grammar (or parse tree).

|>  Each node of a grammar is either a terminal or non-terminal.
|>  You can't process the grammar without knowing which is which.
|>  You have to build a tree of nodes. The node has to be
|>  a discrimniated union of the two kinds.

Typically, you have many more than two types of nodes.

In all of the grammars I've written, I've found polymorphism
(derivation) to be the ideal solution here.  In fact, back when I
still worked in C, my standard solution was to have each node type
start with a pointer to a node descriptor.  Typically, the node
descriptor contains a variety of information; for anything complex
(say calculate the Sethi-Ulmann number), it contained a pointer to a
function.  Given this structure in C, you can imagine how quickly I
switched to C++.
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

In message <KANZE.95Apr19150630@slsvhdt.us-es.sel.de> James Kanze US/ESC 60/3/141
#40763 wrote:

> In article <9510113.6681@mulga.cs.mu.OZ.AU> fjh@munta.cs.mu.OZ.AU
> (Fergus Henderson) writes:
>
> |> Yes.  I agree entirely that discriminated unions are the right way of
> |> handling this sort of thing.  The only problem is that C++ doesn't support
> |> discriminated unions very well.  If you follow this argument to it's
> |> logical conclusion, you have to say that even null pointers were a mistake.
>
> Aren't they?  I've stopped using them, in favor of `Fallible' (e.g.: a
> function will return `Fallible< char* >', rather than simply `char*',
> if it can fail).
>

I've never seen "Fallible". Can you point me to a description, and
possibly an implementation?

Gustav.

--
------------------------------------------------------------------------
Paul Moore                                   gustav@morpheus.demon.co.uk
------------------------------------------------------------------------

... Pets just die on you, where's the fun in that?

In article <FENSTER.95Apr4213620@ground.cs.columbia.edu>
fenster@ground.cs.columbia.edu (Sam Fenster) writes:

|> It's often good design to let an object have an `invalid' state.

Is it?  It's often an acceptable compromize, which will allow
returning a built-in type in a register, rather than a class.  (With
most compilers, returning a class will result in extra overhead, even
if the class would fit in a register easily.)  But it tends to be
limited, and to cause problems in the long run.

Consider the simple case of the Unix system call: signal.  Its second
parameter is a pointer (to a function which handles the signal).  It
uses a special value to signal the default handling, *and* a second
special value to signal ignore.  Oops, maybe we need both NULL and
NULL2 values, and not just NULL.
--
James Kanze         Tel.: (+33) 88 14 49 00        email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
Conseils en informatique industrielle --
                              -- Beratung in industrieller Datenverarbeitung

In article <D6ytsv.HJC@ucc.su.OZ.AU>, John Max Skaller says...

>        1) A transaction processing system.
>
>        There are a finite set of transactions.
>        No polymorphism is involved, you have to process each
>        one differently according to fixed rules dependent on
>        the type.

I see. You want polymophism, but you don't want to CALL it polymophism.
Suit yourself, I guess.

>Note that inheritance and polymorphism are exactly the
>wrong thing to use when unification and discrimination
>is required -- the whole point of an abstract type
>with polymorphic subtypes is that the subtypes ARE
>the supertype by definition.

Subtypes ARE the supertype???!!! I hope this is just a typo!

--
Paul A. Perkins
perkinsp@nando.net
(All things are Fire)

In article <3mmpph$iae@highway.LeidenUniv.nl>,
Jan-Peter de Ruiter <ruiter@ruls41.LeidenUniv.nl> wrote:
>John Max Skaller (maxtal@Physics.usyd.edu.au) wrote:
>
>: The key idea is that you have to "extend" STL.
>: (Actually, you are not extending it but introducing a "subtype")
>: That does not mean adding a template class. It means
>: adding extra PROTOCOL. STL already shows how to specify
>: protocol. It is a good model. Learning how to extend it
>: is a whole new ballgame.
>:
>
>Sure, but if everyone is going to write his or her own 'safe'
>layer on STL, we are not having a standard anymore.

 Do you see a way around this?

 We standardise C++, everyone is going to write their
own programs. They're not "standard". The direct advantage to the
reusable components market is  zero.

 We standardise a set of 10 library components,
everyone uses them differently and continues to invent their own.
The advantage to the reusable components market is 10.

 We standardise a _protocol_ like STL and at least
there is an opportunity for YOUR layers and extensions
to cooperate with mine. At worst your LAYER containers
and iterators will still work with STL itself. So I can
use them even if I do not use your protocol. Remember,
your extensions still have to be STL compliant.

 Advantage to reusable components industry?
What a silly question. This CREATES a reusable components industry
which cannot possibly exist with a mere 10 standard components or
just a standardised language.

 Example: there is a PD hash table extension of STL.
Will you write your own or just use it? I will at least
_try_ it out.

 Is there work to be done, and products to produce which
may succeed or fail? Yes of course.

 How successful will STL be? It remains to be seen.

 I can tell you it has improved my own productivity
enormously -- and I'm only just starting to use it.

>I still don't see how an important library like STL can allow
>the following code to compile and run without even as much as a
>warning:
>
>#include <iostream.h>
>#include <list.h>
>
>int array [] = { 1,2,3,4,5 };
>
>int main ()
>{
>  list<int> l1 (array, array + 5);
>  list<int>::iterator i1 = l1.begin ();
>  while (i1 != l1.end ())
>    cout << *i1++ << endl;
>  int test = *i1; // i1 is now pointing at random nonsense
>  cout << test; // OK, on _some_ systems test will be 0.
>  return 0;
>}
>

 Let me turn your question around. I don't see how
an important library like STL can _prevent_ the code above
from compiling, or even crashing, without imposing a performance penalty
on code not containing such errors.

 However, I'm told by Alex the idea is that STL based
code might be statically checked by tools.

 C++ as a language is not capable of that. Remember in
modern terms, it isn't a very secure language in the first place.

 So as more and more STL code is written, and more
and more people discover particular classes of bugs that are
common -- and experts begin to discover how to recognize these
bugs -- then you can expect tools to be developed to help
you write safer code. (And changes to the C++ language eventually
to accomodate this)

----------------------------------------------------------------
Now let me make a suggestion. Rewrite your code:

  {for
  (
     list<int>::iterator i1 = l1.begin ();
     i1 != l1.end ();
     i1++
  )
  {
    cout << *i1 << endl;
  }}
  int test = *i1; // i1 is now pointing at random nonsense
  // COMPILER ERROR: i1 not declared

When your compiler supports proper scoping of conditionals,
you won't need the { and } around the for statement.

This is an _indication_ of the kind of thing needed for
safe programming -- unfortunately C++ will never
have constructions properly designed to support correctness,
it is based on C which is far too archaic, and was designed
for hand optimisation rather than verification.

However, it is generally known for statements are superior
to while statements because the iteration control is all
in one place. In Pascal they're even better, since you
cannot modify the control variable at all in the body,
and because the increment and compare are generated
correctly by the compiler. That is, in Pascal, for
statements are syntactically guarranteed to be secure.

Similarly, array index violations are impossible in Pascal.
Of course, it _is_ possible to get an error converting
a type to a subrange. So the advantages are small.

However there is a body of theory on checking these things.
I'm told by a professor at QUT that about 90% of all
this class of error can be detected in typical programs.
That is given some integer calculations

 var i : 10..20;
      j : 0..99;
 ...
 j += i + i*i;
 ...
 array[j] ...

it is possible to compute the possible range of values
so that required run time checks can be elided because
it can be proved they will always succeed.

This kind of work indicates that with appropriate language
constructions and coding styles, checked conversions
may be possible without significant loss of efficiency
in future languages. (Well, in this case the work is
being done for Modula II, which is an ISO Standard language)

The point? What you want is being worked on but it is HARD.
There is no magic.

In the end, "correctness" cannot be assured. Testing
is never going to go out of style :-)

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

John Max Skaller (maxtal@Physics.usyd.edu.au) wrote:
: In article <3mmpph$iae@highway.LeidenUniv.nl>,
: Jan-Peter de Ruiter <ruiter@ruls41.LeidenUniv.nl> wrote:
: >John Max Skaller (maxtal@Physics.usyd.edu.au) wrote:
: >
: >: The key idea is that you have to "extend" STL.
: >: (Actually, you are not extending it but introducing a "subtype")
: >: That does not mean adding a template class. It means
: >: adding extra PROTOCOL. STL already shows how to specify
: >: protocol. It is a good model. Learning how to extend it
: >: is a whole new ballgame.
: >:
: >
: >Sure, but if everyone is going to write his or her own 'safe'
: >layer on STL, we are not having a standard anymore.

[a few lines deleted]

:  We standardise a _protocol_ like STL and at least
: there is an opportunity for YOUR layers and extensions
: to cooperate with mine.

So it is a PROTOCOL. No wonder some people thought that STL had
less than they expected. They expected a LIBRARY. But what you say
makes a lot of sense. If you want to templatize, you need a protocol
so that the string replacement in the templates yields something
that actually instantiates. That explains the crazy names, like
push_back to append an element to the end of a list.

A few people had disagreed with the "T" in STL, suggesting that
"standard container library" would have been a better choice. It seems
the "T" is wholly appropriate, but the "L" should have been a "P"
--the Standard Template Protocol.

Cay

In article <0iJjvA2NBh107h@double.actrix.gen.nz>,
Chris Double <chris@double.actrix.gen.nz> wrote:
>In <9510113.6681@mulga.cs.mu.OZ.AU> fjh@munta.cs.mu.OZ.AU (Fergus Henderson) writes:
>>Yes.  I agree entirely that discriminated unions are the right way of
>>handling this sort of thing.  The only problem is that C++ doesn't support
>>discriminated unions very well.  If you follow this argument to it's
>>logical conclusion, you have to say that even null pointers were a mistake.
>
>What are discriminated unions? At a guess it sounds like a union of types
>and some sort of identifier saying which type is actually stored in the
>union. Is this correct?

 In a nutshell, yes.
>
>What sort of area you would use this in?

 Everywhere. The notion is as general and important
as inheritance and of equal utility. It is like AND and OR
of logic. The idea of not having both is just absurd.

>I remember John Skaller saying
>in a previous post that people don't use them enough. Where should they
>be used? Is there an idiom that you can use in C++ to support
>discriminated unions?

 No, there isn't. Not really. So when you need them,
it is very hard to get around this problem. dynamic_cast
with a dummy "object" works, but is invasive.

 Here are two examples where you need them:

 1) A transaction processing system.

 There are a finite set of transactions.
 No polymorphism is involved, you have to process each
 one differently according to fixed rules dependent on
 the type.

 The effects of a transaction permeates the system wholistically,
 and cannot be encapsulated into a single notion.

 2) A grammar (or parse tree).

 Each node of a grammar is either a terminal or non-terminal.
 You can't process the grammar without knowing which is which.
 You have to build a tree of nodes. The node has to be
 a discrimniated union of the two kinds.

Put it this way: subtyping (nice inheritance) takes a subset,
it is the AND operation. Discrimination is by polymorhism.

Unification is an OR operation, it makes a new type which is
the union of two others. Discrimination is subtraction --
a way of getting the original type of an object of a unified
type.

Type casing is a kind of selection on type like a switch.
Indeed when the type discriminant is represented as
testable state, this is how you do the discrimination.

In fact, one can view any subset of states of a type
as another type -- and so EVERY time you make a decision
in a program -- by a switch or an if/then/else -- you are
type casing on an implied type.

Discriminated unions allow "packaging" of these distinctions
with semantics for unification and discrimination.
C unions only permit unification, you have to do the
discrimination manually.

Note that inheritance and polymorphism are exactly the
wrong thing to use when unification and discrimination
is required -- the whole point of an abstract type
with polymorphic subtypes is that the subtypes ARE
the supertype by definition.

Discriminiated unions are the opposite: unification
is imposed afterwards, it isn't inherent.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

John Max Skaller (maxtal@Physics.usyd.edu.au) wrote:

: The key idea is that you have to "extend" STL.
: (Actually, you are not extending it but introducing a "subtype")
: That does not mean adding a template class. It means
: adding extra PROTOCOL. STL already shows how to specify
: protocol. It is a good model. Learning how to extend it
: is a whole new ballgame.
:

Sure, but if everyone is going to write his or her own 'safe'
layer on STL, we are not having a standard anymore.

I still don't see how an important library like STL can allow
the following code to compile and run without even as much as a
warning:

#include <iostream.h>
#include <list.h>

int array [] = { 1,2,3,4,5 };

int main ()
{
  list<int> l1 (array, array + 5);
  list<int>::iterator i1 = l1.begin ();
  while (i1 != l1.end ())
    cout << *i1++ << endl;
  int test = *i1; // i1 is now pointing at random nonsense
  cout << test; // OK, on _some_ systems test will be 0.
  return 0;
}

maxtal@Physics.usyd.edu.au (John Max Skaller) writes:

>Fergus Henderson <fjh@munta.cs.mu.OZ.AU> wrote:
>>
>>I think it's clear that complexity requirements are unenforceable.
>
> I don't think thats quite correct, in a sense I think
>you would agree with -- it is possible to determine in some cases
>what the complexity of a piece of code is.

My point was in fact it is possible to determine the complexity of
_all_ operations (that terminate and that don't involve I/O), but that
in fact worst-case complexity analysis if carried to extremes can give
you results which are simply not at all useful.  As soon as you try to
enforce them, you realize that for enforcement purposes the complexity
requirements are meaningless.

> The point is you have to do this by reading and analysing
>the code, it isn't something that is really "testable" in the sense
>of executing the code and recording and analysing behaviour.
>This would imply that in order to met test requirements, vendors
>would be _required_ to supply actual sources for, say, STL
>Standard Library entities.

An alternative would be to require implementations to document the
appropriate constants.  For example, if some operation was supposed to
be at worst linear in N, the implementation could be required to
document an N0 and a K such that the time for the operation was no
worse than K*max(N,N0) for all N.  That would be quite testable,
although the documentation requirements would probably be too onerous.
But it would not be at all useful, since implementations would supply
ridiculously high values for the appropriate constants.

BTW, the draft implies that the complexity of all operations on iterators
is amortized constant time.  What does this mean when applied to input
iterators, e.g. istream_iterator<char>(cin)?  If it's constant time,
what is the constant?

--
Fergus Henderson            | As practiced by computer science, the study of
fjh@cs.mu.oz.au             | programming is an unholy mixture of mathematics,
http://www.cs.mu.oz.au/~fjh | literary criticism, and folklore. - B. A. Sheil

chris@double.actrix.gen.nz (Chris Double) writes:

>What are discriminated unions? At a guess it sounds like a union of types
>and some sort of identifier saying which type is actually stored in the
>union. Is this correct?

Yes, roughly speaking, and that's one way of implementing discriminated
unions in C++.

Mathematically, if you consider types as sets of values,
then a discriminated union D of types T_1, T_2, ..., T_n is the set

 D = ({ tag_1 } x T_1) U ({ tag_2 } x T2 } U ... U ({ tag_n } x T_n)

where `tag_1' thru `tag_n' are distinct (but otherwise arbitrary) constants,
(`x' denotes cross-product, `U' denotes set union, and `{ X }' denotes
the set containing just X.)

Note that enumerations are equivalent to a special case of discriminated
unions, when all the T_i are the unit type.

Another way of implementing discriminated unions in C++ is with inheritence.
All the types T_1 ... T_n are derived from a common abstract base class.
An element of the discriminated union is represented by a pointer or
reference to the base class, and you use RTTI to determine which of
the different possibilities it actually refers to.

>What sort of area you would use this in?

Whenever a piece of data could be one of a finite, stable set of alternatives.

If it could be one of an unbounded or unstable set of alternatives, then
you should generally use subtyping implemented as inheritence from an
abstract base class (and in this case, you should generally _not_ use RTTI).

In languages with good support for discriminated unions, the most
commonly used examples are types like lists and trees.  For example, a
list of integers is either empty (`nil'), or it is constructed (`cons')
from an integer followed by another intlist:

 datatype intlist = nil
    | cons of int * intlist
    ;

I've borrowed SML syntax.  Note that the `*' in `int * intlist' is a
cross-product, i.e.  the type `int * intlist' is just a pair consisting
of an int and an intlist.

With the above definition, you can write expressions of type intlist
such as `nil', which represents the empty list, and
`cons(1, cons(2, cons(3, nil)))', which represents the list 1, 2, 3.
You can also write functions which act on this recursive data structure,
such as a function to compute the length of a list:

 fun length (list) =
  case list of
   nil =>
    0
   cons (head, tail) =>
    length (tail) + 1;

Another example is a 2-3 tree, which can contain nodes with either two
or three children:

 datatype tree = empty
   | two_node of tree * int * tree
   | three_node of tree * int * tree * int * tree
   ;

Another example, from a hypothetical card came:

 datatype suit = spades | clubs | diamonds | hearts ;
 datatype card = card of suit * rank | joker ;

--
Fergus Henderson            | As practiced by computer science, the study of
fjh@cs.mu.oz.au             | programming is an unholy mixture of mathematics,
http://www.cs.mu.oz.au/~fjh | literary criticism, and folklore. - B. A. Sheil

In article <D6oI19.ILL@reston.icl.com>,
Stephen Carlson <scc@reston.icl.com> wrote:
>In article <D6DwEJ.5M2@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John Max Skaller) writes:
>>In article <D6482o.KtJ@reston.icl.com>,
>>Stephen Carlson <scc@reston.icl.com> wrote:
>>>In article <KINNUCAN.95Mar22150251@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:
>>>>   If iterators are to be a generalization of pointers, shouldn't one of
>>>>   their most important behaviors (the ability to be null) also be part
>>>>   of the generalization?
>>>>No.
>>>Would you care to explain this answer?  Why should a "generalization of
>>>pointers" not include an important property of pointers?
>>
>> Because that property is not useful for making the
>>_algorithms_ of STl work.
>
>Your answer explains why STL Iterators have been misbilled (in the
>documentation no less).  There is no property about generic pointers
>that is being generalized, yet one property, being NULL, has been
>taken away.

>IMHO, the proper way to bill STL Iterators is as a "generalization of
>array pointers."  This simultaneously (a) shows what is being generalized
>("array" to "Container") and (b) explains why there are no null iterators:
>array pointers are simply not null.

True.

The C/C++ pointer is unique; it is a language construct which provides
_two_ mostly disjoint abstractions:

1. the Pascal like pointer, or the `link' of a linked data structure.
This abstraction requires a NULL value.

2. the array iterator model, unique to C/C++, which definitely doesn't
require a NULL value.

In addition to which, it provides:

3. a back-door for pass-by-reference in a (mostly) pass-by-value
language.  Again, it doesn't require a NULL value (or am I missing
something).

The main feature common to all three is that they can be dereferenced.


I can't come up with a scenario where a single pointer simultaneously
acts as both (1) and (2) above.

STL iterators have little, if nothing to do with (1) above, the only
case where a NULL is required.  STL iterators generalize (2) above.

The usage of NULL in other `pointer' contexts is mainly due to a
confusion of abstractions.  We should _not_ carry the same confusion
to the STL iterator, which provides one and only one abstraction.

- Hari
--
--Raja R Harinath-----------`finger harinath@cs.umn.edu' for more info
   "Come to think of it, there already _are_ a million monkeys on a
     million typewriters, but Usenet is _nothing_ like Shakespeare."
                                                  -- Blaire Houghton

maxtal@Physics.usyd.edu.au (John Max Skaller) writes:

>A null value to express "failure" fails to specify what _kind_
>of failure. The correct way to do this is with discriminated union:
>
> enum {success, notfound, emptyrange, duplicatesfound,
>  illformedquery} ..
>
>that is, adding a mere "boolean" flag to an iterator is refusing
>to recognize that the return value of a function might
>be a valid iterator OR any other state information.

Yes.  I agree entirely that discriminated unions are the right way of
handling this sort of thing.  The only problem is that C++ doesn't support
discriminated unions very well.  If you follow this argument to it's
logical conclusion, you have to say that even null pointers were a mistake.

--
Fergus Henderson            | As practiced by computer science, the study of
fjh@cs.mu.oz.au             | programming is an unholy mixture of mathematics,
http://www.cs.mu.oz.au/~fjh | literary criticism, and folklore. - B. A. Sheil

In article <9510113.6681@mulga.cs.mu.OZ.AU>,
Fergus Henderson <fjh@munta.cs.mu.OZ.AU> wrote:
>maxtal@Physics.usyd.edu.au (John Max Skaller) writes:
>
>>A null value to express "failure" fails to specify what _kind_
>>of failure. The correct way to do this is with discriminated union:
>>
>> enum {success, notfound, emptyrange, duplicatesfound,
>>  illformedquery} ..
>>
>>that is, adding a mere "boolean" flag to an iterator is refusing
>>to recognize that the return value of a function might
>>be a valid iterator OR any other state information.
>
>Yes.  I agree entirely that discriminated unions are the right way of
>handling this sort of thing.  The only problem is that C++ doesn't support
>discriminated unions very well.  If you follow this argument to it's
>logical conclusion, you have to say that even null pointers were a mistake.

 Not necessarily: something that I learned from someone else
and which has really stuck in my mind is the answer to the following
question:

 "What is the difference between state and type?"

The answer is:

 "Engineering"

(Substitute "art" if you like :-)

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In <9510113.6681@mulga.cs.mu.OZ.AU> fjh@munta.cs.mu.OZ.AU (Fergus Henderson) writes:
>Yes.  I agree entirely that discriminated unions are the right way of
>handling this sort of thing.  The only problem is that C++ doesn't support
>discriminated unions very well.  If you follow this argument to it's
>logical conclusion, you have to say that even null pointers were a mistake.

What are discriminated unions? At a guess it sounds like a union of types
and some sort of identifier saying which type is actually stored in the
union. Is this correct?

What sort of area you would use this in? I remember John Skaller saying
in a previous post that people don't use them enough. Where should they
be used? Is there an idiom that you can use in C++ to support
discriminated unions?

Regards,
Chris Double.

In article <9510023.17400@mulga.cs.mu.oz.au>,
Fergus Henderson <fjh@munta.cs.mu.OZ.AU> wrote:
>I think it's clear that complexity requirements are unenforceable.

On the contrary.  The complexity specifications are in terms
of the number of operations -- calls to copy constructors
and comparison operators -- which can be counted.  The "constant
factor" can be bounded simply by requiring (for instance) that
a list of 2N elements take no more than twice as long to
process as a list of N elements, for a linear algorithm; no
more than four times as long, for a quadratic algoritm; and
so on.

This "unenforceable" myth keeps popping up, and I don't know why.
It's easy to show that a library doesn't meet such a requirement
for some input, even if you can't prove that it satisfies for all
possible input.  This only means that nobody can certify that a
library does conform; but nobody can do that anyway, because they
can't prove there are no bugs.

Nathan Myers
myersn@roguewave.com

In article <9510023.17400@mulga.cs.mu.OZ.AU>,
Fergus Henderson <fjh@munta.cs.mu.OZ.AU> wrote:
>maxtal@Physics.usyd.edu.au (John Max Skaller) writes:
>
>> It is uncertain if, in fact, such complexity requirements
>>are "normative" since they are deducible from code -- but
>>cannot be measured.
>
>I think it's clear that complexity requirements are unenforceable.

 I don't think thats quite correct, in a sense I think
you would agree with -- it is possible to determine in some cases
what the complexity of a piece of code is.

 The point is you have to do this by reading and analysing
the code, it isn't something that is really "testable" in the sense
of executing the code and recording and analysing behaviour.
This would imply that in order to met test requirements, vendors
would be _required_ to supply actual sources for, say, STL
Standard Library entities.

 That would deny existing permissions that such actual
codes are an implementation detail at best -- and may not
exist.
--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

To those who want NULL iterators, try this:

Define:

 enum zero_t { zero=0; };

Specify:

 A type X providing the syntax

 x == zero

 with a type bool is said to be _pointed_ (mathematical
 term for "have a distinguished value")

Now you can write algorithms and state in the requirements

 "This algorithm requires the type of the input range
 iterators to be pointed".

Thats it. You just extended STL.

Now prove it is useful. If you can others might use your extension.
You'd then have established a defacto standard.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In <D6ntAG.9qu@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John Max Skaller) writes:

> Sigh. And I'd rather have referential transparency.
>Because I want to write algorithms that are correct and
>operate whether or not the container supports the semantics
>efficiently.

> So it seems I need to derive :

> template<class T, template<class> class Container>
> struct IndexedContainer: Container<T> {
>  T& operator[](int) {
>   list<T>::iterator i = begin();
>   while(i--) i++;
>   return i;
>  }
> }

>[which as written has the serious problem of being LESS efficient
>for a container like vector]

Can't you do something like:

  T& operator[](int index) {
   list<T>::iterator n = begin();
   advance(n, index);
   return *n;
  }

Using advance will be efficient for vectors and not for lists.

Regards,
Chris.

ruiter@ruls41.LeidenUniv.nl (Jan-Peter de Ruiter) writes:

>Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
>
>: I see, so the standards committee should add
>: a feature to an already complex language solely on the basis that it
>: will make some programmers happier?
>
>Indeed, it should. Unless it makes other programmers' life
>harder, and that is definitely not the case with null valued iterators.

Unfortunately complexity makes _everyone's_ life harder.
That's why we have to make trade-offs.

--
Fergus Henderson - fjh@munta.cs.mu.oz.au

maxtal@Physics.usyd.edu.au (John Max Skaller) writes:

> It is uncertain if, in fact, such complexity requirements
>are "normative" since they are deducible from code -- but
>cannot be measured.

I think it's clear that complexity requirements are unenforceable.
After all, all real machines have only a finite amount of memory, and
hence only a finite number of different states.  This implies that the
time to complete any action not involving I/O is bounded by the product
of the number of states and the state transition time (presuming that
the action does terminate at all).  Thus any action not involving I/O
is constant-time.  Of course, the constant may be considerably longer
than the lifetime of the universe, so this conclusion isn't useful for
anything other than proving conformance to the letter of some standard,
or pointing out some inherent limitations in complexity analysis.

In fact the draft seems to be only half-serious about complexity anyway -
for allocators, it says that all operations are "expected" to have
amortized constant time.  That doesn't sound like a normative requirement
to me.

Note that a standard doesn't have to be enforceable in order to be
useful.  For example, the Ada standard has lots of useful
"Implementation Advice" sections, which state that implementations
"should" do such-and-such.

--
Fergus Henderson - fjh@munta.cs.mu.oz.au

In article <3m4mam$2le@jupiter.SJSU.EDU>,
Cay Horstmann <horstman@sjsumcs.sjsu.edu> wrote:
>
>In contrast, STL purposefully chose [to do]
>nothing to prevent iterator mishandling.
>I don't think they are wrong, it just is
>different from my own efficiency/safety tradeoff model.

 Why don't we tack this issue head on? Here's a
proposition for you:

 1) STL is right as it is (not providing iterator safety)

 2) STL is powerful and extensible enough to allow
 YOU to provide a layer that does provide iterator safety.

As I write I do not _know_ whether these propositions hold water
or not. But I'd like to find out. In particular, I start
with the proposition as an mental assumption and attempt to
prove it by constructively -- by actually writing a safe iterator layer.

I may need help doing that. Anyhow, lets start:

First, what does "safe" mean? Well, some possibilities are:

 1) An iterator pair representing a range both denote
 the same container

 2) An iterator pair representing a range not only denote
 the same container, they denote a valid subsequence
 of the container (possibly empty)

 3) An iterator requiring dereferening is dereferenceable

Now, to check (1) we require a function

 container_of(iterator)

which tells to what container (if any) an iterator refers.
To tell if an iterator is valid we could require a function

 valid_iterator(iterator)

and to tell if dereferenceable we could define

 dereferenceable(iterator it) {
  return
   valid_iterator(it) &&
   iterator != container_of(itertor) -> end();
 }

To tell if a range is valid is easy if slow:

 valid_range(iterator it1, iterator2 it2) {
  if (container_of(it1) == container_of(it2))
  {
   int len = container_of(it1)->size();
   while(len--) if(it1++==it2) return true;
  }
  return false;
 }

In fact this idea can be used to implement:

 valid_iterator(iterator)

by simply checking all possible iterators.

Now, of COURSE this is inefficient. But the above are templates
and so can be specialised.

Implementing "container_of" can be done by delegation:

 class iterator {
  delegate it;
  container *cont;
  ....
  // use delegation here, possibly with checks
 }

OK, that is only an outline not real code. But it _looks_
possible.

The key idea is that you have to "extend" STL.
(Actually, you are not extending it but introducing a "subtype")
That does not mean adding a template class. It means
adding extra PROTOCOL. STL already shows how to specify
protocol. It is a good model. Learning how to extend it
is a whole new ballgame.



--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <FENSTER.95Apr4213620@ground.cs.columbia.edu>,
   fenster@ground.cs.columbia.edu (Sam Fenster) wrote:
:I agree.  An STL container should always hand back an iterator
:guaranteed to point to valid data, or to an end iterator.  It
:shouldn't require error checking.  But it would be useful if
:*other* functions could return Null iterators, and if iterators
:could be initialized to Null.

Perhaps.  But I don't see the utility of bundling a error state
that has nothing to do with an STL container with an STL iterator,
which, after all, exists to support STL containers.  To me it
smacks of data bundling, which is a bad idea in procedural code and
doesn't get any better in OO code.

:It's often good design to let an object have an `invalid' state.

I don't know about that.  It's better than an object that can have
an invalid state and doesn't tell you about it, but in general I'd
rather have objects that never wind up in invalid states.

:I agree that STL containers should not return invalid iterators.
:But functions which are not part of the STL, and have unrelated or
:expanded functionality, can hand back iterators defining an STL
:range.  (This is similar to how a function that does something
:unrelated to string handling can return a string!)  Null would
:be a particularly compact and convenient way to indicate the
:function's failure.

But again, why burden STL with keeping around error flags for
non-STL functions?  If this is really a problem, why can't the
client class define its own error state and return that, ala:

class STLClient {
    public:
 ErrorState func(iterator& arg);
    };

What could you do with a simple null state, anyway?  Terminate?  At
that point you might as well throw an exception.

 --Paul J. Ste. Marie, pstemari@well.sf.ca.us, pstemari@erinet.com

The Financial Crimes Enforcement Network claims that they capture every
public posting that has their name ("FinCEN") in it.  I wish them good hunting.

Perhaps Null iterators have a drawback as a technique, but not for any of
Paul's three reasons:

> > [Someone else wrote:]
> >   Can you give me a good reason not to define a singular value for an
> >   iterator?  Whether I choose to call it null seems immaterial.

> Paul Kinnucan <kinnucan@hq.ileaf.com> wrote:
> >1. Redundant.
> >You can't do anything with null-valued iterators that you can't
> >do just as easily, clearly, and economically without them.

I think the alternatives are less easy and less economical.  Perhaps they'd be
more clear, but also more verbose and klunky.

> >2. Dangerous.
> >Encourages careless programmers to think that they can safely incr/decr
> >isolated iterators, for example, as one post to this thread
> >suggested, to explore the region around an isolated iterator.

I don't see how it encourages that.  The kind of carelessness you mention
seems unrelated to Null iterators.

> >3. Complex.
> >Null iterators introduce additional (and needless) complexity
> >into the design and implementation of STL containers and applications.

Hardly any complexity at all, and it simplifies users' lives by allowing them
to signal invalidity.

However, Max's criticism makes more sense:

maxtal@Physics.usyd.edu.au (John Max Skaller) writes:
> 4) Inadequate.
> A null value to express "failure" fails to specify what _kind_ of
> failure. The correct way to do this is with discriminated union...

> Certainly a boolean is most commonly useful, but it lacks generality
> appropriate to a Standard Library.
>
> Requiring a returned iterator be valid is therefore a sensible cut off point
> -- the returned value can be immediately reused WITHOUT checking by another
> algorithm....

> IMHO it is a good thing STL does NOT try to make this decision for you.

Matthew Hannigan (matth@extro.ucc.su.OZ.AU) wrote:
: horstman@sjsumcs.sjsu.edu (Cay Horstmann) writes:
: > [ .. ]
: >It is weird that STL is very defensive in one regard (worst-case
: >running time of algorithms) and very rough-and-ready in another
: >(no testability of iterator state).
: > [ .. ]

: Surely that's because the user can do something about the latter
: but not usually about the former.  (without writing another
: implementation)

In theory, "the user" can write perfect programs and never make a pointer/
iterator bug. In theory, "the user" can always pick wonderful hash
functions. In practice, that doesn't seem to pan out, and pragmatically
speaking I'd put "user makes iterator error" into the same category as
"user chooses crappy hash function". And then I'd spend my time defending
against the iterator errors first because their impact on my program is more
dramatic. In contrast, STL purposefully chose not to include hashing
because people might choose crappy hash functions, and they do nothing to
prevent iterator mishandling. I don't think they are wrong, it just is
different from my own efficiency/safety tradeoff model.

Cay

In article <3lp3ko$di5@calum.csclub.uwaterloo.ca>,
Ross Ridge <rridge@calum.csclub.uwaterloo.ca> wrote:
>
>STL already says that.  But I wouldn't slow index operations added to
>standard STL containers, I'd rather have the static checking that tells
>me that I'm violating an assumption.

 Sigh. And I'd rather have referential transparency.
Because I want to write algorithms that are correct and
operate whether or not the container supports the semantics
efficiently.

 So it seems I need to derive :

 template<class T, template<class> class Container>
 struct IndexedContainer: Container<T> {
  T& operator[](int) {
   list<T>::iterator i = begin();
   while(i--) i++;
   return i;
  }
 }

[which as written has the serious problem of being LESS efficient
for a container like vector]

My reason: I want to delay fixing implementation details like this
until I have sorted out the interface design.

This is _exactly_ what using abstract classes does for you --
you can use any working implementation for prototyping and
speed it up by specialisation afterwards (overriding virtual
functions).

In particular you may be wise to do some profiling to balance the
speed/memory tradeoff of your application sensibly.

In a GUI for example, in managing window lists does not matter
as much as blitting pixels around FAST. There's a lot more
pixels than windows :-)

However, in this case lists have a semantic advantage over
vectors -- iterators remain valid in lists after insertion or
deletion, but not in vectors. I still _need_ to index
the windows. It doesn't matter how long it takes, I have to
do it anyhow.

So -- the STL is correct because you can have what you want
and so can I. I just have some more work to do. :-)

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <D6DwEJ.5M2@ucc.su.OZ.AU> maxtal@Physics.usyd.edu.au (John Max Skaller) writes:
>In article <D6482o.KtJ@reston.icl.com>,
>Stephen Carlson <scc@reston.icl.com> wrote:
>>In article <KINNUCAN.95Mar22150251@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:
>>>   If iterators are to be a generalization of pointers, shouldn't one of
>>>   their most important behaviors (the ability to be null) also be part
>>>   of the generalization?
>>>No.
>>Would you care to explain this answer?  Why should a "generalization of
>>pointers" not include an important property of pointers?
>
> Because that property is not useful for making the
>_algorithms_ of STl work.

Your answer explains why STL Iterators have been misbilled (in the
documentation no less).  There is no property about generic pointers
that is being generalized, yet one property, being NULL, has been
taken away.

IMHO, the proper way to bill STL Iterators is as a "generalization of
array pointers."  This simultaneously (a) shows what is being generalized
("array" to "Container") and (b) explains why there are no null iterators:
array pointers are simply not null.

This model, a generalization of array pointers, is a good one:

-  Those who know that it is unsafe to use a single array pointer to
   navigate an array without length/end information, use pairs of iterators.

-  Those who want the convenience and compactness of using a single array
   pointer to navigate an array, terminate the Container with a null value
   (i.e., NULL for a Container of pointers).

-  Those who want to indicate the existance and identity of an element,
   use a pointer.

I submit that most of the impetus for the null pointer position is due
to a desire to make iterators live up to its billing.  Unfortunately,
the billing does not explain the model.

Stephen Carlson
--
Stephen Carlson     :  Poetry speaks of aspirations,  : ICL, Inc.
scc@reston.icl.com  :  and songs chant the words.     : 11490 Commerce Park Dr.
(703) 648-3330      :                 Shujing 2:35    : Reston, VA  22091   USA

OK, I think I understand what you are trying to do.
You want your iterator to serve two purposes:
1. Be an iterator
2. Be an indication that some routine failed miserably.

I agree with John Max Skaller that you should really be returning 2
different results. Since you don't want to use exceptions here, could you
change your Dicts::look_up_word() to return a
pair<error_indicator, Dicts::iterator> instead of just an iterator? Or if
that is not acceptable, add a isValid() method to your iterator. Or even
define your iterators such that there you can assign them to NULL when you
want to indicate that they are invalid- as long as your code makes sure
never to pass a NULL iterator to any STL compliant algorithm there is
nothing wrong with this. There are many ways you can implement that test
without placing additional requirements on the C++ standard.

Nobody has said that your iterators must be limited to the interface
defined by STL. The STL specification defines a minimal set of interfaces
that guarentee interoperability between containers and algorithms. If you
follow the rule that you can only pass valid iterators to the algorithms,
you will only get valid iterators back. If you require that all iterators
support NULL, then you also need to say that all of the STL algorithms need
to check for this NULL value (which adds overhead, which will cause people
to not use these algorithms, which will make STL not as useful as it is).

I believe that the minimal interface requirements of the STL components is
one of the things that makes STL so wonderful. There is nothing preventing
anyone from adding on to the interface when special functions are needed
(which is where I'd classify your examples). In fact, STL uses this model
itself: Forward iterators only require the operations (==, !=, *, pre and
post ++). Bidirectional iterators add to this set the operations (pre and
post --). Random access iterators add even more required operations (+=, +,
-=, -, [], <, <=, >, >=). You have a requirement that your iterators have a
testable invalid state. None of the algorithms or containers that come with
STL require this, and I believe that adding the NULL value to the C++
standard is unnecessary.

Another problem is that STL imposes no restrictions on how iterators can be
implemented. An iterator could be a C style pointer, an enum, an integer, a
complex structure, or anything else. How do you create one value that can
generically be compared against any implementation of an iterator? (I don't
think that this problem can't be overcome, I just don't think that it needs
to be solved.)

eddy
--
ed.dy \'ed-e-\ n [ME (Sc dial.) ydy, prob. fr. ON itha; akin to OHG ith-
   again], L et and 1a: a current of water or air running contrary to the main
   current; esp)X : a small whirlpool 1b: a substance moving similarly  2: a
   contrary or circular current  - eddy vb

In article <FENSTER.95Apr4160955@ground.cs.columbia.edu>,
Sam Fenster <fenster@ground.cs.columbia.edu> wrote:
>> Sam Fenster <fenster@ground.cs.columbia.edu> wrote:
>>>
>>>   #include "dictionaries.h"   // Defines namespace Dicts.
>>>   int main () {
>>>   // Dicts is a module that accesses global containers and files that we
>>>   // can't see.  It is *not* a single container.  Its data is *not* public.
>>>
>>>   Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
>>>   if (it1 == Dicts::Null) error();
>>>
>>>   Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
>>>   if (it2 == Dicts::Null) error();
>>>
>>>   for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
>>>   return 0;}
[...]
>>> The availability of Null would make the design much cleaner.
[...]
>The Dict routines *do* return a range within an STL container.  Dict, as a
>whole, manages many STL containers.  Dict doesn't have to *be* an STL
>container.  And it doesn't have to give direct access to any STL container
>beyond returning a pair of iterators.

OK, say that you really do need NULL. Are you allowed to pass this NULL
value back to Dicts? In your example, Dicts::end_of_section() takes as a
parameter an iterator it created (in Dicts::look_up_word()). Is
Dicts::end_of_section(Dicts::look_up_word("the")) valid in your model, or is
it required that every call to Dicts::look_up_word() be followed by a check?

By saying that interators can have a valid value that is NULL, you are sort
of saying that every function that takes an iterator needs to make sure the
value is not NULL before operating on that iterator. (There is another
thread in comp.std.c++ going on about the difference between references and
pointers where this comes up.) I don't see how making everyone check for
NULL makes the _system_ design any cleaner. It might make the design of
Dicts::look_up_word() easier, but it muddies up the design of other
components of the system.

If you _must_ have testable invalid iterators, I still think that a member
function of the iterator (something like it1.isValid()) makes the design
cleaner than comparing the iterator against NULL.

--
ed.dy \'ed-e-\ n [ME (Sc dial.) ydy, prob. fr. ON itha; akin to OHG ith-
   again], L et and 1a: a current of water or air running contrary to the main
   current; esp)X : a small whirlpool 1b: a substance moving similarly  2: a
   contrary or circular current  - eddy vb

John Max Skaller <maxtal@Physics.usyd.edu.au> wrote:
> But the problem is that well defined operations you may
>need cannot always be linear or meet STL requirements --
>and then you find NO standard container that meets your needs.
>You may well be quite happy with a list with slow indexing,
>the problem is that you have to WRITE the extra code yourself.
>
> It would be easy enough to say: "The performance of
>this algorithm is logarithmic provided the indexing operation
>is constant time" -- and provide the slower indexing operation
>anyhow.

STL already says that.  But I wouldn't slow index operations added to
standard STL containers, I'd rather have the static checking that tells
me that I'm violating an assumption.

      Ross Ridge

--
 l/  //   Ross Ridge -- The Great HTMU, Ook                    +1 519 883 4329
[oo][oo]  rridge@csclub.uwaterloo.ca      http://csclub.uwaterloo.ca/u/rridge/
-()-/()/
 db  //

In article <D6BME0.Ezr@cdf.toronto.edu>, g2devi@cdf.toronto.edu (Robert N. Deviasse) says:
>
>In article <3lfjcl$qvt@druid.borland.com>,
>Pete Becker <pete@borland.com> wrote:
>>In article <D688s0.60A@cdf.toronto.edu>, g2devi@cdf.toronto.edu (Robert N. Deviasse) says:
>>>
>>>BTW, one of the reason people wanted to have a null value is to be able to
>>>check if an iterator has been initialized or made invalid, assuming that
>>>the programmer follows defensive a programming style. For example:
>>>
>>>      Container c;
>>>      Iter i=Null<Iter>();
>>>      ... // (*)
>>>      // we assume that 'i' has been set to a value in 'c'
>>>      assert(i!=Null<Iter>());
>>>      ... // (**)
>>>      invalidate(c);           // append or some other operation
>>>      DEBUG(i=Null<Iter>());   // used only for defensive programming
>>>      ...
>>>
>>>I can't see how we can emulate this with the end() iterator.
>>
>>       No, I don't see offhand how to do it, either. Again, I'll ask: why
>>do you want to do this? There are much safer ways of programming this
>>sort of thing. Don't create the iterator until you are ready to initialize
>>it.
>
>Unfortunately this isn't always possible. Consider this the following:
>      Container c;
>      Iter i=Null<Iter>();
>      if (condition1()){
>         ... set i wrt c appropriately
>      }else if (condition2()){
>         ... set i wrt c appropriately
>      } else {
>         ... set i wrt c appropriately
>      }
>
>      // As a sanity check I want to ensure that i has been initialized.
>      // This also provides *executable* documentation on my expectations at this point.
>      assert(i!=Null<Iter>());
>
>The problem is that sometimes the calculation of an initialization value is
>conditional upon the state of the system at the time of initialization. Given the
>nature of conditionals, how can you *practically* get around this problem?
>

Call a function. Use its result to initialize the iterator.

 Iter Initialize( Container& c )
 {
       if (condition1()){
         ... return iterator set wrt c appropriately
       }else if (condition2()){
         ... return iterator set wrt c appropriately
       } else {
         ... return iterator set wrt c appropriately
       }
 Container c;
       Iter i=Initialize(c);

>> Make sure it goes out of scope when it becomes invalid.
>
>Is this always possible? Remember that if an iterator can become invalid when
>Consider this example:    (Note DEBUG(x) is a macro that expands to x)
>
>    Container c;
>    ...
>    Iter i=c.start();
>    ...
>    Container d;
>    ...
>    Iter j=c.start();
>    do_something_that_changes_a_Container_structure(c);   // now i is invalid
>    DEBUG(i=Null<Iter>());                                // document that i is invalid


"document that i is invalid"? Sounds like a comment. I don't see the point of this code.
It seems at best circular: we need null iterators so that I can write code that uses
null iterators.
>
>At this point, i is invalid, but j is still needed, so we can't just end the scope
>here.
>

 Well, yes, it is certainly possible to write code snippets that appear to
make it hard add blocks that make things out of scope. Rather than respond with a trivial
re-ordering of these statements which makes it simple to add a block, let me repeat my
usual question: can someone provide a fairly complete example that requires this?

>> I don't think that's really the case: it certainly hasn't worked
>>out that way with pointers.
>
>I don't follow. You mean that you've never seen or used assert statements to ensure
>that your pointers are non-null?
>
>> What is behind this urge to import all the
>>known dangers of null pointers into STL?
>
>Dangers? How can null pointers be any more dangerous than an out of boundary pointer
>(which is the alternative)? Neither of these can be safely dereferenced. At least
>with null pointers you can *test* if it is dereferenceable. How is this more dangerous?

Precisely because it encourages programmers to test for validity rather than assure
validity. Write the code so that it is impossible (or very difficult) to dereference a
null pointer. Then you won't have to test for them.
 -- Pete

Tony Cook (tony@online.tmx.com.au) wrote:

: a) is covered well by the way you should be using STL - using start
: and end iterators.  I've never seen a pointer incremented to null
: and I don't really believe it should be.

Well, OK, maybe it shouldn't be, but in C this is or course frequently
happening using strings.

: b) has a stronger root in the common "C" pattern of return null to
: indicate an error (malloc and fopen for example), but this has been
: overshadowed in C++ by exceptions.  Is there some reason you can't
: use exceptions for this?  If you can't have you considered, that if
: you need such a validity test in your own iterators, that you can
: use some sort of test member function, like good() in IOStreams?

I agree with this in the sense that I would like to be able to test
(e.g. using good() ) the validity of an iterator _before_ any exception
is thrown. Whether that is by using a null value or some test function
doesn't matter to me. Our own library does both, and it really is handy
to be able to test validity, especially during debugging.

Greetings,

JP

> fenster@ground.cs.columbia.edu (Sam Fenster) writes:
> |> You compare an iterator against an end iterator when it's
> |> traversing/searching a pre-existing range.  Pre-existing ranges *have*
> |> end iterators.  When someone hands you an iterator that helps *define* a
> |> range, it's serving a different purpose.  There may be no other range in
> |> sight to compare it to.  Then you need Null as a standard way to signal
> |> invalidity.

swf@elsegundoca.ncr.com (Stan Friesen) writes:
> You missed it again!
>
> The way to signal this is to use a begin-end pair where the begin iterator
> tests equal to the end iterator.  This represents an empty range, since
> the end iterator points "one past the end".

You missed it again!

An empty range can be a perfectly valid range.  It is not the same as an error
condition.  As I've pointed out several times already.

In article <3lk13b$7cb@druid.borland.com>,
Pete Becker <pete@borland.com> wrote:
>>        OK, now consider the code fragment
>>
>>        my_container<int>       c;
>>        my_container<int>::iterator     it = c.begin();
>>        c.insert(5);
>>        // does *it == 5 ? Of course not!
>>        // ++it will also be undefined.
>>
>
>Yes. That is true for all STL containers. Don't save off iterators then
>modify the container. Grab the iterators when they are needed. It can be
>very expensive to write iterators that "know" when their container has
>been modified.
[...]
>Seems to me that mentioning NULL pointers doesn't help your case. NULL
>pointers cause many programming problems. It is not at all clear to me
>that they solve more problems than they cause.

I could see NULL being useful to indicate that the container under the
iterator has changed, which invalidates the iterator. But this isn't
practical with STL (as Pete points out, it can get very expensive). There
is nothing stopping anyone from writing STL compliant containers/iterators
that can have the iterator "know" when the container has changed (and set
the iterator to NULL), but I'd prefer to see this as an extension to STL,
rather than an additional requirement on the current STL. The current STL
makes a nice base from which to work.

If you need NULL iterators, build them on top of STL. Just remember that
passing a NULL iterator to one of the STL algorithms will have undefined
behavior (but then passing any "singular" iterator has the same effect).

eddy

--
ed.dy \'ed-e-\ n [ME (Sc dial.) ydy, prob. fr. ON itha; akin to OHG ith-
   again], L et and 1a: a current of water or air running contrary to the main
   current; esp)X : a small whirlpool 1b: a substance moving similarly  2: a
   contrary or circular current  - eddy vb

In article <KINNUCAN.95Mar31170947@candide.hq.ileaf.com>,
Paul Kinnucan <kinnucan@hq.ileaf.com> wrote:
>
>   Can you give me a good reason not to define a singular value for an
>   iterator?  Whether I choose to call it null seems immaterial.
>
>He already has.  In fact, three good reasons have been stated by
>Pete and others in this thread:
>
>1. Redundant.
>
>You can't do anything with null-valued iterators that you can't
>do just as easily, clearly, and economically without them.
>
>2. Dangerous.
>
>Encourages careless programmers to think that they can safely incr/decr
>isolated iterators, for example, as one post to this thread
>suggested, to explore the region around an isolated iterator.
>
>3. Complex.
>
>Null iterators introduce additional (and needless) complexity
>into the design and implementation of STL containers and applications.

 Let me add:

4) Inadequate.

A null value to express "failure" fails to specify what _kind_
of failure. The correct way to do this is with discriminated union:

 enum {success, notfound, emptyrange, duplicatesfound,
  illformedquery} ..


that is, adding a mere "boolean" flag to an iterator is refusing
to recognize that the return value of a function might
be a valid iterator OR any other state information.

Certainly a boolean is most commonly useful, but it lacks
generality appropriate to a Standard Library.

Requiring a returned iterator be valid is therefore a sensible
cut off point -- the returned value can be immediately reused
WITHOUT checking by another algorithm.

If this is not enough it is up to you to return your
own kind of data structure and test is before the iterator
is used -- OR simply throw an exception.

Which choice you make depends on whether you consider the kind
of "failure" involved an error or a valid return. (Throw
exceptions in the former case, and a discriminated union in the
latter)

IMHO it is a good thing STL does NOT try to make this decision for you.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <3lop14$92e@vbohub.vbo.dec.com>,
Ian Johnston <johnston@caiman.enet.dec.com> wrote:
>>
>> But it is my guess there needs to be a layer
>>on top of STL that provides referential transparency
>>_irrespective_ of performance. Because in 90% of code,
>>performance just doesn't matter. Correctness matters
>>in 100% of code :-)
>>
>
>Agreed 110%. (:-))
>
>But seriously, this is absolutely right. I still think STL is fine for
>experts, but will prove less than robust in the hands of less expert
>programmers.
>
>That's a shame.
>
>Yes, I can implement something on top for those less expert programmers.
>
>But then, it would be nice not to have to.

 Yes, but if you think about it it is necessary. In the
next Standard we might standardise such a set of application
layers. But it is _surely_ way too soon to do that now.

 I personally do not yet know exactly what the "best"
way to extend and wrap STL is. I'm still doing basic stuff
like

 a) using the supplied containers

 b) adding my own containers and iterators

 c) fiddling to get my compiler to work with STL at all

 d) doing lots of thinking about extending the
    STL Standard -- ie the protocol -- in various
    ways. For example -- extension to partial
    orderings is obvious. Extension to several
    dimensions is necessary for graphics.

 So what I'm saying is that while I agree with you,
there is no magic. I think STL is pitched at _exactly_ the
right level for this point in time and for this version
of the Standard.

 I'm finding more deficiencies in the C++ language
by using STL than in STL itself. For example, there is NO WAY
to declare a variable of the type of a member of a parameter:

 template<class T> void f(T t) {
  _Typeof<t.m> x = t.m;
 };

I can do this in Metaware HighC/C++ and am doing it. You can use
'typeof()' in GNU. The compiler knows the type of t.m and it can
do overloading on it but you can't make a variable??

In fact what we need is:

 define x = expr;

which means

 typeof(expr) x = expr;

 My point -- the C++ language is far more deficient itself
than STL. STL was designed. C++ was grown, pruned, and hacked.
--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <3loejf$sv7@highway.LeidenUniv.nl>,
   ruiter@ruls41.LeidenUniv.nl (Jan-Peter de Ruiter) wrote:

:Tony Cook (tony@online.tmx.com.au) wrote:
:
:: a) is covered well by the way you should be using STL - using
:: start and end iterators.  I've never seen a pointer incremented
:: to null and I don't really believe it should be.
:
:Well, OK, maybe it shouldn't be, but in C this is or course
:frequently happening using strings.

No, the pointer didn't increment to null, *pointer became null.
This idiom requires an endmarker in the container.  Null is
convienent if the container contains characters or pointers, but is
not so convienent if, for example, it contains floats.

:: b) has a stronger root in the common "C" pattern of return null
:: to indicate an error (malloc and fopen for example), but this
:: has been overshadowed in C++ by exceptions.  Is there some
:: reason you can't use exceptions for this?  If you can't have you
:: considered, that if you need such a validity test in your own
:: iterators, that you can use some sort of test member function,
:: like good() in IOStreams?

:I agree with this in the sense that I would like to be able to
:test (e.g. using good() ) the validity of an iterator _before_ any
:exception is thrown. Whether that is by using a null value or some
:test function doesn't matter to me. Our own library does both, and
:it really is handy to be able to test validity, especially during
:debugging.

The headache with this style of programming is that every line of
code winds up being an if statement.  I'd rather have containers
that were guaranteed to give back a valid iterator than ones which
decided whether or not to hand me back a usable iterator depending
on the phase of the moon or the drizzle in Redmond.

Think about it. A container isn't interfacing with a user or
(in most cases) doing I/O.  All it does is hold the data you put
into it.  It should never have to hand back a bad iterator, and
from a container client perspective, the code is cleaner if flags
indicating various odd states on the part of the client are
seperate and not conflating into the iterator, which could care
less.

 --Paul J. Ste. Marie, pstemari@well.sf.ca.us, pstemari@erinet.com

The Financial Crimes Enforcement Network claims that they capture every
public posting that has their name ("FinCEN") in it.  I wish them good hunting.

In article <3lp2u3$f7u@druid.borland.com>,
   pete@borland.com (Pete Becker) wrote:
[...]
:Precisely because it encourages programmers to test for validity
:rather than assure validity. Write the code so that it is
:impossible (or very difficult) to dereference a null pointer. Then
:you won't have to test for them.

Exactly.  Deal with the problem when it occurs, and you won't need
to replicate the same error-handling code in every last module of
your program.

 --Paul J. Ste. Marie, pstemari@well.sf.ca.us, pstemari@erinet.com

The Financial Crimes Enforcement Network claims that they capture every
public posting that has their name ("FinCEN") in it.  I wish them good hunting.

In article <ncmD6EIC8.Cvy@netcom.com>, Nathan Myers <ncm@netcom.com> wrote:
>
>You can make operator!() start up a Doom session if you like.

 OH! We definitely SHOULD Standardise that. I'll write
a proposal immediately.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

> Sam Fenster <fenster@ground.cs.columbia.edu> wrote:
>>
>>   #include "dictionaries.h"   // Defines namespace Dicts.
>>   int main () {
>>   // Dicts is a module that accesses global containers and files that we
>>   // can't see.  It is *not* a single container.  Its data is *not* public.
>>
>>   Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
>>   if (it1 == Dicts::Null) error();
>>
>>   Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
>>   if (it2 == Dicts::Null) error();
>>
>>   for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
>>   return 0;}

eddy@clipper.robadome.com (eddy Gorsuch) writes:
> Q1: If I change the second call to:
>     Dicts::iterator it2 = Dicts::look_up_word ("zoo");
>     is it2 reachable from it1? (That is, is
>        while (it1 != it2) {++it1;};
>     guarenteed to stop?

No.  The words may be found in different dictionaries.

>     If so, then your dictionary is "logically" one container (even if it
>     might be implemented as many containers).

> Q2: OK, if you still don't think the dictionary is one container, how is
>     your use of Dict::Null any different than using Dict::end() in your
>     above example. The current STL definition uses container::end() the
>     exact same way that you are using Dict::Null in the above code.

No.  `Dicts' is not a single container.  It's just a namespace in which all
the names in a module reside.  The operations in it give access to any number
of privately held containers.  If I were to define end(), it would not be the
end of any particular container.  Note three points here: (1) The semantics of
the interface displayed above do not require knowledge of any larger container
than what is defined by the returned bounds.  (2) If end() were defined, it
could not actually be the end of any single container.  (3) Null is not being
used to indicate the end of anything.  It's an invalid iterator, being used to
indicate a failed operation.

>     (i.e. STL uses the end of the container to indicate an "invalid"
>     iterator...

There is no "*the* container."  There is no "end of *the* container."

>> If someone hands me two iterators, it is useful to be able to check them
>> for validity.  Do you claim that in every such situation, there will be a
>> third iterator somewhere in my environment, the end of some larger range?
>
> If you get 2 STL iterators as a range and the iterators are equal, then
> there are no items in the "container" that is returned. This means that the
> iterators are valid (they point into some container), but that there is no
> valid data to be returned. Why do you need a third iterator?

I don't.  I need Null.  If the iterators are equal, this does not tell me that
the operation failed.  It tells me that the operation succeeded, and the
resulting range exists in a particular container, and it is empty.  If I never
initialized Dict, or if it could not find the dictionaries on disk, or if it
couldn't find a range in any of the dictionaries that satisfied the specified
criteria, I need to return an invalid iterator.  Not an empty range in a
particular container.

Since people are making me repeat myself, I figure I'll repeat the following
as well.  I'm interested in people's thoughts, and no one's addressed it:

> Pete Becker <pete@borland.com> writes:
>> Use exceptions to indicate errors.

Sam Fenster <fenster@ground.cs.columbia.edu> wrote:
> Unfortunately, there is no way to stop exceptions from causing program
> termination, so they should be avoided, except for extreme errors.  If your
> exception-throwing function ever gets called by a destructor during stack
> unwinding for an unrelated exception, for instance, your program will
> terminate.  Or if the copy constructor of the thrown object throws an
> unrelated exception....The more unrelated subsystems use exceptions for
> non-extreme errors, the more likely your program is to terminate
> unexpectedly.

If it were not for these drawbacks in the C++ definition of exceptions, I
would indeed advocate exceptions as *by far* the best way of signaling errors.
There would then be no need for return values indicating invalidity.  There
might still be some need for object states indicating invalidity.  I'm sorely
disappointed that exceptions can't be used safely.  But even if they could,
they are not the only style of programming, or error handling, out there.
(In C++, they don't even have any established practice yet!)

Back to our regularly scheduled programming:

> Sam Fenster <fenster@ground.cs.columbia.edu> wrote:
>> Modeling my collection of dictionaries as a single large container doesn't
>> have any desirable semantics for me!  Why impose an ordering between the
>> last word in one dictionary and the first word in some other dictionary?
>>
>> The availability of Null would make the design much cleaner.

eddy@clipper.robadome.com (eddy Gorsuch) writes:
> But STL is a library of containers (and things you can do with those
> containers). If your programming model doesn't fit this (your dictionary is
> not a container, then it doesn't fit the STL model, and you should not be
> using the STL semantics if they don't fit your model).

The Dict routines *do* return a range within an STL container.  Dict, as a
whole, manages many STL containers.  Dict doesn't have to *be* an STL
container.  And it doesn't have to give direct access to any STL container
beyond returning a pair of iterators.

(See the end of this post for a repost of another example -- a database
query.)

> The example you posted above does (as far as I can tell) fall into the STL
> model. Let me change it slightly:
>
>    Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
>    if (it1 == Dicts::end_of_section(it1)) error();
>    // or maybe: if (it1 == Dicts::end_of_section("the")) error();
>
> it1 is a single value. I can check whether it is a valid item in the
> container of T's by checking it against the end of section of T's.

it1 can only be used to identify a particular container (and its end) if
look_up_word() succeeds.  look_up_word ("#%&*") will not be able to find any
particular container with that string in it.  Should I make a fake container
just so all failed operations can point at its end?  That's a pretty ugly
design.  Why don't I just make up a single fake iterator value instead?  One
that doesn't point into a particular container?  Called, say, `Null'?

> >    Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
> >    if (it2 == Dicts::Null) error();
>
> How can your end_of_section return an invalid iterator? Remember that STL
> iterators point into a container, and possibly one place past the end of the
> container. (i.e. your Dicts::end_of_section() for T's should not return
> "Tzar", but "Tzar" + 1.)

Yes, that's what it would do to indicate the range ["the".."tzar"].

end_of_section(it1) might fail if Dicts wasn't properly initialized, or if its
argument pointed at invalid data.  It would then not be able to return an
iterator pointing to the end of any particular container.  In fact, if it1
were in the `Z' section, it would return the end iterator of a particular
dictionary (or section -- it's all relative) on *success*.  How do I indicate
failure?  Null would be most welcome here.

As a postscript, here's a repost of another example in which Null presents a
solution where an end iterator is not applicable.  Keep in mind the points
I've already made -- (1) A valid, empty returned range is not the same as an
error, and (2) interfaces which return iterator pairs do not need to be
containers themselves:

Sam Fenster <fenster@ground.cs.columbia.edu> wrote:
> I pass a function a database query string.  It passes me back an iterator
> pair which lets me traverse (or search) the list of query results:
>
> #include "db.h"
> int main ()
> {
>    DB_Handle h = db_open("personnel");  // A database of tables
>    if (!h) {error("Couldn't open database"); exit(1);}
>
>    string query;
>    cin.getline(query);  // "select name from employee where salary > 50000"
>
>    DB_iterator_pair p = db_query(h, query);
>    if (p.i1 == Null<DB_iterator>())  {error("Invalid query"); exit(2);}
>
>    for (DB_iterator i=p.i1; i!=p.i2; ++i)  cout << *i << endl;
>    return 0;
> }

> :Tony Cook (tony@online.tmx.com.au) wrote:
> :: b) has a stronger root in the common "C" pattern of return null to
> :: indicate an error (malloc and fopen for example), but this has been
> :: overshadowed in C++ by exceptions.

Not yet, it hasn't.

> :: Is there some reason you can't use exceptions for this?

In a different thread, Sam Fenster <fenster@ground.cs.columbia.edu> wrote
something relevant here:
> Unfortunately, there is no way to stop exceptions from causing program
> termination, so they should be avoided, except for extreme errors.  If any
> exception-throwing function ever gets called by a destructor during stack
> unwinding for an unrelated exception, for instance, your program will
> terminate.  Or if the copy constructor of the thrown object throws an
> unrelated exception.  The more unrelated subsystems use exceptions for
> non-extreme errors, the more likely your program is to terminate
> unexpectedly.

> If it were not for these drawbacks in the C++ definition of exceptions, I
> would indeed advocate exceptions as *by far* the best way of signaling
> errors.  There would then be no need for return values indicating
> invalidity.  There might still be some need for object states indicating
> invalidity.  I'm sorely disappointed that exceptions can't be used safely.
> But even if they could, they are not the only style of programming, or error
> handling, out there.  (In C++, they don't even have any established practice
> yet!)

> :: If you can't [use exceptions,] have you considered, that if you need such
> :: a validity test in your own iterators, that you can use some sort of test
> :: member function, like good() in IOStreams?

pstemari@erinet.com (Paul J. Ste. Marie) writes:
> The headache with this style of programming is that every line of code winds
> up being an if statement.  I'd rather have containers that were guaranteed
> to give back a valid iterator than ones which decided whether or not to hand
> me back a usable iterator depending on the phase of the moon or the drizzle
> in Redmond.
>
> Think about it. A container isn't interfacing with a user or (in most cases)
> doing I/O.  All it does is hold the data you put into it.  It should never
> have to hand back a bad iterator,

I agree.  An STL container should always hand back an iterator guaranteed to
point to valid data, or to an end iterator.  It shouldn't require error
checking.  But it would be useful if *other* functions could return Null
iterators, and if iterators could be initialized to Null.

This is similar to how the address of an array element is guaranteed non-null.
No error checking is required.  But a function may return null, and a pointer
may be initialized to null.

> and from a container client perspective, the code is cleaner if flags
> indicating various odd states on the part of the client are seperate and not
> conflating into the iterator, which could care less.

It's often good design to let an object have an `invalid' state.  I agree that
STL containers should not return invalid iterators.  But functions which are
not part of the STL, and have unrelated or expanded functionality, can hand
back iterators defining an STL range.  (This is similar to how a function that
does something unrelated to string handling can return a string!)  Null would
be a particularly compact and convenient way to indicate the function's
failure.

Like `bool', it would be useful if Null were standard, because its use would
be common, and you don't want every library incompatibly defining its own Null
template.

In article <3lk13b$7cb@druid.borland.com>,
Pete Becker <pete@borland.com> wrote:
>In article <D6BL0K.1w8@nntpa.cb.att.com>, mpl@pegasus.bl-els.att.com (-Michael P. Lindner) says:
>>
>>I promised myself I wasn't going to post any more followups to this
>>thread, but here goes.
>
> Don't quit now! You've posted an example that shows what you're
>trying to do. That makes it a lot easier to understand than simply
>describing it in words.

The reason I promised myself this was because this thread (more like a
knot by now :^) has lost all sense of a logical, calm discussion of
technical merit.  It has degraded into name calling and religious debate.
I feel ashamed that my posts have contributed to the resulting
discussions.

>This problem is not unique to iterators. It can apply to any object that
>you create before you use it. In general, the way I handle this is to
>intialize something with a function call. So, in a perhaps overly simple
>case:

 ... example deleted ...

Sorry, not applicable.  The iterator data member exists to remember the
state of the iterator, so you can't just set it from a function every
time.

>>        my_container<int>       c;
>>        my_container<int>::iterator     it = c.begin();
>>        c.insert(5);
>>        // does *it == 5 ? Of course not!
>>        // ++it will also be undefined.
>
>Yes. That is true for all STL containers. Don't save off iterators then
>modify the container. Grab the iterators when they are needed.

No, that's not true for all STL containers.  To quote from "The Standard
Template Library" by Stepanov and Lee, February 7, 1995 (caps for
emphasis are mine):

Vector:
 "insert causes reallocation if the new size is greater than the
 old capacity. If no reallocation happens ALL THE ITERATORS AND
 REFERENCES BEFORE THE INSERTION POINT REMAIN VALID."

List:
 "INSERT DOES NOT AFFECT THE VALIDITY OF ITERATORS AND
 REFERENCES."

Deque:
 "insert and push invalidate all the iterators and references to
 the deque."

Associative containers:
 not specified in the document, but in practice, INSERT DOES NOT
 AFFECT THE VALIDITY OF IOTERATORS AND REFERENCES.

--
Mike Lindner
mikel@attmail.com
mpl@cmprime.attpls.com
mpl@pegasus.att.com

In article <FENSTER.95Apr4160955@ground.cs.columbia.edu>,
fenster@ground.cs.columbia.edu (Sam Fenster) wrote:
... a long message about hist Dict iterator example including:
> I don't.  I need Null.  If the iterators are equal, this does not tell me that
> the operation failed.  It tells me that the operation succeeded, and the
> resulting range exists in a particular container, and it is empty.

Your Dict class does not appear to be a template. It appears to be a
specific class returning a specific type that is an STL iterator. You
are free to define NULL for that type if you wish.

If, in the general case, you define a template Dict<T> class then you
are free to require that iterator<T> has a NULL value defined. No
changes in STL are required to do this. You are free to define a
class that only works with a subset of iterators and iterators with
NULL defined are a genuine (and arguably useful) subset of iterators.

I think the problem is that the STL is a very general collection of
algorithms and conventions and to add something like NULL restricts
the application of STL without adding anything to STL itself. You are
free to define NULL for any iterator class you create, it is only the
abstract concept of an iterator that lacks a NULL value.

Is there any good reason to define an abstract iterator_with_null?
STL itself does not need it, but you and several other people would
obviously like one. If everyone is going to reinvent it then perhaps
it should be standardised. Ideally if your iterator is a class it
should define the null value within the class 'iter.null()', but this
syntax is not pointer compatible.

Reserving a special value such as 0 would be a pain. The best way I
can think of would be to require that for each iterator type ITER
there should be a function 'nulliterator<ITER>()' that returns the
null value. Since this is needed only for your subset of iterators it
should be part of your library documentation (put it in a namespace?)
and not part of the standard.

--
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
           A little inaccuracy sometimes saves tons of explanation.

In article <D6J4Dz.B7x@ucc.su.OZ.AU>,
maxtal@Physics.usyd.edu.au (John Max Skaller) wrote:
>
>  I'm finding more deficiencies in the C++ language
> by using STL than in STL itself. For example, there is NO WAY
> to declare a variable of the type of a member of a parameter:
>
>  template<class T> void f(T t) {
>   _Typeof<t.m> x = t.m;
>  };
>
> I can do this in Metaware HighC/C++ and am doing it. You can use
> 'typeof()' in GNU. The compiler knows the type of t.m and it can
> do overloading on it but you can't make a variable??
>
> In fact what we need is:
>
>  define x = expr;
>
> which means
>
>  typeof(expr) x = expr;
>
Now don't get me wrong. I completely agree with you about needing
something like typeof() or your define keyword, but can't you do
something like the tricks STL plays to 'fix' your example?

I haven't tried this, and I rather suspect my compiler might crash,
but:

template<class T, class M> void f2(T t, M) {
        M x = t.m;
}

template<class T> void f(T t) { f2(t, t.m) }



>  My point -- the C++ language is far more deficient itself
> than STL. STL was designed. C++ was grown, pruned, and hacked.

And regular pruning ensures a good bushy growth :-)

--
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
              Are you still here?  The message is over.  Go away!

Jan-Peter de Ruiter <ruiter@ruls41.LeidenUniv.nl> wrote:
>However, this debate tends to become almost as silly as the hashtable
>debate, and it will probably end the same way too.

Why?  Because neither proposal will become part of the standard?

>What I find sad is that I have the strong suspicion that the opponents
>of hash tables and null valued iterators are not open minded about it.

Someone other than me opposed hash tables being added to the standard?
As near as I can tell I was only one who did.  Are you really so niave
as to think that absolutely everyone would agree with you?  I'm
surprised you even think my opinion matters so much.

>They cling to STL as if any change in it will mean public humiliation
>for the people who designed and supported STL.

I haven't even made up my mind if I like STL yet.

>That attitude is not very productive.

Neither are speculative attacks, but hey, don't worry, I don't
expect a lot productivity on Usenet.

       Ross Ridge

--
 l/  //   Ross Ridge -- The Great HTMU, Ook                    +1 519 883 4329
[oo][oo]  rridge@csclub.uwaterloo.ca      http://csclub.uwaterloo.ca/u/rridge/
-()-/()/
 db  //

In article <3lp2u3$f7u@druid.borland.com>, pete@borland.com (Pete Becker) writes:
> In article <D6BME0.Ezr@cdf.toronto.edu>, g2devi@cdf.toronto.edu (Robert N. Deviasse) says:
> >
> >In article <3lfjcl$qvt@druid.borland.com>,
> >Pete Becker <pete@borland.com> wrote:
> >> What is behind this urge to import all the
> >>known dangers of null pointers into STL?
> >
> >Dangers? How can null pointers be any more dangerous than an out of boundary pointer
> >(which is the alternative)? Neither of these can be safely dereferenced. At least
> >with null pointers you can *test* if it is dereferenceable. How is this more dangerous?
>
> Precisely because it encourages programmers to test for validity rather than assure
> validity. Write the code so that it is impossible (or very difficult) to dereference a
> null pointer. Then you won't have to test for them.

I'm curious. What is your opinion of exceptions in C++? A great many languages
don't need them, and since you appear to believe that prevention is *always*
preferable to being able to deal with recovery. Surely exceptions encourage
programmers to write sloppy code that can generate exceptions rather than
just prevent the error from ever occuring.

IMO, the problem is that we have to live with tradeoffs. When you redesign
your code to prevent certain types of errors instead of checking for them,
you change, among other things, the readability and maintainability of the
code. Quite often, you can increase the total quality of the code and all
is good. Often you make tradeoffs, but their small and it's more than worth
it. There are times however when it's not worth the tradeoffs. This is the
case with null iterators as well as exceptions. And why all the fuss about
testing for validity. You *already* have to. You can't dereference an
iterator to container.end(). Sure you can (IMO always) avoid the problem
(just provide a function that counts the number of elements in the data
structure and never have the iterator incremented more than that number
of times), but is it always worth it?

John recognized the problem and provided, IMO, a better solution than null
iterators. I concede that using exceptions is better than null iterators.
They are self-monitoring (i.e. they will be checked even if I forget to or I
can't because the code is in some library code), they can say more about
an error than just say that it's invalid, and they can be specific to the
iterator in question. Given this, I don't see a reason for null iterators.


>  -- Pete
>
>
>

Take care
    Robert
--
/----------------------------------+------------------------------------------\
| Robert N. Deviasse               |"If we have to re-invent the wheel,       |
| EMAIL: g2devi@cdf.utoronto.ca    |  can we at least make it round this time"|
+----------------------------------+------------------------------------------/

horstman@sjsumcs.sjsu.edu (Cay Horstmann) writes:
> [ .. ]
>It is weird that STL is very defensive in one regard (worst-case
>running time of algorithms) and very rough-and-ready in another
>(no testability of iterator state).
> [ .. ]

Surely that's because the user can do something about the latter
but not usually about the former.  (without writing another
implementation)

--
 -Matt Hannigan

In article <D6Ks8J.Eo8@nntpa.cb.att.com>, mpl@pegasus.bl-els.att.com (-Michael P. Lindner) says:
>
>In article <3lk13b$7cb@druid.borland.com>,
>Pete Becker <pete@borland.com> wrote:
>>This problem is not unique to iterators. It can apply to any object that
>>you create before you use it. In general, the way I handle this is to
>>intialize something with a function call. So, in a perhaps overly simple
>>case:
>
>        ... example deleted ...
>
>Sorry, not applicable.  The iterator data member exists to remember the
>state of the iterator, so you can't just set it from a function every
>time.
>

I don't understand. Wasn't the example about initialization? I suppose there's
a natural extension of the initialization argument, which suggests that you
ought to be able to explicitly test for an invalid state at any time, even
after the iterator has been set to a valid state, but that really doesn't sound
like the iterator model that STL uses.

>>>        my_container<int>       c;
>>>        my_container<int>::iterator     it = c.begin();
>>>        c.insert(5);
>>>        // does *it == 5 ? Of course not!
>>>        // ++it will also be undefined.
>>
>>Yes. That is true for all STL containers. Don't save off iterators then
>>modify the container. Grab the iterators when they are needed.
>
>No, that's not true for all STL containers.  To quote from "The Standard
>Template Library" by Stepanov and Lee, February 7, 1995 (caps for
>emphasis are mine):
>

 Yes, there are cases where it is permissible to save iterators into
particular types of containers. I should have been clearer: I don't think this
is a good design policy. I think it's much more important to be able to
substitute a different container when application profiling reveals that that
the original choice is less than optimal. Relying on properties that change
from container to container makes this sort of substitution much harder.
 -- Pete

Jan-Peter de Ruiter (ruiter@ruls41.LeidenUniv.nl) wrote:
: Tony Cook (tony@online.tmx.com.au) wrote:

: : a) is covered well by the way you should be using STL - using start
: : and end iterators.  I've never seen a pointer incremented to null
: : and I don't really believe it should be.

: Well, OK, maybe it shouldn't be, but in C this is or course frequently
: happening using strings.

Remember though, with strings you are testing the current item under
the iterator (pointer) - not the pointer itself.  There's nothing
stopping you having an iterator for a collection of type T where T
has a conversion to bool.

: : b) has a stronger root in the common "C" pattern of return null to
: : indicate an error (malloc and fopen for example), but this has been
: : overshadowed in C++ by exceptions.  Is there some reason you can't
: : use exceptions for this?  If you can't have you considered, that if
: : you need such a validity test in your own iterators, that you can
: : use some sort of test member function, like good() in IOStreams?

: I agree with this in the sense that I would like to be able to test
: (e.g. using good() ) the validity of an iterator _before_ any exception
: is thrown. Whether that is by using a null value or some test function
: doesn't matter to me. Our own library does both, and it really is handy
: to be able to test validity, especially during debugging.

The point of exceptions is that they remove the _need_ to worry
about detecting failures of this sort, but you can still do it if
you wish:
 // rough syntax here only
 Dict<T>::iterator s, e;
 try {
  s = d.begin();
  e = d.end();

  while (s < e)
   // do something with it
 }
 catch (...)
 {
  // Oh! It failed - do something about it
 }
--
        Tony Cook - tony@online.tmx.com.au
                    100237.3425@compuserve.com

In article 7pc@druid.borland.com, pete@borland.com (Pete Becker) writes:

>    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"

I'm fonder of:

 extern ostream_iterator &output;
 copy(it1, it2, output) ;


---
~THol()

Thomas Holaday
holaday_thomas@jpmorgan.com
tlhol@ibm.net
70407.534@compuserve.com

In article <3lh8bv$edh@jupiter.SJSU.EDU>, horstman@sjsumcs.sjsu.edu (Cay Horstmann) says:
>
>I do
>not know if anyone has ever thought of organizing the iterator hierarchy
>to have "nice" iterators and "classic" iterators (pointers into C
>arrays).

 A caution on terminology here: STL does not have an "iterator
hierarchy" in the sense of having iterator types that inherit from other
iterator types. It has a family of iterators with increasing power, but
there is no requirement that there be any inheritance involved, and the
only use of inheritance in the iterators as implemented by HP is to
reduce the amount of boilerplate code. That is strictly an implementation
technique and a convenience; STL can be implemented in full conformance
to its specification without using these bases.
 In the broader sense of "heirarchy" as a graded series, it is an
appropriate description of STL iterator types.
 -- Pete

In article <D6BL0K.1w8@nntpa.cb.att.com>, mpl@pegasus.bl-els.att.com (-Michael P. Lindner) says:
>
>I promised myself I wasn't going to post any more followups to this
>thread, but here goes.
>

 Don't quit now! You've posted an example that shows what you're
trying to do. That makes it a lot easier to understand than simply
describing it in words.

>Saying "your problem is that 'Dicts' is not STL compliant" is not a
>viable answer.  Making "Dicts" STl compliant simply pushes the iteration
>problem to the implementor of "Dicts" rather than the caller.
>

Yes. That's where it belongs. The goal here is to provide a uniform way
of accessing containers. That makes writing the containers harder, but
ultimately makes containers easier to use. Since a particular container,
if successful, will only be written once but used many times, this is the
right way to allocate the effort.

>In fact, my quest for the null-valued iterator came about from
>implementing complex STL-compliant containers which were aggregates of
>simpler ones.  The ugliness of not having a null has two forms.
>
>1. Testing code correctness:
>   Since the default constructor leaves the iterator in an undefined
>   state, there is no way to define an iterator with a known value.
>
>   This makes code clumsier to debug and makes some assertions difficult
>   (i.e. impossible without defining dummy containers) to write.
>
>        iterator        it;
>        // complex code which _should_ assign a value to it
>        assert(/* it has been assigned a value? */);
>

This problem is not unique to iterators. It can apply to any object that
you create before you use it. In general, the way I handle this is to
intialize something with a function call. So, in a perhaps overly simple
case:

 iterator it;
 if( condition() )
  {
  // do some computations
  it = resultOfComputations;
  }
 else
  {
  // do some other computations
  it = resultOfOtherComputations;
  }

this becomes:

 iterator getIterator()
 {
 if( condition() )
  {
  // do some computations
  return resultOfComputations;
  }
 else
  {
  // do some other computations
  return resultOfOtherComputations;
  }
 }

 iterator it = getIterator();

This way there are no uninitialized objects hanging around, and no need to
test whether something has been initialized. It seems to me that this is
much cleaner and much less error prone than having objects that may have
a real meaning or may be serving as flags that indicate that they do not
have a real meaning, and having to remember to check the flag before
using the object.

>[example omitted]
>
>        OK, now consider the code fragment
>
>        my_container<int>       c;
>        my_container<int>::iterator     it = c.begin();
>        c.insert(5);
>        // does *it == 5 ? Of course not!
>        // ++it will also be undefined.
>

Yes. That is true for all STL containers. Don't save off iterators then
modify the container. Grab the iterators when they are needed. It can be
very expensive to write iterators that "know" when their container has
been modified.

>
>In summary, just as you can program with pointers without using NULL,
>you can program with iterators without needing a null value.

Seems to me that mentioning NULL pointers doesn't help your case. NULL
pointers cause many programming problems. It is not at all clear to me
that they solve more problems than they cause.
 -- Pete

>I just don't understand this mulish reluctance to define singular
>iterator values.  ...
>Can you give me a good reason not to define a singular value for an
>iterator?  Whether I choose to call it null seems immaterial.

Neither STL, nor the C++ Standard Library, imposes any restriction on
defining singular values for your iterators.  In fact, it imposes very
few restrictions of any kind.  (You can make operator!() start up
a Doom session if you like.)  What it does say is that you cannot
count on any random iterator having any given singular value,
and that no conforming algorithm will check for them.

Nathan Myers
myesn@roguewave.com

Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:

: I see, so the standards committee should add
: a feature to an already complex language solely on the basis that it
: will make some programmers happier?

Indeed, it should. Unless it makes other programmers' life
harder, and that is definitely not the case with null valued iterators.

BTW, on what other basis would a standard committee add features?
Platonic beauty? The Truth? The U.S. deficit? I Ching hexagrams?

:    However, this debate tends to become almost as silly as the hashtable
:    debate, and it will probably end the same way too. What I find sad is
:    that I have the strong suspicion that the opponents of hash tables and
:    null valued iterators are not open minded about it. They cling to STL
:    as if any change in it will mean public humiliation for the people
:    who designed and supported STL.

:    That attitude is not very productive.

: Oh, yes, if all else fails, deplore their motives, shedding a mock tear
: or two.

Well, after reading what you seem to think about the purpose of a
standard committee, I feel like shedding another tear. But of course,
you are right that shedding tears will not work. The question is, what
would? There have been numerous code examples around in this thread,
indicating clearly that null values for iterators are often very
convenient.

JP

In article <3lbuob$4q9@druid.borland.com>,
Pete Becker <pete@borland.com> wrote:
>In article <JAK.95Mar28222502@remington.cs.brown.edu>, jak@cs.brown.edu (Jak Kirman) says:
>>
>>>>>>> "Pete" == Pete Becker <pete@borland.com> writes:
>>In article <3l6qhj$kr3@druid.borland.com> pete@borland.com (Pete Becker) writes:
>>
> STL does not in any way prevent you from writing a linked list with
>an iterator that supports an incredibly slow index operation. You can do that
>if you want to, and you can use that iterator in calls to STL algorithms that
>require an index operation.

> One of the major benefits of talking about complexity is that it is a
>concept that is reasonably well understood and generally useful.
>It provides a sound
>theoretical basis for choosing containers and algorithms,
>something which has been
>sorely lacking in container libraries in the past.
> -- Pete

 But the problem is that well defined operations you may
need cannot always be linear or meet STL requirements --
and then you find NO standard container that meets your needs.
You may well be quite happy with a list with slow indexing,
the problem is that you have to WRITE the extra code yourself.

 It would be easy enough to say: "The performance of
this algorithm is logarithmic provided the indexing operation
is constant time" -- and provide the slower indexing operation
anyhow.

 Lacking such referential transparency a decision
to change the underlying container to support fast lookup
and slower insertions (instead of fast insertions and slow lookup)
would require rewriting of code -- rather than just changing
the name of the container, which is what referential
transparency would permit.

 Worse, a configurable hybrid container -- such as a
linked list with a vector cache -- may or may not meet STL requirements
depending entirely on dynamically configurable parameters. Yet
such containers are very useful (since you can tune performance
by adjusting a few numbers).

 I think what this shows is that STL containers are only
raw building blocks: one can build semantically compliant containers
which are not STL compliant only by virtue of the perforamce
requirements.

 It is uncertain if, in fact, such complexity requirements
are "normative" since they are deducible from code -- but
cannot be measured.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <FENSTER.95Mar28174405@ground.cs.columbia.edu>,
Sam Fenster <fenster@ground.cs.columbia.edu> wrote:

>that it's invalid.  Such a value is useful if an object must be instantiated
>before it's initialized to something useful.  Or if it is the result of some
>operation that may fail.  NULL is used for these purposes with pointers.
>
>If someone hands me two iterators, it is useful to be able to check them for
>validity.  Do you claim that in every such situation, there will be a third
>iterator somewhere in my environment, the end of some larger range?  Do you
>want me to redesign my problem so that such a third iterator exists (and is
>made public), *just so I have something to test my two iterators against*?
>That complicates my interface considerably!

 You are perfectly free to provide NULL iterators for your
containers.

 It simply isn't a requirement that everyone else do this
for theirs. If you write an algorithm which does NULL checks it
will not operate with all iterators, but only that subset
for which a NULL is defined -- it is your perogative
to define extensions of STL.

 STL itself does not mandate NULL iterators simply
because none of the algorithms need such a value.

 STL iterators CANNOT in general be checked for validity.
It is the reponsibility of the CALLER of an algorithm to ensure
the iterators passed meet the pre-conditions of the algorithm.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <MATT.95Mar30102657@physics2.berkeley.edu>,
Matt Austern <matt@physics.berkeley.edu> wrote:

>every iterator type should have one singular value that can be
>tested for.
>
>I don't pretend that this is necessary; I do claim, however, that it
>would be convenient.  There are a few cases where having such a
>testable singular value would make certain tasks easier.  To my mind,
>the benefit of having null iterators is small but nonzero, and the
>cost is zero.

 How can you be sure the cost would be zero?
If you use an unsigned integer as an iterator, what singular value
would you have?

 Any value you pick makes modular arithmetic and
the ability to address a binary power of two memory
locations impossible. This is a serious problem, and it can cost
up to four bytes on some machines to correct. 4 bytes != 0.

 It seems to me that singular values are possessed by
certain types and not others -- and STL does not require
one coerce a type to have an unnatural property.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <D6482o.KtJ@reston.icl.com>,
Stephen Carlson <scc@reston.icl.com> wrote:
>In article <KINNUCAN.95Mar22150251@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:
>>   If iterators are to be a generalization of pointers, shouldn't one of
>>   their most important behaviors (the ability to be null) also be part
>>   of the generalization?
>>
>>No.
>
>Would you care to explain this answer?  Why should a "generalization of
>pointers" not include an important property of pointers?


 Because that property is not useful for making the
_algorithms_ of STl work.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <3lfjcl$qvt@druid.borland.com>,
Pete Becker <pete@borland.com> wrote:

>sort of thing. Don't create the iterator until you are ready to initialize
>it. Make sure it goes out of scope when it becomes invalid. This is just
>sound programming practice.

 If it is possible. And often it isn't.
The simplest case is if the constructor can fail and you want
to trap an exception.  Bad luck, a try block is a block,
so you can't declare the iterator IN the try block, you may have to
write some painful thing like:

 iterator i;
 try { i = make_iterator(); } catch(...) { ... }

[I agree with your comments but just point out not everything
nests cleanly. Now if we had threads or coroutines ... ]

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

Jan-Peter de Ruiter (ruiter@ruls41.LeidenUniv.nl) wrote:
: Well, after reading what you seem to think about the purpose of a
: standard committee, I feel like shedding another tear. But of course,
: you are right that shedding tears will not work. The question is, what
: would? There have been numerous code examples around in this thread,
: indicating clearly that null values for iterators are often very
: convenient.

There have been two main uses for null iterators given here:
 a) indicating the end of a container, ala linked list
 b) indicating an error (a find that found an internal error?).

a) is covered well by the way you should be using STL - using start
and end iterators.  I've never seen a pointer incremented to null
and I don't really believe it should be.

b) has a stronger root in the common "C" pattern of return null to
indicate an error (malloc and fopen for example), but this has been
overshadowed in C++ by exceptions.  Is there some reason you can't
use exceptions for this?  If you can't have you considered, that if
you need such a validity test in your own iterators, that you can
use some sort of test member function, like good() in IOStreams?
--
        Tony Cook - tony@online.tmx.com.au
                    100237.3425@compuserve.com

In article <D69noL.FvK@ucc.su.OZ.AU>, maxtal@Physics.usyd.edu.au (John Max
Skaller) writes:

> I personally believe referential transparency is
>MORE imporatant than performance guarrantees. It is more
>important to get it right first -- and THEN tune performance
>where it matters by switching to a better suited container.
>
> STL actually interferes with that technique --
>which virtual function polymorphism guarrantees.
>
> I'm NOT saying STL is wrong or bad -- it is
>very important to have design goals AND STICK TO THEM.
>
> But it is my guess there needs to be a layer
>on top of STL that provides referential transparency
>_irrespective_ of performance. Because in 90% of code,
>performance just doesn't matter. Correctness matters
>in 100% of code :-)
>

Agreed 110%. (:-))

But seriously, this is absolutely right. I still think STL is fine for
experts, but will prove less than robust in the hands of less expert
programmers.

That's a shame.

Yes, I can implement something on top for those less expert programmers.

But then, it would be nice not to have to.

Ian
--
Consulting for Digital Equipment Corp  johnston@caiman.enet.dec.com
      (33) 92.95.51.74

In article <MATT.95Mar30102657@physics2.berkeley.edu> matt@physics.berkeley.edu writes:

> Some of us, though, suggest a more modest
> change: every iterator type should have one singular value that can be
> tested for.

> I don't pretend that this is necessary; I do claim, however, that it
> would be convenient.  There are a few cases where having such a
> testable singular value would make certain tasks easier.  To my mind,
> the benefit of having null iterators is small but nonzero, and the
> cost is zero.

The cost is certainly not zero, because in general, comparison
operators for iterators will have to cater to the possibility
of such testable singular values.  I will agree, though, that
the cost is small.

But it may well be too late to do anything about it.  At the last
committee meeting, when the proposal for hash tables was brought
up, it was rejected without even considering its technical merits
on the basis that the committee had decided to close the door on
extensions.

Indeed, that sentiment has been growing for well over a year.
STL itself just barely made it in the door before the door closed.
--
    --Andrew Koenig
      ark@research.att.com

In article <D69vJt.Cww@research.att.com>, ark@research.att.com (Andrew Koenig) writes:
|> In article <MATT.95Mar30102657@physics2.berkeley.edu> matt@physics.berkeley.edu writes:
|>
|> > Some of us, though, suggest a more modest
|> > change: every iterator type should have one singular value that can be
|> > tested for.
|>
|> > I don't pretend that this is necessary; I do claim, however, that it
|> > would be convenient.  There are a few cases where having such a
|> > testable singular value would make certain tasks easier.  To my mind,
|> > the benefit of having null iterators is small but nonzero, and the
|> > cost is zero.
|>
|> The cost is certainly not zero, because in general, comparison
|> operators for iterators will have to cater to the possibility
|> of such testable singular values.  I will agree, though, that

Would it really?  I can write code that compares two pointers, and of course,
there is the possibility that one (or both) of them is (are) NULL.  I don't see
how the language definition requires any special treatment to 'cater' to this
possibility.  It seems to me that exactly the same can be done wrt iterators.

|> the cost is small.

 [ stuff deleted ]
|> --
|>     --Andrew Koenig
|>       ark@research.att.com

--
---------------------------------------------------------------------
 Ed Osinski
 Computer Science Department, New York University
 E-mail:  osinski@cs.nyu.edu
---------------------------------------------------------------------
In the early years of the 16th century, to combat the rising tide of
religious unorthodoxy, the Pope gave Cardinal Ximinez of Spain leave
to move without let or hindrance throughout the land, in a reign of
violence, terror and torture that makes a smashing film.  This was
the Spanish Inquisition...
    -- Monty Python's Flying Circus

In article <FENSTER.95Mar27182435@ground.cs.columbia.edu>, fenster@ground.cs.columbia.edu (Sam Fenster) writes:
|>
|> You compare an iterator against an end iterator when it's traversing/searching
|> a pre-existing range.  Pre-existing ranges *have* end iterators.  When someone
|> hands you an iterator that helps *define* a range, it's serving a different
|> purpose.  There may be no other range in sight to compare it to.  Then you
|> need Null as a standard way to signal invalidity.

You missed it again!

The way to signal this is to use a begin-end pair where the begin iterator
tests equal to the end iterator.  This represents an empty range, since
the end iterator points "one past the end".

--
swf@elsegundoca.attgis.com  sarima@netcom.com

The peace of God be with you.

In article <JAK.95Mar29223621@remington.cs.brown.edu> jak@cs.brown.edu (Jak Kirman) writes:

   >>>>> "Pete" == Pete Becker <pete@borland.com> writes:

    Pete> Using the STL model, this code would be written:

    Pete>     Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
    Pete>     if (it1 == Dicts:end()) error();

    Pete>     Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
    Pete>     if (it2 == Dicts:end()) error();

    Pete>     for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
    Pete>     return 0;}

    Pete> I just don't understand this insistence that Null is
    Pete> important. It does not make this example any simpler to write. It
    Pete> does make the entire model more complicated, because you end up
    Pete> with two different ways to say the same thing: return an
    Pete> off-the-end iterator or return a Null iterator. How can I write
    Pete> general purpose code in such a case? Do I always have to check
    Pete> for both?  -- Pete

   I just don't understand this mulish reluctance to define singular
   iterator values.  A past-the-end value is not the same as a singular
   value; how would you distinguish between a successful call to
   Dicts::end_of_section() for the last section, which would presumably
   return the past-the-end iterator value, and a failed call?

I just don't understand this mindless insistence on generalization
for its own sake, without regard to any other consideration. :-)

   Can you give me a good reason not to define a singular value for an
   iterator?  Whether I choose to call it null seems immaterial.

He already has.  In fact, three good reasons have been stated by
Pete and others in this thread:

1. Redundant.

You can't do anything with null-valued iterators that you can't
do just as easily, clearly, and economically without them.

2. Dangerous.

Encourages careless programmers to think that they can safely incr/decr
isolated iterators, for example, as one post to this thread
suggested, to explore the region around an isolated iterator.

3. Complex.

Null iterators introduce additional (and needless) complexity
into the design and implementation of STL containers and applications.
For example, it raises the issue of when, if ever, STL containers
should return null values. It raises the question of how functions
that return a range should indicate an empty range, etc., etc., etc.

In short, defining singular iterators would ruin the elegant symmetry
of STL's design without producing any real benefit in recompense.

- Paul

: BTW, one of the reason people wanted to have a null value is to be
able to
: check if an iterator has been initialized or made invalid, assuming that
: the programmer follows defensive a programming style. For example:

STL plainly doesn't support defensive programming in that regard. You
cannot test if an iterator is dereferenceable. You have to know. The
argument goes that pointers into C arrays have the same property. I do
not know if anyone has ever thought of organizing the iterator hierarchy
to have "nice" iterators and "classic" iterators (pointers into C
arrays).

It is weird that STL is very defensive in one regard (worst-case
running time of algorithms) and very rough-and-ready in another
(no testability of iterator state).


: Still, it seems to me that STL iterators should include more pointer properties
: than it currently does.

I wish you wouldn't ruin your own case by falling back onto C
pointers. C pointers are not safe. You should say "Still, it seems
that STL iterators should behave more like good quality C++ iterators."

Cay

In article <D688s0.60A@cdf.toronto.edu>, g2devi@cdf.toronto.edu (Robert N. Deviasse) says:
>
>BTW, one of the reason people wanted to have a null value is to be able to
>check if an iterator has been initialized or made invalid, assuming that
>the programmer follows defensive a programming style. For example:
>
>      Container c;
>      Iter i=Null<Iter>();
>      ... // (*)
>      // we assume that 'i' has been set to a value in 'c'
>      assert(i!=Null<Iter>());
>      ... // (**)
>      invalidate(c);           // append or some other operation
>      DEBUG(i=Null<Iter>());   // used only for defensive programming
>      ...
>
>I can't see how we can emulate this with the end() iterator.

 No, I don't see offhand how to do it, either. Again, I'll ask: why
do you want to do this? There are much safer ways of programming this
sort of thing. Don't create the iterator until you are ready to initialize
it. Make sure it goes out of scope when it becomes invalid. This is just
sound programming practice. I agree that if you want to write dangerous
code a Null iterator may be able to make you think you've made it a bit
safer. I don't think that's really the case: it certainly hasn't worked
out that way with pointers. What is behind this urge to import all the
known dangers of null pointers into STL?
 -- Pete

In article <FENSTER.95Mar28174405@ground.cs.columbia.edu>,
Sam Fenster <fenster@ground.cs.columbia.edu> wrote:
>>>>    fenster@ground.cs.columbia.edu (Sam Fenster) wrote:
>
>    #include "dictionaries.h"   // Defines namespace Dicts.
>    int main () {
>    // Dicts is a module that accesses global containers and files that we
>    // can't see.  It is *not* a single container.  Its data is *not* public.
>
>    Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
>    if (it1 == Dicts::Null) error();
>
>    Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
>    if (it2 == Dicts::Null) error();
>
>    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
>    return 0;}

Q1: If I change the second call to:
    Dicts::iterator it2 = Dicts::look_up_word ("zoo");
    is it2 reachable from it1? (That is, is
       while (it1 != it2) {++it1;};
    guarenteed to stop? If so, then your dictionary is "logically" one
    container (even if it might be implemented as many containers).

Q2: OK, if you still don't think the dictionary is one container, how
    is your use of Dict::Null any different than using Dict::end() in
    your above example. The current STL definition uses container::end()
    the exact same way that you are using Dict::Null in the above code.
    (i.e. STL uses the end of the container to indicate an "invalid"
    iterator, or

[...]
>If someone hands me two iterators, it is useful to be able to check them for
>validity.  Do you claim that in every such situation, there will be a third
>iterator somewhere in my environment, the end of some larger range?  Do you
>want me to redesign my problem so that such a third iterator exists (and is
>made public), *just so I have something to test my two iterators against*?
>That complicates my interface considerably!

If you get 2 STL iterators as a range and the iterators are equal, then
there are no items in the "container" that is returned. This means that the
iterators are valid (they point into some container), but that there is no
valid data to be returned. Why do you need a third iterator?

>Modeling my collection of dictionaries as a single large container doesn't
>have any desirable semantics for me!  Why impose an ordering between the last
>word in one dictionary and the first word in some other dictionary?  The STL
>philosophy is not, "Make *everything* a container."  You just want me to do it
>so I'll have a phony "end iterator" to compare my real iterators against to
>test for validity.  (I would never even use my phony "start iterator.")
>
>The availability of Null would make the design much cleaner.

But STL is a library of containers (and things you can do with those
containers). If your programming model doesn't fit this (your dictionary is
not a container, then it doesn't fit the STL model, and you should not be
using the STL semantics if they don't fit your model). The example you
posted above does (as far as I can tell) fall into the STL model. Let me
change it slightly:
   Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
   if (it1 == Dicts::end_of_section(it1)) error();
   // or maybe: if (it1 == Dicts::end_of_section("the")) error();

it1 is a single value. I can check whether it is a valid item in the
container of T's by checking it against the end of section of T's.

>    Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
>    if (it2 == Dicts::Null) error();

How can your end_of_section return an invalid iterator? Remember that STL
iterators point into a container, and possibly one place past the end of the
container. (i.e. your Dicts::end_of_section() for T's should not return
"Tzar", but "Tzar" + 1.) There is always an "end of section" iterator in
STL, even though that "end of section" may point to a valid item in the
container. That is, even if you implemented the Dict as one huge container,
where "uang" follows "tzar", end_of_section("the") might be a valid iterator
that happens to point to "uang" (within the whole of the Dict), but if it1
is equal to this iterator, it indicates that it1 is not pointing to a valid
entry in the T section.

eddy

--
ed.dy \'ed-e-\ n [ME (Sc dial.) ydy, prob. fr. ON itha; akin to OHG ith-
   again], L et and 1a: a current of water or air running contrary to the main
   current; esp)X : a small whirlpool 1b: a substance moving similarly  2: a
   contrary or circular current  - eddy vb

In article <3lfjcl$qvt@druid.borland.com>,
Pete Becker <pete@borland.com> wrote:
>In article <D688s0.60A@cdf.toronto.edu>, g2devi@cdf.toronto.edu (Robert N. Deviasse) says:
>>
>>BTW, one of the reason people wanted to have a null value is to be able to
>>check if an iterator has been initialized or made invalid, assuming that
>>the programmer follows defensive a programming style. For example:
>>
>>      Container c;
>>      Iter i=Null<Iter>();
>>      ... // (*)
>>      // we assume that 'i' has been set to a value in 'c'
>>      assert(i!=Null<Iter>());
>>      ... // (**)
>>      invalidate(c);           // append or some other operation
>>      DEBUG(i=Null<Iter>());   // used only for defensive programming
>>      ...
>>
>>I can't see how we can emulate this with the end() iterator.
>
> No, I don't see offhand how to do it, either. Again, I'll ask: why
>do you want to do this? There are much safer ways of programming this
>sort of thing. Don't create the iterator until you are ready to initialize
>it.

Unfortunately this isn't always possible. Consider this the following:
      Container c;
      Iter i=Null<Iter>();
      if (condition1()){
         ... set i wrt c appropriately
      }else if (condition2()){
         ... set i wrt c appropriately
      } else {
         ... set i wrt c appropriately
      }

      // As a sanity check I want to ensure that i has been initialized.
      // This also provides *executable* documentation on my expectations at this point.
      assert(i!=Null<Iter>());

The problem is that sometimes the calculation of an initialization value is
conditional upon the state of the system at the time of initialization. Given the
nature of conditionals, how can you *practically* get around this problem?

> Make sure it goes out of scope when it becomes invalid.

Is this always possible? Remember that if an iterator can become invalid when
Consider this example:    (Note DEBUG(x) is a macro that expands to x)

    Container c;
    ...
    Iter i=c.start();
    ...
    Container d;
    ...
    Iter j=c.start();
    do_something_that_changes_a_Container_structure(c);   // now i is invalid
    DEBUG(i=Null<Iter>());                                // document that i is invalid

At this point, i is invalid, but j is still needed, so we can't just end the scope
here.

> This is just
>sound programming practice. I agree that if you want to write dangerous
>code a Null iterator may be able to make you think you've made it a bit
>safer.

You've misread me. I do follow this preventative style *when* possible, but it's
not always possible, and in those cases I've learned that defensive programming
practises can help tremendously.

> I don't think that's really the case: it certainly hasn't worked
>out that way with pointers.

I don't follow. You mean that you've never seen or used assert statements to ensure
that your pointers are non-null?

> What is behind this urge to import all the
>known dangers of null pointers into STL?

Dangers? How can null pointers be any more dangerous than an out of boundary pointer
(which is the alternative)? Neither of these can be safely dereferenced. At least
with null pointers you can *test* if it is dereferenceable. How is this more dangerous?
Personally I much prefer my programs to crash because of a dereferenced null pointer
over a dereferenced out of boundary pointer. The first type of error can either be
caught by modern operating systems or liberal use of assert statements. The later often
causes nearby data to be corrupted (possibly even the virtual function table pointer of
another object). These sorts of errors are much harder to detect and debug, IMO.

> -- Pete
>


Take care
    Robert
--
/----------------------------------+------------------------------------------\
| Robert N. Deviasse               |"If we have to re-invent the wheel,       |
| EMAIL: g2devi@cdf.utoronto.ca    |  can we at least make it round this time"|
+----------------------------------+------------------------------------------/

In article <KINNUCAN.95Mar29071930@candide.hq.ileaf.com>,
Paul Kinnucan <kinnucan@hq.ileaf.com> wrote:
>In article <3l9976$qek@druid.borland.com> pete@borland.com (Pete Becker) writes:
>
>
>   In article <FENSTER.95Mar27113503@ground.cs.columbia.edu>, fenster@ground.cs.columbia.edu (Sam Fenster) says:
>
>    Fundamentally this argument says: I want to design Dict in a way
>   that does not obey the STL design rules, so I need a Null iterator. There is
>   another possible solution: obey the STL design rules.
>
>Yes. A compulsion to concoct examples that demonstrate a need for
>an alien construct like null iterators can lead proponents to overlook
>simpler solutions based on native STL idioms, e.g.,
>
>Dicts::iterator start;
>Dicts::iterator end;
>Dicts::find_section_starting_at("the", start, end);
>if (start != end)
>  for (Dicts::iterator i=start; i!=end; ++i)  output (*i);
>- Paul

I promised myself I wasn't going to post any more followups to this
thread, but here goes.

Saying "your problem is that 'Dicts' is not STL compliant" is not a
viable answer.  Making "Dicts" STl compliant simply pushes the iteration
problem to the implementor of "Dicts" rather than the caller.

In fact, my quest for the null-valued iterator came about from
implementing complex STL-compliant containers which were aggregates of
simpler ones.  The ugliness of not having a null has two forms.

1. Testing code correctness:
   Since the default constructor leaves the iterator in an undefined
   state, there is no way to define an iterator with a known value.

   This makes code clumsier to debug and makes some assertions difficult
   (i.e. impossible without defining dummy containers) to write.

 iterator it;
 // complex code which _should_ assign a value to it
 assert(/* it has been assigned a value? */);

   It would be nice if there was a way to specify a known value for "it"
   and test for that value without having to have a container to which
   "it" refers.  It would be even nicer if we all wrote perfect code the
   first time, and didn't need to debug or assert anything :^)

2. Writing aggregate STl containers:
   Consider the case of a container that's implemented as a container of
   containers of X.  In order to make this container STL compliant, we
   have to write iterators that conform to the spec.  The most obvious
   way (to me) is something like:

 template<class X>
 class my_container {
  typedef container1<X> rep_rep_type;
  typedef container2<container1<X> > rep_type;
  class iterator;
  friend iterator;
  rep_type rep;
 public:
  class iterator {
   friend my_container<X>;
   rep_type& rep;
   rep_type::iterator a;
   rep_rep_type::iterator b;
   iterator(rep_type&, rep_type::iterator, rep_rep_type);
  public:
   iterator operator ++ () {
    // "while" could be "if" if we knew none
    // of the sub-containers were empty
    while (b == (*a).end()) {
     if (++a != rep.end())
      b = (*a).begin();
     else
      break;
    }
   }
   // other stuff
  }
  iterator begin() {
   return iterator(rep, rep.begin(), rep.begin() !=
    rep.end() ? (*rep.begin()).begin() :
    rep_rep_type::iterator());
  }
  // other stuff
 }

 OK, now consider the code fragment

 my_container<int> c;
 my_container<int>::iterator it = c.begin();
 c.insert(5);
 // does *it == 5 ? Of course not!
 // ++it will also be undefined.

   In fact, each sub-iterator now needs a flag associated with it to
   tell whether it is initialized or not.  Yes, one possible solution is
   to have a class static rep_rep_type in iterator, and have the dummy
   container's end() represent a null, but that's just inventing lots of
   little nulls.  Again, the obvious (to me) solution is a way to
   initialize an iterator to a known value without needing a container
   to which it refers.

In summary, just as you can program with pointers without using NULL,
you can program with iterators without needing a null value.  But I
claim, based on the examples described above, that the code will be
harder to test and debug, or you will wind up inventing sentinal values
to replace the lack of null.

Moving the problem into a container class does not make it go away.

--
Mike Lindner
mikel@attmail.com
mpl@cmprime.attpls.com
mpl@pegasus.att.com

Paul J. Ste. Marie (pstemari@erinet.com) wrote:
> In article <MATT.95Mar25154326@physics7.berkeley.edu>,
>    matt@physics7.berkeley.edu (Matt Austern) wrote:
> :In article <3kvrpd$h3d@druid.borland.com> pete@borland.com (Pete
> Becker) writes:
> :
> :>  Huh? Please provide a sample implementation of an iterator
> :> that can safely increment to null for, say, a vector of objects
> :> of type Foo.
> :
> :Nobody has ever suggested that as an extension to the STL.

I believe someone did, but it died quickly (and rightly so).

> Beg pardon?  Every example I've seen here of a null iterator has
> requested that incrementing an iterator past the end of the
> container would return null.  At least one headache with this is
> that any algorithm that depended on this behavior would be unusable
> with the simplest sort of container/iterator, a vanilla array and a
> pointer into it.

> The usual code snippet cited in the examples has been:

>  while (++iterator)

> which doesn't work with pointers, no way, no how.

The idea is to _increase_ compatibility with pointers, not decrease it. IMVHO,
it only makes sense to:

> :The actual suggestion that has been made is much more modest: for
> :every type of iterator, a null value must be defined.  It must be
> :possible to assign null to an iterator, it must be possible to
> :compare an iterator for equality to null, and it is guaranteed
> :that no valid iterator will ever compare equal to null.  I don't
> :see that this would introduce any overhead at all.

> It wouldn't--but the request was that all invalid iterators would
> compare equal to null, not that no valid iterator would equal null.

Again, the idea is to preserve pointer programming idioms. Not all invalid
pointers compare equal to NULL, so why should iterators? The one-after-last
pointer, in particular, is a useful invalid pointer which does not compare
equal to NULL, and already has an equivalent in STL. However all NULL pointers
are invalid. That is all the suggestion requires for iterators, as well.

      Oren.

--
Life is tough and the hard ships are many.

In article <3lbuob$4q9@druid.borland.com>, pete@borland.com (Pete Becker) says:
>
>In article <JAK.95Mar28222502@remington.cs.brown.edu>, jak@cs.brown.edu (Jak Kirman) says:
>>
>>>>>>> "Pete" == Pete Becker <pete@borland.com> writes:
>>In article <3l6qhj$kr3@druid.borland.com> pete@borland.com (Pete Becker) writes:
>>
>> Pete>  One of the design goals of STL is to be able to guarantee worst
>> Pete> case performance for algorithms. This is done by not supporting operations
>> Pete> that are "too expensive" on a particular container. This results in a
>> Pete> compile-time error when you try to use them, rather than a program that
>> Pete> runs at a glacial pace. It is certainly possible to design a library with
>> Pete> a different set of design goals. Personally, I prefer compile-time errors
>> Pete> to documentation that says "don't do this". I don't see any benefit in
>> Pete> being able to ask for a quicksort() on a linked list when there are much
>> Pete> better ways of sorting a list.
>>
>>For example, to show people how slow it is.  Why would you think people
>>agree on what "too slow" is, anyway?  Asymptotic complexity is often not
>>a very useful criterion.
>>
>
>        STL does not in any way prevent you from writing a linked list with
>an iterator that supports an incredibly slow index operation. You can do that
>if you want to, and you can use that iterator in calls to STL algorithms that
>require an index operation. You can put such a class in a commercial product and
>tell people that your product supports indexing into a linked list. You cannot,
>however, claim that such an iterator conforms to the STL requirements, because it
>violates the requirement that indexing occur in constant time.
>        One of the major benefits of talking about complexity is that it is a
>concept that is reasonably well understood and generally useful. It provides a sound
>theoretical basis for choosing containers and algorithms, something which has been
>sorely lacking in container libraries in the past.
>        -- Pete

 On second thought, I left out something important: if you follow
the performance requirements for STL iterators then you know the worst
case performance of the algorithms that you use. If you provide iterators
that do not meet these requirements, you have no basis for drawing any
conclusions about the performance of the algorithms. This is a significant
loss, regardless of whether you think asymptotic copmlexity is at times
not useful.
 -- Pete

In article <FENSTER.95Mar27182435@ground.cs.columbia.edu>, fenster@ground.cs.columbia.edu (Sam Fenster) says:
>
>    #include "dictionaries.h"
>    int main () {using namespace Dicts;
>    // Dicts is a module that accesses global containers and files that we
>    // can't see.  It is *not* a single container.  Its data is *not* public.
>
>    Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
>    if (it1 == Dicts:Null) error();
>
>    Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
>    if (it2 == Dicts:Null) error();
>
>    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
>    return 0;}

Using the STL model, this code would be written:

    Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
    if (it1 == Dicts:end()) error();

    Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
    if (it2 == Dicts:end()) error();

    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
    return 0;}

I just don't understand this insistence that Null is important. It does not
make this example any simpler to write. It does make the entire model more
complicated, because you end up with two different ways to say the same thing:
return an off-the-end iterator or return a Null iterator. How can I write
general purpose code in such a case? Do I always have to check for both?
 -- Pete

In article <D6482o.KtJ@reston.icl.com> scc@reston.icl.com (Stephen Carlson) writes:

   In article <KINNUCAN.95Mar22150251@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:
   >   If iterators are to be a generalization of pointers, shouldn't one of
   >   their most important behaviors (the ability to be null) also be part
   >   of the generalization?
   >
   >No.

   Would you care to explain this answer.

I apologize for not explaining the curtness of my reply.

I want to avoid boring those who have been following this
thread from the beginning by repeating explanations that I
have given in previous posts.

If you are still interested in why I oppose extending
STL to support null iterators, please contact me by
e-mail. I'd be be glad to explain my position to you
off-line.

- Paul

In article <3lc5hg$7pc@druid.borland.com>,
Pete Becker <pete@borland.com> wrote:
>In article <FENSTER.95Mar27182435@ground.cs.columbia.edu>, fenster@ground.cs.columbia.edu (Sam Fenster) says:
>>
>>    #include "dictionaries.h"
>>    int main () {using namespace Dicts;
>>    // Dicts is a module that accesses global containers and files that we
>>    // can't see.  It is *not* a single container.  Its data is *not* public.
>>
>>    Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
>>    if (it1 == Dicts:Null) error();
>>
>>    Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
>>    if (it2 == Dicts:Null) error();
>>
>>    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
>>    return 0;}
>
>Using the STL model, this code would be written:
>
>    Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
>    if (it1 == Dicts:end()) error();
>
>    Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
>    if (it2 == Dicts:end()) error();
>
>    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
>    return 0;}
>
>I just don't understand this insistence that Null is important. It does not
>make this example any simpler to write. It does make the entire model more
>complicated, because you end up with two different ways to say the same thing:
>return an off-the-end iterator or return a Null iterator. How can I write
>general purpose code in such a case? Do I always have to check for both?

Actually you don't. If x.start()==x.end() then we can't dereference x. Does
it matter whether x.end() is an arbitrary value or Null<Iter>()? There is
one advantage of using Null<Iter>() over x.end() to denote an invalid value,
it does not become undefined if an operation like append() does not invalidate
the iterator. This makes the Null<Iter>() approach a little less error prone.

BTW, one of the reason people wanted to have a null value is to be able to
check if an iterator has been initialized or made invalid, assuming that
the programmer follows defensive a programming style. For example:

      Container c;
      Iter i=Null<Iter>();
      ... // (*)
      // we assume that 'i' has been set to a value in 'c'
      assert(i!=Null<Iter>());
      ... // (**)
      invalidate(c);           // append or some other operation
      DEBUG(i=Null<Iter>());   // used only for defensive programming
      ...

I can't see how we can emulate this with the end() iterator. Why? Replace
Null<Iter>() by c.end() in the above and c.append(value) with the comment
in (*) to get:
      Container c;
      Iter i=c.end();
      c.append(value);
      assert(i!=c.end());
Now even though 'i' wasn't set, the assert statement fails.

Fortunately Null<Iter>() can be simulated in C++ (by declaring a null container
and setting specializing the function iterator Null<Iter>() to the end of
that iterator, or if you're using pointers we can just return NULL).

Still, it seems to me that STL iterators should include more pointer properties
than it currently does.

> -- Pete


Take care
    Robert
--
/----------------------------------+------------------------------------------\
| Robert N. Deviasse               |"If we have to re-invent the wheel,       |
| EMAIL: g2devi@cdf.utoronto.ca    |  can we at least make it round this time"|
+----------------------------------+------------------------------------------/

>>>>> "Pete" == Pete Becker <pete@borland.com> writes:

 Pete> Using the STL model, this code would be written:

 Pete>     Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
 Pete>     if (it1 == Dicts:end()) error();

 Pete>     Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
 Pete>     if (it2 == Dicts:end()) error();

 Pete>     for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
 Pete>     return 0;}

 Pete> I just don't understand this insistence that Null is
 Pete> important. It does not make this example any simpler to write. It
 Pete> does make the entire model more complicated, because you end up
 Pete> with two different ways to say the same thing: return an
 Pete> off-the-end iterator or return a Null iterator. How can I write
 Pete> general purpose code in such a case? Do I always have to check
 Pete> for both?  -- Pete

I just don't understand this mulish reluctance to define singular
iterator values.  A past-the-end value is not the same as a singular
value; how would you distinguish between a successful call to
Dicts::end_of_section() for the last section, which would presumably
return the past-the-end iterator value, and a failed call?

Can you give me a good reason not to define a singular value for an
iterator?  Whether I choose to call it null seems immaterial.

                         Jak Kirman                        jak@cs.brown.edu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Ah, but a man's reach should exceed his grasp, or what's a heaven for?
                                                          -- Robert Browning

Paul Kinnucan <kinnucan@hq.ileaf.com> writes:
> Again, this begs the question as to just what other types of "errors" (other
> than empty range) there could be in properly designed operations involving
> STL iterators.

I think the general point you're missing is that things other than direct,
immediate access to an STL container might yield an iterator pair.  In such
cases, there may be error conditions unrelated to STL containers.  These might
still be most conveniently signaled by returning an invalid value in the
iterator pair.  An empty range would be a valid result, and so would not
signal such a condition.

For instance: I pass a function a database query string.  It passes me back an
iterator pair which lets me traverse (or search) the list of query results:

#include "db.h"
int main ()
{
   DB_Handle h = db_open("personnel");  // A database of tables
   if (!h) {error("Couldn't open table"); exit(1);}

   string query;
   cin.getline(query);  // "select name from employee where salary > 50000"

   DB_iterator_pair p = db_query(h, query);
   if (p.i1 == Null<DB_iterator>())  {error("Invalid query"); exit(2);}

   for (DB_iterator i=p.i1; i!=p.i2; ++i)  cout << *i << endl;
   return 0;
}

Pete Becker <pete@borland.com> writes:
> Use exceptions to indicate errors. And, to anticipate one possible response,
> if you don't want to use exceptions to handle "routine" errors, please
> describe what these errors would be and why an STL algorithm would run into
> such an error.

See above.  Functions that are not "STL algorithms" may return STL ranges.  Of
course, any error handling strategy may be replaced by exceptions.  You could
get rid of NULL pointers and force everyone to use exceptions there, too.

Unfortunately, there is no way to stop exceptions from causing program
termination, so they should be avoided, except for extreme errors.  If your
exception-throwing function ever gets called by a destructor during stack
unwinding for an unrelated exception, for instance, your program will
terminate.  Or if the copy constructor of the thrown object throws an
unrelated exception.  Or if your exception is unhandled.  The more unrelated
subsystems use exceptions for non-extreme errors, the more likely your program
is to terminate unexpectedly.

Pete Becker <pete@borland.com> writes:
> I just don't understand this insistence that Null is important. It does not
> make this example any simpler to write. It does make the entire model more
> complicated, because you end up with two different ways to say the same
> thing: return an off-the-end iterator or return a Null iterator. How can I
> write general purpose code in such a case? Do I always have to check for
> both?

No.  When the iterator you're handed is supposed to indicate an element in a
known iterator range, you compare it to that range's end iterator.  When it's
not, you can't, and don't.  When you are handed an iterator pair, it will
often denote a range that is not drawn from a known larger range.

: My challenge stands: either produce a (properly designed) STL code example that
: requires a null iterator or shut up.
:

Look, Paul, most languages have the theoretical power of the Turing
Machine, and so does C++ + STL. Therefore, it's impossible to think
of a problem that's not solvable using STL-without-null-value-iterators.
What you say is similar to "show me a piece of C code and I show you
how you can write it without function calls".

The numerous examples you believe you have shot down, are all
indications that it's _convenient_ to have null valued iterators.
Convenient, in the cognitive ergonomical sense that the programmer
is happier using them and that another programmer will find it
easier to understand the code.

Granted, these things are subjective, and therefore hard to discuss.
However, I see no advantages in turning the debate in a "challenge",
particularly not the challenge of writing STL code so that you can
show us how YOU (who are apparently sympathetic towards STL's
constraints) would write it. Sure, I believe you'd write it in
a different way, but you're probably not the only programmer that's
going to use STL.

However, this debate tends to become almost as silly as the hashtable
debate, and it will probably end the same way too. What I find sad is
that I have the strong suspicion that the opponents of hash tables and
null valued iterators are not open minded about it. They cling to STL
as if any change in it will mean public humiliation for the people
who designed and supported STL.

That attitude is not very productive.

Jan

In article <JAK.95Mar29223621@remington.cs.brown.edu> jak@cs.brown.edu writes:

> I just don't understand this mulish reluctance to define singular
> iterator values.

What reluctance?  Any iterator can potentially have a singular
value, in which case programs are not allowed to do anything with
the iterator but assign a new value to it.
--
    --Andrew Koenig
      ark@research.att.com

In article <3ldmk4$l6h@highway.LeidenUniv.nl> ruiter@ruls41.LeidenUniv.nl (Jan-Peter de Ruiter) writes:

> However, this debate tends to become almost as silly as the hashtable
> debate, and it will probably end the same way too. What I find sad is
> that I have the strong suspicion that the opponents of hash tables and
> null valued iterators are not open minded about it. They cling to STL
> as if any change in it will mean public humiliation for the people
> who designed and supported STL.

The only opposition to hashtables that was raised at the meeting
was one of schedule: the committee had agreed that it was time to
stop adding stuff and turn our attention to finishing the details
and shipping the thing.
--
    --Andrew Koenig
      ark@research.att.com

tIn article <D69BKA.MM9@research.att.com> ark@research.att.com (Andrew Koenig) writes:

> > I just don't understand this mulish reluctance to define singular
> > iterator values.
>
> What reluctance?  Any iterator can potentially have a singular
> value, in which case programs are not allowed to do anything with
> the iterator but assign a new value to it.

However, there is no way to test whether or not an iterator is
singular.

Now I realize that that is never going to change, nor do I suggest
that it should; distinguishing unfailingly between singular and
nonsingular values would be very expensive, and would require a great
deal of bookkeeping.  Some of us, though, suggest a more modest
change: every iterator type should have one singular value that can be
tested for.

I don't pretend that this is necessary; I do claim, however, that it
would be convenient.  There are a few cases where having such a
testable singular value would make certain tasks easier.  To my mind,
the benefit of having null iterators is small but nonzero, and the
cost is zero.

--

                               --matt

In article <3l6qhj$kr3@druid.borland.com>,
Pete Becker <pete@borland.com> wrote:
>
> One of the design goals of STL is to be able to guarantee worst
>case performance for algorithms. This is done by not supporting operations
>that are "too expensive" on a particular container. This results in a
>compile-time error when you try to use them, rather than a program that
>runs at a glacial pace. It is certainly possible to design a library with
>a different set of design goals. Personally, I prefer compile-time errors
>to documentation that says "don't do this". I don't see any benefit in
>being able to ask for a quicksort() on a linked list when there are much
>better ways of sorting a list.
> -- Pete


 Yes, but this is quite a nasty problem. Not all users
and all uses are interested in performance: correctness is
important.

 I have a case right now where I have linked list of
entities (windows, actually) and I NEED to access them by
position.

 It is a pain in the finger to have to write my
own error prone indexing operation when the STL list
could easily provide that operation -- and optimise it.

 You see, I don't give a pinch how long it takes to
search through a list of 10 windows. What I care about is
being able to write compact succinct code that uses
the same code that has been and is being used by
millions of other people.

 I personally believe referential transparency is
MORE imporatant than performance guarrantees. It is more
important to get it right first -- and THEN tune performance
where it matters by switching to a better suited container.

 STL actually interferes with that technique --
which virtual function polymorphism guarrantees.

 I'm NOT saying STL is wrong or bad -- it is
very important to have design goals AND STICK TO THEM.

 But it is my guess there needs to be a layer
on top of STL that provides referential transparency
_irrespective_ of performance. Because in 90% of code,
performance just doesn't matter. Correctness matters
in 100% of code :-)


--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <FENSTER.95Mar28174405@ground.cs.columbia.edu>, fenster@ground.cs.columbia.edu (Sam Fenster) says:
>
>>>>    fenster@ground.cs.columbia.edu (Sam Fenster) wrote:
>
>    #include "dictionaries.h"   // Defines namespace Dicts.
>    int main () {
>    // Dicts is a module that accesses global containers and files that we
>    // can't see.  It is *not* a single container.  Its data is *not* public.
>
>    Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
>    if (it1 == Dicts::Null) error();
>
>    Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
>    if (it2 == Dicts::Null) error();
>
>    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
>    return 0;}
>
>>>>> I don't know or care which dictionary the range came from.  I did not
>>>>> need to reference any pre-existing containers or iterators before or
>>>>> after calling look_up_word() or end_of_section()....To traverse the
>>>>> range, all I need are the iterators which are returned.  And to find out
>>>>> if the function calls succeeded, what do I compare them against, if not
>>>>> Null?  Why should Dicts (and every similar module) need to declare some
>>>>> special (global?) nonstandard object?
>
>> fenster@ground.cs.columbia.edu (Sam Fenster) says:
>>> The return here is not compared to an end iterator for some larger range.
>>> The programmer is not concerned with any larger range.  Once it1 and it2
>>> are returned, they establish the only range the caller knows about.  If
>>> this range cannot be established, they need to have some value which
>>> indicates this.  This situation is distinct from having them denote an
>>> empty range [it1==it2].
>
>pete@borland.com (Pete Becker) writes:
>>       Fundamentally this argument says: I want to design Dict in a way that
>> does not obey the STL design rules, so I need a Null iterator. There is
>> another possible solution: obey the STL design rules. I see nothing in the
>> statement of the problem that precludes treating the entire contents of Dict
>> as a range.
>
>I got the impression that one of the design rules of STL was that one could
>treat a pair of iterators in isolation as a container.  So if someone wants to
>pass me (or returns me) a possibly empty ordered collection of things, they
>can pass me two iterators and nothing else.
>

Yes.

>It's often good design to let an object have a checkable state which indicates
>that it's invalid.  Such a value is useful if an object must be instantiated
>before it's initialized to something useful.  Or if it is the result of some
>operation that may fail.  NULL is used for these purposes with pointers.
>

Well, maybe it is sometimes good. But having objects that can be invalid can lead to
serious problems. Just look at all the program bugs that programmers cause by dereferencing
null pointers.

The basic rule is that functions that you call must honor the contract that they claim
to support. If that contract says "I'll return two valid iterators that define a range" then
you don't need to check for validity. If that contract says "I'll return two iterators that
may or may not be valid, and if they are both valid they define a range" then you must check
both pointers for validity. The second version is inherently more complex. I just don't see
what the benefit is in doing it that way.

>If someone hands me two iterators, it is useful to be able to check them for
>validity.  Do you claim that in every such situation, there will be a third
>iterator somewhere in my environment, the end of some larger range?  Do you
>want me to redesign my problem so that such a third iterator exists (and is
>made public), *just so I have something to test my two iterators against*?
>That complicates my interface considerably!
>

No, don't add a third pointer. The two are sufficient. If they are equal then you have
no data to operate on.

>Modeling my collection of dictionaries as a single large container doesn't
>have any desirable semantics for me!  Why impose an ordering between the last
>word in one dictionary and the first word in some other dictionary?  The STL
>philosophy is not, "Make *everything* a container."  You just want me to do it
>so I'll have a phony "end iterator" to compare my real iterators against to
>test for validity.  (I would never even use my phony "start iterator.")
>
>The availability of Null would make the design much cleaner.
>

If you don't want to treat your collection of dictionaries as a single large container,
don't do it. STL was not designed to provide integration of loose collections of
containers. I suspect that adding Null wouldn't really get you there.

In article <FENSTER.95Mar27113503@ground.cs.columbia.edu>, fenster@ground.cs.columbia.edu (Sam Fenster) says:
>
>pstemari@erinet.com (Paul J. Ste. Marie) writes:
>>    fenster@ground.cs.columbia.edu (Sam Fenster) wrote:
>>>
>>>     Dicts::iterator it1 = Dicts::look_up_word ("the");
>>>     if (it1 == Dicts:Null) error();
>>
>>>     Dicts::iterator it2 = Dicts::end_of_section (it1);
>>>     if (it2 == Dicts:Null) error();
>>>
>>>     for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i);
>>>
>>> I don't know or care which dictionary the range came from.  I did not need
>>> to reference any pre-existing containers or iterators before or after
>>> calling look_up_word() or end_of_section().  The Dicts module does not need
>>> to make its container(s) public.  To traverse the range, all I need are the
>>> iterators which are returned.  And to find out if the function calls
>>> succeeded, what do I compare them against, if not Null?  Why should Dicts
>>> (and every similar module) need to declare some special (global?)
>>> nonstandard object?
>>
>> A simpler implementation would be:
>>
>>     Dicts::iterator it1 = Dicts::look_up_word ("the");
>>     if (it1 == Dicts::end()) error();
>>
>>     Dicts::iterator it2 = Dicts::end_of_section (it1);
>>     if (it2 == Dicts::end()) error();
>>
>>     for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i);
>
>Why would this be simpler?
>
>> and actually, you don't even need the error checks.
>
>Sure I do.  An error is not the same as an empty range.  Also, if I want to
>detect the absence of the word "the" before calling end_of_section(), how else
>can I do it?
>
>And what is Dicts::end()?  Dicts is not a single STL container.  It returns a
>range that lies in one container of many that are not publicly visible.  If
>end() is a single value designed to signify failure for *any* container,
>shouldn't we call it "Null"?  Don't you think programmers will want such a
>value available and standardized for any iterator they use?
>
>The return here is not compared to an end iterator for some larger range.  The
>programmer is not concerned with any larger range.  Once it1 and it2 are
>returned, they establish the only range the caller knows about.  If this range
>cannot be established, they need to have some value which indicates this.
>This situation is distinct from having them denote an empty range.

 Fundamentally this argument says: I want to design Dict in a way
that does not obey the STL design rules, so I need a Null iterator. There is
another possible solution: obey the STL design rules. I see nothing in the
statement of the problem that precludes treating the entire contents of Dict
as a range. It may take a bit more thought to come up with an iterator
mechanism that works sensibly in this context, but try it before rejecting it.
Adding a Null iterator looks to me like a shortcut to avoid careful design. The
cost that it imposes is that there would be two different ways to do the same
thing, with no clear guidance on when to use which one. I, for one, do not want
to sacrifice the clarity that the STL model gives just to support this shortcut.
 -- Pete

John Max Skaller (maxtal@Physics.usyd.edu.au) wrote:

:  Actually, STL doesn't require anything. The existing
: list class is not construed as a sequence (it only has
: bi-directional iterators) but there is nothing to stop
: you creating a list class that is random access.

True enough. That brings up a question I had for quite some time.
The deque<T> template DOES have random access, although traditionally
a deque could well be implemented as a linked list. In fact, Modena
recommends using a deque instead of a vector on the segmented PC
architecture to avoid the 64K limitation. Is there any deep reason why a
deque is required to have random access?

Cay

Sam> For instance, let's say a module has a set of private
Sam> dictionaries.  Functions in the module's interface may return a
Sam> pair of iterators specifying a subrange in one of those
Sam> dictionaries.  For instance,

If your iterators know how to go from one dictionary to another, I don't
see what the problem is -- you simply say

    Dicts::iterator it1 = Dicts::look_up_word ("the");
    if (it1 == Dicts::error()) error();

    Dicts::iterator it2 = Dicts::end_of_section (it1);
    if (it2 == Dicts::error()) error();
    for (Dicts::iterator i = it1; i != it2; ++i)

The iterator should always be dereferencable between it1 and it2.
You would define Dicts::error to return a singular value.

If your iterators don't know what the sequence of dictionaries is, who
decides where to go after the end of each dictionary?

                         Jak Kirman                        jak@cs.brown.edu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If someone is going to make a mistake costly to me, better for it to be
an understandably incompetent human like myself than a mysteriously
incompetent machine.
                            -- J Weizenbaum, Computer Power and Human Reason

>>>>> "Pete" == Pete Becker <pete@borland.com> writes:
In article <3l6qhj$kr3@druid.borland.com> pete@borland.com (Pete Becker) writes:

 Pete>  One of the design goals of STL is to be able to guarantee worst
 Pete> case performance for algorithms. This is done by not supporting operations
 Pete> that are "too expensive" on a particular container. This results in a
 Pete> compile-time error when you try to use them, rather than a program that
 Pete> runs at a glacial pace. It is certainly possible to design a library with
 Pete> a different set of design goals. Personally, I prefer compile-time errors
 Pete> to documentation that says "don't do this". I don't see any benefit in
 Pete> being able to ask for a quicksort() on a linked list when there are much
 Pete> better ways of sorting a list.

For example, to show people how slow it is.  Why would you think people
agree on what "too slow" is, anyway?  Asymptotic complexity is often not
a very useful criterion.

                         Jak Kirman                        jak@cs.brown.edu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Never doubt that a small group of thoughtful, committed individuals can
change the world.  Indeed, it is the only thing that ever has.
                                  -- Margaret Mead,  American Anthropologist

Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
> In article <1995Mar22.123032.21351@wisipc.weizmann.ac.il> oren@marganit.weizmann.ac.il (Ben-Kiki Oren) writes:
>    Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
>    > I fail to see any advantage to using iterators as a replacement for
>    > pointers. I've repeatedly asked proponents of the null-valued iterator
>    > to give a concrete example of where a null-valued iterator would allow
>    > code to be written more efficiently or more clearly if it were available.
>    > But thus far none has been forthcoming.

>    O.K., I'll bite. How about navigating multiple containers:

>       snip

> There are other more efficient alternatives than the ones that you consider
> and that don't require a null iterator. For example, the level indicator
> in your Symbol struct is not really necessary and could be eliminated, thereby
> saving n * no_of_symbols * no_bytes_per_symbol bytes of storage. Instead, you
> could use static variables to keep track of the current level during
> iteration through your symbol table as follows:

>    foobar.hh:
>     struct Symbol { ... };
>     typedef bool (*checkfp)(Symbol &);

>    bar.cc:
>     int level;
>            SIterator current;
>            SIterator last;
>       snip

> My code preserves your idiom, is as concise and clear as yours,
> requires less memory and is at least as fast. So I don't think
> you've proved your point yet.

However, your code relies on static variables to perform the iteration,
removing any possibility of nested searches, multi-threading, and so on. As
for the level indicator, it might be useful by itself (e.g., when testing for
shadowing).

I don't think that this line of reasoning (`show me a code which really really
needs null iterators') leads anywhere. Obviously it is possible to write any
program you like without them, and I agree that the overhead wouldn't be too
high (in my example, you could pack the iterator and a flag into a single type
which would be the return value of find, or you could add a bool& argument to
it). To reiterate:

>    It depends what your position is in vague issues such as 'orthogonality',
>    'simplicity', and 'completeness'. Saying 'STL iterators work like pointers in
>    SOME respects' requires you to start listing the differences, making the
>    definition longer and the understanding slower. No differences a mean more
>    powerful tool and a simpler definition.
>

> Again, this is an assertion to be proven.

i) The definition is simpler since you can say 'iterators behave just like
pointers', without having to list the exceptions. For example: I was
explaining STL to my brother, who is working on a huge C++ project. I hardly
finished saying that 'iterators behave a lot like pointers', and he already
asked: so you can dereference, increment/decrement, and test for NULL? So I
had to explain that no, NULL you couldn't test for, etc. My description of STL
would have been a lot simpler if I didn't need to go into this.

ii) As for power, obviously you can do "if(it = find())" if you have a null
iterator, and you can't otherwise. That is power. It may be irrelevant, or
rarely needed in practical programs, or bad style, but it does allow the
programmer one more option, to be used where appropriate. C++'s philosophy was
to avoid the 'one true way' syndrom and allow the programmer to choose the
right idiom for the task at hand. IMHO STL should follow this.

>    STL iterators have been presented as supporting pointer-based programming
>    idioms. This is one of the key choices in STL's design. Saying that iterators
>    are not meant as a replacement to pointers is obviously true, but irrelevant.
>    It does not change the fact that STL iterators support only part (a large
>    part, the important part, but still a part) of the pointers programming
>    idioms. Since there is no performance/complexity hit involved, why not support
>    ALL these idioms?

> STL iterators have never been presented (except by misinformed critics)
> as supporting ALL pointer-based programming idioms. This is manifestly
> not true now (for example, STL iterators do not support pointer-based
> idioms for navigating lists)

Obviously it isn't true now! The question is, why not?

> and  cannot be made true simply
> by fiddling with STL's basic constructs.

Now that also is an assertion that needs to be proven. I didn't see any reason
posted why supporting null iterators is not a simple fiddling with STL. It
would allow using the full set of pointer idioms (lists included, if someone
was perverse enough to write such code). So?

Note: I think this thread has lost its point once the hash table addition was
rejected. If there is no time for such a vital change, the minor wart of not
allowing null iterators is not something worth bothering with. Maybe in version
3.0 :-)

      Oren.

--
Life is tough and the hard ships are many.

>>>>> "Pete" == Pete Becker <pete@borland.com> writes:

 >> I don't see this extension being useful all that often, but I do see
 >> it being useful on occasion; the gap it fills is real.  Frankly, I
 >> don't see any reason not to define null iterators.

 Pete>  The reason not to define them is that nobody has shown a need
 Pete> for them, except in vague handwaving like "useful on occasion."
 Pete> Please give an example that requires null iterators and does not
 Pete> start out by misusing STL.

The example in the original posting.  The functions that return
iterators need to be able to indicate an error.  The past-the-end value
is obviously not appropriate here, since it would be used to indicate
an empty range.

It would be convenient in many cases to have a null value already
defined, rather than requiring every class to define its own singular
values.  On the other hand, it is easy enough to define a static
singular iterator of the input and output operators, so it does not seem
that pressing a problem.

                         Jak Kirman                        jak@cs.brown.edu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I know not with what weapons World War III will be fought, but
World War IV will be fought with sticks and stones.
                                                          -- Albert Einstein

In article <3l9976$qek@druid.borland.com> pete@borland.com (Pete Becker) writes:


   In article <FENSTER.95Mar27113503@ground.cs.columbia.edu>, fenster@ground.cs.columbia.edu (Sam Fenster) says:
   >
   >pstemari@erinet.com (Paul J. Ste. Marie) writes:
   >>    fenster@ground.cs.columbia.edu (Sam Fenster) wrote:
   >>>
   >>>     Dicts::iterator it1 = Dicts::look_up_word ("the");
   >>>     if (it1 == Dicts:Null) error();
   >>
   >>>     Dicts::iterator it2 = Dicts::end_of_section (it1);
   >>>     if (it2 == Dicts:Null) error();
   >>>
   >>>     for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i);
   >>>
   >>> I don't know or care which dictionary the range came from.  I did not need
   >>> to reference any pre-existing containers or iterators before or after
   >>> calling look_up_word() or end_of_section().  The Dicts module does not need
   >>> to make its container(s) public.  To traverse the range, all I need are the
   >>> iterators which are returned.  And to find out if the function calls
   >>> succeeded, what do I compare them against, if not Null?  Why should Dicts
   >>> (and every similar module) need to declare some special (global?)
   >>> nonstandard object?

[snip]


    Fundamentally this argument says: I want to design Dict in a way
   that does not obey the STL design rules, so I need a Null iterator. There is
   another possible solution: obey the STL design rules.

Yes. A compulsion to concoct examples that demonstrate a need for
an alien construct like null iterators can lead proponents to overlook
simpler solutions based on native STL idioms, e.g.,

Dicts::iterator start;
Dicts::iterator end;
Dicts::find_section_starting_at("the", start, end);
if (start != end)
  for (Dicts::iterator i=start; i!=end; ++i)  output (*i);


- Paul

- Paul

In article <JAK.95Mar28222502@remington.cs.brown.edu>, jak@cs.brown.edu (Jak Kirman) says:
>
>>>>>> "Pete" == Pete Becker <pete@borland.com> writes:
>In article <3l6qhj$kr3@druid.borland.com> pete@borland.com (Pete Becker) writes:
>
> Pete>  One of the design goals of STL is to be able to guarantee worst
> Pete> case performance for algorithms. This is done by not supporting operations
> Pete> that are "too expensive" on a particular container. This results in a
> Pete> compile-time error when you try to use them, rather than a program that
> Pete> runs at a glacial pace. It is certainly possible to design a library with
> Pete> a different set of design goals. Personally, I prefer compile-time errors
> Pete> to documentation that says "don't do this". I don't see any benefit in
> Pete> being able to ask for a quicksort() on a linked list when there are much
> Pete> better ways of sorting a list.
>
>For example, to show people how slow it is.  Why would you think people
>agree on what "too slow" is, anyway?  Asymptotic complexity is often not
>a very useful criterion.
>

 STL does not in any way prevent you from writing a linked list with
an iterator that supports an incredibly slow index operation. You can do that
if you want to, and you can use that iterator in calls to STL algorithms that
require an index operation. You can put such a class in a commercial product and
tell people that your product supports indexing into a linked list. You cannot,
however, claim that such an iterator conforms to the STL requirements, because it
violates the requirement that indexing occur in constant time.
 One of the major benefits of talking about complexity is that it is a
concept that is reasonably well understood and generally useful. It provides a sound
theoretical basis for choosing containers and algorithms, something which has been
sorely lacking in container libraries in the past.
 -- Pete

In article <JAK.95Mar28224204@remington.cs.brown.edu>, jak@cs.brown.edu (Jak Kirman) says:
>
>>>>>> "Pete" == Pete Becker <pete@borland.com> writes:
>
> >> I don't see this extension being useful all that often, but I do see
> >> it being useful on occasion; the gap it fills is real.  Frankly, I
> >> don't see any reason not to define null iterators.
>
> Pete>  The reason not to define them is that nobody has shown a need
> Pete> for them, except in vague handwaving like "useful on occasion."
> Pete> Please give an example that requires null iterators and does not
> Pete> start out by misusing STL.
>
>The example in the original posting.

I don't have the original posting. Was this the "example" that called
a function named find() with no parameters and with no context to
indicate what it was supposed to be doing? I couldn't tell from the code
what it was trying to illustrate, and the description of what it was
trying to do was so vague that it really didn't contribute anything to
the discussion. If someone has a better example, please post it.

>The functions that return
>iterators need to be able to indicate an error.  The past-the-end value
>is obviously not appropriate here, since it would be used to indicate
>an empty range.
>

Use exceptions to indicate errors. And, to anticipate one possible
response, if you don't want to use exceptions to handle "routine" errors,
please describe what these errors would be and why an STL algorithm
would run into such an error.

>It would be convenient in many cases to have a null value already
>defined, rather than requiring every class to define its own singular
>values.  On the other hand, it is easy enough to define a static
>singular iterator of the input and output operators, so it does not seem
>that pressing a problem.

Singular values cannot be used for anything. In particular, there is no
guarantee that you will get a meaningful result from comparing two
iterators if one of them is a singular value.
 -- Pete

In article <JAK.95Mar28224204@remington.cs.brown.edu> jak@cs.brown.edu (Jak Kirman) writes:

   >>>>> "Pete" == Pete Becker <pete@borland.com> writes:

    >> I don't see this extension being useful all that often, but I do see
    >> it being useful on occasion; the gap it fills is real.  Frankly, I
    >> don't see any reason not to define null iterators.

    Pete>  The reason not to define them is that nobody has shown a need
    Pete> for them, except in vague handwaving like "useful on occasion."
    Pete> Please give an example that requires null iterators and does not
    Pete> start out by misusing STL.

   The example in the original posting.  The functions that return
   iterators need to be able to indicate an error.  The past-the-end value
   is obviously not appropriate here, since it would be used to indicate
   an empty range.

Again, this begs the question as to just what other types of "errors"
(other than empty range) there could be in properly designed operations involving
STL iterators. Your assertion is neither "obvious" nor true in my view
and is just another instance of the sort of hand-waving that characterizes
the claims of null iterator supporters.

My challenge stands: either produce a (properly designed) STL code example that
requires a null iterator or shut up.

- Paul

In article <3l5pf3$co4@highway.LeidenUniv.nl>,
Jan-Peter de Ruiter <ruiter@ruls41.LeidenUniv.nl> wrote:
>Pete Becker (pete@borland.com) wrote:
>
>:  I hope I didn't say this. What I tried to say was that if you
>: accept the elided argument that there is no point in providing subsets of
>: pointer capabilities, then the entire iterator structure in STL collapses,
>: because it is based in large part on providing only the subset of pointer
>: capabilities that a particular container can provide efficiently. This
>: means, for example, that STL doesn't let you index into a linked list,
>: because it is too expensive. This is a very powerful notion, and
>: I would hate to lose it just because people want to think in terms of
>: pointers.
>
>This is something I don't really get - why would a library forbid
>expensive operations? Why is that 'powerful'? If the user is warned
>in the documentation that indexing a linked list is expensive, he
>can decide for himself whether is it _too_ expensive or not. No?
>
>I wouldn't really worry if STL could do stuff that is potentially
>inefficient.

 Actually, STL doesn't require anything. The existing
list class is not construed as a sequence (it only has
bi-directional iterators) but there is nothing to stop
you creating a list class that is random access.

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

pstemari@erinet.com (Paul J. Ste. Marie) writes:
>    fenster@ground.cs.columbia.edu (Sam Fenster) wrote:
>>
>>     Dicts::iterator it1 = Dicts::look_up_word ("the");
>>     if (it1 == Dicts:Null) error();
>
> If this isn't an implicit statement that incrementing the iterator
> enough gives you a null I don't know what is.

Not at all.  There's no "incrementing" going on here.  A Null return is used
to indicate any kind of inability to return an iterator pointing to the
requested item.  Perhaps no dictionaries were currently open.  Perhaps "the"
wasn't in any of the available dictionaries.

This behavior is similar to that of a pointer.  Incrementing it cannot give
you NULL.  But a function can return NULL.  Get it?

>>     Dicts::iterator it2 = Dicts::end_of_section (it1);
>>     if (it2 == Dicts:Null) error();
>>
>>     for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i);
>>
>> I don't know or care which dictionary the range came from.  I did not need
>> to reference any pre-existing containers or iterators before or after
>> calling look_up_word() or end_of_section().  The Dicts module does not need
>> to make its container(s) public.  To traverse the range, all I need are the
>> iterators which are returned.  And to find out if the function calls
>> succeeded, what do I compare them against, if not Null?  Why should Dicts
>> (and every similar module) need to declare some special (global?)
>> nonstandard object?
>
> A simpler implementation would be:
>
>     Dicts::iterator it1 = Dicts::look_up_word ("the");
>     if (it1 == Dicts::end()) error();
>
>     Dicts::iterator it2 = Dicts::end_of_section (it1);
>     if (it2 == Dicts::end()) error();
>
>     for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i);

Why would this be simpler?

> and actually, you don't even need the error checks.

Sure I do.  An error is not the same as an empty range.  Also, if I want to
detect the absence of the word "the" before calling end_of_section(), how else
can I do it?

And what is Dicts::end()?  Dicts is not a single STL container.  It returns a
range that lies in one container of many that are not publicly visible.  If
end() is a single value designed to signify failure for *any* container,
shouldn't we call it "Null"?  Don't you think programmers will want such a
value available and standardized for any iterator they use?

The return here is not compared to an end iterator for some larger range.  The
programmer is not concerned with any larger range.  Once it1 and it2 are
returned, they establish the only range the caller knows about.  If this range
cannot be established, they need to have some value which indicates this.
This situation is distinct from having them denote an empty range.

In article <KINNUCAN.95Mar22150251@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:
>   If iterators are to be a generalization of pointers, shouldn't one of
>   their most important behaviors (the ability to be null) also be part
>   of the generalization?
>
>No.

Would you care to explain this answer?  Why should a "generalization of
pointers" not include an important property of pointers?  For example,
no one considers that whole numbers are a generalization of integers;
in fact, it is the other way around.

>   In fact, it seems more correct, based on their semantics, to claim that
>   iterators are generalization of pointers within an array rather than
>   pointers in general.
>
>Exactly. That's why IMHO a null iterator is a meaningless, useless,
>and dangerously misleading construct.

Exactly.  That's why the phrase "generalization of pointers" is
misleading.  STL iterators are not a generalization of pointers at all,
only a generalization of a restricted subset of pointers.

Stating that null iterators are "meaningless, useless, and dangerously
lisleading" by appealing to a mislabelled model is futile without
explaining why that particular model was chosen in the first place.

Stephen Carlson
--
Stephen Carlson     :  Poetry speaks of aspirations,  : ICL, Inc.
scc@reston.icl.com  :  and songs chant the words.     : 11490 Commerce Park Dr.
(703) 648-3330      :                 Shujing 2:35    : Reston, VA  22091   USA

In article <3l5pf3$co4@highway.LeidenUniv.nl>, ruiter@ruls41.LeidenUniv.nl (Jan-Peter de Ruiter) writes:
|>
|> This is something I don't really get - why would a library forbid
|> expensive operations?

It doesn't *forbid* it, it simply doesn't require it.

There is nothing in STL that prevents you from providing a random-
access iterator for a linked list if you wish.  It is important that
it not *require* expensive operations, since if it did there would be
too much pressure to bypass the standard library for efficiency.

You need to remember, STL is mainly a *framework*, not a complete system.
Each client gets to extend it in any way appropriate for the job at hand.

--
swf@elsegundoca.attgis.com  sarima@netcom.com

The peace of God be with you.

>>>    fenster@ground.cs.columbia.edu (Sam Fenster) wrote:

    #include "dictionaries.h"   // Defines namespace Dicts.
    int main () {
    // Dicts is a module that accesses global containers and files that we
    // can't see.  It is *not* a single container.  Its data is *not* public.

    Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
    if (it1 == Dicts::Null) error();

    Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
    if (it2 == Dicts::Null) error();

    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
    return 0;}

>>>> I don't know or care which dictionary the range came from.  I did not
>>>> need to reference any pre-existing containers or iterators before or
>>>> after calling look_up_word() or end_of_section()....To traverse the
>>>> range, all I need are the iterators which are returned.  And to find out
>>>> if the function calls succeeded, what do I compare them against, if not
>>>> Null?  Why should Dicts (and every similar module) need to declare some
>>>> special (global?) nonstandard object?

> fenster@ground.cs.columbia.edu (Sam Fenster) says:
>> The return here is not compared to an end iterator for some larger range.
>> The programmer is not concerned with any larger range.  Once it1 and it2
>> are returned, they establish the only range the caller knows about.  If
>> this range cannot be established, they need to have some value which
>> indicates this.  This situation is distinct from having them denote an
>> empty range [it1==it2].

pete@borland.com (Pete Becker) writes:
>  Fundamentally this argument says: I want to design Dict in a way that
> does not obey the STL design rules, so I need a Null iterator. There is
> another possible solution: obey the STL design rules. I see nothing in the
> statement of the problem that precludes treating the entire contents of Dict
> as a range.

I got the impression that one of the design rules of STL was that one could
treat a pair of iterators in isolation as a container.  So if someone wants to
pass me (or returns me) a possibly empty ordered collection of things, they
can pass me two iterators and nothing else.

It's often good design to let an object have a checkable state which indicates
that it's invalid.  Such a value is useful if an object must be instantiated
before it's initialized to something useful.  Or if it is the result of some
operation that may fail.  NULL is used for these purposes with pointers.

If someone hands me two iterators, it is useful to be able to check them for
validity.  Do you claim that in every such situation, there will be a third
iterator somewhere in my environment, the end of some larger range?  Do you
want me to redesign my problem so that such a third iterator exists (and is
made public), *just so I have something to test my two iterators against*?
That complicates my interface considerably!

Modeling my collection of dictionaries as a single large container doesn't
have any desirable semantics for me!  Why impose an ordering between the last
word in one dictionary and the first word in some other dictionary?  The STL
philosophy is not, "Make *everything* a container."  You just want me to do it
so I'll have a phony "end iterator" to compare my real iterators against to
test for validity.  (I would never even use my phony "start iterator.")

The availability of Null would make the design much cleaner.

In other applications, it may be even more implausible to postulate some
"larger" ordered container in which the returned range "resides."  For
instance, the returned iterators may be the start and end of a container which
was created on the spot.  In general, the piece of code which is handed a pair
of iterators *shouldn't have to know.*

This is in contrast to code which is handed a single iterator.  A single
iterator is always drawn from a range.  Thus, it can be compared against the
end iterator of that range.

       Sam
P.S.  Pete, I value your contributions here highly.

In article <D5yB6n.HKx@nntpa.cb.att.com>, mpl@pegasus.bl-els.att.com (-Michael P. Lindner) says:
>
>OK, this is not _really_ a followup to Ben-Kiki Oren's post, but moreso
>a followup of both followups to that post, namely by pete@borland.com
>(Pete Becker) and kinnucan@hq.ileaf.com (Paul Kinnucan).
>
>
>>Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
>>> I fail to see any advantage to using iterators as a replacement for
>>> pointers. I've repeatedly asked proponents of the null-valued iterator
>>> to give a concrete example of where a null-valued iterator would allow
>>> code to be written more efficiently or more clearly if it were available.
>>> But thus far none has been forthcoming.
>
>The original post that started this thread had 2 such examples.  One was
>the case where a function was to return an iterator (or pair of
>iterators, more realistically) into a container that is not visible to
>the caller.  The second example was to be able to have a known value to
>initialize an iterator to, and test an iterator against, for the purpose
>of asserting whether the iterator had been modified.  In both these
>cases, a null valued iterator allows code to be written at least more
>clearly, if not more efficiently.

 I thought both of these examples had been thoroughly discredited.
As has been pointed out, both involve code fragments that cannot be
compiled as written. Once you add the information needed to make them
work, there is no benefit whatsoever from having null iterators. In the
first example, in order to do the search in the first place you must have
a range to search. If the search fails it returns an iterator that
compares equal to the upper limit of the range. There is no need for a
null iterator to do this.

>
>As for Mr. Becker's comment on the added overhead for supporting pointer
>semantics, of the two proposals for adding more pointer semantics to STL
>iterators, adding a null value and adding operator ->, neither adds any
>overhead to iterators (except additional lines of header file for the
>compiler to read :^).  Can you (Mr. Becker) please explain why you feel
>the overhead would be excessive?

 Huh? Please provide a sample implementation of an iterator that
can safely increment to null for, say, a vector of objects of type Foo.
Here's the STL style, which doesn't use a null iterator:

 #include <iostream.h>
 #include <iterator.h>
 #include <algo.h>

 template <class Foo, unsigned N> class Array
 {
 public:
  Array() : limit(0), Data(new Foo[N]) {}
  typedef Foo *iterator;
  iterator begin() { return Data; }
  iterator end() { return Data+limit; }
  void insert( const Foo& foo ) { Data[limit++] = foo; }
 private:
  Foo *Data;
  unsigned limit;
 };

 int main()
 {
 Array<int,10> array;
 for( int i = 0; i < 10; i++ )
  array.insert(i);
 copy( array.begin(), array.end(), ostream_iterator<int>(cout,"\n"));
 return 0;
 }

Now, please provide similar functionality with equivalent efficiency for
an iterator that turns into a null.


>
>Nobody has suggested that adding additional pointer semantics
>necessitates folding the different types of iterators into a single
>type.  Mr. Becker correctly states that removing the different types of
>iterators in favor of a single flavor would not be detrimental to
>STL,

 I hope I didn't say this. What I tried to say was that if you
accept the elided argument that there is no point in providing subsets of
pointer capabilities, then the entire iterator structure in STL collapses,
because it is based in large part on providing only the subset of pointer
capabilities that a particular container can provide efficiently. This
means, for example, that STL doesn't let you index into a linked list,
because it is too expensive. This is a very powerful notion, and
I would hate to lose it just because people want to think in terms of
pointers.
 -- Pete

In article <3kvrpd$h3d@druid.borland.com> pete@borland.com (Pete Becker) writes:

>  Huh? Please provide a sample implementation of an iterator that
> can safely increment to null for, say, a vector of objects of type Foo.

Nobody has ever suggested that as an extension to the STL.  The actual
suggestion that has been made is much more modest: for every type of
iterator, a null value must be defined.  It must be possible to assign
null to an iterator, it must be possible to compare an iterator for
equality to null, and it is guaranteed that no valid iterator will
ever compare equal to null.  I don't see that this would introduce any
overhead at all.

I don't see this extension being useful all that often, but I do see
it being useful on occasion; the gap it fills is real.  Frankly, I
don't see any reason not to define null iterators.

--

                               --matt

In article <MATT.95Mar25154326@physics7.berkeley.edu>,
   matt@physics7.berkeley.edu (Matt Austern) wrote:
:In article <3kvrpd$h3d@druid.borland.com> pete@borland.com (Pete
Becker) writes:
:
:>  Huh? Please provide a sample implementation of an iterator
:> that can safely increment to null for, say, a vector of objects
:> of type Foo.
:
:Nobody has ever suggested that as an extension to the STL.

Beg pardon?  Every example I've seen here of a null iterator has
requested that incrementing an iterator past the end of the
container would return null.  At least one headache with this is
that any algorithm that depended on this behavior would be unusable
with the simplest sort of container/iterator, a vanilla array and a
pointer into it.

The usual code snippet cited in the examples has been:

 while (++iterator)

which doesn't work with pointers, no way, no how.

:The actual suggestion that has been made is much more modest: for
:every type of iterator, a null value must be defined.  It must be
:possible to assign null to an iterator, it must be possible to
:compare an iterator for equality to null, and it is guaranteed
:that no valid iterator will ever compare equal to null.  I don't
:see that this would introduce any overhead at all.

It wouldn't--but the request was that all invalid iterators would
compare equal to null, not that no valid iterator would equal null.


 --Paul J. Ste. Marie, pstemari@well.sf.ca.us, pstemari@erinet.com

The Financial Crimes Enforcement Network claims that they capture every
public posting that has their name ("FinCEN") in it.  I wish them good hunting.

In article <D5yB6n.HKx@nntpa.cb.att.com> mpl@pegasus.bl-els.att.com (-Michael P. Lindner) writes:
   OK, this is not _really_ a followup to Ben-Kiki Oren's post, but moreso
   a followup of both followups to that post, namely by pete@borland.com
   (Pete Becker) and kinnucan@hq.ileaf.com (Paul Kinnucan).


   >Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
   >> I fail to see any advantage to using iterators as a replacement for
   >> pointers. I've repeatedly asked proponents of the null-valued iterator
   >> to give a concrete example of where a null-valued iterator would allow
   >> code to be written more efficiently or more clearly if it were available.
   >> But thus far none has been forthcoming.

   The original post that started this thread had 2 such examples.  One was
   the case where a function was to return an iterator (or pair of
   iterators, more realistically) into a container that is not visible to
   the caller.  The second example was to be able to have a known value to
   initialize an iterator to, and test an iterator against, for the purpose
   of asserting whether the iterator had been modified.  In both these
   cases, a null valued iterator allows code to be written at least more
   clearly, if not more efficiently.

No, you can code both cases just as efficiently and clearly without
null iterators as I thought I demonstrated in my response to the
original post.

If you think I'm wrong, demonstrate it with actual code. Give me a
piece of code using a null iterator and challenge me to write code
that is as clear and efficient without a null iterator. I'm sure I
can because whereever you would use a null iterator I'll simply use
a null pointer. (That's because I don't demand that iterators
replace pointers; I feel perfectly free to use them in
combination in my code, using pointers where
appropriate, e.g., passing addresses of objects, and iterators
where appropriate, e.g., navigating containers.)

- Paul

In article <1995Mar22.123032.21351@wisipc.weizmann.ac.il> oren@marganit.weizmann.ac.il (Ben-Kiki Oren) writes:


   Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
   > I fail to see any advantage to using iterators as a replacement for
   > pointers. I've repeatedly asked proponents of the null-valued iterator
   > to give a concrete example of where a null-valued iterator would allow
   > code to be written more efficiently or more clearly if it were available.
   > But thus far none has been forthcoming.

   O.K., I'll bite. How about navigating multiple containers:

   foobar.hh:

    struct Symbol { int level; ... };

    typedef bool (*checkfp)(Symbol &);

   bar.cc:

    SContainer tables[...];

    SIterator find_next(SContainer &, checkfp, SIterator);

    SIterator find_first(SContainer &, checkfp);

    SIterator find_next_interesting(SIterator prev_symbol, checkfp check) {
     int level = prev_symbol ? *prev_symbol.level : 0;
     do {
      prev_symbol = prev_symbol
       ? find_next(tables[level], check, prev_symbol)
       : find_first(tables[level], check);
     } while(!prev_symbol && ++level < levels);
     return(prev_symbol);
    }

   foo.cc:

    bool is_integer(Symbol &symbol) { ... }

    void foo() {
     SIterator symbol; // Null by default?
     while(symbol = find_next_interesting(symbol, is_integer))
      do_something_with(next_item);


    }

   Foo is searching for integer identifiers across all levels of a symbol table.
   It does not care which level the found symbols belong to. Returning a pointer,
   which is the natural thing to do, would make life harder when looking for the
   next symbol.

   Keeping a container just for integers is not practical; is_integer is just one
   of possibly many checks which may be applied. Putting all symbols in one
   container is also impractical since it makes discarding all symbols of the
   deepest level expensive.

   Returning a flag + iterator is possible of course, and is what would be done
   in a case like this using the current STL definition. I admit the difference
   isn't great. But it exists, nevertheless.



There are other more efficient alternatives than the ones that you consider
and that don't require a null iterator. For example, the level indicator
in your Symbol struct is not really necessary and could be eliminated, thereby
saving n * no_of_symbols * no_bytes_per_symbol bytes of storage. Instead, you
could use static variables to keep track of the current level during
iteration through your symbol table as follows:


   foobar.hh:

    struct Symbol { ... };

    typedef bool (*checkfp)(Symbol &);

   bar.cc:

    int level;
           SIterator current;
           SIterator last;

    SContainer tables[levels];

    Symbol* find_next(checkfp check) {
             while (current != last) {
               Symbol& s = *current++;
               if (*check(s))
                  return &s;
             }
             return 0;
           }

    Symbol* find_first(checkfp check) {
             for (level = 0; level<levels;++level) {
                 current = tables[level].first();
                 last = tables[level].last();
                 Symbol* first = find_next(check);
                 if (s)
                   return first;
              }
              return 0;
            }

    Symbol* find_next_interesting(Symbol* curr, checkfp check) {
               if (!curr)
                  return find_first(check);
        while (!next && ++level < levels)
                  next = find_next(check);
               return next;
    }

   foo.cc:

    bool is_integer(Symbol &symbol) { ... }

    void foo() {
     Symbol* symbol = 0;
     while(symbol = find_next_interesting(symbol, is_integer))
      do_something_with(symbol);

    }

My code preserves your idiom, is as concise and clear as yours,
requires less memory and is at least as fast. So I don't think
you've proved your point yet.

   > So far all I've seen are theoretical arguments based on specious reasoning
   > e.g.,  "iterators are like pointers in SOME respects, therefore
   > they would be more useful if they were like pointers in ALL respects."
   > The fact is, the conclusion does not follow from the premise.

   It depends what your position is in vague issues such as 'orthogonality',
   'simplicity', and 'completeness'. Saying 'STL iterators work like pointers in
   SOME respects' requires you to start listing the differences, making the
   definition longer and the understanding slower. No differences a mean more
   powerful tool and a simpler definition.
   ^^^^^^^^

Again, this is an assertion to be proven.

   In a way, this is similar to the 'fat interface' notion. You provide a
   complete set of operations on a class, even if currently you don't see where
   each specific one will be needed.


   > My own view is that STL iterators have not been proposed as a replacement
   > for pointers and therefore there is no need for them to duplicate all
   > the properties of pointers. Iterators are intended to be used to
   > navigate containers; there is no need for a null iterator to do this.
   > If your purpose is to pass an address of an object, use a pointer.

   STL iterators have been presented as supporting pointer-based programming
   idioms. This is one of the key choices in STL's design. Saying that iterators
   are not meant as a replacement to pointers is obviously true, but irrelevant.
   It does not change the fact that STL iterators support only part (a large
   part, the important part, but still a part) of the pointers programming
   idioms. Since there is no performance/complexity hit involved, why not support
   ALL these idioms?

STL iterators have never been presented (except by misinformed critics)
as supporting ALL pointer-based programming idioms. This is manifestly
not true now (for example, STL iterators do not support pointer-based
idioms for navigating lists) and  cannot be made true simply
by fiddling with STL's basic constructs. The point of STL is to make all containers
appear to algorithms like C arrays. By doing this, STL makes it possible to substitute
a pointer whereever an iterator appears in an STL-based algorithm. The
opposite is not true, however. You cannot now and never will be able to
substitute an STL iterator wherever a pointer appears in a program. Iterators and
pointers are not interchangeable constructs in general and attempts to make
them so are misguided.

- Paul

In article <D5t7sB.DHM@reston.icl.com>,
Stephen Carlson <scc@reston.icl.com> wrote:
>
>If iterators are to be a generalization of pointers, shouldn't one of
>their most important behaviors (the ability to be null) also be part
>of the generalization?
>
 STL is a library of generic algorithms which operate
cooperatively via iterators on a variety of containers.

 STL duplicates a subset of pointer semantics so that the
algorithms can be written so they operator on raw pointers.

 Since none of the algorithms include a zero comparison
with an iterator, there is no purpose in STL requiring null
iterators.

>In fact, it seems more correct, based on their semantics, to claim that
>iterators are generalization of pointers within an array rather than
>pointers in general.

 Yes. An array is a container, and pointers into
the array are iterators. STL iterators are not object handles
but container element locators.

 The fact that C only provides plain pointers is a simplification
that leads to LOTS of run-time errors that could be avoided
by a stronger type system.


--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

OK, this is not _really_ a followup to Ben-Kiki Oren's post, but moreso
a followup of both followups to that post, namely by pete@borland.com
(Pete Becker) and kinnucan@hq.ileaf.com (Paul Kinnucan).


>Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
>> I fail to see any advantage to using iterators as a replacement for
>> pointers. I've repeatedly asked proponents of the null-valued iterator
>> to give a concrete example of where a null-valued iterator would allow
>> code to be written more efficiently or more clearly if it were available.
>> But thus far none has been forthcoming.

The original post that started this thread had 2 such examples.  One was
the case where a function was to return an iterator (or pair of
iterators, more realistically) into a container that is not visible to
the caller.  The second example was to be able to have a known value to
initialize an iterator to, and test an iterator against, for the purpose
of asserting whether the iterator had been modified.  In both these
cases, a null valued iterator allows code to be written at least more
clearly, if not more efficiently.

As for Mr. Becker's comment on the added overhead for supporting pointer
semantics, of the two proposals for adding more pointer semantics to STL
iterators, adding a null value and adding operator ->, neither adds any
overhead to iterators (except additional lines of header file for the
compiler to read :^).  Can you (Mr. Becker) please explain why you feel
the overhead would be excessive?

Nobody has suggested that adding additional pointer semantics
necessitates folding the different types of iterators into a single
type.  Mr. Becker correctly states that removing the different types of
iterators in favor of a single flavor would not be detrimental to
STL, but I fail to follow fis claim that adding null values will cause
this to happen.

--
Mike Lindner
mikel@attmail.com
mpl@cmprime.attpls.com
mpl@pegasus.att.com

>>> Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
>>>> I've repeatedly asked proponents of the null-valued iterator to give a
>>>> concrete example of where a null-valued iterator would allow code to be
>>>> written more efficiently or more clearly if it were available.

> mpl@pegasus.bl-els.att.com (-Michael P. Lindner) says:
>> The original post that started this thread had 2 such examples.  One was
>> the case where a function was to return an iterator (or pair of iterators,
>> more realistically) into a container that is not visible to the caller.

For instance, let's say a module has a set of private dictionaries.  Functions
in the module's interface may return a pair of iterators specifying a subrange
in one of those dictionaries.  For instance,

    Dicts::iterator it1 = Dicts::look_up_word ("the");   // "The" starts with T
    if (it1 == Dicts:Null) error();

    Dicts::iterator it2 = Dicts::end_of_section (it1);   // End of the T's
    if (it2 == Dicts:Null) error();

    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i);   // "The"..."Tzar"

I don't know or care which dictionary the range came from.  I did not need to
reference any pre-existing containers or iterators before or after calling
look_up_word() or end_of_section().  The Dicts module does not need to make
its container(s) public.  To traverse the range, all I need are the iterators
which are returned.  And to find out if the function calls succeeded, what do
I compare them against, if not Null?  Why should Dicts (and every similar
module) need to declare some special (global?) nonstandard object?

>>> Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
>>>> If you think I'm wrong...Give me a piece of code using a null iterator
>>>> and challenge me to write code that is as clear and efficient without a
>>>> null iterator. I'm sure I can because whereever you would use a null
>>>> iterator I'll simply use a null pointer.

You say, make it1 and it2 be pointers to iterators?  (Just so they can have a
Null value?)  Are the iterators they point to globals?  Then my code's not
reentrant.  Are they allocated?  Then I have to free them.  Pointers to local
variables?  That requires redundantly passing addresses in.  All three times:
Yuk.

>> The second example was to be able to have a known value to initialize an
>> iterator to, and test an iterator against, for the purpose of asserting
>> whether the iterator had been modified.  In both these cases, a null valued
>> iterator allows code to be written at least more clearly, if not more
>> efficiently.

Another good example where there is currently no clean solution.

pete@borland.com (Pete Becker) writes:
>  In the first example, in order to do the search in the first place you
> must have a range to search. If the search fails it returns an iterator that
> compares equal to the upper limit of the range. There is no need for a null
> iterator to do this.

The first example didn't involve providing "a range to search."  Even if the
function in question is doing a search, the bounds of that search are not
necessarily known to the caller.

>  Huh? Please provide a sample implementation of an iterator that
> can safely increment to null for, say, a vector of objects of type Foo.

pstemari@erinet.com (Paul J. Ste. Marie) writes:
: Nobody has ever suggested that as an extension to the STL.

> Beg pardon?  Every example I've seen here of a null iterator has requested
> that incrementing an iterator past the end of the container would return
> null.

False.  Certain people who weren't thinking clearly said that.  The rest of us
agree with mpl@pegasus.bl-els.att.com (-Michael P. Lindner).

In article <FENSTER.95Mar26180106@ground.cs.columbia.edu>,
   fenster@ground.cs.columbia.edu (Sam Fenster) wrote:
:
:For instance, let's say a module has a set of private
:dictionaries.  Functions in the module's interface may return a
:pair of iterators specifying a subrange in one of those
:dictionaries.  For instance,
:
:    Dicts::iterator it1 = Dicts::look_up_word ("the");
:    if (it1 == Dicts:Null) error();

If this isn't an implicit statement that incrementing the iterator
enough gives you a null I don't know what is.

:
:    Dicts::iterator it2 = Dicts::end_of_section (it1);
:    if (it2 == Dicts:Null) error();
:
:    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i);
:
:I don't know or care which dictionary the range came from.  I did
:not need to reference any pre-existing containers or iterators
:before or after calling look_up_word() or end_of_section().  The
:Dicts module does not need to make its container(s) public.  To
:traverse the range, all I need are the iterators which are
:returned.  And to find out if the function calls succeeded, what
:do I compare them against, if not Null?  Why should Dicts (and
:every similar module) need to declare some special (global?)
:nonstandard object?

A simpler implementation would be:

    Dicts::iterator it1 = Dicts::look_up_word ("the");
    if (it1 == Dicts::end()) error();

    Dicts::iterator it2 = Dicts::end_of_section (it1);
    if (it2 == Dicts::end()) error();

    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i);

and actually, you don't even need the error checks.


 --Paul J. Ste. Marie, pstemari@well.sf.ca.us, pstemari@erinet.com

The Financial Crimes Enforcement Network claims that they capture every
public posting that has their name ("FinCEN") in it.  I wish them good hunting.

>>> mpl@pegasus.bl-els.att.com (-Michael P. Lindner) says:
>>>> The original post that started this thread had 2 such examples.  One was
>>>> the case where a function was to return an iterator (or pair of
>>>> iterators, more realistically) into a container that is not visible to
>>>> the caller.

>> pete@borland.com (Pete Becker) writes:
>>>       In the first example, in order to do the search in the first place
>>> you must have a range to search. If the search fails it returns an
>>> iterator that compares equal to the upper limit of the range. There is no
>>> need for a null iterator to do this.

> fenster@ground.cs.columbia.edu (Sam Fenster) says:
>> The first example didn't involve providing "a range to search."

pete@borland.com (Pete Becker) writes:
>  Of course it did. It may not have had an explicit range, but that most
> likely means that the search was intended to cover the entire container.

No.  (1) The function might return a range from any of *several* containers.
The caller should not need to see those containers and test against each of
their end iterators.

Or (2) the module called may have some private state which restricts the
search to a subrange of a container.  The caller shouldn't have to know what
that subrange's end iterator is.

(3) Even if there is a single range in which the returned iterators are a
subrange, if it's not an "explicit range" because it's part of private state,
you can't test against its end iterator.

In general, STL gives us the capability of using a pair of iterators in
isolation, as if they *are* a container.  So if we are handed a pair of
iterators, they should be able to assume a value signaling invalidity without
reference to some *other* pair of iterators.

You compare an iterator against an end iterator when it's traversing/searching
a pre-existing range.  Pre-existing ranges *have* end iterators.  When someone
hands you an iterator that helps *define* a range, it's serving a different
purpose.  There may be no other range in sight to compare it to.  Then you
need Null as a standard way to signal invalidity.

>> Even if the function in question is doing a search, the bounds of that
>> search are not necessarily known to the caller.

>  They most certainly are. They are either a range specified by the
> caller or they are the entire container. How can you possibly write a
> function that searches for something without describing where to search?

Encapsulation.  Data hiding.  Modularity.  Object-oriented programming.

"Where to search" may be specified by the caller in a way that has nothing to
do with the implementation detail of what container(s) are used inside the
called module or object.  And yet, as part of its interface, the module may
return a range to the user.  As below:

    #include "dictionaries.h"
    int main () {using namespace Dicts;
    // Dicts is a module that accesses global containers and files that we
    // can't see.  It is *not* a single container.  Its data is *not* public.

    Dicts::iterator it1 = Dicts::look_up_word ("the"); // "The" starts with T
    if (it1 == Dicts:Null) error();

    Dicts::iterator it2 = Dicts::end_of_section (it1); // End of the T's
    if (it2 == Dicts:Null) error();

    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i); // "The"..."Tzar"
    return 0;}

Pete Becker (pete@borland.com) wrote:

:  I hope I didn't say this. What I tried to say was that if you
: accept the elided argument that there is no point in providing subsets of
: pointer capabilities, then the entire iterator structure in STL collapses,
: because it is based in large part on providing only the subset of pointer
: capabilities that a particular container can provide efficiently. This
: means, for example, that STL doesn't let you index into a linked list,
: because it is too expensive. This is a very powerful notion, and
: I would hate to lose it just because people want to think in terms of
: pointers.

This is something I don't really get - why would a library forbid
expensive operations? Why is that 'powerful'? If the user is warned
in the documentation that indexing a linked list is expensive, he
can decide for himself whether is it _too_ expensive or not. No?

I wouldn't really worry if STL could do stuff that is potentially
inefficient.

JP

In article <FENSTER.95Mar26180106@ground.cs.columbia.edu>, fenster@ground.cs.columbia.edu (Sam Fenster) writes:
|> I don't know or care which dictionary the range came from.  I did not need to
|> reference any pre-existing containers or iterators before or after calling
|> look_up_word() or end_of_section().  The Dicts module does not need to make
|> its container(s) public.  To traverse the range, all I need are the iterators
|> which are returned.  And to find out if the function calls succeeded, what do
|> I compare them against, if not Null?  Why should Dicts (and every similar
|> module) need to declare some special (global?) nonstandard object?

No.  The ranges are inclusive below, exclusive above.  A function that
fails simply returns a range with the lower bound equal to the upper bound,
which indicates an empty range. [I think the previous posters misunderstood
the example, and faield to realize you were suggesting functions that *return*
a range].

This is a generalization of the usual "pointer one past the end" idiom used
in C programming.

--
swf@elsegundoca.attgis.com  sarima@netcom.com

The peace of God be with you.

In article <MATT.95Mar25154326@physics7.berkeley.edu>, matt@physics7.berkeley.edu (Matt Austern) says:
>
>In article <3kvrpd$h3d@druid.borland.com> pete@borland.com (Pete Becker) writes:
>
>>       Huh? Please provide a sample implementation of an iterator that
>> can safely increment to null for, say, a vector of objects of type Foo.
>
>Nobody has ever suggested that as an extension to the STL.  The actual
>suggestion that has been made is much more modest: for every type of
>iterator, a null value must be defined.  It must be possible to assign
>null to an iterator, it must be possible to compare an iterator for
>equality to null, and it is guaranteed that no valid iterator will
>ever compare equal to null.  I don't see that this would introduce any
>overhead at all.

OK.

>
>I don't see this extension being useful all that often, but I do see
>it being useful on occasion; the gap it fills is real.  Frankly, I
>don't see any reason not to define null iterators.

 The reason not to define them is that nobody has shown a need for
them, except in vague handwaving like "useful on occasion." Please give
an example that requires null iterators and does not start out by misusing
STL.
 -- Pete

Pete Becker (pete@borland.com) wrote:
> In article <1995Mar22.123032.21351@wisipc.weizmann.ac.il>, oren@marganit.weizmann.ac.il (Ben-Kiki Oren) says:
> >   snip
> >It does not change the fact that STL iterators support only part (a large
> >part, the important part, but still a part) of the pointers programming
> >idioms. Since there is no performance/complexity hit involved, why not support
> >ALL these idioms?
>
> There is a significant performance/complexity hit involved in making
> iterators support all of the usages that pointers support.

I don't see how supporting a null iterator causes a significant performance or
complexity hit. The only potentially effected operators are == and !=. I don't
see why they should be severely effected: to implement null iterators, you need
some 'special value' which no iterator can normally get. If your iterator
contains pointers, use NULL. If it contains indices, use -1, etc. Then, the
code for == and != does not need to be changed at all, since NULL will be
unequal to any valid pointer, -1 will be unequal to any valid index, etc.

As for the ++ and -- operators, then (i) since it is illegal to apply them to
null iterators, so they don't need to check for this case, and (ii) they can
never produce a null iterator as their result. In short, their implementation
is blissfully ignorant of the existence of null iterators, and therefore is not
effected in any way. Likewise for the '*' operator.

> Further, if
> you accept this argument, you must also accept that the five types of
> iterators in STL should be dropped in favor of a single type of iterator
> that does everything. This would be a major loss in clarity and
> expressibility.

I'd accept that as soon as the three types of pointers in C++ will be dropped
in favor of a single type which does everything :-)

      Oren.

--
Life is tough and the hard ships are many.

In article <FENSTER.95Mar26180106@ground.cs.columbia.edu>, fenster@ground.cs.columbia.edu (Sam Fenster) says:
>
>>>> Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
>>>>> I've repeatedly asked proponents of the null-valued iterator to give a
>>>>> concrete example of where a null-valued iterator would allow code to be
>>>>> written more efficiently or more clearly if it were available.
>
>> mpl@pegasus.bl-els.att.com (-Michael P. Lindner) says:
>>> The original post that started this thread had 2 such examples.  One was
>>> the case where a function was to return an iterator (or pair of iterators,
>>> more realistically) into a container that is not visible to the caller.
>
>For instance, let's say a module has a set of private dictionaries.  Functions
>in the module's interface may return a pair of iterators specifying a subrange
>in one of those dictionaries.  For instance,
>
>    Dicts::iterator it1 = Dicts::look_up_word ("the");   // "The" starts with T
>    if (it1 == Dicts:Null) error();

 if( it1 == Dicts::end() ) error();

>
>    Dicts::iterator it2 = Dicts::end_of_section (it1);   // End of the T's
>    if (it2 == Dicts:Null) error();

 if( it2 == Dicts::end() ) error();

>
>    for (Dicts::iterator i=it1; i!=it2; ++i)  output (*i);   // "The"..."Tzar"
>

 for( Dicts::iterator i = it1; i != it2; ++i )
  output( *i );

 What is the benefit from having a thing named Null? In particular, since STL
iterators expect to use a range, code that talks to them should be written in terms
of a range. Why, then, is it beneficial to provide another way of doing the same thing?
 -- Pete

In article <FENSTER.95Mar26180106@ground.cs.columbia.edu>, fenster@ground.cs.columbia.edu (Sam Fenster) says:
>
>pete@borland.com (Pete Becker) writes:
>>       In the first example, in order to do the search in the first place you
>> must have a range to search. If the search fails it returns an iterator that
>> compares equal to the upper limit of the range. There is no need for a null
>> iterator to do this.
>
>The first example didn't involve providing "a range to search."

 Of course it did. It may not have had an explicit range, but that most
likely means that the search was intended to cover the entire container.

>Even if the
>function in question is doing a search, the bounds of that search are not
>necessarily known to the caller.
>

 They most certainly are. They are either a range specified by the caller
or they are the entire container. How can you possibly write a function that
searches for something without describing where to search?
 -- Pete

In article <3l5pf3$co4@highway.LeidenUniv.nl>, ruiter@ruls41.LeidenUniv.nl (Jan-Peter de Ruiter) says:
>
>Pete Becker (pete@borland.com) wrote:
>
>:       I hope I didn't say this. What I tried to say was that if you
>: accept the elided argument that there is no point in providing subsets of
>: pointer capabilities, then the entire iterator structure in STL collapses,
>: because it is based in large part on providing only the subset of pointer
>: capabilities that a particular container can provide efficiently. This
>: means, for example, that STL doesn't let you index into a linked list,
>: because it is too expensive. This is a very powerful notion, and
>: I would hate to lose it just because people want to think in terms of
>: pointers.
>
>This is something I don't really get - why would a library forbid
>expensive operations? Why is that 'powerful'? If the user is warned
>in the documentation that indexing a linked list is expensive, he
>can decide for himself whether is it _too_ expensive or not. No?
>

 One of the design goals of STL is to be able to guarantee worst
case performance for algorithms. This is done by not supporting operations
that are "too expensive" on a particular container. This results in a
compile-time error when you try to use them, rather than a program that
runs at a glacial pace. It is certainly possible to design a library with
a different set of design goals. Personally, I prefer compile-time errors
to documentation that says "don't do this". I don't see any benefit in
being able to ask for a quicksort() on a linked list when there are much
better ways of sorting a list.
 -- Pete

In article <KINNUCAN.95Mar17093019@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:
>In article <1995Mar14.235439.14903@wisipc.weizmann.ac.il> oren@hadar.weizmann.ac.il (Ben-Kiki Oren) writes:
>>It allows STL to support the full range of programming idioms which are now
>>being used for pointer-based programming, whereas the current situation allows
>>just a subset. I'd say thats a big `something'.

>So far all I've seen are theoretical arguments based on specious reasoning
>e.g.,  "iterators are like pointers in SOME respects, therefore
>they would be more useful if they were like pointers in ALL respects."
>The fact is, the conclusion does not follow from the premise.

I think that the conclusion that iterators should behave like pointers
in many respects is a reasonable one from the following statements in
the documentation, Alexander Stepanov & Meng Lee, The Standard Template
Library, section 5, p. 6 (February 7, 1995):

 "Iterators are a generalization of pointers that allow a
    programmer to work with different data structures (containers)
    in a uniform manner."

and

    "Since iterators are a generalization of pointers, their semantics
    is a generalization of the semantics of pointers in C++."

>My own view is that STL iterators have not been proposed as a replacement
>for pointers and therefore there is no need for them to duplicate all
>the properties of pointers. Iterators are intended to be used to
>navigate containers; there is no need for a null iterator to do this.
>If your purpose is to pass an address of an object, use a pointer.

If iterators are to be a generalization of pointers, shouldn't one of
their most important behaviors (the ability to be null) also be part
of the generalization?

In fact, it seems more correct, based on their semantics, to claim that
iterators are generalization of pointers within an array rather than
pointers in general.

Stephen Carlson
--
Stephen Carlson     :  Poetry speaks of aspirations,  : ICL, Inc.
scc@reston.icl.com  :  and songs chant the words.     : 11490 Commerce Park Dr.
(703) 648-3330      :                 Shujing 2:35    : Reston, VA  22091   USA

Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
> I fail to see any advantage to using iterators as a replacement for
> pointers. I've repeatedly asked proponents of the null-valued iterator
> to give a concrete example of where a null-valued iterator would allow
> code to be written more efficiently or more clearly if it were available.
> But thus far none has been forthcoming.

O.K., I'll bite. How about navigating multiple containers:

foobar.hh:

 struct Symbol { int level; ... };

 typedef bool (*checkfp)(Symbol &);

bar.cc:

 SContainer tables   ...   ;

 SIterator find_next(SContainer &, checkfp, SIterator);
 SIterator find_first(SContainer &, checkfp);

 SIterator find_next_interesting(SIterator prev_symbol, checkfp check) {
  int level = prev_symbol ? *prev_symbol.level : 0;
  do {
   prev_symbol = prev_symbol
    ? find_next(tables   level   , check, prev_symbol)
    : find_first(tables   level   , check);
  } while(!prev_symbol && ++level < levels);
  return(prev_symbol);
 }

foo.cc:

 bool is_integer(Symbol &symbol) { ... }

 void foo() {
  SIterator symbol; // Null by default?
  while(symbol = find_next_interesting(symbol, is_integer))
   do_something_with(next_item);
 }

Foo is searching for integer identifiers across all levels of a symbol table.
It does not care which level the found symbols belong to. Returning a pointer,
which is the natural thing to do, would make life harder when looking for the
next symbol.

Keeping a container just for integers is not practical; is_integer is just one
of possibly many checks which may be applied. Putting all symbols in one
container is also impractical since it makes discarding all symbols of the
deepest level expensive.

Returning a flag + iterator is possible of course, and is what would be done
in a case like this using the current STL definition. I admit the difference
isn't great. But it exists, nevertheless.

> So far all I've seen are theoretical arguments based on specious reasoning
> e.g.,  "iterators are like pointers in SOME respects, therefore
> they would be more useful if they were like pointers in ALL respects."
> The fact is, the conclusion does not follow from the premise.

It depends what your position is in vague issues such as 'orthogonality',
'simplicity', and 'completeness'. Saying 'STL iterators work like pointers in
SOME respects' requires you to start listing the differences, making the
definition longer and the understanding slower. No differences a mean more
powerful tool and a simpler definition.

In a way, this is similar to the 'fat interface' notion. You provide a
complete set of operations on a class, even if currently you don't see where
each specific one will be needed.

> My own view is that STL iterators have not been proposed as a replacement
> for pointers and therefore there is no need for them to duplicate all
> the properties of pointers. Iterators are intended to be used to
> navigate containers; there is no need for a null iterator to do this.
> If your purpose is to pass an address of an object, use a pointer.

STL iterators have been presented as supporting pointer-based programming
idioms. This is one of the key choices in STL's design. Saying that iterators
are not meant as a replacement to pointers is obviously true, but irrelevant.
It does not change the fact that STL iterators support only part (a large
part, the important part, but still a part) of the pointers programming
idioms. Since there is no performance/complexity hit involved, why not support
ALL these idioms?

      Oren.

P.S. All that said, I doubt that it makes sense to modify STL at this stage to
support the null iterator idiom. It's just a minor annoyance, after all. Hash
tables, now... Adding them was more important by orders of magnitude.

---

In article <1995Mar22.123032.21351@wisipc.weizmann.ac.il>, oren@marganit.weizmann.ac.il (Ben-Kiki Oren) says:
>
>STL iterators have been presented as supporting pointer-based programming
>idioms. This is one of the key choices in STL's design. Saying that iterators
>are not meant as a replacement to pointers is obviously true, but irrelevant.

 It is not irrelevant when someone says "iterators are supposed
to be just like pointers, so they should do XXXX."


>It does not change the fact that STL iterators support only part (a large
>part, the important part, but still a part) of the pointers programming
>idioms. Since there is no performance/complexity hit involved, why not support
>ALL these idioms?
>

There is a significant performance/complexity hit involved in making
iterators support all of the usages that pointers support. Further, if
you accept this argument, you must also accept that the five types of
iterators in STL should be dropped in favor of a single type of iterator
that does everything. This would be a major loss in clarity and
expressibility.
 -- Pete

In article <D5t7sB.DHM@reston.icl.com> scc@reston.icl.com (Stephen Carlson) writes:


   In article <KINNUCAN.95Mar17093019@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:
   >In article <1995Mar14.235439.14903@wisipc.weizmann.ac.il> oren@hadar.weizmann.ac.il (Ben-Kiki Oren) writes:
[snip]
   >My own view is that STL iterators have not been proposed as a replacement
   >for pointers and therefore there is no need for them to duplicate all
   >the properties of pointers. Iterators are intended to be used to
   >navigate containers; there is no need for a null iterator to do this.
   >If your purpose is to pass an address of an object, use a pointer.

   If iterators are to be a generalization of pointers, shouldn't one of
   their most important behaviors (the ability to be null) also be part
   of the generalization?

No.

   In fact, it seems more correct, based on their semantics, to claim that
   iterators are generalization of pointers within an array rather than
   pointers in general.

Exactly. That's why IMHO a null iterator is a meaningless, useless,
and dangerously misleading construct.

- Paul

In article <3kdf13$bhb@druid.borland.com> pete@borland.com (Pete Becker) writes:


   In article <KINNUCAN.95Mar17093019@candide.hq.ileaf.com>, kinnucan@hq.ileaf.com (Paul Kinnucan) says:
   >
   >Hash tables are now available with STL.
   >

    Sorry. They were voted down at the ANSI/ISO meeting last week. Not
   because of any technical flaws in the proposal, but because adding them
   was simply too much work to take on at this point in the standards
   process. They're a great idea for release 2.
    -- Pete

Thanks for the update.

I should have been more precise. Someone has released a draft version
of an STL-compliant hash table template at ftp://butler.hpl.hp.com/stl
the hp butler ftp site. I have
a copy but haven't yet had a chance to evaluate it.

- Paul

In article <3kdf13$bhb@druid.borland.com>,
Pete Becker <pete@borland.com> wrote:
>In article <KINNUCAN.95Mar17093019@candide.hq.ileaf.com>, kinnucan@hq.ileaf.com (Paul Kinnucan) says:
>>
>>Hash tables are now available with STL.
>>
>
> Sorry. They were voted down at the ANSI/ISO meeting last week. Not
>because of any technical flaws in the proposal, but because adding them
>was simply too much work to take on at this point in the standards
>process. They're a great idea for release 2.
> -- Pete

 No need to be sorry. Hash tables are still available for STL.
Just because they're not in the Standard Library does not mean they're
not available :-)

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <1995Mar14.235439.14903@wisipc.weizmann.ac.il> oren@hadar.weizmann.ac.il (Ben-Kiki Oren) writes:


   Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:

[snip]

   > If that's
   > the case, providing a null-valued iterator buys you nothing (at least as far
   > as I can see).

   It allows STL to support the full range of programming idioms which are now
   being used for pointer-based programming, whereas the current situation allows
   just a subset. I'd say thats a big `something'.


I fail to see any advantage to using iterators as a replacement for
pointers. I've repeatedly asked proponents of the null-valued iterator
to give a concrete example of where a null-valued iterator would allow
code to be written more efficiently or more clearly if it were available.
But thus far none has been forthcoming.

So far all I've seen are theoretical arguments based on specious reasoning
e.g.,  "iterators are like pointers in SOME respects, therefore
they would be more useful if they were like pointers in ALL respects."
The fact is, the conclusion does not follow from the premise.

My own view is that STL iterators have not been proposed as a replacement
for pointers and therefore there is no need for them to duplicate all
the properties of pointers. Iterators are intended to be used to
navigate containers; there is no need for a null iterator to do this.
If your purpose is to pass an address of an object, use a pointer.


   Again, the basic ability which
   is missing is:

    if(p) {
     // Given p was not obtained from ++,
     // we can assume *p is safe
    }

Again, the basic ability is not missing because pointers have not
suddenly disappeared from the face of the earth with the advent
of STL iterators. You're example illustates the danger of arguing
from isolated pieces of code deprived of any context. In the above
example, if the function in which the statement is embedded needs
to deal with a container as a container, then it should require
that it be passed a pair of pointers. If instead, it is only
concerned with a specific item, it should require
that it be passed a pointer (possibly null-valued), e.g.

void ProcessItem(TItem* item)
{
   if (item) {
     // Do something to item
   }
}

TIterator i find(TIterator start, TIterator end, const TItem& value)
{
   while (start != end) {
     if (*start == value)
       return start;
     ++start;
   }
   return end;
}

TIterator i = find(List.begin(), List.end(), value);

if (i != List.end())
   ProcessItem(&*i);

   This ability is used in two extremely common idioms:

    if(p = find())
     ...

This idiom is not lost. If all you want from find(...) is
an object, then define find(...) to return a pointer
instead of an iterator.


    class A {
     p pointer_to_some_object_which_might_not_exist;
    }

Again, no problem. Define p as a pointer.


   I'd say the above two idioms, the first in particular, are widespread enough
   to justify the addition of null-iterator, especially since it does not effect
   the efficiency of the implementation in any way.



   Note that this missing ability is _not_ used in the common idiom:

    for(p = low; p < high; p++)
     ...

Actually, only == and != are defined for STL iterators.

   Which is already compatible with STL. However, the idiom:

    while(p = get_next_according_to_some_criteria())
     ...

   Is currenty not supported by STL.

It's not "supported" by the language, either. That's why you
have to write a "get-next" function yourself. Put another way,
STL supports "get-next" idioms to the same extent as the
language.

   STL tries to make all containers look like
   arrays, for which the first idiom is the most natural.

The benefit is that algorithms can be written independent of data
structures. Some see that as a major advantage.

   Yet, even for arrays,
   the second idiom is sometimes useful. Further, the second idiom can be used
   for a loop that spans multiple containers, while the first can't. In short,
   each has its place. Why should STL decide for me which one to use?

It doesn't. You're free to choose any idiom you like. It's just that
null-pointer idioms just turn out to be not particularly useful
for navigating containers, which is what iterators are for.


   P.S. STL needs hash tables!


Hash tables are now available with STL.

- Paul

Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
> In article <1995Feb28.213943.6880@wisipc.weizmann.ac.il> oren@hadar.weizmann.ac.il (Ben-Kiki Oren) writes:

> >  Once you give up on the idea of ruining the compatibility with pointers, that
> >  is you are resigned to supporting _both_ `end' _and_ `null' values, then
> >  implementation becomes trivial and cheap. For example, there is no need to
> >  mess up the ++ and --; I would say it is the programmer's responsibility not
> >  to apply them to a null iterator, just as it is his responsibility not to
> >  apply them to a null pointer. The null value is also a great candidate for the
> >  default value of an iterator.
>
> >  What is wrong with such a scheme?

> >  (For example),
> >  what if the neighbourhood of the found item needs to be examined? Something
> >  like:

> >    T *p = find();
> >    if(!p)
> >     return(not_found());
> >    ...
> >    while(p-- > low_limit && interesting(p))
> >     do_something(p);

> If you already have low_limit, you don't need null to figure out whether
> the iterator is invalid, e.g.

>         T*p = find();
>         if (p == low_limit)
>            return;

What if the code was trying to explore the elements "above" the
result of `find' instead of "below"? `find' can't tell in which context it is
going to be used in. Should it return low_limit or high_limit as a signal of
`not found'?

Further, this assumes `low_limit' is the absolute minimum of the container.
However, it happens that the results of a 'find' need to be tested vs. a
sub-range of the container. In such a case, the `low_limit' and `high_limit'
available to the caller of `find' have nothing to do with the real limits of
the container. Passing these limits to `find' also won't work because the
caller may be interested to know about hits outside the sub-range...

I think that the only workable scheme given the lack of a `null' iterator is
for `find' to use an additional bool flag for success/fail.

> The argument for providing a null value for iterators is that it allows
> you to pass only one iterator around.

Yes, just as it is useful to pass just one pointer around. What can you do
with an isolated pointer, after all?

- Check that the pointer is _not_ dereferencable: if it is NULL, it is
guaranteed not to be. This idiom is not supported by STL, but would be trivial
to add.

- Check that the pointer _is_ dereferencable: by convention, if any function
returns a non-NULL value, it is. The ++ and related operators are the only
ways which may produce the after-the-last pointer; their result therefore
needs to be tested against the boundaries. STL supports the
test-against-boundaries idiom, but since it does not support the NULL value,
it mandates it for every use of an iterator instead of just for values
obtained from ++. If NULL was supported, this idiom could be fully
supported as well.

- Look at the nearby elements: only if you have either an explicit (in the
form of low and high limits), or an implicit (in any other form) guarantee for
the number of elements "above" and "below" the pointer. This idiom is
supported by STL.

- Dereference the pointer. Supported by STL.

What I suggest is to add the one thing you can do to a pointer and not to
an iterator, to complete the support for pointer-based programming idioms.
It doesn't cost in efficiency and it doesn't complicate the implementation
(for any reasonable implementation, at least).

> You don't need to pass iterators
> that point to the bounds of the container.

I thought that the iterators pair can indicate any sub-range? Anyway, you'd
need to pass a pair of iterators in exactly the same places you needed to pass
a pair of pointers. Wasn't that the point of STL?

> In other words, you
> don't need low_limit. However, if you don't have access to the bounds, you can only
> safely dereference the iterator; you can't safely "explore the surroundings."

Correct, and again just like the case for pointers; I find the ability to
`only' safely dereference them rather useful, and I'd rather not give it up
:-)

> If that's
> the case, providing a null-valued iterator buys you nothing (at least as far
> as I can see).

It allows STL to support the full range of programming idioms which are now
being used for pointer-based programming, whereas the current situation allows
just a subset. I'd say thats a big `something'. Again, the basic ability which
is missing is:

 if(p) {
  // Given p was not obtained from ++,
  // we can assume *p is safe
 }

This ability is used in two extremely common idioms:

 if(p = find())
  ...

 class A {
  p pointer_to_some_object_which_might_not_exist;
 }

I'd say the above two idioms, the first in particular, are widespread enough
to justify the addition of null-iterator, especially since it does not effect
the efficiency of the implementation in any way.

Note that this missing ability is _not_ used in the common idiom:

 for(p = low; p < high; p++)
  ...

Which is already compatible with STL. However, the idiom:

 while(p = get_next_according_to_some_criteria())
  ...

Is currenty not supported by STL. STL tries to make all containers look like
arrays, for which the first idiom is the most natural. Yet, even for arrays,
the second idiom is sometimes useful. Further, the second idiom can be used
for a loop that spans multiple containers, while the first can't. In short,
each has its place. Why should STL decide for me which one to use? I thought
C++ was supposed to support many programming idioms, allowing the programmer
to pick the one appropriate for the task. Says so right on the introduction to
Bjarne's book :-)

> On the other hand, if you do provide a null-valued iterator,
> you encourage naive users to think that they can safely explore the surroundings,
> given an isolated iterator, which is not a good thing.

People which did:

 a = *++p;

Without thought for array boundaries, would write in STL:

 i = find();
 a = *i;

Without thought for failed find()s. It just proves the saying that "You can't
make a system fool-proof, because fools are so talented". I'd say that "naive
users" should read the manual, which would surely mention the dangers of
dereferencing NULL pointers/iterators and advancing pointers/iterators out of
their bounds. IMHO the fact that the two idioms are identical would make it
easier for them to remember the rules.

      Oren.

P.S. STL needs hash tables!

BTW, I've seen an ad for a commercial STL library in the latest C++ report
(the one that covered STL). Their STL version included multi-thread support
and, yes, hash tables. Does the current state of the standard allow for hash
tables and multi-threading or is this an "extension"? I thought both are very
difficult to implement while preserving the standard interface. I'm going to
need a multi-threaded library (_and_ hash tables), so the possibility of
getting them in STL is good news for me; but I don't want to get stuck with
some private extension scheme.

      Oren.

--
Life is tough and the hard ships are many.

In article <1995Mar14.235439.14903@wisipc.weizmann.ac.il>,
oren@hadar.weizmann.ac.il says...
>
>Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
>> In article <1995Feb28.213943.6880@wisipc.weizmann.ac.il>
oren@hadar.weizmann.a
>c.il (Ben-Kiki Oren) writes:
>
>> >  Once you give up on the idea of ruining the compatibility with
pointers, th
>at
>> >  is you are resigned to supporting _both_ `end' _and_ `null' values,
then
>> >  implementation becomes trivial and cheap. For example, there is no
need to
>> >  mess up the ++ and --; I would say it is the programmer's
responsibility no
>t
>> >  to apply them to a null iterator, just as it is his responsibility
not to
>> >  apply them to a null pointer. The null value is also a great
candidate for
>the
>> >  default value of an iterator.
>>
>> >  What is wrong with such a scheme?
>
>> >  (For example),
>> >  what if the neighbourhood of the found item needs to be examined?
Something
>> >  like:
>
>> >        T *p = find();
>> >        if(!p)
>> >                return(not_found());
>> >        ...
>> >        while(p-- > low_limit && interesting(p))
>> >                do_something(p);
>
>> If you already have low_limit, you don't need null to figure out
whether
>> the iterator is invalid, e.g.
>
>>         T*p = find();
>>         if (p == low_limit)
>>            return;
>
>What if the code was trying to explore the elements "above" the
>result of `find' instead of "below"? `find' can't tell in which context
it is
>going to be used in. Should it return low_limit or high_limit as a
signal of
>`not found'?
>

 Since this code is syntactically wrong and the discussion says
nothing about the context in which it is operating, it is very hard to
figure out what the point is. However, the STL requirement is that find()
returns the upper limit of the range if it does not succeed. The test is
always the same, and find() doesn't have to "tell in which context it is
going to be used". That's the programmer's job.

 template <class Iterator>
 void GuessWhatThisDoes( Iterator start,
    Iterator end,
    Value value )
 {
 Iterator res = find( start, end, value );
 if( res == end )
  return;
 // your code goes here
 }

You can do whatever you like at the end of this function: look at things
in [res,end), look at things in [start,res), whatever. This has nothing
to do with find().
 -- Pete

In article <3jl1is$5i9@hpsystem1.informatik.tu-muenchen.de>,
Ulf Schuenemann <schuenem@Informatik.TU-Muenchen.DE> wrote:
>
>In article <3j9fpd$q88@hustle.rahul.net>, "Ronald F. Guilmette" <rfg@rahul.net> writes:
>[..]
>|> >One could reduce the problem "why only 0 and not also 1" to a problem of type:
>|> >"Because 0 is of different type than 1. Only the type of 0 is allowed here"...
>|>
>|> I hope that you are not forgetting that the following is valid in that
>|> _other_ language that C++ is supposed to be compatible with:
>
>You should know that are already some incompatibilies to C [By good reasons].
>
>|>  char *cp = (1-1);
>And
> virtual void pure () = 2 * 3 - 6;
> virtual void pure () = (int) 0.0;
>
>To be honest, I would prefere if `const integral expression evaluating to 0'
>would be thrown out of C++ and the rules would be changed to literal '0'.
>
>Can you give me an example for the usefullness of C's rule ?

Oh, I didn't claim this sort of sillyness was useful.  I was just noting
what the rules of the game are (for the edification of anyone who was not
already aware of this particular one).

--

-- Ron Guilmette, Sunnyvale, CA ---------- RG Consulting -------------------
---- E-mail: rfg@segfault.us.com ----------- Purveyors of Compiler Test ----
-------------------------------------------- Suites and Bullet-Proof Shoes -

In article <3j9fpd$q88@hustle.rahul.net>, "Ronald F. Guilmette" <rfg@rahul.net> writes:
[..]
|> >One could reduce the problem "why only 0 and not also 1" to a problem of type:
|> >"Because 0 is of different type than 1. Only the type of 0 is allowed here"...
|>
|> I hope that you are not forgetting that the following is valid in that
|> _other_ language that C++ is supposed to be compatible with:

You should know that are already some incompatibilies to C [By good reasons].

|>  char *cp = (1-1);
And
 virtual void pure () = 2 * 3 - 6;
 virtual void pure () = (int) 0.0;

To be honest, I would prefere if `const integral expression evaluating to 0'
would be thrown out of C++ and the rules would be changed to literal '0'.

Can you give me an example for the usefullness of C's rule ?

[ It just makes compilerwriting more difficult, as I found with gcc:

const int i = 0;
const float f = 0;

struct X {
 virtual void pure1 () = 0;
 virtual void pure2 () = (1-1);
 virtual void pure3 () = 2*3-6;
 virtual void pure4 () = (int) f;
 virtual void pure5 () = (int) 0.0;
 virtual void pure6 () = (int) i; // ERROR: invalid initializer
 virtual void pure7 () = i;  // ERROR: invalid initializer
 virtual void pure8 () = (void*) i;
 virtual void pure9 () = (void*) 0;
};
]

Ulf Schuenemann

--------------------------------------------------------------------
Ulf Sch   nemann
Institut f   r Informatik, Technische Universit   t M   nchen.
email: schuenem@informatik.tu-muenchen.de
WWW:   http://www.informatik.tu-muenchen.de/cgi-bin/nph-gateway/hphalle2/~schuenem/index.html

In article <1995Mar4.015359.25680@news.snu.ac.kr>, nedkonz@gate.net writes:
>In <3iupvd$hg5@vbohub.vbo.dec.com>, johnston@caiman.enet.dec.com (Ian
>Johnston) writes:
>[...]

[snip]

>Sure, for raw vectors of data iterated using pointers you don't
>necessarily
>have thread safety, but what's to keep you from making a derived vector
>class that is thread-safe?
>
>

If I'm going to move away from using raw pointers to using classes
everywhere, then why stick to the STL model anyway?

--
Consulting for Digital Equipment Corp  johnston@caiman.enet.dec.com
      (33) 92.95.51.74

In article <3jjo56$nif@vbohub.vbo.dec.com> johnston@caiman.enet.dec.com (Ian Johnston) writes:



   In article <1995Mar4.015359.25680@news.snu.ac.kr>, nedkonz@gate.net writes:
   >In <3iupvd$hg5@vbohub.vbo.dec.com>, johnston@caiman.enet.dec.com (Ian
   >Johnston) writes:
   >[...]

   [snip]

   >Sure, for raw vectors of data iterated using pointers you don't
   >necessarily
   >have thread safety, but what's to keep you from making a derived vector
   >class that is thread-safe?
   >
   >

   If I'm going to move away from using raw pointers to using classes
   everywhere, then why stick to the STL model anyway?

It lets you apply algorithms written for thread-unsafe data structures
to thread-safe structures (derived from STL containers), thereby
reducing development costs.

I don't see how a threaded environment in any way nullifies the advantages
of STL over other container libraries in current use.

- Paul

>>>>> Ulf Schuenemann <schuenem@Informatik.TU-Muenchen.DE> writes:

>  virtual void pure () = 2 * 3 - 6;
>  virtual void pure () = (int) 0.0;

Actually, these are ill-formed.  The grammar calls for a literal zero in
this case.  gcc does not diagnose these errors, but that's a bug.

Jason

matt@physics2.berkeley.edu (Matt Austern) writes:

>In article <3j9fpd$q88@hustle.rahul.net> "Ronald F. Guilmette" <rfg@rahul.net> writes:


>> I hope that you are not forgetting that the following is valid in that
>> _other_ language that C++ is supposed to be compatible with:
>>
>>  char *cp = (1-1);

>But C++ isn't supposed to be compatible with C: there are lots of
>valid C programs that are either erronious C++ or that give different
>results depending on whether they're compiled as C or as C++.

>char *cp = (1-1) is currently legal C and C++, but I really have
>trouble believing that anybody uses it in their code except as part of
>an Obfuscated ANSI C Contest.  If special rules for "0" really did
>clean up the C++ language in any important way, and if the only
>downside were that code like "char *cp = 1-1" would no longer compile,
>then that would be a very small price.

I would agree with this.  However it would be easy to make my '0' idea
be compatable by saying that any constant integer expression that evaluates
to zero has the type '0'.  Then the above assignment is legal and is
parsed as a cast from a '0'.

Bill Spitzak

In article <3j9di1$p2b@hustle.rahul.net>,
Ronald F. Guilmette <rfg@rahul.net> wrote:
>In article <3isoqn$ln2@bcarh8ab.bnr.ca>, Brian White <bcwhite@bnr.ca> wrote:
>>In article <D4LEJq.8M5@ucc.su.OZ.AU>,
>>
>>... You can always compare the value of a pointer even if it
>>doesn't point to anything...
>
>This is not true in standard C, and I also hope (for the sake of the imple-
>mentors) that it will be not true in standard C++.

I don't understand what you mean.  There is absolutely no checking in C or
C++ on pointer ranges.  You can always compare the value of a pointer (not
necessarily the value of what it _points_ to).

eg: void* something;
 void* somethingelse;
 if (something != somethingelse) exit(1);

You'll get warnings from the compiler about using an uninitialized variable,
but it will still compile and execute.  If this isn't comparing the value
of a pointer that doesn't point to anything, I don't know what is.

                                        Brian
                                 ( bcwhite@bnr.ca )

-------------------------------------------------------------------------------
    In theory, theory and practice are the same.  In practice, they're not.

In article <3j9fpd$q88@hustle.rahul.net> "Ronald F. Guilmette" <rfg@rahul.net> writes:


> I hope that you are not forgetting that the following is valid in that
> _other_ language that C++ is supposed to be compatible with:
>
>  char *cp = (1-1);

But C++ isn't supposed to be compatible with C: there are lots of
valid C programs that are either erronious C++ or that give different
results depending on whether they're compiled as C or as C++.

char *cp = (1-1) is currently legal C and C++, but I really have
trouble believing that anybody uses it in their code except as part of
an Obfuscated ANSI C Contest.  If special rules for "0" really did
clean up the C++ language in any important way, and if the only
downside were that code like "char *cp = 1-1" would no longer compile,
then that would be a very small price.
--

                               --matt

In <3iupvd$hg5@vbohub.vbo.dec.com>, johnston@caiman.enet.dec.com (Ian Johnston) writes:
[...]

>Personally, I prefer the style that STL seems to make difficult. It seems
>more natural to me that an iterator is a fully-empowered object, which
>knows about the container it is attached to. This is also critical for safe
>multi-threaded handling of containers. Unless anyone can show me how
>to handle this in the STL style of iterator, this is a make or break
>feature for me.

Since you can make iterators for any particular kind of container, it is
not difficult to make the iterator refer to the container. For instance, make
sure that myContainer<T>::iterator has a reference to the container. Then
myContainer<T>::begin(), etc. can make sure that the iterator is constructed
with a reference to the container.

Sure, for raw vectors of data iterated using pointers you don't necessarily
have thread safety, but what's to keep you from making a derived vector
class that is thread-safe?

John Dubois (dubois@uwo.ca) wrote:
< In article <3ii4l7$pre@bcarh8ab.bnr.ca> bcwhite@bnr.ca (Brian White) writes:
< >In article <KINNUCAN.95Feb22182657@candide.hq.ileaf.com>,

< >It is useful for returning an error condition (eg. Item not found) instead
< >of having to return both the pointer/iterator _and_ a flag saying whether
< >it was sucessful or not.

< Why not just overload the ==(int) & !=(int) for all iterators? that way you
< can write: ....

That's what operator bool is for.  It does seem sensible to me that
operator bool be defined for iterators.  I've always done it where it made
'pointer-like' sense.

A.

--
Andrew Fitzgibbon (Research Associate),                     andrewfg@ed.ac.uk
Artificial Intelligence, Edinburgh University.               +44 031 650 4504
<a href=http://www.dai.ed.ac.uk/staff/personal_pages/andrewfg> Home Page </a>
              "I have seven friends, they taught me all I know.
        Their names are who, why, what, where, which, when, and how."

>
>
>   Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
>
>   : An iterator, by definition, can never by invalid; it can have only
>   : three states: 1) pointing to the beginning of a container, 2) pointing
>   : to the end, or 3) pointing to an item.
>
horstman@sjsumcs.sjsu.edu (Cay Horstmann) writes:
>Sadly, that is not true. Not only is 1) a subcase of 3), but more importantly
>the iterator is invalidated if the container is modified through another
>iterator or a member function, replaced through assignment, or the
>container is destroyed.

Paul Kinnucan <kinnucan@hq.ileaf.com> wrote:
>Yes, you're correct. I went back and read the STL documentation
>more carefully over the weekend. You can also decrement or
>increment an iterator past the beginning or end of a container, making
>it invalid. Uninitialized iterators are also invalid (cannot be
>dereferenced.)

Worse.  As interators have all the limitations of pointers, invalid
iterators may not be even referenced.

      Ross Ridge

--
 l/  //   Ross Ridge -- The Great HTMU, Ook                    +1 519 883 4329
[oo][oo]  rridge@csclub.uwaterloo.ca      http://csclub.uwaterloo.ca/u/rridge/
-()-/()/
 db  //

In article <3isoqn$ln2@bcarh8ab.bnr.ca>, Brian White <bcwhite@bnr.ca> wrote:
>In article <D4LEJq.8M5@ucc.su.OZ.AU>,
>
>... You can always compare the value of a pointer even if it
>doesn't point to anything...

This is not true in standard C, and I also hope (for the sake of the imple-
mentors) that it will be not true in standard C++.

--

-- Ron Guilmette, Sunnyvale, CA ---------- RG Consulting -------------------
---- E-mail: rfg@segfault.us.com ----------- Purveyors of Compiler Test ----
-------------------------------------------- Suites and Bullet-Proof Shoes -

In article <3j53j8$g2c@hpsystem1.informatik.tu-muenchen.de>,
Ulf Schuenemann <schuenem@Informatik.TU-Muenchen.DE> wrote:
>
>Regarding 0 as a constant of a type different from other integral or pointer
>types could give the special treatment of 0 a better basis. E.g.:
>
>- The literal value 0 (known as NULL) is compatible to any pointertype;
>  this is not the case for any other literal value.
>- By appending '= 0' to a memberfunctiondeclaration you make it pure virtual.
>  No other pointers or integral values or names of any type are allowed.
>
>One could reduce the problem "why only 0 and not also 1" to a problem of type:
>"Because 0 is of different type than 1. Only the type of 0 is allowed here"...

I hope that you are not forgetting that the following is valid in that
_other_ language that C++ is supposed to be compatible with:

 char *cp = (1-1);

--

-- Ron Guilmette, Sunnyvale, CA ---------- RG Consulting -------------------
---- E-mail: rfg@segfault.us.com ----------- Purveyors of Compiler Test ----
-------------------------------------------- Suites and Bullet-Proof Shoes -

In article <3ijvmj$q09@hollywood.cinenet.net>, spitzak@news.cinenet.net (Bill Spitzak) writes:
[..]
|> Silly Idea:

There are no silly ideas. Interesting idea!

|> I would like to see '0' treated as a special built-in type in C++.
|> You cannot declare variables with type '0', the only way to get an
|> instance is with an explicit constant of '0'.  The only time you can
|> use the type '0' is when declaring a function argument:
|>
|>  // these define two different functions!
|>  int foo(int a, 0);
|>  int foo(int a, int b);
|>
|> There is a built-in cast from '0' to all built-in types.  User-defined
|> classes may define casts (class X {X(0);...}) to convert from zero.
|>
|> This is most useful for making "pointer-like" objects that can be
|> compared to and assigned a value of zero.
[..]

Regarding 0 as a constant of a type different from other integral or pointer
types could give the special treatment of 0 a better basis. E.g.:

- The literal value 0 (known as NULL) is compatible to any pointertype;
  this is not the case for any other literal value.
- By appending '= 0' to a memberfunctiondeclaration you make it pure virtual.
  No other pointers or integral values or names of any type are allowed.

One could reduce the problem "why only 0 and not also 1" to a problem of type:
"Because 0 is of different type than 1. Only the type of 0 is allowed here"
Explainations in terms of types are a widespread technique, well known
and understood.

N.B.: Allthough inconvineant, this could be a chance to REDUCE THE COMPLEXITY
of the desciption of C++.

[
BTW another application of the 'silly idea':
For functions with parameter of pointer or integral type you could prevent
calling it with a literall 0 by provinding another (private member-)
function.
(This is similar to references: You can't call a function with a literal 0,
but with an expression evaluating to 0 of some pointertype.)
E.g. for builtin integral types T one could say that
"T T::operator/ (0)" should be private to T.
]

|> Bill Spitzak


Ulf Schuenemann

--------------------------------------------------------------------
Ulf Sch   nemann
Institut f   r Informatik, Technische Universit   t M   nchen.
email: schuenem@informatik.tu-muenchen.de

In article <KINNUCAN.95Feb27180242@candide.hq.ileaf.com>,
kinnucan@hq.ileaf.com (Paul Kinnucan) writes:
>
>Your iterator increment operator must set the iterator to 0 when it
>reaches the end of the container. This cannot be done with STL
>iterator incrementing without defeating the very purpose of STL. As far as
>I
>know, it is not possible to use an STL iterator safely in isolation
>for iteration, (only for dereferencing and then only if you provided
>a null value.) That is why STL algorithms
>always take pairs of iterators as arguments, one pointing to the
>beginning of a range of items, the other to the end. So you're argument
>that providing a null iterator value would allow an iterator to
>be passed around in isolation both for container navigation and
>dereferencing is not valid for STL and cannot be made valid as far
>as I can see without destroying the very basis of STL.
>

This whole thing about iterators seems to be getting out of hand.

I think the crucial question to ask is 'what is the most useful paradigm'?
STL proponents seem to feel that making iterators look just like using
pointers is a key advantage. Others seem to feel that being able to use
iterators in isolation is a convenient and useful technique. Clearly, STL,
as it stands, supports the first approach but not the second.

The question then becomes one more of coding style. STL supports
one coding style, but not the other.

Personally, I prefer the style that STL seems to make difficult. It seems
more natural to me that an iterator is a fully-empowered object, which
knows about the container it is attached to. This is also critical for safe
multi-threaded handling of containers. Unless anyone can show me how
to handle this in the STL style of iterator, this is a make or break
feature for me.

I did post a question a couple of months back about how STL and
multi-threading would coexist, but no-one ever replied -:( I'm still
keeping my fingers crossed that it isn't an issue that has just been
overlooked.


--
Consulting for Digital Equipment Corp  johnston@caiman.enet.dec.com
      (33) 92.95.51.74

There is something I don't get about this discussion.

Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
> An iterator, by definition, can never by invalid; it can have only
> three states: 1) pointing to the beginning of a container, 2) pointing
> to the end, or 3) pointing to an item. So there can never by any
> circumstance in which an iterator (as an iterator) can naturally (i.e.,
> via iterator operations) acquire a null value. Apparently, STL
> critics want a null iterator value so that it can be used to indicate
> whether an iterator points to an item (as opposed to the beginning
> or end of a container).

Why is it taken for granted that adding a null iterator value _replaces_ the
`beginning' and `end' values?

Consider: a pointer, originally pointing to an array element, can also be
in only one of the above three states, and no `natural' operation (such as ++)
can change this fact. And in fact, when looping on arrays, the idiom
`for(p = first; p < last; p++)' is common in C, without any use of NULLs.

However, the idiom of using the null value to indicate a failed search,
an invalid position, etc., is even more common. The two do not proclude each
other! Even in C:

p == NULL -=> *p does not exist.
p != NULL -=> *p typically exists, unless p points to the `end' position.

STL claims to allow idioms from pointer-based programming to be transferred to
generic container programming. Disallowing NULL iterators prohibits the use of
many such programming idioms, especially in searches, keeping locations of
items which do not always exist, etc. Example: its fine to say that a `find'
can return a pointer, so it'll be easy to indicate a `not-found' result. But
what if the neighbourhood of the found item needs to be examined? Something
like:

 T *p = find();
 if(!p)
  return(not_found());
 ...
 while(p-- > low_limit && interesting(p))
  do_something(p);

Once you give up on the idea of ruining the compatibility with pointers, that
is you are resigned to supporting _both_ `end' _and_ `null' values, then
implementation becomes trivial and cheap. For example, there is no need to
mess up the ++ and --; I would say it is the programmer's responsibility not
to apply them to a null iterator, just as it is his responsibility not to
apply them to a null pointer. The null value is also a great candidate for the
default value of an iterator.

What is wrong with such a scheme?

     Oren.

P.S.

STL needs hash tables!

     Oren.

--
Life is tough and the hard ships are many.

In article <3im5l7$3vm@jupiter.SJSU.EDU> horstman@sjsumcs.sjsu.edu (Cay Horstmann) writes:


   Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:

   : An iterator, by definition, can never by invalid; it can have only
   : three states: 1) pointing to the beginning of a container, 2) pointing
   : to the end, or 3) pointing to an item.

   Sadly, that is not true. Not only is 1) a subcase of 3), but more importantly
   the iterator is invalidated if the container is modified through another
   iterator or a member function, replaced through assignment, or the
   container is destroyed.


Yes, you're correct. I went back and read the STL documentation
more carefully over the weekend. You can also decrement or
increment an iterator past the beginning or end of a container, making
it invalid. Uninitialized iterators are also invalid (cannot be
dereferenced.)

- Paul

In article <D4LEJq.8M5@ucc.su.OZ.AU>,
John Max Skaller <maxtal@Physics.usyd.edu.au> wrote:
>In article <MATT.95Feb15133154@physics10.berkeley.edu>,
>Matt Austern <matt@physics.berkeley.edu> wrote:
>>
>>The STL is not the ideal standard library; neither is C++ the ideal
>>language.  In the real world of engineering we don't want "ideal"; we
>>want "good enough to be useful".
>
> With all due respect to Matt, this attitude is a sign of
>the downfall of America. What ever happened to the pursuit of excellence?

I must disagree.  Any good businessman will tell you that "good right now"
is infinitely better than "perfect sometime in the future".

This is why America is the business and innovative giant that it is;
because viable, "usable" technology is made available today instead of
waiting for better to be developed tomorrow.

Pursuit of excellence is a grand idea and nobody is suggesting doing
otherwise.  However, you have use what is available today or you have
nothing to build upon but ideas.

C++ is widely used not because it is perfect, but because it built upon
an already large acceptance of C.  Had it not been compatible, it would
be a "niche" language and largely ignored except for a side column in some
magazine procaiming its excellence.


> I want ideal. That doesn't mean I won't use what is available,
>ideal or not, but I will always be searching for something better.

Good.  C++ and STL are what is available.  Use them and make them better.


>>The STL is already a very useful collection class library; adding a
>>standard way of testing whether or not an iterator is valid would make
>>the STL more useful.
>
> But it isn't possible. Because a raw pointer is an iterator,
>and it can't be tested for "validity". Indeed, by definition,
>if the pointer does not have a valid value you can't do ANYTHING
>with it. So a validity test is useless.

Not true.  You can always compare the value of a pointer even if it
doesn't point to anything.  A "raw pointer" is tested for validity
by comparing it to NULL.  If I "can't do ANYTHING with it", then I
would never know if 'malloc' suceeded or failed.  Ask people who
use programs that crash when they run out of memory if this validity
test is useless.


>>My father once told me that there were two kinds of PhD theses: the
>>perfect ones, and the finished ones.  The same is true of language
>>standards.
>
> A tired old argument. If you aim for perfection but have
>a balanced expectation of resources, you will produce a good
>work. If you start off being prepared to accept crap, that is surely
>what you will get.

That was his point.  If you only accept perfection (hence the "perfect
theses"), then you will never finish.

In article <1995Feb28.213943.6880@wisipc.weizmann.ac.il> oren@hadar.weizmann.ac.il (Ben-Kiki Oren) writes:


   There is something I don't get about this discussion.

   Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
   > An iterator, by definition, can never by invalid; it can have only
   > three states: 1) pointing to the beginning of a container, 2) pointing
   > to the end, or 3) pointing to an item. So there can never by any
   > circumstance in which an iterator (as an iterator) can naturally (i.e.,
   > via iterator operations) acquire a null value. Apparently, STL
   > critics want a null iterator value so that it can be used to indicate
   > whether an iterator points to an item (as opposed to the beginning
   > or end of a container).

   Why is it taken for granted that adding a null iterator value _replaces_ the
   `beginning' and `end' values?

   Consider: a pointer, originally pointing to an array element, can also be
   in only one of the above three states, and no `natural' operation (such as ++)
   can change this fact. And in fact, when looping on arrays, the idiom
   `for(p = first; p < last; p++)' is common in C, without any use of NULLs.

   However, the idiom of using the null value to indicate a failed search,
   an invalid position, etc., is even more common. The two do not proclude each
   other! Even in C:

   p == NULL -=> *p does not exist.
   p != NULL -=> *p typically exists, unless p points to the `end' position.

   STL claims to allow idioms from pointer-based programming to be transferred to
   generic container programming. Disallowing NULL iterators prohibits the use of
   many such programming idioms, especially in searches, keeping locations of
   items which do not always exist, etc. Example: its fine to say that a `find'
   can return a pointer, so it'll be easy to indicate a `not-found' result. But
   what if the neighbourhood of the found item needs to be examined? Something
   like:

    T *p = find();
    if(!p)
     return(not_found());
    ...
    while(p-- > low_limit && interesting(p))
     do_something(p);

   Once you give up on the idea of ruining the compatibility with pointers, that
   is you are resigned to supporting _both_ `end' _and_ `null' values, then
   implementation becomes trivial and cheap. For example, there is no need to
   mess up the ++ and --; I would say it is the programmer's responsibility not
   to apply them to a null iterator, just as it is his responsibility not to
   apply them to a null pointer. The null value is also a great candidate for the
   default value of an iterator.

   What is wrong with such a scheme?


If you already have low_limit, you don't need null to figure out whether
the iterator is invalid, e.g.

        T*p = find();
        if (p == low_limit)
           return;

The argument for providing a null value for iterators is that it allows
you to pass only one iterator around. You don't need to pass iterators
that point to the bounds of the container. In other words, you
don't need low_limit. However, if you don't have access to the bounds, you can only
safely dereference the iterator; you can't safely "explore the surroundings." If that's
the case, providing a null-valued iterator buys you nothing (at least as far
as I can see). On the other hand, if you do provide a null-valued iterator,
you encourage naive users to think that they can safely explore the surroundings,
given an isolated iterator, which is not a good thing.

- Paul

Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:

: An iterator, by definition, can never by invalid; it can have only
: three states: 1) pointing to the beginning of a container, 2) pointing
: to the end, or 3) pointing to an item.

Sadly, that is not true. Not only is 1) a subcase of 3), but more importantly
the iterator is invalidated if the container is modified through another
iterator or a member function, replaced through assignment, or the
container is destroyed.

Cay

matt@physics2.berkeley.edu (Matt Austern) writes:

>kinnucan@hq.ileaf.com (Paul Kinnucan) writes:
>
>> A function that searches an STL data structure can indicate a failure
>> by returning an iterator that points to the end-of-the-container (not the
>> same as the last item in the container), e.g.
>>
>> Suppose Container is an object of a container class derived from
>> an STL container with the addition of a find() member.
>>
>> TItemIterator i = Container.find("foo");
>>
>> if (i != Container.end()) {
>
>The problem with this is that testing such a result requires that you
>keep around both the iterator and the container to which it belongs:

No it doesn't.  You can keep around a pair of iterators instead.

>This is rather contrary to the spirit of the STL: all of the STL
>generic algorithms use iterator ranges, instead of whole containers.
>Note that this isn't just a convenience, but that it's absolutely
>crucial to how the STL is supposed to be used: by passing around
>pointers, instead of containers, you can do operations without knowing
>or caring just what sort of container you're dealing with.  It's a bit
>of a wart that in this one single respect, testing the validity of an
>iterator, you do need to pass around the original container.

No, you can pass around the "end" iterator instead.

--
Fergus Henderson - fjh@munta.cs.mu.oz.au
all [L] (programming_language(L), L \= "Mercury") => better("Mercury", L) ;-)

kinnucan@hq.ileaf.com (Paul Kinnucan) writes:

      (matt:)
      The problem with this is that testing such a result requires that you
      keep around both the iterator and the container to which it belongs:
      i != Container.end() makes no sense unless you still have Container.

   If this is a problem in a given application, simply define find() to return
   a pointer to an object instead of an iterator.

Well, sure, but being able to ask an iterator whether it still
'points' to a valid value from the container could still be
_convenient_ sometimes, right?

   via iterator operations) acquire a null value. Apparently, STL
   critics want a null iterator value so that it can be used to indicate
   whether an iterator points to an item (as opposed to the beginning
   or end of a container). This usage would require that somewhere in
   an application program that every iterator be tested after an incr/decr
   operation and then assigned to the null value if it points to the
   beginning or end of the container. In other words, adding a null
   iterator would add an extra step to application code, which defeats
   the very reason for adding it in the first place.

I do not believe that, if properly implemented, this extra step would
be very costly.

   If you're concern is to use a single object to convey
   the location of an item in memory or the fact that the item could not be
   found, then pass a pointer around.

No, that would be really inconvenient, for then one would lose the
incr/decr navigation functions, right? In a linked list, doing a ++ on
a pointer to an element will fail to produce the intended result. The
whole point is to have both an iterator that one can pass around
(without needing to pass around the container) _and_ a way to test
whether it is pointing to a 'real' object, and not the begin/end
value. We have built such iterators in our own library and it proves
to be very convenient. Not absolutely essential (there's always a way
to hack around anything), but really convenient. One thing we like
about our iterators-with-null-values is that it allows for loops like:

while(someIterator++) dosomethingwith(someIterator);

which is, especially for C oriented programmers, a rather natural
way of expressing things.

      (matt:)
      The STL is extremely useful even without null iterators and hash
      tables.  I hope, though, that both hash tables and null iterators
      do get added to the standard.

   I agree only if a real benefit can be demonstrated by such an
   addition. So far I haven't seen a benefit demonstrated.

I hope you only refer to the null iterators. I've followed the
discussion about hash tables and I found it unbelievable that some STL
defenders actually try to maintain that hash tables aren't a useful
addition to STL.

Jan Peter

This discussion reminds me of a class presented in
the book Scientific and Engineering C++: (etc.).

They had a template class Falliable. Faliiable<int>
would be just like an int, but would have a
isValid function. If you tried to access the
int when isValid() returns 0, you get a
run time error. You could use the invalid
state as you null iterator and use

Fallible<iterator>

instead of iterator.

Mark Stankus

nagle@netcom.com (John Nagle) writes:

>      For the pointer case, the "valid" predicate is simply a bounds
>check against the container limits.  (With an STL container, unlike a
>raw C/C++ array, you can find out how big it is.)  This is implemented
>as a simple inline function with an expression like
>
> (p >= low && p <= high)
>
>which should generate good fast code.

That code is not strictly conforming code, since pointer comparisons
are only guaranteed to work if the pointers point to within the same
array.  The above expression might always evaluate to true.

--
Fergus Henderson - fjh@munta.cs.mu.oz.au
all [L] (programming_language(L), L \= "Mercury") => better("Mercury", L) ;-)

In article <MATT.95Feb15133154@physics10.berkeley.edu>,
Matt Austern <matt@physics.berkeley.edu> wrote:
>
>The STL is not the ideal standard library; neither is C++ the ideal
>language.  In the real world of engineering we don't want "ideal"; we
>want "good enough to be useful".

 With all due respect to Matt, this attitude is a sign of
the downfall of America. What ever happened to the pursuit of excellence?

 I want ideal. That doesn't mean I won't use what is available,
ideal or not, but I will always be searching for something better.

>The STL is already a very useful collection class library; adding a
>standard way of testing whether or not an iterator is valid would make
>the STL more useful.

 But it isn't possible. Because a raw pointer is an iterator,
and it can't be tested for "validity". Indeed, by definition,
if the pointer does not have a valid value you can't do ANYTHING
with it. So a validity test is useless.

>My father once told me that there were two kinds of PhD theses: the
>perfect ones, and the finished ones.  The same is true of language
>standards.

 A tired old argument. If you aim for perfection but have
a balanced expectation of resources, you will produce a good
work. If you start off being prepared to accept crap, that is surely
what you will get.

 There are reasonable notions of precision, for example
the MODULA Standard is specified using formal semantics. They aimed
high and achieved something excellent. Perhaps not perfect.

 IMHO STL is a superb architecture because it was designed
as _consequences_ of a core architectural model. It is that
kernel of assumptions that IS in fact the real STL. The containers
iterators and algorithms are just examples which happen to be
convenient.

 What is being Standardised is principally the ARCHITECTURE.
It is that -- without a single class or piece of code -- which
makes Standardisation of STL one of the most important advances
in C++ since the ARM. Basically:

 C + Templates == STL

 That is, STL is very close to being IDEAL. It is not
necessarily ideal _for a given application_: I have not found
much use for it in my graphics library yet.

 Which is what makes it ideal. I can take it, or any part
of it, or I can leave it.

 Just try doing that with any "Root Object" style of library.
 Try it with mixins. (more flexible but still limited)

--
        JOHN (MAX) SKALLER,         INTERNET:maxtal@suphys.physics.su.oz.au
 Maxtal Pty Ltd,
        81A Glebe Point Rd, GLEBE   Mem: SA IT/9/22,SC22/WG21
        NSW 2037, AUSTRALIA     Phone: 61-2-566-2189

In article <JANPETER.95Feb25170259@mpih16> janpeter@mpih16 (Jan Peter de Ruiter) writes:

   kinnucan@hq.ileaf.com (Paul Kinnucan) writes:

  (matt:)
  The problem with this is that testing such a result requires that you
  keep around both the iterator and the container to which it belongs:
  i != Container.end() makes no sense unless you still have Container.

      If this is a problem in a given application, simply define find() to return
      a pointer to an object instead of an iterator.

   Well, sure, but being able to ask an iterator whether it still
   'points' to a valid value from the container could still be
   _convenient_ sometimes, right?

Yes, but I can't think of a case where it's particularly advantageous and
I can think of a real danger in providing it. Namely, if you provide a null
iterator, then people will think it's okay to use iterators in isolation
for navigating containers. The problem is that an STL
iterator by itself is useful only for dereferencing. You can't increment
or decrement it safely because you don't know where it is pointing relative to
other objects in the container. However, if you encourage people to pass
isolated iterators around (by providing a null value), you run the risk
of encouraging people to think that they can use isolated iterators to
navigate a container, which they can't do (at least not safely).

      via iterator operations) acquire a null value. Apparently, STL
      critics want a null iterator value so that it can be used to indicate
      whether an iterator points to an item (as opposed to the beginning
      or end of a container). This usage would require that somewhere in
      an application program that every iterator be tested after an incr/decr
      operation and then assigned to the null value if it points to the
      beginning or end of the container. In other words, adding a null
      iterator would add an extra step to application code, which defeats
      the very reason for adding it in the first place.

   I do not believe that, if properly implemented, this extra step would
   be very costly.

No, but what's the benefit?

      If you're concern is to use a single object to convey
      the location of an item in memory or the fact that the item could not be
      found, then pass a pointer around.

   No, that would be really inconvenient, for then one would lose the
   incr/decr navigation functions, right? In a linked list, doing a ++ on
   a pointer to an element will fail to produce the intended result. The
   whole point is to have both an iterator that one can pass around
   (without needing to pass around the container) _and_ a way to test
   whether it is pointing to a 'real' object, and not the begin/end
   value. We have built such iterators in our own library and it proves
   to be very convenient. Not absolutely essential (there's always a way
   to hack around anything), but really convenient. One thing we like
   about our iterators-with-null-values is that it allows for loops like:

   while(someIterator++) dosomethingwith(someIterator);

   which is, especially for C oriented programmers, a rather natural
   way of expressing things.

Your iterator increment operator must set the iterator to 0 when it
reaches the end of the container. This cannot be done with STL
iterator incrementing without defeating the very purpose of STL. As far as I
know, it is not possible to use an STL iterator safely in isolation
for iteration, (only for dereferencing and then only if you provided
a null value.) That is why STL algorithms
always take pairs of iterators as arguments, one pointing to the
beginning of a range of items, the other to the end. So you're argument
that providing a null iterator value would allow an iterator to
be passed around in isolation both for container navigation and
dereferencing is not valid for STL and cannot be made valid as far
as I can see without destroying the very basis of STL.


  (matt:)
  The STL is extremely useful even without null iterators and hash
  tables.  I hope, though, that both hash tables and null iterators
  do get added to the standard.

      I agree only if a real benefit can be demonstrated by such an
      addition. So far I haven't seen a benefit demonstrated.

   I hope you only refer to the null iterators. I've followed the
   discussion about hash tables and I found it unbelievable that some STL
   defenders actually try to maintain that hash tables aren't a useful
   addition to STL.

No, I think extending STL to support hash tables is an excellent idea. I'm
also not opposed to null-valued iterators if someone can show how they could
be incorporated into the STL model in a way that preserves the integrity of
the model while at the same time offering demonstrable benefits. I've been
using a class libary for years (Borland's Class Lib) that offers null-valued
iterators but lacks the theoretical underpinnings, functionality, efficiency, and
elegance of STL. As far as I'm concerned, the loss of a null iterator value
is a very small price to pay for the benefits of STL.

- Paul

In article <nagleD49GM1.Bqu@netcom.com>, nagle@netcom.com (John Nagle) wrote:

:      Actually, it might be possible to do that.
:
:      First, you'd like to have some functions that test iterators for
: validity.  While every iterator is associated with a container, there's
: no way to find a container from one of its iterators.  So we won't
: be able to write
:
:         if (valid(iter)) { ...}
:
: but will have to have syntax like
:
:         if (contr.valid(iter)) { ... }
:
: (where "contr" is a container and "iter" is an iterator from that container)
:
: So that's what I'd suggest adding: a bool-valued function "valid" for
: each container type which returns "true" if the iterator passed to it
: is dereferencable.
:
: That's clear enough, and easy to understand and use.  And it's a
: straightforward extension, not a modification of existing syntax.

Named like the global set membership tests ...

      bool includes( const_iterator ) const;

it would fit nicely in all containers.

:       This would, I think, provide the capabilities of the original
: proposal in a simpler way.
:
:                                         John Nagle

Agreed.

   Lars Farm

--
Lars.Farm@nts.mh.se

In article <MATT.95Feb21142523@physics10.berkeley.edu> matt@physics10.berkeley.edu (Matt Austern) writes:

   In article <KINNUCAN.95Feb21213256@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:

   > I was replying to you. An iterator is a dimensionless object. It
   > indicates a location in a container. An iterator itself literally
   > cannot have a beginning or an end. Therefore your demand that
   > STL provide an "end of iterator" object is as literally meaningless
   > as it would be to ask that C provide an "end of pointer" object.
   > Unless, of course, you meant to say "end of container." In this
   > case, both C and STL provide such an object: it is a pointer (or an
   > iterator) that points to the location immediately following the
   > last item in array (container).

   C also provides a pointer that, by definition, cannot point to any
   valid object.  C defines pointer semantics so that it is possible
   to create pointers with this value and it is possible to compare
   any pointer for equality with this value.


Defining a "null" iterator to mark the end of a container would
defeat the very raison d'etre of STL, namely to unify data structures
such that  algorithms that manipulate data structures will
work regardless of the type of  the underlying data structure.

This is because the use of null pointers to mark the end of
single-ended lists (which is what I assume critics of STL
have in mind when they talk about the usefulness of null
pointers) does not generalize to navigation of other
types of data structures. Whereas the pointer incr/decr model
works for many (all?) types of containers. Some container libraries
(e.g. Borland's) define the incr operator to overfow to a "null"
value when it reaches the end of the container. This, however, is
contrary to the behavior of the built-in C/C++ incr operator. Moreover,
the demand that STL provide "null" as an end of container indicator
begs the whole question of how to mark the beginning of a container
for either forwards or backwards iteration.

   Null pointers are very convenient, and most C code that manipulates
   pointers uses null pointers in one way or another.  STL iterators
   don't have any analogue of null pointers; it would be nice if they
   did.

   The lack of null iterators (for lack of a better term) isn't a
   crippling limitation: it's still possible to do useful work with the
   STL even in their absence.  It is annoying, though.


Could you give a specific example of where a "null" iterator would
come in handy? I am really having a hard time seeing what all the fuss is
about and concrete examples might help me and others to understand why
this lack of a null iterator is such a big issue with STL critics.


- Paul

In article <KINNUCAN.95Feb22182657@candide.hq.ileaf.com>,
kinnucan@hq.ileaf.com says...
>
>This is because the use of null pointers to mark the end of
>single-ended lists (which is what I assume critics of STL
>have in mind when they talk about the usefulness of null
>pointers) does not generalize to navigation of other
>types of data structures. Whereas the pointer incr/decr model
>works for many (all?) types of containers. Some container libraries
>(e.g. Borland's) define the incr operator to overfow to a "null"
>value when it reaches the end of the container. This, however, is
>contrary to the behavior of the built-in C/C++ incr operator.

 Iterators in BIDS don't try to imitate pointers. They use a
different model, based on argv. So the BIDS iterators use a 0 to indicate
the end of the container. As a practical matter, this means that
iterators typically must be able to examine the container that they point
into in order to determine when they are done. If I had seen STL before I
designed those iterators I would probably have adopted STL's model.
 -- Pete

In article <3ii4l7$pre@bcarh8ab.bnr.ca> bcwhite@bnr.ca (Brian White) writes:

   In article <KINNUCAN.95Feb22182657@candide.hq.ileaf.com>,
   Paul Kinnucan <kinnucan@hq.ileaf.com> wrote:
   >Could you give a specific example of where a "null" iterator would
   >come in handy? I am really having a hard time seeing what all the fuss is
   >about and concrete examples might help me and others to understand why
   >this lack of a null iterator is such a big issue with STL critics.

   It is useful for returning an error condition (eg. Item not found) instead
   of having to return both the pointer/iterator _and_ a flag saying whether
   it was sucessful or not.

A function that searches an STL data structure can indicate a failure
by returning an iterator that points to the end-of-the-container (not the
same as the last item in the container), e.g.

Suppose Container is an object of a container class derived from
an STL container with the addition of a find() member.

TItemIterator i = Container.find("foo");

if (i != Container.end()) {
   // do whatever
}
else
  report_error();

Or, if you prefer, you can implement find() to return a pointer (or 0)
to the found object.

   Most C/C++ functions that return pointers (eg. malloc & new) return NULL
   if they failed.  Doing the same for all iterators would be an obvious
   extension.

malloc and new are not container navigation functions and hence your
analogy does not apply.

STL iterators generalize the use of pointers as iterators, i.e., as pointers
to specific locations within a data structure that can be incremented
or decremented from one item to the next. Since 0 can never be
a position within a valid data structure, the concept of a null iterator
is literally meaningless. Granted, fictions, even meaningless ones, can
sometimes be useful, but you have not provided an example of a case where
the fiction of a null iterator allows something to be done that can't be
done just as easily without it.

- Paul

In article <3igrn7$rlh@druid.borland.com> pete@borland.com (Pete Becker) writes:

       >From: pete@borland.com (Pete Becker)
       >Newsgroups: comp.std.c++,comp.lang.c++
       >Date: Wed, 22 Feb 95 18:24:24 PST
       >Organization: Borland
       >
       >In article <KINNUCAN.95Feb22182657@candide.hq.ileaf.com>,
       >kinnucan@hq.ileaf.com says...
       >>
       >>This is because the use of null pointers to mark the end of
       >>single-ended lists (which is what I assume critics of STL
       >>have in mind when they talk about the usefulness of null
       >>pointers) does not generalize to navigation of other
       >>types of data structures. Whereas the pointer incr/decr model
       >>works for many (all?) types of containers. Some container libraries
       >>(e.g. Borland's) define the incr operator to overfow to a "null"
       >>value when it reaches the end of the container. This, however, is
       >>contrary to the behavior of the built-in C/C++ incr operator.
       >
       > Iterators in BIDS don't try to imitate pointers. They use a
       >different model, based on argv. So the BIDS iterators use a 0 to indicate
       >the end of the container. As a practical matter, this means that
       >iterators typically must be able to examine the container that they point
       >into in order to determine when they are done. If I had seen STL before I
       >designed those iterators I would probably have adopted STL's model.
       > -- Pete
       >

That's an interesting comment. I'd like to know what the designers of
existing commercial (or at least widespread) class libraries that
cover roughly the same functionality think about STL. It looks impressing
to me (that's probably a synonym for "I don't really understand it" :-)),
and it'd be interesting to hear comments from the designers of other
libraries. Would you write the library in an "STL way" if you started
again? Are there plans to build libraries (like Tools.h++, ICLCC etc.)
on top of STL in the future? Does STL need to be encapsulated to be as
easy to use as, say, Tools.h++?

--
 Stefan Tilkov                                MLC Ratingen, Germany
                                                 st@mlc-ratingen.de
                                         Phone +49 (0) 2102 8506 20
                                         Fax   +49 (0) 2102 8506 30

In article <KINNUCAN.95Feb23174429@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:

> A function that searches an STL data structure can indicate a failure
> by returning an iterator that points to the end-of-the-container (not the
> same as the last item in the container), e.g.
>
> Suppose Container is an object of a container class derived from
> an STL container with the addition of a find() member.
>
> TItemIterator i = Container.find("foo");
>
> if (i != Container.end()) {

The problem with this is that testing such a result requires that you
keep around both the iterator and the container to which it belongs:
i != Container.end() makes no sense unless you still have Container.

This is rather contrary to the spirit of the STL: all of the STL
generic algorithms use iterator ranges, instead of whole containers.
Note that this isn't just a convenience, but that it's absolutely
crucial to how the STL is supposed to be used: by passing around
pointers, instead of containers, you can do operations without knowing
or caring just what sort of container you're dealing with.  It's a bit
of a wart that in this one single respect, testing the validity of an
iterator, you do need to pass around the original container.

On a more mundane level, note that the "p != Container.end()" idiom is
just as usable in ordinary C as it is in C++ with STL containers,
since C arrays have off-the-end elements too.  Nonetheless, C
programmers still find null pointers useful.  I don't think I'd like
to write C or C++ code without being allowed to use null pointers.
Generally, STL iterators are supposed to be generalizations of C
pointers; I think it's a pity that this generalization leaves out one
of the most useful aspects of C pointers.  It's not a crippling
limitation, but it is a limitation.

The STL is extremely useful even without null iterators and hash
tables.  I hope, though, that both hash tables and null iterators
do get added to the standard.
--

                               --matt

In article <3igrn7$rlh@druid.borland.com> pete@borland.com (Pete Becker) writes:

   In article <KINNUCAN.95Feb22182657@candide.hq.ileaf.com>,
   kinnucan@hq.ileaf.com says...
   >
   >This is because the use of null pointers to mark the end of
   >single-ended lists (which is what I assume critics of STL
   >have in mind when they talk about the usefulness of null
   >pointers) does not generalize to navigation of other
   >types of data structures. Whereas the pointer incr/decr model
   >works for many (all?) types of containers. Some container libraries
   >(e.g. Borland's) define the incr operator to overfow to a "null"
   >value when it reaches the end of the container. This, however, is
   >contrary to the behavior of the built-in C/C++ incr operator.

    Iterators in BIDS don't try to imitate pointers. They use a
   different model, based on argv. So the BIDS iterators use a 0 to indicate
   the end of the container. As a practical matter, this means that
   iterators typically must be able to examine the container that they point
   into in order to determine when they are done. If I had seen STL before I
   designed those iterators I would probably have adopted STL's model.
    -- Pete

What amazes me about STL is that it actually improves in some respects on Borland's
Class Library, a major achievement in its own right and one from
which I have benefited greatly.

- Paul

In article <KINNUCAN.95Feb22182657@candide.hq.ileaf.com>,
Paul Kinnucan <kinnucan@hq.ileaf.com> wrote:
>Could you give a specific example of where a "null" iterator would
>come in handy? I am really having a hard time seeing what all the fuss is
>about and concrete examples might help me and others to understand why
>this lack of a null iterator is such a big issue with STL critics.

It is useful for returning an error condition (eg. Item not found) instead
of having to return both the pointer/iterator _and_ a flag saying whether
it was sucessful or not.

Most C/C++ functions that return pointers (eg. malloc & new) return NULL
if they failed.  Doing the same for all iterators would be an obvious
extension.

                                        Brian
                                 ( bcwhite@bnr.ca )

-------------------------------------------------------------------------------
    In theory, theory and practice are the same.  In practice, they're not.

dubois@uwo.ca (John Dubois) writes:

>while(X++ != 0)

Even a quick reading of the STL spec (which I admit is all I have
done) will show that an "array" is specified with TWO iterators, one
pointing at the start and one at (just after) the end.  Incrementing
the interator off the end does not turn it into any magic constant, it
turns it into a value equal to this end pointer!

However I do thing a NULL value is very useful for:

>In article <3ii4l7$pre@bcarh8ab.bnr.ca> bcwhite@bnr.ca (Brian White) writes:
>>It is useful for returning an error condition (eg. Item not found) instead
>>of having to return both the pointer/iterator _and_ a flag saying whether
>>it was sucessful or not.

But I am not going to say it is necessary or anything because I have
not used STL.

I would guess supporting NULL in an iterator is a pain because you
must be able to convert '0' to a null so the code "itr=0" works.  I
don't see how you do this without allowing other integers or pointers
to convert.  I don't see any way to solve this except:

Silly Idea:

I would like to see '0' treated as a special built-in type in C++.
You cannot declare variables with type '0', the only way to get an
instance is with an explicit constant of '0'.  The only time you can
use the type '0' is when declaring a function argument:

 // these define two different functions!
 int foo(int a, 0);
 int foo(int a, int b);

There is a built-in cast from '0' to all built-in types.  User-defined
classes may define casts (class X {X(0);...}) to convert from zero.

This is most useful for making "pointer-like" objects that can be
compared to and assigned a value of zero.

It may also have use for static variables if you also add the rules: A
static uninitialized instance of a class with a cast from '0' is
filled as though initialized with '0'.  A class with a default
constructor but no cast from '0' defines a default cast from '0' that
will return the "same thing that a static instance is initialized
with".

There is a default cast of any type to bool which does "x!=0".

Bill Spitzak

In article <3ii4l7$pre@bcarh8ab.bnr.ca> bcwhite@bnr.ca (Brian White) writes:
>From: bcwhite@bnr.ca (Brian White)
>Subject: Re: Proposal for Addition to STL
>Date: 23 Feb 1995 14:04:23 GMT

>In article <KINNUCAN.95Feb22182657@candide.hq.ileaf.com>,
>Paul Kinnucan <kinnucan@hq.ileaf.com> wrote:
>>Could you give a specific example of where a "null" iterator would
>>come in handy? I am really having a hard time seeing what all the fuss is
>>about and concrete examples might help me and others to understand why
>>this lack of a null iterator is such a big issue with STL critics.

>It is useful for returning an error condition (eg. Item not found) instead
>of having to return both the pointer/iterator _and_ a flag saying whether
>it was sucessful or not.

>Most C/C++ functions that return pointers (eg. malloc & new) return NULL
>if they failed.  Doing the same for all iterators would be an obvious
>extension.

>                                        Brian
>                                 ( bcwhite@bnr.ca )

>-------------------------------------------------------------------------------
>    In theory, theory and practice are the same.  In practice, they're not.


Why not just overload the ==(int) & !=(int) for all iterators? that way you
can write:

NiftyIterator X(MyClass);

while(X++ != 0)
 ...

Thus there is no need for null iterator, is there? (doesn't matter if you
return pointers or references, or whatever normally (unless you normally
return int, then even the null iterator would be ambigous..))

John Dubois.

In article q09@hollywood.cinenet.net, spitzak@news.cinenet.net (Bill Spitzak) writes:
> dubois@uwo.ca (John Dubois) writes:
>
> >while(X++ != 0)
>
> Even a quick reading of the STL spec (which I admit is all I have
> done) will show that an "array" is specified with TWO iterators, one
> pointing at the start and one at (just after) the end.  Incrementing
> the interator off the end does not turn it into any magic constant, it
> turns it into a value equal to this end pointer!
>
> However I do thing a NULL value is very useful for:
>
> >In article <3ii4l7$pre@bcarh8ab.bnr.ca> bcwhite@bnr.ca (Brian White) writes:
> >>It is useful for returning an error condition (eg. Item not found) instead
> >>of having to return both the pointer/iterator _and_ a flag saying whether
> >>it was sucessful or not.
>
> But I am not going to say it is necessary or anything because I have
> not used STL.
>
> I would guess supporting NULL in an iterator is a pain because you
> must be able to convert '0' to a null so the code "itr=0" works.  I
> don't see how you do this without allowing other integers or pointers
> to convert.  I don't see any way to solve this except:
>
> Silly Idea:
>
> I would like to see '0' treated as a special built-in type in C++.
> You cannot declare variables with type '0', the only way to get an
> instance is with an explicit constant of '0'.  The only time you can
> use the type '0' is when declaring a function argument:
>
>  // these define two different functions!
>  int foo(int a, 0);
>  int foo(int a, int b);

For smart-pointers and iterators I have defined the following:
common.h:
struct NIL {};
extern NIL nil;
common.cc:
NIL nil;

now I can write:

 if(p == nil) ...;

where the op== is defined as

 bool operator==(const NIL &) const { return fp == 0; }

>
> There is a built-in cast from '0' to all built-in types.  User-defined
> classes may define casts (class X {X(0);...}) to convert from zero.
>
> This is most useful for making "pointer-like" objects that can be
> compared to and assigned a value of zero.
>
> It may also have use for static variables if you also add the rules: A
> static uninitialized instance of a class with a cast from '0' is
> filled as though initialized with '0'.  A class with a default
> constructor but no cast from '0' defines a default cast from '0' that
> will return the "same thing that a static instance is initialized
> with".
>
> There is a default cast of any type to bool which does "x!=0".
>
> Bill Spitzak

In article <3ijvmj$q09@hollywood.cinenet.net>,
Bill Spitzak <spitzak@news.cinenet.net> wrote:
>I would guess supporting NULL in an iterator is a pain because you
>must be able to convert '0' to a null so the code "itr=0" works.  I
>don't see how you do this without allowing other integers or pointers
>to convert.  I don't see any way to solve this except:

The proposal was to create a "null_iterator<Type>" class that would be
the functional equivalent of "NULL".

                                        Brian
                                 ( bcwhite@bnr.ca )

-------------------------------------------------------------------------------
    In theory, theory and practice are the same.  In practice, they're not.

In article <MATT.95Feb23170121@physics2.berkeley.edu> matt@physics2.berkeley.edu (Matt Austern) writes:


   In article <KINNUCAN.95Feb23174429@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:

   > A function that searches an STL data structure can indicate a failure
   > by returning an iterator that points to the end-of-the-container (not the
   > same as the last item in the container), e.g.
   >
   > Suppose Container is an object of a container class derived from
   > an STL container with the addition of a find() member.
   >
   > TItemIterator i = Container.find("foo");
   >
   > if (i != Container.end()) {

   The problem with this is that testing such a result requires that you
   keep around both the iterator and the container to which it belongs:
   i != Container.end() makes no sense unless you still have Container.

If this is a problem in a given application, simply define find() to return
a pointer to an object instead of an iterator.

   This is rather contrary to the spirit of the STL: all of the STL
   generic algorithms use iterator ranges, instead of whole containers.
   Note that this isn't just a convenience, but that it's absolutely
   crucial to how the STL is supposed to be used: by passing around
   pointers, instead of containers, you can do operations without knowing
   or caring just what sort of container you're dealing with.  It's a bit
   of a wart that in this one single respect, testing the validity of an
   iterator, you do need to pass around the original container.

An iterator, by definition, can never by invalid; it can have only
three states: 1) pointing to the beginning of a container, 2) pointing
to the end, or 3) pointing to an item. So there can never by any
circumstance in which an iterator (as an iterator) can naturally (i.e.,
via iterator operations) acquire a null value. Apparently, STL
critics want a null iterator value so that it can be used to indicate
whether an iterator points to an item (as opposed to the beginning
or end of a container). This usage would require that somewhere in
an application program that every iterator be tested after an incr/decr
operation and then assigned to the null value if it points to the
beginning or end of the container. In other words, adding a null
iterator would add an extra step to application code, which defeats
the very reason for adding it in the first place.

If you're concern is to use a single object to convey
the location of an item in memory or the fact that the item could not be
found, then pass a pointer around. If your concern is to operate
on a container, than pass iterators around. Passing an iterator range allows
you to operate safely on any data structure without knowing about the
data structure's type. Trying to reduce the number of iterators that need
to be passed around to one by introducing a null value for iterators
would break the integrity of the STL model. For example, you would
then have to define increment and decrement operators that overflow
to 0 when they reach the bounds of a data structure. This behavior
differs from the behavior of the built-in incr/decr operators, which
is truly contrary to the spirit of STL.


   On a more mundane level, note that the "p != Container.end()" idiom is
   just as usable in ordinary C as it is in C++ with STL containers,
   since C arrays have off-the-end elements too.

Yes, that's the whole point of STL! Any algorithm that uses this idiom
will apply to any type of STL data structure as well as to C/C++
arrays. The same cannot be said for algorithms that depending on using
a null pointer to indicate the end of a data structure. Such algorithms
work for lists, for example, but not for vectors.

   Nonetheless, C
   programmers still find null pointers useful.  I don't think I'd like
   to write C or C++ code without being allowed to use null pointers.
   Generally, STL iterators are supposed to be generalizations of C
   pointers; I think it's a pity that this generalization leaves out one
   of the most useful aspects of C pointers.  It's not a crippling
   limitation, but it is a limitation.

Only if you think of iterators as a replacement for pointers. The fact
is, the advent of STL does not mean the end of pointers. It simply
means that you have to learn when to use pointers
and when iterators. For example, as I pointed out above, it may be
appropriate that a find() function return a pointer to an item
rather than an iterator. There is nothing in STL that says you can't
do this. In this case, you can, as in the past, use a null value
to indicate a failed search. On the other hand, in circumstances where
you once would have passed a pointer to a container around, you can
now pass an iterator range, thereby abstracting your program from
dependency on a particular data structure type.

   The STL is extremely useful even without null iterators and hash
   tables.  I hope, though, that both hash tables and null iterators
   do get added to the standard.

I agree only if a real benefit can be demonstrated by such an
addition. So far I haven't seen a benefit demonstrated.

- Paul

In article <dubois.77.00DED740@uwo.ca>, John Dubois <dubois@uwo.ca> wrote:
>
>Why not just overload the ==(int) & !=(int) for all iterators? that way you
>can write:
>
>NiftyIterator X(MyClass);
>
>while(X++ != 0)
> ...

That takes care of testing for null, but how do you set something to
null?  Overloading assignment and construction from int would be a
dangerous thing to do, as it allows things like:

 NiftyIterator it(3);
 it = 42;

to be written.

--
Mike Lindner
mikel@attmail.com
mpl@cmprime.attpls.com
mpl@pegasus.att.com

>>>>> Jan Reimers <janr@molienergy.bc.ca> writes:

> OK, so how does the copy algorithm, which you imply uses an iterator,
> know when to stop moving bytes???

It takes a start iterator and an end iterator.  Try to read the
documentation before posting questions like this.

Jason

mpl@pegasus.bl-els.att.com (-Michael P. Lindner) writes:
>Some time ago, I began using STL seriously.  In constructing complex
>objects using the STL container classes, and in developing my own
>STL-compliant classes, I believe I have discovered a shortcoming of the
>STL interface requirements.  I found using STL iterators in real
>programs to be more cumbersome and error-prone than using pointers
>because of the lack of a "null" value.  There is no requirement for
>iterators to be constructed with any known value, (other than the copy
>constructor) nor any way to test whether an iterator is dereferenceable.

>I have not experienced this problem in using other container classes'
>iterators, because typically there is a member function to test whether
>the iterator is dereferenceable, and typically the default constructor
>for the iterator ensures that it is not dereferenceable.  Because of the
>intent that pointers be usable as STL iterators, it is not possible to
>provide exactly this functionality.

     Actually, it might be possible to do that.

     First, you'd like to have some functions that test iterators for
validity.  While every iterator is associated with a container, there's
no way to find a container from one of its iterators.  So we won't
be able to write

 if (valid(iter)) { ...}

but will have to have syntax like

 if (contr.valid(iter)) { ... }

(where "contr" is a container and "iter" is an iterator from that container)

So that's what I'd suggest adding: a bool-valued function "valid" for
each container type which returns "true" if the iterator passed to it
is dereferencable.

That's clear enough, and easy to understand and use.  And it's a
straightforward extension, not a modification of existing syntax.
Is it efficiently implementable for all STL containers?  Yes.
Here's how.

      There are two basic cases for STL iterators.  The first is the
pointer-like case where the underlying collection is really an array,
and the second is everything else.

      For the pointer case, the "valid" predicate is simply a bounds
check against the container limits.  (With an STL container, unlike a
raw C/C++ array, you can find out how big it is.)  This is implemented
as a simple inline function with an expression like

 (p >= low && p <= high)

which should generate good fast code.

      For the more general case, a different implementation is required,
but the "valid" function can still be provided.  For non-array collections,
the approach is to have a defined way to recognize singular values,
initialize iterators to some singular value (NULL, or whatever),
require that all the operations put a recognizable singular value
in the iterator when reaching the end or beginning of the collection,
and provide a "valid" test that recognizes singular values.  For
a list collection, for example, we might just use "NULL" as the
singular value, and the "valid" test becomes

 (p != NULL)

which again generates good fast code and can go in an inline function.

      It's the job of the operations on iterators for non-array collections
to insure that any operation on a valid iterator generates either another
valid iterator or a recognizable singular value.

      This would, I think, provide the capabilities of the original
proposal in a simpler way.

     John Nagle

In article <nagleD49GM1.Bqu@netcom.com> nagle@netcom.com (John Nagle) writes:

>       For the pointer case, the "valid" predicate is simply a bounds
> check against the container limits.  (With an STL container, unlike a
> raw C/C++ array, you can find out how big it is.)  This is implemented
> as a simple inline function with an expression like
>
>  (p >= low && p <= high)
>
> which should generate good fast code.

Unfortunately, such code is not guaranteed to work, because if p and
q are pointers then p>=q is undefined unless p and q are pointers into
the same array (or one element beyond the last element of that array).

This infelicity is inherited from C, where it is motivated by a desire
to allow purveyors of compilers for machines with segmented architectures
to optimize away the comparison of segmenent addresses in contexts where
they `know' they will be the same anyway.

C++ cannot remove this restriction on pointer comparison, because doing
so would give it a significant speed disadvantage compared with C on
such machines.  Moreover, such machines and compilers are not exactly rare.
--
    --Andrew Koenig
      ark@research.att.com

In article <3i1vld$js6@highway.LeidenUniv.nl> vosse@ruls41.LeidenUniv.nl (Theo Vosse) writes:


   Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
   : Actually, STL containers do that for you automatically.
   Well, then there is no problem whatsoever, is there?

Not in the case of STL containers.

   : BTW, what do you mean by an "end_of-iterator" object anyway?
   Read the original posting.

I was replying to you. An iterator is a dimensionless object. It
indicates a location in a container. An iterator itself literally
cannot have a beginning or an end. Therefore your demand that
STL provide an "end of iterator" object is as literally meaningless
as it would be to ask that C provide an "end of pointer" object.
Unless, of course, you meant to say "end of container." In this
case, both C and STL provide such an object: it is a pointer (or an
iterator) that points to the location immediately following the
last item in array (container). So in this respect STL iterators
are strictly analogous to standard C pointers.

   :    : The fact that adding some more pointer-like
   :    : functionality to iterators allows one to write even more code in
   :    : a generic fashion should not come as a surprise
   :    Yes, but only so if all iterators have a common end-of-data
   :    indicator, since otherwise even a copy algorithm from an
   :    ordinary T[] to an STL-structure cannot be written, where
   :    T is an arbitrary type.

   : What's your point? It's certainly possible to write copy algorithms
   : from ordinary C arrays to STL containers.

   The thing I don't get, is this: when advocating STL, people say it's
   so easy to use because you can use the pointer syntax, and then go
   on about their generic algorithms. However, this does not enable you
   to write

       void somefun(char s[])
       {
    vector<char> v;

    v = s;
    ...
       }

You can't do this in standard C between character arrays, either, because
C doesn't support dynamic arrays.

   So, it's not so generic after all. So why bother with the pointer
   syntax? Since it doesn't generalize to non-STL types, any syntax will do.

It's generic in the sense that any algorithm you can write using standard
C pointers you can write using STL iterators and vice versa. The
example that you give above does not contradict this inasmuch as it
doesn't use pointers or iterators.

   : It's not irrational to think that negative criticism based
   : on misinformation can retard progress.
   Ok, I overdid some of it. But STL is not progress if you define it as
   the standard. It might be progress if it just another free library,
   which proves useful to the community. STL still contains shortcomings
   (do I need to mention the word 'hash table'?) and, according to the
   original poster, also lacks and end-of-iterator value, so determining
                                   ^^^^^^^^^^^^^^

Again, a meaningless concept.

   this to be the standard is something that stands in the way of progress
   or will do so within a few years. We all know the history of FORTRAN...

   :    3) Everyone who likes it may use STL as much as they want. However, I
   :       dislike having it forced down my throat.
   : Nobody is forced to use STL.

   There's that argument again. Nobody is forced to use the '\0' convention
   either, but we all end up using it. Why bother defining it as a standard
   then?

Because


1) Adoption of a standard utility library will benefit the C++
   community as a whole by reducing duplication of effort.

2) STL is the most advanced C++ utility library in existence
   by almost any measure (efficiency, functionality, theoretical
   underpinnings, framework for growth). In fact,
   aside from any action of the C++
   standards committee, STL would be the standard (in the
   sense of "best of its kind") by which any other potential
   candidate libraries present or future would be measured. Hence,
   it's adoption as "the" standard is only logical.

3) Its adoption as a formal standard allows vendors and users alike
   to move forward confident that their investment in this critical
   area of C++ technology will not be wasted.

- Paul

In article <KINNUCAN.95Feb21213256@candide.hq.ileaf.com> kinnucan@hq.ileaf.com (Paul Kinnucan) writes:

> I was replying to you. An iterator is a dimensionless object. It
> indicates a location in a container. An iterator itself literally
> cannot have a beginning or an end. Therefore your demand that
> STL provide an "end of iterator" object is as literally meaningless
> as it would be to ask that C provide an "end of pointer" object.
> Unless, of course, you meant to say "end of container." In this
> case, both C and STL provide such an object: it is a pointer (or an
> iterator) that points to the location immediately following the
> last item in array (container).

C also provides a pointer that, by definition, cannot point to any
valid object.  C defines pointer semantics so that it is possible
to create pointers with this value and it is possible to compare
any pointer for equality with this value.

Null pointers are very convenient, and most C code that manipulates
pointers uses null pointers in one way or another.  STL iterators
don't have any analogue of null pointers; it would be nice if they
did.

The lack of null iterators (for lack of a better term) isn't a
crippling limitation: it's still possible to do useful work with the
STL even in their absence.  It is annoying, though.
--

                               --matt

-Michael P. Lindner (mpl@pegasus.bl-els.att.com) wrote:
: To all Netters other than Theo Vosse:
Thank you.

: not post further responses to Theo's post to the Net.  Rather, respect
: the time of those readers interested in C++, and send any further
: remarks directly to Theo, as will I.  Thank you.
What, afraid of a little discussion regarding the merits of
the inclusion of STL in the ANSI standard?

: The originators of STL have indeed written programs of more than 10 lines.
Why did they never run into the problem of not having an 'end_of_iterator'
value then? Or do these programs remember which is the highest value or
the number of elements in the container?

: If you had bothered to read my post before replying,
I actually did do that.

: my impetus is to improve the interface to support writing more robust
: code through the use of post conditions, and to simplify certain
Your good intentions have little to do with the usability of STL.
Merely the fact that you can propose a rather serious change in
the interface shows that STL is not the ideal standard library
it should be before inclusion in a standard such as ANSI.
Besides, robustness should be increased by using pre-conditions
in my terminology, but it might mean the same.

: It is not meant to imply that either the originators of
: STL, nor the ANSI committee, nor any of the people on the Net who've
: been using the library for months have been negligent in their analysis
: of STL.
Then why don't they do something about it other than saying 'No, you're
wrong; we won't change it; it's the standard; you don't *have* to use it'?

: STL was not designed to replace pointers with iterators.
No kiddin'...

: It was designed to provide for the use of pointers as iterators.
It hardly matters what you use, as long as you're not completely
implementation independent, does it?
It is not as if it is possible to change the STL declaration
for the storage of a sequence of ints by int[] and still do
the same kind of operations on it (e.g. inserting or whatever).
Therefore, it would not be a problem if the iterator syntax
differed from the way one *might* iterate over an array of ints.

: The fact that adding some more pointer-like
: functionality to iterators allows one to write even more code in
: a generic fashion should not come as a surprise
Yes, but only so if all iterators have a common end-of-data
indicator, since otherwise even a copy algorithm from an
ordinary T[] to an STL-structure cannot be written, where
T is an arbitrary type.

: As to your claims about layers and overhead, you further reveal your
: ignorance.  STL was chosen by ANSI in part because it avoids those
: drawbacks, something which cannot be said about many container
: implementations.
Aha. I do most certainly not encourage the use of any other type
of containter class presently available. Most of these indulge
in inheritance and provide very cumbersome access.

: As to your dislike of pointer syntax, if you have a better proposal, try
: making a positive contribution towards the language and let us all know
: what it is, instead of harping on the useful work of people who actually
: want to advance the state of computer science.

Four points
1) It is irrational to think that only contributions in the form
   of so-called 'positive' proposals can create progress. If you disagree,
   try reading some Popper, Lakatos, Feyerabend, whichever you prefer,
   who all (and all in a different way) stress the importance of
   'negative' results.
2) Everyone is entitled to their opinion. Trying to break off a discussion
   on the basis of a dubious moral and/or authority is very very bad
   ethics.
3) Everyone who likes it may use STL as much as they want. However, I
   dislike having it forced down my throat.
4) Whatever its merits, STL does not 'advance the state of computer
   science'...

BTW, if you don't like my or somebody else's opinion, just don't read it.

--
 Theo Vosse
 ----------
 Unit for Experimental Psychology
 University of Leiden
 The Netherlands

In article <3hssmn$2qr@highway.LeidenUniv.nl> ruiter@ruls41.LeidenUniv.nl (Jan-Peter de Ruiter) writes:
>-Michael P. Lindner (mpl@pegasus.bl-els.att.com) wrote:
>
>
>Well, mr. Vosse has a point, right? The iterators as they are now are
>fine for the abstract algorithms (copy, etc), but if the _user_
>would like to use iterators in a way similar to pointers in C,
>you'd expect and END_OF_ITERATOR test to be present.


OK, so how does the copy algorithm, which you imply uses an iterator,
 know when to stop moving bytes???
Any iterator that doesn't know whem to stop should be renamed as
a "SegmentationFaulter" !!!

>
>I can fully imagine someone wondering what kind of programs
>the developers themselves write using STL, given that this
>functionality is not as yet present in STL.
>
>Actually, I believe you could see mr. Vosse's posting as _encouraging_
>for it underlines the need for having validity tests for iterators
>in STL, which is exactly what you've proposed.
>
>Cheers,
>
>JP


--

################################

        Jan N. Reimers

Paul Kinnucan (kinnucan@hq.ileaf.com) wrote:
: Actually, STL containers do that for you automatically.
Well, then there is no problem whatsoever, is there?

: BTW, what do you mean by an "end_of-iterator" object anyway?
Read the original posting.

:    : The fact that adding some more pointer-like
:    : functionality to iterators allows one to write even more code in
:    : a generic fashion should not come as a surprise
:    Yes, but only so if all iterators have a common end-of-data
:    indicator, since otherwise even a copy algorithm from an
:    ordinary T[] to an STL-structure cannot be written, where
:    T is an arbitrary type.

: What's your point? It's certainly possible to write copy algorithms
: from ordinary C arrays to STL containers.

The thing I don't get, is this: when advocating STL, people say it's
so easy to use because you can use the pointer syntax, and then go
on about their generic algorithms. However, this does not enable you
to write

    void somefun(char s[])
    {
 vector<char> v;

 v = s;
 ...
    }

So, it's not so generic after all. So why bother with the pointer
syntax? Since it doesn't generalize to non-STL types, any syntax will do.

: It's not irrational to think that negative criticism based
: on misinformation can retard progress.
Ok, I overdid some of it. But STL is not progress if you define it as
the standard. It might be progress if it just another free library,
which proves useful to the community. STL still contains shortcomings
(do I need to mention the word 'hash table'?) and, according to the
original poster, also lacks and end-of-iterator value, so determining
this to be the standard is something that stands in the way of progress
or will do so within a few years. We all know the history of FORTRAN...

:    3) Everyone who likes it may use STL as much as they want. However, I
:       dislike having it forced down my throat.
: Nobody is forced to use STL.

There's that argument again. Nobody is forced to use the '\0' convention
either, but we all end up using it. Why bother defining it as a standard
then?

: [stuff deleted]
The 'stuff deleted', might that be the phrase 'If you don't like my
comments, you're not forced to read them'?

--
 Theo Vosse
 ----------
 Unit for Experimental Psychology
 University of Leiden
 The Netherlands

In article <3hqgmg$t3f@bcarh8ab.bnr.ca> bcwhite@bnr.ca (Brian White) writes:

> What about using the !-operator, as many people do?  (i.e.:
>
>  iterator it = some_lookup_function(some_data);
>  if (!it) {
>   // some_data not found
>  } else {
>   // do something
>  }

The problem isn't a syntactic one.  Whether you write it==0 or !it or
it.invalid(), you're asking the same thing: is the iterator invalid?
The STL does not provide any way of testing whether an iterator is
invalid.
--

                               --matt

-Michael P. Lindner (mpl@pegasus.bl-els.att.com) wrote:

: I normally prefer to just let "crackpot" posts lie, so as not to waste
: net bandwidth or my time, but since it was my post which caused Theo to
: make such ridiculous and clearly unthinking statements and broadcast
: his/her ignorance to the entire net world, I shall respond.  Please do
: not post further responses to Theo's post to the Net.  Rather, respect
: the time of those readers interested in C++, and send any further
: remarks directly to Theo, as will I.  Thank you.

Well, mr. Vosse has a point, right? The iterators as they are now are
fine for the abstract algorithms (copy, etc), but if the _user_
would like to use iterators in a way similar to pointers in C,
you'd expect and END_OF_ITERATOR test to be present.

I can fully imagine someone wondering what kind of programs
the developers themselves write using STL, given that this
functionality is not as yet present in STL.

Actually, I believe you could see mr. Vosse's posting as _encouraging_
for it underlines the need for having validity tests for iterators
in STL, which is exactly what you've proposed.

Cheers,

JP

This post contains a long discussion of a proposal I have sent to the
chair of the ANSI C++ committee involved in the Standard Template
Library (STL).  This proposal has been discussed with several people
involved in the STL, with positive results (although I have been
informed that time pressure may prevent it from being considered in the
ANSI proposal.  I am posting it to allow a larger audience to critique
it, and to ensure that should it not be considered by ANSI, at least it
can be considered for STL compliant library vendors' implementations.

Some time ago, I began using STL seriously.  In constructing complex
objects using the STL container classes, and in developing my own
STL-compliant classes, I believe I have discovered a shortcoming of the
STL interface requirements.  I found using STL iterators in real
programs to be more cumbersome and error-prone than using pointers
because of the lack of a "null" value.  There is no requirement for
iterators to be constructed with any known value, (other than the copy
constructor) nor any way to test whether an iterator is dereferenceable.

I have not experienced this problem in using other container classes'
iterators, because typically there is a member function to test whether
the iterator is dereferenceable, and typically the default constructor
for the iterator ensures that it is not dereferenceable.  Because of the
intent that pointers be usable as STL iterators, it is not possible to
provide exactly this functionality.

Can we use a past-the-end value to test if an iterator can be
dereferenced?  Not if we don't know which container's off-the-end value
to test for.  Two examples I've come across where it is difficult
(because of complexity) or impossible (because of encapsulation) to know
what container an iterator refers to are:

1. checking post conditions.

 iterator it;
 // long complicated series of statements with many paths through
 // them that is intended to set "it" to some dereferenceable value
 assert(it != ???); // what can I say about "it"?

    Note that the times when this is hardest to write using off-the-end
    values are the times this assertion is needed most.

    Alex Stepanov proposed that we might amend the STL requirements to
    guarantee that:

 static iterator it;

    produced the same (null valued) iterator, so that code could become

 static iterator null_iterator;
 iterator it(null_iterator);
 // long complicated....
 assert(it != null_iterator);

    However, that is in my mind a less than desirable solution, and
    still requires additions to STL (and changes, as in the default
    constructor for istream_iterator, etc.).

    Without additions to the STL, the only solution I see is something
    like

 container empty;
 iterator it = empty.end();
 // long complicated....
 assert(it != empty.end());

    which requires creating an extra container for each scope in which
    you want to perform a check (and additional code to reset "it" to
    "empty.end()" if an intermediate operation sets it to another
    container's "end()").

2. "Pointer-like" return values.

 iterator it = some_lookup_function(some_data);
 if (it != ???)
  // do something
 else
  // some_data not found

    this can be remedied by replacing the iterator return value of
    some_lookup_function() with a pair<iterator, bool>, where the bool
    member of the pair specifies whether the iterator member is
    dereferenceable, as in

 pair<iterator, bool> p = some_lookup_function(some_data);
 if (p.second)
  // do something
 else
  // some_data not found

    which I guess in some sense could be called superior to the previous
    way, since it explicitly returns a "success/failure" indicator
    instead of relying on an out-of-band value.  However, it does not
    seem as natural to programmers used to C/C++ (which I guess is true
    of many of the STL constructs).

In other words, there is nothing here that _can't_ be done with
iterators the way they are, it's just that the way they are requires
more complicated ways of doing some things which are very simple and
natural if we were using pointers.  Since pointers are STL iterators,
and since STL iterators have the semantics of pointers otherwise, it
seems as useful to have null values for iterators as it is for
pointers.

While I understand that it might be considered unnecessary overhead to
require iterators to be initialized to a null value by default (and
impossible for pointer types), providing a mechanism to construct
iterators with a null value and defining a test for a null value would
be a very useful facility.

My proposed change to the STL portion of the ANSI C++ standard is the
addition of a class to represent null values for pointer types as well
as more complex STL iterators.

New class:

    template<class T>
    class   null_pointer {
    public:
     operator T* () const { return 0; }
     bool operator == (const null_pointer<T>) const { return true; }
     bool operator != (const null_pointer<T>) const { return false; }
    };

    // provide for commutativity
    template<class T, class U>
    inline bool operator == (const null_pointer<T>, const U& t) {
       return t == null_pointer<T>(); }
    template<class T, class U>
    inline bool operator != (const null_pointer<T>, const U& t) {
       return t == null_pointer<T>(); }

Additional requirements for STL iterators:

An iterator which refers to a type T

    1. Must be able to be constructed from a const null_pointer<T>.  The
       value of the resulting iterator is singular (not dereferenceable).

    2. Must have a test for equality/inequality such that an
       iterator constructed from a null_pointer<T> is equal to another
       iterator constructed from a null_pointer<T>, and is not equal to
       any dereferenceable or past the end iterator.

The following requirements do not "require" anything of the
implementation.  They are implied by the previous requirements, but are
stated here explicitly in keeping with the style of the STL iterator
requirements, which state what expressions are legal, rather than how
they are implemented.

    3. Must allow assignment from a const null_pointer<T> or const
       null_pointer<T>&.

       Note that requirement 1, along with the existing requirement for
       assignment fulfills this requirement.  Implementors may choose to
       overload assignment from null_pointer<T> to avoid the overhead of
       an extra constructor call.

    4. Must allow tests for equality/inequality with const
       null_pointer<T> or const null_pointer<T>& such that the
       expression "it == null_pointer<T>()" returns the same value as
       the expression "it == iterator(null_pointer<T>())" (where the
       type of "it" is "iterator").

       Note that requirements 1, and 2 fulfill this requirement.
       Implementors may choose to overload equality/inequality with
       null_pointer<T> to avoid the overhead of an extra constructor
       call.

Rationale for null_pointer<T> class:

    - null_pointer must be a type rather than a value to allow
      overloading of constructors.

    - null_pointer must be a templated class to allow pointers to be
      used with it, as in

     int *ip = null_pointer<int>();

      where there must be a conversion defined from null_pointer<int> to
      int *

    - default constructor, destructor, copy constructor, address of
      operator and assignment operator are all sufficient, since the
      class has no data members.

    - operator == (const null_pointer<T>) and operator != (const
      null_pointer<T>) are provided for cases when an expression reduces
      to

     null_pointer<T> == null_pointer<T>

    - templated functions operator == (const null_pointer<T>, const U& t)
      and operator != (const null_pointer<T>, const U& t) are provided
      for cases such as

     null_pointer<T> == x

      since many programmers are used to writing the expression

     0 == x

      to avoid accidentally using "=" instead of "=="

Rationale for iterator requirements:

    - Construction from null_pointer is provided as separate from the
      default constructor because we cannot change the semantics of the
      default constructor for pointers.

      For iterators like istream_iterator, which have a default
      constructor that already initializes the iterator to a well-known
      value, the issue is moot.  I think having a null_pointer
      constructor even for these iterators fulfills the principle of
      least surprise.

    - operator == and != are defined to be able to determine if an
      iterator's value is equivalent to the null value.

    - conversion of an iterator to bool is not provided, because it
      would make the expressions

     it + 1

      and

     1 + it

      do very different things (the former would express an iterator one
      element beyond "it", the latter would express an integer which is
      "1" or "2" depending on the value of "it").

For example, an iterator might have the following declaration:

      template<class T>
      class   my_container {
      public:
          typedef T   value_type;
          class iterator {
          public:
              iterator(const null_pointer<value_type>);
              // rest of declaration
          };
          // rest of declaration
      };

One can then write:

      my_container<int>::iterator it = null_pointer<int>();
      char* cp = null_pointer<char>();

to explicitly initialize the iterator to the null value.
Likewise, one can test an iterator for equality to the null value:

      if (it == null_pointer<int>())
       // do something
      if (cp == null_pointer<char>())
       // do something else
      assert(it != null_pointer<int>);
      assert(cp != null_pointer<char>);

I believe my proposed solution (the null_pointer class) has the
properties of:

efficiency:  the null_pointer class itself is empty - iterators can be
    written in such a way that it is just used as a type name to select
    the appropriate function from the iterator class.  In the case of
    pointers, instances of null_pointer will simplify to "0".

elegance:  it does not violate the rule of least surprise, and fits into
    the "look and feel" of programming with STL.  Even without looking
    at the class itself, it is easy to guess what "null_pointer<int>()"
    means.

simplicity:  I believe this to be the simplest way to provide a null
    value that will work with both pointers and complex iterators.

--
Mike Lindner
mikel@attmail.com
--
Mike Lindner
mikel@attmail.com
mpl@cmprime.attpls.com
mpl@pegasus.att.com