In article <941lbe$l1g$1@nnrp1.deja.com>,
  wmm@fastdial.net wrote:
> In article <93nel8$md1$1@nnrp1.deja.com>,
>   Jim Hyslop <jim.hyslop@leitch.com> wrote:
[snip]
> > Right, so if the first sentence [of 2.1p1 phase 3] is a summary of >
> the results, then
> > why
> > have the whole phrase "white space characters (including comments)"?
> > Why
> > not just state the desired output, i.e. "white space"?
>
> Because that's not the result of phase 3.  "White space" may
> contain comments, and comments are explicitly replaced by space
> characters as part of the processing of phase 3.  The only things
> left are preprocessing tokens and white-space characters.
Yes, of course, I should have realized that.

You've convinced me - I withdraw my defect report if I am allowed to do
so.

--
Jim
To suppress Deja's product links, add this header:
x-no-productlinks: yes


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <949o0f$ip6$1@nnrp1.deja.com>,
  Jim Hyslop <jim.hyslop@leitch.com> wrote:
> In article <941lbe$l1g$1@nnrp1.deja.com>,
>   wmm@fastdial.net wrote:
> > In article <93nel8$md1$1@nnrp1.deja.com>,
> >   Jim Hyslop <jim.hyslop@leitch.com> wrote:
> [snip]
> > > Right, so if the first sentence [of 2.1p1 phase 3] is a summary
of >
> > the results, then
> > > why
> > > have the whole phrase "white space characters (including
comments)"?
> > > Why
> > > not just state the desired output, i.e. "white space"?
> >
> > Because that's not the result of phase 3.  "White space" may
> > contain comments, and comments are explicitly replaced by space
> > characters as part of the processing of phase 3.  The only things
> > left are preprocessing tokens and white-space characters.
> Yes, of course, I should have realized that.
>
> You've convinced me - I withdraw my defect report if I am allowed to
do
> so.

No problem -- and thanks!  We always have more to do at the
meetings than time to do it, so anything that turns out not to
be an issue after all is appreciated.

--
William M. Miller, wmm@fastdial.net
Vignette Corporation (www.vignette.com)


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <93ka7d$tkl$1@nnrp1.deja.com>,
  wmm@fastdial.net wrote:
> In article <93d3cl$2i7$1@nnrp1.deja.com>,
>   Jim Hyslop <jim.hyslop@leitch.com> wrote:
> > The problem I had was the phrase "sequence of white space characters
> > (including comments)" in [lex.phases]. The use of that phrase
> > implied
> > that "white space" did not necessarily mean "sequence of white space
> > characters (including comments)".
If I may elaborate on this slightly (in the event my reasoning here is
not clear)...

My reasoning when I read that phrase (and in subsequent conversations on
other email lists I have come across others who apparently followed the
same reasoning as I) was:

16.3.2 [cpp.stringize] just says "white space" which I know is defined
elsewhere. 2.1 [lex.phases] para 3 says "sequence of white space
characters (including comments)". Hmmm... this isn't the definition, and
I know the definition appears elsewhere, so there must be a reason 2.1
explicitly states what it means, therefore it must mean something other
than just generic "white space".

I know, that is faulty reasoning because I did not look up the
definition of white space :-)

[snip]
> I'm still not personally convinced that a change is required.
> The first sentence of the phase 3 description in 2.1p1 is a
> summary of the actions performed in phase 3, with elaboration
> and clarification in the following text.  The reason that
> "(including comments)" belongs with "sequences of white-space
> characters" is precisely because "Each comment is replaced by
> one space character" -- the result of phase 3 is that there
> are _only_ white-space characters and preprocessing tokens.
Right, so if the first sentence is a summary of the results, then why
have the whole phrase "white space characters (including comments)"? Why
not just state the desired output, i.e. "white space"?

> Both "preprocessing tokens" and "white-space characters" are
> forward references, and there's an explicit cross-reference to
> the place where they are defined (2.4).
Ah, yes, of course - I have an unfortunate tendency to gloss over
cross-references in my attempts to absorb the meaning of the sentence.

> The Standard isn't
> required to define every concept the first place it occurs,
> but we do try to have cross-references to the definition when
> a forward reference is used.  I think it's more appropriate that
> the definition of white-space character be in the section that
> actually deals with preprocessing lexical analysis, 2.4.
I can understand that, and it sounds reasonable to me. As I said earlier
my main source of confusion was the use of the entire phrase (not to
mention that the repetition is unnecessarily redundant [sic]).

OK, here is my modified proposal, then:

Given that the first sentence in 2.1 [lex.phases] paragraph 3, is
intended to be a summary of the results of phase 3, and given that the
repetition of the definition of "white space" in this sentence can be
misleading and is certainly redundant, I suggest that the first sentence
in 2.1 [lex.phases] paragraph 3 be reworded to read:

"The source file is decomposed into preprocessing tokens (2.4) and white
space."

--
Jim
To suppress Deja's product links, add this header:
x-no-productlinks: yes


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <93nel8$md1$1@nnrp1.deja.com>,
  Jim Hyslop <jim.hyslop@leitch.com> wrote:
> In article <93ka7d$tkl$1@nnrp1.deja.com>,
>   wmm@fastdial.net wrote:
> > I'm still not personally convinced that a change is required.
> > The first sentence of the phase 3 description in 2.1p1 is a
> > summary of the actions performed in phase 3, with elaboration
> > and clarification in the following text.  The reason that
> > "(including comments)" belongs with "sequences of white-space
> > characters" is precisely because "Each comment is replaced by
> > one space character" -- the result of phase 3 is that there
> > are _only_ white-space characters and preprocessing tokens.
> Right, so if the first sentence is a summary of the results, then why
> have the whole phrase "white space characters (including comments)"?
Why
> not just state the desired output, i.e. "white space"?

Because that's not the result of phase 3.  "White space" may
contain comments, and comments are explicitly replaced by space
characters as part of the processing of phase 3.  The only things
left are preprocessing tokens and white-space characters.

The problem (to the extent that there is one) lies in the fact
that 16.3.1, which is part of phase 4, describes how "white
space" is handled, when in fact only the processing for white-
space characters need be specified.  It's not inaccurate,
since white-space characters are a subset of "white space."  In
fact, I'm not convinced that recasting 16.3.1 in terms of white-
space characters, in the interests of precision and minimal
specification, would actually be an improvement -- the result
would be the same even if comments were not removed in phase 3.
That means that describing the processing in terms of "white
space" makes 16.3.1 more stand-alone, that is, clear even
without reference to the phases of translation.  Reducing the
amount of far-away context required to understand a given section
is a good thing, IMHO.

(I would point out in passing that some of the processing in
phase 4 does actually depend on the removal of comments in phase
3.  See, for instance, the wording in 16p2 and 16.3p6, which
deals only with white-space characters, on the assumption that
there are no more comments.)

--
William M. Miller, wmm@fastdial.net
Vignette Corporation (www.vignette.com)


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

Jim Hyslop wrote:
>
> In article <934lu4$hb1$1@nnrp1.deja.com>,
>   wmm@fastdial.net wrote:
[snip]
> > I'll add this to the core language issues list.  However,
> > speaking personally, I have a very hard time seeing any
> > justification for the "each whitespace character"
> > interpretation.
> The problem I had was the phrase "sequence of white space characters
> (including comments)" in [lex.phases]. The use of that phrase implied
> that "white space" did not necessarily mean "sequence of white space
> characters (including comments)".

I see a difference between "white space" meaning a non-empty sequence of
white space characters and "sequence of white space characters
(INCLUDING COMMENTS)" (emphasis is mine). As I read it, the second
phrase means a sequence of white space characters and possibly
interspersed comments. "White space" surely excludes comments.

Regards,
Heinz

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <937gqr$q6a$1@nnrp1.deja.com>,
  Jim Hyslop <jim.hyslop@leitch.com> wrote:
> In article
> <E00FFF0572D3D21185C70008C7B2BDC40865ACC4@stork.mars.leitch.com>,
>   "Jim.Hyslop" <Jim.Hyslop@Leitch.com> wrote:
> [snip]
> > Does the phrase "Each occurrence of white space" mean "Each
whitespace
> > character", or does it mean "Each nonempty sequence of whitespace
> > characters" (the phrase used in 2.1 [lex.phases] for phase 3)?
> By the way, what *is* the intent of that phrase? Anyone care to
comment?

Our posts crossed, so I've already expressed my opinion.  It
might be clearer, however, if I give my paraphrase of the
two sentences (the one you are concerned about and the one
following it that I mentioned in my other post), because I do
think the relationship between them is important.

The existing wording is:

    Each occurrence of white space between the argument   s
    preprocessing tokens becomes a single space character in
    the character string literal. White space before the first
    preprocessing token and after the last preprocessing token
    comprising the argument is deleted.

My paraphrase:

    White space in the argument is normalized in the resulting
    character string literal.  White space at the beginning and
    end of the argument does not appear in the result; white
    space between preprocessing tokens is represented by a
    single space character in the result.

I think this interpretation is supported by 16.3p1 ("all
white-space separations are considered identical" in determining
whether a redefinition is benign).  For instance,

    #define X a b c
    #define X a /**/ b /**/ c

is well-formed.  The implication of allowing this as a benign
redefinition is, I think, that using X as an argument to the
# operator will produce the same string, regardless of which
definition is in force at the time.  That is, if you could use
stringizing to tell the difference between the definitions, I
don't think 16.3p1 would have said that "all white-space
separations are considered identical."

--
William M. Miller, wmm@fastdial.net
Vignette Corporation (www.vignette.com)


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <93fuep$9tv$1@nnrp1.deja.com>, wmm@fastdial.net writes
>    White space in the argument is normalized in the resulting
>    character string literal.  White space at the beginning and
>    end of the argument does not appear in the result;

I think we need 'contiguous' here

>white
>    space between preprocessing tokens is represented by a
>    single space character in the result.

Francis Glassborow      Association of C & C++ Users
64 Southfield Rd
Oxford OX4 1PA          +44(0)1865 246490
All opinions are mine and do not represent those of any organisation

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <3A5AD1E2.5414D09F@elbanet.co.at>,
  Heinz Huber <Heinz.Huber@elbanet.co.at> wrote:
>
>
> Jim Hyslop wrote:
> >
> > In article <934lu4$hb1$1@nnrp1.deja.com>,
> >   wmm@fastdial.net wrote:
> [snip]
> > > I'll add this to the core language issues list.  However,
> > > speaking personally, I have a very hard time seeing any
> > > justification for the "each whitespace character"
> > > interpretation.
> > The problem I had was the phrase "sequence of white space characters
> > (including comments)" in [lex.phases]. The use of that phrase
implied
> > that "white space" did not necessarily mean "sequence of white space
> > characters (including comments)".
>
> I see a difference between "white space" meaning a non-empty sequence
> of
> white space characters and "sequence of white space characters
> (INCLUDING COMMENTS)" (emphasis is mine).
No, the definition of white space in 2.4/2 explicitly includes comments,
so as William Miller pointed out, the phrase "sequence of white space
characters (including comments)" is identical to "white space".

> As I read it, the second
> phrase means a sequence of white space characters and possibly
> interspersed comments. "White space" surely excludes comments.
Phase 3 of translation converts comments into white space.

--
Jim
To suppress Deja's product links, add this header:
x-no-productlinks: yes


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <93d3cl$2i7$1@nnrp1.deja.com>,
  Jim Hyslop <jim.hyslop@leitch.com> wrote:
> The problem I had was the phrase "sequence of white space characters
> (including comments)" in [lex.phases]. The use of that phrase implied
> that "white space" did not necessarily mean "sequence of white space
> characters (including comments)".
>
> It seems to me now that [lex.phases] is the inconsistent clause, and
> perhaps it should be re-worded instead, to something like:
>
> "The source file is decomposed into preprocessing tokens (2.4) and
white
> space." and either add a new clause in 1.3 for a definition of "white
> space", or move the definition in 2.4/2 to 1.3. That is:
>
> [lex.phases] (2.1) paragraph 3 becomes (using pseudo-HTML to indicate
> font changes):
> "The source file is decomposed into preprocessing tokens (2.4) and
> <ITAL>white space</ITAL>; this consists of comments (2.7), or
> white   space characters (space, horizontal tab, new   line, vertical tab,
> and form   feed),or both."
>
> [lex.pptoken] (2.4) paragraph 2 becomes, in part:
> "... Preprocessing tokens can be separated by white space. As
described
> in clause 16 ... "
>
> Is it possible to amend a DR once it has been submitted?

Sure.  This is not an automated system, and I often include
subsequent discussion in the writeup of an issue that is
submitted via this newsgroup.

I'm still not personally convinced that a change is required.
The first sentence of the phase 3 description in 2.1p1 is a
summary of the actions performed in phase 3, with elaboration
and clarification in the following text.  The reason that
"(including comments)" belongs with "sequences of white-space
characters" is precisely because "Each comment is replaced by
one space character" -- the result of phase 3 is that there
are _only_ white-space characters and preprocessing tokens.
Both "preprocessing tokens" and "white-space characters" are
forward references, and there's an explicit cross-reference to
the place where they are defined (2.4).  The Standard isn't
required to define every concept the first place it occurs,
but we do try to have cross-references to the definition when
a forward reference is used.  I think it's more appropriate that
the definition of white-space character be in the section that
actually deals with preprocessing lexical analysis, 2.4.

However, I acknowledge that this is a point where there can be
legitimate difference of opinion and will write up the issue
accordingly.

--
William M. Miller, wmm@fastdial.net
Vignette Corporation (www.vignette.com)


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

 [Moderator's note: this defect report has been
 forwarded to the C++ committee. -moderator(fjh).]

[cpp.stringize], 16.3.2 paragraph 2 reads, in part:

"Each occurrence of white space between the argument's preprocessing
 tokens becomes a single space character in the character string
 literal."

Does the phrase "Each occurrence of white space" mean "Each whitespace
character", or does it mean "Each nonempty sequence of whitespace
characters" (the phrase used in 2.1 [lex.phases] for phase 3)?

I recommend that 16.3.2 [cpp.stringize] be re-worded to one of the two
phrases presented
above, to remain consistent with other parts of the standard (i.e. 2.1
[lex.phases]).

Jim
---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]

In article
<E00FFF0572D3D21185C70008C7B2BDC40865ACC4@stork.mars.leitch.com>,
  "Jim.Hyslop" <Jim.Hyslop@Leitch.com> wrote:
>  [Moderator's note: this defect report has been
>  forwarded to the C++ committee. -moderator(fjh).]
>
> [cpp.stringize], 16.3.2 paragraph 2 reads, in part:
>
> "Each occurrence of white space between the argument's preprocessing
>  tokens becomes a single space character in the character string
>  literal."
>
> Does the phrase "Each occurrence of white space" mean "Each whitespace
> character", or does it mean "Each nonempty sequence of whitespace
> characters" (the phrase used in 2.1 [lex.phases] for phase 3)?
>
> I recommend that 16.3.2 [cpp.stringize] be re-worded to one of the two
> phrases presented
> above, to remain consistent with other parts of the standard (i.e. 2.1
> [lex.phases]).

I'll add this to the core language issues list.  However,
speaking personally, I have a very hard time seeing any
justification for the "each whitespace character"
interpretation.  As far as I can tell, every occurrence
of the phrase "white space" (as opposed to "white space
character(s)") in the Standard means "a contiguous sequence
of white-space characters and/or comments."  See, for
instance,

2.4p2:

    Preprocessing tokens can be separated by white space;
    this consists of comments (2.7), or white-space
    characters (space, horizontal tab, new-line, vertical
    tab, and form-feed), or both.

16.3p1:

    Two replacement lists are identical if and only if the
    preprocessing tokens in both have the same number,
    ordering, spelling, and white-space separation, where
    all white-space separations are considered identical.

16.3.2p2:

    White space before the first preprocessing token and
    after the last preprocessing token comprising the
    argument is deleted.

I find the last quote particularly convincing, since it is
the very next sentence after the one you suggest changing.

Whenever the Standard intends "white space character," it
says so explicitly.
--
William M. Miller, wmm@fastdial.net
Vignette Corporation (www.vignette.com)


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article
<E00FFF0572D3D21185C70008C7B2BDC40865ACC4@stork.mars.leitch.com>,
  "Jim.Hyslop" <Jim.Hyslop@Leitch.com> wrote:
[snip]
> Does the phrase "Each occurrence of white space" mean "Each whitespace
> character", or does it mean "Each nonempty sequence of whitespace
> characters" (the phrase used in 2.1 [lex.phases] for phase 3)?
By the way, what *is* the intent of that phrase? Anyone care to comment?

--
Jim
To suppress Deja's product links, add this header:
x-noproductlinks: yes


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <934lu4$hb1$1@nnrp1.deja.com>,
  wmm@fastdial.net wrote:
> In article
> <E00FFF0572D3D21185C70008C7B2BDC40865ACC4@stork.mars.leitch.com>,
>   "Jim.Hyslop" <Jim.Hyslop@Leitch.com> wrote:
> >  [Moderator's note: this defect report has been
> >  forwarded to the C++ committee. -moderator(fjh).]
> >
> > [cpp.stringize], 16.3.2 paragraph 2 reads, in part:
> >
> > "Each occurrence of white space between the argument's preprocessing
> >  tokens becomes a single space character in the character string
> >  literal."
> >
> > Does the phrase "Each occurrence of white space" mean "Each
whitespace
> > character", or does it mean "Each nonempty sequence of whitespace
> > characters" (the phrase used in 2.1 [lex.phases] for phase 3)?
> >
> > I recommend that 16.3.2 [cpp.stringize] be re-worded to one of the
two
> > phrases presented
> > above, to remain consistent with other parts of the standard (i.e.
2.1
> > [lex.phases]).
>
> I'll add this to the core language issues list.  However,
> speaking personally, I have a very hard time seeing any
> justification for the "each whitespace character"
> interpretation.
The problem I had was the phrase "sequence of white space characters
(including comments)" in [lex.phases]. The use of that phrase implied
that "white space" did not necessarily mean "sequence of white space
characters (including comments)".

[snip]
> 2.4p2:
>
>     Preprocessing tokens can be separated by white space;
>     this consists of comments (2.7), or white-space
>     characters (space, horizontal tab, new-line, vertical
>     tab, and form-feed), or both.
Actually, this is the most compelling quote for me, since the term
"white-space" is in italics, which indicates that its definition
immediately follows.
[snip]
> Whenever the Standard intends "white space character," it
> says so explicitly.
It seems to me now that [lex.phases] is the inconsistent clause, and
perhaps it should be re-worded instead, to something like:

"The source file is decomposed into preprocessing tokens (2.4) and white
space." and either add a new clause in 1.3 for a definition of "white
space", or move the definition in 2.4/2 to 1.3. That is:

[lex.phases] (2.1) paragraph 3 becomes (using pseudo-HTML to indicate
font changes):
"The source file is decomposed into preprocessing tokens (2.4) and
<ITAL>white space</ITAL>; this consists of comments (2.7), or
white   space characters (space, horizontal tab, new   line, vertical tab,
and form   feed),or both."

[lex.pptoken] (2.4) paragraph 2 becomes, in part:
"... Preprocessing tokens can be separated by white space. As described
in clause 16 ... "

Is it possible to amend a DR once it has been submitted?


--
Jim
To suppress Deja's product links, add this header:
x-no-productlinks: yes


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]