Topic: tokens and preprocessing


Author: gennaro_prota@my-deja.com
Date: Wed, 7 Feb 2001 15:51:44 GMT
Raw View
On Wed,  7 Feb 2001 01:31:24 GMT, wmm@fastdial.net wrote:

>In article <95pkbj$m8s$1@nnrp1.deja.com>,
>  gennaro_prota@my-deja.com wrote:
>>
>>  if I write
>>    #define SIZE -1
>>
>>  what happens when writing, for instance
>>
>>   int i = -SIZE;
>>
>> ?
>>
>> My opinion is that, since tokenization occurs during translation
phase
>> 3,  when macro invocations are expanded (phase 4) -1 has already been
>> parsed as a single preprocessing token (pp-number).
>
>You are correct that the tokenization in phase 3 determines the
>answer, but you don't have the correct definition of the tokens.
>A pp-number (defined in 2.9) does _not_ have a sign operator.
>Thus -1 is _two_ preprocessing tokens -- the "-" becomes a
>preprocessing-op-or-punc (2.12) and "1" becomes a pp-number
>(integer-literal).

Thanks a lot for the clarification! I read 2.9 but I was not careful
enough about the sign: I thought "-1" was a single token. Shame on
me !:-(

>
>> So
>>
>>    int i = -SIZE;
>>
>> yelds
>>
>>  int i = - -1;
>>
>>  (note the space between the two minus signs).
>
>That's the effect; however, a standard-conforming preprocessor
>produces preprocessing tokens, not streams of characters, so
>the space character is just illustrative -- one way you could
>write the source to get the same effect as the macro expansion.

Yes, I know that. I used the space character just to highlight we had
two separate tokens. (I should have said "has the same effect of",
instead of "yields").

Thanks again! :-)))


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]





Author: Jim Hyslop <jim.hyslop@leitch.com>
Date: Wed, 7 Feb 2001 20:55:22 GMT
Raw View
In article <3A805FFB.3097D0C6@acm.org>,
  James Dennett <jdennett@acm.org> wrote:
> gennaro_prota@my-deja.com wrote:
[snip]
> > Anyway, if I write
> >
> >   int i = 1 SIZE;
> >
> > I should get a syntax error (two consecutive numbers,
> > as if I wrote int a = 1 1).
> >
> > This time, compilers don't agree.
>
> There might be a typo in your post?
I think (if I may be so bold as to interpret gennaro_prota's words) that
the point was based on an incorrect assumption that "-1" tokenized to a
single preprocessing token representing a number. Therefore, according
to the (incorrect) reasoning,

int i = 1-1;

gets tokenized as

TOKEN(int) TOKEN(i) TOKEN(=)
TOKEN(number<value: 1>) TOKEN(number<value: -1>) TOKEN(semicolon)

What the OP failed to take into account, however, is whitespace - to a
compiler "1-1" is identical to "1 - 1", so

int j = 1 - 1;

would be tokenized as

TOKEN(int) TOKEN(j) TOKEN(=)
TOKEN(number<value: 1>) TOKEN(minus) TOKEN(number<value: 1>)
TOKEN(semicolon)

which is clearly not the same sequence of tokens.

--
Jim
To suppress Deja's product links, add this header:
x-no-productlinks: yes


Sent via Deja.com
http://www.deja.com/

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]