+1 vote
in Compilers & Operating Systems by

1 Answer

+1 vote
selected by (user.guest)
Best answer


A token is a pair consisting of a token name and an optional attribute value. The token name is an abstract symbol representing a kind of lexical unit, e.g., a particular keyword, or a sequence of input characters denoting an identifier. The token names are the input symbols that the parser processes. In what follows, we shall generally write the name of a token in boldface. We will often refer to a token by its token name. 

A pattern is a description of the form that the lexemes of a token may take [ or match]. In the case of a keyword as a token, the pattern is just the sequence of characters that form the keyword. For identifiers and some other tokens, the pattern is a more complex structure that is matched by many strings.

A lexeme is a sequence of characters in the source program that matches the pattern for a token and is identified by the lexical analyzer as an instance of that token. 

Example: In the following C language statement ,

printf ("Total = %d\n‖, score) ;

both printf and score are lexemes matching the pattern for token id, and "Total = %d\n‖

is a lexeme matching literal [or string].

Related questions

0 votes
1 answer 608 views
+1 vote
1 answer 508 views
+1 vote
1 answer 279 views
asked May 24, 2021 in Compilers & Operating Systems by anonymous
+1 vote
1 answer 418 views
+3 votes
1 answer 8.8k views
+1 vote
1 answer 280 views
+1 vote
1 answer 156 views
0 votes
1 answer 80 views
+1 vote
1 answer 44 views
+1 vote
1 answer 3.6k views
+1 vote
1 answer 194 views
+1 vote
1 answer 213 views
+1 vote
1 answer 1.2k views
Welcome to CPENTalk.com
Solution-oriented students of computer engineering on one platform to get you that