Add lexer test runner and whitespace characters tests
Whitespace characters for the tests are U+000B, U+000C, U+000D, U+0085, U+2028, U+2029
This commit is contained in:
+93
@@ -0,0 +1,93 @@
|
||||
val ('val')
|
||||
WHITE_SPACE (' ')
|
||||
IDENTIFIER ('x1')
|
||||
WHITE_SPACE (' ')
|
||||
EQ ('=')
|
||||
WHITE_SPACE (' ')
|
||||
OPEN_QUOTE ('"')
|
||||
REGULAR_STRING_PART ('.
.')
|
||||
CLOSING_QUOTE ('"')
|
||||
WHITE_SPACE ('\n\n')
|
||||
val ('val')
|
||||
WHITE_SPACE (' ')
|
||||
IDENTIFIER ('x2')
|
||||
WHITE_SPACE (' ')
|
||||
EQ ('=')
|
||||
WHITE_SPACE (' ')
|
||||
OPEN_QUOTE ('"')
|
||||
REGULAR_STRING_PART ('. .
')
|
||||
CLOSING_QUOTE ('"')
|
||||
WHITE_SPACE ('\n\n')
|
||||
val ('val')
|
||||
WHITE_SPACE (' ')
|
||||
IDENTIFIER ('x3')
|
||||
WHITE_SPACE (' ')
|
||||
EQ ('=')
|
||||
WHITE_SPACE (' ')
|
||||
OPEN_QUOTE ('"')
|
||||
REGULAR_STRING_PART ('
. .')
|
||||
CLOSING_QUOTE ('"')
|
||||
WHITE_SPACE ('\n\n')
|
||||
val ('val')
|
||||
WHITE_SPACE (' ')
|
||||
IDENTIFIER ('x4')
|
||||
WHITE_SPACE (' ')
|
||||
EQ ('=')
|
||||
WHITE_SPACE (' ')
|
||||
CHARACTER_LITERAL (''
'')
|
||||
WHITE_SPACE ('\n\n')
|
||||
val ('val')
|
||||
WHITE_SPACE (' ')
|
||||
IDENTIFIER ('x5')
|
||||
WHITE_SPACE (' ')
|
||||
EQ ('=')
|
||||
WHITE_SPACE (' ')
|
||||
OPEN_QUOTE ('"""')
|
||||
REGULAR_STRING_PART ('.
.')
|
||||
CLOSING_QUOTE ('"""')
|
||||
WHITE_SPACE ('\n\n')
|
||||
val ('val')
|
||||
WHITE_SPACE (' ')
|
||||
IDENTIFIER ('x6')
|
||||
WHITE_SPACE (' ')
|
||||
EQ ('=')
|
||||
WHITE_SPACE (' ')
|
||||
OPEN_QUOTE ('"""')
|
||||
REGULAR_STRING_PART ('. .
')
|
||||
CLOSING_QUOTE ('"""')
|
||||
WHITE_SPACE ('\n\n')
|
||||
val ('val')
|
||||
WHITE_SPACE (' ')
|
||||
IDENTIFIER ('x7')
|
||||
WHITE_SPACE (' ')
|
||||
EQ ('=')
|
||||
WHITE_SPACE (' ')
|
||||
OPEN_QUOTE ('"""')
|
||||
REGULAR_STRING_PART ('
. .')
|
||||
CLOSING_QUOTE ('"""')
|
||||
WHITE_SPACE ('\n\n')
|
||||
val ('val')
|
||||
WHITE_SPACE (' ')
|
||||
IDENTIFIER ('x8')
|
||||
WHITE_SPACE (' ')
|
||||
EQ ('=')
|
||||
WHITE_SPACE (' ')
|
||||
OPEN_QUOTE ('"""')
|
||||
REGULAR_STRING_PART ('.')
|
||||
REGULAR_STRING_PART ('\n')
|
||||
REGULAR_STRING_PART ('
.')
|
||||
CLOSING_QUOTE ('"""')
|
||||
WHITE_SPACE ('\n\n')
|
||||
val ('val')
|
||||
WHITE_SPACE (' ')
|
||||
IDENTIFIER ('x9')
|
||||
WHITE_SPACE (' ')
|
||||
EQ ('=')
|
||||
WHITE_SPACE (' ')
|
||||
OPEN_QUOTE ('"""')
|
||||
REGULAR_STRING_PART ('.')
|
||||
REGULAR_STRING_PART ('\n')
|
||||
REGULAR_STRING_PART ('
')
|
||||
REGULAR_STRING_PART ('\n')
|
||||
REGULAR_STRING_PART ('.')
|
||||
CLOSING_QUOTE ('"""')
|
||||
Reference in New Issue
Block a user