## mars.venus.editors.jeditsyntax.tokenmarker
Class Token

    java.lang.Object
      mars.venus.editors.jeditsyntax.tokenmarker.Token

    public class Token

extends Object

A linked list of tokens. Each token has three fields - a token identifier, which is a byte value that can be looked up in the array returned by `SyntaxDocument.getColors()` to get a color value, a length value which is the length of the token in the text, and a pointer to the next token in the list.

Author:
  Slava Pestov

Field Summary
`static byte` `COMMENT1` Comment 1 token id.
`static byte` `COMMENT2` Comment 2 token id.
`static byte` `END` The token type, that along with a length of 0 marks the end of the token list.
`byte` `id` The id of this token.
`static byte` `ID_COUNT` The total number of defined token ids.
`static byte` `INTERNAL_FIRST` The first id that can be used for internal state in a token marker.
`static byte` `INTERNAL_LAST` The last id that can be used for internal state in a token marker.
`static byte` `INVALID` Invalid token id.
`static byte` `KEYWORD1` Keyword 1 token id.
`static byte` `KEYWORD2` Keyword 2 token id.
`static byte` `KEYWORD3` Keyword 3 token id.
`static byte` `LABEL` Label token id.
`int` `length` The length of this token.
`static byte` `LITERAL1` Literal 1 token id.
`static byte` `LITERAL2` Literal 2 token id.
`static byte` `MACRO_ARG` Macro parameter token.
`Token` `next` The next token in the linked list.
`static byte` `NULL` Normal text token id.
`static byte` `OPERATOR` Operator token id.

Constructor Summary
`Token(int length, byte id)` Creates a new token.

Method Summary
`String` `toString()` Returns a string representation of this token.

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

### NULL

    public static final byte NULL

  Normal text token id. This should be used to mark normal text.

  See Also:
    Constant Field Values

### COMMENT1

    public static final byte COMMENT1

  Comment 1 token id. This can be used to mark a comment.

  See Also:
    Constant Field Values

### COMMENT2

    public static final byte COMMENT2

  Comment 2 token id. This can be used to mark a comment.

  See Also:
    Constant Field Values

### LITERAL1

    public static final byte LITERAL1

  Literal 1 token id. This can be used to mark a string literal (eg, C mode uses this to mark "..." literals)

  See Also:
    Constant Field Values

### LITERAL2

    public static final byte LITERAL2

  Literal 2 token id. This can be used to mark an object literal (eg, Java mode uses this to mark true, false, etc)

  See Also:
    Constant Field Values

### LABEL

    public static final byte LABEL

  Label token id. This can be used to mark labels (eg, C mode uses this to mark ...: sequences)

  See Also:
    Constant Field Values

### KEYWORD1

    public static final byte KEYWORD1

  Keyword 1 token id. This can be used to mark a keyword. This should be used for general language constructs.

  See Also:
    Constant Field Values

### KEYWORD2

    public static final byte KEYWORD2

  Keyword 2 token id. This can be used to mark a keyword. This should be used for preprocessor commands, or variables.

  See Also:
    Constant Field Values

### KEYWORD3

    public static final byte KEYWORD3

  Keyword 3 token id. This can be used to mark a keyword. This should be used for data types.

  See Also:
    Constant Field Values

### OPERATOR

    public static final byte OPERATOR

  Operator token id. This can be used to mark an operator. (eg, SQL mode marks +, -, etc with this token type)

  See Also:
    Constant Field Values

### INVALID

    public static final byte INVALID

  Invalid token id. This can be used to mark invalid or incomplete tokens, so the user can easily spot syntax errors.

  See Also:
    Constant Field Values

### MACRO_ARG

    public static final byte MACRO_ARG

  Macro parameter token. Added for MARS 4.3.

  See Also:
    Constant Field Values

### ID_COUNT

    public static final byte ID_COUNT

  The total number of defined token ids.

  See Also:
    Constant Field Values

### INTERNAL_FIRST

    public static final byte INTERNAL_FIRST

  The first id that can be used for internal state in a token marker.

  See Also:
    Constant Field Values

### INTERNAL_LAST

    public static final byte INTERNAL_LAST

  The last id that can be used for internal state in a token marker.

  See Also:
    Constant Field Values

### END

    public static final byte END

  The token type, that along with a length of 0 marks the end of the token list.

  See Also:
    Constant Field Values

### length

    public int length

  The length of this token.

### id

    public byte id

  The id of this token.

### next

    public Token next

  The next token in the linked list.

Constructor Detail

### Token

    public Token(int length,
                 byte id)

  Creates a new token.

  Parameters:
    `length` - The length of the token
    `id` - The id of the token

Method Detail

### toString

    public String toString()

  Returns a string representation of this token.

  Overrides:
    `toString` in class `Object`