Expand description
Lexer: converts a pattern string into a flat token stream.
Structs§
- Token
- A lexed token: its kind plus the source span it was read from.
Enums§
- Token
Kind - The kind of a lexed token.
Functions§
- tokenize
- Tokenize a STIX pattern string into a vector of tokens.