Package-level declarations

Types

Link copied to clipboard
class CharRangeTokenRecognizer(val detectedCharRange: ClosedRange<Char>) : TokenRecognizer

A token recognizer made for recognizing a ClosedRange of characters.

Link copied to clipboard

Indicates that the state should be changed to the default state

Link copied to clipboard

Indicates that the state should be changed to the state with stateLabel as its label.

Link copied to clipboard
class IgnoreMatchResult(val tokenEndsAt: Int, val nextStateBehavior: NextStateBehavior) : MatcherResult

Indicates that the match was successful, but no token should be created for this match.

Link copied to clipboard
class MatchedTokenResult(val token: Token, val nextStateBehavior: NextStateBehavior) : MatcherResult

Indicates that the match was successful and a token was created.

Link copied to clipboard
sealed class MatcherResult

Subclasses of this class represent the different possible outputs for matchers.

Link copied to clipboard
sealed class NextStateBehavior

Subclasses of this class represent a kind of behavior that can be followed when deciding whether to change state or not, usually after a successful match.

Link copied to clipboard

Indicates that the match was not successful, that there was no match.

Link copied to clipboard

Indicates that the state should not be changed

Link copied to clipboard
class RepeatedRecognizer(val baseRecognizer: TokenRecognizer, val min: Int = 1, val max: Int? = null) : TokenRecognizer

A recognizer that, using another "base" recognizer, will recognize a repetition of the other recognizer.

Link copied to clipboard
class StringRecognizer(val toRecognize: String) : TokenRecognizer

Implementation of a TokenRecognizer that attempts to recognize a given string exactly. It will return a pair with the matched string and the ending index (exclusive) if recognized.

Link copied to clipboard
class StringSetTokenRecognizer(stringsToRecognize: List<String>) : TokenRecognizer

A StringSetTokenRecognizer is a TokenRecognizer specifically built to be able to recognize whether a substring of the input matches at a relatively high speed. The speed improvements are mostly noticeable when the input consists of strings of characters of the same length.

Link copied to clipboard
abstract class TokenMatcher(nextStateBehavior: NextStateBehavior = NoStateChange)

A token matcher is an object that can determine whether a string at a given offset matches some pattern. The matcher then returns a corresponding token or null if no match is found.

Link copied to clipboard
interface TokenRecognizer

A token recognizer has the ability to detect a pattern within a string (the exact pattern being entirely up to the recognizer) and then returns the matched pattern as well as the ending index (that is, the index of the last matched character + 1, this is considered to be an exclusive index).

Link copied to clipboard
class TokenRecognizerIgnored(val recognizer: TokenRecognizer, nextStateBehavior: NextStateBehavior) : TokenMatcher

A type of matcher that ignores anything that matches the recognizer and provides no result otherwise.

Link copied to clipboard
class TokenRecognizerMatched(val recognizer: TokenRecognizer, val tokenType: TokenType, nextStateBehavior: NextStateBehavior = NoStateChange) : TokenMatcher

This class can be used to associate a TokenRecognizer with a token type, forming a complete TokenMatcher

Properties

Link copied to clipboard

Create a recognizer that recognizes the given recognizer or pseudo-recognizer 1 or more times in a row.

Functions

Link copied to clipboard
fun anyOf(vararg s: String): TokenRecognizer

Create a recognizer that recognizes any of the strings provided as parameters.

Link copied to clipboard

Create a recognizer that recognizes the given regular expression. Use this before isToken to create a matcher that matches against a regular expression.

Link copied to clipboard
fun Any.repeated(min: Int = 1, max: Int? = null): RepeatedRecognizer

Create a recognizer that recognizes the given recognizer or pseudo-recognizer min to max (inclusive) times in a row. By default, the (pseudo)-recognizer is recognized from 1 to an infinite amount of times.

Link copied to clipboard

Universal function to turn an object into a recognizer. This function returns a token recognizer.