Parsing and lexing

はじめに
インストール/設定
- 要件
- インストール手順
定義済み定数
Pattern matching — Parle pattern matching
例
- Lexer examples
- Parser examples
Parle\Lexer — The Parle\Lexer class
- Parle\Lexer::advance — Process next lexer rule
- Parle\Lexer::build — Finalize the lexer rule set
- Parle\Lexer::callout — Define token callback
- Parle\Lexer::consume — Pass the data for processing
- Parle\Lexer::dump — Dump the state machine
- Parle\Lexer::getToken — Retrieve the current token
- Parle\Lexer::insertMacro — Insert regex macro
- Parle\Lexer::push — Add a lexer rule
- Parle\Lexer::reset — Reset lexer
Parle\RLexer — The Parle\RLexer class
- Parle\RLexer::advance — Process next lexer rule
- Parle\RLexer::build — Finalize the lexer rule set
- Parle\RLexer::callout — Define token callback
- Parle\RLexer::consume — Pass the data for processing
- Parle\RLexer::dump — Dump the state machine
- Parle\RLexer::getToken — Retrieve the current token
- Parle\RLexer::insertMacro — Insert regex macro
- Parle\RLexer::push — Add a lexer rule
- Parle\RLexer::pushState — Push a new start state
- Parle\RLexer::reset — Reset lexer
Parle\Parser — The Parle\Parser class
- Parle\Parser::advance — Process next parser rule
- Parle\Parser::build — Finalize the grammar rules
- Parle\Parser::consume — Consume the data for processing
- Parle\Parser::dump — Dump the grammar
- Parle\Parser::errorInfo — Retrieve the error information
- Parle\Parser::left — Declare a token with left-associativity
- Parle\Parser::nonassoc — Declare a token with no associativity
- Parle\Parser::precedence — Declare a precedence rule
- Parle\Parser::push — Add a grammar rule
- Parle\Parser::reset — Reset parser state
- Parle\Parser::right — Declare a token with right-associativity
- Parle\Parser::sigil — Retrieve a matching part of a rule
- Parle\Parser::sigilCount — Number of elements in matched rule
- Parle\Parser::sigilName — Retrieve a rule or token name
- Parle\Parser::token — Declare a token
- Parle\Parser::tokenId — Get token id
- Parle\Parser::trace — Trace the parser operation
- Parle\Parser::validate — Validate input
Parle\RParser — The Parle\RParser class
- Parle\RParser::advance — Process next parser rule
- Parle\RParser::build — Finalize the grammar rules
- Parle\RParser::consume — Consume the data for processing
- Parle\RParser::dump — Dump the grammar
- Parle\RParser::errorInfo — Retrieve the error information
- Parle\RParser::left — Declare a token with left-associativity
- Parle\RParser::nonassoc — Declare a token with no associativity
- Parle\RParser::precedence — Declare a precedence rule
- Parle\RParser::push — Add a grammar rule
- Parle\RParser::reset — Reset parser state
- Parle\RParser::right — Declare a token with right-associativity
- Parle\RParser::sigil — Retrieve a matching part of a rule
- Parle\RParser::sigilCount — Number of elements in matched rule
- Parle\RParser::sigilName — Retrieve a rule or token name
- Parle\RParser::token — Declare a token
- Parle\RParser::tokenId — Get token id
- Parle\RParser::trace — Trace the parser operation
- Parle\RParser::validate — Validate input
Parle\Stack — The Parle\Stack class
- Parle\Stack::pop — Pop an item from the stack
- Parle\Stack::push — Push an item into the stack
Parle\Token — The Parle\Token class
Parle\ErrorInfo — The Parle\ErrorInfo class
Parle\LexerException — The Parle\LexerException class
Parle\ParserException — The Parle\ParserException class

The Parle\Lexer class

はじめに

Single state lexer class. Lexemes can be defined on the fly. If the particular lexer instance is meant to be used with Parle\Parser, the token IDs need to be taken from there. Otherwise, arbitrary token IDs can be supplied. This lexer can give a certain performance advantage over Parle\RLexer, if no multiple states are required. Note, that Parle\RParser is not compatible with this lexer.

クラス概要

Parle\Lexer

class Parle\Lexer {

/* 定数 */

const int Parle\Lexer::ICASE = 1;

const int Parle\Lexer::DOT_NOT_LF = 2;

const int Parle\Lexer::DOT_NOT_CRLF = 4;

const int Parle\Lexer::SKIP_WS = 8;

const int Parle\Lexer::MATCH_ZERO_LEN = 16;

/* プロパティ */

public bool $bol = false;

public int $flags = 0;

public int $state = 0;

public int $marker = 0;

public int $cursor = 0;

/* メソッド */

public void advance()

public void build()

public void callout(int $id, callable $callback)

public void consume(string $data)

public void dump()

public Parle\Token getToken()

public void insertMacro(string $name, string $regex)

public void push(string $regex, int $id)

public void reset(int $pos)

}

定義済み定数

Parle\Lexer::ICASE
Parle\Lexer::DOT_NOT_LF
Parle\Lexer::DOT_NOT_CRLF
Parle\Lexer::SKIP_WS
Parle\Lexer::MATCH_ZERO_LEN

プロパティ

bol: Start of input flag.
flags: Lexer flags.
state: Current lexer state, readonly.
marker: Position of the latest token match, readonly.
cursor: Current input offset, readonly.

The Parle\RLexer class

はじめに

Multistate lexer class. Lexemes can be defined on the fly. If the particular lexer instance is meant to be used with Parle\RParser, the token IDs need to be taken from there. Otherwise, arbitrary token IDs can be supplied. Note, that Parle\Parser is not compatible with this lexer.

クラス概要

Parle\RLexer

class Parle\RLexer {

/* 定数 */

const int Parle\RLexer::ICASE = 1;

const int Parle\RLexer::DOT_NOT_LF = 2;

const int Parle\RLexer::DOT_NOT_CRLF = 4;

const int Parle\RLexer::SKIP_WS = 8;

const int Parle\RLexer::MATCH_ZERO_LEN = 16;

/* プロパティ */

public bool $bol = false;

public int $flags = 0;

public int $state = 0;

public int $marker = 0;

public int $cursor = 0;

/* メソッド */

public void advance()

public void build()

public void callout(int $id, callable $callback)

public void consume(string $data)

public void dump()

public Parle\Token getToken()

public void insertMacro(string $name, string $regex)

public void push(string $regex, int $id)

public void push(
    string $state,
    string $regex,
    int $id,
    string $newState
)

public void push(string $state, string $regex, string $newState)

public int pushState(string $state)

public void reset(int $pos)

}

定義済み定数

Parle\RLexer::ICASE
Parle\RLexer::DOT_NOT_LF
Parle\RLexer::DOT_NOT_CRLF
Parle\RLexer::SKIP_WS
Parle\RLexer::MATCH_ZERO_LEN

プロパティ

bol: Start of input flag.
flags: Lexer flags.
state: Current lexer state, readonly.
marker: Position of the latest token match, readonly.
cursor: Current input offset, readonly.

The Parle\Parser class

はじめに

Parser class. Rules can be defined on the fly. Once finalized, a Parle\Lexer instance is required to deliver the token stream.

クラス概要

Parle\Parser

class Parle\Parser {

/* 定数 */

const int Parle\Parser::ACTION_ERROR = 0;

const int Parle\Parser::ACTION_SHIFT = 1;

const int Parle\Parser::ACTION_REDUCE = 2;

const int Parle\Parser::ACTION_GOTO = 3;

const int Parle\Parser::ACTION_ACCEPT = 4;

const int Parle\Parser::ERROR_SYNTAX = 0;

const int Parle\Parser::ERROR_NON_ASSOCIATIVE = 1;

const int Parle\Parser::ERROR_UNKNOWN_TOKEN = 2;

/* プロパティ */

public int $action = 0;

public int $reduceId = 0;

/* メソッド */

public void advance()

public void build()

public void consume(string $data, Parle\Lexer $lexer)

public void dump()

public Parle\ErrorInfo errorInfo()

public void left(string $tok)

public void nonassoc(string $tok)

public void precedence(string $tok)

public int push(string $name, string $rule)

public void reset(int $tokenId = ?)

public void right(string $tok)

public string sigil(int $idx)

public int sigilCount()

public string sigilName(int $idx)

public void token(string $tok)

public int tokenId(string $tok)

public string trace()

public bool validate(string $data, Parle\Lexer $lexer)

}

定義済み定数

Parle\Parser::ACTION_ERROR
Parle\Parser::ACTION_SHIFT
Parle\Parser::ACTION_REDUCE
Parle\Parser::ACTION_GOTO
Parle\Parser::ACTION_ACCEPT
Parle\Parser::ERROR_SYNTAX
Parle\Parser::ERROR_NON_ASSOCIATIVE
Parle\Parser::ERROR_UNKNOWN_TOKEN

プロパティ

action: Current parser action that matches one of the action class constants, readonly.
reduceId: Grammar rule id just processed in the reduce action. The value corresponds either to a token or to a production id. Readonly.

The Parle\RParser class

はじめに

Parser class. Rules can be defined on the fly. Once finalized, a Parle\RLexer instance is required to deliver the token stream.

クラス概要

Parle\RParser

class Parle\RParser {

/* 定数 */

const int Parle\RParser::ACTION_ERROR = 0;

const int Parle\RParser::ACTION_SHIFT = 1;

const int Parle\RParser::ACTION_REDUCE = 2;

const int Parle\RParser::ACTION_GOTO = 3;

const int Parle\RParser::ACTION_ACCEPT = 4;

const int Parle\RParser::ERROR_SYNTAX = 0;

const int Parle\RParser::ERROR_NON_ASSOCIATIVE = 1;

const int Parle\RParser::ERROR_UNKNOWN_TOKEN = 2;

/* プロパティ */

public int $action = 0;

public int $reduceId = 0;

/* メソッド */

public void advance()

public void build()

public void consume(string $data, Parle\RLexer $rlexer)

public void dump()

public Parle\ErrorInfo errorInfo()

public void left(string $tok)

public void nonassoc(string $tok)

public void precedence(string $tok)

public int push(string $name, string $rule)

public void reset(int $tokenId = ?)

public void right(string $tok)

public string sigil(int $idx = ?)

public int sigilCount()

public string sigilName(int $idx)

public void token(string $tok)

public int tokenId(string $tok)

public string trace()

public bool validate(string $data, Parle\RLexer $lexer)

}

定義済み定数

Parle\RParser::ACTION_ERROR
Parle\RParser::ACTION_SHIFT
Parle\RParser::ACTION_REDUCE
Parle\RParser::ACTION_GOTO
Parle\RParser::ACTION_ACCEPT
Parle\RParser::ERROR_SYNTAX
Parle\RParser::ERROR_NON_ASSOCIATIVE
Parle\RParser::ERROR_UNKNOWN_TOKEN

プロパティ

action: Current parser action that matches one of the action class constants, readonly.
reduceId: Grammar rule id just processed in the reduce action. The value corresponds either to a token or to a production id. Readonly.

The Parle\Stack class

はじめに

Parle\Stack is a LIFO stack. The elements are inserted and removed only from one end.

クラス概要

Parle\Stack

class Parle\Stack {

/* プロパティ */

public bool $empty = true;

public int $size = 0;

public mixed $top;

/* メソッド */

public void pop()

public void push(mixed $item)

}

プロパティ

empty: Whether the stack is empty, readonly.
size: Stack size, readonly.
top: Element on the top of the stack.

The Parle\Token class

はじめに

This class represents a token. Lexer returns instances of this class.

クラス概要

Parle\Token

class Parle\Token {

/* 定数 */

const int Parle\Token::EOI = 0;

const int Parle\Token::UNKNOWN = -1;

const int Parle\Token::SKIP = -2;

/* プロパティ */

public int $id;

public string $value;

/* メソッド */

}

プロパティ

id: Token id.
value: Token value.

定義済み定数

Parle\Token::EOI: End of input token id.
Parle\Token::UNKNOWN: Unknown token id.
Parle\Token::SKIP: Skip token id.

The Parle\ErrorInfo class

はじめに

The class represents detailed error information as supplied by Parle\Parser::errorInfo

クラス概要

Parle\ErrorInfo

class Parle\ErrorInfo {

/* プロパティ */

public int $id;

public int $position;

public mixed $token;

/* メソッド */

}

プロパティ

id: Error id.
position: Position in the input, where the error occurred.
token: If applicable - the Parle\Token related to the error, otherwise null.

The Parle\LexerException class

はじめに

クラス概要

Parle\LexerException

class Parle\LexerException extends Exception implements Throwable {

/* 継承したプロパティ */

protected string $message = "";

private string $string = "";

protected int $code;

protected string $file = "";

protected int $line;

private array $trace = [];

private Throwablenull $previous = null;

/* メソッド */

/* 継承したメソッド */

final public string Exception::getMessage()

final public Throwablenull Exception::getPrevious()

final public int Exception::getCode()

final public string Exception::getFile()

final public int Exception::getLine()

final public array Exception::getTrace()

final public string Exception::getTraceAsString()

public string Exception::__toString()

private void Exception::__clone()

}

The Parle\ParserException class

はじめに

クラス概要

Parle\ParserException

class Parle\ParserException extends Exception implements Throwable {

/* 継承したプロパティ */

protected string $message = "";

private string $string = "";

protected int $code;

protected string $file = "";

protected int $line;

private array $trace = [];

private Throwablenull $previous = null;

/* メソッド */

/* 継承したメソッド */

final public string Exception::getMessage()

final public Throwablenull Exception::getPrevious()

final public int Exception::getCode()

final public string Exception::getFile()

final public int Exception::getLine()

final public array Exception::getTrace()

final public string Exception::getTraceAsString()

public string Exception::__toString()

private void Exception::__clone()

}