Lexer
in package
RFC 5322 header token lexer
Tokenizes a header name/value string into atoms, quoted-strings, structural delimiters, and folding whitespace, per RFC 5322 §3.2. Comments ("(...)", nesting-aware) are recognized so they don't corrupt quote/delimiter tracking, but are discarded rather than emitted as tokens. Malformed input (an unterminated quoted-string or unbalanced comment) degrades gracefully rather than throwing - this is a best-effort scanner for real-world mail/HTTP headers, some of which are slightly off-spec.
Tags
Table of Contents
Constants
- ATOM = 'ATOM'
- DELIMITER = 'DELIMITER'
- FWS = 'FWS'
- QUOTED_STRING = 'QUOTED_STRING'
- DELIMITERS = [';' => true, ',' => true, '=' => true, ':' => true, '<' => true, '>' => true]
- WHITESPACE = [' ' => true, "\t" => true, "\r" => true, "\n" => true]
Properties
Methods
- __construct() : mixed
- findTopLevelDelimiter() : Token|null
- Find the first occurrence of $delimiterChar that is a top-level DELIMITER token in $input (i.e. not inside a quoted-string or comment).
- tokenize() : array<string|int, Token>
- unfold() : string
- Collapse obs-fold sequences (CRLF followed by one-or-more SP/TAB) into a single space, leaving other CRLFs untouched as line separators.
- readAtom() : Token
- readQuotedString() : Token
- readWhitespace() : Token
- skipComment() : void
Constants
ATOM
public
mixed
ATOM
= 'ATOM'
DELIMITER
public
mixed
DELIMITER
= 'DELIMITER'
FWS
public
mixed
FWS
= 'FWS'
QUOTED_STRING
public
mixed
QUOTED_STRING
= 'QUOTED_STRING'
DELIMITERS
protected
mixed
DELIMITERS
= [';' => true, ',' => true, '=' => true, ':' => true, '<' => true, '>' => true]
WHITESPACE
protected
mixed
WHITESPACE
= [' ' => true, "\t" => true, "\r" => true, "\n" => true]
Properties
$input
protected
string
$input
$length
protected
int
$length
$pos
protected
int
$pos
= 0
Methods
__construct()
public
__construct(string $input) : mixed
Parameters
- $input : string
findTopLevelDelimiter()
Find the first occurrence of $delimiterChar that is a top-level DELIMITER token in $input (i.e. not inside a quoted-string or comment).
public
static findTopLevelDelimiter(string $input, string $delimiterChar) : Token|null
Parameters
- $input : string
- $delimiterChar : string
Return values
Token|nulltokenize()
public
tokenize() : array<string|int, Token>
Return values
array<string|int, Token>unfold()
Collapse obs-fold sequences (CRLF followed by one-or-more SP/TAB) into a single space, leaving other CRLFs untouched as line separators.
public
static unfold(string $input) : string
Parameters
- $input : string
Return values
stringreadAtom()
protected
readAtom() : Token
Return values
TokenreadQuotedString()
protected
readQuotedString() : Token
Return values
TokenreadWhitespace()
protected
readWhitespace() : Token
Return values
TokenskipComment()
protected
skipComment() : void