net.thevpc.nuts.runtime.standalone.format.tson.parser.custom

Class TsonCustomLexer

java.lang.Object

net.thevpc.nuts.runtime.standalone.format.tson.parser.custom.TsonCustomLexer

All Implemented Interfaces:

NGenerator<NElementTokenImpl>

public class TsonCustomLexer
extends Object
implements NGenerator<NElementTokenImpl>

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TsonCustomLexer.LinesAndContent

Constructor Summary

Constructors

Constructor and Description
TsonCustomLexer(Reader reader)
TsonCustomLexer(String reader)

Method Summary

Modifier and Type	Method and Description
List<NElementTokenImpl>	all()
NElementTokenImpl	continueReadBullet(NElementTokenType type, String chars, int line, int column, long pos)
static String	encodeStringContent(String value, char c, int N) Encodes the interior of a fenced string (encoding section 2.1): internal maximal delimiter runs of length k are written as k quotes when k < N, and as k+1 quotes when k >= N (the extra quote prevents an internal run from being mistaken for the closing fence).
NElementTokenImpl	next()
NElementTokenImpl	readBlockString()
NElementTokenImpl	readFenced(char c0, int N, NElementTokenType tokenType, NElementType elementType, int line, int column, long pos) Reads a fenced (quoted) string with fence length N using the generalized P-run counting rule:
NElementTokenImpl	readFencedString(int c, int line, int column, long pos) Reads a fenced (quoted) string whose opening fence is the maximal run of quote characters at the current position.
NElementTokenImpl	readLineString()
NElementTokenImpl	readMultiLineString(String marker, NElementTokenType tokenType)
NElementTokenImpl	readQuoted(char c0, NElementTokenType elementTokenType, NElementType elementType)
NElementTokenImpl	readTripleQuoted(char c0, NElementTokenType tokenType, NElementType elementType)
static String	toFencedString(String value, char c, int N, boolean unterminated) Serializes a string value using fence length N (encoding section 2.1 + boundary space trimming): internal delimiter runs of length k < N are written as-is, runs of length k >= N are written as k+1 quotes, and a single sacrificial space is added at the very start/end of content when the content starts/ends with a delimiter character (so the literal quote cannot merge into the fence).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TsonCustomLexer

public TsonCustomLexer(String reader)

TsonCustomLexer

public TsonCustomLexer(Reader reader)

Method Detail

all

public List<NElementTokenImpl> all()

continueReadBullet

public NElementTokenImpl continueReadBullet(NElementTokenType type,
                                            String chars,
                                            int line,
                                            int column,
                                            long pos)

next

public NElementTokenImpl next()

Specified by:

next in interface NGenerator<NElementTokenImpl>

readFencedString

public NElementTokenImpl readFencedString(int c,
                                          int line,
                                          int column,
                                          long pos)

Reads a fenced (quoted) string whose opening fence is the maximal run of quote characters at the current position.

• run of 1 → N=1 (simple quoted string)
• run of exactly 2 → empty string (open + immediately closed, both quotes consumed). N=2 is not a valid fence length.
• run of 3+ → N = run length (arbitrary fenced string)

readFenced

public NElementTokenImpl readFenced(char c0,
                                    int N,
                                    NElementTokenType tokenType,
                                    NElementType elementType,
                                    int line,
                                    int column,
                                    long pos)

Reads a fenced (quoted) string with fence length N using the generalized P-run counting rule:

 P < N : all P quotes are literal content, string stays open
 P == N : the string terminates here
 P > N : P-1 literal quotes, 1 discarded (escape signal), string stays open
 

Reaching EOF while the string is still open produces an error token (unterminated fenced string); the reader is left at EOF and the token image is the raw, undosed text.

readQuoted

public NElementTokenImpl readQuoted(char c0,
                                    NElementTokenType elementTokenType,
                                    NElementType elementType)

readTripleQuoted

public NElementTokenImpl readTripleQuoted(char c0,
                                          NElementTokenType tokenType,
                                          NElementType elementType)

toFencedString

public static String toFencedString(String value,
                                    char c,
                                    int N,
                                    boolean unterminated)

Serializes a string value using fence length N (encoding section 2.1 + boundary space trimming): internal delimiter runs of length k < N are written as-is, runs of length k >= N are written as k+1 quotes, and a single sacrificial space is added at the very start/end of content when the content starts/ends with a delimiter character (so the literal quote cannot merge into the fence). When unterminated is true, no closing fence is written.

encodeStringContent

public static String encodeStringContent(String value,
                                         char c,
                                         int N)

Encodes the interior of a fenced string (encoding section 2.1): internal maximal delimiter runs of length k are written as k quotes when k < N, and as k+1 quotes when k >= N (the extra quote prevents an internal run from being mistaken for the closing fence). Every other character is written verbatim.

readLineString

public NElementTokenImpl readLineString()

readBlockString

public NElementTokenImpl readBlockString()

readMultiLineString

public NElementTokenImpl readMultiLineString(String marker,
                                             NElementTokenType tokenType)