Stateless multi-delimiter tokenizer for SWMM input lines. More...

#include <Tokenizer.hpp>

Collaboration diagram for openswmm::input::Tokenizer:

Public Member Functions
	Tokenizer ()=delete
	Disallow tokenizer instances; all operations are static.

Static Public Member Functions
static std::string_view	strip_comment (std::string_view line) noexcept
	Borrow the line prefix before its first unquoted semicolon.

static std::vector< std::string >	tokenize (std::string_view line)
	Split a line into owned tokens using whitespace and positional comma rules.

static std::vector< std::string_view >	tokenize_views (std::string_view line)
	Split an unquoted line into borrowed nonempty token views.

static void	tokenize_views_into (std::string_view line, std::vector< std::string_view > &out)
	Replace a reusable vector with borrowed nonempty token views.

static void	to_upper_inplace (std::string &s) noexcept
	Uppercase bytes in place using the current C locale.

static std::string	to_upper (std::string_view s)
	Copy bytes and uppercase the copy using the current C locale.

static std::string_view	trim (std::string_view s) noexcept
	Borrow the substring without leading/trailing C-locale whitespace.

static bool	is_numeric (std::string_view sv) noexcept
	Test whether a trimmed token parses completely as a double.

static bool	is_boolean (std::string_view sv) noexcept
	Recognize an untrimmed boolean token.

static bool	parse_boolean (std::string_view sv) noexcept
	Interpret an untrimmed affirmative boolean token.

Detailed Description

Stateless multi-delimiter tokenizer for SWMM input lines.

All methods are static and stateless. Instantiation is not needed.

Constructor & Destructor Documentation

◆ Tokenizer()

openswmm::input::Tokenizer::Tokenizer ( )

delete

Disallow tokenizer instances; all operations are static.

Member Function Documentation

◆ is_boolean()

bool openswmm::input::Tokenizer::is_boolean ( std::string_view sv )

staticnoexcept

Recognize an untrimmed boolean token.

Parameters

[in] sv Borrowed token; locale-uppercase comparison accepts YES/NO/TRUE/FALSE/1/0.

Returns: True for those complete tokens; false for whitespace-padded, unknown or >=8-byte tokens.

◆ is_numeric()

bool openswmm::input::Tokenizer::is_numeric ( std::string_view sv )

staticnoexcept

Test whether a trimmed token parses completely as a double.

Special values and leading-plus acceptance follow the selected numeric backend. This is not a finiteness predicate.

Parameters

[in] sv Borrowed token; leading/trailing C-locale whitespace is trimmed.

Returns: True for successful complete conversion by from_chars_double(); false for empty, invalid, partial or reported overflow.

Here is the call graph for this function:

◆ parse_boolean()

bool openswmm::input::Tokenizer::parse_boolean ( std::string_view sv )

staticnoexcept

Interpret an untrimmed affirmative boolean token.

Use is_boolean() first when an invalid token must be distinguished from false.

Parameters

[in] sv Borrowed token compared after C-locale byte uppercasing.

Returns: True for YES/TRUE/1; false for NO/FALSE/0 and all invalid tokens.

Here is the caller graph for this function:

◆ strip_comment()

std::string_view openswmm::input::Tokenizer::strip_comment ( std::string_view line )

staticnoexcept

Borrow the line prefix before its first unquoted semicolon.

Double quotes toggle quote state; backslash/CSV doubled-quote escaping is not interpreted. No allocation; returned storage must outlive the view.

Parameters

[in] line Borrowed bytes; caller retains storage for the duration of this call.

Returns: View into line, unchanged if no unquoted semicolon exists.

Here is the caller graph for this function:

◆ to_upper()

std::string openswmm::input::Tokenizer::to_upper ( std::string_view s )

static

Copy bytes and uppercase the copy using the current C locale.

Parameters

[in] s Borrowed source bytes, including embedded NUL bytes.

Returns: Independent uppercase string; input is preserved.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ to_upper_inplace()

void openswmm::input::Tokenizer::to_upper_inplace ( std::string & s )

staticnoexcept

Uppercase bytes in place using the current C locale.

Preserves byte count; does not perform Unicode case mapping. Unlike engine name equality helpers this uses the C locale, not a fixed ASCII-only fold.

Parameters

[in,out] s Mutable string whose bytes are converted via unsigned-char-safe std::toupper.

Here is the caller graph for this function:

◆ tokenize()

std::vector< std::string > openswmm::input::Tokenizer::tokenize ( std::string_view line )

static

Split a line into owned tokens using whitespace and positional comma rules.

Strips unquoted semicolon comments. Space/tab delimit; commas delimit at boundaries or in a line without unquoted whitespace. Leading/consecutive CSV commas can create empty tokens; a final comma does not create a final empty field. A leading quoted token loses its enclosing quotes; embedded quotes remain. Unclosed quotes are tolerated through end of line. No escape decoding or CR/LF normalization is performed here.

Parameters

[in] line Borrowed bytes; caller retains storage for the duration of this call.

Returns: Independent owned strings; empty for a blank/comment-only line.

Here is the call graph for this function:

◆ tokenize_views()

std::vector< std::string_view > openswmm::input::Tokenizer::tokenize_views ( std::string_view line )

static

Split an unquoted line into borrowed nonempty token views.

Comment/comma rules mirror tokenize(), but quoted tokens are unsupported and empty CSV columns are discarded. Allocates the result vector; token bytes are not copied.

Parameters

[in] line Borrowed bytes; caller retains storage for the duration of this call.

Returns: Vector owning the views, whose referenced bytes remain owned by line.

Here is the call graph for this function:

◆ tokenize_views_into()

void openswmm::input::Tokenizer::tokenize_views_into	(	std::string_view	line,
		std::vector< std::string_view > &	out )

static

Replace a reusable vector with borrowed nonempty token views.

Quoted tokens are unsupported; empty CSV columns are discarded. Reserves at least eight entries, so first use and larger rows can allocate. Retained views are invalid once input storage changes or dies.

Parameters

[in]	line	Borrowed bytes; caller retains storage for the duration of this call.
[out]	out	Caller-owned vector cleared first; its capacity is reused and grown when needed. Views borrow line storage, which must not alias storage invalidated by clearing out.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ trim()

std::string_view openswmm::input::Tokenizer::trim ( std::string_view s )

staticnoexcept

Borrow the substring without leading/trailing C-locale whitespace.

Uses unsigned-char-safe std::isspace; does not remove interior whitespace.

Parameters

[in] s Borrowed input bytes; may be empty.

Returns: View into s, possibly empty; caller must preserve underlying storage.

Here is the caller graph for this function:

The documentation for this class was generated from the following files:

/home/runner/work/openswmm.engine/openswmm.engine/src/engine/input/Tokenizer.hpp
/home/runner/work/openswmm.engine/openswmm.engine/src/engine/input/Tokenizer.cpp

Public Member Functions

Static Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ Tokenizer()

Member Function Documentation

◆ is_boolean()

◆ is_numeric()

◆ parse_boolean()

◆ strip_comment()

◆ to_upper()

◆ to_upper_inplace()

◆ tokenize()

◆ tokenize_views()

◆ tokenize_views_into()

◆ trim()