String to token vector for expression parsing

Question

My objective is to create a linear algebra calculator that would be able calculate expressions containing vectors, matrices, (eventually complex numbers/imaginary; quaternions), and real numbers. I've got the code for parsing tokens from a given string into a vector of Token object pointers:

// Function for an operator to implement.
typedef Value*(*EvaluationFunction)(Value** args);

// The associativity of the operator.
enum Associativity : unsigned char
{
    NONE = 0, LEFT = 1, RIGHT = 2
};

// The evaluator structure that would be an
// operator and a function.
struct Evaluator
{
    EvaluationFunction evalFunc;
    unsigned char args; // The argument count expected.
};

// A structure for an operator.
struct Operator : public Evaluator
{
    String id;
    Associativity association;
    unsigned int precedence;

    // A constructor to create this struct
    // anonymously.
    Operator(String&& idp, unsigned int precedencep,
        Associativity assoc = Associativity::NONE)
    {
        id = idp;
        precedence = precedencep;
        association = assoc;
        args = 2;
    }
};

...

// The list of active operators.
static std::vector<Operator*> smOperators;

// Amount of space to reserve for vector when getting tokens.
static constexpr unsigned char RESERVE_MORE_VECTOR_SPACE{ 5 };

...

// Definitions for the token types.
// 'U' indicates unused beyond token parsing.
#define U_TOKEN_LEFT_BRACKET '['
#define U_TOKEN_RIGHT_BRACKET ']'
#define TOKEN_LEFT_PARENTHESIS '('
#define TOKEN_RIGHT_PARENTHESIS ')'
#define TOKEN_OPERATOR 'o'
#define TOKEN_FUNCTION 'f'
#define TOKEN_VALUE 'v'
#define TOKEN_COMMA ','

// A token structure used to represent
// expressional tokens.
struct Token
{
    String source;
    char type;
    union 
    {
        const Operator* op;
        const Function* func;
        Value* value;
    };
};

// Gets the specified operator.
static Operator* GetOperator(const char* src)
{
    for (auto& ptr : smOperators)
    {
        if (ptr->id == src)
        {
            return ptr;
        }
    }
    return NULL;
}

// Gets the tokens from an expression string.
static void GetTokens(const String& str, std::vector<Token*>& output)
{
    if (str.GetLength() == 0)
        return;

    output.reserve(RESERVE_MORE_VECTOR_SPACE);
    bool targetIsAlphaNumeric{ IsCharAlphaNumeric(str.At(0)) };
    bool readingMatrix{ false };
    unsigned int subStrIndex{ 0 };
    for (unsigned int i = 0; i < str.GetLength(); i++)
    {
        char cat = str.At(i);

        // Checks whether a matrix is being read in.
        if (readingMatrix)
        {
            if (cat == U_TOKEN_RIGHT_BRACKET)
            {
                output.emplace_back(CreateToken(str.Sub(subStrIndex - 1, i + 1))); // subStrIndex - 1 is there to include the left bracket
                                                                                   // and i + 1 is to include the right bracket.
                subStrIndex = i + 1;
                readingMatrix = false;
            }
            continue;
        }

        // If it's a non-unary, non-function, operator.
        if (cat == TOKEN_LEFT_PARENTHESIS
            || cat == TOKEN_RIGHT_PARENTHESIS
            || cat == TOKEN_COMMA
            || cat == U_TOKEN_LEFT_BRACKET)
        {
            // Checks whether the character is
            // the beginning of a matrix definition.
            if (cat == U_TOKEN_LEFT_BRACKET)
            {
                readingMatrix = true;
            }

            // Checks whether there should be a token
            // pushed onto the tokens vector
            if (subStrIndex != i)
            {
                output.emplace_back(CreateToken(str.Sub(subStrIndex, i)));
            }

            // Checks if a value is followed by a left parentheses.
            // Then it will insert a multiplication operator.
            if (output.size() > 0)
            {
                auto& lastElem = output.at(output.size() - 1);
                if ((cat == TOKEN_LEFT_PARENTHESIS 
                     || cat == U_TOKEN_LEFT_BRACKET
                     && (lastElem->type == TOKEN_VALUE
                         || lastElem->type == TOKEN_RIGHT_PARENTHESIS
                         || lastElem->type == U_TOKEN_RIGHT_BRACKET)))
                {
                    Token* t = new Token;
                    t->source = "*";
                    t->op = GetOperator("*");
                    t->type = TOKEN_OPERATOR;
                    output.emplace_back(t);
                }
            }

            // If the operator is not a comma or left bracket, it will
            // be pushed on the vector.
            if (cat != TOKEN_COMMA && cat != U_TOKEN_LEFT_BRACKET)
            {
                output.emplace_back(CreateToken(str.Sub(i, i + 1)));
            }

            subStrIndex = i + 1;

            // If there are still more characters to read,
            // then the target will be reassigned and the
            // substring index recalculated.
            if (i + 1 < str.GetLength())
            {
                targetIsAlphaNumeric = IsCharAlphaNumeric(str.At(i + 1));
                subStrIndex = i + 1;
            }

            // If the vector is out of capacity, it will allocate
            // 5 more slots just for efficiency.
            if (output.capacity() == output.size())
            {
                output.reserve(output.capacity() + RESERVE_MORE_VECTOR_SPACE);
            }
        }
        // If the end of the current token has been reached:
        else if((!targetIsAlphaNumeric && IsCharAlphaNumeric(cat))
                || (targetIsAlphaNumeric && !IsCharAlphaNumeric(cat)) )
        {
            // Places the token into the vector and adjusts
            // the target and substring index.
            Token* newToken = CreateToken(str.Sub(subStrIndex, i));

            // Makes sure that if the expression starts with an
            // operator (usually a negative sign), it will be, for example,
            // "0 - x" instead of "- x"; so it won't break.
            if (newToken->type == TOKEN_OPERATOR && output.size() == 0)
            {
                Token* t = new Token;
                t->source = "0";
                t->value = new Number(0);
                t->type = TOKEN_VALUE;
                output.emplace_back(t);
            }

            // Checks to see if a right parenthesis is followed by
            // a number, and if so it will insert a multiplication operator
            // between the number and the parenthesis.
            if (output.size() > 0 
                && newToken->type == TOKEN_VALUE 
                && output.at(output.size() - 1)->type == TOKEN_RIGHT_PARENTHESIS)
            {
                Token* t = new Token;
                t->source = "*";
                t->op = GetOperator("*");
                t->type = TOKEN_OPERATOR;
                output.emplace_back(t);
            }

            output.emplace_back(newToken);
            subStrIndex = i;
            targetIsAlphaNumeric = !targetIsAlphaNumeric;

            // If the vector is out of capacity, it will allocate
            // 5 more slots just for efficiency.
            if (output.capacity() == output.size())
            {
                output.reserve(output.capacity() + RESERVE_MORE_VECTOR_SPACE);
            }
        }
    }

    // Checks to see if another token can
    // be placed into the vector.
    if (subStrIndex != str.GetLength())
    {
        Token* newToken = CreateToken(str.Sub(subStrIndex, str.GetLength()));

        // Checks to see if a right parenthesis is followed by
        // a number, and if so it will insert a multiplication operator
        // between the number and the parenthesis.
        if (output.size() > 0 
            && newToken->type == TOKEN_VALUE 
            && output.at(output.size() - 1)->type == TOKEN_RIGHT_PARENTHESIS)
        {
            Token* t = new Token;
            t->source = '*';
            t->op = GetOperator("*");
            t->type = TOKEN_OPERATOR;
            output.emplace_back(t);
        }

        output.emplace_back(newToken);
    }

    // Resizes the vector to fit the contents
    // (may not be necessary, but for now it will stay).
    output.shrink_to_fit();
}

String.h:

#ifndef _STRING_H
#define _STRING_H

#include <vector>
#include <iostream>

// Enumeration that will define how
// a string will be constructed.
enum StringMode : char 
{
    DEFAULT = 0, REPLACE_WHITE_SPACE = 1
};

// Sets the mode for which the
// Strings will be input through istream
// by.
void SetStringInputMode(StringMode mode);

// Gets the mode for which the
// Strings will be input through istream
// by.
StringMode GetStringInputMode();

// This class should containg a pointer
// to a character/character array, but should
// use less memory than the std::string. The main
// purpose for this class is to simplify the
// "routing" of char*s when evaluating expressions.
class String final 
{
    // Uses the std::istream to input a string
    // object.
    friend std::istream& operator>>(std::istream&, String&);

    // Uses the std::ostream to output the specified string.
    friend std::ostream& operator<<(std::ostream&, const String&);
private:
    char* c_str; // The "c string"
    unsigned int length; // The length of the "c string"

    // Compares the characters between the two strings starting
    // at the specified index in this string.
    void inline CompareFrom(unsigned int, const String&, bool&) const;
public:
    // A default constructor just in case
    // we need the string to be input from
    // an istream.
    String();

    // Defining a copy constructor to make
    // sure the pointers are transfered correctly
    // and the string is generated with a specified mode.
    String(const String&, StringMode = StringMode::DEFAULT);

    // The constructor for the String class
    // that takes the original C string, and it
    // will copy it into a new heap allocated space
    // with a specified mode.
    String(const char*, StringMode = StringMode::DEFAULT);

    // Creates a new string that takes in a
    // buffer size but sets all of the values
    // to 0 / null.
    String(unsigned int size);

    // A destructor is defined for this class
    // to destroy the allocated heap memory.
    ~String();

    // Deletes the current c_str and
    // redefines this string with the specified
    // length.
    void Redefine(unsigned int);

    // Resizes this string to match the
    // specified length.
    void Resize(unsigned int);

    // Gets the pointer to the heap allocated
    // char*
    const char* GetPointer() const;

    // Gets a substring. First index is inclusive;
    // second index is exclusive.
    String Sub(unsigned int, unsigned int) const;

    // Gets the length of this string
    unsigned int GetLength() const;

    // Places the specified character c into
    // the index position 'index'
    void Put(char, unsigned int);

    // Gets the character at the specified
    // index.
    char At(unsigned int) const;

    // Finds the first instance of the specified
    // character in this string.
    unsigned int Find(char) const;

    // Attempts to find an instance of a specified
    // string within this string.
    unsigned int Find(const String&) const;

    // Attempts to find all matches of the specified
    // string within this string.
    void FindAll(const String&, std::vector<unsigned int>&) const;

    // An assignment operator to make sure
    // the operator from one string gets
    // transfered correctly to this string.
    void operator=(const String&);

    // A comparison operator to check whether
    // two strings are identical.
    // True = identical; False = not identical.
    bool operator==(const String&) const;

    // A comparison operator to check whether
    // two strings are not identical.
    // True = not identical; False = identical.
    bool operator!=(const String&) const;

    // A comparison operator to check whether
    // two strings are identical.
    // True = identical; False = not identical.
    bool operator==(const char*) const;

    // A comparison operator to check whether
    // two strings are not identical.
    // True = not identical; False = identical.
    bool operator!=(const char*) const;

    // A comparison operator to check whether
    // this string and specified char are identical.
    // True = identical; False = not identical.
    bool operator==(const char) const;

    // A comparison operator to check whether
    // the string and char are not identical.
    // True = not identical; False = identical.
    bool operator!=(const char) const;
};

// This class will be used when an
// unknown number of characters need to
// be put into a string.
class StringBuffer final
{
private:
    char** buffers; // The collection of buffers.
    unsigned int unitBufferSize; // The size of one individual buffer.
    unsigned int capacity; // The capacity of the buffers array.
    unsigned int currentBuffer; // Keeps track of an open buffer location.
    unsigned int iterator; // Iterator that will keep track of the current buffer location.

    // Places a character into the buffer using
    // and inline method.
    inline void PutChar(char c);

    // Adds n amount of buffers.
    void AddBuffers(unsigned int);

    // Locates a character at a specified
    // location.
    inline char At(unsigned int) const;
public:
    // A default constructor that will be used to
    // initialize the currentBuffer pointer to a
    // heap allocated one.
    explicit StringBuffer(unsigned int = 0, unsigned int = 64);

    // No need to have a copy constructor.
    StringBuffer(StringBuffer&) = delete;

    // A destructor to remove the heap allocated
    // buffers array.
    ~StringBuffer();

    // Gets the size of a unit buffer.
    unsigned int GetUnitBufferSize() const;

    // Sets the size of a unit buffer.
    void SetUnitBufferSize(unsigned int);

    // Gets the count of unit buffers.
    unsigned int GetUnitBufferCount() const;

    // Gets the length of the buffer.
    unsigned int GetLength() const;

    // Input a character into the buffer.
    void operator<<(char);

    // Inputs a string into the buffer.
    void operator<<(const String&);

    // Inputs a string into the buffer.
    void operator<<(const char*);

    // Copies the buffer data into a string object.
    void CopyInto(String&) const;

    // Converts the buffers into a String object.
    String Collect() const;

    // Clears any data stored in the buffers.
    // The size remains the same.
    void Reset(unsigned int = 0);

    // No need for an assignment operator.
    void operator=(StringBuffer&) = delete;

    // Checks whether a string matches what is in
    // this buffer.
    bool operator==(const String&) const;
};

#endif // _STRING_H

String.cpp:

#include <stdlib.h>
#include <string.h>
#include <stdexcept>
#include "String.h"

#define PRIVATE

// ----- NON-MEMBER FUNCTIONS -----

static StringMode INPUT_MODE{ StringMode::DEFAULT }; // Mode by which strings are input by.

// Sets the mode for which the
// Strings will be input through istream
// by.
void SetStringInputMode(StringMode mode)
{
    INPUT_MODE = mode;
}

// Gets the mode for which the
// Strings will be input through istream
// by.
StringMode GetStringInputMode()
{
    return INPUT_MODE;
}

// ------ STRING -------

// Uses the std::istream to input a string
// object.
std::istream& operator>>(std::istream& input, String& output)
{
    StringBuffer buffer{ 1 };
    char n;
    while (input.get(n))
    {
        if (n == 10)
            break;

        if (n == ' ' && INPUT_MODE == StringMode::REPLACE_WHITE_SPACE)
            continue;

        buffer << n;
    }
    buffer.CopyInto(output);
    return input;
}

// Uses the std::ostream to output the specified string.
std::ostream& operator<<(std::ostream& output, const String& str)
{
    output << str.c_str;
    return output;
}

// A default constructor just in case
// we need the string to be input from
// an istream.
String::String()
    : c_str{ (char*)malloc(sizeof(char)) }, length{ 0 }
{
    *(c_str) = NULL;
}

// Defining a copy constructor to make
// sure the pointers are transfered correctly
// and the string is generated with a specified mode.
String::String(const String& str, StringMode mode)
    : length{ 0 }
{
    if (mode == StringMode::REPLACE_WHITE_SPACE)
    {
        for (unsigned int i = 0; i < str.length; i++)
            if (*(str.c_str + i) != ' ')
                length++;

        if (length == 0)
        {
            c_str = nullptr;
            return;
        }

        c_str = (char*)malloc(length * sizeof(char) + 1);
        unsigned int whiteSpaceCount{ 0 };
        for (unsigned int i = 0; i < str.length; i++)
        {
            if (*(str.c_str + i) != ' ')
            {
                whiteSpaceCount++;
                continue;
            }

            *(c_str + i - whiteSpaceCount) = *(str.c_str + i);
        }
        *(c_str + length) = 0;
    }
    else
    {
        length = str.length;
        c_str = (char*)malloc(length * sizeof(char) + 1);
        strcpy(c_str, str.c_str);
    }
}

// The constructor for the String class
// that takes the original C string, and it
// will copy it into a new heap allocated space
// with a specified mode.
String::String(const char* defined, StringMode mode)
    : length{ 0 }
{
    if (mode == StringMode::REPLACE_WHITE_SPACE)
    {
        for (unsigned int i = 0; defined[i] != 0; i++)
        {
            if (*(defined + i) != ' ')
                length++;
        }

        if (length == 0)
        {
            c_str = nullptr;
            return;
        }

        c_str = (char*)malloc(length * sizeof(char) + 1);
        unsigned int whiteSpaceCount{ 0 };
        for (unsigned int i = 0; defined[i] != 0; i++)
        {
            if (*(defined + i) == ' ')
            {
                whiteSpaceCount++;
                continue;
            }

            *(c_str + i - whiteSpaceCount) = *(defined + i);
        }
        *(c_str + length) = 0;
    }
    else 
    {
        length = strlen(defined);
        c_str = (char*)malloc(length * sizeof(char) + 1);
        strcpy(c_str, defined);
    }
}

// Creates a new string that takes in a
// buffer size but sets all of the values
// to 0 / null.
String::String(unsigned int size)
    : length{ size } 
{
    if (size < 0)
    {
        c_str = nullptr;
        length = 0;
        return;
    }

    c_str = (char*)malloc(length + 1);
    for (unsigned int i = 0; i < length; i++)
        *(c_str + i) = 0;
    *(c_str + length) = 0;
}

// A destructor is defined for this class
// to destroy the allocated heap memory.
String::~String() 
{
    delete c_str;
}

// Compares the characters between the two strings starting
// at the specified index in this string.
PRIVATE void inline String::CompareFrom(unsigned int si, const String& ss, bool& so) const
{
    so = true;
    for (unsigned int i = 0; i < ss.GetLength(); i++)
    {
        if (*(c_str + i + si) != *(ss.c_str + i))
        {
            so = false;
            break;
        }
    }
}

// Deletes the current c_str and
// redefines this string with the specified
// length.
void String::Redefine(unsigned int nSize)
{
    free(c_str);
    length = nSize;
    c_str = (char*)malloc(length * sizeof(char) + 1);
    *(c_str + nSize) = 0;
}

// Resizes this string to match the
// specified length.
void String::Resize(unsigned int nSize)
{
    if (nSize < 0)
    {
        return;
    }

    char* new_str = (char*)malloc(nSize * sizeof(char) + 1);
    for (unsigned int i = 0; i < nSize; i++)
    {
        if (i < length)
            *(new_str + i) = *(c_str + i);
        else
            break;
    }
    *(new_str + nSize) = 0;
    free(c_str);
    c_str = new_str;
    length = nSize;
}

// Gets the pointer to the heap allocated
// char*
const char* String::GetPointer() const 
{
    return c_str;
}

// Gets a substring.
String String::Sub(unsigned int begin, unsigned int end) const
{
    if (begin < 0 || begin >= length || end < 0 || end > length || begin >= end)
    {
        return String{ static_cast<unsigned int>(0) };
    }

    String sub{ end - begin };
    for (unsigned int i = 0; i < sub.length; i++)
    {
        *(sub.c_str + i) = *(c_str + begin + i);
    }
    return sub;
}

// Gets the length of this string
unsigned int String::GetLength() const 
{
    return length;
}

// Places the specified character c into
// the index position 'index'
void String::Put(char c, unsigned int index)
{
    if (index < 0 || index >= length)
        return;

    *(c_str + index) = c;
}

// Gets the character at the specified
// index.
char String::At(unsigned int index) const
{
    if (index < 0 || index >= length)
        return NULL;

    return *(c_str + index);
}

// Finds the first instance of the specified
// character in this string.
unsigned int String::Find(char c) const
{
    for (unsigned int i = 0; i < length; i++)
        if (*(c_str + i) == c)
            return i;

    return length;
}

// Attempts to find an instance of a specified
// string within this string.
unsigned int String::Find(const String& str) const
{
    if (str.GetLength() > length || str.GetLength() == 0 || length == 0)
    {
        return length;
    }

    bool output;
    for (unsigned int i = 0; i <= length - str.GetLength(); i++)
    {
        CompareFrom(i, str, output);
        if (output)
        {
            return i;
        }
    }
    return length;
}

// Attempts to find all matches of the specified
// string within this string.
void String::FindAll(const String& str, std::vector<unsigned int>& ind) const
{
    unsigned int strLen = str.GetLength();
    if (strLen > length || strLen == 0 || length == 0)
    {
        return;
    }
    for (unsigned int i = 0; i < length - strLen; i++)
    {
        bool found{ false };
        CompareFrom(i, str, found);
        if (found)
        {
            ind.emplace_back(i);
            i += strLen - 1;
        }
    }
}

// An assignment operator to make sure
// the operator from one string gets
// transfered correctly to this string.
void String::operator=(const String& str) 
{
    free(c_str);
    c_str = (char*)malloc(str.length * sizeof(char) + 1);
    strcpy(c_str, str.c_str);
    length = str.length;
}

// A comparison operator to check whether
// two strings are identical.
// True = identical; False = not identical.
bool String::operator==(const String& str) const
{
    if (str.length != length)
        return false;

    for (unsigned int i = 0; i < length; i++)
        if (*(c_str + i) != *(str.c_str + i))
            return false;

    return true;
}

// A comparison operator to check whether
// two strings are not identical.
// True = not identical; False = identical.
bool String::operator!=(const String& str) const
{
    if (str.length != length)
        return true;

    for (unsigned int i = 0; i < length; i++)
        if (*(c_str + i) != *(str.c_str + i))
            return true;

    return false;
}

// A comparison operator to check whether
// two strings are identical.
// True = identical; False = not identical.
bool String::operator==(const char* str) const
{
    unsigned int len = strlen(str);
    if (length != len)
        return false;

    for (unsigned int i = 0; i < len; i++)
        if (*(c_str + i) != *(str + i))
            return false;

    return true;
}

// A comparison operator to check whether
// two strings are not identical.
// True = not identical; False = identical.
bool String::operator!=(const char* str) const
{
    unsigned int len = strlen(str);
    if (length != len)
        return true;

    for (unsigned int i = 0; i < len; i++)
        if (*(c_str + i) != *(str + i))
            return true;

    return false;
}

// A comparison operator to check whether
// this string and specified char are identical.
// True = identical; False = not identical.
bool String::operator==(const char c) const
{
    if (length != 1)
        return false;

    return *c_str == c;
}

// A comparison operator to check whether
// the string and char are not identical.
// True = not identical; False = identical.
bool String::operator!=(const char c) const
{
    if (length != 1)
        return true;

    return *c_str != c;
}

// ------ STRING BUFFER -------

// A default constructor that will be used to
// initialize the currentBuffer pointer to a
// heap allocated one.
StringBuffer::StringBuffer(unsigned int cap, unsigned int unitBufSize)
    : unitBufferSize{ unitBufSize < 1 ? 1 : unitBufSize }, iterator { 0 }, currentBuffer{ 0 }, capacity{ cap < 1 ? 1 : cap }
{
    buffers = (char**)malloc(sizeof(char*) * capacity);

    for (unsigned int i = 0; i < capacity; i++)
    {
        *(buffers + i) = (char*)malloc(sizeof(char) * unitBufferSize);
    }
}

// A destructor to remove the heap allocated
// buffers array.
StringBuffer::~StringBuffer()
{
    for (unsigned int i = 0; i < capacity; i++)
    {
        free(*(buffers + i));
        *(buffers + i) = nullptr;
    }
    free(buffers);
}

// Gets the size of a unit buffer.
unsigned int StringBuffer::GetUnitBufferSize() const
{
    return unitBufferSize;
}

// Sets the size of a unit buffer.
void StringBuffer::SetUnitBufferSize(unsigned int nBuffSize)
{
    if (currentBuffer == 0 && iterator == 0) // Nothing has been appended so far.
        unitBufferSize = nBuffSize == 0 ? 1 : nBuffSize;
}

// Gets the count of unit buffers.
unsigned int StringBuffer::GetUnitBufferCount() const
{
    return capacity + 1;
}

// Gets the length of the buffer.
unsigned int StringBuffer::GetLength() const
{
    return (unitBufferSize * currentBuffer) + iterator;
}

// Places a character into the buffer using
// and inline method.
PRIVATE inline void StringBuffer::PutChar(char c)
{
    if (iterator == unitBufferSize)
    {
        if (currentBuffer + 1 == capacity)
            AddBuffers(1);

        currentBuffer++;
        iterator = 0;
    }
    *(*(buffers + currentBuffer) + iterator++) = c; // place char into next available spot
}

// Resizes the buffers array to a specified amount.
PRIVATE void StringBuffer::AddBuffers(unsigned int amt)
{
    char** nAlloc = (char**)malloc(sizeof(char*) * (capacity + amt));
    for (unsigned int i = 0; i < amt + capacity; i++)
    {
        if (i < capacity && buffers != nullptr)
            *(nAlloc + i) = *(buffers + i);
        else
            *(nAlloc + i) = (char*)malloc(sizeof(char) * unitBufferSize);
    }

    capacity += amt;
    free(buffers);
    buffers = nAlloc;
}

// Locates a character at a specified
// location.
PRIVATE inline char StringBuffer::At(unsigned int loc) const
{
    return (*(*(buffers + (loc / unitBufferSize)) + (loc % unitBufferSize)));
}

// Input a character into the buffer.
void StringBuffer::operator<<(char n)
{
    PutChar(n);
}

// Inputs a string into the buffer.
void StringBuffer::operator<<(const String& str)
{
    for (unsigned int i = 0; i < str.GetLength(); i++)
    {
        PutChar(str.At(i));
    }
}

// Inputs a string into the buffer.
void StringBuffer::operator<<(const char* str)
{
    if (str == nullptr)
        return;

    unsigned int i{ 0 };
    while (*(str + i) != 0)
    {
        PutChar(*(str + i++));
    }
}

// Copies the buffer data into a string object.
void StringBuffer::CopyInto(String& str) const
{
    if (currentBuffer == 0 && iterator == 0)
        return;

    unsigned int nStrLen = (currentBuffer * unitBufferSize) + iterator;
    str.Redefine(nStrLen);
    for (unsigned int i = 0; i < nStrLen; i++)
    {
        str.Put(At(i), i);
    }
}

// Converts the buffers into a String object.
String StringBuffer::Collect() const
{
    if (currentBuffer == 0 && iterator == 0)
        return String{};

    unsigned int nStrLen = (currentBuffer * unitBufferSize) + iterator;
    String output{ nStrLen };
    for (unsigned int i = 0; i < nStrLen; i++)
    {
        output.Put(At(i), i);
    }
    return output;
}

// Clears any data stored in the buffers.
// The size remains the same.
void StringBuffer::Reset(unsigned int cap)
{
    for (unsigned int i = 0; i < capacity; i++)
    {
        free(*(buffers + i));
    }
    capacity = cap < 1 ? 1 : cap;
    free(buffers);
    buffers = (char**)malloc(sizeof(char*) * capacity);

    for (unsigned int i = 0; i < capacity; i++)
    {
        *(buffers + i) = (char*)malloc(sizeof(char) * unitBufferSize);
    }

    currentBuffer = 0;
    iterator = 0;
}

// Checks whether a string matches what is in
// this buffer.
bool StringBuffer::operator==(const String& str) const
{
    if (str.GetLength() != iterator + (currentBuffer * unitBufferSize))
    {
        return false;
    }

    for (unsigned int i = 0; i < str.GetLength(); i++)
    {
        if (str.At(i) != At(i))
            return false;
    }
    return true;
}

Edit: After going back through the String class, I honestly have no idea what I was doing with "routing char pointers", but I also wanted to input a string directly from std::cin and be able to replace the white-space as the input was coming through. Would it be better to use std::string?

Even though the calculator in general currently evaluates an average expression such as 5+[(2+2)*5;5] very fast (1/10 millisecond), from my looks the GetTokens function can use some work. I'm specifically concerned about improving the second if block in the main for loop as I feel comparing the character again within the block is redundant. I've tried somehow fitting it into a switch statement, but it would never work out the way I want it to.

That's pretty much it. Since I'm also relatively new to C++ (and "low level" programming in general), please do comment on any improvements in the code as a whole, and if anything can be done better/more efficiently.

Answer 1

Overall I think this is fairly reasonable. I think the biggest issues are a tendency to reinvent the wheel, and the need to break things into smaller pieces to make it more readable. Here are some suggestions.

Macros

Using #define for constants is a C-ism. In C++ you should prefer to use either enums or const values. So for the various token definitions, I would probably do something like this:

enum TokenType : char {
    U_TOKEN_LEFT_BRACKET = '[',
    U_TOKEN_RIGHT_BRACKET = ']',
    // ... etc.
};

Or if you'd rather use const, then something like this:

static const char U_TOKEN_LEFT_BRACKET = `[`;
static const char U_TOKEN_RIGHT_BRACKET = ']';
// ... etc.

Reinventing the Wheel

You seem to have written a lot of code that already exists elsewhere. For example, the String class, the IsCharAlphaNumeric() function, etc. You should look things up online before implementing something of your own. Some specifics:

You say:

I also wanted to input a string directly from std::cin and be able to replace the white-space as the input was coming through. Would it be better to use std::string?

Almost certainly, but try it and see. std::string is used in thousands of real-world programs run by millions of users. It's well-tested and efficient. But the only way to know for sure whether it's better than what you're currently using is to try it and then profile the code to see if it got faster.

The other thing is that your String class has some issues. The constructor doesn't check the return value of malloc(), so it's possible that there's no memory allocated for the string, but it gets written to anyway. It's also doing more work than it needs to. For example in the copy constructor, in the else case, you could simply call strdup() instead of malloc() followed by strcpy().

The IsCharAlphaNumeric() function can be replace with isalnum() in <ctype.h>. The GetOperator() function can be replaced with a call to std::find_if().

Simplify

I see a number of things that could be simplified as well. The Token class has no constructors. You could simply make 3 of them - one for each part in the union. That would make the 4 places where you manually allocate one (without checking if it succeeded) and then manually fill it out into a single line.

Also, why is output a vector of pointers to Tokens? If it were just a vector of Token objects you could use emplace_back() to add them without an extra call to CreateToken(). (Speaking of which, I don't see the code for that function. Given the name it looks an awful lot like the constructors I suggested making in the previous paragraph.) In any event, if you deal with fewer pointers, you have less chance of forgetting to check if allocating them succeeded.

This:

if((!targetIsAlphaNumeric && IsCharAlphaNumeric(cat))
            || (targetIsAlphaNumeric && !IsCharAlphaNumeric(cat)) )

can be simplified to this:

if (targetIsAlphaNumeric != IsCharAlphaNumeric(cat))

Have you verified that your attempts to reserve more space are actually more efficient? std::vector is pretty efficient already. (Also, do you mean efficient in space or in time?)

You can make the loop easier to read and understand by breaking it out into functions, like this:

for (unsigned int i = 0; i < str.GetLength(); i++)
{
    if (readingMatrix)
    {
        ParseMatrixCharacter(cat, i, output);
    }
    else if (isRegularOperator(cat))
    {
        ParseRegularOperator(cat, i, output);
    }
    else if (targetIsAlphaNumeric != IsCharAlphaNumeric(cat))
    {
        ParseEnd(cat, i, output);
    }
}

Naming

Most of your names are understandable. There were just a few I would suggest working on. What does cat mean? "character at"? If so, call it that. Why not call t token? It's only a few more characters and is more readable.