2
\$\begingroup\$

I'm trying to make a lex (flex) grammar that kicks out tokens while always being aware of the current line number.

As an exercise, I made a simple grammar that accepts raw strings '...' with no escape sequences and prints out the line range after a string is completed.

Here's the grammar

%{

#include <stdio.h>

int linum = 1;

int token_line_count(const char *text)
{
    int count = 0;
    const char *it = text;
    if (it == NULL) { return 0; }

    while (*it) {
        if (*it == '\n') {
            ++count;
        }
        ++it;
    }
    return count;
}

%}

%option noyywrap

%%

'[^']*' {
    int begin_linum = linum;
    int linum_len = token_line_count(yytext);
    int end_linum = linum + linum_len;

    printf("found string from line %d to %d\n", begin_linum, end_linum);

    linum += linum_len;
}

\n {
    ++linum;
}

. {
    /* do nothing */
}

%%

int main()
{
    yylex();
    return 0;
}

and here's an example source file to be parsed:

hello

garbage

other stuff

'first raw string one line long on line 7'

'second raw string is 2 lines long. from line 9
to line 10'

Here's an example of the grammar in action, parsing the file above (correctly).

% ./rawstr < examplerawstr.txt
found string from line 7 to 7
found string from line 9 to 10

And, for completeness, here's the makefile used to create the executable.

CFLAGS := -O2 -Wall -pedantic -std=c99

rawstr: rawstr.o
    $(CC) -o $@ $^

clean:
    $(RM) rawstr.c rawstr.o rawstr

It doesn't use the ability to "name" regexes in the preamble... and I'm a little concerned that a single matched string can be of arbitrary length ... is there a way to split a string up into "beginning", "quoted char", and "end" tokens and have them "add or remove" contexts?

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.