Search Results
| Search type | Search syntax |
|---|---|
| Tags | [tag] |
| Exact | "words here" |
| Author |
user:1234 user:me (yours) |
| Score |
score:3 (3+) score:0 (none) |
| Answers |
answers:3 (3+) answers:0 (none) isaccepted:yes hasaccepted:no inquestion:1234 |
| Views | views:250 |
| Code | code:"if (foo != bar)" |
| Sections |
title:apples body:"apples oranges" |
| URL | url:"*.example.com" |
| Saves | in:saves |
| Status |
closed:yes duplicate:no migrated:no wiki:no |
| Types |
is:question is:answer |
| Exclude |
-[tag] -apples |
| For more details on advanced search visit our help page | |
Results tagged with lexical-analysis
Search options answers only
not deleted
user 6172
Lexical analysis is the process of converting a sequence of characters into a sequence of tokens.
2
votes
Simple tokenizer v2 - reading all matching chars at once
A couple of tiny tidbits:
You could easily make Token immutable (removing the property setters) by passing type and index into the constructor as such:
public Token((bool Success, string Token, int …
- 14.2k
5
votes
Simple language tokenizer in C#
These are just some quick thoughts off the top of my head.
Is there any particular functional reason for TokenType to be ushort? In general, you don't modify from the base type unless it's intended …
- 14.2k