Get the latest tech news
Fast Tokenizers with StringScanner (2023)
-09-02 @ 13:00 Lately I’ve been messing around with writing a GraphQL parser called TinyGQL. I wanted to see how fast I could make a GraphQL parser without writing any C extensions.
The@ symbol in the inspect output shows where the cursor currently points, and the ratio at the beginning gives you kind of a “progress” counter. Since the parser doesn’t actually need to know about whitespace or comments in order to understand the document, it is fine for the lexer to just skip them. This makes our benchmark a little bit awkward now because we’ve shifted the blame of “identifier” allocations from the lexer to the parser.
Or read this on Hacker News