feat(gnovm): custom go parser for gas counting and nesting limitation #4027
+13,535
−45
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The Go parser from the standard library has been imported in gnovm/pkg/parser, with a Makefile to generate and apply custom patch. The patch consist to add a new function ParseFileStats which performs accounting to enable gas computation and set the maximum nesting level as an input parameter to instead of a constant. Parsing is instrumented to measure and control the nesting level and have a first cost estimate of parsing to compute gas, before type checking itself. The advantages are that there is almost no overhead, early catch of problems (even before the AST memory is allocated), and a simple gas formula (no distinction between tokens except the nesting level). No extra pass is added, we just exploit some internal parser metadata for prevention of too deep nesting and gas counting. The drawback is to fork and maintain a different version of the Go parser, but most of it is automated. Note: the Go parser has been imported from go1.24.1, as it has been simplified and has no internal dependencies, as opposite to previous versions.
github-project-automation
bot
added this to
💪 Bounties & Worx and 🧙♂️gno.land core team
🛠 PR Checks SummaryAll Automated Checks passed. ✅ Manual Checks (for Reviewers):
Read More🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers. ✅ Automated Checks (for Contributors):🟢 Maintainers must be able to edit this pull request (more info) ☑️ Contributor Actions:
☑️ Reviewer Actions:
📚 Resources:Debug
|
Codecov ReportAttention: Patch coverage is 📢 Thoughts on this report? Let us know! |
piux2
reviewed
Apr 9, 2025
| // nesting level reached during the parsing, which reflect both the size | ||
| // and the complexity of the AST. | ||
| return types.Gas(stats.NumTok * math.Ilogb(float64(1+stats.TopNest))) | ||
| } |
There was a problem hiding this comment.
Nice! Gas is a measure of computation time, benchmarked in nanoseconds. Is it possible to benchmark this?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Go parser from the standard library has been imported in gnovm/pkg/parser, with a Makefile to generate and apply custom patch.
The patch consist to add a new function ParseFileStats which performs accounting to enable gas computation and set the maximum nesting level as an input parameter to instead of a constant.
Parsing is instrumented to measure and control the nesting level and have a first cost estimate of parsing to compute gas, before type checking itself.
The advantages are that there is almost no overhead, early catch of problems (even before the AST memory is allocated), and a simple gas formula (no distinction between tokens except the nesting level). No extra pass is added, we just exploit some internal parser metadata for prevention of too deep nesting and gas counting.
The drawback is to fork and maintain a different version of the Go parser, but most of it is automated.
Note: the Go parser imported through the makefile corresponds to the go version installed on host (currently go1.23.7).
The maximum parsing nesting limit has been fixed to 10000, instead of previous 100000. It correspond to a maximum AST depth of approximately half of it (ie now 5000 instead of 50000). The limit is now a parameter of
ParseFileStats()instead of a private constant.The original behaviour and limits are preserved for existing functions, as
ParseFile.Some cosmetic patches to parser are added, to comply with gno lint and format rules.
This is a possible alternative to #3974