Get the latest tech news
Python extensions should be lazy
Python's memory model is a performance bottleneck
When I run this through Valgrind with the 500k LoC as input, the results show a massive number of memory allocations ( malloc) and a significant amount of time spent on garbage collection. In the case of ASTs, one could imagine a kind of ‘query language’ API for Python that operates on data that is owned by the extension - analogous to SQL over the highly specialized binary representations that a database would use. AST query engine diagramA compiled extension for Python is much faster when it can lazily build PyObjects only when necessary - keeping as much of the data compactly within its own language as possible.
Or read this on Hacker News