Module manager

source code

Manage resources.

We can attempt to control resource consumption by closing generators - the problem is which generators to close?

At first it seems that the answer is going to be connected to tree traversal, but after some thought it's not so clear exactly what tree is being traversed, and how that identifies what generators should be closed. In particular, an "imperative" implementation with generators does not have the same meaning of "depth" as a recursive functional implementation (but see the related discussion in the manual).

A better approach seems to be to discard those that have not been used "for a long time". A variation on this - keep a maximum number of the youngest generators - is practical. But care is needed to both in identifying what is used, and when it starts being unused, and in implementing that efficiently.

Here all generators are stored in a priority queue using weak references. The "real" priority is given by the "last used date" (epoch), but the priority in the queue is frozen when inserted. So on removing from the queue the priority must be checked to ensure it has not changed (and, if so, it must be updated with the real value and replaced).

Note that the main aim here is to restrict resource consumption without damaging performance too much. The aim is not to control parse results by excluding certain matches. For efficiency, the queue length is increased (doubled) whenever the queue is filled by active generators.