Package lepl :: Package core :: Module manager
[hide private]
[frames] | no frames]

Module manager

source code

Manage resources.

We can attempt to control resource consumption by closing generators - the problem is which generators to close?

At first it seems that the answer is going to be connected to tree traversal, but after some thought it's not so clear exactly what tree is being traversed, and how that identifies what generators should be closed. In particular, an "imperative" implementation with generators does not have the same meaning of "depth" as a recursive functional implementation (but see the related discussion in the manual).

A better approach seems to be to discard those that have not been used "for a long time". A variation on this - keep a maximum number of the youngest generators - is practical. But care is needed to both in identifying what is used, and when it starts being unused, and in implementing that efficiently.

Here all generators are stored in a priority queue using weak references. The "real" priority is given by the "last used date" (epoch), but the priority in the queue is frozen when inserted. So on removing from the queue the priority must be checked to ensure it has not changed (and, if so, it must be updated with the real value and replaced).

Note that the main aim here is to restrict resource consumption without damaging performance too much. The aim is not to control parse results by excluding certain matches. For efficiency, the queue length is increased (doubled) whenever the queue is filled by active generators.

Classes [hide private]
  _GeneratorManager
A 'Monitor' (implements MonitorInterface, can be supplied to Configuration) that tracks (and can limit the number of) generators.
  GeneratorRef
This contains the weak reference to the GeneratorWrapper and is stored in the GC priority queue.
Functions [hide private]
 
GeneratorManager(queue_len)
A 'Monitor' (implements MonitorInterface, can be supplied to Configuration) that tracks (and can limit the number of) generators.
source code
Function Details [hide private]

GeneratorManager(queue_len)

source code 

A 'Monitor' (implements MonitorInterface, can be supplied to Configuration) that tracks (and can limit the number of) generators. It is also coupled to the size of stacks during search (via the generator_manager_queue_len property).

This is a helper function that "escapes" the main class via a function to simplify configuration.