Release 0.10 or later of Clozure CL uses a di erent memory management scheme than previous versions did. Those earlier versions would allocate a block of memory (of speci ed size) at startup and would allocate lisp objects within that block. When that block lled with live (non-GCed) objects, the lisp would signal a heap full condition. The heap size imposed a limit on the size of the largest object that could be allocated.
The new strategy involves reserving a very large (2GB on DarwinPPC32, 1GB on LinuxPPC, very large on 64-bit implementations) block at startup and consuming (and relinquishing) its contents as the size of the live lisp heap data grows and shrinks. After the initial heap image loads and after each full GC, the lisp kernel will try to ensure that a speci ed amount (the lisp-heap-gc-threshold ) of free memory is available. The initial value of this kernel variable is 16MB on 32-bit implementations and 32MB on 64-bit implementations; it can be manipulated from Lisp (see below.)
The large reserved memory block consumes very little in the way of system resources; memory that's actually committed to the lisp heap (live data and the threshold area where allocation takes place) consumes nite resources (physical memory and swap space). The lisp's consumption of those resources is proportional to its actual memory usage, which is generally a good thing.
This scheme is much more exible than the old one, but it may also increase the possibility that those resources can become exhausted. Neither the new scheme nor the old handles that situation gracefully; under the old scheme, a program that consumes lots of memory may have run into an arti cial limit on heap size before exhausting virtual memory.
The -R or --heap-reserve command-line option can be use to limit the size of the reserved block and therefore bound heap expansion. Running
would provide an execution environment that's very similar to that provided by earlier Clozure CL versions.
[Function]lisp-heap-gc-threshold
This function returns the value of the kernel variable that speci es the amount of free space to leave in the heap after full GC.
[Function]set-lisp-heap-gc-threshold new-threshold
This function sets the value of the kernel variable that speci es the amount of free space to leave in the heap after a full GC to new-threshold, which must be a non-negative xnum. Returns the new value of that kernel variable (which may be somewhat larger than what was speci ed).
[Function]use-lisp-heap-gc-threshold
Tries to grow or shrink lisp's heap space, so that the free space is (approximately) equal to the current heap threshold. Returns nil.
For many programs, the following observations are true to a very large degree:
Ephemeral (or generational) garbage collectors try to exploit these observations: by concentrating on frequently reclaiming newly-created objects quickly, it's less often necessary to do more expensive GCs of the entire heap in order to reclaim unreferenced memory. In some environments, the pauses associated with such full GCs can be noticeable and disruptive, and minimizing the frequency (and sometimes the duration) of these pauses is probably the EGC's primary goal (though there may be other bene ts, such as increased locality of reference and better paging behavior.) The EGC generally leads to slightly longer execution times (and slightly higher, amortized GC time), but there are cases where it can improve overall performance as well; the nature and degree of its impact on performance is highly application-dependent.
Most EGC strategies (including the one employed by Clozure CL) logically or physically divide memory into one or more areas of relatively young objects ( generations ) and one or more areas of old objects. Objects that have survived one or more GCs as members of a young generation are promoted (or tenured ) into an older generation, where they may or may not survive long enough to be promoted to the next generation and eventually may become old objects that can only be reclaimed if a full GC proves that there are no live references to them. This ltering process isn't perfect a certain amount of premature tenuring may take place but it usually works very well in practice.
It's important to note that a GC of the youngest generation is typically very fast (perhaps a few milliseconds on a modern CPU, depending on various factors), Clozure CL's EGC is not concurrent and doesn't o er realtime guarantees.
Clozure CL's EGC maintains three ephemeral generations; all newly created objects are created as members of the youngest generation. Each generation has an associated threshold, which indicates the number of bytes in it and all younger generations that can be allocated before a GC is triggered. These GCs will involve the target generation and all younger ones (and may therefore cause some premature tenuring); since the older generations have larger thresholds, they're GCed less frequently and most short-lived objects that make it into an older generation tend not to survive there very long.
The EGC can be enabled or disabled under program control; under some circumstances, it may be enabled but inactive (because a full GC is imminent.) Since it may be hard to know or predict the consing behavior of other threads, the distinction between the active and inactive state isn't very meaningful, especially when native threads are involved.
Enables the EGC if arg is non-nil, disables the EGC otherwise. Returns the previous enabled status.
Although this function is thread-safe (in the sense that calls to it are serialized), it doesn't make a whole lot of sense to be turning the EGC on and o from multiple threads...
Returns t if the EGC was enabled at the time of the call, and nil otherwise.
Returns t if the EGC was active at the time of the call, and nil otherwise. Since this is generally a volatile piece of information, it's not clear whether this function serves a useful purpose when native threads are involved.
Returns, as multiple values, the sizes in kilobytes of the thresholds associated with the youngest ephemeral generation, the middle ephemeral generation, and the oldest ephemeral generation.
[Function]configure-egc g0-size g1-size g2-size
Puts the indicated threshold sizes (speci ed in kilobytes) into e ect. Each threshold indicates the total size that may be allocated in that and all younger generations before a GC is triggered. Disables EGC while setting the values. The provided threshold sizes are rounded up to a multiple of 64 Kbytes.
After a full GC nishes, it'll try to ensure that at least (LISP-HEAP-GC-THRESHOLD) of virtual memory are available; objects will be allocated in this block of memory until it lls up, the GC is triggered, and the process repeats itself.
Many programs reach near stasis in terms of the amount of logical memory that's in use after full GC (or run for long periods of time in a nearly static state), so the logical address range used for consing after the Nth full GC is likely to be nearly or entirely identical to the address range used by the N + 1th full GC.
By default (and traditionally in Clozure CL), the GC's policy is to release the pages in this address range: to advise the virtual memory system that the pages contain garbage and any physical pages associated with them don't need to be swapped out to disk before being reused and to (re-)map the logical address range so that the pages will be zero- lled by the virtual memory system when they're next accessed. This policy is intended to reduce the load on the VM system and keep Clozure CL's working set to a minimum.
For some programs (especially those that cons at a very high rate), the default policy may be less than ideal: releasing pages that are going to be needed almost immediately and zero- ll-faulting them back in, lazily incurs unnecessary overhead. (There's a false economy associated with minimizing the size of the working set if it's just going to shoot back up again until the next GC.) A policy of retaining pages between GCs might work better in such an environment.
Functions described below give the user some control over this behavior. An adaptive, feedback-mediated approach might yield a better solution.
Tries to in uence the GC to retain/recycle the pages allocated between GCs if arg is true, and to release them otherwise. This is generally a tradeo between paging and other VM considerations.
Returns t if the GC tries to retain pages between full GCs and nil if it's trying to release them to improve VM paging performance.
SAVE-APPLICATION identi es code vectors and the pnames of interned symbols and copies these objects to a pure area of the image le it creates. (The pure area accounts for most of what the ROOM function reports as static space.)
When the resulting image le is loaded, the pure area of the le is now memory-mapped with read-only access. Code and pure data are paged in from the image le as needed (and don't compete for global virtual memory resources with other memory areas.)
Code-vectors and interned symbol pnames are immutable : it is an error to try to change the contents of such an object. Previously, that error would have manifested itself in some random way. In the new scheme, it'll manifest itself as an unhandled exception error in the Lisp kernel. The kernel could probably be made to detect a spurious, accidental write to read-only space and signal a lisp error in that case, but it doesn't yet do so.
The image le should be opened and/or mapped in some mode which disallows writing to the memory-mapped regions of the le from other processes. I'm not sure of how to do that; writing to the le when it's mapped by Clozure CL can have unpredictable and unpleasant results. SAVE-APPLICATION will delete its output le's directory entry and create a new le; one may need to exercise care when using le system utilities (like tar, for instance) that might overwrite an existing image le.
In general, a weak reference is a reference to an object which does not prevent the object from being garbage-collected. For example, suppose that you want to keep a list of all the objects of a certain type. If you don't take special steps, the fact that you have a list of them will mean that the objects are always live , because you can always reference them through the list. Therefore, they will never be garbage-collected, and their memory will never be reclaimed, even if they are referenced nowhere else in the program. If you don't want this behavior, you need weak references.
Clozure CL supports weak references with two kinds of objects: weak hash tables and populations.
Weak hash tables are created with the standard function make-hash-table, which is extended to accept the keyword argument :weak. Hash tables may be weak with respect to either their keys or their values. To make a hash table with weak keys, invoke make-hash-table with the option :weak t, or, equivalently, :weak :key. To make one with weak values, use :weak :value. When the key is weak, the equality test must be #’eq (because it wouldn't make sense otherwise).
When garbage collection occurs, key-value pairs are removed from the hash table if there are no non-weak references to the weak element of the pair (key or value).
In general, weak-key hash tables are useful when you want to use the hash to store some extra information about the objects you look up in it, while weak-value hash tables are useful when you want to use the hash as an index for looking up objects.
A population encapsulates an object, causing certain reference from the object to be considered weak. Clozure CL supports two kinds of populations: lists, in which case the encapsulated object is a list of elements, which are spliced out of the list when there are no non-weak references to the element; and alists, in which case the encapsulated object is a list of conses which are spliced out of the list if there are no non-weak references to the car of the cons.
If you are experimenting with weak references interactively, remember that an object is not dead if it was returned by one of the last three interactively-evaluated expressions, because of the variables *, **, and ***. The easy workaround is to evaluate some meaningless expression before invoking gc, to get the object out of the REPL variables.
[Function]make-population &key :type :initial-contents
This function creates and returns a new population. A population encapsulates an object, causing certain references from the object to be considered weak.
The :type argument determines what kind of population to create. Its value may be either :list (the default) or :alist.
The :initial-contents argument is a sequence of elements (or conses, for type :alist) to be used to initialize the population. The sequence itself (including the conses in the case of an alist) is not stored in the population; a new list or alist is created to hold the elements.
[Function]population-type population
This function returns the type of population, either :list or :alist.
[Function]population-contents population
This function returns the list encapsulated in population.
Note that as long as there is a direct (non-weak) reference to this list, it will not be modi ed by the garbage collector. Therefore it is safe to traverse the list, and even modify it, just as with any other list. If you want the elements of the list to become subject to garbage collection again, you must stop referring to the list directly.
setf may be used with population-contents to replace the list encapsulated by the population. The new list is not copied; it is used directly.