std.experimental.allocator - D Programming Language
Description: D Programming Language
Keywords: D programming language

Menu

- Learn
- Documentation
  - Language Reference
  - Library Reference
  - Command-line Reference
  - Feature Overview
  - Articles
- Packages
- Community
  - Blog
  - Orgs using D
  - Twitter
  - Forums
  - IRC
  - Wiki
  - GitHub
  - Issues
  - Foundation
  - Donate
- Resources
  - Books
  - Tutorials
  - Tools
  - Editors
  - IDEs
  - Visual D
  - Acknowledgments
  - D Style
  - Glossary
  - Sitemap

Search
Report a bug
If you spot a problem with this page, click here to create a Bugzilla issue.
Improve this page
Quickly fork, edit online, and submit a pull request for this page. Requires a signed-in GitHub account. This works well for small changes. If you'd like to make larger changes you may want to consider using a local clone.

# std.experimental.allocator

High-level interface for allocators. Implements bundled allocation/creation and destruction/deallocation of data including structs and classes, and also array primitives related to allocation. This module is the entry point for both making use of allocators and for their documentation.

Synopsis:

    // Allocate an int, initialize it with 42
    int* p = theAllocator.make!int(42);
    assert(*p == 42);
    // Destroy and deallocate it
    theAllocator.dispose(p);

    // Allocate using the global process allocator
    p = processAllocator.make!int(100);
    assert(*p == 100);
    // Destroy and deallocate
    processAllocator.dispose(p);

    // Create an array of 50 doubles initialized to -1.0
    double[] arr = theAllocator.makeArray!double(50, -1.0);
    // Append two zeros to it
    theAllocator.expandArray(arr, 2, 0.0);
    // On second thought, take that back
    theAllocator.shrinkArray(arr, 2);
    // Destroy and deallocate
    theAllocator.dispose(arr);

## Layered Structure

D's allocators have a layered structure in both implementation and documentation:

1. A high-level, dynamically-typed layer (described further down in this module). It consists of an interface called IAllocator, which concret; allocators need to implement. The interface primitives themselves are oblivious to the type of the objects being allocated; they only deal in void[], by necessity of the interface being dynamic (as opposed to type-parameterized). Each thread has a current `allocator` it uses by default, which is a thread-local variable theAllocator of type IAllocator. The process has a global allocator called processAllocator, also of type IAllocator. When a new thread is created, processAllocator is copied into theAllocator. An application can change the objects to which these references point. By default, at application startup, processAllocator refers to an object that uses D's garbage collected heap. This layer also include high-level functions such as make and dispose that comfortably allocate/create and respectively destroy/deallocate objects. This layer is all needed for most casual uses of allocation primitives.
2. A mid-level, statically-typed layer for assembling several allocators into one. It uses properties of the type of the objects being created to route allocation requests to possibly specialized allocators. This layer is relatively thin and implemented and documented in the std.experimental.allocator.typed module. It allows an interested user to e.g. use different allocators for arrays versus fixed-sized objects, to the end of better overall performance.
3. A low-level collection of highly generic heap building blocks— Lego-like pieces that can be used to assemble application-specific allocators. The real allocation smarts are occurring at this level. This layer is of interest to advanced applications that want to configure their own allocators. A good illustration of typical uses of these building blocks is module std.experimental.allocator.showcase which defines a collection of frequently- used preassembled `allocator` objects. The implementation and documentation entry point is std.experimental.allocator.building_blocks. By design, the primitives of the static interface have the same signatures as the IAllocator primitives but are for the most part optional and driven by static introspection. The parameterized class CAllocatorImpl offers an immediate and useful means to package a static low-level allocator into an implementation of IAllocator.
4. Core allocator objects that interface with D's garbage collected heap (std.experimental.allocator.gc_allocator), the C malloc family (std.experimental.allocator.mallocator), and the OS (std.experimental.allocator.mmap_allocator). Most custom allocators would ultimately obtain memory from one of these core allocators.

## Idiomatic Use of std.experimental.allocator

As of this time, std.experimental.allocator is not integrated with D's built-in operators that allocate memory, such as new, array literals, or array concatenation operators. That means std.experimental.allocator is opt-in—applications need to make explicit use of it.
For casual creation and disposal of dynamically-allocated objects, use make, dispose, and the array-specific functions makeArray, expandArray, and shrinkArray. These use by default D's garbage collected heap, but open the application to better configuration options. These primitives work either with theAllocator but also with any `allocator` obtained by combining heap building blocks. For example:

    void fun(size_t n)
    {
        // Use the current allocator
        int[] a1 = theAllocator.makeArray!int(n);
        scope(exit) theAllocator.dispose(a1);
        ...
    }

To experiment with alternative allocators, set theAllocator for the current thread. For example, consider an application that allocates many 8-byte objects. These are not well supported by the default allocator, so a free list allocator would be recommended. To install one in main, the application would use:

    void main()
    {
        import std.experimental.allocator.building_blocks.free_list
            : FreeList;
        theAllocator = allocatorObject(FreeList!8());
        ...
    }

### Saving the IAllocator Reference For Later Use

As with any global resource, setting theAllocator and processAllocator should not be done often and casually. In particular, allocating memory with one `allocator` and deallocating with another causes undefined behavior. Typically, these variables are set during application initialization phase and last through the application.
To avoid this, long-lived objects that need to perform allocations, reallocations, and deallocations relatively often may want to store a reference to the allocator object they use throughout their lifetime. Then, instead of using theAllocator for internal allocation-related tasks, they'd use the internally held reference. For example, consider a user-defined hash table:

    struct HashTable
    {
        private IAllocator _allocator;
        this(size_t buckets, IAllocator allocator = theAllocator) {
            this._allocator = allocator;
            ...
        }
        // Getter and setter
        IAllocator allocator() { return _allocator; }
        void allocator(IAllocator a) { assert(empty); _allocator = a; }
    }

Following initialization, the HashTable object would consistently use its allocator object for acquiring memory. Furthermore, setting HashTable.allocator to point to a different allocator should be legal but only if the object is empty; otherwise, the object wouldn't be able to deallocate its existing state.

### Using Allocators without IAllocator

Allocators assembled from the heap building blocks don't need to go through IAllocator to be usable. They have the same primitives as IAllocator and they work with make, makeArray, dispose etc. So it suffice to create `allocator` objects wherever fit and use them appropriately:

    void fun(size_t n)
    {
        // Use a stack-installed allocator for up to 64KB
        StackFront!65536 myAllocator;
        int[] a2 = myAllocator.makeArray!int(n);
        scope(exit) myAllocator.dispose(a2);
        ...
    }

In this case, myAllocator does not obey the IAllocator interface, but implements its primitives so it can work with makeArray by means of duck typing.
One important thing to note about this setup is that statically-typed assembled allocators are almost always faster than allocators that go through IAllocator. An important rule of thumb is: "assemble `allocator` first, adapt to IAllocator after". A good `allocator` implements intricate logic by means of template assembly, and gets wrapped with IAllocator (usually by means of allocatorObject) only once, at client level.
License:
Boost License 1.0.
Authors:
Andrei Alexandrescu

Source: std/experimental/allocator

interface `IAllocator`;
  Dynamic allocator interface. Code that defines allocators ultimately implements this interface. This should be used wherever a uniform type is required for encapsulating various allocator implementations.
  Composition of allocators is not recommended at this level due to inflexibility of dynamic interfaces and inefficiencies caused by cascaded multiple calls. Instead, compose allocators using the static interface defined in std.experimental.allocator.building_blocks, then adapt the composed allocator to `IAllocator` (possibly by using CAllocatorImpl below).
  Methods returning Ternary return Ternary.yes upon success, Ternary.no upon failure, and Ternary.unknown if the primitive is not implemented by the allocator instance.
  abstract @property uint `alignment`();
    Returns the `alignment` offered.
  abstract size_t `goodAllocSize`(size_t `s`);
    Returns the good allocation size that guarantees zero internal fragmentation.
  abstract void[] `allocate`(size_t, TypeInfo `ti` = null);
    Allocates n bytes of memory.
  abstract void[] `alignedAllocate`(size_t `n`, uint `a`);
    Allocates `n` bytes of memory with specified alignment `a`. Implementations that do not support this primitive should always return null.
  abstract void[] `allocateAll`();
    Allocates and returns all memory available to this allocator. Implementations that do not support this primitive should always return null.
  abstract bool `expand`(ref void[], size_t);
    Expands a memory block in place and returns true if successful. Implementations that don't support this primitive should always return false.
  abstract bool `reallocate`(ref void[], size_t);
    Reallocates a memory block.
  abstract bool `alignedReallocate`(ref void[] `b`, size_t `size`, uint `alignment`);
    Reallocates a memory block with specified `alignment`.
  abstract Ternary `owns`(void[] `b`);
    Returns Ternary.yes if the allocator `owns` `b`, Ternary.no if the allocator doesn't own `b`, and Ternary.unknown if ownership cannot be determined. Implementations that don't support this primitive should always return Ternary.unknown.
  abstract Ternary `resolveInternalPointer`(void* `p`, ref void[] `result`);
    Resolves an internal pointer to the full block allocated. Implementations that don't support this primitive should always return Ternary.unknown.
  abstract bool `deallocate`(void[] `b`);
    Deallocates a memory block. Implementations that don't support this primitive should always return false. A simple way to check that an allocator supports deallocation is to call `deallocate`(`null`).
  abstract bool `deallocateAll`();
    Deallocates all memory. Implementations that don't support this primitive should always return false.
  abstract Ternary `empty`();
    Returns Ternary.yes if no memory is currently allocated from this allocator, Ternary.no if some allocations are currently active, or Ternary.unknown if not supported.
nothrow @nogc @property @safe IAllocator `theAllocator`();
nothrow @nogc @property @safe void `theAllocator`(IAllocator `a`);
  Gets/sets the allocator for the current thread. This is the default allocator that should be used for allocating thread-local memory. For allocating memory to be shared across threads, use processAllocator (below). By default, `theAllocator` ultimately fetches memory from processAllocator, which in turn uses the garbage collected heap.
  Examples:

      // Install a new allocator that is faster for 128-byte allocations.
      import std.experimental.allocator.building_blocks.free_list : FreeList;
      import std.experimental.allocator.gc_allocator : GCAllocator;
      auto oldAllocator = theAllocator;
      scope(exit) theAllocator = oldAllocator;
      theAllocator = allocatorObject(FreeList!(GCAllocator, 128)());
      // Use the now changed allocator to allocate an array
      const ubyte[] arr = theAllocator.makeArray!ubyte(128);
      assert(arr.ptr);
      //...

@property IAllocator `processAllocator`();
@property void `processAllocator`(IAllocator `a`);
  Gets/sets the allocator for the current process. This allocator must be used for allocating memory shared across threads. Objects created using this allocator can be cast to shared.
auto `make`(T, Allocator, A...)(auto ref Allocator `alloc`, auto ref A `args`);
  Dynamically allocates (using `alloc`) and then creates in the memory allocated an object of type T, using `args` (if any) for its initialization. Initialization occurs in the memory allocated and is otherwise semantically the same as T(`args`). (Note that using `alloc`.`make`!(T[]) creates a pointer to an (empty) array of Ts, not an array. To use an allocator to allocate and initialize an array, use `alloc`.makeArray!T described below.)
  Parameters:

  T | Type of the object being created.
  Allocator `alloc` | The allocator used for getting the needed memory. It may be an object implementing the static interface for allocators, or an IAllocator reference.
  A `args` | Optional arguments used for initializing the created object. If not present, the object is default constructed.

  Returns:
  If T is a class type, returns a reference to the created T object. Otherwise, returns a T* pointing to the created object. In all cases, returns `null` if allocation failed.
  Throws:
  If T's constructor throws, deallocates the allocated memory and propagates the exception.
  Examples:

      // Dynamically allocate one integer
      const int* p1 = theAllocator.make!int;
      // It's implicitly initialized with its .init value
      assert(*p1 == 0);
      // Dynamically allocate one double, initialize to 42.5
      const double* p2 = theAllocator.make!double(42.5);
      assert(*p2 == 42.5);

      // Dynamically allocate a struct
      static struct Point
      {
          int x, y, z;
      }
      // Use the generated constructor taking field values in order
      const Point* p = theAllocator.make!Point(1, 2);
      assert(p.x == 1 && p.y == 2 && p.z == 0);

      // Dynamically allocate a class object
      static class Customer
      {
          uint id = uint.max;
          this() {}
          this(uint id) { this.id = id; }
          // ...
      }
      Customer cust = theAllocator.make!Customer;
      assert(cust.id == uint.max); // default initialized
      cust = theAllocator.make!Customer(42);
      assert(cust.id == 42);

      // explicit passing of outer pointer
      static class Outer
      {
          int x = 3;
          class Inner
          {
              auto getX() { return x; }
          }
      }
      auto outer = theAllocator.make!Outer();
      auto inner = theAllocator.make!(Outer.Inner)(outer);
      assert(outer.x == inner.getX);

T[] `makeArray`(T, Allocator)(auto ref Allocator `alloc`, size_t `length`);
T[] `makeArray`(T, Allocator)(auto ref Allocator `alloc`, size_t `length`, auto ref T `init`);
Unqual!(ElementEncodingType!R)[] `makeArray`(Allocator, R)(auto ref Allocator `alloc`, R `range`)
if (isInputRange!R && !isInfinite!R);
T[] `makeArray`(T, Allocator, R)(auto ref Allocator `alloc`, R `range`)
if (isInputRange!R && !isInfinite!R);
  Create an array of T with `length` elements using `alloc`. The array is either default-initialized, filled with copies of `init`, or initialized with values fetched from `range`.
  Parameters:

  T | element type of the array being created
  Allocator `alloc` | the allocator used for getting memory
  size_t `length` | `length` of the newly created array
  T `init` | element used for filling the array
  R `range` | `range` used for initializing the array elements

  Returns:
  The newly-created array, or `null` if either `length` was 0 or allocation failed.
  Throws:
  The first two overloads throw only if `alloc`'s primitives do. The overloads that involve copy initialization deallocate memory and propagate the exception if the copy operation throws.
  Examples:

      import std.algorithm.comparison : equal;
      static void test(T)()
      {
          T[] a = theAllocator.makeArray!T(2);
          assert(a.equal([0, 0]));
          a = theAllocator.makeArray!T(3, 42);
          assert(a.equal([42, 42, 42]));
          import std.range : only;
          a = theAllocator.makeArray!T(only(42, 43, 44));
          assert(a.equal([42, 43, 44]));
      }
      test!int();
      test!(shared int)();
      test!(const int)();
      test!(immutable int)();

bool `expandArray`(T, Allocator)(auto ref Allocator `alloc`, ref T[] `array`, size_t `delta`);
bool `expandArray`(T, Allocator)(auto ref Allocator `alloc`, ref T[] `array`, size_t `delta`, auto ref T `init`);
bool `expandArray`(T, Allocator, R)(auto ref Allocator `alloc`, ref T[] `array`, R `range`)
if (isInputRange!R);
  Grows `array` by appending `delta` more elements. The needed memory is allocated using `alloc`. The extra elements added are either default- initialized, filled with copies of `init`, or initialized with values fetched from `range`.
  Parameters:

  T | element type of the `array` being created
  Allocator `alloc` | the allocator used for getting memory
  T[] `array` | a reference to the `array` being grown
  size_t `delta` | number of elements to add (upon success the new length of `array` is `array`.length + `delta`)
  T `init` | element used for filling the `array`
  R `range` | `range` used for initializing the `array` elements

  Returns:
  `true` upon success, `false` if memory could not be allocated. In the latter case `array` is left unaffected.
  Throws:
  The first two overloads throw only if `alloc`'s primitives do. The overloads that involve copy initialization deallocate memory and propagate the exception if the copy operation throws.
  Examples:

      auto arr = theAllocator.makeArray!int([1, 2, 3]);
      assert(theAllocator.expandArray(arr, 2));
      assert(arr == [1, 2, 3, 0, 0]);
      import std.range : only;
      assert(theAllocator.expandArray(arr, only(4, 5)));
      assert(arr == [1, 2, 3, 0, 0, 4, 5]);

bool `shrinkArray`(T, Allocator)(auto ref Allocator `alloc`, ref T[] `array`, size_t `delta`);
  Shrinks an `array` by `delta` elements.
  If `array`.length < `delta`, does nothing and returns false. Otherwise, destroys the last `array`.length - `delta` elements in the `array` and then reallocates the `array`'s buffer. If reallocation fails, fills the `array` with default-initialized data.
  Parameters:

  T | element type of the `array` being created
  Allocator `alloc` | the allocator used for getting memory
  T[] `array` | a reference to the `array` being shrunk
  size_t `delta` | number of elements to remove (upon success the new length of `array` is `array`.length - `delta`)

  Returns:
  true upon success, false if memory could not be reallocated. In the latter case, the slice `array`[$ - `delta` .. $] is left with default-initialized elements.
  Throws:
  The first two overloads throw only if `alloc`'s primitives do. The overloads that involve copy initialization deallocate memory and propagate the exception if the copy operation throws.
  Examples:

      int[] a = theAllocator.makeArray!int(100, 42);
      assert(a.length == 100);
      assert(theAllocator.shrinkArray(a, 98));
      assert(a.length == 2);
      assert(a == [42, 42]);

void `dispose`(A, T)(auto ref A `alloc`, T* `p`);
void `dispose`(A, T)(auto ref A `alloc`, T `p`)
if (is(T == class) || is(T == interface));
void `dispose`(A, T)(auto ref A `alloc`, T[] `array`);
  Destroys and then deallocates (using `alloc`) the object pointed to by a pointer, the class object referred to by a class or interface reference, or an entire `array`. It is assumed the respective entities had been allocated with the same allocator.
auto `makeMultidimensionalArray`(T, Allocator, size_t N)(auto ref Allocator `alloc`, size_t[N] `lengths`...);
  Allocates a multidimensional array of elements of type T.
  Parameters:

  N | number of dimensions
  T | element type of an element of the multidimensional arrat
  Allocator `alloc` | the allocator used for getting memory
  size_t[N] `lengths` | static array containing the size of each dimension

  Returns:
  An N-dimensional array with individual elements of type T.
  Examples:

      import std.experimental.allocator.mallocator : Mallocator;

      auto mArray = Mallocator.instance.makeMultidimensionalArray!int(2, 3, 6);

      // deallocate when exiting scope
      scope(exit)
      {
          Mallocator.instance.disposeMultidimensionalArray(mArray);
      }

      assert(mArray.length == 2);
      foreach (lvl2Array; mArray)
      {
          assert(lvl2Array.length == 3);
          foreach (lvl3Array; lvl2Array)
              assert(lvl3Array.length == 6);
      }

void `disposeMultidimensionalArray`(T, Allocator)(auto ref Allocator `alloc`, T[] `array`);
  Destroys and then deallocates a multidimensional `array`, assuming it was created with makeMultidimensionalArray and the same allocator was used.
  Parameters:

  T | element type of an element of the multidimensional `array`
  Allocator `alloc` | the allocator used for getting memory
  T[] `array` | the multidimensional `array` that is to be deallocated

  Examples:

      struct TestAllocator
      {
          import std.experimental.allocator.common : platformAlignment;
          import std.experimental.allocator.mallocator : Mallocator;

          alias allocator = Mallocator.instance;

          private static struct ByteRange
          {
              void* ptr;
              size_t length;
          }

          private ByteRange[] _allocations;

          enum uint alignment = platformAlignment;

          void[] allocate(size_t numBytes)
          {
               auto ret = allocator.allocate(numBytes);
               _allocations ~= ByteRange(ret.ptr, ret.length);
               return ret;
          }

          bool deallocate(void[] bytes)
          {
              import std.algorithm.mutation : remove;
              import std.algorithm.searching : canFind;

              bool pred(ByteRange other)
              { return other.ptr == bytes.ptr && other.length == bytes.length; }

              assert(_allocations.canFind!pred);

               _allocations = _allocations.remove!pred;
               return allocator.deallocate(bytes);
          }

          ~this()
          {
              assert(!_allocations.length);
          }
      }

      TestAllocator allocator;

      auto mArray = allocator.makeMultidimensionalArray!int(2, 3, 5, 6, 7, 2);

      allocator.disposeMultidimensionalArray(mArray);

CAllocatorImpl!A `allocatorObject`(A)(auto ref A `a`)
if (!isPointer!A);
CAllocatorImpl!(A, Yes.indirect) `allocatorObject`(A)(A* `pa`);
  Returns `a` dynamically-typed CAllocator built around `a` given statically- typed allocator `a` of type A. Passing `a` pointer to the allocator creates `a` dynamic allocator around the allocator pointed to by the pointer, without attempting to copy or move it. Passing the allocator by value or reference behaves as follows.
  - If A has no state, the resulting object is allocated in static shared storage.
  - If A has state and is copyable, the result will store `a` copy of it within. The result itself is allocated in its own statically-typed allocator.
  - If A has state and is not copyable, the result will move the passed-in argument into the result. The result itself is allocated in its own statically-typed allocator.
  Examples:

      import std.experimental.allocator.mallocator : Mallocator;
      IAllocator a = allocatorObject(Mallocator.instance);
      auto b = a.allocate(100);
      assert(b.length == 100);
      assert(a.deallocate(b));

      // The in-situ region must be used by pointer
      import std.experimental.allocator.building_blocks.region : InSituRegion;
      auto r = InSituRegion!1024();
      a = allocatorObject(&r);
      b = a.allocate(200);
      assert(b.length == 200);
      // In-situ regions can deallocate the last allocation
      assert(a.deallocate(b));

class `CAllocatorImpl`(Allocator, Flag!"indirect" indirect = No.indirect): IAllocator;
  Implementation of IAllocator using Allocator. This adapts a statically-built allocator type to IAllocator that is directly usable by non-templated code.
  Usually `CAllocatorImpl` is used indirectly by calling theAllocator.
  ref Allocator `impl`();
    The implementation is available as a public member.
  this(Allocator* `pa`);
    The implementation is available as a public member.
  @property uint `alignment`();
    Returns impl.`alignment`.
  size_t `goodAllocSize`(size_t `s`);
    Returns impl.`goodAllocSize`(`s`).
  void[] `allocate`(size_t `s`, TypeInfo `ti` = null);
    Returns impl.`allocate`(`s`).
  void[] `alignedAllocate`(size_t `s`, uint `a`);
    If impl.`alignedAllocate` exists, calls it and returns the result. Otherwise, always returns null.
  Ternary `owns`(void[] `b`);
    If Allocator implements `owns`, forwards to it. Otherwise, returns Ternary.unknown.
  bool `expand`(ref void[] `b`, size_t `s`);
    Returns impl.`expand`(`b`, `s`) if defined, `false` otherwise.
  bool `reallocate`(ref void[] `b`, size_t `s`);
    Returns impl.`reallocate`(`b`, `s`).
  bool `alignedReallocate`(ref void[] `b`, size_t `s`, uint `a`);
    Forwards to impl.`alignedReallocate`.
  bool `deallocate`(void[] `b`);
    If impl.`deallocate` is not defined, returns Ternary.unknown. If impl.`deallocate` returns void (the common case), calls it and returns Ternary.yes. If impl.`deallocate` returns bool, calls it and returns Ternary.yes for `true`, Ternary.no for `false`.
  bool `deallocateAll`();
    Calls impl.`deallocateAll`() and returns Ternary.yes if defined, otherwise returns Ternary.unknown.
  Ternary `empty`();
    Forwards to impl.`empty`() if defined, otherwise returns Ternary.unknown.
  void[] `allocateAll`();
    Returns impl.`allocateAll`() if present, `null` otherwise.

Copyright © 1999-2017 by the D Language Foundation | Page generated by Ddoc on Mon Apr 10 10:54:47 2017