The Mallocator

The Mallocator

  • Comments 23

A common question from programmers who have an intermediate amount of experience with using the STL is, "How do I write an STL allocator?".  Writing an STL allocator is not especially difficult - only two member functions are interesting, allocate() and deallocate().  However, STL allocators must satisfy a number of other requirements (given by section 20.1.5 of the International Standard for C++, ISO/IEC 14882:2003), and the code to do so takes roughly 80 editor lines.  Figuring out the code from the requirements can be overwhelming, but once you see the code, it's easy.

 

One thing some programmers try is to derive from std::allocator.  I recommend against this; it's more trouble than it's worth.  Looking at std::allocator's implementation is also painful.

 

Therefore, I've written an example STL allocator, whose purpose in life is to wrap malloc() and free(), which I've imaginatively called Mallocator.  I've carefully implemented all of the integer overflow checks and so forth that would be required in real production code.  And I've exhaustively commented which parts of Mallocator are boilerplate (common to all, virtually all, or all stateless allocators), and which parts you would have to customize.  Hopefully, this should demystify the implementation of STL allocators:

 

 

C:\Temp>type mallocator.cpp

// The following headers are required for all allocators.

#include <stddef.h>  // Required for size_t and ptrdiff_t and NULL

#include <new>       // Required for placement new and std::bad_alloc

#include <stdexcept> // Required for std::length_error

 

// The following headers contain stuff that Mallocator uses.

#include <stdlib.h>  // For malloc() and free()

#include <iostream>  // For std::cout

#include <ostream>   // For std::endl

 

// The following headers contain stuff that main() uses.

#include <list>      // For std::list

 

template <typename T> class Mallocator {

public:

 

    // The following will be the same for virtually all allocators.

    typedef T * pointer;

    typedef const T * const_pointer;

    typedef T& reference;

    typedef const T& const_reference;

    typedef T value_type;

    typedef size_t size_type;

    typedef ptrdiff_t difference_type;

 

    T * address(T& r) const {

        return &r;

    }

 

    const T * address(const T& s) const {

        return &s;

    }

 

    size_t max_size() const {

        // The following has been carefully written to be independent of

        // the definition of size_t and to avoid signed/unsigned warnings.

        return (static_cast<size_t>(0) - static_cast<size_t>(1)) / sizeof(T);

    }

 

 

    // The following must be the same for all allocators.

    template <typename U> struct rebind {

        typedef Mallocator<U> other;

    };

 

    bool operator!=(const Mallocator& other) const {

        return !(*this == other);

    }

 

    void construct(T * const p, const T& t) const {

        void * const pv = static_cast<void *>(p);

 

        new (pv) T(t);

    }

 

    void destroy(T * const p) const; // Defined below.

 

 

    // Returns true if and only if storage allocated from *this

    // can be deallocated from other, and vice versa.

    // Always returns true for stateless allocators.

    bool operator==(const Mallocator& other) const {

        return true;

    }

 

 

    // Default constructor, copy constructor, rebinding constructor, and destructor.

    // Empty for stateless allocators.

    Mallocator() { }

 

    Mallocator(const Mallocator&) { }

 

    template <typename U> Mallocator(const Mallocator<U>&) { }

 

    ~Mallocator() { }

 

 

    // The following will be different for each allocator.

    T * allocate(const size_t n) const {

        // Mallocator prints a diagnostic message to demonstrate

        // what it's doing. Real allocators won't do this.

        std::cout << "Allocating " << n << (n == 1 ? " object" : "objects")

            << " of size " << sizeof(T) << "." << std::endl;

 

        // The return value of allocate(0) is unspecified.

        // Mallocator returns NULL in order to avoid depending

        // on malloc(0)'s implementation-defined behavior

        // (the implementation can define malloc(0) to return NULL,

        // in which case the bad_alloc check below would fire).

        // All allocators can return NULL in this case.

        if (n == 0) {

            return NULL;

        }

 

        // All allocators should contain an integer overflow check.

        // The Standardization Committee recommends that std::length_error

        // be thrown in the case of integer overflow.

        if (n > max_size()) {

            throw std::length_error("Mallocator<T>::allocate() - Integer overflow.");

        }

 

        // Mallocator wraps malloc().

        void * const pv = malloc(n * sizeof(T));

 

        // Allocators should throw std::bad_alloc in the case of memory allocation failure.

        if (pv == NULL) {

            throw std::bad_alloc();

        }

 

        return static_cast<T *>(pv);

    }

 

    void deallocate(T * const p, const size_t n) const {

        // Mallocator prints a diagnostic message to demonstrate

        // what it's doing. Real allocators won't do this.

        std::cout << "Deallocating " << n << (n == 1 ? " object" : "objects")

            << " of size " << sizeof(T) << "." << std::endl;

 

        // Mallocator wraps free().

        free(p);

    }

 

 

    // The following will be the same for all allocators that ignore hints.

    template <typename U> T * allocate(const size_t n, const U * /* const hint */) const {

        return allocate(n);

    }

 

 

    // Allocators are not required to be assignable, so

    // all allocators should have a private unimplemented

    // assignment operator. Note that this will trigger the

    // off-by-default (enabled under /Wall) warning C4626

    // "assignment operator could not be generated because a

    // base class assignment operator is inaccessible" within

    // the STL headers, but that warning is useless.

private:

    Mallocator& operator=(const Mallocator&);

};

 

// A compiler bug causes it to believe that p->~T() doesn't reference p.

 

#ifdef _MSC_VER

    #pragma warning(push)

    #pragma warning(disable: 4100) // unreferenced formal parameter

#endif

 

// The definition of destroy() must be the same for all allocators.

template <typename T> void Mallocator<T>::destroy(T * const p) const {

    p->~T();

}

 

#ifdef _MSC_VER

    #pragma warning(pop)

#endif

 

 

int main() {

    using namespace std;

 

    cout << "Constructing l:" << endl;

 

    list<int, Mallocator<int> > l;

 

    cout << endl << "l.push_back(1729):" << endl;

 

    l.push_back(1729);

 

    cout << endl << "l.push_back(2161):" << endl;

 

    l.push_back(2161);

 

    cout << endl;

 

    for (list<int, Mallocator<int> >::const_iterator i = l.begin(); i != l.end(); ++i) {

        cout << "Element: " << *i << endl;

    }

 

    cout << endl << "Destroying l:" << endl;

}

 

C:\Temp>cl /EHsc /nologo /W4 mallocator.cpp

mallocator.cpp

 

C:\Temp>mallocator

Constructing l:

Allocating 1 object of size 4.

Allocating 1 object of size 12.

 

l.push_back(1729):

Allocating 1 object of size 12.

 

l.push_back(2161):

Allocating 1 object of size 12.

 

Element: 1729

Element: 2161

 

Destroying l:

Deallocating 1 object of size 12.

Deallocating 1 object of size 12.

Deallocating 1 object of size 12.

Deallocating 1 object of size 4.

 

 

To explain this output:

 

The object of size 4 is the aux object

 

The objects of size 12 are doubly-linked list nodes.

 

In VC's implementation, default-constructed lists allocate a sentinel node, which is where end iterators point.  (End iterators have to be decrementable, so they can't point at NULL.)

 

 

Stephan T. Lavavej

Visual C++ Libraries Developer

  • I did connect with some support, and have sent details.  I referenced the blog to them so they may locate your team...

    ciao

  • >> The memset() call looks OK.

    > Only because it->a happened to refer to an array, and the name of an array decays into a pointer to its first element. [...] Either way, casting to void * the first argument to memset() is exceedingly dangerous and buys you nothing.

    I agree 100% except that some sort of cast would be necessary in this case to at least cast away const-ness.  I meant 'OK' as in 'no bug here', not as in 'good stuff'.

  • I am just wondering why doesn't Microsoft release a set of Allocators that are most common with Visual Studio - for instance, a memory pool allocator.

    Do Boost have special allocators?

  • Boost has a pool allocator library, but no other special allocators that I'm aware of.

  • re: std lib names

    > I used to be very careful about that. [...] In reality and in C++0x, including <cfoo> is no safeguard against everything getting declared in the global namespace anyways.

    I can follow you up to this point. However, this

    > That's why I'm ceasing to bother with <cfoo>.

    doesn't seem to follow from the above.

    In C++, those headers are indeed named <cstddef> etc. Yes, one can get away with using the old C names and not using 'std', but I don't see what advantage it offers. On the other hand it does come with some disadvantages. ("Oh, this file seems to be pretty old, predating C++89!" <goes off and spends half an hour hunting for possible problems> Or it's just plain confusing to mix the header name styles.)

    What's your rationale for using the old header names? Is there any besides "I've been doing this for three decades before they came and changed it!" (which isn't really a valid one, IMHO)?

    Thanks for the code, BTW!

    Schobi

  • [Tom]

    > I am just wondering why doesn't Microsoft release a set of Allocators that are most common with Visual Studio - for instance, a memory pool allocator.

    Have you seen the Dinkum Allocators Library at http://www.dinkumware.com/manuals/default.aspx?manual=compleat&page=index_alloc.html ?

    [Hendrik Schober]

    > What's your rationale for using the old header names?

    There's no difference in cleanliness between <cstddef> and <stddef.h>. (As I explained, there was supposed to be a difference, but it never materialized, and C++0x gave up trying.) Therefore, it's a style decision. <stddef.h> is visually distinct from C++ headers like <vector> and <regex>. I alphabetically sort my headers, after sorting them by group (C/C++/Boost/etc.), and having the C headers look different from the C++ headers is nice (because they certainly contain different guts).

    Like I said, I used to use <cstddef>, but now I'm switching to use <stddef.h>.

    > Is there any besides "I've been doing this for three decades before they came and changed it!" (which isn't really a valid one, IMHO)?

    I'm not even three decades old. I began learning C in mid-2000 and C++ in early 2002.

  • Wow, thanks alot for the information. This is exactly what I was looking for and your comments are great.

  • Hmm.. hi again.

    Aren't throw() specifications required on most of the methods?

    And how would you write the above if it had two template parameters ie. where either the first or the second template param would make allocator comparison etc incorrect? (ie. not mixing well semantically or in possible implementation/internal usage).

    Cheers..

    M/[e]/K

Page 2 of 2 (23 items) 12