CEGUI+Custom memory allocator

[e3d_alive2]

greetings
i'v read in ogre roadmap from here http://www.cegui.org.uk/wiki/index.php/ ... nt_Roadmap that there shoud'v been custom memory allocator.
Has it been implemented in CEGUI, or is it still in plans?

[CrazyEddie]

That page is out of date and abandoned, and it's best not to take a lot of notice of what's put there

To answer the specific query, no this has not been implemented, and it's certainly not on my TODO list at the moment

CE.

[e3d_alive2]

so like last time[with d3d11 render], will anyone object if i implement it?
because it would be very nice if CEGUI had it

it will reduce amount of crashes becaue of debug/release dll and on subscriber as well
it may speed up memory allocations and raise memory requrements for cegui[using aligned memory]
i was thinking of implementing it either as overloading operator new or using placement new, whatever will work

[Kulik]

I would really like a custom memory allocator support. If you make it clean, generic and optional, I think it could make it into CEGUI (it's CE's call though )

[Jamarr]

i was thinking of implementing it either as overloading operator new or using placement new, whatever will work

If you are only implementing a custom allocator then I think you want operator-new as that is specifically for allocation, whereas placement-new is specifically for placement. You would not typically use placement-new except when you want an object constructed at a particular memory address, say if some other process/hardware expects a certain object at a specific address. It is important to keep a distinction between allocation and placement.

In any case, I would not recommend overloading operator new. According to the standard, you can only overload operator new in the global or class scope. Seeing as this would be specifically for CEGUI, you do not want to pollute the global scope. However there are far so many classes that overloading each class' operator new would be a maintenance nightmare. So you are basically left with two options: 1) replace all new/delete calls within CEGUI with calls to a CEGUI allocate/deallocate function/manager or 2) override operator new/delete on a base class and force all CEGUI classes to inherit from them.

Personally I like #1 because there is no coupling between the class and its allocator. The client is free to mix/match allocators based on the needs of the application or each individual class, and can be easily swapped with other allocators should those needs change. I believe the standard library also uses this approach.

I do like this idea of being able to easily implement custom allocation schemes. It can used to debug/trace allocation/deallocation bugs, implement memory pools, group related objects in memory, etc.

edit: moved this thread to the Library Development forum.

[e3d_alive2]

yes, thank you Jamarr i'm gonna go with 1 ) "replace all new/delete calls within CEGUI with calls to a CEGUI allocate/deallocate "
in my case i go with define CEGUI_NEW -> which will capture filename and line where new happens and CEGUI_DELETE
which will call custom operator new(sz,"test",__FILE__,__LINE__) and custom operator delete for it, maybe with template destructor
user can override it with his own allocation function

inheriting classes in CEGUI would be a good idea, except there are TOO MANY CLASSES[inherit them all will be a nightmare]

if you have any other sugestions i will be glad to hear them

[CrazyEddie]

Bit late to the party here, but yes, it's fine / good for you to implement this (thanks ), and I agree with Jamarr's suggested implementation (1).

Cheers,

CE.

[e3d_alive2]

here's my snippet

Code: Select all


class CEGUI_Allocator
{
public:
   CEGUI_Allocator(){};
   ~CEGUI_Allocator(){};
        void* Alloc(size_t sz,size_t Amount,CEGUI_MEM_Arr Creator,CEGUI_MEM_Arr Destroyer,char* FileName=NULL,long Line=0);
        bool Dealloc(void *obj);
} CEGUI_ALLOC;

inline void* operator new(size_t sz,CEGUI_Allocator &a,CEGUI_MEM_Arr Killer,char* FileName,long Line)
{
   return a.Alloc(sz,1,NULL,Killer,FileName,Line);
}

#define CEGUI_NEW(T) new(CEGUI_ALLOC,CEGUI_ArrKiller<T,1,1>,__FILE__,__LINE__) T
#define CEGUI_NEWARRAY(T,Amt) (T*)CEGUI_ALLOC.Alloc(sizeof(T),Amt,CEGUI_ArrCreator<T,Amt,sizeof(T)>,CEGUI_ArrKiller<T,Amt,sizeof(T)>,__FILE__,__LINE__)
#define CEGUI_DEL(obj) CEGUI_ALLOC.Dealloc(obj);
#define CEGUI_DELARRAY(obj) CEGUI_ALLOC.Dealloc(obj);


you use it like that

Code: Select all

    test1 *a=CEGUI_NEW(test1)(34);//where first is typename and second is constructor
    CEGUI_DEL(a);

    test1 *aa=CEGUI_NEWARRAY(test1,50);//where first is typename and the second is amount[constructor and destructor will be called for each element]
    CEGUI_DELARRAY(aa);


test1 is a test class with constructor and destructor that prints messages
i'm gonna integrate it into CEGUI via replacing new/delete with this[gonna be tricky, but i try my best]
waiting on comments on design[based it off ogre and my own]

debug structure for each var or entire array of vars is this

Code: Select all

typedef void (*CEGUI_MEM_Arr)(void*);
struct CEGUI_MemHDR
{
    char* FileName;//SHOULD BE REMOVED IN RELEASE
    long Line;////SHOULD BE REMOVED IN RELEASE
    CEGUI_MEM_Arr Killer;
};


eats 12 bytes, without Line and FileName it will eat even less[4]
p.s. class CEGUI_Allocator here is just for simplicity, it can be replaced with 2 functions for better integration

p.s.s. helpers

Code: Select all

template<class T,size_t count,size_t off>
inline void CEGUI_ArrCreator(void *obj)
{
   T* basePtr;//=(T*)obj;
   for (size_t i = 0; i < count; i++)
   {
       basePtr=(T*)((size_t)obj+i*off);
       new (basePtr) T();
   }
}

template<class T,size_t count,size_t off>
inline void CEGUI_ArrKiller(void *obj)
{
   T* basePtr;
   for (size_t i = 0; i < count; i++)
   {
      basePtr=(T*)((size_t)obj+i*off);
      basePtr->~T();
   }
}



edit: class updated to 2 functions instead of 3 already
edit: fixed issue with arrays
edit: compressed data with better templates
edit: more fixes

[e3d_alive2]

i'v integrated it in most of places, but there is still crash
i'm working on it

[Kulik]

What if CEGUI users were able to redefine CEGUI_NEW and CEGUI_DELETE (perhaps even CEGUI_MALLOC and CEGUI_DEALLOC for some other stuff) themselves and CEGUI coped with that? Users could delegate to Ogre's memory allocators or their own allocators.

To make this work, there must be a std::allocator inherited class that wraps these macros for STL. Sort of like this http://www.ogre3d.org/docs/api/html/Ogr ... ource.html but with CEGUI_MALLOC, etc... By default, CEGUI would define CEGUI_NEW as "new", etc...

This would make it much easier for you whilst making it much more powerful for developers because they could use anything they wanted, not just options CEGUI provides for them.

My idea how to do custom allocators (if they were already implemented).

In config.h (generated by ./configure):

Code: Select all

#define CEGUI_MEM_INCLUDE MyAwesomeMemoryAllocators.h
#define CEGUI_NEW(x) myNew(x)
#define CEGUI_DELETE(x) myDelete(x)


Then in CEGUIBase.h:

Code: Select all

#ifndef CEGUI_NEW
#define CEGUI_NEW new
#endif


etc...

[CrazyEddie]

I think the more generic; the more opportunities the user has to override or replace behaviours the better. I tend to favour this approach over trying to give the user "solutions to all their problems", since most of the time such solutions are a mile wide of the target. Or in other words, I like to facilitate the user hooking into things, while not necessarily providing anything other than a basic / example implementation

CE

[e3d_alive2]

Kulik i'l keep that in mind
the only big problem i'm currently facing is runtime crashes on the exit from CEGUI
+what to do with CEGUI script(lua) module, should i replace it, or keep it like it is, if' it's gonna be replaced by tolua++ anyway or where to modify tolua++ so it would print CEGUI_NEW(type,a) instead of new a?
+cegui::system and cegui::direct3d11render require destructors to be public in order to kill them with that allocator, should i make them public or is there any other way?
here's my current implementation[point me to an errors and other stuff in it]

Code: Select all

struct CEGUI_MemHDR
{
   char* FileName;//SHOULD BE REMOVED IN RELEASE
   long Line;////SHOULD BE REMOVED IN RELEASE
   CEGUI_MEM_Fnc Killer;
   size_t ElSz;//element size
   size_t NumEls;//num elements
};

const size_t CEGUI_MemHDR_sz=sizeof(CEGUI_MemHDR);


void *MyAlloc(size_t sz)
{
   void *o=malloc(sz);
   memset(o,0,sz);
   return o;
}

void MyDealloc(void *obj)
{
   free(obj);
}

CEGUI_MEM_UAlloc CEUserAlloc=MyAlloc;
CEGUI_MEM_UDealloc CEUserDealloc=MyDealloc;

int Allocs=0;


CEGUIEXPORT void* CEGUI_Alloc( size_t sz,size_t Amount,CEGUI_MEM_Fnc Creator,CEGUI_MEM_Fnc Destroyer,char* FileName/*=NULL*/,long Line/*=0*/ )
{
   if(sz==0 || Amount==0)
   {
      CEGUI_THROW("CEGUI_Alloc: Invalid Size Or Amount Is Specified!");
      return NULL;
   }
   size_t FullSize=sz*Amount+CEGUI_MemHDR_sz;
   CEGUI_MemHDR* AdminArea=(CEGUI_MemHDR*)malloc(FullSize);
   if(!AdminArea)
   {
      CEGUI_THROW("CEGUI_Alloc: Unable To Allocate Memory Of Size!"+FullSize);
      return NULL;
   }
   //memset(AdminArea,0,FullSize);
   void *UserArea=(void*)((size_t)AdminArea+CEGUI_MemHDR_sz);

   AdminArea->FileName=FileName;
   AdminArea->Line=Line;

   AdminArea->Killer=Destroyer;

   AdminArea->NumEls=Amount;
   AdminArea->ElSz=sz;
   if(Creator) Creator(UserArea,Amount,sz);
   return UserArea;
}

//TODO If ehc enabled do try catch here
CEGUIEXPORT bool CEGUI_Dealloc( void *obj )
{
   if(!obj) return false;
   CEGUI_MemHDR* AdminArea=(CEGUI_MemHDR*)((size_t)obj-CEGUI_MemHDR_sz);
   if(AdminArea->Killer) AdminArea->Killer(obj,AdminArea->NumEls,AdminArea->ElSz);
   free(AdminArea);
   return true;
}



[Kulik]

TBH I have no idea about low level allocators, so I doubt I will see any errors just by looking at the code unfortunately That's why I wanted to delegate allocation to other libraries or the user. Especially delegation to Ogre's optimised allocators sounds great to me...

If you are not interested in such an idea, I will code the patch myself... It could even work with your custom allocators, so it's a win win

[e3d_alive2]

here's more stable version of allocer
it writes debug dump into file[pretty simple but enough to identify who crashes]
btw CEGUI::String does like a 5000 allocations/deallocation[the leader of all] and may be greatly improved with mempool
http://rapidshare.com/files/424888479/TestAlloc2.patch
it requires windows.h, mostly for mutexes and locks
after some tests and time it will be removed

[e3d_alive2]

JUST Checked samples from svn and realize that i was't initializing cegui with awesome bootstrap, so with it no more leaks, hopefully there won't be any crashes either.