#pentagram@irc.freenode.net logs for 6 Mar 2005 (GMT)

Archive Today Yesterday Tomorrow
Pentagram homepage

[00:11:29] <-- sbx has left IRC ("BRB")
[00:20:09] --> pF-SBX has joined #pentagram
[00:20:18] --- pF-SBX is now known as sbx
[03:37:36] <-- Chetic has left IRC (Read error: 104 (Connection reset by peer))
[03:38:00] --> Chetic has joined #pentagram
[04:25:53] --> sbx|afk has joined #pentagram
[04:26:21] <-- sbx|afk has left IRC (Client Quit)
[04:29:39] --> Colourless has joined #Pentagram
[04:29:39] --- ChanServ gives channel operator status to Colourless
[04:33:12] --> EsBee-Eks has joined #pentagram
[04:44:05] <-- sbx has left IRC (Read error: 110 (Connection timed out))
[05:06:00] <-- EsBee-Eks has left IRC ("casts gate travel")
[10:47:31] --> Fl00der has joined #pentagram
[11:38:37] <-- Fl00der has left IRC ()
[11:49:14] <-- Lightkey has left IRC (Read error: 60 (Operation timed out))
[11:57:31] --> Lightkey has joined #Pentagram
[12:58:21] <-- Kirben has left IRC ("System Meltdown")
[16:16:39] * watt has been thinking.. which is never a good sign... would it be a good idea to pool memory for processes an gumps?
[16:19:38] <Colourless> no idea
[16:19:46] <Colourless> do we need to do that?
[16:20:46] <Colourless> i'd say it would most likely just end up causing problems
[16:23:22] <Colourless> but if done right it would reduce memory fragmentation over a long session
[16:23:46] <Colourless> and possibly spead the engine up
[16:25:13] <watt> speed would be the underlying motivation. And naturally, it'd have to be tested heavily.
[16:25:55] <Colourless> pooled item memory could be a larger speed gain
[16:26:04] <watt> I think it might be a good idea.. and seeing that it's c++, it's not too incredibly had to implement
[16:26:35] <watt> good point.... at least better than gumps...
[16:26:47] <Colourless> and arguably easier too
[16:27:17] <Colourless> most items all have the same size memory foot print
[16:27:26] <Colourless> could just allocate a hunk of memory
[16:29:10] <Colourless> would be pretty easy to actually do... like 'really' easy
[16:29:54] <Colourless> just need to ensure call the correct delete operator when the item is deleted
[16:32:52] <Colourless> might have something like an ItemAllocater class
[16:33:09] <Colourless> would allocate say a meg of memory
[16:33:10] <watt> hardest bit may be making sure its not actually slower than new ;-)
[16:33:28] <Colourless> i think i could create a quick implementation
[16:33:36] <Colourless> unused memory blocks would be a linked list
[16:34:20] <watt> I was thinking a MemManager having specific Allocators with Pools of a meg a piece in a vector.
[16:34:44] <watt> yeah... a list would be good.
[16:34:57] <watt> .. for the unused pieces
[16:36:01] <Colourless> an allocator for UCProcesses and Items would be a good thing
[16:37:19] <Colourless> other things i don't think it would be necessary
[16:37:35] <Colourless> maybe NPCs
[16:38:33] * watt thinks the new operator would inherit properly... covering Item and Process properly may cover them all.
[16:38:57] <Colourless> could also have a memory pool all fixed items
[16:39:10] <Colourless> so don't fragment memory too badly when changing maps
[16:40:08] <Colourless> since all the fixed items get deleted at map change, can just reuse the pool from the beginning on map change
[16:44:38] <Colourless> thinking delete would look somehting like this
[16:45:05] <Colourless> no wait.. just thinking
[16:45:19] <watt> hmmm... I though quake2 pooled... quess not so much... they used malloc but added a linked list node to the front of the memory and chained all allocated memory together.
[16:45:36] <watt> * thought
[16:46:24] <watt> interesting way to avoid leaking.
[16:46:39] <Colourless> quake 2 did pool memory
[16:46:47] <Colourless> though in a really basic fashion
[16:46:56] <Colourless> on map change it like just cleared everything
[16:47:03] <Colourless> and reloaded everything in it's memory pools
[16:48:11] <watt> ohh.... they could simply walk the list and see what was "Free" and use it if they needed it.
[16:49:22] <wjp> easiest might be to use a different Allocator for the object/process maps in ObjectManager/Kernel
[16:50:00] <wjp> although I don't know how messy that would become if we want to only handle Item/UCProcess differently
[16:50:09] <wjp> hm, dinner; bbl
[16:52:37] <Colourless> deleteing an Item would work something like this...
[16:52:42] <Colourless> if (item->memorypool & FIXED_POOL) FixedAllocator::delete item;
[16:52:44] <Colourless> else if (item->memorypool & ITEM_POOL) ItemAllocator::delete item;
[16:52:45] <Colourless> else delete item;
[16:54:01] <Colourless> thouse & should really be =
[16:54:06] <watt> yup
[16:54:12] <Colourless> == even
[16:55:00] <Colourless> and on map change might have something like FixedAllocater::reset()
[16:55:13] <watt> although... is that overloading operator delete?
[16:56:18] <Colourless> i am not entirely sure... need to read up on the workings of operator delete
[16:56:28] <watt> if so... the fallback is malloc on new and free on delete
[17:07:50] <Colourless> well, i think i'll test it out (sort of)
[17:08:01] <watt> already?!?!?
[17:08:12] <Colourless> no :-)
[17:08:17] <Colourless> just testing the overloading
[17:08:20] <watt> hehe.
[17:08:27] <Colourless> create a dummy class
[17:09:04] <Colourless> see if it works
[17:09:27] <watt> I would just overload new and delete on item.. tie the to malloc and delete and spit console messages out (which would be oh so plentiful)
[17:12:57] <Colourless> it didn't like this
[17:12:58] <Colourless> DeleteClass::delete this; // delete self.
[17:13:10] <Colourless> d:\Pentagram\world\Item.cpp(1133): error C2834: 'operator delete' must be globally qualified
[17:14:31] <Colourless> so i made this: class DeleteClass
[17:14:31] <Colourless> {
[17:14:31] <Colourless> public:
[17:14:31] <Colourless> static void operator delete (void * p)
[17:14:31] <Colourless> {
[17:14:32] <Colourless> pout << "Deleting: " << p << std::endl;
[17:14:34] <Colourless> ::operator delete (p);
[17:14:36] <Colourless> }
[17:14:38] <Colourless> static void DeleteItem(Item *p) { delete p; }
[17:14:40] <Colourless> };
[17:14:42] <Colourless> and am using
[17:14:44] <Colourless> DeleteClass::DeleteItem(this); // delete self.
[17:15:44] <watt> ah yea... static... hmm
[17:22:53] <watt> ... wow.. that's a lot of calls the new....
[17:24:12] <watt> trying to think if theres a good way to get member variables... seeing that the operators are static, it causes a small problem
[17:24:30] <Colourless> doesn't seem to be working on delete...
[17:25:13] <watt> mine was:
[17:25:17] <watt> #include <stdio.h>
[17:25:17] <watt> void * Item::operator new(size_t size)
[17:25:18] <watt> {
[17:25:18] <watt> »···pout << "Allocating Item: " << size << std::endl;
[17:25:19] <watt> »···return malloc(size);
[17:25:19] <watt> }
[17:25:20] <watt> void Item::operator delete(void * ptr)
[17:25:22] <watt> {
[17:25:24] <watt> »···pout << "Free Item: " << ptr << std::endl;
[17:25:26] <watt> »···free(ptr);
[17:25:28] <watt> }
[17:27:02] <Colourless> yeah that would work
[17:27:17] <Colourless> can't use another class to do it
[17:27:36] <Colourless> so you can just do all the required allocator magic in the item new and delete operators
[17:28:13] <Colourless> can cast the delete ptr to Item* safely
[17:28:29] <watt> yup... and then get class type.
[17:29:19] <Colourless> and there is some funky trickery you can do with operator new
[17:29:41] <watt> get the memory and set variables before construction.
[17:29:45] <watt> yup.
[17:30:12] <Colourless> you can actually pass arguments to operator new
[17:30:19] <Colourless> IF you know what you are doing
[17:30:43] <watt> oh? I though it would just be size_t
[17:31:47] <Colourless> void * operator new (size_t size, char * name);
[17:31:48] <Colourless> void operator delete (void * p, char * name);
[17:31:48] <Colourless> and call the special new using this syntax:
[17:31:48] <Colourless> Foo * p = new ("special") Foo;
[17:32:27] <watt> neat... but yikes! That would be mucho code change.
[17:32:28] <Colourless> the overloaded delete though is ONLY called in the case of an exception in the constructor
[17:33:32] <Colourless> yes and no. you could have something like this
[17:33:59] <Colourless> void * operator new (size_t size, Pentagram::Allocator *allocator)
[17:34:27] <Colourless> and store allocator as a member in the Item struct
[17:34:41] <watt> hmm.. my cast ((Item *) ptr)->GetClassType().classname is always returning "Object"
[17:35:38] <Colourless> hmm... could be because of calling of destructors
[17:35:48] <watt> I think so.
[17:36:23] <Colourless> so then I'm thinking put operator new and delete in the object class
[17:36:35] <Colourless> and use a stored allocator pointer
[17:41:43] <Colourless> so the operators would look like this
[17:41:43] <Colourless> static void *operator new (size_t size, Pentagram::Allocator *alloc) {
[17:41:44] <Colourless> if (alloc) return alloc->Create(size);
[17:41:44] <Colourless> else ::operator new(size);
[17:41:44] <Colourless> }
[17:41:44] <Colourless> static void *operator delete (void * p, Pentagram::Allocator *alloc) {
[17:41:46] <Colourless> if (alloc) alloc->Destroy(p);
[17:41:48] <Colourless> else ::operator delete(p);
[17:41:50] <Colourless> }
[17:41:52] <Colourless> static void operator delete (void * p) {
[17:41:54] <Colourless> Object *obj = reinterpret_cast<Object *>(p);
[17:41:56] <Colourless> operator delete (p,obj->allocator);
[17:41:58] <Colourless> }
[17:42:42] <watt> won't the else ::operator delete(p) cause a loop?
[17:42:44] <Colourless> uh.. sort of
[17:43:01] <Colourless> no loop
[17:43:06] <Colourless> :: means use global
[17:43:25] <Colourless> is a slight problem with my code
[17:43:35] <Colourless> since you need to pass alloc to the constructor
[17:45:11] <Colourless> a trick that can be done is to add a few bytes before object that you would store info
[17:46:09] <Colourless> but i don't particularly like that idea
[17:46:20] <Colourless> would rather store the allocator as a member
[17:47:47] <watt> seems hard to access members due to the destructor though.
[17:48:21] <Colourless> yeah it is a bit problematic...
[17:51:41] <watt> It's a bit of memory play, but I think a struct on the front of the memory would be the best way to store info... except then we've assumed that struct is always there (or we check some magic on the front of it)
[17:51:58] <Colourless> so the alternative is like this
[17:52:15] <Colourless> very basic this:
[17:52:15] <Colourless> static void *operator new (size_t size, Pentagram::Allocator *alloc) {
[17:52:15] <Colourless> void *ret = 0;
[17:52:15] <Colourless> if (alloc) return ret = alloc->Create(size+sizeof(Pentagram::Allocator *));
[17:52:15] <Colourless> else ret = ::operator new(size+sizeof(Pentagram::Allocator *));
[17:52:16] <Colourless> *reinterpret_cast<Pentagram::Allocator *>(ret) = alloc;
[17:52:18] <Colourless> return reinterpret_cast<uint8*>(ret)+sizeof(Pentagram::Allocator *);
[17:52:20] <Colourless> }
[17:52:22] <Colourless> static void *operator delete (void * p, Pentagram::Allocator *alloc) {
[17:52:24] <Colourless> if (alloc) alloc->Destroy(p);
[17:52:26] <Colourless> else ::operator delete(p);
[17:52:28] <Colourless> }
[17:52:30] <Colourless> static void operator delete (void * p) {
[17:52:32] <Colourless> p = reinterpret_cast<uint8*>(p)-sizeof(Pe
[17:52:46] <Colourless> p = reinterpret_cast<uint8*>(p)-sizeof(Pentagram::Allocator *);
[17:52:47] <Colourless> operator delete (p,*reinterpret_cast<Pentagram::Allocator *>(p));
[17:52:47] <Colourless> }
[17:53:23] <Colourless> i think there is a few coding bugs there
[17:53:45] <Colourless> like the reinterpret_cast<Pentagram::Allocator *> should actually be reinterpret_cast<Pentagram::Allocator **>
[17:54:05] <Colourless> but putting a struct at the beginning would be cleaner
[17:54:42] <Colourless> and would allow for 'easy' maintanace
[17:59:04] <Colourless> that code is actually quite terrible :-)
[17:59:08] <Colourless> lots of probs
[17:59:34] <Colourless> an idea i have is if the allocator fails to allocate the memory, we could have it default back to the global new
[17:59:40] <Colourless> (or some other default allocator)
[18:00:13] <Colourless> that way if the allocator runs out of memory, we don't crash or have to do any other special handling
[18:00:24] <Colourless> just need to make sure the correct allocator pointer is set in the info struct
[18:01:30] <watt> Classes do indeed use the new and delete operators of their parent.
[18:01:56] <Colourless> good good
[18:02:14] <watt> llocating Item: 0x9e72dc0: 144
[18:02:14] <watt> Allocated Actor: 0x9e72dc0
[18:02:14] <watt> Allocating Item: 0x9e73008: 144
[18:02:15] <watt> Allocated Actor: 0x9e73008
[18:03:27] <watt> Freeing Actor: 0x9ac1210
[18:03:27] <watt> Free: 0x9ac1210
[18:03:27] <watt> Freeing Actor: 0x9ac12a8
[18:03:28] <watt> Free: 0x9ac12a8
[18:04:38] <watt> Item has the new and delete. Actor has console messages in the constructor and destructor.
[18:05:15] <watt> damn.. It's been too long since I last did any real C++, forgot how fun it could be sometimes.
[18:06:08] <Colourless> can you try this make the operator new have arguements like this (size_t size, int foo = 0)
[18:06:33] <Colourless> (will also need to make a matching operator delete too)
[18:07:06] <watt> umm.. might require a #define to make it quick to code, but sure
[18:07:45] <watt> if I could figure out how to #define every new Item() statement
[18:08:04] <watt> hmm.. is that possible?
[18:08:25] <Colourless> no
[18:08:32] <Colourless> i want to see if it will take the default arguement
[18:08:37] <watt> perhaps I'll just try it on new Actor
[18:08:39] <Colourless> and not require a new () Item()
[18:09:22] <Colourless> if default arguement does work it would greatly simplify things
[18:09:44] <watt> hmm.. yeah.
[18:11:47] <Colourless> if it doesn't work, we could just have 2 new functions anyway
[18:13:29] <watt> don't think defaults work..
[18:14:19] <watt> but that's ok
[18:17:09] <watt> and if you define a new or delete with more parameters, it seems you have to define the normal ones too
[18:26:56] <watt> interesting error:
[18:26:59] <watt> world/Item.cpp:82: error: invalid conversion from `const void*' to `void*'
[18:26:59] <watt> world/Item.cpp:82: error: initializing argument 2 of `void* operator
[18:26:59] <watt> new(unsigned int, void*)'
[18:27:19] <watt> that void  is defined as a const char 
[18:28:45] <watt> casting fixes it... but huh
[18:28:47] <Colourless> operator new (size_t, void*) is a standard type
[18:29:08] <watt> oh.. crap.
[18:29:43] <Colourless> void * operator new (size_t, void *p) { return p; }
[18:30:04] <Colourless> creates a new object out of an existing buffer
[18:30:40] <watt> I did not know that. Similiar to realloc?
[18:30:55] <Colourless> no :-)
[18:31:52] <Colourless> p is a pointer to an uninitialized buffer, and it will construct an object in p
[18:31:55] <Colourless> usage
[18:32:08] <Colourless> new (p) Foo(bar)
[18:32:27] <watt> that could be useful
[18:33:50] <Colourless> calling delete though after one of those = BAD
[18:33:53] <Colourless> :-)
[18:34:13] <watt> I'd imagine so
[18:37:42] <Colourless> well, i should probably be of
[18:37:43] <Colourless> f
[18:37:52] <Colourless> getting quite late... again
[18:37:56] <watt> later.
[18:38:22] <Colourless> have fun investigating operators new and delete :-)
[18:38:28] <-- Colourless has left IRC ("casts improved invisibility")
[18:38:41] <watt> I did get a message of "New Actor" passed to the operator new(size, msg)
[18:53:47] --> sbx has joined #pentagram
[19:28:26] <watt> We may be overthinking the pooling... if we're a little more... err... wasteful... it could be really fast... make pools for different size blocks (256, 512, 1024 bytes) and waste what isn't used per node
[19:29:53] <watt> the node only needs to have an ID, a status variable to tell if it's free, and some other identifier (pointer?) to the Pool it is in.
[19:31:29] <watt> The pool could use a simple array based stack to know where the free nodes are.
[19:35:03] <watt> I don't think we need to worry about separate operators either if we just say that the objecst always will be pooled no matter what happens. Still need to allow a PoolManager of some sort simply create another pool when on fills up.
[19:36:07] <watt> think we should pool all Objects
[19:48:35] <watt> ok.. the most allocated items I've seen is 20079 after quickly flying through 3 maps.
[19:53:03] <wjp> that many?
[19:53:21] <wjp> oh, wait, all nonfixed items of all maps are always loaded of course
[19:53:56] <watt> looks like a 256 byte pool with 32767 nodes would handle all the objects decently.. another pool if it grows higher
[19:55:23] <watt> 8388352 bytes to play with per pool...
[19:55:34] <watt> 8 megs at a time... hmm.
[20:00:52] <watt> I think I'd feel more comfort with 4 meg pools
[20:05:12] <sbx> Your whole conversation went a little above my head.
[20:05:15] <sbx> Does anyone know a good web resource explaining memory management/pooling in C++?
[20:05:43] <sbx> no, not necessarily good, just anything
[20:06:26] <watt> well, I got a few resources... but nothing that does much in the way of explaining full memory management...
[20:06:40] <watt> http://www.devx.com/tips/Tip/5608
[20:07:26] <sbx> something that tells what memory pooling is?
[20:07:35] <watt> http://www.bearcave.com/software/c++_mem.html
[20:08:06] <watt> ummm.... hmmm...
[20:08:17] <sbx> those look good
[20:09:03] <sbx> I've done memory allocation in blocks before, for speed.
[20:09:07] <watt> more or less... the concept is that dynamic memory allocation and deallocation (new and delete) is slow when done too heavily
[20:10:40] <sbx> ah is that all
[20:10:51] <watt> You can limit this theoritically by allocating larger chunks and using those chunks as a pool for smaller structures
[20:10:57] <sbx> #ifdef _BRAIN_DAMAGED_IBM_
[20:12:13] <sbx> Oh I've only gotten as far as "(de)allocate a block of N structures", but not "allocate a chunk of X MB to be used for anything".
[20:12:39] <sbx> but I saw that quake2 did something like that
[20:12:40] <watt> hehe.. yeah... I don't agree with him... he could have easily solved that with typedefs
[20:12:55] <watt> as for the brain damage
[20:13:41] <sbx> yeah
[20:13:46] <watt> I think quake2 allocated small memory chunks and reused them mostly.
[20:15:24] <sbx> I didn't understand what it was doing at the time, just something related to memory management.
[20:15:48] <sbx> thanks for the help and links
[20:16:04] <watt> np
[20:19:03] <watt> hmm... there seems to never be more than 21903 objects allocated at any given point
[20:23:09] <wjp> heh, show me a place with 21903 object and I'll make it allocate 21904 ;-)
[20:33:58] <-- watt has left #pentagram ()
[20:36:01] --> watt has joined #pentagram
[20:36:48] --- ChanServ gives channel operator status to watt
[20:37:51] <watt> ack.. pentagram froze up and I had to quit X to regain control
[20:38:53] <watt> looks like the max size for objects is around 208 right now, so 256 sounds about right for the node block size
[20:42:55] <-- sbx has left IRC ("brb")
[20:43:30] <watt> probably 256 byte and 1024 byte pools would suffice, and the 1024 byte pools likely will need only 1024 items per pool.
[20:44:20] <watt> soo... 16 megs to run pentagram successfully as a minimum requirement?
[20:45:07] <watt> U8 recommended 8
[20:46:24] <watt> hehe... good times with my blazing fast 90 Mhz 486... good times
[20:51:00] --> sbx has joined #pentagram
[21:45:39] --> Fl00der has joined #pentagram
[21:45:48] <sbx> heya Fl00der
[21:45:58] <Fl00der> evening
[23:00:30] --> Kirben has joined #pentagram
[23:00:30] --- ChanServ gives channel operator status to Kirben
[23:23:19] <-- Kirben has left IRC (leguin.freenode.net irc.freenode.net)
[23:23:19] <-- pizzasleep has left IRC (leguin.freenode.net irc.freenode.net)
[23:23:19] <-- oaQ^ has left IRC (leguin.freenode.net irc.freenode.net)
[23:25:52] --> oaQ^ has joined #pentagram
[23:38:00] --> pizzasleep has joined #pentagram
[23:49:13] <-- Lightkey has left IRC (Read error: 60 (Operation timed out))
[23:57:40] --> Lightkey has joined #Pentagram