Improving Assert

Posted in Code on February 4th, 2012 by Pyroka

My coding style is probably best described as defensive, I usually pepper my code with liberal use of asserts, but don’t then attempt to fix the issue, so, I will assert that a pointer is non-null, but if that pointer is null in a compile where asserts are removed, my code will crash, there won’t be an if check following the assert to attempt to handle the null pointer because in my mind, as assert should be fired for data that the code cannot handle.

This has worked pretty well for me, esspecially in the land of gameplay programming where things change fast, I attempt to have an assert for every assumption my code makes even if ‘there’s no way that could fail’ because the code will change, and then it will fail and an assert will help you to track it down (I also always assert in switch ‘default’ statemnets, when it’s not a valid option, since enums change and it can also help track down memory errors, if you happen to mangle somthing that passes through a switch statement).

Basically, I love assert, so I read with interest this AltDevBlogADay article on making asserts even more useful. The basic premise is that, through large amounts of macro pre-processor vodoo, you end up with an assert that you can follow up with a comma seperated list of variables, and have the name and value of that variable printed along with the assert. So instead of doing this:

Assert(index > 0 && index < m_Size, "Index: %d is out of bounds size: %u", index, m_Size);

You can do this:

Assert(index > 0 && index < m_Size, "Index is out of bounds")(index, m_Size);

And get this:


Assert in Array.cpp, line 27: "Index is out of bounds".
- Index (int) = 5
- m_Size (u32) = 2

Which saves some typing, but it also does so very much more. The first extra thing this new assert does that I love is that it forces you to put the extra brackets after the assert statement, they can be empty (if they really need to be) but they have to be there, which encourages the use of lobbing every variable that might be some help in there. Secondly, it calls code you wrote to output the value of the variable, meaning it supports custon types (through the use of a bodyless template function that you can specialise to take care of your types). So if I add a Vector3 to that list, it prints out the X, Y and Z components in a nice way without me hafing to type “X: %f Y: %f Z: %f” all the time and then cursing when that Vector3 is now a Vector4 and I could have really done with printing out the W bit.

It took a while to convert all the asserts in my engine code to use this new version (because we all love the errors caused when a marco evaluates to something different than it used to…) but I’m really happy I took the time. I also modified the code in the article to include the functionality that existed already to have different code take-over the reporting of the assert. For example in editor builds an assert will show a dialog box with the information, where as in game code the assert is either output to the debugger/console window or dumped to the log file (if no debugger is attached).

Tags: ,

Tyr: Transforming Nodes

Posted in Code on December 11th, 2011 by Pyroka

It’s been a good week, I managed to complete the to-do list from my last post early in the week, node transforming (translate, rotate and scale) went in pretty fast, and it was very nice to be able to drag nodes around the screen. I also buffed up the properties slightly, adding a hash property (created by hashing the name of the property with MurmurHash) and saves that when saving the properties, when loading properties it finds a property that matches that hash, and then checks that the type is what is expected. This should allow me to add, remove and move around properties and it will ‘just work’ (previously it assumed that the order of the properties would be the same loading as it was saving, and that properties were all the correct type, very dangerous assumptions but that was just first-pass temporary code).

Fixing the issue with dragging and dropping items from the scene graph TreeCtrl turned out to be a one-line fix, I forgot that the tree control items own the data you associate with them, and it was being deleted as part of the drag/drop action, and then causing a crash (well, hitting an assert actually) when that item was then selected (as that data was expected to be there).

So with those tasks sorted, I moved on to the next big job: Making it so that you can select multiple nodes and then transform them with the mouse, this was a bit of a pain and is still not entirely complete (currently the properties PropGrid only shows if you have exactly one node selected, in future it will show all common properties shared between all selected nodes, and allow you to change them for all selected nodes at once), but as a task it wasn’t particularly difficult, just a case of going through the code and sorting out each of the places that assumed there was only ever one selected node (not that many, as it turned out).

Then I added a ‘selection marker’, this is a cross that follows the selected node or nodes (assuming the average position in the case of more that one node being selected) and has a (small) hit-area, you click can click and drag that to translate, rotate and scale the selected node(s), thus allowing the click-and-drag transforms to work on nodes that have a size of 0, which previously didn’t work.

Finally I added some extra buttons to the editor to expose some align functions, these allow the user to select a group of nodes and align them to the left, center, right, top, middle or bottom, which uses the nodes personal bounding boxes (as in, the bounding box that encompasses that node only, excluding it’s children) to align the nodes, for example if you select a bunch of nodes and click ‘align-left’ then the nodes will be arranged so that the minimum extent on the x-axis of their personal bounding boxes will all equal that of the left-most node, quite a useful operation.

So then, next week I want to make sure that loading and saving scenes works as intended, and I might move on to attempting to build-up a more complex, realistic scene… I’ve been putting some thought into the game I might make to test features of the engine as I go along, nothing has really solidified yet but I have some interesting (at-least for me) ideas. I also want to make sure it compiles on Linux but progress on that front may be slightly slower (I have to be in a particular mood to comb through GCC compile errors…)

Tags:

Tyr: Now Using Premake

Posted in Articles on December 3rd, 2011 by Pyroka

So, no blog post last week, I had actually managed to get some things done, just not really enough to blog about, my attention was irratic, jumping from task to task so much that I didn’t really get anything in a fit state to talk about. This week however, I managed to finish off some of the things I started last week, or at-least get them to the stage where I can write about them.

So the most major thing I did is that I finally moved the project over to using Premake, I’ve wrote about Premake before, but basically, it allows you to define project file that can then be compiled into Visual Studio, XCode or makefile projects (and more) so that you only have to maintain one file, (in my case, one file per-project and a master-file for the solution) and you can build on the 3 major platforms.

I also added the repository to Cerberus, (and it pretty much built first-time) so whenever I push to the main repository (hosted on BitBucket), Cerberus will download and compile the project on all the platforms I’ve set-up (currently only Windows, as I’m in the middle of setting up a Linux VM on my laptop) and e-mail me the result, it’s kinda cool to see my own CI software compiling my own engine, and then e-mailing me telling me it’s ok.

I’m anticipating some issues when I attempt to get Tyr compiling on Linux for the first time, I got a bit sloppy blocking off my Win32 specific code and I’m sure I will have fallen-foul of some of the warnings GCC has enabled that VS doesn’t, so I want to try and get this working as soon as possible, so I can sort these issues before I get too far with things. But I’m also tempted to leave it till the new year, where I shall be constructing a dedicated build machine or two (ideally one per-platform, but I’m debating just getting some OSX compatible hardware and VMing everything).

I also started the input library, (thinking I’d need it to progress with the editor) which saw me referring to this article by Keith Judge, which gave me a start diving into the weird and wonderful world of the Win32 RawInput API (I’d really love to talk to someone at MS who could attempt to justify some of those decisions…) which seems to be the way to go input-wise, and fits in with my whole taking care of things myself aspect.

Finally I looked into the editor, more precisely making it so you can translate, rotate and scale nodes using the mouse, like one would expect from a civilised program. I’d actually been working on that for a few days but apparently I wasn’t very awake when I was, it got to the stage where it was nearly working but not quite, but then the next day I had a look and it was glaringly obvious that the approach I was taking was making it stupidly hard for-myself and I re-wrote the thing in about half the lines of code of the original solution, and it worked just fine. So now you can drag nodes around the level… As long as those nodes have a size greater than zero, I’m attempting to come up with a solution for selecting/transforming nodes with no size in a way that won’t pollute the game-side code and only be available for the editor.

So that’s it for the past two weeks, this week I hope to clear up how I’m accessing properties (I’m planning some form of compile-time hash so grabbing properties by string is ok) and finish off the editor transforming code and add the buttons to the editor. Then I really need to look at dragging/dropping nodes from one part of the scene-graph tree control to another, as it currently very much doesn’t work but is a rather vital operation.

 

Tags:

Tyr: Debug Rendering

Posted in Code on November 13th, 2011 by Pyroka

This second post this week, is to make up for the very very high likely-hood that there will be no post next week due to work and such.

It’s gotten to the stage where the editor could do with drawing some stuff, stuff like markers to show the position/size of the currently selected node, and this raises an interesting problem. It would also be good for game-nodes to be able to draw debug graphics, from anywhere in their update loop (or functions called from that).

So, I have need of a way to store rendering commands for later use, my idea was to have some form of buffer that is cleared every frame, and methods to add common shapes to that buffer (rectangles, circles, etc.). Due to the way that scenes work in Tyr (there can be more than one scene active at the same time) I need a buffer per-scene, and a global buffer, for things like the editor who will want to draw out-side of a nodes update loop.

Now, the simplest method of drawing shapes with OpenGL is with the glBegin and glEnd methods, basically, you call glBegin, passing it a mode (To draw lines, quads, triangles, etc.) and then repeatedly call glVertex, glTexCoord, glColour, describing the geometry you’d like to draw. So I decided to create a class that holds to std::vectors, the fist one is a list of commands (to be passed to glBegin) and an index into the second buffer, the second buffer will hold only floats, used to express the colour and vertex positions of the items I wish to draw. I made the decision that any primitive drawn by this debug-drawing code will have all vertecies drawn in the same colour, to save on the amount of information I’m saving (and to stress that this is for debug-only drawing, since it will be rather slow).

Primitives are added to this buffer by calling one of several methods (DrawRect, DrawCircle, etc.) all of which take a RGBA colour, and the relevant per-primitive information, this calls internal methods to add commands to the command list and float to the float list, this is all done so that the drawing code can be as simple as possible.

void Renderer::Render( DebugBuffer* pBuffer )
{
	glDisable(GL_TEXTURE_2D);
	glDisable(GL_DEPTH_TEST);
	glBindTexture(GL_TEXTURE_2D, 0);
	g_ShaderManager.Bind(INVALID_SHADER);

	size_t commandIndex = 0;
	size_t floatIndex = 0;

	while(floatIndex < pBuffer->m_FloatList.size())
	{
		GLuint startIndex = pBuffer->m_CommandList[commandIndex++];
		TYR_ASSERT_MSG(floatIndex == startIndex, "Start index does not match current float index");
		GLuint command = pBuffer->m_CommandList[commandIndex++];

		// Grab the colours
		GLfloat red	= pBuffer->m_FloatList[floatIndex++];
		GLfloat green	= pBuffer->m_FloatList[floatIndex++];
		GLfloat blue	= pBuffer->m_FloatList[floatIndex++];
		GLfloat alpha	= pBuffer->m_FloatList[floatIndex++];

		// We should end before the next set
		GLuint nextStartIndex = commandIndex < pBuffer->m_CommandList.size() ?
								pBuffer->m_CommandList[commandIndex] :
								pBuffer->m_FloatList.size();

		// Draw it
		glBegin(command);
		for(;floatIndex < nextStartIndex;)
		{
			float x = pBuffer->m_FloatList[floatIndex++];
			float y = pBuffer->m_FloatList[floatIndex++];
			float z = pBuffer->m_FloatList[floatIndex++];

			glColor4f(red, green, blue, alpha);
			glVertex3f(x, y, z);
		}
		glEnd();
	}
}

This is the code that renders a DebugBuffer, it simply loops through the commands, grabbing the needed floats from the float buffer that are then used to render the primitive. Each command knows the index of the first float that it needs to access (the red value for the colour) and by looking ahead to the next command, we can get the total number of floats that command uses.

The system as it stands seems to work quite well, as I mentioned earlier, each scene has it’s own debug buffer and there is a global one too, the rendering order is scene (and child nodes), the scene’s DebugBuffer and finally the global DebugBuffer, so that the debug information is always available (I may have to alter this when I add support for post-process effects, as those will want to be applied before the debug rendering. I also need to extent this method to allow the rendering of text, but I’m not entirely sure how to go about doing that one yet. This system is rather new (it was implemented mostly on Saturday night) so it may change over time when it is used more extensively.

Tags: , ,

Tyr: Managing Managers

Posted in Code on November 8th, 2011 by Pyroka

Like most game engines, Tyr has a number of managers, these managers are typically responsible for loading, saving and manipulating certain objects, they are also (in Tyr at-least) responsible for owning the objects, meaning that they control the lifetime of said objects and are responsible for deletion (to this end, I usually make objects owned by a manager have a private destructor, so that attempting to delete them not through the manager is a compile error).

Now there is a set order, logically that these managers should themselves be created, granted for some of them it doesn’t matter (should the texture manager be created before the shader manager?), whilst for others, it is pretty easy (the Logger, responsible for logging warnings and errors, should probably be first) and if there is an order they should be constructed, they should be destructed in reverse order, otherwise nasty things may occur.

So, I set about trying to come up with a nice way to make such an arrangement happen, this post will document my journey.

First things first, we have some common behaviour, all the managers should have a priority, and throw in Init and Shutdown methods for good measure (these will be called based on the priority) so for that we need a BaseManager (which I was very tempted to call ManagerManager), the base manager doesn’t do much at the moment:

class BaseManager : public NonCopyable
{
public:
	//! Constructor
	BaseManager(
		u8 priority		//!< Priority of the manager (lower is higher)
		);

	//! Destructor
	virtual ~BaseManager();

	//! Initialise (called in order of priority)
	//! \return True on success
	virtual bool Init() = 0;

	//! Shut-down
	//! \return True on success
	virtual bool Shutdown() = 0;

	//! Get the priority of the manager
	//! \return The priority
	u8 GetPriority() const;

private:
	u8 m_Priority;		//!< Priority of this manager, lower higher in list
};

The main point of node is the priority member variable, which is a u8, allowing 255 managers (should be enough for anyone, right?), now the comment for this member talks about a list, and we'll get to that in a moment, but allow be to indulge into a brief tangent on static initialisation in C++.

Lots of people would implement these managers using the Singleton design pattern, indeed it used to be my weapon of choice for such tasks, and it has a lot of merits, especially when you get to the more advanced versions which allow you to control when they are constructed and deleted, however, I grew tired of singletons for a variety of reasons, for one they give you a false feeling of safety (even though it is pretty easy to maliciously, or accidentally create more than one) they can have issues being used in .dlls, and also, typing out Singleton::Instance() (or your equivalent) is a tad verbose and does get tiring after a while.

So now I use global variables straight-up, rather than attempting to hide behind a design pattern to make them go away, global variables are of-course bad, as are macros, and both will be used judiciously here, generally, all the 'bad' things that you 'should never use' end up being useful after a while, you just have to know the risks and decide they're worth it.

So, back to static initialisation (forgive my tangent from my tangent). See all my managers are declared by an extern, something like:

extern TextureManager g_TextureManager;

This is nice and simple but it does mean that the manager object is constructed during static initialisation and thus I have no say in whether this manager is constructed before that manager, which is slightly ok, after-all we made that Init() method so that we could initialise the managers in a different order to the one they were constructed in. The problem, as it turns out, lies with the fact that we have to store references to these managers in a list, and is we want to register managers in their constructor (which we do otherwise we will have to manually update a list of managers and that just sounds horrendous) we need to make sure that the list of managers is constructed before any of the managers themselves.

Lucky for all, all is not undefined in the land of static initialisation, static local variables, declared in a function, will be initialised the first time that function is called. So, to solve our manager-list issue we merely need to define a member function on the BaseManager like so:

ManagerList& BaseManager::GetManagerList()
{
	static ManagerList managerList;
	return managerList;
}

Then, in the constructor of the BaseManager add:

GetManagerList().push_back(this);

This will make sure that our list of managers is constructed before we attempt to add a manager to it (which is nice, I'm sure you'd agree). So, we have our list, we have our managers that register with said list, sorting the list is a trivial issue, as is calling the Init/Shutdown functions in correct orders, so now we turn our attention to how best to define the priority and thus the order of these managers.

Now, you can just manually define them, that would be ok, for a while, but when you have more than a few managers the relationships get pretty complex and going back and changing numbers is very tedious, no, what we need is a list, a list with the names of the managers in a human readable format, and their position in that list to be their priority. What we need, is an enum.

When this idea hit me I immediately liked the cleanness of it, just define an enum, perhaps in one of the higher-up components of the engine (in the Engine.lib, actually) and all is well, you can re-order the entries in the enum, and all the priority values will change and there will be no errors due to forgetting that one manager. There are, however, two main issues in this technique. Firstly, by making the priority of the manager be defined else-where, we are making that library implicitly rely on another library, which I want to attempt to avoid, however, by marking the priorities as 'extern' I am merely stating that something, somewhere has to define a value for that, if they want to use this library, I personally don't think that is too bad. Secondly, if I am defining this list in a high-level library (and I am) there is a very high-chance that I will create a circular dependency (Engile lib defines the priorities, Engine lib relies on RenderLib to do rendering, RenderLib relies on the priorities defined in EngineLib) however, both Visual Studio and GCC can deal with this (you just have to tell GCC to deal with it using --start-group and --end-group) so that isn't too bad either.

Now all that is left is to make the process of defining these managers and the priority list easier and less prone to errors, and for that, I chose to break-out the macros.

In the BaseManager header, I define 3 macros:

#define TYR_GET_MANAGER_PRIORITY_VAR(name)		name##_Priority
#define TYR_DEFINE_MANAGER_PRIORITY_VAR(name)	extern	u8 TYR_GET_MANAGER_PRIORITY_VAR(name);
#define TYR_MANAGER_CTOR(name) \
	TYR_DEFINE_MANAGER_PRIORITY_VAR(name) \
	name::name() \
		: BaseManager(TYR_GET_MANAGER_PRIORITY_VAR(name))

The first one, simply makes it convenient to get the name of the variable correct everywhere it's used, the second one declares an external variable that is used as the priority, and the final one uses the first two to describe the constructor of a new manager class, which will have the priority assigned to the variable _Priority. Only the final macro is used directly, like so:

TYR_MANAGER_CTOR(TextureManager)
{

}

Which expands to:

extern u8 TextureManager_Priority;
TextureManager::TextureManager()
    : BaseManager(TextureManager_Priority)
{

}

Now, in EngineLib, I have a file called ManagerConfig.cpp (it has a corresponding header, but it is empty), which contains the following:

#define TYR_DECLARE_MANAGER_PRIORITY_VAR(name)\
	u8 TYR_GET_MANAGER_PRIORITY_VAR(name) = ManagerPriorityList::name;

namespace ManagerPriorityList
{
	enum
	{
		Logger,
		TextureManager,
		ShaderManager,
		SceneManager,
	};
};

TYR_DECLARE_MANAGER_PRIORITY_VAR(Logger);
TYR_DECLARE_MANAGER_PRIORITY_VAR(SceneManager);
TYR_DECLARE_MANAGER_PRIORITY_VAR(TextureManager);
TYR_DECLARE_MANAGER_PRIORITY_VAR(ShaderManager);

Another macro, this one defines the values in the format expected by the previously declared extern, this macro assigns the value from a named value in the enum below, and at the very bottom, the list of managers defined using the macro. Note that the order of the managers at the bottom doesn't matter, so I only have to change the order of the enum if I want to change the order of the managers.

I am still slightly annoyed that I have to write the manager name twice in this file, I'm tempted to switch the enum out for some form of static variable that is incremented as part of the macro, but I'm not sure if that would work and it's good enough for now. I hope you enjoyed this unexpected mid-week posting, I'll try to have something more to tell by the weekend, as expected taking a break from tool programming to dive back into the engine proper has boosted my enthusiasm for the project, so progress is picking up somewhat.

Tags: ,

Tyr: Textures and Shaders

Posted in Code on November 5th, 2011 by Pyroka

There was no blog post last week, sorry about that, but truthfully I’d not really done enough to justify one, it was a week of research more than coding, but I did find the answers to some questions (mostly about efficient sprite rendering).

So, this week I decided to take a bit of a break from the Asset Manager, and take the time to actually implement textures properly, loading the compiled textures and sending the image data off to OpenGL turned out to be rather easy, OpenGL supports rendering DXT compressed textures via the ‘ubiquitous’ GL_COMPRESSED_RGBA_S3TC etc. extentions, these aren’t part of the official spec, but being labelled as ‘ubiquitous’ means that most hardware supports them, so I decided it was safe to use them.

I then moved on to dragging my sprite renderer slightly into the modern era by switching from ye-olde fixed-function rendering to using shaders, I created a simple sprite vertex and fragment shader (it’s about 4 lines long) that basically just replaces what the fixed-function version did (render a texture, multiplied by a colour, at positions specified by the vertices).

I’d never set-up shaders in OpenGL before, and a fair amount of the documentation I found used the ARB extentions to load and compile the shaders, howver, combining a few different websites (mainly this one) I managed to come up with a solution that works.

In the process of attempting to get shaders working, I made much use of my logger class, and have extended it so that, when asserts are enabled, it will trigger a break-point whenever it logs an ‘Error’ or ‘Fatal’ level message.

I’m having fun working on the main engine again, so I’ll likely not work on the Asset Manager this week, instead I want to attempt to make it easier to move, rotate and scale sprites around the small test scene I’m making.

Tags: , ,

Asset Manager: Adding Multiple Textures

Posted in Code on October 23rd, 2011 by Pyroka

So this week I set out to add some more polish to the process of adding textures, even though it’s only likely to be me ever using this tool, a little time spent on polish can reap some serious time-saving benefits, the main tasks were to add support for adding multiple images at once, and to have the texture information fields auto-fill from the source texture, I decided to tackle the latter task first.

This turned out to be pretty simple, I just made the ‘Source’ (location of the source asset) input the top-most on the add texture dialogue, capture the event when that changed, then attempt to load that source path as an image, to extract the width and the height, and extract the name from the file-path.

Adding support for adding multiple textures at once was slightly more tricky, I decided to swap out the ‘Source’ file-select dialogue for a list-box with ‘Add’ and ‘Delete’ buttons under it, the file-select dialogue now shows when the user clicks the ‘Add’ button and allows them to select multiple files, the code then processes each file-path the user selected, for each file, pre-populating the width, height, name and parent folder, and storing those results in a std::vector of structs. If the file cannot be opened, or is not a valid image, an error dialogue is shown to the user but the adding process continues.

The user is then able to select individual files and make any changes they desire, when the user clicks the ‘Ok’ button, each entry in the std::vector is validated (making sure the width, height, parent folder etc. are valid), if an invalid entry is found, an error dialogue is shown, and that item is selected in the list, allowing the user to quickly amend the error.

Once this was all up and working, it was time to test it with a large amount of data, luckily, I had a folder full of sprites (700+) to test with (the sprites in question are the ‘Last Guardian’ sprites available here under a Creative Commons licence), so I selected all 700-odd sprites and clicked ok to add them to the database, Asset Manager froze for a couple of minuets, just as I was about to start to attempt debugging the presumed infinite loop, it resumed responding.

So, the code worked, although it was woefully slow, I suspected the issue was with the database (as there really wasn’t much of my code being executed), and sure enough, a look at the SQLite FAQ explained the problem (that by default each INSERT query created and then committed a transaction) and the solution (manually start a transaction before INSERTing the records, and COMMIT it when all the records are added), I added a few transaction-related functions to the database wrapper and implemented this solution, when I re-ran the test, re-adding the sprites (after having manually cleared the database, there is still no way to remove items currently…) the operation completed instantly.

 

Adding the Last Guardian sprites to Asset Manager

In the coming week I need to work on completing the basic functionality for Asset Manager, allowing the removal of textures and folders, and allowing items to be moved by dragging them around the tree-control, once this is done I think I’ll move back over to the engine proper and get textures loading.

Tags: , , ,

Asset Manager: Adding Textures

Posted in Code on October 16th, 2011 by Pyroka

Another Asset Manager update this week, and, as the title would suggest, you can now add textures (and folders), you can also update the properties for existing textures and a lot more has been going on behind the scenes, mainly with commands.

Command is the class I use to implement undo/redo functionality, Commands have Execute and Unexecute methods which perform the steps needed to carry out or roll-back the command. Now, it is slightly tedious to write a whole bunch of commands that merely use accessors to set a value on on object, so I decided to spend some time attempting to come up with a better solution (warning: template code ahead).

So, I needed some code that would, when constructed, use a ‘get’ method to get the current value of a property, then on Execute(), set a new value and on Unexecute(), return the value to what it was when the command was constructed, this is what I came up with:

//! Templated class for a command that calls a getter/setter to undo and redo
template < typename Obj, typename Type, Type (Obj::*Getter)() const, void (Obj::*Setter)(Type) >
class GetSetCommand : public Command
{
public:
//! Constructor
GetSetCommand(
    Obj* pObj,          //!< Object to act upon
    const Type&; value  //!< Initial value to set
)
: m_pObject(pObj)
, m_NewVal(value)
, m_OldVal()
{
    m_OldVal = ((*m_pObject).*(Getter))();
}

//! Destructor
~GetSetCommand()
{
}

//! Execute the command
//! \return True if change was successful, false otherwise
bool Execute()
{
    ((*m_pObject).*(Setter))(m_NewVal);
    return true;
}

//! Reverse the command
//! \return True if reversal was successful, false otherwise
bool UnExecute()
{
    ((*m_pObject).*(Setter))(m_OldVal);
    return true;
}

private:
    Obj* m_pObject; //!< Object to act upon
    Type m_NewVal;  //!< Value to set upon execute
    Type m_OldVal;  //!< Value to set upon unexecute
};

This lovely bit of template magic does the trick, it has 4 type augments, the type of the object that owns the property we are changing, the type of the property, and a gettter and setter functions. This allows me to typedef versions of this template, like so:

typedef GetSetCommand < AssetDatabaseItem, std::string, &AssetDatabaseItem::GetName, &AssetDatabaseItem::SetName > NameCommand;

So whenever I need to use that command, I can simply create a new AssetDatabaseItem::NameCommand and all the template black magic is hidden. I actually have two versions of this template, one for intrinsic types, shown above, and one for complex types, where the getter function is defined as taking a const reference parameter, to avoid the copying costs.

Other than that, I’ve implemented adding textures and folders, and updating properties, I’ve also made it easy for the asset manager to determine if the properties of an asset have changed since the last time that asset was compiled (by adding a unix timestamp ‘modified’ column to the database) so I can tell if an asset needs to be compiled again.

This week I plan to add some more polish to the process of adding textures, currently you have to enter all the information manually, however I feel it would be better if, when you selected the source image, if the output width, height and name were all filled in automatically (to values that matched the source asset), I also want to make it easier to add multiple assets at a time, as I think that adding a folder full of textures will be something that happens quite often.

Tags: ,

More Asset Manager Refactoring

Posted in Code on October 8th, 2011 by Pyroka

So this week has been a pretty good week for my work on the asset manager, mostly refactoring the SQLite wrapper (again).

Previously, to get a list of items from the database, you called a method on the AssetDatabase class, which returns a vector of items, both the vector and the items are allocated during this time, and it is the responsibility of the calling code to delete these, this bothered me while writing it, but I couldn’t think of a better solution at the time.
This approach became worse when it occured to me that if the GetItems method were to be called multiple times, there would be seperate objects in seperate lists pointing to the same database item, which makes updating items a nightmare, as the database and all items that pointed to that database item would need to be kept in sync.

This was the final straw for the code as it was, now being decidedly not fit for purpose, I refactored this into a better system. The new system starts with changes to the AssetDatabase class, now when a database is opened all the items are retrieved from the database, and stored in per-table std::maps with the rowid of the item as the map key the AssetDatabaseItem structs have been promoted to classes, containing a pointer to the AssetDatabase they belong to, the public member variables have been replaced with accessors, and all the ‘set’ functions call a method on the AssetDatabase to update the SQLite database with the new value.

Any code that wishes to get a list of database items needs to allocate a list and pass it by reference into a function that will query the SQLite database for the relevant rows, and then use that as a reference to the various std::maps to retrieve the relevant AssetDatabaseItems

This new system means that any number of asset lists can be gloating around and they will contain pointers to the same objects, managed by the AssetDatabase they came from, and any changes to any properties of any of these items will be reflected in the SQLite database.

As well as this rather epic refactoring, I also created a custom wxImageHandler to save and load images in the engine format, added support for all the different types of dxt compression and added properties to the compiled image. Next I need to actually detect and handle the changes in properties, something which is complicated slightly by my wish to be able to undo and redo such actions, meaning that they will have to be turned into a command class, whilst this complicates matters slightly, the advantage of having functioning undo and redo buttons in any application is worth the effort.
After that I will add the ability to add and remove database items (all my work currently is based around a test database created from hard-coded data) and make sure that works, I can then test the system with some more realistic data.

I have had some more ideas for the job queue, but I think I need to read up on some existing implementations before I commit to an approach, I also feel the need to plan this system out properly before starting coding, as it will be an important system and debugging threading issues can be tricky.

I’m in London this weekend so no coding will be done (I’m writing this on my phone on the train) so the next update may be a bit sparse, depending on how much I manage to get done in the evenings this week.

Tags: , ,

Asset Manager: Update 2

Posted in Code on October 3rd, 2011 by Pyroka

Bit of a late update this week, most of my free time was taken up by some web work that needed doing (not for this site) so I didn’t really get a chance to work on the asset manager untill this weekend, however, I managed to have a very productive weekend.

I improved the AssetListPanel wxControl, so it now nicely shows a list of assets (in either a list view or a tree view), polished that a bit more too (making it so that the selected item is mirrored on the other view, and adding a custom wxEvent for an item being selected). I created a BitmapWindow, which is a form of wxScrolledWindow for displaying bitmaps (the background is the grey and white check pattern that will be familiar to anyone who uses Photoshop or other graphics tools) and I created a TextureViewPanel, again a custom wxWidget for displaying a texture.

Here’s what it looks like:

Asset Manager showing a texture in it's original, and compiled versions

The image on the left is the original source image, and the image on the right is the compiled version, I still need to add some stats for the compiled version and add them to the property grid the same as they are for the source image.
I also, (kinda) have compiling textures in, but it needs a lot of work, currently if you select a texture and it can’t find the compiled version, it will compile it, however, currently the compiling only supports DTX1 format and it takes ages blocking the main thread the entire time it does it. So, next on the todo list is to move that on-to another thread and stick a progress bar on there. I’m using LibSquish to encode the image for me, which took a little while to getworking but seems to produce good results and doesn’t involve too much code.

The current todo-list, in no particular order, is:

  • Add support for the other DTX formats
  • Crete a custom wxImageHandler to handle loading of the complied textures
  • Create a simple job system (that the engine will also use) and use that to take care of compiling textures
  • Add an ‘update’ button to the TextureViewPanel that will save off any changes to the compiled texture and recompile it
  • Add buttons to lock the source and compiled texture view scroll bars
  • Add buttons to add folders and textures

I think I’ll get all the basic tasks done before moving on-to the job system, as that will be a bit more involved (and hopefully more fun)

Tags: , ,