So this week I set out to add some more polish to the process of adding textures, even though it’s only likely to be me ever using this tool, a little time spent on polish can reap some serious time-saving benefits, the main tasks were to add support for adding multiple images at once, and to have the texture information fields auto-fill from the source texture, I decided to tackle the latter task first.

This turned out to be pretty simple, I just made the ‘Source’ (location of the source asset) input the top-most on the add texture dialogue, capture the event when that changed, then attempt to load that source path as an image, to extract the width and the height, and extract the name from the file-path.

Adding support for adding multiple textures at once was slightly more tricky, I decided to swap out the ‘Source’ file-select dialogue for a list-box with ‘Add’ and ‘Delete’ buttons under it, the file-select dialogue now shows when the user clicks the ‘Add’ button and allows them to select multiple files, the code then processes each file-path the user selected, for each file, pre-populating the width, height, name and parent folder, and storing those results in a std::vector of structs. If the file cannot be opened, or is not a valid image, an error dialogue is shown to the user but the adding process continues.

The user is then able to select individual files and make any changes they desire, when the user clicks the ‘Ok’ button, each entry in the std::vector is validated (making sure the width, height, parent folder etc. are valid), if an invalid entry is found, an error dialogue is shown, and that item is selected in the list, allowing the user to quickly amend the error.

Once this was all up and working, it was time to test it with a large amount of data, luckily, I had a folder full of sprites (700+) to test with (the sprites in question are the ‘Last Guardian’ sprites available here under a Creative Commons licence), so I selected all 700-odd sprites and clicked ok to add them to the database, Asset Manager froze for a couple of minuets, just as I was about to start to attempt debugging the presumed infinite loop, it resumed responding.

So, the code worked, although it was woefully slow, I suspected the issue was with the database (as there really wasn’t much of my code being executed), and sure enough, a look at the SQLite FAQ explained the problem (that by default each INSERT query created and then committed a transaction) and the solution (manually start a transaction before INSERTing the records, and COMMIT it when all the records are added), I added a few transaction-related functions to the database wrapper and implemented this solution, when I re-ran the test, re-adding the sprites (after having manually cleared the database, there is still no way to remove items currently…) the operation completed instantly.

Adding the Last Guardian sprites to Asset Manager

In the coming week I need to work on completing the basic functionality for Asset Manager, allowing the removal of textures and folders, and allowing items to be moved by dragging them around the tree-control, once this is done I think I’ll move back over to the engine proper and get textures loading.

Another Asset Manager update this week, and, as the title would suggest, you can now add textures (and folders), you can also update the properties for existing textures and a lot more has been going on behind the scenes, mainly with commands.

Command is the class I use to implement undo/redo functionality, Commands have Execute and Unexecute methods which perform the steps needed to carry out or roll-back the command. Now, it is slightly tedious to write a whole bunch of commands that merely use accessors to set a value on on object, so I decided to spend some time attempting to come up with a better solution (warning: template code ahead).

So, I needed some code that would, when constructed, use a ‘get’ method to get the current value of a property, then on Execute(), set a new value and on Unexecute(), return the value to what it was when the command was constructed, this is what I came up with:

//! Templated class for a command that calls a getter/setter to undo and redo
template < typename Obj, typename Type, Type (Obj::*Getter)() const, void (Obj::*Setter)(Type) >
class GetSetCommand : public Command
{
public:
//! Constructor
GetSetCommand(
    Obj* pObj,          //!< Object to act upon
    const Type&; value  //!< Initial value to set
)
: m_pObject(pObj)
, m_NewVal(value)
, m_OldVal()
{
    m_OldVal = ((*m_pObject).*(Getter))();
}

//! Destructor
~GetSetCommand()
{
}

//! Execute the command
//! \return True if change was successful, false otherwise
bool Execute()
{
    ((*m_pObject).*(Setter))(m_NewVal);
    return true;
}

//! Reverse the command
//! \return True if reversal was successful, false otherwise
bool UnExecute()
{
    ((*m_pObject).*(Setter))(m_OldVal);
    return true;
}

private:
    Obj* m_pObject; //!< Object to act upon
    Type m_NewVal;  //!< Value to set upon execute
    Type m_OldVal;  //!< Value to set upon unexecute
};

This lovely bit of template magic does the trick, it has 4 type augments, the type of the object that owns the property we are changing, the type of the property, and a gettter and setter functions. This allows me to typedef versions of this template, like so:

typedef GetSetCommand < AssetDatabaseItem, std::string, &AssetDatabaseItem::GetName, &AssetDatabaseItem::SetName > NameCommand;

So whenever I need to use that command, I can simply create a new AssetDatabaseItem::NameCommand and all the template black magic is hidden. I actually have two versions of this template, one for intrinsic types, shown above, and one for complex types, where the getter function is defined as taking a const reference parameter, to avoid the copying costs.

Other than that, I’ve implemented adding textures and folders, and updating properties, I’ve also made it easy for the asset manager to determine if the properties of an asset have changed since the last time that asset was compiled (by adding a unix timestamp ‘modified’ column to the database) so I can tell if an asset needs to be compiled again.

This week I plan to add some more polish to the process of adding textures, currently you have to enter all the information manually, however I feel it would be better if, when you selected the source image, if the output width, height and name were all filled in automatically (to values that matched the source asset), I also want to make it easier to add multiple assets at a time, as I think that adding a folder full of textures will be something that happens quite often.

So this week has been a pretty good week for my work on the asset manager, mostly refactoring the SQLite wrapper (again).

Previously, to get a list of items from the database, you called a method on the AssetDatabase class, which returns a vector of items, both the vector and the items are allocated during this time, and it is the responsibility of the calling code to delete these, this bothered me while writing it, but I couldn’t think of a better solution at the time.
This approach became worse when it occured to me that if the GetItems method were to be called multiple times, there would be seperate objects in seperate lists pointing to the same database item, which makes updating items a nightmare, as the database and all items that pointed to that database item would need to be kept in sync.

This was the final straw for the code as it was, now being decidedly not fit for purpose, I refactored this into a better system. The new system starts with changes to the AssetDatabase class, now when a database is opened all the items are retrieved from the database, and stored in per-table std::maps with the rowid of the item as the map key the AssetDatabaseItem structs have been promoted to classes, containing a pointer to the AssetDatabase they belong to, the public member variables have been replaced with accessors, and all the ‘set’ functions call a method on the AssetDatabase to update the SQLite database with the new value.

Any code that wishes to get a list of database items needs to allocate a list and pass it by reference into a function that will query the SQLite database for the relevant rows, and then use that as a reference to the various std::maps to retrieve the relevant AssetDatabaseItems

This new system means that any number of asset lists can be gloating around and they will contain pointers to the same objects, managed by the AssetDatabase they came from, and any changes to any properties of any of these items will be reflected in the SQLite database.

As well as this rather epic refactoring, I also created a custom wxImageHandler to save and load images in the engine format, added support for all the different types of dxt compression and added properties to the compiled image. Next I need to actually detect and handle the changes in properties, something which is complicated slightly by my wish to be able to undo and redo such actions, meaning that they will have to be turned into a command class, whilst this complicates matters slightly, the advantage of having functioning undo and redo buttons in any application is worth the effort.
After that I will add the ability to add and remove database items (all my work currently is based around a test database created from hard-coded data) and make sure that works, I can then test the system with some more realistic data.

I have had some more ideas for the job queue, but I think I need to read up on some existing implementations before I commit to an approach, I also feel the need to plan this system out properly before starting coding, as it will be an important system and debugging threading issues can be tricky.

I’m in London this weekend so no coding will be done (I’m writing this on my phone on the train) so the next update may be a bit sparse, depending on how much I manage to get done in the evenings this week.

Bit of a late update this week, most of my free time was taken up by some web work that needed doing (not for this site) so I didn’t really get a chance to work on the asset manager untill this weekend, however, I managed to have a very productive weekend.

I improved the AssetListPanel wxControl, so it now nicely shows a list of assets (in either a list view or a tree view), polished that a bit more too (making it so that the selected item is mirrored on the other view, and adding a custom wxEvent for an item being selected). I created a BitmapWindow, which is a form of wxScrolledWindow for displaying bitmaps (the background is the grey and white check pattern that will be familiar to anyone who uses Photoshop or other graphics tools) and I created a TextureViewPanel, again a custom wxWidget for displaying a texture.

Here’s what it looks like:

Asset Manager showing a texture in it's original, and compiled versions

The image on the left is the original source image, and the image on the right is the compiled version, I still need to add some stats for the compiled version and add them to the property grid the same as they are for the source image.
I also, (kinda) have compiling textures in, but it needs a lot of work, currently if you select a texture and it can’t find the compiled version, it will compile it, however, currently the compiling only supports DTX1 format and it takes ages blocking the main thread the entire time it does it. So, next on the todo list is to move that on-to another thread and stick a progress bar on there. I’m using LibSquish to encode the image for me, which took a little while to getworking but seems to produce good results and doesn’t involve too much code.

The current todo-list, in no particular order, is:

• Add support for the other DTX formats
• Crete a custom wxImageHandler to handle loading of the complied textures
• Create a simple job system (that the engine will also use) and use that to take care of compiling textures
• Add an ‘update’ button to the TextureViewPanel that will save off any changes to the compiled texture and recompile it
• Add buttons to lock the source and compiled texture view scroll bars
• Add buttons to add folders and textures

I think I’ll get all the basic tasks done before moving on-to the job system, as that will be a bit more involved (and hopefully more fun)