Using GTATools

This page aims to provide a short introduction to using gtatools.

Note that gtatools has currently no stable API, so parts might be changed drastically, be removed or be rewritten at any time. This is because I don't always know where the project is going to go next, and sometimes new features can only be added by changing internals of the engine. If you want to use gtatools, you should either stick to a specific revision, or be prepared to change your code a lot.

The GTATools Programs

Apart from the core libraries, gtatools comes with a few programs.

The command-line programs gtaimg and gtatxd are utilities for working with IMG and TXD archives. Both utilities support reading the files. gtaimg also has experimental writing support. gtaimg can list, extract, append and remove IMG archive entries, and create IMG archives. gtatxd can list textures inside TXD archives and extract them as raw data or PNG files.

You can run gtaimg --help for instructions on how to use the program. The same goes for gtatxd.

The command-line programs nxcommon-test and gtaformats-test provide some automated unit tests for the libnxcommon and libgtaformats libraries. They still only test a few parts of these libraries. You can run them with the option --nxcommon-test-help or --gtaformats-test-help respectively to see how to use them.

gtatools-gui is a GUI program based on Qt that can display all kinds of GTA files. It can e.g. render DFF files, show textures inside TXD files, display RWBS files (DFF and TXD) low-level using a section tree and hex editor, render IFP animations on a specific mesh and other stuff. It can also display IMG files as if they were directories. It is mainly used for playing around with files and debugging the engine, and will frequently crash, so beware. Currently, it works (well, if it works) by setting up "profiles" for specific game installations in the settings. This way, you can e.g. switch between San Andreas and Vice City, or different installations of the same game.

gtatools-test is a highly chaotic prototyping program and the main program that I use for testing the engine. It does whatever I currently want to test, and can serve as a demo tool for the rendering capabilities of the engine. It runs off a configuration file called config.xml which you need to place in either the directory the executable is found in, or the directory it is running from (working directory). You can find an example config.xml here.

libnxcommon

libnxcommon is not directly part of gtatools, but kinda maintained along with it. I use it for my other projects, too, and it is meant to contain some general-purpose utility classes and functions as well as some platform abstraction layers. I like to think of it as my personal extended C++ standard library. Many classes (like CString or File) are heavily used throughout gtatools.

libgtaformats

libgtaformats is one of the two core libraries of gtatools, and represents the lower-end functionality. It contains all the format loaders for the GTA formats that gtatools supports. You can use it to load a DFF file into a C++ class structure that you can use to do whatever you want for example, like converting it to another mesh file format. You can use it to read textures from TXD archives and display or extract them. You can traverse IMG archives and extract files from them. All kinds of weird and wonderful stuff. It has some experimental writing support for IMG, TXD and RWBS (the low-level format of DFF/TXD) files.

I have written a small example program that lists all files inside the gta3.img file of a GTA game and extracts the first file from it. You might use it to test whether your gtatools build is working. Just change the GTA_ROOT_PATH define to the root of your game.

On my system, I would use the following command to compile the test program:

    g++ -fPIC --std=c++11 -I/home/alemariusnexus/Qt/5.3/gcc_64/include -I/home/alemariusnexus/gtatools-install/include -L/home/alemariusnexus/gtatools-install/lib -o imgtest imgtest.cpp -lgtaformats -lnxcommon

Be sure to specify library and include file search paths to all libraries using -I and -L. The option --std=c++11 is also necessary, because some C++11 features are used in gtatools' headers.

If you want to know more about using it, look at the header files in the format subdirectories. You can also look at the source code of the gtaimg / gtatxd command-line programs for examples on IMG and TXD, at the source code of libgta for its usage inside the engine (like going from a DFF file to a renderable mesh in OpenGL), or at the source code of the format subdirectories of gtatools-gui for using it to display all kinds of GTA file formats.

libgta

"Wow. That is some messed-up and convoluted library", I hear you say. That's true, because I don't really know where that one is going. It is built upon the features that libgtaformats provides and implements the engine of GTATools. It is roughly made up of the following parts:

Resource Indexer System: This part indexes files, so that things like meshes or textures in TXD files can be quickly found by name.
Resource Management System: Provides a transparent cache for resources like meshes and textures, so that when a specific resource is requested, if it is not yet cached, a resource loader will load this resource from one of the GTA files (using the resource indexer system) and put it into the cache. If a requested resource is in the cache, it can be delivered very quickly because it is kept in main memory. This system works using resource pointers that can be dereferenced to actually access resources.
Scene System: Provides a means of putting together different SceneObjects to create a scene. A SceneObject can be a static, animated or timed map object, a vehicle (currently in testing), a pedestrian and other stuff. This system also contains the routines for loading these SceneObjects from GTA files (IDE/IPL), handles streaming of objects that are far away from the camera and determines rough object visibility (with the potentially visible set algorithm, PVS) to optimize the rendering process.
Rendering Entity Generation System: This system converts SceneObjects to RenderingEntities that can be displayed by the rendering system.
Rendering System: This system is responsible for actually displaying a a scene graphically. It uses OpenGL as its rendering backend and provides order-independent transparency (OIT) using the Depth Peeling algorithm. It has some code for shadow volumes, but that is currently disabled and some code is missing. It provides basic lighting support.

libgta needs quite a modern graphics card to run currently. You will definitely need OpenGL 3.X. 3.1 may or may not work, but I recommend at least 3.2. Specifically, this is why the open-source GPU drivers on Linux do not work right now, because they only provide OpenGL 3.0. This requirement could generally be a bit lower, but OpenGL 2.x can only be supported with some features disabled or some very sinister trickery, and 1.x is definitely not enough, because I'm using shaders all over the place.

I can't really give usage instructions for libgta, because it changes so much. If you think you can take the pain and want to try using it in your own program, you might want to look at the source code of gtatools-test for an example of how to use it. Some parts of gtatools-gui's code might also be interesting, especially the DFF format subdirectory, and the IFPAnimationViewer stuff which use libgta to render models and animations.

Troubleshooting

Here's a list of things that might go wrong and how to deal with them:

When gtatools-test complains about a missing config.xml, look at the section on gtatools-test above. You need to download the file config.xml.
When gtatools-test complains that it can't find the GTA installation root directory, make sure you have to correct path to your GTA installation set in config.xml.
When gtatools-test crashes with some OpenGL related error, especially something with framebuffers or an error while compiling the shaders, you might need a more recent graphics card and/or graphics driver with support for a modern version of OpenGL. As stated above, the open-source drivers on Linux will currently not work, you have to use the official drivers instead.
It might also just be that something is broken in libgta. I test this mostly on an Nvidia GTX 460 on Linux with the official drivers, so I don't know about specific quirks of other GPUs/drivers.
If you get random crashes when doing random stuff, or when it deletes your hard drive (kinda unlikely because I refrain from using recursive delete ;) )... well... I told you this project is experimental!