Clover.moe

Clover’s Toy Box 22.10

Clover’s Toy Box development: adding Wolfenstein model formats, MP3 with metadata, and vertex tangents.

After posts covering 4 months (22.04), 3 months (22.07), and 2 months (22.09) of development it seems only natural for this post to cover 1 month (22.10).

Compared to Spearmint, Toy Box now supports all audio codecs (no audio mixer yet), all image formats (except uncompressed DDS), and has partial support for all model formats. There is still a few difficult issues for models but it seem like these areas (audio, images, models) could have feature parity with Spearmint in the near future. There will be some unsupported model behavior that is unlikely to be used.

The other main areas that need work are Quake 3 .BSP (levels), collision, and networking. Toy Box is also missing many rendering features and the actual game-logic. So yeah, there is still a long way to go but it’s nice to feel like there is progress.

Models

  • Added support for Return to Castle Wolfenstein .MDS skeletal models.
  • Added support for Wolfenstein: Enemy Territory .MDM skeletal mesh and .MDX skeletal animation.
  • Added support for Wolfenstein: Enemy Territory .TAG models.
  • Added support for Kingpin: Life of Crime .MDX models (unrelated to Wolf:ET).
  • Added more support for vertex tangents.
  • For Enemy Territory: Quake Wars mod kit (SDK); Support vertex colors and missing bind pose in .MD5MESH.
  • Added support for Misfit Model 3D .MM3D skeletal “points” (tags in Quake terms).
  • Added support for vertex tangents and colors in Inter-Quake Export .IQE models.

RTCW / Wolf:ET

I added support for Return to Castle Wolfenstein .MDS. It’s another version of MD4 with unique method of bone joint storage and uses collapse map for level of detail support. I don’t support the dynamic torso rotation, torso animation, or collapse map yet.

I added support for Wolfenstein: Enemy Territory .MDM and .MDX. They are RTCW .MDS split into mesh and animation files; similar to FAKK/Alice .SKB and .SKA. MDM adds tags that are relative to a bone and bones are only in MDX.

For Wolf:ET model formats, I decided to try to properly support separate mesh (without bind pose joints) and skeletal animation models. (For FAKK/Alice .SKB/.SKA I cheat and load the other from the same directory, though this may not be correct.) There were some pretty intrusive changes to support skeletal mesh model without bind pose joints. I have to support models with no joints rendering using the joints in the (animation) frameModel and support MDM adding additional tag joints on top. The number of joints/tags is based on the combination of the model and (animation) frameModel, not either model on it’s own.

The Wolf:ET models work surprisingly well but there is some visible issues. I don’t correctly handle MDM vertexes with multiple bone joint influences yet due to missing bind pose joints and not improperly getting them from an animation.

I want to figure out how to calculate usable bind pose matrices using the vertex bone-space positions as it would be less intrusive model support through the engine and allow for GPU bone skinning. Though I currently only support 4 joint influences per-vertexes for GPU bone skinning and the Wolf:ET MDM models use up to 5. I may change to 8 in the future (it requires additional GLSL vertex attributes and shader variants). Worst case, I have to deal with storing and using the bone-space positions when rendering using CPU model animation.

I also added Wolf:ET .TAG “models” which are Quake 3 .MD3 models reduced to just one frame of MD3 tags. It’s used by the Wolf:ET server game-logic for attachment points on vehicles. The Wolf:ET client renderer doesn’t support them. In Spearmint I specifically decided not to add it to the engine because the game-logic can load it itself (even though Spearmint allows the server game-logic to load models through the engine and get the tags). I changed my mind for Toy Box, it’s easy to add it in the model code and I can also render debug axis for the tags and view the .TAG models in Clover Resource Utility.

Kingpin

A year ago I was looking into formats used by Quake 2-based games and started adding Kingpin Life of Crime .MDX based on Quake 2 .MD2. MDX adds a couple sections I don’t support; effects connected to a vertex and per-mesh bounding boxes. Quake 2 MD2 has separate vertex texture coords for software and OpenGL. Kingpin MDX however removed the software texture coords which are what I was using in Toy Box. So I had to parse the MD2 OpenGL commands list of triangle fans and strips and convert it indexed triangles using the triangle’s per-vertex texture coords.

Vertex Tangents

Normal maps allow per-texel lighting direction to give an appearance of more detail on a model. It’s one of the latest and greatest rendering feature of ~2003. Vertex tangent vectors are needed for normal map support to know the up and side orientation of the texture in 3D space. (The vertex normal is the forward orientation.)

IQM and IQE may include vertex tangents. For IQE and other formats they must be generated. Vertex tangents are generated by Doom 3 for .MD5MESH and by ioquake3/Spearmint opengl2 renderers for model formats they support (excluding IQM, which should have them).

I already supported vertex tangents for IQM models (and applying one normal map to all models as a test). I added vertex tangents to the generic vertex so that it’s possible for dynamic geometry, sprites, and CPU animated models to have vertex tangents. I added debug lines for vertex tangents which is very useful when generating them.

I added support to my general model handling (MeshBuilder API) for explicit and generated vertex tangents and vertex colors. This generate vertex tangents for all skeletal formats except IQM (MD4, MDR, MDS, MDM, SKB, MD5MESH, MMD3). I still need to add vertex tangent generation for formats that don’t use MeshBuilder; IQM and formats based on MD2 and MD3.

Conveniently Wolf:ET .MDM contains vertex texture coords and normals so it’s possible to generate tangents even though I don’t know the vertex model-space positions.

ET:QW

I previously added support for loading the Enemy Territory: Quake Wars variant of .MD5MESH (version 11). (The ET:QW game doesn’t use them but the ET:QW SDK contains them, albeit without any .MD5ANIM animations.)

I added support for vertex colors in ET:QW .MD5MESH. All models in the SDK have vertex colors but all the colors are white (“1 1 1 1”). It’s not very exciting but it reused the work I did for .IQE vertex color support.

I added support for loading “*_lod3.md5mesh” which are lower level of detail (LOD) models that do not contain the bind pose joints. This reused the work I did for Wolf:ET .MDM without joints. The LOD models can be drawn using the joints from the separate “*.md5mesh” file with the same base name. I handle this in Clover Resource Utility by treating the base model as the animation frameModel. This does not support animations yet (the ET:QW SDK doesn’t have any though) as it needs to connect the LOD model to the base model to get the joints instead of treating base model as an animation.

Misfit

Misfit Model 3D .MM3D models contain skeletal points (tags) connected to joints similar to Wolf:ET .MDM but with multiple joint influences like vertexes. I added support for this along with Wolf:ET .MDM tags which made it more complicated.

Inter-Quake Export

Inter-Quake Export .IQE, the text version of Inter-Quake Model .IQM, supports vertex colors and vertex tangents. I added support for them using my new common code in my MeshBuilder API.

Audio

  • Added .MP3 playback using public domain dr_mp3 library.
  • Added support for ID3v2 metadata in .MP3 and .WAV.
  • Cleaned up my .WAV loading.
  • Added support for LIST INFO metadata in .WAV.
  • Fixed a large .Opus file crashing the game.

MP3

I got side tracked processing the ID3 metadata that is typically in MP3 files before I added MP3 audio support. I thought it would be fairly simple key-value pairs but it turned out to be pretty complex.

ID3v1 (“TAG” data identifier) is 128 bytes at the end of the file. There is fixed length fields for title, artist, etc. (Values for specific keys, if you will.) Most (all my?) MP3 files have this and it’s very simple to load but it’s limited.

I detect audio files by the content. MP3 files usually start with ID3v2 metadata (“ID3” data identifier). The ID3v2 spec boasts it can be used with other audio codecs. I thought I could parse ID3v2 metadata to skip it and then check the audio data. It turned out MP3 files are frame based and scans for the next frame so arbitrary data can be mixed in (possible in error). I can’t check the data immediately after the ID3v2 metadata to detect MP3. ID3v2 in WAV files is stored as a RIFF chunk; not simply at the start of the file like MP3 files. So my idea of general ID3v2 handling at the start of the file turned out to not be useful and I moved it to MP3 and WAV specific code.

I had some difficulty understanding the ID3v2 specifications. As I understood more it reveled that ID3v2 is kind of full blown archive format with entries. It has separate text encoding per-entry and even offers deflate compression like .zip files. It also has a “unsynchronization” option for when the entry data is altered to avoid looking like a MP3 frame header. (I haven’t added support for deflate compression or “unsynchronization” flag.) There is three incompatible versions of ID3v2—I have all three in my meager library of MP3s.

I spent a bunch of time with trying to understand it and also converting text encodings (UTF-16 big and little endian to UTF-8). After a few hours of working on ID3 support, I realized I probably should of looked for a library for ID3 support but I was already too deep to want to stop.

I found of several weird cases when testing my library of MP3s. Such as placing the string terminator at the beginning of the entry… possibly a side affect of ID3 using a string terminator as a separator between values in the entry but these only had one value. Some files including a second ID3 metadata block (with the same content?). Some files there is garbage between the ID3 metadata and the MP3 audio data (in one file there is ID3v1 metadata, which is suppose to be at the end of the file).

So yeah, ID3v2 is complicated, weird edge cases, and not really a joy to implement. I worked on it three days and didn’t implement the full specifications.

Adding MP3 audio support using dr_mp3 was really easy though. It took like 2 hours to add using the dr_mp3 “pull API” to request samples with read and seek callbacks. I may want to change later to work with MP3 frames so I can handle ID3v2 tags in the middle of the stream. Though this is probably only likely to happen in Internet radio streams.

WAV

My .WAV loading somewhat bizarrely read 20kib of the file and then processed the byte array to find the audio format and start of the audio sample data. If the sample data started more than 20kib in (very unlikely), it would fail. Now it streams the file and processes it byte by byte with the sizes defined in the .WAV file.

I added support for the WAV metadata in Microsoft’s WAV specification (a “LIST” chunk with “INFO” identifier) and then proceeded to add support for “id3 ” chunk that contains ID3v2 metadata. Audacity exports both which gives a way to test it. Audacity also includes the ID3v2 “extended header” to show me yet another way I was incorrectly parsing ID3v2. The ID3v2 header size includes extended header size. I glossed over the wording in the specification; it’s there.

Opus

I tried loading a 8-hour 488 MB Opus file but it crashed. The 64-bit sample data size was 6.0 GB which is over max signed 32-bit integer value (2.4 GB). The size was passed to Opus library op_read_stero() as 32-bit and overflowed to negative and crashed in the opusfile library. After I fixed my code to limit the read size, it took 2 minutes to load the 8 hour Opus file, then reported an error because my sound effect loader only supports 32-bit size. So I moved the error up to before allocating and loading 6.0 GB of audio sample data. Large files should be streamed—and I support streamed reading for all audio codecs—but I don’t support playing streamed audio yet.

Posted

in

, ,

by

The Turtle Man 🃑

Tags: