I’ve completed most of the things I want to do in my open source projects (Spearmint, Maverick, etc). There was of course a lot of other ideas and many are now redirected at Clover’s Toy Box. At this point I mainly work on Maverick and Quake 3 related open source projects for the sake of other people. While I like assisting, it’s not satisfying and can be stressful.

I’m withdrawing from providing user support and fulfilling requests for open source projects. I’m not interested in continuing to discuss these projects. I’ve made the Clover.moe Community Discord server be read-only after giving a month notice. I’ve thought about this for some time; I decided to do this in March.

I will continue to contribute to Quake 3 open source projects if it’s something I personally want. There is still a few loose ends I want to deal with and I may find other things in the future. So it’s not entirely the end of me working on these projects.

* 650% improvement in one level on one set of hardware.

Performance

It took a couple years to get my reimplementation of the Quake 3 renderer (“Toy Box renderer for Spearmint”) to the performance of the ioquake3/Spearmint “opengl1” renderer. It very challenging to meet the performance despite using more modern OpenGL features and it was still missing many features. I also had to disable curved surfaces for it to be faster.

My renderer fell behind again after upgrading to new better hardware and adding additional features (notably Quake 3 materials).

Most of the official Quake 3 maps ran at 1000 frames per-second (FPS). The slowest case that I was aware of was at the center of the Quake 3 add-on level ct3ctf2. It ran at only 100 FPS. Using the Spearmint opengl1 renderer on ct3ctf2 is somewhere between 500 and 666 FPS and the OpenGL2 renderer is somewhere between 800 and 1000 FPS. That’s kind of disappointing for my renderer. (Higher frames per-second is better.)

After making several changes I’ve clawed my way from 100 FPS to 650 FPS at the center of ct3ctf2. A 650% improvement. (These changes will improve other levels as well but it’s not 650% everywhere.) This is hopefully only the tip of the performance ice burg.

The main goal for improving the frame rate is being able to render more content at a lower frame rate or use less power to render the same content (which is particularly relevant for mobile devices).

BeginPerformanceQuery

I reviewed what it was drawing for the ct3ctf2 level. It was uploading 300,000 vertexes and issuing 1,000 draw calls per-frame at 100 FPS.

The curved surfaces were using materials that required using the CPU vertex shader and uploaded 3x vertexes (for each material layer) each frame. Additionally the surface order interleaved flat surfaces (using static vertex buffer) and curves (using dynamic vertex buffer). This resulted in not merging them into the same draw call despite using the same materials.

Materials had for example a base texture, environment map (using CPU vertex shader), and then standalone lightmap (using CPU vertex shader).

1. BSP surface merging

I changed the Quake 3 level surface sorting to sort flat (static vertex buffer) and curved (dynamic vertex buffer) surfaces together so they can be merged into single flat or curve draw calls.

2. Standalone lightmap

Lightmaps are often merged with an adjacent layer to use multi-texture. Both layers are drawn in a single draw call. However multiply blended lightmaps cannot be merged with the additive blended environment map and some other effects. This separate lightmap layer used the CPU vertex shader to copy the lightmap texture coordinate from the multi-texture slot to the base slot.

I could do the same thing in modern OpenGL shaders by adding a texcoord source or additional shader type (both that have more over head). For OpenGL 1.x fixed-function, I would potentially need to rebind the vertex attributes per draw call for the world. It seemed like a mess to do both of these in the same backend.

Instead I made standalone lightmap layers use multi-texture with a white base image. No CPU vertex shader needed or complicated re-architecture the backend. (OpenGL 1.2 is needed for multi-texture so OpenGL 1.1 still falls back to the CPU vertex shader.)

3. Duplicate vertexes

The CPU vertex shader has to add draw calls for each layer. This added vertexes for each layer with a TODO for separating vertex upload from layers. As a quick solution, I made layers that do not require the CPU vertex shader share the same unmodified vertexes. This allowed the base and standalone lightmap layers to use the same vertexes.

4. Hardware tcGen

In the previous article on adding Quake 3 material support I talked about hardware texture coordinate generation and how it can be done in hardware. I added support to OpenGL shaders so it doesn’t need the CPU vertex shader. (I haven’t implemented it for OpenGL fixed-function and Wii yet.)

5. View frustum culling

I’ve supported Quake 3 level’s Potentially Visible Set for a while to drop rendering areas of the map. However this doesn’t work well for large open area.

I added view frustum culling to drop drawing level surfaces and models that are not in front of the camera within the view angle. I wrote most of this code in 2021 or earlier but I didn’t merge it due to a rendering issue that was apparently already resolved since then (a few specific surfaces in some maps disappeared when clearly on screen).

6. Curve tessellation

The curved surfaces in the Quake 3 level formats are Bézier curves that have 3 by 3 patch of control points to define the mathematical curve. I convert it to triangles (tessellate) in order to draw it.

I originally implement it as tessellating it each frame and at a fixed number of triangles; even if say, it’s flat square and only needs 2 triangles. I knew it was slow but it seemed easier at the time when I was trying to get it to work at all. There was an issue that some vertical rounded corners had incorrect lightmap texture coordinates. I tried unsuccessfully to fix it two or three times over the last 3 years. Improving performance was kind of on hold for this reason.

I found the lightmap texture coordinate issue while experimenting with limiting the number of rows/columns. It turns out Quake 3 levels have invalid lightmap texcoords if the control points are not equal distance apart. The vertical rounded corners are flat vertically and rounded horizontally. Quake 3 doesn’t add any rows; it’s just single triangles from the top to bottom. It doesn’t use the middle row of control points with invalid lightmap texture coordinates.

I completely rewrote the tessellation code. Curves are now tessellated at level load instead of each frame. The curves are now subdivided based on how curved it is (less triangles in most cases). Curves could have gaps between touching patches due to how they’re subdivided now. Curves are now stitched together by detect common edges and adding rows and columns to adjacent patches.

Curves are now faster to draw and match Quake 3 visually. (Though I haven’t added dynamic lower detail far away yet.)

EndPerformanceQuery

The center of ct3ctf2 has moved from uploading 300,000 vertexes per-frame to only 3,500 vertexes and from 1,000 draw calls to 500 draw calls. 100 FPS to now 650 FPS. I still have more ideas for improving performance but I got sidetracked on adding features again.

Mirrors and Portals

I previously added mirror and portal rendering support in Toy Box but it has fallen into disrepair. I hadn’t ever hooked it up for Quake 3 maps or “Toy Box renderer for Spearmint”.

I fixed the mirrors in Toy Box to handle framebuffer object support that was added ages ago and fixed OpenGL 1 clip plane rotating based on whatever GL_MODELVIEW matrix was previously set.

However remember that new view frustum culling? Yeah, the culling for the main view applied to the mirror views so mirrors didn’t draw anything behind the player. Sprites also faced the main view instead of the mirror view. I was in mirror hell for a month and a half. I didn’t want to work on it for whatever reason but felt like I shouldn’t do something else so I just didn’t work on Toy Box very much as a result.

(If this show up in Inksie’s analytics, this is just a game programming blog where I talk to myself that isn’t profitable and I really like your shorts. And maybe mirrors are the problem.)

a: want to know who is to blame for your low self-esteem?
b: yes!
a: close your eyes and I’ll show you.
a: open.
a: that’s right-
b: mirrors!
a: no.

When the level model was added to the scene it immediately performed culling and added entities for the visible geometry. I was able to add mirror/portal entities here.

In my mirror system, it drew a model the stencil buffer and only drew the mirror view in the marked area. (This allows multiple mirrors in the main view without issues.) I was hung up for a while with how to draw the surface for Quake 3 mirrors. As a initial hack I just used the Quake 3 explosion model (it’s just a square) so I could continue working on it.

If I add the mirror/portal after the CPU vertex shader processes the material, it may have the wrong surface normal for the camera. So the mirror needs to use the source surface. However the material could move around so limiting the mirror to the source surface area is not correct. Quake 3 only draws one mirror/portal view and it draws on the whole screen and then draw the level over it. That’s ultimately what I decided to do. It’s essentially what I was doing with the square model scaled up but with less steps. This allowed mirrors to work but with the wrong view culling and sprite orientation.

I changed adding the level model to just create an entity with the information and later when processing entities for a scene actually add the entities for the visible geometry and mirrors/portal surfaces. This was not entirely straight forward but it had been my on TODO list for a while.

After this I made processing entities for a scene end with looping through the mirrors and add mirror view entities with have their own list of entities to draw. This included culling and added level models and generating sprite vertexes for the mirror view.

In Quake 3 levels mirror/portal surfaces do not directly specific the destination view. This is specified by the game code at run-time but it doesn’t directly specific which surface it’s for. How/when to connect this was logistic problem. However the bigger problem is it just specific a vector for the view directory and a bunch of options for how to set up the view axis (even though it literally passes the view direction in a view axis).

I still haven’t entirely implemented it. One option is the roll for the camera and it’s mainly used to just fix Quake 3 being terrible at setting it correctly. So currently there is an inconsistency in an add-on level that the view in upside-down in Quake 3 but right-sideup in Toy Box (this doesn’t look like it’s intentional upside-down).

This was working pretty well but things were getting unexpected culled in mirrors. I thought it might be a problem with like the matrix math for modifying the mirror view by the main view or the view frustum for culling. I spent a fair amount trying to debug it by drawing the camera location. This was a annoying problem of how do you draw the mirror view location in the main view and vice versa when I don’t easily have access to it with how this is structured? Two static variables and flipping the which you set and read.

However this wasn’t the problem at all. It turn out I was using the mirror camera location modified by the current main view location for the Quake 3 level’s Potentially Visible Set and it moved through walls and obscured parts of the level and so they were not drawn. Using the actual mirror location solved the issue.

I added support for only drawing models in the main view or in portal views so that Quake 3 player models draw in mirrors when using first person model. Now I’m just disappointed I went though all this work and Quake 3 only has like 7 unique mirrors/portals in it.

Whatever, I’m out of mirror hell for now.

Large level support

Rendering has a maximum distance. I set it fairly high but it cuts off some large levels (such as Quake 3: Team Arena mpterra[1-3] maps) and q3map2 _skybox entity. Setting it higher reduces depth precision and (for Quake 3 support) I don’t have control or ability to review all of the content to set the max depth distance to fit the content.

I added dynamic depth near and far plane by tracking the bounds of the 3D scene and then calculating the minimum and maximum distance from the camera. I haven’t added bounds tracking for all render commands yet. Though I also need it for adding view frustum culling for everything.

I had to rework the skybox drawing as it needs to have the geometry inside the max depth but also be depth value farther than everything else. I use glDepthRange( 1, 1 ) to set it to the max depth value and expand the scene bounds to include the skybox size. Though recently there seems to be some issues in mirrors. (Mirror hell doesn’t end.)

General

• Add dark mode color scheme for model background
• Add frames, FPS, and loop to New Animation window
• Add Edit to animation mode for changing the same fields as New Animation window
• Make Animation Sets window use New Animation dialog
• Replace Rename in Animation Sets window with Edit for name, frames, FPS, and loop
• Rename texture to material in Edit Groups window
• Change max frames for an animation to 9999 (previously 999 but convert to frame animation was limited to 99)
• Fix not updating frame count in animation mode until focus changes, text editing is disabled now
• Fix ‘paste animation frame’ without selecting an animation
• Fix misplaced “fi”s for configure
• Fix ignoring user CXXFLAGS in configure
• Fix compiling on GNU Hurd

Documentation

• Add dark mode support for web browsers
• Fix viewing in-app help in dark mode

Model Formats

• Add Quake 3 player export for IQE

GNU/Linux

• Add changelog to Linux appstream metadata
• Update Flatpak runtime to org.kde.Platform 5.15-23.08
• Don’t use Wayland unless requested (environment variable QT_QPA_PLATFORM=wayland)
• Fix finding Qt translations in Flatpak

macOS

• Fix configure failure on newer macOS
• Fix installing Qt translation in the appbundle

How not to update Qt

The most recent build of Maverick for macOS had been 1.3.12 released in 2019. Maverick uses the Qt GUI framework. The Maverick 1.3.12 release uses Qt 5.11. I wanted to update it to Qt 5.15 but Qt 5.11 was the last version to support macOS 10.11 which my MacBook Pro is limited to.

I forked the several repositories required for building Qt and patched it to cross-compile from Linux for macOS (as it must be built using a newer macOS SDK) and readd support for macOS 10.11. Building had some hard coded paths specific to my machine and it didn’t handle new Qt rendering classes correctly for macOS 10.11.

I thought I would also use this customized Qt for other applications I develop (in Toy Box). Ultimately I dropped interest in using Qt aside from maintaining Maverick. I don’t want to maintain Qt or feel like I need to ensure the customized source code (1 GB zip) remains available long term.

Continuing to use the existing version of Qt was a problem due to sort of breaking the install. I had updated Qt to a non-working version and Homebrew package manager was in a broken state. I did get Homebrew and Qt 5.11 working again at some point.

Conclusion

There is now Maverick 1.3.13 and 1.3.14 macOS builds using Qt 5.11. I don’t think they should be any worse than using Maverick 1.3.12.

These new builds are still made on a 2008 Intel MacBook Pro. Thank you to the people who donated toward new Mac hardware for being able to update Maverick. The money is still set aside for Mac hardware whenever I have enough.

Background: Clover’s Toy Box (2017—) is my private 3D game engine from scratch project. It’s a continuation of ideas for Spearmint (2008—), my enhanced version of the Quake 3 (1999) engine. I cut my teeth modifying SRB2 / the Doom (1993) engine in 2006-2008. I’ve done computer programming for about 18 years.

Materials

About a year ago I started working on adding support for Quake 3 materials (*.shader files) to Clover’s Toy Box and Toy Box renderer for Spearmint (my private reimplementation of the Spearmint renderer using Toy Box).

Quake 3 materials define how to draw an image on a 3D model, game level surface, or in the menu. They allow for multiple blended images, animated image sequences, and dynamic effects such as scrolling an image, flashing color, or changing the position of the surface.

It’s difficult to implement the Quake 3 material system as there isn’t a complete definition of how it works, it’s complicated, and it interacts with all rendering. It kind of spiraled out into implement or fix all the rest of Quake 3 renderer features in Toy Box renderer for Spearmint.

Most of the Quake 3 material features are supported by Toy Box now. It’s missing the sky dome, fog, and a few position modifiers (deformVertexes). It has some of the additions made in Spearmint but I haven’t focused on it. However there is still many issues for rendering Quake 3 that are not directly part of the material system.

My Toy Box renderer—including the implementation of the Quake 3 material system—runs on OpenGL, OpenGL ES, and WebGL as well as the Wii console. Though the Wii console runs out of memory loading level textures (only 88 MB of RAM) and it’s missing an implementation of polygonOffset for preventing decals from flickering.

It runs on modern and legacy OpenGL. (OpenGL is a programming interface for hardware accelerated graphics rendering on a graphics card.) Toy Box is compatible with modern OpenGL 3.2+ Core Profile and streams geometry using OpenGL 4.4 persistent mapped buffers. I’m particularly proud (amused?) of the Quake 3 material system being fully implemented on legacy OpenGL 1.1 which Quake 3 supported in 1999.

(I also fixed ioquake3 and Spearmint to fully support a sky box using OpenGL 1.1 which may be useful for some Intel graphics under Windows 10 stuck with Microsoft’s generic OpenGL 1.1 driver.)

Implementation

Some parts of the material system were straight forward to add. Others not so much. I spent a lot of time testing Quake 3 format levels (official, add-ons, and other games). I found issues and made a list of them to look into. Looking into issues often found it was a different manifestation of a known issue that I hadn’t fixed yet.

I have a list of like 50 issues and not everything made it onto the list. It’s honestly not that exciting to recount. Things were broken and then I fixed them.

The material system can kind of be broken down into five categories:

• Material file parsing, implicit keyword behavior, and sort order.
• Changing OpenGL rendering state.
• Changing vertex attributes (position, normal, color, texture coordinates).
• Changing vertex attributes but behavior very specific to Quake 3.
• Special handling for the sky and fog volumes.

Material parsing

The Toy Box Quake 3 material loader handles various implicit behavior. If a material has “rgbGen vertex”, it default to using “alphaGen vertex”. However “rgbGen exactVertex” uses opaque alpha which is inconsistent. Using image blending disables writing depth which is to prevent surfaces behind it drawing over it. There is several things that affect the implicit sort order. It’s just a lot of random stuff to fill out the full Quake 3 material definitions correctly.

The Quake 3 material definitions are converted to a separate material system that has some additional features and different implementation of some features. My intention is to create a new material file format that doesn’t depend on Quake 3’s implicit behavior.

OpenGL state

Changing the OpenGL rendering state was mostly straight forward to add. Many of the material keywords are easy to infer how they directly map to the OpenGL API. Face culling, blend modes, alpha test, depth test, depth write, and polygon offset.

Though colors being floating-point (i.e., 1.0) in the material definition and converted to an 8-bit integer (i.e., 255) when it’s loaded was not obvious and affected alpha testing for conditional transparency.

A material for an unofficial Quake 3 add-on level (xccc_dm4) specified using alpha 0.5 and set the alpha test to greater-or-equal to 0.5. This caused transparency to flicker because OpenGL rendering doesn’t have perfect precision and alpha values vary slightly above and below 0.5.

Quake 3 converts the alpha 0.5 to 8-bit integer (0.5 × 255 = 127.5) and rounds it to 127 and then the OpenGL driver compares 127 ÷ 255 as a floating-point value 0.498039216 with alpha test reference 0.5. This way there is no flickering. Though it doesn’t draw at all as the alpha is always lower than 0.5. This is apparently what the creator intended as it’s not visible in a video they made of the level either.

So I convert the floating-point value from the Quake 3 material to an 8-bit integer and then back to a floating-point value as that’s what I’m using for colors in Toy Box.

Vertex attributes

There are vertex attribute generators/modifiers for vertex position, normal (direction), texture coordinates, and color. I implemented all of them in the CPU vertex shader (it’s just a function that outputs new vertexes). This way they work on all graphics APIs (OpenGL fixed-function and shaders, Wii, …).

The CPU vertex shader which I started quite some time ago (see Toy Box 22.04 § “OpenGL 1.1 fixed-function rendering”) had to be expanded to allow for multiple material layers with separate generated/modified vertex attributes and to support many new effects.

I also finally replaced the initial CPU vertex shader specific to OpenGL 1.x (low-level, non-VBO compatible) with the API-independent code. Now OpenGL fixed-function rendering can use persistent mapped buffers and vertex buffer objects.

I mark material layers for which attributes need to be processed on the CPU and then tell the GPU to just use the submitted values for that attribute. This allows mixing CPU and GPU vertex attribute handling. For example, processing the position modifiers on the CPU and using the GPU to apply the texture matrix for texture coordinate modifiers.

Most of the Quake 3 position modifiers need to be applied per-vertex (opposed to a matrix multiply). This is always done in the CPU vertex shader as it isn’t compatible with graphics APIs for the Wii or OpenGL 1. They could be implemented using OpenGL 2 GLSL shaders but it’s very specific to each position modifier. I’m trying to keep the graphics backends kind of generic.

Currently if any attribute requires the CPU vertex shader it results in animating the model on the CPU instead of using OpenGL 2 GPU skeletal or frame animation. This is kind of a disappointment for “skeletal models with mixed CPU and GPU vertex attributes” as the performance is probably largely affected by animation. Though I haven’t considered what cases it would be beneficial to optimize this for; a lot of the CPU-only effects use the vertex position.

Vertex colors

I split up some of the Quake 3 material keywords so they could be handled in a more general way.

Spearmint has twelve RGB vertex color generators. In Toy Box the color generators are split up into four base color generators (white, vertex, one minus vertex, model lighting) and five color modifiers (constant color, waveform for color intensity, entity color, one minus entity color, and underbright to counteract Quake 3 scene overbright).

This replaces having lightingDiffuse and lightingDiffuseEntity for model lighting without and with entity color, and const, wave, and colorWave for white with constant color, a waveform for color intensity, and both.

It’s easy to tell the Wii to use white or vertex color but one minus vertex and model lighting use the CPU vertex shader to generate the color arrays. The color modifiers can all be combined and applied using the Wii GPU. Most of the color generators are GPU accelerated on the Wii. One minus vertex which is rarely used (I haven’t actually checked if it’s used by anything). Eventually I need to figure out GPU model lighting for supporting normal maps (per-pixel light direction).

OpenGL 2 GLSL can do all of the base types + apply the color modifier on the GPU. Though I currently have one minus vertex disabled because I’m not sure it’s worth supporting it in the backend.

OpenGL 1 can only do white and vertex color (like the Wii) but it can only apply color modifiers on the GPU if it uses white and not vertex colors or model lighting. So the renderer just marks the material layers using vertex colors or model lighting to use the CPU vertex shader for color modifiers. Model lighting already uses the CPU vertex shader so it doesn’t really affect much.

Vertex texture coordinates

Spearmint has six texture coordinate generators (base texture, lightmap, vector w/ matrix, cel-shading, two types of reflection-mapping) and in Toy Box they’re split up into a five base generators (base texture, lightmap, position, normal, and reflect) and a matrix if needed. Reflect is shared by the two reflection mapping methods for Quake 3 and Raven Software’s Quake 3 based games.

This doesn’t reduce a lot but it does make it easier to implement GPU support as a base type + texture matrix. I haven’t added them yet but OpenGL 1.3 fixed-function and 2.0 GLSL can support all of them. The Wii has GPU support for all generators except reflect.

I tested generating reflect vectors and storing them in the vertex normals on OpenGL for using reflect via the normal generator and it works. This may be a way for the Wii to utilize CPU reflect with GPU texture matrix. Eventually Wii can have at least partial GPU support for all of the texture coordinate generators.

Vertex attribute conclusion

The Toy Box material handling is being designed around GPU color multiply and texture matrix multiply. Quake 3 has more specific handling for combining some generators and modifiers to allow for better CPU optimization. Scrolling a texture only needs two additions per-vertex, not a full matrix multiply.

My understanding is that the Wii always does a texture matrix multiply. It can be set to an identity matrix but it can’t be disabled. So it may not require extra processing to use a texture matrix multiply aside from the data transfer of the matrix to the GPU.

Very specific behavior

Some of the material keywords are very specific to Quake 3 and it would be very difficult to accurately remake them without basing it on the math from Quake 3. Math can be patented but not copyrighted. So it seems that using these equations would not be copyright infringement. Though they aren’t really necessary for creating a new game and I’d probably leave them out of a new game.

Keywords such as tcMod turb for turbulent lava texture scroll and alphaGen lightingSpecular for slightly non-standard specular with a hard coded light origin. I haven’t decided if I should try to remake the sky dome texture coordinate generator and so the sky dome is currently missing. Quake 3 uses these on basically every official level.

Future plans

I had thought about releasing closed-source commercial software (content tools, game engine) compatible with Quake 3 data formats as a way to try to monetize Toy Box. Though it’s very unrealistic to generate the level of revenue I would like it to and it would probably just become a burden.

I’m no longer planning to release any software based on Toy Box supporting Quake 3 data formats. I’m not planning to pursue selling or releasing a content application or game engine. I’m also not planning to reprogram and release Turtle Arena (without bots) on Toy Box.

I’m satisfied with what I’ve accomplished in Clover’s Toy Box so far and I intend to continue developing it. Currently the only planned software to release based on Toy Box is original games (which I don’t actually have plans for).

I want to move in the direction of focusing on art with the end goal being images and videos instead of a full game. Though that’s not necessarily related to Toy Box and there is a lot of stuff I need to deal with before that.

New Git Repo

The source code has moved from a flexible_hud branch and tags in zturtleman/ioq3 repository to a new zturtleman/flexible-hud-for-ioq3 repository.

Flexible HUD was treated as a one-off throw away project in 2013. The source code for first two releases was previously missing from the Git repo. They had “git diff” patches in the Flexible HUD pk3 downloads. Now the code is committed and tagged in the Git repo.

ioquake3

I merged in the latest changes from ioquake3 (2019 to 2024). It includes these changes I made in 2019 to 2021 that affect the mod code:

• Restore bots crushing unseen player on q3tourney6 in non-CTF (restores the original harder difficulty of the final boss)
• Fix team orders menu not listing clients with lower clientnums (it wasn’t possible to order bots that joined before you on a server using the Q3 menu)
• Fix duplicate team join center print for bots and g_teamAutoJoin (too many messages could cause an error and kick players off the server)
• Fix lightning gun handling for corpses and single player podiums (this shouldn’t affect anything)

Config

I added my 2018 config file for applying Spearmint’s “Very High Quality” graphics settings to ioquake3. The Flexible HUD web page says how to apply it.

This was previously a standalone download. I don’t remember where if anywhere it’s linked. It seems relevant to users of Flexible HUD so I included it.

Rounding it out

The Flexible HUD downloads and ztm-ioq3-veryhighquality.cfg are now hosted on GitHub instead of clover.moe/downloads. This was actually the main motivation for all of this. I think about moving to a static website and I don’t want to add a bunch of zip files to the website Git repository.

SDL

Simple DirectMedia Layer is a cross-platform abstraction layer for window creation, graphics initialize, audio, input, etc that is used by most games on Linux. (SDL source code)

I mentioned custom window decoration “hit testing” resizing issues at the end of my Toy Box on Wayland post. Someone fixed Linux (X11 and Wayland) to display cursors when hovering over resizable areas. It assumed the cursors were double arrows like on Windows and KDE. Resize left and right both had a single arrow pointing right on GNOME. I fixed it to have the correct cursors on GNOME using 8 cursors with each resize direction.

I helped explain an issue with Nintendo GameCube Adapter controller mapping so it could be fixed on Windows and Android.

I try to read most of the SDL 3 commits by following an RSS feed. I pointed out a few minor issues as commit comments.

ioquake3

ioquake3 is a project to maintain and extend the source code for the 1999 first-person shooter Quake III Arena. (ioq3 source code)

I made 25 posts to help solve issues on the ioquake3 forum.

Platform

I updated SDL libraries for Windows and macOS from SDL 2.0.14 to SDL 2.24.0. I cross-compiled mingw-w64 and macOS libraries on Linux. I did have to use Windows to build the MSVC libraries.

I made it so macOS can be built separate for “legacy” Universal Bundle (x86/x86_64/powerpc) and “modern” Universal Bundle 2 (x86_64/Apple Sillicon). Legacy macOS App Bundle was updated to SDL 2.0.22 as newer versions (jumping to SDL 2.24.0) dropped support for macOS 10.6. (Building for Apple Sillicon was added by Tom Kidd in 2022.)

I fixed ioquake3 failing to start from macOS terminal due to how the URI scheme handler support was added.

I updated the Windows “NSIS installer” script. ioquake3 and games based on it tend to just use a .zip file instead of an installer. It would be useful to use the installer as it adds integration for the “quake3://connect/127.0.0.1” URI scheme handler to Windows.

QVM

QVM (Quake Virtual Machine) is a cross-platform bytecode format used by Quake 3 that allows game modifications to be compiled on one platform and run on all platforms. (A similar recent technology being WebAssembly.)

I fixed compiling QVMs on Linux if the source code has Windows line endings. MS-DOS used CR+LF to end lines in text files (Carriage return, line feed; commands for a text printer). However Unix-like platforms used only LF. The Windows file API in text mode automatically reads CR+LF as LF but other platforms did not and caused the QVM tools to fail with strange syntax errors.

I made it so QVMs are compiled on all platforms by default even if they do not have a run-time QVM just-in-time compiler. The QVM interpreter works on all platforms. This made QVMs be built on Linux/macOS ARM64 by default.

OpenGL2 renderer

Compared to the opengl1 renderer—which closely resembles the original Quake 3 renderer—the ioquake3 OpenGL2 render has a lot of issues and (in my opinion) poor design decisions. I think it’s too much work to fix it. However when prompted about issues, it’s not that I can’t fix them…

I fixed the edge border for smaller view size (cg_viewsize) being drawn to an framebuffer object and not blit to the screen when using HDR or framebuffer multisample anti-alias with post-process. This was fixed to draw to the screen directly like other 2D drawing.

I fixed updating the loading screen with r_cubeMapping 1. The changes for the previous issue caused Quake 3’s drawing all images to the screen—to ensure the driver loads them—to be visible. However it shouldn’t be visible because the loading screen draws over it. It turns out generating cube maps for the level left the culling state set to hiding the front of polygons and the loading screen was culled until the level is loaded to reset the culling state. This was broken for years but it seemed like it was just really fast after loading the level.

I fixed the developer option to clear the screen to magenta (r_clear 1) before rendering when using HDR or framebuffer multisample anti-alias. This makes it obvious when some part of the screen isn’t drawn or is transparent and the previous frame is visible (hall of mirrors effect or random flashing between two different frames; it’s just bad).

I fixed framebuffer multisample anti-alias on AMD Windows driver. The driver incorrectly requires GL_EXT_direct_state_access extension to bind a renderbuffer for it to be valid. This should only be required for OpenGL Core context direct state access or GL_ARB_direct_state_access extension. Though yes, ioquake3 should probably stop using the EXT extension in an OpenGL Core context.

I fixed q3map2 lightstyles effects for dynamic pulsing lightmaps with r_mergeLightmaps 1 and r_sunlightMode 1.

r_mergeLightmaps 1 combines internal 128×128 pixel lightmaps into a larger atlas and modifies the geometry texture concordances. This broke materials using both internal and external lightmap and materials using texture concordance offset with internal lightmaps (two obscure things I didn’t know q3map2 did). I corrected external lightmap to apply a transform to convert the texture concordances back to the original and made offset for internal lightmaps use the scale of the texture atlas.

r_sunlightMode 1 changed lightmap stages in materials to support cascading sun shadows by using white image modulated by the lightmap texture. However it didn’t work with some blend mode that use the alpha of the lightmap texture. I fixed it to only apply to normal lightmap stage blend modes.

I fixed parsing q3gl2_sun without two additional “optional” tokens. Quake 3’s text parser has an option to not allow line breaks but it still sets the text pointer to after the line break. So effectively it can only check for an optional token once. And then must not check for any more because it will be on the next line. I fixed parsing q3gl_sun to only check for additional optional tokens if the previous optional tokens existed. (I previously fixed the parser in Spearmint to stop at end of line so it doesn’t have this problem but I’m more concerned about compatibility with weird content in ioquake3.)

World of Padman

World of Padman is a freeware first-person shooter based on the ioquake3 engine. (WoP source code)

My widescreen HUD support from “ZTM’s Flexible HUD for ioq3” was merged into World of Padman for version 1.7. I helped fix two widescreen issues with health station icons (distance independent 2D sprites on the HUD) and lens flares.

Some of the OpenGL2 renderer changes in ioquake3 were for World of Padman or at request of one of the World of Padman developers. q3map2 lightstyles effects are used by the wop_trashmap level.

Q3Rally

Q3Rally is a freeware racing and third-person car shooter based on the ioquake3 engine. (Q3Rally source code)

Some changes I made include:

• Bots can drive around more race maps now (“Fix bots going in reverse for no reason in races”)
• Fixed spectator observer camera to rotate smoothly
• Fixed intermission view angles
• Fixed players in race have an invisible chainsaw
• Fixed client error dropping to menu when player dies if sv_maxclients is too high (“Fix out of range death events”)
• Updated to latest ioquake3
• Various other bug fixes

I fixed game network compatibility and enabled ARM64 support for the new Q3Rally Flatpak package for Linux.

Spearmint

Spearmint is my enhanced engine and game-logic based on ioquake3. (engine source code, game-logic source code)

opengl1 renderer

I fixed two issues in the opengl1 renderer due to adding changes from Wolfenstein: Enemy Territory.

Wolfenstein: Enemy Territory added far plane culling. The level BSP node bounds doesn’t include surfaces for q3map2 skybox portal hack (_skybox entity) so the far plane wasn’t set far enough and the skybox scene was cut off. Instead the bounds of surfaces in the BSP node should be used, which Wolfenstein: Enemy Territory also added.

Wolfenstein: Enemy Territory fixed surface culling for two sided level surfaces but this broke snow flakes in rgoer_seasons user created level for Quake 3 using “deformVertexes move” material feature to change the surface position. I changed back to Quake 3 behavior of not using exact view culling for two sided surfaces.

OpenGL2 renderer

I fixed bullet/explosion marks on doors and moving platforms in splitscreen player views. (It was pointed out to me that I fixed this in opengl1 years ago but I forgot to apply it to the OpenGL2 renderer.)

Game-Logic

I added an option to disable view kick when receiving damage.

I added setting the frame rate in the graphics options menu.

Maverick Model 3D

Maverick Model 3D is a 3D model editor and animator that I maintain that supports some game specific formats. It’s based on Misfit Model 3D that ceased development. (Maverick source code)

I developed/released Maverick Model 3D 1.3.14 in April.

In addition to that:

I fixed a couple issues for macOS and made GitHub Actions build and upload macOS build. If you’re logged into to GitHub, the macOS build can be downloaded from the bottom of the “Actions summary” page for each commit. I haven’t tested it as it requires newer macOS than I have access to.

I fixed a couple issues that I found through the Debian package tracker several years ago. CXXFLAGS are now respected by configure and it compiles for GNU Hurd. Though OpenGL—required for displaying the model—didn’t work in the Debian 2023 GNU Hurd virtual machine image.

I added support for exporting a Quake 3 player model with three separate models (head, upper, lower) to IQE format which can be converted to IQM for use with ioquake3 and derivative projects.

IQM uses skeletal animation which allows for less memory usage and better animation quality compared to Quake 3’s MD3 format that stores lower precision vertex positions for each frame and interpolates vertexes in a straight line between frames which may cause meshes to deform. Though IQM may be slower to draw.

I got IQE Quake 3 player model export working in 2018 (as reference in this ioquake3 forum thread where I was working on improving performance and fixing issues with ioquake3’s IQM support). I wanted to move Quake 3 player model export out on the individual format exporters but that never happened. So now it’s in IQE exporter.

P.S.

Most of these changes are a result of interacting with people rather than my own ideas or done for the sake of hypothetical other people.

If you found this useful, consider donating on Ko-fi (no account required).

Wii

I ported Clover’s Toy Box to the Nintendo Wii console. It supports rendering, audio output, networking, Wii/GameCube controllers, and USB keyboard and mouse.

I used the Wii homebrew SDK (devkitPPC, libogc, and satellite libraries) which does most of the difficult work.

Wii Remote pointing on the screen (IR sensor) is supported but I haven’t integrated support for the accelerometer motion sensor or Wii Motion Pus (gyro sensor).

The Wii Remote speaker is not support by the Wii homebrew SDK. Using Wii Motion Plus with Nunchuk extension isn’t support by the homebrew SDK either. So adding support for those would be more involved.

Getting the renderer working was easier than I expected. I mainly had to implement the functions to draw triangles/lines and upload textures (not obvious how for RGBA8) to get something to display. There wasn’t any set up needed beyond the example Wii video init. The GX graphics API is very similar to fixed-function OpenGL. In some cases separate OpenGL functions are combined as a single GX function or slightly lower-level.

I added support for blend modes, depth test/write, alpha test, cull modes, multi-texture (lightmaps), texture-less (color only) rendering, vertex color, color multiply, and texture coordinates matrix. There is no support for normal maps and frame post-processing yet.

The ground work started in 2020 with getting Toy Box to compile for GameCube/Wii with stubbed out functionality and adding OpenGL 1.1 support in 2022 partially to make it easier to add fixed-function rendering on the Wii.

I need to improve performance. In some parts of the Turtle Arena Subway level it runs at 60 frames per-second. If it renders the whole map it drops to 20 frames per-second which is way too low for the level size.

I need to better handle allocating memory and out of memory errors. The Wii only has 88 MB of RAM (which is split into two separate parts) and I basically pretend Toy Box will never run out of memory.

GameCube

The GameCube and Wii consoles are similar. The graphics features are actually identical between the two consoles. The GameCube CPU/GPU are slower and it only has 27 MB of RAM. The Wii has a bunch of extra stuff added like internal flash memory, Bluetooth, etc.

I keep Toy Box able to compile for GameCube as well. However I don’t know an easy way to add data files for the GameCube build like placing them on the Wii’s SD card. So it just draws flat colored triangles and lines in the menu. I don’t have a way to run homebrew on a physical GameCube but it runs on the Dolphin emulator. (Technically the GameCube build should run on the Wii but I haven’t tested it.)

GX RGBA8 format

GameCube/Wii 8-bit RGBA format uses 4×4 pixel block tiles (similar to S3TC) with alpha and red stored in 32 bytes and then green and blue in next 32 bytes. I think I found this referenced somewhere in YAGCD (Yet Another GameCube Documentation) and had to dig into some asset converter to figure it out.

4×4 tile linear sequential memory (spaces / line break for readability):

ARARARAR ARARARAR ARARARAR ARARARAR

GBGBGBGB GBGBGBGB GBGBGBGB GBGBGBGB

It’s the pixel colors left-to-right top-to-bottom for the 4×4 block with alpha and red and then separately for green and blue. Additional blocks continue after this to make a 8×4 image or whatever size.

Other formats like RGB565 seem to work as commonly expected (not using tile blocks).

Doom

I have a private fork of the original Doom source code for Linux that uses Clover’s Toy Box. I updated it to run on the Wii.

My Doom fork is mainly for messing with “porting an old game” to use code from Clover’s Toy Box but Doom is already really portable to many platforms that there wasn’t much to do. There is already multiple versions of Doom for the Wii available, this isn’t anything new.

I fixed compiling the Doom big endian byte-swap functions, disabled Doom networking and audio which don’t compile for Wii, added uploading the Doom software rendered game frame as a GX texture, and added remapping controller buttons to Doom key values. And it basically worked.

Pressing the key to move backward, moved forward and it was faster than the forward key. The forward_move and side_move input command was a “char” value. It turns out on the Wii, char defaults to unsigned (range 0 to 255, instead of -128 to 127) so forward_move = -25 became 231 (256 – 25).

To avoid potential issues with using the homebrew SDK libraries, I opted to change Doom input to use “signed char” instead of overriding the default for “char” with a compiler option.

It supports Wii and GameCube controller joysticks/buttons and USB mouse and keyboard but pointing the Wii Remote for aiming isn’t supported.

For Doom on all platforms, I would need to fix audio and improve input integration.

Interlude

Following a Twitter trend, I made a top 25 games list.

It was hard to think of 25. There are other games but it’s been a long time since I played them or thought about them. I mostly replaced playing games with software dev and watching TV shows a decade ago.

(For whatever reason I included with the demo of Metroid Prime Hunters that was bundled with the Nintento DS. It’s kind of a weird game selection.)

I first played seven of the games on the GameCube or Wii.

• PC: 9 games
• GameCube: 4 games
• Wii: 3 games
• PlayStation 3: 3 games
• PS Vita: 2 games
• Nintendo DS: 1 game
• Nintendo 3DS: 1 game
• SEGA Genesis: 1 game
• SEGA Dreamcast: 1 game

What this doesn’t represent well is that I may have played 75 games across the GameCube and Wii consoles compared to 5 or less per-consoles since then (PS3, PS4, PS Vita, 3DS) and I haven’t played as many new games on PC (Steam) since then either.

Why?

The simple answer is I ported Toy Box to the Wii because I wanted to even though there is no real use case anymore.

[This post was original titled Clover’s Toy Box 23.04 but I put off rewriting the Why? section for months to not be very long and off-topic.]

Summer of 2006

I started programming in the summer of 2006 by modifying Sonic Robo Blast 2 based on the Doom engine. I followed the online hype of the GameCube successor—the Revolution—that became the Wii released in November 2006. I was also looking forward to the Ninja Turtles 2003 TV series resuming in the fall, continuations of Sonic and TMNT 2003 game series on the Wii, and some other things eventually happening.

On the one hand, I was bored in the summer of 2006 so I started trying to make a game. On the other hand, I was excited looking forward to several things.

However nothing I was looking forward to met my expectations and some didn’t happen at all.

I wanted to make my own successor to the Ninja Turtle GameCube games that would run on the Wii. This a subject that I continue to work on over the years; whether it be Turtle Arena, controller support on desktop computers, or trying to run software on the Wii.

The Eternal Summer of 2006

I sarcastically refer to Clover’s Toy Box as taking place in “the eternal summer of 2006”. It was a point of excitement and looking forward to the possibility of the future.

And so, I ported Clover’s Toy Box (and Doom) to the Wii because I can.

Though I’m kind of bored.

Wayland is a newer application display/input protocol on Linux and some other operating systems. I recently got an AMD graphics card that works with Wayland and switched from using GNOME with X11 to Wayland.

Clover’s Toy Box and “Toy Box renderer for Spearmint” work on GNOME with Wayland using SDL 2 library.

However my Qt app Clover Resource Utility’s 3D model background was transparent and failed to clear the model between drawn frames which created a hall of mirrors effect.

It took 8 hours to understand and fix the Resource Utility issue yesterday. Though I did take a break at one point and was distracted by watching YouTube the entire time.

Fix it

What do I know:

• I had previously read that OpenGL on Wayland (and also unrelated, Vulkan in general without enabling an extension) display alpha instead of displaying the window as opaque.
• The Clover Resource Utility background has been broken for awhile—displaying as black—unrelated to Wayland.
• The 3D scene is rendered to a framebuffer object and then drawn to the widget framebuffer (either blended or copied directly).

So combining these things, it seemed I broke the background to be transparent black. I got the background to work fairly easily by changing the 3D scene to be “transparent” which caused it to blend with background instead of overwriting the background alpha with 0 alpha of the 3D scene. This fixed the background color as well. This was the older behavior of the application.

Fix it again

The 3D scene was specifically not transparent/blended because of issues with Quake 3 map materials writing 0 alpha and expecting it to be opaque.

I previously added a hack to make non-blended materials write 1.0 alpha (only necessary when a transparent/blended framebuffer object was used). However this detection failed in simple cases like blending a texture over a lightmap. It basically only fixed textures without a Quake 3 shader.

Testing this older behavior however had very strange effect that I couldn’t guess causation and behaved differently on X11 and Wayland. On Wayland the transparent materials were accumulatively added to the window each frame with hall of mirrors effect until the surface was white. However I clear the 3D scene and widget framebuffers each frame. It kind of seemed like premultiplied alpha without clearing the widget framebuffer but that doesn’t make any sense either. Where is it coming from?

I tried to fix it by forcing writing 1.0 alpha when drawing the 3D scene framebuffer to the widget framebuffer. However I changed the color multiplier in the vertex shader, not the actual alpha of the 3D scene framebuffer texture in the fragment shader, and I didn’t realize it for hours.

Fix it again, correctly

This left me with an issue I could not explain the cause of and searching online lead to nothing (despite finding and reading about other issues related to Qt Wayland not displaying OpenGL content, general OpenGL transparency, etc).

Qt problem? GNOME with Wayland problem?

I was comparing using both of the Qt X11 and Wayland backends and testing changing various things, semi-random because it made no sense. I occasionally thought I fixed it but then realizing I need to test maps besides q3dm1 because my “non-blended shader hack” fixes it but I didn’t manged to fix the alpha. There was a lot of truly strange results.

Eventually I tried enabling alpha for the Qt OpenGL context. (I had code for it commented out and I was just semi-randomly testing things.) This fixed the weird “accumulatively added to the window each frame” and made X11 and Wayland behave the same.

After that I was able to understand, I did not correctly fix it to write 1.0 alpha when drawing the 3D scene framebuffer texture to the widget framebuffer. Correcting that to actually work, fixed the issue.

Conclusion

Make sure to write 1.0 alpha to the OpenGL framebuffer on Wayland if OpenGL alpha buffer isn’t enabled.

That’s how I spent 8 hours to fix the transparent HOM background while still correctly ignoring Quake 3 map materials writing weird alpha.

I thought about trying to use RenderDoc to inspect it at one point and I don’t really have a reason for why I didn’t.

Maybe if I took a break for the day, I would of realized that in order for non-1.0 alpha to misbehave when I’m clearing the 3D scene and widget framebuffers each frame, non-1.0 alpha must be making to the final framebuffer.

If having OpenGL alpha issues on Wayland (or maybe in general), enabling alpha in the OpenGL context so it’s displayed (correctly) may be helpful.

I tried to capture the Quake 3 map bug here by recording a video: surface is opaque, screenshot: surface is transparent. I love undefined behavior.

Future

The titlebar and resizing on GNOME with Wayland depends on the application to implement it. SDL supports it using the libdecor library. (libdecor will be added in Flatpak freedesktop 23.08 SDK releasing around the end of the month.) I would want to ship libdecor along with SDL in my application for full self-contain Wayland support.

(Outside of Flatpak) I would prefer not to deal with shipping libdecor and it’s plugins that assume that cairo or GTK 3 are installed. It seems like it would be simpler to add support for handling the titlebar and resizing in my application.

However SDL seems to need improvement for “hit testing” for determining if the cursor is over the titlebar/edge to work correctly. (In my test, resize doesn’t work on Windows, resize cursors don’t appear when hovering over edge on Linux (X11/Wayland), and I haven’t tested macOS.)

SDL 3 has gained support for popup menu and tooltip windows and displaying the default titlebar context menu across Windows, Linux, and macOS. I would like to build on it to make a simple GUI framework to replace using Qt for Clover Resource Utility.

Toon Cel Outline

It’s now possible to create a toon cel outline effect (expanded mesh, drawing triangle backside). It works with skeletal animated models but it’s not supported with frame animations. Skeletal animations can be converted to frame animations afterward though.

1. Disable drawing back-facing triangles using Render -> Render Options -> Hide back-facing triangles
2. Select all meshes to have toon cel outline and then use the Geometry -> Duplicate command (copy and paste doesn’t keep the joint influences)
3. Add a material for the outline color in Materials -> Edit Materials
4. Create a new group in Materials – > Edit Groups and add the selected duplicate meshes (Assign As Group) and set materials to your outline color material
5. Use Geometry -> Vertices -> Weld Vertices so that vertices in the same place don’t move apart
6. Use Geometry -> Normals -> Invert Normals so that triangle faces inward
7. Use Geometry -> Vertices -> Offset by Normal to give the cel outline the desired size

Changes

General

• Enable Qt HiDPI support by default and fix a few layout issues
• Add Offset by Normal command to move vertex along average normal
• Make Duplicate command keep all joint influences
• Make Edge Divide command set group and texture coords
• Make Select Vertex tool automatically select and unselect triangles
• Automatically select new face made by Make Face From Vertices
• Make texture coord window reset map scheme on selection change
• Fix texture coord window displaying last texture if no triangle has a texture

Merge Model / Import Animations

• Add support for frame animations to Import Animations (must have the same number of vertexes and order)
• Make Merge Model copy frame animation frames-per second (FPS) setting
• Make Merge Model and Import Animations copy looping setting
• Fix Merge Model and Import Animations to set skeletal animation FPS on the correct animation

Background Image

• Fix model using background image with Canvas mode Flat
• Fix model viewports not updating immediately after setting background image
• Fix drawing left/right brackground image in some cases
• Fix background image not blending in some cases

Model Formats

cal3d

• Add support for exporting cal3d .cfg files

mm3d

• Fix writing background image filename

GNU/Linux

• Update Flatpak runtime to org.kde.Platform 5.15-22.08 from 5.15 (20.08)

Download

Download the Windows x86 installer and GNU/Linux x86_64 Flatpak Bundle at the Maverick Model 3D webpage.

This release was created with a week of development. If you find it useful, consider donating at Ko-fi.

It should build and run on macOS. However like Maverick 1.3.13 there is no macOS build as I haven’t sorted it out / purchased a new mac that Qt 5.15 still supports.

Audio

I added support for playing multiple audio files at once. My initial goal is more-or-less feature parity with Spearmint’s audio mixer. I haven’t completed that yet.

Quake 3 has a pretty basic audio mixer; OpenAL 1.1 and SDL_mixer support most of the features directly. The main complex features would need to be implemented on top of the mixer are ambient sounds (volume affected by multiple locations but only one playing sound with one max volume) and game-specific logic for stopping sounds when starting a sound (like limit per-object).

In Spearmint, I added support for multiple listeners for splitscreen. OpenAL 1.1 and SDL_mixer do not directly support this. It should be possible using OpenAL with multiple AL contexts to have “multiple listeners” by duplicating all audio buffers and sources. It would be possible to manually spatialize sounds for SDL_mixer.

OpenAL?

I originally planned to use OpenAL 1.1 (specifically mojoAL and OpenAL-Soft). The main reason to use OpenAL was to not have to deal with multi-threading myself and (using OpenAL-Soft) up to 7.1 surround sound mixing. (Though I didn’t find OpenAL implementation for GameCube/Wii which I pretend to target.)

Using OpenAL doesn’t seem too difficult for Spearmint features (though I’m not sure ambient sounds can be implemented correctly) but I think additional features will become very complicate or unnecessary memory usage.

Using OpenAL wasn’t as simple as I expected. I still had to write the framework of an audio mixer but without the literal audio mix function. Having to deal with multiple OpenAL devices and contexts with duplicated audio buffers is complicated and unnecessary memory usage. I plan to add basic effects like volume fading … which OpenAL 1.1 does not support. These kind of push me to want to streaming the audio for each source to OpenAL. (I handle the actual full audio buffers and preprocessed effects for the source and OpenAL just does spaticalizing/mixing with short temporary per-source audio buffers.)

I had a basic incomplete OpenAL mixer working and then I ripped it apart to just add my own audio mix function which at it’s core it just adding audio samples scaled by volume.

Toy Box audio mixer for Spearmint

I modified Spearmint to support an external audio mixer library similar to external renderer library. I made a small library to implement Spearmint sound interface over Toy Box mixer. This is how I did most of the testing of the audio mixer.

This hasn’t been merged into the public Spearmint git repo as it’s not really useful outside this test and I didn’t convert the built in mixers to use it.

Toy Box audio mixer

I implemented audio sources with instance IDs so that individual sources can be modified or stopped. (The instance IDs prevent reallocated sources being affected by old out of date source handles.) There is also object IDs to allow updating position of all sounds by object like in Quake 3.

I made a list of about 50 features in the Spearmint audio mixer and reviewed the RTCW/ET features I haven’t implemented in Spearmint yet. I implemented most of them.

I implemented some long wished for features: pausing audio sources and (RTCW/ET) per-source volume and range. It would be possible to pause audio sources when the game is paused now. Though to be more practical I probably need to add some setting for whether it’s a game or menu sound.

Spearmint has four separate audio source systems for playing sounds (sfx, looping sfx / real looping sfx, music / streaming sounds, and raw samples from videos/VoIP). Toy Box has one audio source system. The somewhat strange “stop looping sound if not added this frame” is not directly supported; it’s the applications job, so it’s implemented in “Toy Box audio mixer for Spearmint” library.

Missing features compared to with Spearmint:

• Audio resampling (needed for playing sounds at correct speed/pitch)
• Doppler (dynamic audio resampling)
• Stop sound when starting sound (limit count per-object, free oldest when out of sources).
• Limit total volume of looping sounds by sfx
• Audio recording
• Audio recording volume meter

DDS Images

I added support for uncompressed DDS image formats to have feature parity with image support in Spearmint opengl1 renderer. I added a few others DDS features that are supported by ioquake3/Spearmint opengl2 renderer.

• Add support for uncompressed image formats, in additional to “ABGR8” format.
• Add support for uncompressed mipmaps.
• Add support for cube maps.
• Add support for red and red-green channel-only compressed images (BC4/BC5).

(ioquake3 opengl2 renderer supports some additional compressed formats—BC4S, BC5S, BC6H and BC7—that Spearmint does not and I haven’t added support to Toy Box.)

The DDS image format (DirectDraw Surface) is a container format used by Windows DirectX SDK that supports image data in many different formats. Toy Box had support for DDS images with DirectX texture compression 1-5 (DXT1-5, also known as Block Compression 1-3) and uncompressed 8-bit RGBA.

I added support to Toy Box for loading DDS images with uncompressed RGBA image data with various bit depths and color channels present such as RGB565, L8A8 (luminance-alpha), and RGB10A2. All are converted to RGBA8 internally; known as “ABGR8” in DDS terms as the names use reversed channel order. (Spearmint also converts them to RGBA8.)

Mipmaps are a series of 50% size copies of the image that are used to improve image quality further away from the camera.

I added support to Toy Box for using the uncompressed RGBA mipmaps from the DDS file instead of generating new mipmaps. This can be used for faster loading or effects purposely using different images for the mipmap levels. (Spearmint incorrectly generates new mipmaps but ioquake3 opengl2 renderer uses the mipmaps from uncompressed RGBA DDS images.)

Cube maps are 6 images for the sides of a cube that can be used for reflections and the sky. I added support for loading DDS cube maps and added basic cube map reflection rendering.

There are variants of Block Compression for red and red-green channel-only compression (known as Block Compression 4/5, OpenGL RGTC1/2, DDS ATI1/2, and 3Dc+/3Dc). It uses the same compression format as DXT5/BC3 alpha block but for red and green blocks. Red-green compression is useful for normal-map compression. Red-green compression offers better precision for gradients than RGB compression (DXT1/BC1) and the normal-map light direction is normalized so the third missing value can be calculated in a shader. Red and red-green compression are supported by ioquake3’s opengl2 renderer but not Spearmint (different DDS loaders).

I added support for red and red-green compression formats to Toy Box. I support uploading R/RG compressed images to OpenGL directly or first manually decompress the image if it’s not supported by the OpenGL driver. My image writing/converting code will also decompressed them if writing a image format besides DDS (such as PNG or JPG).

ioquake3’s opengl2 renderer’s DDS loader also supports BC6H and BC7 compression. I’m not planning to add them to Toy Box at this point. Just passing the data to OpenGL isn’t complicated but decompression is complicated. I want to be able to decompress all images to load them on OpenGL 1.1 or write them to a PNG image.

I’m only targeting feature parity with Spearmint opengl1 renderer but BC4/5 red/red-green compression wasn’t very difficult to add to Toy Box. I will be ignoring the other parts for now.

Year’s End

There was 406 code revisions adding 27,000 new lines of code (including comment and blank lines). It’s about 25% of Clover’s Toy Box ~106,000 lines of code (not including third party code).

Changes of the year:

• Improved rendering performance and fix various issues.
• Added support for seven more skeletal model formats; Quake 3 MD4, Doom 3 MD5, and some derivatives of them.
• Added support for OpenGL 1.1 and OpenGL ES 1.1 and to prefer OpenGL ES 2 via ANGLE on Windows over OpenGL 1.1.
• Added an audio mixer and loading six audio file formats.
• Improved game text input and added support for buttons on newer controllers.
• Added browsing archive files and viewing text files to Cover Resource Utility.
• Added basic support for running libretro cores.
• Added support for controlling PC “RGB lighting” using OpenRGB network protocol.
• Initial Android port.
• Fixed running on Windows XP and ReactOS.

Year six of development ends. Year seven begins.

After posts covering 4 months (22.04), 3 months (22.07), and 2 months (22.09) of development it seems only natural for this post to cover 1 month (22.10).

Compared to Spearmint, Toy Box now supports all audio codecs (no audio mixer yet), all image formats (except uncompressed DDS), and has partial support for all model formats. There is still a few difficult issues for models but it seem like these areas (audio, images, models) could have feature parity with Spearmint in the near future. There will be some unsupported model behavior that is unlikely to be used.

The other main areas that need work are Quake 3 .BSP (levels), collision, and networking. Toy Box is also missing many rendering features and the actual game-logic. So yeah, there is still a long way to go but it’s nice to feel like there is progress.

Models

• Added support for Return to Castle Wolfenstein .MDS skeletal models.
• Added support for Wolfenstein: Enemy Territory .MDM skeletal mesh and .MDX skeletal animation.
• Added support for Wolfenstein: Enemy Territory .TAG models.
• Added support for Kingpin: Life of Crime .MDX models (unrelated to Wolf:ET).
• Added more support for vertex tangents.
• For Enemy Territory: Quake Wars mod kit (SDK); Support vertex colors and missing bind pose in .MD5MESH.
• Added support for Misfit Model 3D .MM3D skeletal “points” (tags in Quake terms).
• Added support for vertex tangents and colors in Inter-Quake Export .IQE models.

RTCW / Wolf:ET

I added support for Return to Castle Wolfenstein .MDS. It’s another version of MD4 with unique method of bone joint storage and uses collapse map for level of detail support. I don’t support the dynamic torso rotation, torso animation, or collapse map yet.

I added support for Wolfenstein: Enemy Territory .MDM and .MDX. They are RTCW .MDS split into mesh and animation files; similar to FAKK/Alice .SKB and .SKA. MDM adds tags that are relative to a bone and bones are only in MDX.

For Wolf:ET model formats, I decided to try to properly support separate mesh (without bind pose joints) and skeletal animation models. (For FAKK/Alice .SKB/.SKA I cheat and load the other from the same directory, though this may not be correct.) There were some pretty intrusive changes to support skeletal mesh model without bind pose joints. I have to support models with no joints rendering using the joints in the (animation) frameModel and support MDM adding additional tag joints on top. The number of joints/tags is based on the combination of the model and (animation) frameModel, not either model on it’s own.

The Wolf:ET models work surprisingly well but there is some visible issues. I don’t correctly handle MDM vertexes with multiple bone joint influences yet due to missing bind pose joints and not improperly getting them from an animation.

I want to figure out how to calculate usable bind pose matrices using the vertex bone-space positions as it would be less intrusive model support through the engine and allow for GPU bone skinning. Though I currently only support 4 joint influences per-vertexes for GPU bone skinning and the Wolf:ET MDM models use up to 5. I may change to 8 in the future (it requires additional GLSL vertex attributes and shader variants). Worst case, I have to deal with storing and using the bone-space positions when rendering using CPU model animation.

I also added Wolf:ET .TAG “models” which are Quake 3 .MD3 models reduced to just one frame of MD3 tags. It’s used by the Wolf:ET server game-logic for attachment points on vehicles. The Wolf:ET client renderer doesn’t support them. In Spearmint I specifically decided not to add it to the engine because the game-logic can load it itself (even though Spearmint allows the server game-logic to load models through the engine and get the tags). I changed my mind for Toy Box, it’s easy to add it in the model code and I can also render debug axis for the tags and view the .TAG models in Clover Resource Utility.

Kingpin

A year ago I was looking into formats used by Quake 2-based games and started adding Kingpin Life of Crime .MDX based on Quake 2 .MD2. MDX adds a couple sections I don’t support; effects connected to a vertex and per-mesh bounding boxes. Quake 2 MD2 has separate vertex texture coords for software and OpenGL. Kingpin MDX however removed the software texture coords which are what I was using in Toy Box. So I had to parse the MD2 OpenGL commands list of triangle fans and strips and convert it indexed triangles using the triangle’s per-vertex texture coords.

Vertex Tangents

Normal maps allow per-texel lighting direction to give an appearance of more detail on a model. It’s one of the latest and greatest rendering feature of ~2003. Vertex tangent vectors are needed for normal map support to know the up and side orientation of the texture in 3D space. (The vertex normal is the forward orientation.)

IQM and IQE may include vertex tangents. For IQE and other formats they must be generated. Vertex tangents are generated by Doom 3 for .MD5MESH and by ioquake3/Spearmint opengl2 renderers for model formats they support (excluding IQM, which should have them).

I already supported vertex tangents for IQM models (and applying one normal map to all models as a test). I added vertex tangents to the generic vertex so that it’s possible for dynamic geometry, sprites, and CPU animated models to have vertex tangents. I added debug lines for vertex tangents which is very useful when generating them.

I added support to my general model handling (MeshBuilder API) for explicit and generated vertex tangents and vertex colors. This generate vertex tangents for all skeletal formats except IQM (MD4, MDR, MDS, MDM, SKB, MD5MESH, MMD3). I still need to add vertex tangent generation for formats that don’t use MeshBuilder; IQM and formats based on MD2 and MD3.

Conveniently Wolf:ET .MDM contains vertex texture coords and normals so it’s possible to generate tangents even though I don’t know the vertex model-space positions.

ET:QW

I previously added support for loading the Enemy Territory: Quake Wars variant of .MD5MESH (version 11). (The ET:QW game doesn’t use them but the ET:QW SDK contains them, albeit without any .MD5ANIM animations.)

I added support for vertex colors in ET:QW .MD5MESH. All models in the SDK have vertex colors but all the colors are white (“1 1 1 1”). It’s not very exciting but it reused the work I did for .IQE vertex color support.

I added support for loading “*_lod3.md5mesh” which are lower level of detail (LOD) models that do not contain the bind pose joints. This reused the work I did for Wolf:ET .MDM without joints. The LOD models can be drawn using the joints from the separate “*.md5mesh” file with the same base name. I handle this in Clover Resource Utility by treating the base model as the animation frameModel. This does not support animations yet (the ET:QW SDK doesn’t have any though) as it needs to connect the LOD model to the base model to get the joints instead of treating base model as an animation.

Misfit

Misfit Model 3D .MM3D models contain skeletal points (tags) connected to joints similar to Wolf:ET .MDM but with multiple joint influences like vertexes. I added support for this along with Wolf:ET .MDM tags which made it more complicated.

Inter-Quake Export

Inter-Quake Export .IQE, the text version of Inter-Quake Model .IQM, supports vertex colors and vertex tangents. I added support for them using my new common code in my MeshBuilder API.

Audio

• Added .MP3 playback using public domain dr_mp3 library.
• Added support for ID3v2 metadata in .MP3 and .WAV.
• Cleaned up my .WAV loading.
• Added support for LIST INFO metadata in .WAV.
• Fixed a large .Opus file crashing the game.

MP3

I got side tracked processing the ID3 metadata that is typically in MP3 files before I added MP3 audio support. I thought it would be fairly simple key-value pairs but it turned out to be pretty complex.

ID3v1 (“TAG” data identifier) is 128 bytes at the end of the file. There is fixed length fields for title, artist, etc. (Values for specific keys, if you will.) Most (all my?) MP3 files have this and it’s very simple to load but it’s limited.

I detect audio files by the content. MP3 files usually start with ID3v2 metadata (“ID3” data identifier). The ID3v2 spec boasts it can be used with other audio codecs. I thought I could parse ID3v2 metadata to skip it and then check the audio data. It turned out MP3 files are frame based and scans for the next frame so arbitrary data can be mixed in (possible in error). I can’t check the data immediately after the ID3v2 metadata to detect MP3. ID3v2 in WAV files is stored as a RIFF chunk; not simply at the start of the file like MP3 files. So my idea of general ID3v2 handling at the start of the file turned out to not be useful and I moved it to MP3 and WAV specific code.

I had some difficulty understanding the ID3v2 specifications. As I understood more it reveled that ID3v2 is kind of full blown archive format with entries. It has separate text encoding per-entry and even offers deflate compression like .zip files. It also has a “unsynchronization” option for when the entry data is altered to avoid looking like a MP3 frame header. (I haven’t added support for deflate compression or “unsynchronization” flag.) There is three incompatible versions of ID3v2—I have all three in my meager library of MP3s.

I spent a bunch of time with trying to understand it and also converting text encodings (UTF-16 big and little endian to UTF-8). After a few hours of working on ID3 support, I realized I probably should of looked for a library for ID3 support but I was already too deep to want to stop.

I found of several weird cases when testing my library of MP3s. Such as placing the string terminator at the beginning of the entry… possibly a side affect of ID3 using a string terminator as a separator between values in the entry but these only had one value. Some files including a second ID3 metadata block (with the same content?). Some files there is garbage between the ID3 metadata and the MP3 audio data (in one file there is ID3v1 metadata, which is suppose to be at the end of the file).

So yeah, ID3v2 is complicated, weird edge cases, and not really a joy to implement. I worked on it three days and didn’t implement the full specifications.

Adding MP3 audio support using dr_mp3 was really easy though. It took like 2 hours to add using the dr_mp3 “pull API” to request samples with read and seek callbacks. I may want to change later to work with MP3 frames so I can handle ID3v2 tags in the middle of the stream. Though this is probably only likely to happen in Internet radio streams.

WAV

My .WAV loading somewhat bizarrely read 20kib of the file and then processed the byte array to find the audio format and start of the audio sample data. If the sample data started more than 20kib in (very unlikely), it would fail. Now it streams the file and processes it byte by byte with the sizes defined in the .WAV file.

I added support for the WAV metadata in Microsoft’s WAV specification (a “LIST” chunk with “INFO” identifier) and then proceeded to add support for “id3 ” chunk that contains ID3v2 metadata. Audacity exports both which gives a way to test it. Audacity also includes the ID3v2 “extended header” to show me yet another way I was incorrectly parsing ID3v2. The ID3v2 header size includes extended header size. I glossed over the wording in the specification; it’s there.

Opus

I tried loading a 8-hour 488 MB Opus file but it crashed. The 64-bit sample data size was 6.0 GB which is over max signed 32-bit integer value (2.4 GB). The size was passed to Opus library op_read_stero() as 32-bit and overflowed to negative and crashed in the opusfile library. After I fixed my code to limit the read size, it took 2 minutes to load the 8 hour Opus file, then reported an error because my sound effect loader only supports 32-bit size. So I moved the error up to before allocating and loading 6.0 GB of audio sample data. Large files should be streamed—and I support streamed reading for all audio codecs—but I don’t support playing streamed audio yet.