User:MascaraSnake/UDMF

From SRB2 Wiki
Jump to: navigation, search
NoteIcon.png Note
I am no longer working on UDMF. All the information presented below is purely archival.

As you might have heard already, Nev3r and I are working on adding UDMF support to SRB2. This page gives an overview on what we're doing, why it's useful, and what you can do to participate.

What is UDMF and why is it good?

UDMF stands for Universal Doom Mapping Format. It's a text-based map format developed by the Doom community, the idea being that it supports all the different map features from all the various Doom source ports, and that it can be easily extended to support new features in the future. Before I go into detail about how it works and why we want it, let me motivate why it was created in the first place.

What is the problem with Doom's original map format?

Doom's original map format (which SRB2 uses to this day) is what you call a binary format – it pretty much takes the map data as it is stored in the game's memory during runtime and dumps it on the hard drive as raw, non-human-readable binary data. The advantage is that is that it's really fast and simple for the game to read and write because you don't need to do any conversion. But the big downside is that it's not extensible. Let's look at the way the Doom map format handles Thing as an example. Each Thing is stored in the map data as a sequence of five 16-bit integers:

  • X position
  • Y position
  • Angle
  • Thing type
  • Flags

The THINGS lump of a map is nothing more than a bunch of these five-integer sequences, one for each Thing. It works the same way for sectors, linedefs, and so forth. Each map element has a set of properties – which I will call fields from now on – and the map format just dumps these fields into a file as raw data, one after the other.

You will notice that Things have a field for X and Y position, but not for Z position (i.e., height). This is because the original Doom didn't have Thing heights! So what does SRB2 do to support Thing heights? It steals 12 bits from the "Flags" field and shoves the Z position information in there. This is why SRB2 only has four Thing flags – because there are only 4 bits left over for the actual flags. It's also why you can't have Thing heights beyond 4096, because that's the most you can fit in 12 bits. These kinds of hacks are why we need specialized map editors like Zone Builder instead of just using Doom Builder or GZDoom Builder straight out of the box.

You might be asking yourself why we don't just add a sixth "Z position" field. We can't do this because it would change the map format and therefore break map editor support in a much more drastic way. Telling Zone Builder to interpret the last 12 bits of the "Flags" field as the Thing's height is pretty easy to do. If we load an original Doom map that doesn't use that hack in Zone Builder, or if we load an SRB2 map in Doom Builder, it will show some of the Thing information incorrectly, but it will still basically work. However, if we added a sixth field to the Thing definition, cross-editor support would break completely. Zone Builder expects the THINGS lump to have exactly five 16-bit fields for each Thing. If we add a sixth field and don't tell Zone Builder about it, it will garble the Thing data completely and fail to load the map properly. We could change Zone Builder to recognize that sixth field, but then it wouldn't be able to read all those old maps with the original five-field format, so you'd need to keep an old version of Zone Builder around to load the old maps.

So this is the core of the problem: We could extend the map format to add new fields if we wanted to do, but every time we do so, we completely break support for older maps and lose any semblance of compatibility with the existing map editors for Doom. Some years ago, the creators for all the different Doom source ports were facing a similar problem: Sure they could add new map functionality by adding new fields and changing the map format, but then you'd need a specialized map editor just for that one source port, which wouldn't be compatible with all the other source ports. And so they decided to come up with a new map format that they could all use and that didn't have that problem.

How does UDMF work?

UDMF is entirely text-based. All the map data (except for the stuff created by the nodebuilder) is stored in a single text lump called TEXTMAP. Here's what the entry for a Thing looks like:

thing // 0
{
x = 0.000;
y = -768.000;
angle = 90;
type = 1;
flags = 3;
}

As you can see, each entry is a simple list of the fields and their values. If a field is unused, you can simply leave it out and it will fall back on a default value. Take this for example:

thing // 0
{
x = 0.000;
y = -768.000;
angle = 90;
type = 1;
}

Here the flags value is missing, so the "Flags" field will default to 0.

Sectors, vertices, linedefs and sidedefs are also defined in text blocks just like this one. The TEXTMAP lump, then, is nothing more than a list of these blocks, one for each map element.

Why is this better?

The key difference to the binary format is that each field is labeled with its name. This allows us to leave out fields and have them fall back on a default value. This makes it possible to add new fields and still have backwards compatibility. Let's say we want to add a Z position field for Things. The Thing entry would then look something like this:

thing // 0
{
x = 0.000;
y = -768.000;
z = 32.000;
angle = 90;
type = 1;
flags = 3;
}

If the z field is missing, it simply defaults to 0. This future-proofs the map format: If we decide to add a new field in a future version, maps from earlier versions will still load fine both in the game and the map editor – the missing field will simply be set to its default value. What makes this even better is that GZDoom Builder (which Zone Builder is based on) has a "Custom" tab in its Thing/linedef/etc window that allows you to add and edit any field you want. This means that if we ever add a new field, ZB won't need to be updated to allow you to use it (although we will probably update it eventually to integrate it more nicely into the interface).

In the context of the Doom community, UDMF allows source ports to share a common map format even though they don't all support the same fields. The map loading code simply ignores any field that it doesn't recognize, so you can add a new field that other source ports don't support without breaking anything. In the SRB2 context, this means that source code mods and spinoffs like SRB2Kart will be able to extend SRB2's UDMF implementation as they see fit, without having to worry about downwards compatibility to SRB2 and map editor support. It may even be possible to allow Lua to define new map fields.

Why should I care?

If for some reason you're not yet convinced that UDMF is the best thing since sliced bread, here's a few examples of the things UDMF will allow us to do. These come in two varieties: getting rid of old hacks that made life unnecessarily hard for mappers, and adding entirely new features that weren't possible before.

Linedef arguments

Because the binary map format has a limited amount of fields, we sometimes have to get creative in order to stuff extra information into the map. This is especially common with linedef specials. A lot of our linedef types are parametrized – their behavior depends on various settings that the mapper can control. Because we don't have any dedicated fields where we can store these parameters, we typically shove them into unrelated fields, like the texture offsets of the control linedef, or the floor height of the control sector. Sometimes even the length of the control linedef is used as a parameter, which leads to control sectors with bizarre contorted shapes. All of this can make it pretty tedious to set up special effects, because you have to look up which field controls which setting on the Wiki, and if it's something like linedef length, changing the setting requires more work than just swapping out one number for another.

In UDMF, none of this will be necessary because linedefs have a number of dedicated "argument" fields, arg0 through arg4. These fields exist solely for the purpose of supplying arguments to linedef specials so you can customize their behavior. As an example, let's look at linedef type 60, Activate Moving Platform (Adjustable Speed). Right now, you need the following to set it up:

  • Linedef length: Speed
  • Front X texture offset: Time delay when the movement changes direction
  • Front Y texture offset: Time delay before the platform starts moving
  • Not Climbable flag: Platform begins by moving upward

In UDMF, it could be something like this:

  • arg0: Speed
  • arg1: Time delay when the movement changes direction
  • arg2: Time delay before the platform starts moving
  • arg3: 1 if platform begins by moving upward, 0 if it begins by moving downward

In Zone Builder, the labels of the arg fields can be customized for each effect, so the arg0 field would be labeled as "Speed:", for example. Fields where you need to choose from different options, like arg3 in this example, can be displayed as drop-down lists. Fields that are a collection of flags that are either toggled on or off can be displayed as checkboxes.

The result of this will be that you can set up most linedef effects by simply typing the parameters you want into the arg fields. There will be no more need for shoving parameters into unrelated fields or encoding them in the map geometry. And you won't have to look up which setting goes where because the map editor will tell you.

Native flat alignment

Another consequence of not being able to add new fields is that some things that really should be sector properties are instead hacked in via linedef specials. The most obvious example is flat alignment. In Doom's map format, sidedefs have fields for texture offsets, so you can align wall textures just by tweaking those numbers. But sectors don't have anything like that, so we hacked it in via linedef type 7, Sector Flat Alignment. This is pretty tedious to set up because you have to tweak settings on the control sector until it looks right in the target sector. In UDMF, there won't be any need for this mess: Sectors will simply have fields that allow you to set the floor and ceiling offsets directly, and you will be able to adjust them in ZB's Visual Mode just like wall textures.

Native vertex slopes

Vertex-based slopes are somewhat tedious to set up right now because you have to place three slope vertex Things and then link them together via a linedef special. In UDMF, vertices can have a Z position, just like Things. This makes it trivial to make sloped sector – you just make a triangular sector and set the vertex heights to what you want them to be, and the surface will interpolate between them automatically.

Multitagging

ZDoom's UDMF implementation has support for applying multiple tags to one sector/linedef/Thing. This is extremely useful when you have two special effects (such as FOFs) whose tagged areas overlap partially but not completely. In the current map format, you have to give the overlapping area a new tag and then make duplicates of the two special effects that use that tag instead. With multitagging, you'd just give the overlapping area two tags – one for each effect. Multitagging support requires a bit more work from the code side than some of the other features, but we're already working on it and it's looking pretty doable so far.

What does this have to do with me?

We need you as guinea pigs! UDMF gives us an opportunity to basically redesign our entire mapping setup. All the existing linedef, sector and Things types need to be given a long, hard look to figure out the best way to set them up under the new system. Some may become obsolete (see the flat alignment example above). Many will become more comfortable to set up. A lot of similar linedef types may be merged now that we have more space for different options in the linedef arguments (this is the case for a lot of FOF types especially). The goal is to make things as convenient as possible for mappers. This requires feedback from as many mappers as possible, so we invite you to participate.

The development of UDMF will be happening entirely on the public GitLab repository, so you can follow our progress. Once it's in a usable state, we will start distributing experimental builds so you can mess around with UDMF and start making test maps before it's added to an official release. Your feedback on these builds will be crucial for shaping the final implementation of UDMF.

Discussion of UDMF will be happening in the #srb2-udmf channel on the Discord server. The channel is visible for everybody, but talking rights are restricted to keep the conversation focused. If you want participate, all you have to do is notify an admin or mod so they can give you talk permissions. Everyone is welcome to participate!

Links

  • Original UDMF specification: [1]
  • ZDoom's extensions: [2]
  • Current state of SRB2's UDMF specification: /Current
  • Outdated notes on SRB2's UDMF specification: /Specification
  • Nev3r's UDMF page: User:Nev3r/UDMF
  • udmf-next branch on the GitLab repository: [3]