Aug 2, 2013

The Framework of Riggs

The following blog post is an explanation of how I structured the automated rigging system I wrote in python and PyMEL for Double Fine Productions called Riggs (after the main character Eddie Riggs from Brutal Legend), intended for individuals interested in rigging or character pipelines. Here's a demo of it:





INTRO

At a company like Double Fine the characters are generally very stylized, which usually means a lot of trial and error with rigging. There are many revisions to a character by the time the games ship. As is usually the case, over the course of my time at Double Fine the demands of the characters governed the tools that I wrote. In this post I'll go over what I learned from the first system I wrote in MEL and the issues with it that led me to design a system like Riggs, as well as a general explanation of the nuts and bolts of Riggs.

The standard artist disclaimer:  This might go without saying but there are many other ways to accomplish what I've done, this is merely an account of what I did and learned.  I am an animator and rigger first, and a programmer second.  I was taking my first steps with scripting while I wrote the first system as well as my first steps with python while I wrote Riggs.



THE FIRST SYSTEM

During some down time after we finished making Psychonauts one of our programmers named Nathan Martz held a class at work teaching the basics of MEL scripting for artists. After that(and with the guidance and patience of a super helpful Nathan and other programmers) I started writing what I thought would be a cool way to speed up rigging. These basic scripts only intended for myself eventually turned into a system the entire animation team used during the production of Brutal Legend.

In the first very basic iteration the user would start with a preset biped layout loaded from a file, add the joints they wanted, and then with one button create all of the animation controls on the rig based on joint names. This rig had 2 skeletons, a game skeleton that the geometry was bound to and was exported to the game engine, and a control skeleton that housed all the animation controls that drove the game skeleton.  There were two main reasons for having separate skeletons.  One was if you needed make changes to the skeleton for game optimizations later on you could do so without affecting the animation in Maya.  The other was to keep the elements of the rig that got exported to game data separate and easy to access with a collada export script to get it to the game.

Eventually people wanted to make changes to fully built rigs (at Double Fine, the stylized characters tend to go through revisions at any stage in the process) so rather than starting over each time I added a way to recreate the layout, modify joint hierarchy and names, and then build a new rig with one button. One issue with this was that people had to redo everything they had done after the rig was built, such as driven keys, the shape of the nurbs curves, orient switches they added, etc. Not only was this a lot of lost work, if someone didn't recreate everything correctly it could break that character's animations. To solve this I added some more functionality that would save a few of these modifications the user made during rig regeneration, but it was very limited and problematic.

This first system definitely served its purpose and got us through the 150+ rigs of Brutal Legend, but because I was learning how to script while I wrote it, by the time we were halfway through production I was already eager to completely start over to make it faster, cleaner, etc. Of course we had to finish the game, and everything was already in place, so I kept adding to the existing scripts.



A NEW DESIGN

After stretching the first system to its limit, I had a pretty long list of improvements when designing a new rigging system while learning python.  These are some of the main items:

  • Names should be irrelevant: I learned not to rely on users to follow your naming conventions to identify anything that scripts function on. Obviously they don't know, and shouldn't be responsible for, what's going on behind the scenes. I did a lot of wrestling with people to get them to follow naming conventions and rules that the scripts required, but it was an uphill battle, things would break often, and I always felt like the bad guy for scolding them. For Riggs I wanted users to be able to name joints whatever they wanted without it affecting anything in script. Naming conventions are important, but not important enough to break things.
  • More user control: Joint colors, nurbs curve shapes, and anything else that a user could control should be saved as data instead of automatically assigned in script. This relates to the last improvement in that I wanted to automate less and make things more flexible. Set an initial state for things in script, and then recreate the changes that the user had made.
  • Seamless editing: Switching between layout and rig should be as fast and seamless as possible. I wanted people to be less afraid of making rig changes so they could make more, learn more, and the quality would end up better. Joint positions are extremely valuable for proper deformation and it's a great learning experience to be able quickly change the position of joints in a rig while you're painting weights.
  • Reliability: Any supported changes a user makes to a full rig must be stored and recreated exactly the same when going to and from the layout. In the old system if one thing they modified didn't save during rig generation, users stopped trusting that anything they modified would be saved, so they were afraid to make changes to the rig. Even when you fix the bug, even when you spell out the fix in an email, unless you tell them in person, they assume it's still broken. Obviously everyone wants cleaner code with fewer bugs, and this was definitely a priority.
  • Only one skeleton: Users never knew which skeleton was which, and that caused problems.  It was especially problematic for programmers or audio folks that didn't have much maya experience and just wanted to place events on the skeleton for scripts or audio cues.  Also because the skeletons were right on top of each other, people had a lot of trouble selecting either one.
  • Modular: All control types (Leg IK, Arm IK, Spline IK, etc) must be usable anywhere on any rig. This was a problem because for anything special, or basically anything other than a super basic biped rig, a custom MEL script had to be written for it, which made me a bottle neck in the pipeline.
  • Smarter layouts: In the old system each layout joint had an aim constraint pointing it at the children of the locator it was point constrained to, which made it so x always oriented correctly, but using the aim constraint's offsets to change the orientation of the joint was counter intuitive and often the user wanted to just orient the joint with the rotate tool.
With this list I set about designing and building Riggs.  I've continued to add to and refine this system for about the last 4 years.  Now I'm going to jump ahead a bit and explain the 2 modes that Riggs has, "Layout" and "Rig", and then I'll get into how it jumps between the 2.



LAYOUT MODE

The purposes of layout mode are to place the skeleton to fit your geometry, to specify which animation controls will end up being built on which joints, and to set the local rotation axis on the joints in the skeleton. These user options live on the joints themselves, and what I called "Tags", and "Aimers". For Tags and Aimers the options are set up as channel box attributes to keep the GUI less cluttered, and it's quicker for the user than getting used to a new GUI

  • Aimers are used to modify local rotation axes. An Aimer's default state is to aim its X Axis at its children with a world up vector of Y, like the default Maya joint orient direction, but you can can tell it which children to aim at, set it to orient it to the world, or to point the Y up vector between its parent and children, like the reverse of a pole vector. This setting I called "Wedge". In the case of an elbow, this wedge mode ensures that when the elbow bends, the wrist will always travel toward the shoulder. The user can also just rotate the Aimer to adjust it to a preferred orientation.  When the rig is built the joint's orientation will match the Aimer.
  • Tags define which animation controls are built on which joints.  The Tag's options mimic the arguments of the python script that builds the control itself.  The control options are just attributes added to the the Tag's transform that live in the channel box.  For example for a leg IK, the Tag has arguments for the Knee, Ankle, Toe and Toe_End.  These arguments are set up as enum attributes containing a list of all the descendants of the joint containing the tag.
  • Users can save Limbs or parts of a rig by selecting the root of the section of joints they want to save and saving off a file.  These can then be loaded back onto another rig with all of the previous user data included(this never proved to be all that useful).
  • Everything will mirror from left to right, so users only have to work on one side and things on the other will update as they work.
  • The user can modify joint placement by translating, rotating, and scaling the joints, and their transforms get frozen when the rig is built, which speeds up the placement process.
  • In Layout mode there's a separate hierarchy of the nodes that mimic the structure of the skeleton and contain the Aimers and Tags.  Each joint has an associated "Holder" node that is point constrained to it, and holds the Aimers and Tags. This makes traversing the hierarchy in script super simple.



RIG MODE

In rig mode, the rig is animation ready with all of the controls/IK/etc fully built. In this state the user can bind geometry and paints skin weights, modify the cvs of the control curves, add/modify driven keys, space switches, lock/hide attributes, author blend shapes, and more.  If the user wants to add or remove joints, or change the position/orientation of joints, they switch back to layout mode, make their changes and build all the controls again with one button.

The rig's control hierarchy is separate from the skeleton but it builds in such a way that for every joint there is an associated set of nodes:

  • "Holder": Groups everything in the control rig associated with that joint.
  • "Connector": The only part of the rig that connects to the joint.
  • "InverseScaler": Used to mimic the inverse scaling that Maya's joints do so that the scaling behavior is the same.

Each joint's animation control lives under its associated Holder and drives its Connector, which then drives the joint.  For more complex set ups like IK, a separate chain of joints is used, and it lives under the Holder as well (This is something I'd like to change the next time around. If each set of controls lives in its own separate group, all grouped into 1 group and parent constrained as necessary, the rig is much easier to manage in script).

Now I needed to get from layout to rig and back in a quick and fluid way so that users wouldn't be afraid to make changes while they were working.  To do this I decided that in one state all of the nodes and data from the other state should be removed from the scene, and the data necessary to recreate the other state would have be stored before deleting it.



THE MEAT

One of the things I learned after writing the old system was to avoid special cases. In other words each process should be as unaware of other processes as possible. It forces you to rethink structure quite a bit and what you end up getting out of it is a far more robust and modular system. With this in mind I came up with the following setup for dealing with the meat of the system, which is the handling of the data being passed between layouts, rigs, and files(XML files that save and load all layout/rig data). During all the transitions (rig/layout controls to joints, joints to rig/layout controls, joints to file, file to joints) I extract the user data and write it to the joints, delete everything except the joints, and then read from the data saved on those joints. Here's a chart:
As I mentioned I wrote each of the read/write functions to be as unaware as possible. The writing script doesn't know what it's writing, and the reading script doesn't know what it's reading, they simply know how to walk through the joint hierarchy and blindly read or write what they find. This helps avoid special cases that tend to pile up and cause bugs. Here's a step by step explanation of what the script is doing when going from one rig state to the other:

  1. The first step in the process is to gather data necessary for recreating everything before breaking it down.  Using the skeleton as a map, the reading function runs through the joint hierarchy and looks for associated controls (nodes I flagged with an identifying attribute) to grab user data from, ignoring everything else.  This data is then saved on the joints as string attributes that contain python scripts so that the I/O scripts just execute whatever python code it found. The data is separated as build scripts that build the controls in a default state, and then edit scripts that recreate the changes the user made.
  2. Disable the skin clusters and disconnect them from the joints.
  3. After all the info is gathered and skin is disconnected the other control hierarchy is deleted, and any remaining nodes are cleaned away.  If going from layout to rig, the joints are oriented to the Aimer node.  If going from rig to layout, the joints are oriented to the world (so that mirroring works).
  4. Then once the joints are built, a new empty control hierarchy is built, either layout or rig.  In both cases there's a Holder for every joint. In rig mode there's a Connector and InverseScaler for every joint as well.
  5. Then it runs through the joint hierarchy one more time looking for attributes that contain the python scripts to execute that build the controls in their default state.  After executing the attributes are then deleted because they'll be recreated again in step 2 on the way back through.
  6. Now a second script pass this time going through the joints and looking for attributes that contain edit scripts to edit the controls to match the way the user left them, and recreate any extra changes the user made when they were last in that state.
  7. If going to rig mode, enable the skin clusters and reconnect them to the joints.
  8. Finally a cleanup pass, and it packages everything up, neat and tidy.
As I mentioned the data I was storing I separated into two types, build scripts and edit scripts.  The build scripts were the python scripts used to initially build the controls in a default state, and the edit scripts were python scripts that would recreate any changes the user made to the control after it was built, such as setting attributes or changing the shape of the nurbs curve.  Thinking in terms of building in a default state first and making user edits second helped with the decision making while writing this system. I also used the edit scripts to store any user added functionality I supported down the line for things that needed to be executed after everything was built, such as driven keys, space switches, and blend shape controls.

NOTE: The approach of saving the data as executable python scripts worked great for simplicity in script, but I did run into a lot of string formatting issues. Another issue was that the names of the joints were hard coded into the scripts, which proved to be a little unstable and meant that when a user renamed a joint, I had to make sure that the name was updated anywhere that name was stored.



.RIG FILES

Saving and loading rig files was important for 2 reasons.  One, I wanted the user to be able to extract and load rigs easily between scenes, and using Maya files for exporting/importing can be problematic as it takes a lot of crap along with the things you're trying to export.  Two, I wanted the default rigs for bipeds, quadrupeds, props, etc to be stored as a file in a sub folder of the Riggs directory.  This made authoring the default rigs easy, and the GUI could just look in that folder to list all the default rigs for a user to start with.

To save/load these rig files I used python's ElementTree which writes/reads XML files.  This sped up the process immensely and made things quite simple.  Since I had a system in place that gathered all the data necessary to recreate controls and saved it on joints, all I had to do was run though the joint hierarchy, save the joint's name, its world space position/orientation, its parent, and grab the user defined attributes containing the python scripts to execute.  I was also able to use ElementTree's XML hierarchy for the joint hierarchy, meaning the XML items have children and parents, and I could use that same XML tree info to store the joint tree info.



THE GUI

Working at a company with lots of proprietary Maya tools you naturally end up with a lot of windows cluttering up your workspace, so I wanted to design a GUI that didn't take up much space.  To keep the size down, and to make things clearer and less cluttered for the user, I made a GUI that would dynamically change whenever appropriate so that the user would only be seeing what they needed to see at any given time.  The animation control options would live in the channel box, not in the GUI.  While in Layout mode, the user would see the layout tools, in Rig mode, the user would see rig tools, etc.  The beauty of this approach is that it makes the system easier to approach for new users because they are only seeing the things they need, and gradually get introduced to the rest of the functionality, rather than bombarding them with a screen full of buttons and options all at once. Riggs has since expanded quite a bit from the first iteration without cluttering the GUI too much.

Knowing that I was going to be continually expanding on the system, I wanted to put an automated system in place for populating the GUI with available animation control types(IK, etc).  This made adding new control types quick and painless.  Each control type had its own python file that the GUI script pulled related information from and populated menus with.

As I mentioned earlier, naming conventions are great when you're the only one using them, but trying to get a team of people to follow your conventions is an endless battle.  I had so many problems related to people not following conventions that I wanted to eliminate them completely, but found that I still needed users to define left and right sides using prefixes so that Mirroring could work, so I put all of the prefixes in a naming window with drop down menus when users create a new joint.  That way users weren't responsible for remembering, and it worked out quite well. 



FILE STRUCTURE

All of the related read/write scripts I described in "The Meat" section lived in a single file.  I then kept everything rig related in a utils_rig.py file, everything layout related in a utils_layout.py, and a utils_general for things like hard coded naming conventions, getting/setting data, progress windows, etc.  All of the scripts to build the controls lived in a folder called ctrl_scripts, default layouts lived in a layouts folder, and plugins in a plugins folder.  Using python's __init__.py file I gather and source everything in the respective folders, and automatically load plugins in the plugins folder when riggs is sourced. Inside the ctrl_scripts/__init__.py file I wrote some functions to assist in gathering info about the available ctrl scripts. I'm pretty sure this is not the intended python structure, but it worked okay for my purposes, except that I ran into some import problems. Files in the ctrl_scripts folder needed to import files in the main folder like utils_rig.py, and I couldn't seem to figure out how to reload the scripts when I made changes.  I could reload them in Maya, and Maya would see the latest changes, but the ctrl_scripts files would still be referencing the older revision of the file.  I had to reboot Maya to get around it.



OTHER NOTABLE FEATURES

  • I wrote a script to disconnect and reconnect skinning data from its influences.  This enabled all the skinning data to remain in place, attached to the geometry in a disabled state, while making edits to the skeleton in Layout mode.  When disconnecting the joints I save a list of joint names as an attribute on the skin cluster.  To reconnect I could just list skin clusters with that attribute on them and reconnect.  This opened up the ability to reconnect the skin in Layout mode, to edit both the geometry and the joints at the same time. Bonus.  Also because the joints were no longer connected, the geometry could be exported with all of the skinning data attached then imported and reattached to any joints with the same names in any other scene.
  • When a user creates new joints, the system does its best to guess what the naming might be.  Not only is this a time saver, it also serves to keep naming consistent.  If the user duplicates a joint with children, they can select from the prefix drop down menus to add a prefix to the new chain, or perform a find/replace on all the children at once to avoid name clashing, and if the chain of joints uses the the numbering format "JointA1" "JointA2", the scripts will automatically create a JointB chain with joints "JointB1" "JointB2".  This is a great time saver when you're rigging cloth or anything that requires several related chains. If they duplicate a joint called "PropA", it will just create a "PropB" joint. 
  • For editing Layouts I was able to override Maya's built in MEL commands for basic hierarchy operations like parenting and duplicating to handle all the custom operations needed when making changes to Layouts, like matching sides while mirroring.  This makes working on Layouts more intuitive and fast.  I had to have buttons in the GUI for some operations though(like renaming) and it took a fair amount of work to get all the mirroring to work during all of these operations. Having custom layout nodes from a plugin would probably be a better approach for layouts.
  • When saving the nurbs curve shapes, I generate a size value based on joint distances, and then I divide the point positions by that value.  Then when I set the point positions I then multiply by that value.  That way if the distance between the joints has changed, like if a character's limbs are scaled to a different size, the nurbs curve will match the scale changes as well.
  • One of the control scripts I implemented is a python script that is run when the control is built.  This works great for any sort of custom controls. This was something I built primarily for myself knowing that most people using the system wouldn't know how to use it.


WRAP UP

This system works great for generating game engine friendly rigs quickly(very quickly) and easily, keeping rigs consistent, keeping absolute control over your joint hierarchy and orientations, and making changes to already rigged characters.  It's simple enough for people in the 3D world that aren't very technical(or just non-riggers) to use, and robust enough to handle rigs of all shapes, sizes, number of limbs, etc. On the development side, adding new features is quick and painless. Simplicity, editability, and quick rig authoring speed were the primary goals when designing this system and it works great for an environment like Double Fine Productions. They're continuing to use it and expand on it in my absence.

With what I learned while writing Riggs I have a new long list of improvements in mind for the next system, one with more efficient data handling, one that most likely will have a node graph(it's all the rage right now), one with custom nodes that simplify the complex IK setups. Some parts of Riggs are cluttered and unstable, like the edit scripts that are saved between layout and rig that I keep packing more features into. I'd also like to be able to only rebuild only pieces of the rig, rather than editing the entire thing at once. There are things that took longer than I liked to code and still have bugs here and there, like all of the layout operations, especially when taking mirroring into account. I learned that with an understanding of PyMEL and Maya's class system there are ways of getting at and setting data that are much faster than Maya's scripts. Even with all these improvements in mind, I'm sure I'll learn things along the way that I'll be wishing I knew when I started. I suppose that's what keeps it interesting.


Thanks for reading and feel free to question/comment at the bottom of the blog page.

6 Comments:

armin

Hi Tyler,

You wrote that you rewrote some of Maya's commands like parenting and duplicating. I'm curious why you had to do that and what are the changes that you made?
And when creating segments, like an IK limb for example, did you write it in an object oriented way and made changes to that object or did you generate them proceduraly and rebuilt them every time you had to change something?

Thanks
Armin

Tyler Hurd

Hey Armin thanks for the questions,

The Riggs layouts had so many features that were affected when a user would parent/delete/duplicate that I needed a python script that ran a bunch of custom operations each time so the rest of the layout stayed up to date. To make things simpler on people so they don't need to remember to press a separate button for parenting/duplicating/deleting instead of just pressing the p/ctrl+d/delete keys, I just took Maya's MEL commands for these operations and added my own code to them so that when users pressed the keys, it would recognize that they were selecting something in Riggs, and run my custom stuff.

I used Maya's built in objects and just saved my own data on top of them. Things like IK setups were so complicated that it would have been a nightmare to try to edit them in their fully built state. It's much easier to just tear them down and rebuild them again and again from whatever positions the joints are in. Does that answer your question?

armin

Hey Tyler,

Yeah, that answers my question. Thanks.
It makes sense to tear everything down and just keep track of changes. Instead of updating everything.

By the way Riggs looks really impressive. I'll be looking forward to a showreel with the next version of rigs.

Tyler Hurd

Thanks!

Danny Wynne

Hey, thanks for post! Im currently designing a rigging system where I work. I'm wondering how your system handles editing rigs that already have loads of animation files. Is hopping between layout and rig mode more for the initial build of the rig? If you're allowing users to easily move joints, wouldn't that make their edits incompatible with everything thats been created? When we make edits to bone layouts here, we have to batch out all old animations, update a state machine, etc etc. It can be quite a hassle. From a pipeline perspective, that harder it is to change joint layout, the better.

I'd love to hear some examples/stories of your system in action.

Right now Im pulling a lot of inspiration from David Hunt's Modular Procedural Rigging at Bungie GDC '09 talk. It addresses my current issue dealing with tons of already animated scenes. I recommend checking it out.

Anyways, thanks for the post, good work!

Tyler Hurd

Hi Danny, thanks for the interest.

The hopping between layout and rig mode is mostly for the initial build and when adding parts onto existing rigs it doesn't affect the animations at all. You can definitely make changes to rigs that have animation sets already, but it requires a batch export of the animations afterward like you mentioned. If you have a batch exporter that step is pretty painless. Certain changes do break the animations though, especially on limbs with IK, if you change a local rotation axis and its translation was keyed, or anything that has a constraint on it in the animation file. The batch exporter at DF can also run a MEL script in each file before exporting so a lot of times if manual changes needed to be made to the animation curves to fix up the animation, and the changes were straight forward enough to be automated, then I would write a MEL script to fix up all the files in 1 go. For instance if you just want to scale the entire rig you just need to scale all the translate keys in the animations that same scale factor and everything works perfectly. I can see if you're making a 1st person shooter or something this would be WAY more of an issue than it is for the games of Double Fine as the animation sets would be way more complicated.

After some trial and error it becomes like anything else in game development. You start to know in advance how long a change will take, what it will affect, the risks involved, and the tools at your disposal to deal with the errors that come up, so you can decide whether or not the change is worth making within the scope of your project.

That said I was actually pretty surprised when making a change to a rig with an animation set how decent it turned out. I always assumed the animation would turn to crap but the motion the animator was using to communicate would usually come through fine.

I've always wanted to try to record the changes a user makes to a rig and automatically apply the animation fixes needed based on those changes the next time the user opens an animation file, buuuuut I never went down that road as it seems like a long one.

Nice, I actually attended that David Hunt talk and it was super interesting and inspiring. I took a little bit away from it but Bungie makes very different games from Double Fine with very different dev challenges so most of it was not applicable for us.

I'm going to stop writing now.




Post a Comment