RenderMonkey Beginner’s Tutorial

This time I’ll be presenting a beginner’s tutorial about RenderMonkey, which is a shader development environment created by AMD that facilitates the task of creating and testing shader programs. Although AMD has stopped developing it, you can sill get it for free.

I’ll be using HLSL for the scope of this tutorial, but I intend to focus only on how to get RenderMonkey up and running, so I won’t be teaching you how to program in HLSL. If you don’t have any experience, here’s a nice resource for getting started with HLSL – kudos to Catalin Zima for the wonderful tutorial. After learning the syntax, you should also learn how to implement some basic shaders. The XNA education catalogue has a series just for that!

Shader Code

The shader used in this tutorial mixes ambient and diffuse lighting with a diffuse texture map. Here’s the code for the shader:

And what it’s supposed to look like:

So let’s get started!

Creating the Effect

When starting RenderMonkey you’ll get an Empty Workspace that’s ready to be used. Your application should look like this:

A lot of your work will be done on the Workspace Window on the left, which by default shows a single empty workspace. So let’s start by adding a new Effect to it. Right-click on Effect Workspace and add a new Default Effect of your desired type (2). Since I’m programming in HLSL, I created a new DirectX shader. There are many templates ready that you can use when starting your own shaders, but for this tutorial I’ll just choose “DirectX” from the list which creates the most basic shader possible (i.e. a vertex shader that simply applies a WorldViewProjection transformation and a pixel shader that always outputs the same color).

Once you have your Effect created, your window should look like this:

Now we’re ready to start setting everything up!

Changing the Model

You probably noticed that the default model used by RenderMonkey is a sphere. That’s not the best choice for testing your shaders, so let’s start by changing it. On your Workspace Window simply right-click on Model and select a new one:

I’ll use a simple teapot:

Setting Variables

The next step is to create a link between RenderMonkey and the global variables used in our shader. The global variables used by our shader are as follows:

We’ll need to create a new variable inside our Effect on the Workspace Window for each global variable in our code, so that RenderMonkey can interact with our shader. Make sure to give them the same name and the appropriate data type too. Creating a new variable is as simple as right-clicking on your Effect, going under Add Variable and picking the right data type from the list:

You can start by creating a Float3 called LightDirection and a Color called Ambient.

As for defining our matrixes (such as WorldViewProjection and WorldInverseTranspose), these usually have special meanings, and we’d like RenderMonkey to establish the right connection between them and our scene (e.g. link the View matrix to our camera). To do that, simply use one of the Predefined matrixes that RenderMonkey provides, as each of them already come with the appropriate semantic assigned:

Finally, for creating the texture variable that will be sampled by the shader, we need to do it in two steps.

First, right-click on your Effect, select Add Texture and load a 2D Texture of your choice:

Next, scroll down to Pass 0, right-click on it and add a new Texture Object, linking it to the texture you just created. Be sure to name this Texture Object the same as the sampler used in your code (in our case, Texture):

I also got rid of the matrix that came with the default effect since I won’t be needing it. When you’re done, your Effect tree should look like:

Binding Shader Inputs

One other step we need to take, that is frequently forgotten, is to bind RenderMonkey to our vertex shader inputs so that the correct data is streamed. Looking back on our code, we had the following vertex shader input structure:

We’ll need to mimick this structure on RenderMonkey. Inside your Effect tree, right-click on Stream Mapping and choose Edit:

On the window that appears, simply add two new channels and configure them such that they’re compatible with the data types and semantics of your vertex shader input. In our case:

Create the Code

Now for the most important part, entering the code. Scroll down to Pass 0 and right-click on Vertex Shader, choosing Edit from the list. That should open the Code Editor for your effect, with the default implementation still on it:

The most relevant thing to notice here is that unlike most shaders you write, in RenderMonkey you will write your vertex shader and your pixel shader in separate tabs, and the variables you define in one of the tabs are local to it (i.e. can’t be accessed by the other tab).

This means that you’ll have to split your shader code in two, the vertex half and the pixel half. That’s no big deal though. Here’s what it looks like after the split (refer to the full implementation on top for comparison).

Under the Vertex Shader tab:

Under the Pixel Shader tab:

The only points worth noting is that I repeated my VS_OUTPUT structure since it was needed on both sides, and I cleaned my global variables so that only the ones that were used in each file were present.

Also, ensure that your vertex shader and pixel shader functions names match the ones defined under Entry Point. By default, these should be “vs_main” and “ps_main” respectively.

Now let’s see the code in action! Simply press the Compile button on the toolbar:

And this is what you should get:

It looks all white! But don’t worry, that’s only because your LightDirection and Ambient variables are still with their default values. RenderMonkey provides an interesting way for you to edit your variables, so that you may see your shader updating in real-time as you change them!

Tweaking Variables

There are two variables we’d like to tweak here: LightDirection and Ambient. Start by right-clicking on your LightDirection variable and setting it as an Artist Variable. Colors are set as Artist Variables by default so you don’t need to repeat the process for your Ambient color:

Now fire up the Artist Editor window:

And this is what you’ll see on the right side of your application:

Now all you have to do is tweak the variables to your liking. Expand your options in the Artist Editor window so that you have better control over them. In my case I specified that my LightDirection vector should be normalized, and I set my Ambient color to a very dark color:

Texture Sampler Options

Just to throw in one last tip, the texture on my teapot looks a bit rough. I’m guessing it’s probably being sampled using point filtering. Let’s change that!

Right-click on your Texture Object and choose Edit. That’s where you can configure everything about your sampler! I’ll simply change my magnification/minification/mipmapping filters to linear, which should make the texture appear smoother:


And that’s it! You can now maximize your preview window and enjoy. Oh, and you can use the camera controls located on the toolbar to navigate the scene:

Here’s the final result:

I hope you enjoyed this tutorial, and as usual feel free to post any questions on the comment section below.

  1. parkpd says:

    Hi, I like your RenderMonkey tutorial blogs.
    Can I translate these to Korean and post to my blog?

  2. parkpd says:

    I’ve translated your post. You can see that at I will translate also. Thanks in advance. 🙂

  3. Izabella says:

    Hi, just wanted to thank you for this great tutorial, I´m an MMD user.
    The link above is MMD.
    Anyhow, half the time we spend making models instead of animations, but no one really makes effects.
    This dude is the only one, cept we ppl dont speak japanese much so I thought of making my own. So arigato again!

  4. Jin says:

    That’s a very helpful. Thanks for sharing.

  5. faTWave says:

    Oh man, thank you so much Catalinzima! I’m no student of these sorts of matters but I do have a need for the ability to do “simple” effects, such as lighting for a particular side, given what I do. This tutorial of yours will finally make my rendermonkey be of some use. This, along with catalinzima’s tutorial…oh I thank you so much. Know that I will forever credit you two whenever I use effects that I will create perhaps someday in the future…

  1. By Adding a Skybox to RenderMonkey | David Gouveia on September 12, 2011 at 8:37 pm

    […] I wrote a tutorial about the basics of RenderMonkey. This time I’ll be showing how to replace the solid colored background that RenderMonkey uses […]

Leave a Reply