Render to texture questions

[fedyakin]

Hi, I am trying to make a window which shows a preview of a player's character. My current approach is to create a scene containing that character and render it to a texture. I then take that texture and create a CEGUI imageset and image from it. The last step is to set the Image property of a StaticImage to the created image. Up to this point everything works fine. The problem I am encountering is that when the scene changes, the interface is not updated. Images / imagesets in cegui appear to be intended for images that do not change.

I know when the contents of the render texture were changed (I can use an Ogre::RenderTargetListener). As a hack I called invalidate on the window from the callback, and got the desired results. However, this is not a general solution as there is no way to know all windows that are using an image in an imageset.

Should I be trying a different approach? I thought about trying to setup the window to have a CEGUI::RenderingSurface, and rendering my scene to that. However, that doesn't seem possible without making a lot of modifications to CEGUI (mostly in the Ogre Renderer). In addition it wasn't clear to me if that would solve my problem, or if the use of the rendering surface was also cached. The only other approach I can think of is to create either a custom window / window renderer that renders directly from an Ogre Texture, and could invalidate itself when necessary.

Does anyone have any suggestions, or insight into how I might solve this problem? Is there a better approach than what I tried?

[CrazyEddie]

Hi,

The cause of the issue you are seeing is the window imagery caching that's being performed; the Windows are drawn and cached to texture. When you subsequently change the content of the texture used for the Imageset, the change is not reflected in the output because CEGUI is using the cached version of that imagery.

As far as this approach goes, there are two things to do; invalidate the Window, or invalidate the RenderingSurface, from a technical standpoint invalidating the RenderingSurface is more correct, since you don't need the window to regenerate it's geometry, you just want to 're-cache' the imagery. So, your hack is actually a valid solution.

this is not a general solution as there is no way to know all windows that are using an image in an imageset.

Who is setting this image to be used then? Surely if the app is setting the image, the app knows which windows it is set to?

Depending upon how often the RTT content is going to be updated, and which other requirements you have (meaning if you need CEGUI rotation or effects), it could be better if the rendering surface for the window(s) hosting the RTT based imagery to be disabled; this way the content is not cached and will appear to 'update' whenever System::renderGUI is called. Again this requires that you correctly manage which windows are likely to be used in this way (unless you disable surfaces for all windows, though you will likely pay a performance penalty for doing that, and lose access to some other features - such as rotations and other effects).

I thought about trying to setup the window to have a CEGUI::RenderingSurface, and rendering my scene to that. However, that doesn't seem possible without making a lot of modifications to CEGUI (mostly in the Ogre Renderer). In addition it wasn't clear to me if that would solve my problem, or if the use of the rendering surface was also cached. The only other approach I can think of is to create either a custom window / window renderer that renders directly from an Ogre Texture, and could invalidate itself when necessary.

A few points to cover from this paragraph:
1) As alluded to above, the window is likely already using a RenderingSurface and this is actually causing the 'issue' (though it's not a real issue since it's behaving as it was designed ).
2) I don't think any modifications would be needed to render your own content to the RenderingSurface used by a Window - whether a surface created by CEGUI, or one you supplied yourself. This kind of ability was 'in mind' as part of the rendering back end rewrite.
3) This should, in theory, be possible without much in the way of custom code; perhaps a couple of event handlers, not much else though.

Ok, now I've said all that, you're probably going to want me to provide an example If you do want this, I can try to do this over the weekend; it's unlikely I'll get to it today

CE.

[fedyakin]

Hi

Thanks for the quick response. What I was planning on doing was during interface creation programmatically creating various images (character previews, animated textures, etc). The names of these images would then be available to interface developers to use anywhere they wanted in the actual interface. Under this model I as an engine developer wouldn't know what windows end up using the images I generated. That's why I said invalidating the window wasn't a general solution (It could be made to work, by adding an extension to the interface scripting language to register which windows are using what images...)

As you suggested, setting AutoRenderingSurface to false on the FrameWindow containing the StaticImage worked. This is a viable solution, although it would be nice to come up with one that allows the rest of the window to be cached.

In regards to rendering to the RenderingSurface, you are right; all of the underlying Ogre objects appear to be accessable. My attempts to render to it didn't turn out well, but it's likely what I tried to do was wrong. I basically got the Ogre::RenderTexture of the window's RenderingSurface and added a viewport to it, using the camera in my scene. The snippet below shows this attempt. A sample of how to do this would be awesome and much appreciated, but if I'm close, a few pointers would probably be sufficient...

Thanks again for the help.

Code: Select all

// Ensure the window has an auto rendering surface
if( !aWindow.isUsingAutoRenderingSurface() )
    aWindow.setUsingAutoRenderingSurface(true);

// Get the ogre render texture from the window
CEGUI::RenderingSurface *surface = aWindow.getRenderingSurface();
CEGUI::RenderTarget &baseTarget = surface->getRenderTarget();
CEGUI::OgreTextureTarget &target = dynamic_cast<CEGUI::OgreTextureTarget&>(baseTarget);
CEGUI::Texture &baseTexture = target.getTexture();
CEGUI::OgreTexture &texture = dynamic_cast<CEGUI::OgreTexture&>(baseTexture);
Ogre::TexturePtr ogreTexture = texture.getOgreTexture();

// Add a viewport to it
Ogre::RenderTexture *rttTexture = aTextureTarget->getBuffer()->getRenderTarget();
Ogre::Viewport *viewport = rttTexture->addViewport(mCamera, 1);
viewport->setClearEveryFrame(true);
viewport->setBackgroundColour(Ogre::ColourValue::Black);
viewport->setOverlaysEnabled(false);


[CrazyEddie]

What I was planning on doing was during interface creation programmatically creating various images (character previews, animated textures, etc). The names of these images would then be available to interface developers to use anywhere they wanted in the actual interface.

Ah, right. As you say, there are some ways around this initial 'not knowing', but it's a kind of extra step

setting AutoRenderingSurface to false on the FrameWindow containing the StaticImage worked. This is a viable solution, although it would be nice to come up with one that allows the rest of the window to be cached.

Yeah, that's actually somewhat difficult due to the hierarchical nature of the windows and such. It's theoretically possible by having the FrameWindow itself not use the AutoRenderingSurface, the StaticImage not use AutoRenderingSurface, but have all other content contained within a DefaultWindow that does have AutoRenderingSurface enabled. Obviously, again, this requires that UI designers know about such arrangements and plan appropriately.

I'm going to work up an example of rendering directly to the surface. Not because you're a million miles away or anything, but because while I said it was 'in mind' as part of the rendering redesign, and it was, but it's something that's not been tried or tested much - so perhaps for Ogre I've overlooked something, or there are bugs lurking - my example should expose such shortcomings (as well as providing the beginnings of a new tutorial ).

CE.

[CrazyEddie]

Note: The example code below relies on a new function added in branches/v0-7 @ r2470.

I worked up a demo for this and early on it became apparent that there were vaious 'levels' of implementation for this. Basically, the code you previously posted is the simplest of the levels; you say your appempts to render like that "didn't turn out well" - I wonder what that means exactly?

My initial code was basically the same as you posted - the content was rendered to the surface of the FrameWindow overdrawing the other window content. Adjusting the dimensions of the added viewport allows us to target specific areas of the texture so we can more easily share the surface with regular CEGUI content. After doing this, and playing around some more, there are two main issues with this approach:

1) Because the rendering is by default controlled by Ogre, things are drawn out of sequence as far as CEGUI is concerned, and this means you get visible flickering as the CEGUI and viewport content are drawn over each other.

2) Resizing a window that we have 'hooked into' in this fashion can result in the RenderTarget being deleted and/or reallocated in order to cope with the resize; the best case here is that the viewport content no longer gets drawn to the Window surface, and the worst case is that we seg (depending on how we have things arranged).

So, to tackle these issues we have to jump some hoops. I'll briefly discuss solutions to these issues separately.

1) The basic solution to this is to call setAutoUpdated(false) for the Ogre::RenderTarget that we extracted from the CEGUI::RenderingSurface:

Code: Select all

// tell Ogre not to draw this target automatically (else you get
// visible flickering).
mRttTexture->setAutoUpdated(false);

We'll then update it manually at an appropriate time. Ok, so when is an appropriate time and how do we know when it arrives?

An appropriate time would be after all CEGUI texture caching is done, but before the results of that is actually rendered to the screen. Each CEGUI::RenderingSurface has a number of rendering queues, and where these exist (they're created on the fly upon first usage), events are fired before and after each queue is rendered for each individual CEGUI::RenderingSurface. We can use these events to be informed that the rendering is about to be done to the screen - so we subscribe a handler to CEGUI::RenderingSurface::EventRenderQueueStarted for the default rendering root, we do not subscribe to the surface we're going to draw to, because that only gets updated when the window changes, so we use something higher up in the hierarchy and for our purposes here, the RenderingRoot (which is itself a RenderingSurface) returned via CEGUI::Renderer::getDefaultRenderingRoot is what should use - and this would suffice for many, many other scenarios too:

Code: Select all

// Since we set the Ogre RTT above to not be auto updated, we'll use an
// event subscription to decide when to render that content manually.
// Basically this needs to be done after other UI content is drawn but
// before anything gets put on screen.  We need to use an appropriate
// RenderingSurface for this - not the surface used for the window we
// will render directly to - but something higher up.  For this demo
// (and many other scenarios) the RenderingRoot will suffice.  We use
// the EventRenderQueueStarted because we want to update the content
// /before/ it's drawn to the surface (here that would be the screen).
mRenderer->getDefaultRenderingRoot().subscribeEvent(
    RenderingSurface::EventRenderQueueStarted,
    Event::Subscriber(&OgreRTTDemo::updateRTTContent, this));

Within the handler we subscribe we check to see which queue the event is for. As mentioned queues are created as-needed, so checking for a queue that has not been 'touched' elsewhere will not result in any match. One queue we can be sure exists is CEGUI::RQ_BASE, and so we'll check for that:

Code: Select all

// There are many rendering queues that may (or may not) be present for
// a RenderingSurface.  For our purposes, we can hook the 'default' of
// RQ_BASE. - and we need to do our rendering /before/ this queue
if (static_cast<const CEGUI::RenderQueueEventArgs&>(args).queueID != CEGUI::RQ_BASE)
    return false;

Once we know we have the right queue, we just update the Ogre::RenderTarget:

Code: Select all

// get the target to draw itself.
mRttTexture->update();

Hmmm. One snag, though Doing this obliterates the render states that CEGUI so carefully set up earlier for it's rendering. Which means it's likely that nothing will actually be seen due to this. Because there is no state management in the Ogre::RenderSystem, we have to reinitialise all the states again - this is done via CEGUI::OgreRenderer::initialiseRenderStateSettings - and activate the default rendering root for rendering:

Code: Select all

// Reset the initial CEGUI rendering states.
mRenderer->initialiseRenderStateSettings();
// (Re)Activate the appropriate CEGUI render target.
mRenderer->getDefaultRenderingRoot().getRenderTarget().activate();

With this in place, rendering is now done in the correct order and there is not more flickering or other such atefacts.

2) To solve this second issue we again use an event handler. Basically we subscribe to hear when the window with the CEGUI::RenderingSurface we are targetting is resized:

Code: Select all

// subscribe to hear about resize events.  This is needed because
// when the underlying texture gets resized, all the associated objects
// must be assumed to be lost also
fw->subscribeEvent(Window::EventSized,
                   Event::Subscriber(&OgreRTTDemo::handleResize, this));

In the handler, we see if our pointer to the Ogre::RenderTarget has changed - indicating the old one will no longer be valid - and if it is changed, we update our set up to use the new Ogre::RenederTarget instead. One thing to note here is that the Ogre::Viewport dimensions are relative to the entire surface of the Ogre::RenderTarget, so shrinking the window again does not automatically result in shrinking of the viewport - to overcome this we would adjust the Viewport dimensions based on the CEGUI::Window size and how that relates to the Ogre::RenderTarget size (this was not done in my demo, but just involves fetching and dividing the two dimensions and multiplying the viewport dimensions by the result):

Code: Select all

// get rendering surface used by FrameWindow.
RenderingSurface* surface =
    static_cast<const CEGUI::WindowEventArgs&>(args).window->
        getRenderingSurface();

// if there's no surface, skip the RTT setup parts!
if (surface)
    initialiseRTTViewport(surface);

Here is a video I made of my demo app:


Having gone through all this, I'm not sure how appropriate this is to your scenarion However, it's a good demonstration of how - with a little effort, at least - it's possible to directly render anything you like to a CEGUI window.

Below are the complete functions that are relevant to the above discussion. I may clean this up into a Wiki article in the coming weeks (if any one else wants to do it instead, that would be great!).

CE


CEGUI Initialisation - called at start up:

Code: Select all

//----------------------------------------------------------------------------//
void OgreRTTDemo::setupCEGUI()
{
    using namespace CEGUI;
    mRenderer = &OgreRenderer::bootstrapSystem();

    // Load scheme and set up defaults.
    SchemeManager::getSingleton().create("TaharezLook.scheme");
    System::getSingleton().setDefaultMouseCursor("TaharezLook", "MouseArrow");

    // create root window.
    WindowManager& wmgr(WindowManager::getSingleton());
    Window* root = wmgr.createWindow("DefaultWindow");
    System::getSingleton().setGUISheet(root);

    // create frame window for demo.
    Window* fw = wmgr.createWindow("TaharezLook/FrameWindow");
    root->addChildWindow(fw);
    fw->setSize(UVector2(UDim(0.5f, 0), UDim(0.5f, 0)));
    fw->setText("Ogre / CEGUI Direct Rendering Demo");

    // create a button
    Window* btn = wmgr.createWindow("TaharezLook/Button");
    fw->addChildWindow(btn);
    btn->setPosition(UVector2(UDim(0.25f, 0), UDim(0.7f, 0)));
    btn->setSize(UVector2(UDim(0.5f, 0), UDim(0.15f, 0)));
    btn->setText("This is a button!");

    // get rendering surface used by FrameWindow.
    RenderingSurface* surface = fw->getRenderingSurface();
   
    // if there's no surface, skip the RTT setup parts!
    if (surface)
    {
        initialiseRTTViewport(surface);

        // Since we set the Ogre RTT above to not be auto updated, we'll use an
        // event subscription to decide when to render that content manually.
        // Basically this needs to be done after other UI content is drawn but
        // before anything gets put on screen.  We need to use an appropriate
        // RenderingSurface for this - not the surface used for the window we
        // will render directly to - but something higher up.  For this demo
        // (and many other scenarios) the RenderingRoot will suffice.  We use
        // the EventRenderQueueStarted because we want to update the content
        // /before/ it's drawn to the surface (here that would be the screen).
        mRenderer->getDefaultRenderingRoot().subscribeEvent(
            RenderingSurface::EventRenderQueueStarted,
            Event::Subscriber(&OgreRTTDemo::updateRTTContent, this));

        // subscribe to hear about resize events.  This is needed because
        // when the underlying texture gets resized, all the associated objects
        // must be assumed to be lost also
        fw->subscribeEvent(Window::EventSized,
                           Event::Subscriber(&OgreRTTDemo::handleResize, this));
    }
}

initialiseRTTViewport Function that initialises the RTT options and viewport:

Code: Select all

//----------------------------------------------------------------------------//
void OgreRTTDemo::initialiseRTTViewport(CEGUI::RenderingSurface* const surface)
{
    const Ogre::RenderTexture* const old_rtt = mRttTexture;

    // extract the Ogre render target at for the surface
    mRttTexture = dynamic_cast<CEGUI::OgreTexture&>(
        dynamic_cast<CEGUI::OgreTextureTarget&>(
            surface->getRenderTarget()).getTexture()).
                getOgreTexture()->getBuffer()->getRenderTarget();

    // only do set up if target is changed.
    if (old_rtt != mRttTexture)
    {
        // tell Ogre not to draw this target automatically (else you get
        // visible flickering).
        mRttTexture->setAutoUpdated(false);

        // setup viewport from the same camera as the main scene.
        mRttTexture->addViewport(mCamera, 0, 0.02, 0.09, 0.5, 0.5)
            ->setOverlaysEnabled(false);
    }
}

The handler to do the RTT content update:

Code: Select all

//----------------------------------------------------------------------------//
bool OgreRTTDemo::updateRTTContent(const CEGUI::EventArgs& args)
{
    // There are many rendering queues that may (or may not) be present for
    // a RenderingSurface.  For our purposes, we can hook the 'default' of
    // RQ_BASE. - and we need to do our rendering /before/ this queue
    if (static_cast<const CEGUI::RenderQueueEventArgs&>(args).queueID != CEGUI::RQ_BASE)
        return false;

    // get the target to draw itself.
    mRttTexture->update();

    // Reset the initial CEGUI rendering states.
    mRenderer->initialiseRenderStateSettings();
    // (Re)Activate the appropriate CEGUI render target.
    mRenderer->getDefaultRenderingRoot().getRenderTarget().activate();

    return true;
}

The handler called when the window is resized:

Code: Select all

//----------------------------------------------------------------------------//
bool OgreRTTDemo::handleResize(const CEGUI::EventArgs& args)
{
    using namespace CEGUI;

    // get rendering surface used by FrameWindow.
    RenderingSurface* surface =
        static_cast<const CEGUI::WindowEventArgs&>(args).window->
            getRenderingSurface();

    // if there's no surface, skip the RTT setup parts!
    if (surface)
        initialiseRTTViewport(surface);

    return true;
}

[fedyakin]

Thanks a lot for the example, it helped tons. What you posted didn't work for me; I believe the combination of Ogre 1.7 RC1, direct3D renderer, and octree scene manager behaves differently then what you tested with. What happens with this combination is that CEGUI::System::renderGUI results in a call to beginScene, then in the EventRenderQueueStarted handler, updating the Ogre::RenderTexture also calls beginScene. The second beginScene fails as there has been no endScene yet. I changed when rendering occurs and got things working. I suspect that I might be ordering things slightly wrong as I need to call invalidate on the window, where your example didn't need to do that.

Instead of subscribing to EventRenderQueueStarted I turned off the automatic rendering in CEGUI and added a frame listener of my own. Now neither CEGUI or the Ogre::RenderTexture auto update so the application can order them.

Code: Select all

// Manually control when the gui is rendered
mRenderer->setRenderingEnabled(false);
Ogre::Root::getSingleton().addFrameListener(this);


When it is time to render, I first invalidate the window whose surface is going to be rendered to, then have the ogre texture render to the window, and finally have CEGUI render the gui. If I don't call invalidate, then child windows get overdrawn by ogre rendering to the window. If I call invalidateSurface instead of invalidate then the parent frame window often flickers out of existence.

Code: Select all

bool InterfaceManager::frameRenderingQueued( const Ogre::FrameEvent &aEvent )
{
    // Only render when CEGUI is initialized
    if( mSystem )
    {
        // get the targets to draw themselves.
        mWindow->invalidate(false);
        mRttTexture->update();

        // Now render the gui
        mSystem->renderGUI();
    }

    // True indicates that Ogre should continue rendering
    return true;
}


With the above changes, I can now render my preview scene directly onto a cegui window though!

[CrazyEddie]

I'm glad you have a working solution. The issue with _beginScene/_endScene should be solvable by changing the setting that makes those calls:

Code: Select all

myOgreRenderer->setFrameControlExecutionEnabled(false);

Note that this will also automatically set rendering enabled state to false, since the latter is dependant upon the former.

With regards to the rendered view overdrawing the other child content, that's obviously clearly dependant upon when each part gets drawn, and you should be solvable by hooking some other queue, depending on the actual arrangement of windows

It's all quite interesting stuff

CE.