[SOLVED] CEGUI -> Ogre3D Render-to-Texture Alpha Weirdness

[mmixLinus]

Hi folks,

I'm running CEGUI 0.8.3 and Ogre3D 1.9.0. I am having a slight cosmetic problem with my second GUI context. It is drawn using a CEGUI::OgreRenderer, based on the recommended CEGUI RTT code. The picture below shows the corner of the yellowish album Cadmium by the group Sky as a texture on a mesh in Ogre3D, with a CEGUI ImageButton superimposed.


Problem: The shadow of the World button has become completely opaque.

The CEGUI ImageButton I am displaying on top of the yellow texture uses this graphic from the Crystal Clear collection by Everaldo Coelho. As you can see, it has a semitransparent shadow, but the world button rendered above has lost all transparency/alpha information.

The Ogre::TextureUnitState for my "CEGUI" texture, that is to be blended with the underlying yellow texture, is blended using these operations:

Code: Select all

tus->setColourOperationEx(Ogre::LBX_BLEND_TEXTURE_ALPHA);
tus->setAlphaOperation(Ogre::LBX_BLEND_TEXTURE_ALPHA);


I have tried setting the TUS operations to "LBX_SOURCE1" so I only see the CEGUI source, and it shows a transparent texture with an opaque button with an opaque shadow so I guess it is the actual rendering by CEGUI that's ruining the shadow.
The actual button is this:

Code: Select all

    <Window type="Cube/ImageButton" name="wwwPicSearch">
      <Property name="InheritsAlpha"              value="False" />
       <Property name="MousePassThroughEnabled"    value="False" />
      <Property name="Position"            value="{{0.0,0},{0.9,0}" />
      <Property name="Size"                value="{{0.1,0},{0.1,0}}" />
      <Property name="Image"                      value="Buttons/wwwPicSearch" />
      <Property name="AlwaysOnTop"                value="True" />
      <Property name="Alpha"                        value="0.7" />
      <Property name="TooltipText"                    value="Album Art search..." />
    </Window>


and the ImageButton Look'n'Feel:

Code: Select all

  <!--

    ***************************************************
        CubeLook/ImageButton
    ***************************************************
    -->
 
  <WidgetLook name="CubeLook/ImageButton">
    <PropertyDefinition name="Image" initialValue="" redrawOnWrite="true" />
    <PropertyDefinition name="VertImageFormatting" initialValue="Stretched" redrawOnWrite="true" />
    <PropertyDefinition name="HorzImageFormatting" initialValue="Stretched" redrawOnWrite="true" />
    <PropertyDefinition name="hoverColours" initialValue= "tl:FFAFAFAF tr:FFAFAFAF bl:FFFFFFFF br:FFFFFFFF" redrawOnWrite="true" />
    <PropertyDefinition name="pushedColours" initialValue="tl:FFAFAFAF tr:FFAFAFAF bl:FFAFAFAF br:FFAFAFAF" redrawOnWrite="true" />
    <PropertyDefinition name="pushedoffColours" initialValue="tl:FFAFAFAF tr:FFAFAFAF bl:FF7F7F7F br:FFFFFFFF" redrawOnWrite="true" />
    <PropertyDefinition name="disabledColours" initialValue="tl:803F3F3F tr:803F3F3F bl:803F3F3F br:803F3F3F" redrawOnWrite="true" />
    <Property name="WantsMultiClickEvents" value="False" />
    <ImagerySection name="normal">
      <ImageryComponent>
        <Area>
          <Dim type="LeftEdge" >
            <AbsoluteDim value="0" />
          </Dim>
          <Dim type="TopEdge" >
            <AbsoluteDim value="0" />
          </Dim>
          <Dim type="Width" >
            <UnifiedDim scale="1" type="Width" />
          </Dim>
          <Dim type="Height" >
            <UnifiedDim scale="1" type="Height" />
          </Dim>
        </Area>
        <ImageProperty name="Image" />
        <VertFormatProperty name="VertImageFormatting" />
        <HorzFormatProperty name="HorzImageFormatting" />
      </ImageryComponent>
    </ImagerySection>
    <StateImagery name="Normal">
      <Layer>
        <Section section="normal" />
      </Layer>
    </StateImagery>
    <StateImagery name="Hover">
      <Layer>
        <Section section="normal">
          <ColourRectProperty name="hoverColours" />
        </Section>
      </Layer>
    </StateImagery>
    <StateImagery name="Pushed">
      <Layer>
        <Section section="normal">
          <ColourRectProperty name="pushedColours" />
        </Section>
      </Layer>
    </StateImagery>
    <StateImagery name="PushedOff">
      <Layer>
        <Section section="normal">
          <ColourRectProperty name="pushedoffColours" />
        </Section>
      </Layer>
    </StateImagery>
    <StateImagery name="Disabled">
      <Layer>
        <Section section="normal">
          <ColourRectProperty name="disabledColours" />
        </Section>
      </Layer>
    </StateImagery>
  </WidgetLook>


So, where's the shadow gone, or "who's mucking with my alpha"? Me, Ogre or CEGUI?

[Ident]

We talked about this in IRC already but I will reply here again with some new info: What I need to know is if for example Vanilla-Skin widgets which have elements like boarders with transparencies are also displayed wrong.

If yes this is definitely not a CEGUI bug ( which i consider unlikely anyways) and must have to do with the settings of the Ogre RenderTarget. CEGUI creates Ogre RenderTargets as well. These should work. Do you use the OgreRenderer function for creating RenderTextureTargets or do you create it customly?

[mmixLinus]

Thanks for your help



This is the result of my VanillaSkin test. On the left you see two, slightly overlapping RadioButtons, rendered as part of a regular CEGUI window in the system default GUIContext (the "main context"). (Sorry the background is so dark. However you can still see the relevant details.) On the right you can see the previously mentioned background texture with the superimposed RTT/GUIContext2 containing a window with two similar radio buttons. The right texture (RTT) has a full resolution of 512x512, and it is covering roughly 1k x 1k, so I suppose the smoothing you see there is the upscaling antialiasing. The buttons take up all area where alpha != 0.

But the smoothing to the left seems to be the alpha channel blending things nicely. The buttons look ROUND.

Yes, I use the OgreRenderer to create the RTT. This is the code in all its glory

Code: Select all

CEGUI::GUIContext* CApplication::createGUIContext(const Cube::String& sContextName, int iRootWidth, int iRootHeight)
{
   USES_CONVERSION;
   CEGUI::System& ceguiSystem = CEGUI::System::getSingleton();

   CEGUI::OgreRenderer* ogRrr   = static_cast<CEGUI::OgreRenderer *>(ceguiSystem.getRenderer());
   CEGUI::Sizef size(static_cast<float>(iRootWidth), static_cast<float>(iRootHeight));
   
   // We create a CEGUI texture target and create a GUIContext that will use it.
   CEGUI::TextureTarget* renderTextureTarget = ogRrr->createTextureTarget();
   renderTextureTarget->declareRenderSize(size);
   CEGUI::GUIContext *pNewGUIContext = &ceguiSystem.createGUIContext(static_cast<CEGUI::RenderTarget&>(*renderTextureTarget) );

   CEGUI::Texture& ceTex = renderTextureTarget->getTexture();
   Ogre::TexturePtr ogTex = static_cast<CEGUI::OgreTexture&>(ceTex).getOgreTexture();
   //pNewGUIContext->setRootWindow(new CEGUI::Window("DefaultWindow", "root"));

   CUBE_OUT(_T("GUIContext '%S' created. It has Ogre Texture '%s'"), STRING_CUBE_TO_ASCII(sContextName), STRING_OGRE_TO_TCHAR(ogTex->getName()));

   m_RTTContexts[sContextName] = pNewGUIContext;
   m_RTTOgreTextures[sContextName] = ogTex;

   return pNewGUIContext;
}


Not sure my image is clear enough, let me know otherwise.

[Ident]

It is very clear. Like said in my last post: Check the CEGUIOgreRenderer code to see how we create the RenderTextureTarget and compare it to how you do it to determine the difference regarding alpha settings.

[mmixLinus]

Sorry, I don't get it Am I not actually letting CEGUI create the RTT and be in control of the actual rendering?

Code: Select all

CEGUI::TextureTarget* renderTextureTarget = ogRrr->createTextureTarget();

[Ident]

You are correct on this. But CEGUI::Window uses auto-rendering surfaces optionally (for example FrameWindows have it switched "on" by default) and it works for them correctly. And as far as i know this is also true for OGRERenderer. So i do not know what could cause this in your case if it works in CEGUI internally. I quickly looked into the CEGUI code and I dont see any changes to alpha being made for auto rendering windows. I suggest you to look on your own too maybe you notice some alpha changes being done somewhere regarding that. Also you can try to use a FrameWindow in your non-RTT context and switch the auto-rendering surface on and see if alpha works inside it. If it does I would 100% bet on that somewhere in CEGUI the alpha settings are changed after the rendertarget is created. Otherwise it might be something else,..

[mmixLinus]

Nice clarification. Thanks!
I'll definitely look closer into this. I must admit, my project is getting big and unwieldy, so yes, things could be happening "somewhere else"

[Ident]

I didn't say it is something you are doing extra, what i was saying is that CEGUI might do something extra inside the CEGUIOgreRenderer after the creation of the RTT, and that I might have not found that.

I suggest you to try out what I told you with the auto-rendersurface and tell us the result.

[mmixLinus]

Ok, made a small discovery.

The RTT Window being rendered (containing the semitransparent buttons) is a FrameWindow with Alpha=0.0 (the buttons have InheritAlpha=False). If I change the FrameWindow Alpha to <> 0.0 I see the buttons blended with the FrameWindow background according to the buttons' alpha channels. Nice! So the alpha works, but I can't use this because .. the background is visible Changing to a DefaultWindow instead, the original problem persists.

I'm beginning to see a pattern: Alpha information isn't (and can't be?) conveyed over to my final Ogre Texture. CEGUI wants to render my buttons on top of something, and when doing so forms an image without alpha. For example, rendering the buttons in a DefaultWindow or an Alpha = 0.0 FrameWindow, blends the transparent bits (the button shadows) with BLACK, resulting in BLACK (alpha=1). Just a guess.

[Ident]

I read your text twice but i cant make any conclusion from it. Maybe you can figure what causes the strange behaviour and then I can figure for you how to fix it. Did you however try what I told you regarding a FrameWindow and autorendering surface (OUTSIDE RTT)?

[mmixLinus]

There seems to be NO effect setting AutoRenderingSurface to True or False in a non-RTT FrameWindow. The RadioButtons look nice/correct.

[Ident]

This means that the rendertarget you retrieve should also be fully functional.

tus->setColourOperationEx(Ogre::LBX_BLEND_TEXTURE_ALPHA);
tus->setAlphaOperation(Ogre::LBX_BLEND_TEXTURE_ALPHA);

I have no idea what this is about

[mmixLinus]

This means that the rendertarget you retrieve should also be fully functional.

tus->setColourOperationEx(Ogre::LBX_BLEND_TEXTURE_ALPHA);
tus->setAlphaOperation(Ogre::LBX_BLEND_TEXTURE_ALPHA);

I have no idea what this is about

Except that is probably where my problem is. I dumped the alpha channel of the texture that CEGUI renders to, and looking at the bottom left corner of the 512x512 bitmap, where the World Button is (see top image), I get this:

Code: Select all

Line 487 000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000000000
Line 488 000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000000000
Line 489 000000fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff30000000000000000
Line 490 000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffa90000000000000000
Line 491 000000f4ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff560000000000000000
Line 492 00000093ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000000000000000
Line 493 00000004fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc000000000000000000
Line 494 00000408fcffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff34040000000000000000
Line 495 0004081830ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1b0c0400000000000000
Line 496 0008142c50feffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffad34180800000000000000
Line 497 040c22446dadffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff754a261000000000000000
Line 498 0414345c86abfcffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffbc8e643a1800000000000000
Line 499 081b3e6d94b7cffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff6bd9c75471e00000000000000
Line 500 081e47759abcd3e9ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffd7c1a27c502600000000000000
Line 501 081e44729abcd2e2f1ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe4d7bfa2794d2200000000000000
Line 502 081b3a6791b4cddee8f8ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffebe0d1b9986f441e00000000000000
Line 503 0414305680a5c2d7e3eaf3ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffebe5d9c7ab875f371800000000000000
Line 504 040c1b3a648cadc7d9e4eaf1fffffffffffffffffffffffffffffffffffffffffffffffffffffffffeebe6dccbb2936944220c00000000000000
Line 505 0004102644698eafc7d7e2e9ecfefffffffffffffffffffffffffffffffffffffffffffffffffff4e9e4dacab494724d2c140800000000000000
Line 506 0000081426446486a6bed1dce4e8f0fffffffffffffffffffffffffffffffffffffffffffffce9e4ded2c2ab8c694a2c14080400000000000000
Line 507 0000040810223a597797afc4d1dae2e6e9fffffffffffffffffffffffffffffffffffffce6e3dcd3c7b49a7e5f412a140c040000000000000000
Line 508 00000000040c1b2c445f7993a9bcc9d2d9dee2f4fefffffffffffffffffffffffbecdfdad3cabead977e644a301b0c0804000000000000000000
Line 509 00000000000408101e3041596f879aa9b7c1c9d1d5d7d9dcdddddddddcdad7d6d1cac4b9ad9c8c755f47341e1408080000000000000000000000
Line 510 000000000000040408101b26344756697987949fa6afb2b7b9babab9b7b4afa8a1978a7c6d5c4a372a1b10080404000000000000000000000000
Line 511 00000000000000000004080c101b26303a47545c646f75797c7e7e7c79756f675f544a3e30261b140c0808000000000000000000000000000000


This shows clearly that the totally black button shadow, is in fact represented by a fading alpha value, so the background texture (orange) should be visible through the fading alpha. But it isn't. So I am blending incorrectly. And CEGUI is rendering correctly.

'Nuff said

[Ident]

SO did you solve it and if yes how?

[mmixLinus]

Not solved. Working on it.