Hit test decoupled from alpha transparency

[Suthiro]

Hello,
I implemented an AlphaHitWindow widget using http://cegui.org.uk/wiki/Hit_testing_of_non_rectangular_windows_with_CEGUI_0.7.x for 0.8.7 with minor changes to satisfy new function names.
While this class is a great and simple way to achieve non-rectangular windows, it has significant drawback: transparency and hit test is the same. To avoid this, I added a custom

Code: Select all

<PropertyDefinition name="HitTestRegion" type="Image"/>
to <WidgetLook name="TaharezLook/AlphaHitButton">.

This property is designated to keep hit-test information.
But I don't know a way to map it to a buffer except to render it once again. I think I miss something, because there is no need to "render" it - just copy it from

Code: Select all

CEGUI::Image* imag=getProperty<CEGUI::Image*>("HitTestRegion");

to appropriate buffer. There is d_texture, d_area and other d_*s in protected area of CEGUI::Image class that give me the idea of straightforward copying. Moreover, I don't like the idea to render something in the CEGUI::RenderingSurface::EventRenderQueueEnded callback.

Should I make an inherited from CEGUI::Image wrapper class to expose d_*s mentioned above and use it to construct a buffer?
Should I render to texture (in Event callback or elsewhere)?
Is there another (sane) way to achieve alpha transparency decomposed from hit test?

---- Version: 0.8.7 (Build: Feb 3 2019 Debug Microsoft Windows MSVC++ Unknown MSVC++ version 64 bit) ----
---- Renderer module is: CEGUI::OpenGLRenderer - Official OpenGL based 2nd generation renderer module. TextureTarget support enabled via FBO extension. ----
---- XML Parser module is: CEGUI::ExpatParser - Official expat based parser module for CEGUI ----
---- Image Codec module is: SILLYImageCodec - Official SILLY based image codec ----
---- Scripting module is: None ----

If this is important, I use glfw-3.2.1 for OpenGL. MSVC version is Visual Studio Community 2017.

Thank you very much in advance.

[Suthiro]

Hi again!

So I ended with wrapper class for CEGUI::BasicImage.
The code is below:

Code: Select all

//a simple wrapper class to expose protected members.
class OverridenImage : public CEGUI::BasicImage
{
public:
   CEGUI::Texture* getTexture() const
   {
      return d_texture;
   };
   const CEGUI::Rectf& getArea() const
   {
      return d_area;
   }
};

Code: Select all

//----------------------------------------------------------------------------//
bool AlphaHitWindow::renderingEndedHandler(const CEGUI::EventArgs& args)
{
   if (static_cast<const CEGUI::RenderQueueEventArgs&>(args).queueID != CEGUI::RQ_BASE)
      return false;

   // rendering surface needs to exist and needs to be texture backed
   CEGUI::RenderingSurface* const rs = getRenderingSurface();
   if ((nullptr == rs) || !rs->isRenderingWindow())
      return false;

   CEGUI::TextureTarget& tt =
      static_cast<CEGUI::RenderingWindow* const>(rs)->getTextureTarget();

   CEGUI::Texture& texture = tt.getTexture();
   const CEGUI::Sizef tex_sz(texture.getSize());

   const size_t reqd_capacity =
      static_cast<int>(tex_sz.d_width) * static_cast<int>(tex_sz.d_height);

   // see if we need to reallocate buffer:
   if (reqd_capacity > d_hitBufferCapacity)
   {
      delete[] d_hitTestBuffer;
      d_hitTestBuffer = nullptr;
      d_hitBufferCapacity = 0;
   }

   // allocate buffer to hold data if it's not already allocated
   if (nullptr==d_hitTestBuffer)
   {
      d_hitTestBuffer = new CEGUI::uint32[reqd_capacity]{ 0u };
      d_hitBufferCapacity = reqd_capacity;
   }
   d_hitBufferSize = tex_sz;

   //check whether custom property is defined in the WidgetLook/code.
   if (isPropertyPresent("HitTestRegion"))
   {
      //try to obtain Image and cast it to wrapper OverridenImage class. It does nothing except exposing d_texture and d_area protected members.
      auto imag = static_cast<const OverridenImage * const>(getProperty<CEGUI::Image*>("HitTestRegion"));
      //if there is no/Invalid HitTest information given, fall to hit test based on main Image alpha channel.
      if (nullptr != imag)
      {
         //get the pristine texture pointer
         auto texture = imag->getTexture();
         //check if the texture exists
         if (nullptr != texture)
         {
            //see if the texture or the size of hit test region have changed since the last time.
            //while I feel that comparison based only on pointers is not enough, it is the best for now
            if ((texture != d_fullTexture) || (tex_sz != d_hitTestRegionSize))
            {
               //assign new size and texture
               d_hitTestRegionSize = CEGUI::Sizef(tex_sz);
               d_fullTexture = texture;

               //get size of pristine texture
               const CEGUI::Sizef fullTextureSize(texture->getSize());
               //calculate temporary buffer capacity
               const size_t fullTextureRequiredCapacity =
                  static_cast<int>(fullTextureSize.d_width) * static_cast<int>(fullTextureSize.d_height);
               //allocate temporary buffer
               auto d_textureBuffer = new CEGUI::uint32[fullTextureRequiredCapacity]{ 0u };
               //store texture into the buffer
               texture->blitToMemory(d_textureBuffer);
               //get pristine dimensions of the HitTestRegion image
               const CEGUI::Rectf area(imag->getArea());
               //calculate scaling coefficients to use later during mapping the hit test region to rendered window
               const CEGUI::Sizef multi
               (
                  tex_sz.d_width / area.getSize().d_width,
                  tex_sz.d_height / area.getSize().d_height
               );
               //iterate through the hit test texture
               for (auto ii = 0; tex_sz.d_height > ii; ++ii)
               {
                  for (auto jj = 0; tex_sz.d_width > jj; ++jj)
                  {
                     //index for hit test region
                     auto ind = int(ii*tex_sz.d_width + jj);
                     //horizontal index in hit-test texture
                     auto indX = std::round(jj / multi.d_width + area.d_min.d_x);
                     //vertical index in hit-test texture
                     auto indY = std::round(ii / multi.d_height + area.d_min.d_y);
                     //assign hit test buffer with appropriate texture buffer value
                     d_hitTestBuffer[ind] = d_textureBuffer[int(indY*fullTextureSize.d_width + indX)];
                  }
               }
               //I'm not sure whether it is always false in case of no-rendering
               d_hitBufferInverted = false;
               //delete temporary buffer
               delete[] d_textureBuffer;
            }
            //if the texture is the same and size has not changed, we do not do anything,
            //but the event is processed
            return true;
         }
      }
   }
   // save details about what will be in the buffer
   d_hitBufferInverted = tt.isRenderingInverted();
   // grab a copy of the data.
   texture.blitToMemory(d_hitTestBuffer);

   return true;
}

I wonder why there is no getTexture() and getArea() methods in BasicImage? Is there a reason for this?
And are there any pitfalls in the solution above?
Thank you.

[Ident]

Hi, that's a very interesting topic you picked up there.

We discussed hit-testing for widgets some years ago. The guide you referenced was one that the great CrazyEddie himself wrote back in the days, when I asked him how this could be done in CEGUI The solution is straight-forward to understand and editable

One solution we were considering is doing an inside-outside polygon test, which would be quite efficient (especially on memory) and you would just need to somehow be able to easily draw these hit polygons in a tool for your widgets. Unfortunately, tooling is what takes time and what makes this approach difficult, although I would consider it the "best" approach due to the efficiency.

The other option is to go by the pixels rendered and make all alpha==0 pixels considering a hit-fail. This is the approach you were trying out - it is CrazyEddie's original and straightforward proposition to solve the problem.

This latter approach works works fine on its own but you already mentioned a big drawback: you can't separate collision-check and visualisation in this case at all! That was one reason we considered a polygonal hit-test approach. But it is also completely valid and maybe even more intuitve to allow making some sort of hit-image that can be laid over an entire widget for sampling if it is hit or not, as this can be modified by artists and designers easily whenever adjustments are needed. Instead of RGBA you probably only need to store a single-channel image, and likely even only one bit per pixel, which would ultimately be really small. The original image can be in another format of course, but it's good to keep the memory sleek.

I agree it would be the right approach to override the BasicImage (0.8.X) or Image (default branch) base class for this and then, most of all, prevent all rendering related stuff from happening for the hit-test image. We don't need the image on the GPU, just available to the CPU on your local memory. And changing the format as described above would definitely make things go faster and more memory-efficiently.

[Ident]

I wonder why there is no getTexture() and getArea() methods in BasicImage? Is there a reason for this?

I think it was just not needed? We generally don't make interfaces if there is no reason to provide them. This hit-test feature is a special case of its own kind.

And are there any pitfalls in the solution above?

I will look at it in detail later on. Conceptually everything seems fine but it will take me a bit to think everything through.

[Suthiro]

...
One solution we were considering is doing an inside-outside polygon test, which would be quite efficient (especially on memory) and you would just need to somehow be able to easily draw these hit polygons in a tool for your widgets. Unfortunately, tooling is what takes time and what makes this approach difficult, although I would consider it the "best" approach due to the efficiency.
...

Very interesting, I was not even considering to use a polygon test. However, nowadays we have a lot of memory, so I'm not sure if it does really worth the effort.

...
But it is also completely valid and maybe even more intuitve to allow making some sort of hit-image that can be laid over an entire widget for sampling if it is hit or not, as this can be modified by artists and designers easily whenever adjustments are needed. Instead of RGBA you probably only need to store a single-channel image, and likely even only one bit per pixel, which would ultimately be really small. The original image can be in another format of course, but it's good to keep the memory sleek.

I decided to keep RGBA format and hit-test images in the same texture as displayed ones. Maybe I switch to 1-bit data later.

I agree it would be the right approach to override the BasicImage (0.8.X) or Image (default branch) base class for this and then, most of all, prevent all rendering related stuff from happening for the hit-test image. We don't need the image on the GPU, just available to the CPU on your local memory. And changing the format as described above would definitely make things go faster and more memory-efficiently.

That's exactly what I did in the code posted above, except I use the texture stored in GPU. While it is not necessary, it makes things simpler. And to keep things fast, I recalculate the hit-test buffer only when it is needed.

I think it was just not needed? We generally don't make interfaces if there is no reason to provide them. This hit-test feature is a special case of its own kind.

Okay, thanks for the info.

I will look at it in detail later on. Conceptually everything seems fine but it will take me a bit to think everything through.

Thank you again!

Quick question: is there a way to display windows in isometric projection? I found info about rendering GUI to texture, but it looks like it will be kind of non-interactable image. I seek interactable variant. Maybe I should start another topic..

Thanks!

[Ident]

Quick question: is there a way to display windows in isometric projection? I found info about rendering GUI to texture, but it looks like it will be kind of non-interactable image. I seek interactable variant. Maybe I should start another topic..

CEGUI is rendered with a projection matrix that is somewhat hard-coded as far as I remember by default. But the renderer should allow you to set your own matrix.

You can rotate all widgets, once you rotate them, naturally the perspective projection will start kicking in. There is a hit-test calculation that should reverse-project your mouse position to tell where on the widget you click but afaik it is broken. Shouldn't be too difficult to fix though. Is this what you were asking about? and yes: new topic would be better for such things

[Ident]

Very interesting, I was not even considering to use a polygon test. However, nowadays we have a lot of memory, so I'm not sure if it does really worth the effort.

You might have a lot more memory, but reading the memory is the biggest bottleneck we have today, the read-speeds didn't increase as much as the size did. There is plenty computational power however. The memory read costs are especially relevant if you want to sequentially read data from gpu memory into local memory first you really don't want to let everything wait for that.