[CLOSED] Performance problem: Font loading & resizing

[mmixLinus]

This may be an unusual use-case (or at least affect rather few users) but affects us quite a bit.

One of my goals is to have awesome Unicode coverage. For quite some time, MMiX.Me has been using the Unicode true type font DejaVuSans.ttf, roughly 3000 glyphs. However, it doesn't cover CJK - Chinese, Japanese, Korean. So in a bold step towards the awesomeness we now load Code2000.ttf which covers er, umm, "everything". That's when I noted: It takes quite some time to load 53000 glyphs, and even longer to stretch them (ie., when you resize the main window and the font's AutoScaled flag is set to true). [I'm using CEGUI 0.7.4 and FreeType 2.3.11(Ogre3D 1.7.3)]

It just so happens that we were loading three sizes of the font originally, and this is (as far as I can see) unnecessary - we will manage with one. And as it turns out, we won't be stretching any fonts either ( AutoScaled="false"). But I'm still left with ONE big font load. One load takes my computer almost 5 seconds, and I estimated the font stretch to be more than 10 seconds (so resizing the main window took more than 30 seconds with NO action). Again, the stretching issue is obsolete, for me.

My computer is a 2.8 GHz "Lynnfield" (Quad Core i5), but others will be slower, of that I'm sure
(The CEGUI.log snippet looks funny, but I just modified the DejaVuSans-N.font files to point to code2000.ttf. Note the times)

Code: Select all

26/10/2011 20:57:48 (Std)    Started creation of Font from XML specification:
26/10/2011 20:57:48 (Std)    ---- CEGUI font name: DejaVuSans-10
26/10/2011 20:57:48 (Std)    ----       Font type: FreeType
26/10/2011 20:57:48 (Std)    ----     Source file: code2000.ttf in resource group: (Default)
26/10/2011 20:57:48 (Std)    ---- Real point size: 10
26/10/2011 20:57:52 (Info)    Successfully loaded 53068 glyphs
26/10/2011 20:57:52 (Info)    Finished creation of Font 'DejaVuSans-10' via XML file. (0B0FB158)
26/10/2011 20:57:52 (Std)    Started creation of Font from XML specification:
26/10/2011 20:57:52 (Std)    ---- CEGUI font name: DejaVuSans-8
26/10/2011 20:57:52 (Std)    ----       Font type: FreeType
26/10/2011 20:57:52 (Std)    ----     Source file: code2000.ttf in resource group: (Default)
26/10/2011 20:57:52 (Std)    ---- Real point size: 8
26/10/2011 20:57:56 (Info)    Successfully loaded 53068 glyphs
26/10/2011 20:57:56 (Info)    Finished creation of Font 'DejaVuSans-8' via XML file. (0B0F79A0)
26/10/2011 20:57:56 (Std)    Started creation of Font from XML specification:
26/10/2011 20:57:56 (Std)    ---- CEGUI font name: DejaVuSans-14
26/10/2011 20:57:56 (Std)    ----       Font type: FreeType
26/10/2011 20:57:56 (Std)    ----     Source file: code2000.ttf in resource group: (Default)
26/10/2011 20:57:56 (Std)    ---- Real point size: 14
26/10/2011 20:58:01 (Info)    Successfully loaded 53068 glyphs
26/10/2011 20:58:01 (Info)    Finished creation of Font 'DejaVuSans-14' via XML file. (0B0D8520)


Do you think CEGUI::FreeTypeFont::updateFont() could be sped up? I mean, would it even be possible with regards to the FreeType library API, and would the CEGUI Team consider working with this?

Thanks/Linus

[CrazyEddie]

Hi,

Yeah this is something of a known issue, a PITA, and something that we would like to address for sure. The main issue being that, using your example, 53000 glyphs is a lot! Comparing this to what happens with ASCII based text, which would typically be using much less than 100 glyphs.

The issue is exacerbated since there is no longer[*] any direct user control over precisely what glyphs get rendered - they are rendered in 'pages' of glyphs on demand (so the first time a glyph is used in drawn text, a page of glyphs is generated containing the glyph used and some number of glyphs surrounding the used glyph). This arrangement works fine for a lot of cases, but falls down horribly when dealing with CJK and perhaps other languages, as you have found.

Can this process be sped up? I think there is always improvement to be had, however, I doubt that the existing mechanism can be optimised to such a degree that these delays are eliminated - I rather think that more substantial reworking is required. There is a set of changes on mantis that purport to address this issue, however due to the way in which those changes are presented, we are unable to use them in the current form. However, if that work were to be either i)improved / reworked and resubmitted or ii) had a lot of community support (i.e in that case I myself might then look at reworking it), then it could go in. If you're brave, perhaps you can check it out and see if it's any good? (http://www.cegui.org.uk/mantis/view.php?id=574)

So, in conclusion: Yeah, known issue that we're not happy about. And we're willing to try and work with people - as far as possible - to address it.

CE.

[*] "No longer" because originally the user had this level of control, but that was unfortunately removed as part of a contributed font rewrite that went into the 0.5.x releases.

[Kulik]

I also think that the current way CEGUI does fonts exposes too much of the pain to the library user (as I've said numerous times but lets say it again ). IMO user should specify several things, give enough info to CEGUI for it to make good performance/caching decisions but in the end you should specify one font for one font, not 3 fonts for 3 sizes of one font. We could integrate some sort of a fast rectangle packing algorithm (and since we can make a lot of assumptions about symbols this wouldn't be hard I think) and do a proper on-demand symbol rendering. The problem I see is how to purge the cache of unused symbols. Perhaps some reference counting or such should be used for this. If the symbols are rendered in a different thread this wouldn't even cause any sudden performance hits (we would just not render the symbol and only advance the proper offset until it's loaded).

This would open up a lot more formatting/polishing options without any added maintenance pain. All of this would require someone who is interested in typography, kerning and all of that and is experienced in C++ and can design a good API.

A decent GSoC project idea

[Hanmac]

my thinking about font stuff is this:

have a Font class that are connected to multiple font files:(normal,bold,italic and bold italic)

then have a class FontInstance which point to an Font object.
this instance object can hold multible properties like size, bold and italic (for the right font file),stoke,underbar, (sup and sub* ok this chould be otherwere)

with this, the sourcefiles does not need to be loaded multiple times ...

[mmixLinus]

@CrazyEddie Well, had a look at the Font patch, and yes I can see its shortcomings... I suppose I could clean it up a bit, and try to verify it, albeit not today or tomorrow... I think the most difficult part is writing proper method comments.. because that'll somehow mean that I have to understand what's going on I diffed it against 0.7.4 (that I'm running) and pre0.8 tip (Hg) and my guess it's based on 0.7.5. If I touch it, it'll be towards latest in mercurial. Plenty of things on the proverbial plate already, you know In a while maybe?

@Kulik I actually think unloading is unnecessary. At least automatic unloading of unused symbols/pages. Why would you need to do that? Because you're randomly using characters in many different fonts or from many different pages, so you're eating up memory? I think that by far the most common use-case is to use TWO areas of symbols, your own locale + "basic latin", and nothing else. Checkout Code2000 to see it's arrangement. One "problem" with code2000 might be "Hangul Syllables" for Korea which is 11000 glyphs, if there IS a problem. I've actually used MetaFont+TeX a bit, but no, GSoC seems fine

I'll have a go at patching my 0.7.4 to see what happens.

[mmixLinus]

No.

I can't do it. Sorry

So the CEGUIFreeTypeFont.cpp diff was much larger and more complex than I thought. Just trying to understand what the patch is based on isn't easy. It is 0.7.1 or earlier, but heavily patched (I based this in part on the "Succsessfully loaded %d glyphs" string typo, which is in the patch, but not in 0.7.2).

It also doesn't compile in 0.7.4 due to missing pixel format CEGUI::Texture::PF_A8L8 (line 281). There are plenty of float <--> int mismatches too, indicating an old CEGUI.

I can only suggest trying to get the original author to rebase on 0.8. In other words, convince the author to prepare for merging to 0.8 ASAP

[CrazyEddie]

Firstly, thanks so much for giving this a try. From what you say it seems like their version contains a lot of other changes not included in the 'patch', especially that PF_A8L8 pixel format which is something we've never had (even now, after adding some additional formats to the PixelFormat enum we do not have that one!).

Kulik emailed the originator of the patch a while ago highlighting some of the issues we have with it and suggesting it be resubmitted in the proper form, but there has been no response as of this writing. So I guess the chances of an 0.8 compatible update being submitted is slim to none

I can only say that perhaps I might look at it and see what I can make of it. Of course, there are no promises, given that the two people who already looked at it (yourself and Kulik) have had 'bad' things to say

CE.

[mmixLinus]

Thanks CE. Again, sorry I couldn't do much with it.

My temp solution for the time being will be to load DejaVu as default,
and Code2000 optionally for those who need it (and who will be forced to take
the penalty of a few seconds extra startup time. Not a huge issue, imho)

/Linus

PS Closing issue.

[Montred]

Reviving this discussions since it seems to have the most up to date information about current state of Unicode (please correct me if I'm wrong).

I'm interested in tackling this. Basically, I would like to be able to render any character, but huge load times aren't cool.

Wouldn't it be easier to just rely on the Operating System to render text? It seems to me that it's a waste to load those huge fonts in memory when Windows/MacOsX/Linux already have them. CEGUI could use the same approach (prerender character in pages) but for any character that isn't available, it could ask the operating system for a render.

This isn't portable, but we could tackle Windows first. I know Windows may not be a popular platform here but I'm doing a game that will be a Windows only application initially, so I'd be willing to work on a solution for Windows if you guys would accept such a patch.

Another solution would be to just let the user handle font rendering. Can we do that? I don't know how slow would it be to just ask windows to render every character on real time, but we could use the current paging approach. If my understanding is correct, CEGUI currently pre-renders ranges of unicode characters to individual textures. We could have a callback for that.

bool PrerenderFont(
char *image_buffer,
int width, int height, int bitdepth,
int character_width, int character_height,
int unicode_start, int unicode_height);

Cegui could call an user defined function with that signature for example, then I could just use GDI to render the characters in Windows. And when the time comes to port to Mac I could use whatever the Mac equivalent is, etc...

[Montred]

I had another idea to prevent the 5 seconds load time. Sorry for double post.

What if one could just take the huge-ass full unicode font, let's call it unicode.ttf into, say 256 smaller files (remember full unicode range is 2^16) with a maximum of, let's say, 256 characters each, maybe 1024 in 64 files would be a better number, but w/e.

So you would have:

unicode_000.ttf, unicode_001.ttf, etc, up to unicode_255.ttf.

I bet there's a tool for this somewhere, but if not I volunteer to code it.

Then we could change CEGUI to accept the following parameters in font definitions:

<Font Name="unicode" Filename="unicode.ttf" MultiPart="Yes" PartSize="256" Type="FreeType" Size="10" NativeHorzRes="800" NativeVertRes="600" AutoScaled="true"/>

The multipart and PartSize are the new bit, it tells CEGUI that the font is split and that each block has 256 characters. So that it can figure out which files needs to load without checking an index. So now, every time that CEGUI needs new characters to prerender it would just load the corresponding ttf's and it'd be very fast.

So with this minimum change we can have every character ever without the 5 seconds slowdown for parsing the font that OP was mentioning.

EDIT: I may have found an even easier way to solve this. It seems like you can tell FreeType to only load certain glyphs, so maybe we can just load the glyphs we need on demand.

[CrazyEddie]

Hi,

Thanks for the input on this issue, it's still important to us to get this addressed. I'll deal with a couple of things you mention before saying some words on the actual topic

Re: Windows platform
MS Windows support is important to us. The majority of our users are on the platform and - while it's not the primary development platform of any of the active developers on the project - we understand the situation here and I would say most of the things that get said, especially by myself (such as "I want to drop Windows support"), can be taken fairly lightly

Re: Patches / modifications that are "Only for WIndows, to begin with"
This is slightly tougher to deal with in a way that i) does not upset contributors and ii) makes us developers happy. The submission guidelines basically say that patches/submissions should cover all the platforms where possible and where they do not, they will be rejected. Now, obviously some platform has to come first and we understand that. The issue the guideline is meant to cover is one where if we accept such modifications, CEGUI devs then have to implement code and fixes on the remaining plaforms such that we end do not end up with broken or inferior code on any of the platforms for extended periods of time. It is always more important that we stick to that for a first code contribution or - not to put too finer point on it - until we know we can trust the contributor to complete the work for the remaining platforms Note that this is not an outright rejection of all stuff done for one platform - if something is really awesome, of course it will be accepted, and we would happily help in porting unportable modifications to the remaining platforms.

With regards to the specific ideas...

I don't think splitting the TTF is really viable (as in, "most people" would not be happy about doing that) and in any case the majority of the delay is rendering the glyphs to textures. Tthough I appreciate that having massive fonts loaded all the time and maybe multiple times is not good resource usage, though that is a seperate issue to the delay one (of course, a solution could exist to resolve both issues simultaneously).

Having the OS render the text would be nice, although I believe it would be slow as hell. BUT... If someone were to demonstrate otherwise, I would be a happy man. I think overall, some kind of interface to allow better font implementations to be plugged in (like what we have done for the Image system in CEGUI 1.0) would be the kind of direction to head in. There is already some abstraction there, but it's nowhere near enough IMO and is definitely limited because client code can not hook into that abstraction in order to make use of it (i.e you can register your MySuperFont class with CEGUI and have it used). Anyway, I've kind of gone off track a little here.

To restate the situation at the moment: Each time a glyph is used for the first time, a 256 glyph 'page' is generated that contains the needed glyph. This is fine for western languages and some other languages also, but for asian languages and such, where you end up needing loads of 'pages', that is where the delay comes in. The original font implementation did not suffer this issue, because it was possible to pre-declare required glyphs or ranges of glyphs, although that was not an ideal situation either

Anyway, to conclude: I still don't know what the solution should be. I don't think it should require too much effort on the part of the user (splitting TTF files), but anythng else would certainly be considered. I'm happy to have further input and discussion before anyone starts any work, though

[Montred]

Thanks for the comments. It is indeed a challenging issue.

However, the particular issue that preoccupies me now is the loading time of the font itself (at least that's what I think it is).

Let me explain, right now I have a very agile application that starts up in 0.8 seconds. If I do:

CEGUI::FontManager::getSingleton().create( "unicode.font" );

With unicode.font pointing to a 5mb ttf with 50,000 glyphs, the application now takes 2.8 seconds to start. You can say 2 seconds is not a big deal, with games usually taking 20 seconds to start, but right now my application starts really fast and if I can keep it that way I'd like to. And I think we can, easily, so why not.

So I'm assuming that since I'm not using the font in any widget yet, I'm just issuing that call, that those 2 seconds are mainly occupied in calls to FT_Load_Char() in CEGUIFreeTypeFont.cpp, and not in glyph pre-rendering, am I right? This is the same function that OP was asking to be sped up, so I'm not alone in this. I'll profile to find out of this is the case tomorrow.

If so, what do you say, do you think this is worth tackling? Maybe we could make it so that FT_Load_Char() is only called for the glyphs that are used. It could be an option in the font definition.

<Font Name="unicode" Filename="unicode.ttf" DeferredLoading="Yes" ...

[CrazyEddie]

Yes you are correct that the pre-load of glyph information will take time and in the scenario you describe this is almost certainly where the time is going.

To be completely honest, I don't know why that pre-load step is done for all glyphs in the font and am not sure whether it is totally required or not (the current font code is already a contributed rewrite of my original font class) - I will need to study it a bit to refresh myself on exactly what is happening and why If the pre-load of glyph information is not 100% required for some reason, then i see no reason why a modification / option to have it skip the pre-load should not go in.

[Montred]

So I did some testing and it seems that the reason that every glyph is being loaded is merely to find out the horizontal advance of every glyph.

As an experiment I changed this:

Code: Select all

        if (FT_Load_Char(d_fontFace, codepoint,
                         FT_LOAD_DEFAULT | FT_LOAD_FORCE_AUTOHINT))
            continue; // glyph error

        float adv = d_fontFace->glyph->metrics.horiAdvance * float(FT_POS_COEF);

        // create a new empty FontGlyph with given character code
        d_cp_map[codepoint] = FontGlyph(adv);


To this:

Code: Select all

       d_cp_map[codepoint] = FontGlyph(10);


And now the font loads in 20ms instead of 2 seconds, although obviously, every character is now separated by the same distance . So yeah, it's just a matter of doing this on demand. Since FontGlyph is an abstract structure used by all the font implementations it wouldn't be correct to pass it the FT_Face etc to make that calculation on demand, so I guess you may want to do some restructuring to fix this.

Or if you'd like me to help, just tell me how to approach it and I'll send you a patch. It's just a matter of how to do it in a nice way that you like.

[CrazyEddie]

Ok, it's cool that this was your issue and that it is possible to get around it. I will have to look at the code closely to see what I think the best way of going about that would be; I'll get back to you in a couple of days about that