[Bug][Linux][HUD] Hang when using UI

We’ve moved over to the paradox forums. Please come visit us there to discuss:
You can still read the collective wisdom - and lolz - of the community here, but posting is no longer possible.

Home Forums Update v1.5 – Open Beta Linux Specific [Bug][Linux][HUD] Hang when using UI

Tagged: , ,

This topic contains 18 replies, has 2 voices, and was last updated by  tezeriusz 6 years, 8 months ago.

Viewing 19 posts - 1 through 19 (of 19 total)
  • Author
    Posts
  • #170604

    tezeriusz
    Member

    Hi

    To the point.

    build 15434
    Linux Manjoro 64b (I will provide more info if needed)

    When I using game from time to time my UI just frees and then game need to be killed. I can’t reproduce this kind of error (usually when I load saved game this kind of crash is postponed but it happen eventually). But looking at log file will give you more info about what is going on.

    gl guys

    Attachments:
    1. log0322_1308.txt
    #171178

    Lascha
    Moderator

    Thanks for the report. Does this also happen when you play without Debug Mode?

    #171217

    tezeriusz
    Member

    As far as I remember yes it does happen. However (for now) new build 15513 is working fine.

    #172770

    tezeriusz
    Member

    This happen again in 15566 not in debug mode. If you want next time I will try to attache with debuger and generate core for it?

    Cheers

    #173367

    tezeriusz
    Member

    Still present in new build

    Attachments:
    1. log0327_1912.txt
    #173373

    tezeriusz
    Member

    Ok this time I duped core and if you want me to do something special with it say something and for now look at attached pstacks.txt

    Cheers

    Attachments:
    1. pstacks.txt
    #174138

    Lascha
    Moderator

    Thanks! Looks like a deadlock. I’ll check it out.

    #174139

    Lascha
    Moderator

    Btw how did you get this core output? It’s a pretty useful feature!

    #174153

    tezeriusz
    Member

    Hi
    I got this core dump that way:

    (gdb) attach <pid>
    (gdb) generate-core-file <optional-filename>
    (gdb) detach

    There is more http://en.wikibooks.org/wiki/Linux_Applications_Debugging_Techniques/Core_files

    And then gdb –batch –quiet -ex ‘thread apply all bt full’ -ex ‘quit’ {progName} {core} will output pstack like output.

    Cheers

    #174185

    Lascha
    Moderator

    Ok, looks like it’s not a deadlock after all. For some reason your computer failed to create a new thread. Looking at the possible documented reasons for why this could happen are copied below.

    EAGAIN Insufficient resources to create another thread.

    Could be related to our of memory. Creating a new thread probably reserves some memory for the stack, and this could fail because of out of memory. This is probably the case if it always happens after a while of playing. Note: Recent memory leak fix in other thread.

    EAGAIN A system-imposed limit on the number of threads was
            encountered.  There are a number of limits that may trigger
            this error: the RLIMIT_NPROC soft resource limit (set via
            setrlimit(2)), which limits the number of processes and
            threads for a real user ID, was reached; the kernel's system-
            wide limit on the number of processes and threads,
            /proc/sys/kernel/threads-max, was reached (see proc(5)); or
            the maximum number of PIDs, /proc/sys/kernel/pid_max, was
            reached (see proc(5)).

    This could be worth double-checking.

    #174196

    tezeriusz
    Member

    Kernel threads limit
    $ cat /proc/sys/kernel/threads-max
    256708

    pid threads limit
    $ cat /proc/sys/kernel/pid_max
    32768

    Maybe this have something to do with shell(bash) or maybe with starting script? I didn’t look at it yet.

    Any other ideas what can be wrong?

    #174204

    tezeriusz
    Member

    Maybe you set somewhere in code this limit with setrlimit() ?

    more info related to bash limits:
    $ ulimit -a
    core file size (blocks, -c) 0
    data seg size (kbytes, -d) unlimited
    scheduling priority (-e) 30
    file size (blocks, -f) unlimited
    pending signals (-i) 128354
    max locked memory (kbytes, -l) unlimited
    max memory size (kbytes, -m) unlimited
    open files (-n) 1024
    pipe size (512 bytes, -p) 8
    POSIX message queues (bytes, -q) 819200
    real-time priority (-r) 99
    stack size (kbytes, -s) 8192
    cpu time (seconds, -t) unlimited
    max user processes (-u) 128354
    virtual memory (kbytes, -v) unlimited
    file locks (-x) unlimited

    gl

    #175768

    Lascha
    Moderator

    We don’t set limit. It’s memory fragmentation issue because there’s no memory available. It’s a 32-bit process so memory is limited to 4GB even if you have 4+GB (unless you limited it yourself).

    There’s simply not enough empty pages left, all allocations become scattered over pages after some time, so when a big allocation happens it and it cannot find an empty page then you can be out of memory even with 2GB usage. There is no easy fix, but right now I’m looking into enabling our own pools so memory fragmentation can be avoided. That’s the only possible fix for this issue.

    #175777

    tezeriusz
    Member

    Yeah as dev i know what memory fragmentation is but this is usually problem when you want to manage memory on you own (mem pools) and when you use normal malloc system can find way to reallocate memory and give you big chunk if possible. So I’m little confused right now.

    #175912

    Lascha
    Moderator

    The OS kernel also has memory fragmentation but it generally handles this very well so you don’t really notice much of it. But if AoW3 is leaking then even the OS will suffer from memory fragmentation. Imagine small leaks inbetween the big chunks then eventually the big chunks will be filled with many small allocations in between, and there is no more big chunk left. So instead of realloc it will just use a new block where big chunk is available. It leaks again, etc. And eventually there’s no more big chunk available. The problem is that there are small leaks so if there weren’t any, then this would not be such big of a problem anymore.

    But I’m making progress with getting custom memory heaps to work on Linux. Example: Tactical combat has its own heap. If we leak there, it’s OK, because at the end of the tactical combat the heap is destroyed and all leaks are destroyed with it. I hope this makes sense… I’m bad at explaining stuff πŸ™

    #175922

    tezeriusz
    Member

    Yes It make sense. I’m experienced in programming under linux in c++(almost 10 years) so it is clear for me.

    BTW. on Linux there is possibility to use something like LD_PRELOAD and many apps use that as a way to use its own malloc, socket or whatever in entire process without changing a bit in code. If you have some spare time πŸ™‚ look at dmalloc/jwmalloc maybe this utils will solve some problems or speed up development.

    #175935

    Lascha
    Moderator

    We use this one: http://g.oswego.edu/dl/html/malloc.html

    But all allocators will have the same problem. I’ll try to explain the exact problem:

    On Mac & Linux when you overload the ‘new’/’delete’ operator globally, then any other external .so file will also use the overloaded new operator. We overload those operators and redirect it to our malloc. This means that all external middleware hooks in on our memory management, but we don’t want that because of various reasons.

    The solution is to prevent external libraries from calling the overloaded new/delete. To do this we have to remove the new/delete symbols from the final binary. On Mac there was something called -unexport_file_list. But on linux I can’t find anything yet. I tried with visibility attributes, but that doesn’t work. Still looking for a solution, I’ll have it soon πŸ™‚

    #175943

    Lascha
    Moderator

    Found the solution, yay πŸ˜‰ So hopefully the next patch will be more memory friendly for you πŸ™‚

    objcopy -N

    http://linux.die.net/man/1/objcopy

    #176090

    tezeriusz
    Member

    Good job πŸ™‚
    thx for sharing

Viewing 19 posts - 1 through 19 (of 19 total)

You must be logged in to reply to this topic.