compton not working at all (all windows become invisible) at this date: 2012-09-26

[valr]

Hi,

First of all, thanks a lot for the excellent work you've done on compton.

The current bersion of compton doesn't work (anymore) at this date: 2012-09-26.
It was working on 2012-09-24.

When I run it with these parameters...
compton -Cc -fF -I-.015 -O-.03 -D1 -t-1 -l-3 -r4.2 -o.5 &
...all windows become invisible.

I run openbox under archlinux.

Any idea?
Do you need more info?

Cheers
valr

[richardgv]

I hope you do understand what is a correct value for -I and -O, valr. They are the steps to fade in and fade out. compton limits them to the range of 0.0 - 1.0, so when you specify a negative value, compton would try to fade in all your windows with the speed of 0.0 -- so all your windows vanish. Remove the minus sign in the front of those two values, and set -D (fade delta) to a more sensible value -- 1 means the opacity changes in one step every 1 millisecond, this basically means no fading effects would ever be noticeable unless you have quick eyes and photographic memory -- and you are ready.

compton -Cc -fF -I.015 -O.03 -D10 -t-1 -l-3 -r4.2 -o.5

Oh, of course, I'm talking about the situation after I change the fading mechanism, so we expect that you use the latest version from the master branch. Why a negative value was working beforehand I truly have no idea, it shouldn't have worked.

[valr]

Thanks for your answer, richardgv.

1. Indeed negative value shouldn't have been used. I fully agree with that.
   I remember having copied those value from a website years ago. Yeah, I known, stupid me :)

2. There is still something strange. Whatever value I put for -I (except -I1, which I guess kind of deactivate it) makes all my window disappear.

So, if I use the updated command you've mentionned above (compton -Cc -fF -I.015 -O.03 -D10 -t-1 -l-3 -r4.2 -o.5) all windows completely disappear (I mean permanently).

If I use the example mentionned in the README (compton -cC -fF -I 0.065 -O 0.065 -D 6 -m 0.8 -i 0.6 -e 0.6), all windows disappear.

If I use this: compton -cC -fF -I 0.9 -O 0.9, all windows disappear.

Unless I don't understand it at all (which might be an option, and if so I would be happy you explain me why) there is something wrong, isn't it?

(using very last github version)

Thanks
Cheers

[valr]

On top of my above comment, I have another problem with these compton options:compton -Cc -fF -t-1 -l-3 -r4.2 -D1 -O.03
When an application closes (so, window closed), the background is not refreshed.
If I remove the option -O.03 it works.

[valr]

...and finally, the CPU consumption of the current version is much above the one of the "previous" version.
I use these parameters which works for both versions: compton -Cc -fF -t-1 -l-3 -r4.2 -D1

With the "previous" version, the cpu consumption is not noticeable (using htop).
With the current version, it's around 4% (with peaks to 9%).

Just let me know if you want me to provide more info, create separate bug reports, ...

Cheers
Valr

[valr]

Ok, here are more info.
I've cloned the repo and performed 'git reset --hard HEAD~1' until I find a version without -I & -O problems.

After multiple tries, I reach that step: 'HEAD is now at 0d67243 Feature: Issue #29: Alternative shadow blacklist implementation' ---> I still have problems.

Then I do another 'git reset --hard HEAD~1'

Then I reach that step: 'HEAD is now at 8724101 Bug fix: Detect and mark WM windows as active' .
---> the -I & -O problem is gone.

So, it seems it's the commit 'HEAD is now at 0d67243 Feature: Issue #29: Alternative shadow blacklist implementation' that is the problem.

Cheers
Valr

[richardgv]

Thank you for your efforts to diagnose the issue. All the problems you reported sound really scary... And I really have no luck reproducing even a single one of them. Tried fvwm and Openbox, simply no luck. And chjj, seemingly, didn't notice the problem, either.

The only problem I noticed in your commandline switches is -r (shadow-radius) wants an integer, not a floating point number, but that shouldn't break things so horribly.

I suspect this is a timing-related issue, which are, super-annoying.

I would recommend you to find out the minimal commandline switches that could trigger this issue. -F has no use (it has no effect (master branch) or equals -f (richardgv-dev branch) right now) and it doesn't need to be tested. Just check if it happens with compton -f -I0.03, for example.

Whatever value I put for -I (except -I1, which I guess kind of deactivate it) makes all my window disappear.

Nope. Although -I1 looks abnormal, it's handled in the identical way other -I values are. Well, slightly there's a difference in timing.

And does this happen if you don't use -I or use its default value (-I 0.028)?

So, if I use the updated command you've mentionned above (compton -Cc -fF -I.015 -O.03 -D10 -t-1 -l-3 -r4.2 -o.5) all windows completely disappear (I mean permanently).

Permanently? Hmm, you just mean compton does not render the windows, right?

On top of my above comment, I have another problem with these compton options:compton -Cc -fF -t-1 -l-3 -r4.2 -D1 -O.03
When an application closes (so, window closed), the background is not refreshed.
If I remove the option -O.03 it works.

Which is strange, as the default -O value is 0.03.

...and finally, the CPU consumption of the current version is much above the one of the "previous" version.
I use these parameters which works for both versions: compton -Cc -fF -t-1 -l-3 -r4.2 -D1

I compared the CPU usage of old/new compton with pidstat. I'm using more options, and during the tests I minimize, restore, close, open, and move windows with roughly the same speed. I noticed no significant different in the CPU usage of compton process. (The CPU usage of X process caused by compton's requests couldn't be accurately measured.) My CPU is Intel Core i7 3770K, so it's not surprising if my CPU usage values look lower than yours.

So, it seems it's the commit 'HEAD is now at 0d67243 Feature: Issue #29: Alternative shadow blacklist implementation' that is the problem.

It's so weird because 0d67243 isn't a commit about fading at all. I don't actually see a line of code about fading changed in this commit.

[valr]

-r option indeed changes nothing. I've set it as integer now.

with minimal command: compton -f -I0.03 => problem already occurs with that. I mean no window is rendered (not even bmpanel2)

compton -f -I0.028 doesn't work either
compton -f -I1 works
compton -f works

Let's forget the cpu usage thing now.

I've tried to kill the processes that might (maybe, who knows) interfere with compton (unclutter, bmpanel2, cbatticon, volumeicon) but not change.

Any idea of the next steps ?

[richardgv]

This sounds absurd. Absurd indeed... valr, is it possible for you to come to #compton on FreeNode right now? I couldn't reproduce this issue on my side, and I would probably need somebody to add temporary debugging code and execute them. It could be much more efficient if there's a way to contact instantly.

Update:
Make sure your locale don't use comma as the "dot", like some European locales. Make sure you don't have a configuration file. And what is your system architecture? amd64 or i386?

[valr]

As discussed, it's a locale problem.

Indeed, numeric values under my locale (fr_BE.UTF-8) e.g. 0.03 should be written 0,03 to work.
So, let's do it like this: "LC_ALL=C compton the_options" until the fix is committed.

Thanks a million, richardgv!

[richardgv]

A big big problem caused by a small small thing indeed. :-D fr_BE.UTF-8 uses comma instead of dot in a floating point value, and atof(), which we uses for parsing floating point values in commandline arguments, fail to recognize the dot format under the locale, and stop at the dot point, so -I 0.03 is recognized as -I 0, then all windows disappear because they are fading in with the speed of 0.

0d67243 sets libc locale to system locale in order to get the non-ASCII characters in window titles right, thus triggering the issue.

Thanks for all your help in the investigation, valr! :-)

The performance problem still needs attention, therefore, not closing yet.

[valr]

1. I confirm the commit in your branch works perfectly.
2. I confirm a higher cpu consumption...if I'm not misusing pidstat (which I use for the very first time).
   Please, tell me if I should use it differently.

Here are some stats with: pidstat -u -C "compton" 1 20

before version (20120918)

Moyenne : PID %usr %system %guest %CPU CPU Command
Moyenne : 11047 0,10 0,05 0,00 0,15 - compton

current version

Moyenne : PID %usr %system %guest %CPU CPU Command
Moyenne : 11372 1,65 1,75 0,00 3,40 - compton

Now, I have to do profiling to see where is the problem, if any.
Another thing I've never done (yet), any advice is welcome.

My machine: dell inspiron L501X, core I7, nvidia GeForce GT 435M.

[richardgv]

I did another test, and this is embarrassing: Looks like the old compton is using 50% more CPU: https://gist.github.com/3797583

compton-old.txt comes from 2a566e8, the point before I start patching compton, using switches compton -cCGfFz -m 0.8 -i 0.8 -e 0.7 -r 7 -l -7 -t -7. compton-new.txt from the current richardgv-dev branch, with compton.sample.conf, which does basically the same thing as all those switches. 120 seconds, doing basically the same sequence: triggering Firefox popup menus, minimizing and restoring Firefox window and another smaller one, moving and resizing Firefox window, switching focus. Are you using much different action sequences?

I tried your switches compton -Cc -fF -t-1 -l-3 -r4.2 -D1, too. 0.95% vs 0.85%, new compton wins. Comparing 0d4c6fc5cc (master branch before the second time chjj merge my branch) with latest richardgv-dev, same result.

My system: Intel Core i7 3770K, nVidia Geforce GTX 670, pf-sources-3.5.4, x11-drivers/nvidia-drivers-304.51. I don't think graphic card has anything to do with the CPU usage of the compton process -- it should only affect X process.

If you wish to do profiling, I personally use OProfile-0.9.8. Thanks. :-)

[valr]

It's amazing how different systems react differently :-)

My above figures were reported without taking any action. I mean I had 3 xterm opened and was just doing nothing (no window move, ...) and without any activity the cpu consumption is much above. Of course it might not be a fully valid test case but I find those figures interesting.

I'll look at it i.e. use your options and your action sequences and see the result.
Thanks also for the profiling tool!

Note that it might take me few days before I can post any result (plenty of stuffs for work & at home). But the most critical thing (windows disappearing) has already been fixed.

Thanks for all.

[richardgv]

It's absurd that doing nothing could let compton use so much CPU resources. 200 seconds, during which I do nothing but typing in Firefox window, and the average CPU usage of compton is 0.16%. I suspect there's some windows on your system causing the excessive CPU usage. Please try closing all the windows one by one. Enabling event debugging (add -DDEBUG_EVENTS to CFLAGS) and check which one is generating many events may also help. Experimenting with different commandline switches could provide some hints.

[valr]

Ok, here are some more info: this is the commit that causes problem on my system: 778a8b1 Improvement: Change fading mechanism, and if I'm mistaken, you know it very well! Just kidding :)

I've put some 'printf' in the code and activated some debug info and what's causing me a problem is that compton is flooded by damage events. The result is that the function 'paint_preprocess' is called veeeeeeery often. I've not checked much deeper but it seems to be the problematic part.

So, why am I flooded by damage events, I've no clue. I've tried to switch from openbox to twm (was already installed on my machine as I was testing in the train back home) no change. I've tried to change from xterm to urxvt, no change. Note that I've deactivated all the other background X processes I have (volumeicon, cbatticon, ...).

So, do you have any idea why I'm flooded by damage events when doing nothing ? Do you think the 'paint_preprocess' function should only be called under certain conditions ?

Edit 1: even without damage events, when this test is true...

    if (poll(&ufd, 1, fade_timeout()) == 0) {
      break;
    }

...in the main loop, it triggers dozens of successive calls to the 'paint_preprocess' function.

Edit 2: after some profiling (with sysprof, I can't get oprofile working), it indeed shows and important % of cpu used for paint_preprocess (sorry for the layout):

[./compton] 0,00%
__libc_start_main 0,00%
__poll_nocancel 0,00%
- - kernel - - 0,00%
system_call_fastpath 0,00%
main 1,73%
paint_preprocess 9,09%
fade_timeout 0,43%
No map [./compton] 0,00%
__poll_nocancel 0,00%
recv 0,00%
In file [heap] 0,00%

[richardgv]

You will get DamageNotify if the content of one of your windows changes. So if you are watching the debugging output of compton in an xterm -- so a window with content constantly changing -- you will definitely be flooded with DamageNotify, and this could drastically increase compton's CPU usage. You must close your terminal windows and use a virtual console to watch the debugging output of compton.

compton by default doesn't print out DamageNotify events in DEBUG_EVENTS mode. You could turn it on. It's a few lines after the start of ev_handle(). If a windows is emitting too many DamageNotify, you will be able to see them this way.

Compton will only initiate paint_preprocess() if it receives an event or it reaches next fading turn (so every 10 milliseconds, by default). But it will not necessary call paint_all() afterwards, so you could see paint_preprocess() been called more than paint_all(). It's possible to stop calling paint_preprocess() if there's literally nothing that it needs to do -- no events, no fading. But you shouldn't expect reducing calls to paint_preprocess() if there are events firing. I will see if I can reduce the code in paint_preprocess() further.

By the way, I've working on a way that decreases CPU usage of compton when a window is fading with shadows. 20 seconds of constantly fading in and out a window, and the average CPU usage dropped from 0.70% to 0.35%. Yet, it breaks colored shadow support...

...in the main loop, it triggers dozens of successive calls to the 'paint_preprocess' function.

Are you sure it's really generating dozens of calls, without receiving events (not even DamageNotify) or getting to the next fading turn?

after some profiling (with sysprof, I can't get oprofile working), it indeed shows and important % of cpu used for paint_preprocess (sorry for the layout):

When compton is perfectly idling -- no events or whatever -- its CPU usage should low enough (somewhere around 0.20% here, could be further minimized) that it should not be concerned. If it's higher than that number, it is probably receiving extra events. Just close all your windows and watch compton in a virtual console, make sure compton is receiving no events, and let's see if the total CPU usage of the compton process is still a problem.

It is the expected behavior that paint_preprocess() takes the most CPU time when idling, and it could be fixed -- but when compton is idling, for a desktop user his system is usually idling, too, and whether compton is using 0.20% or 0.13% CPU doesn't really matter. I care more about the CPU usage when compton is working under normal workload, actually.

[valr]

Thanks for the explanation of DamageNotify. Indeed the flood of damage events comes from the fact that I ran compton inside the xterm. So, I've run it and checked from a virtual console (thanks for the trick!).

Regarding your question: "Are you sure it's really generating dozens of calls, without receiving events (not even DamageNotify) or getting to the next fading turn?"

Well, I see not only dozens, but hundreds of calls without events (when idling), which seems to be normal if I understand well the thing about the fading turn of 10 milliseconds).

But I'm still at 4% when idling.

Just a question (because I don't know where to look at now, I must admit): in the previous code, the function fade_timeout was doing this test:

if (!fades) return -1;

Now, it doesn't do it anymore, as it seems 'fades' variable doesn't exist anymore (change in the logic).
Do you think it might be the origin of my problem as the 'poll' call is not a blocking one anymore ?

[richardgv]

Well, I see not only dozens, but hundreds of calls without events (when idling), which seems to be normal if I understand well the thing about the fading turn of 10 milliseconds).

Yes, that is the normal behavior right now. Although I don't think the CPU usage when idling is significant, I will try to improve it, but tomorrow or later. It's getting later today and I should sleep earlier today. I have sore throat and had a slight fever this afternoon...

But I'm still at 4% when idling.

Look, it's simply 0.20% here... If you are really sure there's no events, probably then we still need profiling data to figure out what the hell is happening on your side. sysstat is a system-wide profiler, isn't it? You could check if other profiling tools (perf, gprof, callgrind (comes with valgrind), etc.) could provide some hints.

Just a question (because I don't know where to look at now, I must admit): in the previous code, the function fade_timeout was doing this test:

if (!fades) return -1;

Now, it doesn't do it anymore, as it seems 'fades' variable doesn't exist anymore (change in the logic).
Do you think it might be the origin of my problem as the 'poll' call is not a blocking one anymore ?

Originally there's a fading queue fades, I merged it into the window list. I've said in the commit message that this may decrease performance -- I'm kinda trading performance for better program organization. As I've said above I will try to improve it tomorrow.

[richardgv]

Oh, well, for another day I'm staying up late... I pushed two performance related commits to richardgv-dev. Grab them and test, if you wish.

Modern CPUs are less powerful than what I thought initially. I didn't realize that 0.20% of a Core i7 3770K could purely be used by calling paint_process() 100 times a second. Now compton's CPU usage should drop to 0% on idling, I suppose.

By the way, here's a part of an oprofile profiling report of the compton process under my normal workload:

CPU: Intel Ivy Bridge microarchitecture, speed 3.901e+06 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (No unit mask) count 100000
samples  %        image name               symbol name
-------------------------------------------------------------------------------
1218     30.8511  no-vmlinux               /no-vmlinux
  1218     100.000  no-vmlinux               /no-vmlinux [self]
-------------------------------------------------------------------------------
558      14.1337  libX11.so.6.3.0          /usr/lib64/libX11.so.6.3.0
  558      100.000  libX11.so.6.3.0          /usr/lib64/libX11.so.6.3.0 [self]
-------------------------------------------------------------------------------
414      10.4863  libxcb.so.1.1.0          /usr/lib64/libxcb.so.1.1.0
  414      100.000  libxcb.so.1.1.0          /usr/lib64/libxcb.so.1.1.0 [self]
-------------------------------------------------------------------------------
311       7.8774  compton                  paint_preprocess
  311      100.000  compton                  paint_preprocess [self]
-------------------------------------------------------------------------------
263       6.6616  libXrender.so.1.3.0      /usr/lib64/libXrender.so.1.3.0
  263      100.000  libXrender.so.1.3.0      /usr/lib64/libXrender.so.1.3.0 [self]
-------------------------------------------------------------------------------
146       3.6981  libXfixes.so.3.1.0       /usr/lib64/libXfixes.so.3.1.0
  146      100.000  libXfixes.so.3.1.0       /usr/lib64/libXfixes.so.3.1.0 [self]
-------------------------------------------------------------------------------
82        2.0770  compton                  find_win
  82       100.000  compton                  find_win [self]
-------------------------------------------------------------------------------
70        1.7730  libc-2.16.so             malloc
  70       100.000  libc-2.16.so             malloc [self]
-------------------------------------------------------------------------------
58        1.4691  libc-2.16.so             _int_malloc
  58       100.000  libc-2.16.so             _int_malloc [self]
-------------------------------------------------------------------------------
53        1.3425  libc-2.16.so             __memcpy_ssse3_back
  53       100.000  libc-2.16.so             __memcpy_ssse3_back [self]
-------------------------------------------------------------------------------
53        1.3425  libc-2.16.so             _int_free
  53       100.000  libc-2.16.so             _int_free [self]
-------------------------------------------------------------------------------
52        1.3171  libc-2.16.so             pthread_mutex_lock
  52       100.000  libc-2.16.so             pthread_mutex_lock [self]
-------------------------------------------------------------------------------
51        1.2918  libXdamage.so.1.1.0      /usr/lib64/libXdamage.so.1.1.0
  51       100.000  libXdamage.so.1.1.0      /usr/lib64/libXdamage.so.1.1.0 [self]
-------------------------------------------------------------------------------
50        1.2665  compton                  paint_all
  50       100.000  compton                  paint_all [self]
-------------------------------------------------------------------------------
41        1.0385  compton                  make_shadow
  41       100.000  compton                  make_shadow [self]
-------------------------------------------------------------------------------
41        1.0385  compton                  set_fade_callback
  41       100.000  compton                  set_fade_callback [self]
-------------------------------------------------------------------------------
41        1.0385  libc-2.16.so             free
  41       100.000  libc-2.16.so             free [self]

paint_preprocess() still uses more than 10%, well, even if it really doesn't do many things! I have a few plans to accelerate make_shadow, but since it's using only 1% of the total number, it's pretty much useless to increase its performance. paint_all() could be further improved to stop rendering hidden windows, yet that may require a lot of extra code.

[valr]

I have sore throat and had a slight fever this afternoon...

Man, sleep and take all time you need for you!!!

I'm kinda trading performance for better program organization

And I thank you for that, really.
It's so great to have a simple/stable/working composite manager.

Oh, well, for another day I'm staying up late

Sleeeeeeeeep!!! And forget about this. :)

I pushed two performance related commits to richardgv-dev. Grab them and test

Grabbed and tested...and results are excellents!
It's perfect situation now. Back to almost no cpu usage.

By the way, here's a part of an oprofile profiling report of the compton process under my normal workload

I'll try to use oprofile, I promise.

Edit 1: ok, I'm able to run oprofile, but...well, I'm not sure I'm using it the right way. I've just done this:

opcontrol --deinit
echo 0 > /proc/sys/kernel/nmi_watchdog
opcontrol --no-vmlinux
opcontrol --init
opcontrol --reset
opcontrol --start
./compton -Cc -fF -D1 -t-1 -l-3 -r4 -o.5
opcontrol --stop
opreport -l ./compton

...and even with that I'm not sure how to interpret the result.
Here it is anyway.

Here is before (i.e. with issue on cpu usage):

CPU: Intel Core/i7, speed 1.734e+06 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (No unit mask) count 100000
samples  %        image name               symbol name
3204     78.1654  no-vmlinux               /no-vmlinux
360       8.7826  compton                  paint_preprocess
70        1.7077  [vdso] (tgid:4712 range:0x7fff4bd7a000-0x7fff4bd7b000) [vdso] (tgid:4712 range:0x7fff4bd7a000-0x7fff4bd7b000)
67        1.6345  compton                  make_shadow
45        1.0978  compton                  main
41        1.0002  compton                  check_fade_fin
39        0.9515  compton                  set_fade_callback
32        0.7807  libX11.so.6.3.0          /usr/lib/libX11.so.6.3.0
25        0.6099  libxcb.so.1.1.0          /usr/lib/libxcb.so.1.1.0
21        0.5123  compton                  run_fade
19        0.4635  compton                  get_opacity_percent
19        0.4635  libc-2.16.so             __memcpy_ssse3_back
16        0.3903  libm-2.16.so             /usr/lib/libm-2.16.so
15        0.3659  libc-2.16.so             __memset_sse2
15        0.3659  libc-2.16.so             poll
14        0.3415  compton                  get_time_in_milliseconds
12        0.2928  libc-2.16.so             __poll_nocancel
9         0.2196  libXrender.so.1.3.0      /usr/lib/libXrender.so.1.3.0
9         0.2196  libc-2.16.so             _int_malloc
4         0.0976  ld-2.16.so               do_lookup_x
4         0.0976  libc-2.16.so             __memcpy_sse2
4         0.0976  libc-2.16.so             malloc
4         0.0976  libc-2.16.so             memchr
4         0.0976  libc-2.16.so             recv
3         0.0732  compton                  fade_timeout
...

And after (with your new version i.e. no cpu issue anymore):

CPU: Intel Core/i7, speed 1.734e+06 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (No unit mask) count 100000
samples  %        image name               symbol name
321      59.7765  no-vmlinux               /no-vmlinux
55       10.2421  libX11.so.6.3.0          /usr/lib/libX11.so.6.3.0
28        5.2142  libxcb.so.1.1.0          /usr/lib/libxcb.so.1.1.0
22        4.0968  libc-2.16.so             __memcpy_ssse3_back
14        2.6071  compton                  make_shadow
6         1.1173  libc-2.16.so             _int_free
6         1.1173  libc-2.16.so             _int_malloc
5         0.9311  compton                  paint_preprocess
5         0.9311  libc-2.16.so             malloc
4         0.7449  ld-2.16.so               _dl_lookup_symbol_x
4         0.7449  libc-2.16.so             __memcpy_sse2
4         0.7449  libc-2.16.so             __memset_sse2
4         0.7449  libc-2.16.so             recv
3         0.5587  [vdso] (tgid:4385 range:0x7fffd0fff000-0x7fffd1000000) [vdso] (tgid:4385 range:0x7fffd0fff000-0x7fffd1000000)
3         0.5587  compton                  paint_all
3         0.5587  ld-2.16.so               do_lookup_x
3         0.5587  libXrender.so.1.3.0      /usr/lib/libXrender.so.1.3.0
3         0.5587  libc-2.16.so             __poll_nocancel
3         0.5587  libc-2.16.so             writev
2         0.3724  compton                  find_client_win
2         0.3724  compton                  find_win
...

[richardgv]

Man, sleep and take all time you need for you!!!

Thanks. I'm getting better today. :-)

And I thank you for that, really.
It's so great to have a simple/stable/working composite manager.

Mostly it's chjj's work, oh, and the xcompmgr authors. Thanks should go to them. :)

Grabbed and tested...and results are excellents!
It's perfect situation now. Back to almost no cpu usage.

I'm still not quite ready to accept that that executing paint_preprocess() 100 times per second takes 4% CPU...

Edit 1: ok, I'm able to run oprofile, but...well, I'm not sure I'm using it the right way. I've just done this:

Well, I guess I must have lead you on the wrong way, sorry. When I indicated oprofile initially I said Oprofile-0.9.8 as that version introduces easy per-application profiling. This is what I did:

# My compton executable is in current directory
$ ./compton -b
$ operf -p $(pgrep compton)
operf: Profiler started
operf: Press Ctl-c or 'kill -SIGINT 2931' to stop profiling
# Kill it with Ctrl-C
^C
Profiling done.
$ opreport -l ./compton -c | less

I might need more knowledge to understand the profiling data, hmm... But definitely, thanks for all your help! :-)

[valr]

Here are info without your performance commits:

CPU: Intel Core/i7, speed 1.734e+06 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (No unit mask) count 100000
samples  %        image name               symbol name
-------------------------------------------------------------------------------
7002     73.6975  no-vmlinux               /no-vmlinux
  7002     100.000  no-vmlinux               /no-vmlinux [self]
-------------------------------------------------------------------------------
1239     13.0407  compton                  paint_preprocess
  1239     100.000  compton                  paint_preprocess [self]
-------------------------------------------------------------------------------
266       2.7997  [vdso] (tgid:13309 range:0x7fff8d2e7000-0x7fff8d2e7fff) [vdso] (tgid:13309 range:0x7fff8d2e7000-0x7fff8d2e7fff)
  266      100.000  [vdso] (tgid:13309 range:0x7fff8d2e7000-0x7fff8d2e7fff) [vdso] (tgid:13309 range:0x7fff8d2e7000-0x7fff8d2e7fff) [self]
-------------------------------------------------------------------------------
184       1.9366  compton                  main
  184      100.000  compton                  main [self]
-------------------------------------------------------------------------------
169       1.7788  compton                  set_fade_callback
  169      100.000  compton                  set_fade_callback [self]
-------------------------------------------------------------------------------
139       1.4630  compton                  get_opacity_percent
  139      100.000  compton                  get_opacity_percent [self]
-------------------------------------------------------------------------------
124       1.3051  compton                  check_fade_fin
  124      100.000  compton                  check_fade_fin [self]
-------------------------------------------------------------------------------
100       1.0525  compton                  run_fade
  100      100.000  compton                  run_fade [self]
-------------------------------------------------------------------------------
90        0.9473  libc-2.16.so             __poll_nocancel
  90       100.000  libc-2.16.so             __poll_nocancel [self]
-------------------------------------------------------------------------------
71        0.7473  compton                  get_time_in_milliseconds
  71       100.000  compton                  get_time_in_milliseconds [self]
-------------------------------------------------------------------------------
70        0.7368  libm-2.16.so             /usr/lib/libm-2.16.so
  70       100.000  libm-2.16.so             /usr/lib/libm-2.16.so [self]
-------------------------------------------------------------------------------
29        0.3052  libc-2.16.so             poll
  29       100.000  libc-2.16.so             poll [self]
-------------------------------------------------------------------------------
13        0.1368  compton                  fade_timeout
  13       100.000  compton                  fade_timeout [self]
etc...

and with the performance commits:

CPU: Intel Core/i7, speed 1.734e+06 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (No unit mask) count 100000
samples  %        image name               symbol name
-------------------------------------------------------------------------------
43       58.1081  no-vmlinux               /no-vmlinux
  43       100.000  no-vmlinux               /no-vmlinux [self]
-------------------------------------------------------------------------------
10       13.5135  libX11.so.6.3.0          /usr/lib/libX11.so.6.3.0
  10       100.000  libX11.so.6.3.0          /usr/lib/libX11.so.6.3.0 [self]
-------------------------------------------------------------------------------
4         5.4054  compton                  find_win
  4        100.000  compton                  find_win [self]
-------------------------------------------------------------------------------
4         5.4054  libc-2.16.so             malloc
  4        100.000  libc-2.16.so             malloc [self]
-------------------------------------------------------------------------------
3         4.0541  libxcb.so.1.1.0          /usr/lib/libxcb.so.1.1.0
  3        100.000  libxcb.so.1.1.0          /usr/lib/libxcb.so.1.1.0 [self]
-------------------------------------------------------------------------------
2         2.7027  compton                  paint_preprocess
  2        100.000  compton                  paint_preprocess [self]
-------------------------------------------------------------------------------
2         2.7027  libXrender.so.1.3.0      /usr/lib/libXrender.so.1.3.0
  2        100.000  libXrender.so.1.3.0      /usr/lib/libXrender.so.1.3.0 [self]
-------------------------------------------------------------------------------
1         1.3514  [vdso] (tgid:13428 range:0x7fff6d7ff000-0x7fff6d7fffff) [vdso] (tgid:13428 range:0x7fff6d7ff000-0x7fff6d7fffff)
  1        100.000  [vdso] (tgid:13428 range:0x7fff6d7ff000-0x7fff6d7fffff) [vdso] (tgid:13428 range:0x7fff6d7ff000-0x7fff6d7fffff) [self]
-------------------------------------------------------------------------------
1         1.3514  compton                  discard_ignore
  1        100.000  compton                  discard_ignore [self]
-------------------------------------------------------------------------------
1         1.3514  compton                  get_opacity_percent
  1        100.000  compton                  get_opacity_percent [self]
-------------------------------------------------------------------------------
1         1.3514  compton                  paint_all
  1        100.000  compton                  paint_all [self]
-------------------------------------------------------------------------------
1         1.3514  libc-2.16.so             __memmove_ssse3_back
  1        100.000  libc-2.16.so             __memmove_ssse3_back [self]
-------------------------------------------------------------------------------
1         1.3514  libc-2.16.so             __poll_nocancel
  1        100.000  libc-2.16.so             __poll_nocancel [self]
-------------------------------------------------------------------------------

Do you want me to do something specific now ?
What might be interesting (maybe) is that I try to find inside 'paint_preprocess' what takes all that time, what do you think ?

[richardgv]

Thanks again for your awesome work. Oh, well, I've never worked much on projects previously and I'm not too sure how to deal with the profiling data. :-) Are you noticing any big performance issues? Is the CPU usage of compton a concern for you right now?

I'm actually doubting if we still need optimizations on performance side. I don't currently see any points that could be optimized significantly and cheaply (without loss of modularity or massive code to write). The shadow calculation part may have space for optimizations yet they are not actually called very frequently, hmm... Callgrind provided me much interesting data about the time usage inside paint_preprocess() but analysis would take time, if I could figure out how to install KCacheGrind without all those KDE dependencies...

Update: KCacheGrind shows 20% of cost of paint_preprocess() is in solid_picture(), which is used to generate pictures for alpha transparency. What do you think if we round window opacity to N * 3%? Basically, do you think it's acceptable to round 4.5% opacity to 6%, 54.5% opacity to 56%? This way we could pre-generate 33 pictures of different opacity, then totally avoid calling solid_picture() thereafter.

[valr]

Is the CPU usage of compton a concern for you right now?

Well, I have a small concern when an "idle" process uses 4% of cpu (as it's the problem I have on my machine).
Not because it uses cpu but because it uses the battery power of my laptop for "nothing".

I don't currently see any points that could be optimized significantly and cheaply (without loss of modularity or massive code to write).

I definitely don't want an important rework for this case.
I guess you don't like the fix you've implemented in your repo, do you?
At least it solves (hide?) the problem on my machine.

What I will do now is to profile compton with the same tools you've used (callgrind & KCacheGrind) and see the results I get from there (maybe I'll get something different on my machine).

What do you think if we round window opacity to N * 3%?

Uuuuh, absolutely no idea. I guess this might impact the visual effect, right?

[richardgv]

Not because it uses cpu but because it uses the battery power of my laptop for "nothing".

Oh, ah, I wasn't considering this, sorry. I've almost never used a laptop.

I guess you don't like the fix you've implemented in your repo, do you?
At least it solves (hide?) the problem on my machine.

I wasn't liking it initially, but eventually I found I had overestimated the power of modern CPU and the modification saved more CPU resources than what I expected (and the implementation is easier than my expectation, too). So, it is worthwhile, and I should definitely thank you for showing me this. :-)

What I will do now is to profile compton with the same tools you've used (callgrind & KCacheGrind) and see the results I get from there (maybe I'll get something different on my machine).

Ah, cool, if you have the time. We may just get something different depending on what we usually do when compton is running.

Uuuuh, absolutely no idea. I guess this might impact the visual effect, right?

Yeah, the difference is, the opacity of a window could deviate from its correct value for up to 1.5%. (We could save more CPU time by increasing the inaccuracy, but not so significantly.) 1.5% shouldn't be very noticeable, do you think?