macos: error when starting both mouse and keyboard listeners

[Djiit]

Hi there,

Tested on last MacOS version, python 3.6.2, with root privileges to ensure keyboard is correctly monitored.

Using both keyboard and mouse listener in a Kivy application throw me this error :

 Traceback (most recent call last):
   File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
     self.run()
   File "/Users/jtanay/.local/share/virtualenvs/crtxpy-N0We0K7I/lib/python3.6/site-packages/pynput/_util/__init__.py", line 122, in run
     self._run()
   File "/Users/jtanay/.local/share/virtualenvs/crtxpy-N0We0K7I/lib/python3.6/site-packages/pynput/keyboard/_darwin.py", line 183, in _run
     super(Listener, self)._run()
   File "/Users/jtanay/.local/share/virtualenvs/crtxpy-N0We0K7I/lib/python3.6/site-packages/pynput/_util/darwin.py", line 186, in _run
     loop_source = Quartz.CFMachPortCreateRunLoopSource(
   File "/Users/jtanay/.local/share/virtualenvs/crtxpy-N0We0K7I/lib/python3.6/site-packages/objc/_lazyimport.py", line 163, in __getattr__
     raise AttributeError(name)
 AttributeError: CFMachPortCreateRunLoopSource

Tested without Kivy context (adapting examples from the doc to use the .start() method instead of the context manager ), it works perfectly. Any idea of what could be causing this ?

Please tell me if I can help :)

[Djiit]

Hi there,

Any news ?

Thanks !

[moses-palmer]

Hi,

Sorry for the late reply. Could you provide a minimal test that causes the error? I started investigating this, but kivy installation using pip install kivy failed with an error message indicating Cython was missing.

[Djiit]

Hi there,

Yep, the kivy docs are a bit messy sometime but as they state here, you have to install Cython first. On my macbook, i use kivy 0.10.0 with Cython 0.26.1.

Thanks again for investigating !

[Djiit]

Here is a minimal example :

from pynput import mouse

def on_move(x, y):
    print('Pointer moved to {0}'.format(
        (x, y)))

def on_click(x, y, button, pressed):
    print('{0} at {1}'.format(
        'Pressed' if pressed else 'Released',
        (x, y)))
    if not pressed:
        # Stop listener
        return False

def on_scroll(x, y, dx, dy):
    print('Scrolled {0} at {1}'.format(
        'down' if dy < 0 else 'up',
        (x, y)))

from pynput import keyboard

def on_press(key):
    try:
        print('alphanumeric key {0} pressed'.format(
            key.char))
    except AttributeError:
        print('special key {0} pressed'.format(
            key))

def on_release(key):
    print('{0} released'.format(
        key))
    if key == keyboard.Key.esc:
        # Stop listener
        return False


import kivy
kivy.require('1.10.0')

from kivy.app import App
from kivy.uix.label import Label


class MyApp(App):

    def build(self):
        # Collect events until released
        with mouse.Listener(
                on_move=on_move,
                on_click=on_click,
                on_scroll=on_scroll) as listener:
            listener.join()

        # Collect events until released
        with keyboard.Listener(
                on_press=on_press,
                on_release=on_release) as listener:
            listener.join()
        return Label(text='Hello world')


if __name__ == '__main__':
    MyApp().run()

[moses-palmer]

Again, sorry for the late reply. My mac development station is a Mac Mini from 2009 running El Capitan, so it's not exactly a pleasure to use ;-)

When I run pip install Cython=0.26.1 && pip install pillow && pip install kivy && python kivy-test.py, where kivy-test.py is the minimal example from above, I get a program that first waits for me to click, then for me to press escape and finally crashes when trying to create the Label, complaining about a missing window provider:

[CRITICAL] [Window      ] Unable to find any valuable Window provider.                      
pygame - ImportError: No module named 'pygame'                                              
  File "/Users/moses/src/pynput.git/venv/lib/python3.5/site-packages/kivy/core/__init__.py", line 59, in core_select_lib
    fromlist=[modulename], level=0)           
  File "/Users/moses/src/pynput.git/venv/lib/python3.5/site-packages/kivy/core/window/window_pygame.py", line 8, in <module>
    import pygame

pygame does not install, complaining about missing SDL headers.

[gabycperezdias]

Hi, I am having the same issue (I don't use kivy though..)
I have the same error as Djiit and sometimes this one (that also seems to be related)
All these errors started when I updated my MacOS (along with Xcode)

Exception in thread Thread-7:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pynput-1.3.7-py3.6.egg/pynput/_util/init.py", line 122, in run
self._run()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pynput-1.3.7-py3.6.egg/pynput/keyboard/_darwin.py", line 183, in _run
super(Listener, self)._run()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pynput-1.3.7-py3.6.egg/pynput/_util/darwin.py", line 192, in _run
Quartz.CGEventTapEnable(tap, True)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyobjc_core-4.2.2-py3.6-macosx-10.9-x86_64.egg/objc/_lazyimport.py", line 163, in getattr
raise AttributeError(name)
AttributeError: CGEventTapEnable`

[gabycperezdias]

@Djiit did you fix it? Maybe some fork? Workaround?

[moses-palmer]

@gabycperezdias It appears that you run pynput outside of a virtualenv, and that the version of pyobjc-core shipped with High Sierra is 4.2. pynput has been tested with pyobjc-core 3.

Have you tried running it inside of a virtualenv with the dependency versions specified in setup.py?

[gabycperezdias]

Hi, I can't really use a virtualenv because of the integration with wxpython (there's a issue that this library won't work inside a virtualenv).

But, I've downgraded my pyojbc to 3.2.1 (which is whithin your range right?) I still have a mix of 'random' errors, I've got a print from the same error as Djiit:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pynput-1.3.7-py3.6.egg/pynput/_util/__init__.py", line 122, in run
    self._run()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pynput-1.3.7-py3.6.egg/pynput/keyboard/_darwin.py", line 183, in _run
    super(Listener, self)._run()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pynput-1.3.7-py3.6.egg/pynput/_util/darwin.py", line 186, in _run
    loop_source = Quartz.CFMachPortCreateRunLoopSource(
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/objc/_lazyimport.py", line 163, in __getattr__
    raise AttributeError(name)
AttributeError: CFMachPortCreateRunLoopSource

[arikgelman]

Any update regarding this issue? I am trying to listen the mouse and the keyboard but I am getting the same error. The error is only on Mac os, windows working fine.

[moses-palmer]

Could you provide a sample script exhibiting the error, and an accompanying stack trace? As noted on Stack Overflow, this might be related to threading.

[arikgelman]

Ok, I found the solution. It is a threading problem. Putting the keyboard listener in separate thread fixed the problem.

def start_keyboard_listener():
    listener = keyboard.Listener(
                on_press=on_press,
                on_release=on_release)
    listener.start()

t1 = threading.Thread(target=start_keyboard_listen)
t1.deamon = True
t1.start()

with mouse.Listener(
                on_move=on_move,
                on_click=on_click,
                on_scroll=on_scroll) as mouse_listener:
            mouse_listener.join()

[moses-palmer]

Thank you for your prompt response.

Can you please provide your original, non-working script as well? I am curious to see what might cause this issue.

[arikgelman]

No problem, non-working code:

listener = keyboard.Listener(
                on_press=on_press,
                on_release=on_release)
listener.start()
with mouse.Listener(
                on_move=on_move,
                on_click=on_click,
                on_scroll=on_scroll) as mouse_listener:
            mouse_listener.join()

[moses-palmer]

Thank you.

I guess that the lazy-loading used by Quartz / pyobjc is not thread safe. In that case, your working code above may also fail intermittently. The correct solution would require changes to pynput to work around the issues in pyobjc.

[arikgelman]

Is this a known issue? Any timeline for a solution?

[moses-palmer]

I quick search suggests that it was a known issue in pyobjc, but was fixed in version 4.2. Perhaps you could test locally upgrading the version? If that success your issue, I would very much appreciate a PR.

[1over137]

Any updates now? I am also experiencing the same problem on Python 3.8.5 and macOS 11.0 beta.
Even if I use two threads via threading it stil produces the same error.

[apollokit]

So I've run into a similar problem on my mac, though I have been able to get a mouse and keyboard listener to work at the same time (I confirmed this over many tests. I never saw problems with both listeners running at the same time). What I'm encountering though is that when I try to add a keyboard controller to the mix, it breaks things. What I see:

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/Users/<me>/.config/virtualenvs/bdict_venv/lib/python3.8/site-packages/pynput/_util/__init__.py", line 193, in run
    self._run()
  File "/Users/<me>/.config/virtualenvs/bdict_venv/lib/python3.8/site-packages/pynput/_util/darwin.py", line 199, in _run
    loop_source = Quartz.CFMachPortCreateRunLoopSource(
  File "/Users/<me>/.config/virtualenvs/bdict_venv/lib/python3.8/site-packages/objc/_lazyimport.py", line 207, in __getattr__
    raise AttributeError(name)
AttributeError: CFMachPortCreateRunLoopSource

Here's a minimum example to repro the error:

from time import sleep
from pynput import keyboard
from pynput import mouse

from pynput.keyboard import Controller, Key

# works totally fine if I comment out this line and another line below.... vv
keyboard_controller = Controller()

def on_press(*args):
    print('press')

def on_click(*args):
    print('click')

keyb_listener = keyboard.Listener(
    on_press=None,
    on_release=on_press)

keyb_listener.start()

mouse_listener = mouse.Listener(
    on_move=None,
    on_click=on_press,
    on_scroll=None)

mouse_listener.start()

def spin():
    while True:
        print("spinning...")
        # ...this line ^^
        keyboard_controller.type('type')
        sleep(1)

with ThreadPoolExecutor(max_workers=5) as executor:
    futures = []
    futures.append(executor.submit(
            spin))

I'm on:

python 3.8
pynput==1.7.3
MacOS Catalina 10.15.6
pyobjc-framework-Cocoa==7.1
pyobjc-framework-Quartz==7.1

Kinda unfortunate. I've been using pynput for months on ubuntu 18.04 and loved it, never had any problems. Trying to run the same codebase on mac, I ran into this 😞

Anybody have any suggestions for fixes yet?

[fohara]

I have had success running mouse and keyboard listeners at the same time on macOS using multiprocessing instead of threads. The mouse listener and the keyboard listener each get their own child process and communicate the events back to the parent process via a Queue.

[apollokit]

@fohara gotcha, thanks. Good to know, I was thinking I'd have to split my code into different scripts and use a socket or something. Multiprocessing sounds a lot more straightforward.

[ronaldoussoren]

PyObjC developer here, this is a bug in PyObjC that I intend to fix. The lazy loader contains a race condition when resolving an attribute from multiple threads.

[ronaldoussoren]

As a workaround, all changes in pynput/_util/darwin.py:

• Change 'import Foundation; import Quartz' to:

from CoreFoundation import CFRelease, CFMachPortCreateRunLoopSource, CFRunLoopGetCurrent, CFRunLoopAddSource, kCFRunLoopDefaultMode, CFRunLoopRunInMode, kCFRunLoopRunTimedOut, CFRunLoopStop
from Quartz import CGEventTapEnable, CGEventTapCreate, kCGSessionEventTap, kCGHeadInsertEventTap, kCGEventTapOptionListenOnly, kCGEventTapOptionDefault

• Directory use those functions, basically remove all instances of "Quartz." and "CoreFoundation."

This way the symbols are resolved eagerly at import time and that avoids the race condition. That race condition will be fixed in PyObjC 8, which will be released sometime around the release of macOS Monterey later this year.

[moses-palmer]

Thank you @ronaldoussoren for yout input!

Would you advise to also change the keyboard and mouse implementations?

[ronaldoussoren]

I haven't look at the implementation for those, but in general doing "from ... import ..." for the symbols from CoreFoundation and Quartz that you use should avoid the race condition you're running in because import statements are serialized by a lock in the import system.

I have a fix for the race condition in PyObjC that will land in PyObjC 8, but that doesn't help you right now.

[git-iason]

Hi there, are we sure this was fixed in pyobjc 8.0? i am experiencing this issue with pyobjc 8.4.1 and pynput 1.7.6

I see you reverted the pynput fix in Nov '21 (0c0c9b3)

[jack-141122]

Hi there,
I was struggling with the same issue for a few days now. I was not able to get it to work till now.
I work an a Mac with M1 and should have the newest version of pynput installed.

Setting a delay between the starting of the two listeners fixed the problem.

Here is an example for recreating the problem that occurred:
`
from pynput import mouse
from pynput import keyboard

def on_press(key):
print("key pressed")

def on_release(key):
print("key released")

def on_click(x, y, button, pressed):
print("mouse clicked")

keyboard listener

keyb = keyboard.Listener(
on_press=on_press,
on_release=on_release)

mouse listener

mous = mouse.Listener(on_click=on_click)

start both listeners

keyb.start()
mous.start()

while True:
pass`

The Error is something like this: (one of the two listens will not work)
"KeyError: 'CFRunLoopAddSource'"

With a delay tho, both listeners worked fine at my setup:
`import time
from pynput import mouse
from pynput import keyboard

def on_press(key):
print("key pressed")

def on_release(key):
print("key released")

def on_click(x, y, button, pressed):
print("mouse clicked")

keyboard listener

keyb = keyboard.Listener(
on_press=on_press,
on_release=on_release)

mouse listener

mous = mouse.Listener(on_click=on_click)

start keyboard listener

keyb.start()

add delay

time.sleep(1)

start mouse listener

mous.start()

while True:
pass`

The important part here is the "time.sleep(1)".

Maybe this solution can help someone who runs himself into this problem.
(It should work with all delays >= 0.2; time.sleep(x>=0.2))

[ghost]

You can also use mous.wait() docs. That worked for me quite well.

[hughperkins]

@ronaldoussoren Your fix worked for me 🎉 But I have pyobjc-core==10.1, which should per your comment already be fixed?

[RidaShamasneh]

@lettow-humain Thanks man! worked like a charm <3