Thursday, February 10, 2011

pyvnc realxtend addon

The old Python-VNC-Viewer project went to great effort to gain the best performance, even optimizing the image bit-depth, direct buffer copies into SDL surfaces, and grouped blitting - but it still remains unuseably slow. The readme file even contains this: "What is it good for? Nothing ;-) Use the original VNC viewer for better performance." This got me thinking, maybe I should give up, sounds like Python can't handle this problem?

At first I considered using twistedmatrix and leveraging the work already done by Python-VNC-Viewer. I knew that using ctypes and going direct to libvncserver would have much better performance; but not as much as I expected - at least at first. It turns out twisted is infact pretty fast, even compared to a compiled C library. Testing single threaded ctypes-libvnc vs single threaded twisted-vnc, I found that ctypes-libvnc is only modestly faster - not many times faster. The bottleneck is in reading bytes from the server, the VNC (rfb) protocol sends lots of data despite being jpeg compressed and chunked. The slowness comes from single-threading, and having to block when reading these bytes. Thats the end of the story for twisted, since its threads are under the GIL.

The GIL and Ctypes (our Friends):

I often read people bashing the Python and the GIL (global interpreter lock), and that Python lacks true support for threads. In fact Python does have threads, its just that they lock every time you touch a variable, at first this sounds like bad news. However, what is not being point out more often, is that ctypes automatically releases the GIL when calling into C. The GIL now becomes our friend because it reduces the work needed to make things thread-safe. Making PyVNC thread-safe was trivial. The final results show that dual-threaded ctypes-vnc is many times faster than single-threaded twisted-vnc.

Test Machine 2ghz pent4
python-vnc-viewer single threaded (twistedmatrix)
average refresh 0.6 seconds
pyvnc dual threaded (ctypes libvnc)
average refresh 0.07 seconds

Naali (Tundra) Addon:

1. get this addon script and save it as
2. download addon xml file and save it some where
3. run: 'python pyvnc.addon /path/to/naali/bin/pymodules'

The script will then take care of downloading the modules required for your platform, now when you start Tundra the vnc-server and vnc-client windows should appear. If the addon script does not work for you for some reason, you can get the latest source for pyvnc on google code here. Note you can run pyvnc standalone, use the addon script just give a different path to save it, then run it by: 'python' (as server), 'python --client' (as client).

Special thanks to Adminotech LTD who sponsored this project, and everybody in IRC #realxtend-dev who helped me in understanding Naali and Qt.