Eli Bendersky's website In an earlier post, I discussed one way of combining blocking socket I/O with a GUI, by means of a separate thread in which the I/O runs. I've also mentioned that one alternative is using asynchronous I/O with callbacks integrated into the GUI event loop. Here I want to present some sample code for this alternative.
One of the first things Python programmers will think about then considering asynchronous I/O is Twisted, which is an event-driven networking library written in pure Python. Twisted is a huge library, and it's quite a job to learn how to use it properly. Here I'll demonstrate just enough to accomplish the task at hand - re-create the simple socket client from the previous post that works as part of a PyQt GUI.
The full code sample is available for download - socket_client_twisted_pyqt. Here is the socket client class using Twisted:
import struct
from twisted.internet.protocol import Protocol, ClientFactory
from twisted.protocols.basic import IntNStringReceiver
class SocketClientProtocol(IntNStringReceiver):
""" The protocol is based on twisted.protocols.basic
IntNStringReceiver, with little-endian 32-bit
length prefix.
"""
structFormat = "<L"
prefixLength = struct.calcsize(structFormat)
def stringReceived(self, s):
self.factory.got_msg(s)
def connectionMade(self):
self.factory.clientReady(self)
class SocketClientFactory(ClientFactory):
""" Created with callbacks for connection and receiving.
send_msg can be used to send messages when connected.
"""
protocol = SocketClientProtocol
def __init__(
self,
connect_success_callback,
connect_fail_callback,
recv_callback):
self.connect_success_callback = connect_success_callback
self.connect_fail_callback = connect_fail_callback
self.recv_callback = recv_callback
self.client = None
def clientConnectionFailed(self, connector, reason):
self.connect_fail_callback(reason)
def clientReady(self, client):
self.client = client
self.connect_success_callback()
def got_msg(self, msg):
self.recv_callback(msg)
def send_msg(self, msg):
if self.client:
self.client.sendString(msg)
A couple of notes:
- Since I want to re-create the same length-prefixed protocol from the previous post, the IntNStringReceiver protocol class from twisted.protocols.basic comes in handy - it's designed especially for strings prefixed by integer length headers. In our case, the prefix is a 4-byte little-endian unsigned integer, which I specify with the structFormat and prefixLength attributes. In addition I implement a couple of callbacks of the IProtocol interface.
- SocketClientFactory is a straightforward subclass of Twisted's ClientFactory, implementing the callbacks we're interested in here.
This is all quite simple. The bigger problem was finding how to interface Twisted with PyQt. Since Twisted and a typical GUI library are both event-loop based, to make them work together we should use a custom reactor. Unfortunately, due to licensing issues, Twisted doesn't come with a reactor for PyQt pre-packaged, and it should be obtained separately. Even more unfortunately, the PyQt reactor (qt4reactor.py) doesn't appear to have a single well-defined address on the web - several slightly different versions of it can be found floating online. In my code sample I've included a version of qt4reactor.py which I found to work for my needs.
So, back to the code. This is the implementation of the PyQt client GUI that uses Twisted. Similarly to the thread-based sample, it keeps drawing a nice circle animation to demonstrate it's never blocked. The full code is in the zip archive, here is only the interesting part:
class SampleGUIClientWindow(QMainWindow):
def __init__(self, reactor, parent=None):
super(SampleGUIClientWindow, self).__init__(parent)
self.reactor = reactor
self.create_main_frame()
self.create_client()
self.create_timer()
def create_main_frame(self):
self.circle_widget = CircleWidget()
self.doit_button = QPushButton('Do it!')
self.doit_button.clicked.connect(self.on_doit)
self.log_widget = LogWidget()
hbox = QHBoxLayout()
hbox.addWidget(self.circle_widget)
hbox.addWidget(self.doit_button)
hbox.addWidget(self.log_widget)
main_frame = QWidget()
main_frame.setLayout(hbox)
self.setCentralWidget(main_frame)
def create_timer(self):
self.circle_timer = QTimer(self)
self.circle_timer.timeout.connect(self.circle_widget.next)
self.circle_timer.start(25)
def create_client(self):
self.client = SocketClientFactory(
self.on_client_connect_success,
self.on_client_connect_fail,
self.on_client_receive)
def on_doit(self):
self.log('Connecting...')
# When the connection is made, self.client calls the on_client_connect
# callback.
#
self.connection = self.reactor.connectTCP(SERVER_HOST, SERVER_PORT, self.client)
def on_client_connect_success(self):
self.log('Connected to server. Sending...')
self.client.send_msg('hello')
def on_client_connect_fail(self, reason):
# reason is a twisted.python.failure.Failure object
self.log('Connection failed: %s' % reason.getErrorMessage())
def on_client_receive(self, msg):
self.log('Client reply: %s' % msg)
self.log('Disconnecting...')
self.connection.disconnect()
def log(self, msg):
timestamp = '[%010.3f]' % time.clock()
self.log_widget.append(timestamp + ' ' + str(msg))
def closeEvent(self, e):
self.reactor.stop()
#-------------------------------------------------------------------------------
if __name__ == "__main__":
app = QApplication(sys.argv)
try:
import qt4reactor
except ImportError:
# Maybe qt4reactor is placed inside twisted.internet in site-packages?
from twisted.internet import qt4reactor
qt4reactor.install()
from twisted.internet import reactor
mainwindow = SampleGUIClientWindow(reactor)
mainwindow.show()
reactor.run()
The most important part of this code is in the last section, where the Twisted reactor and PyQt application are set up. A few steps have to be performed in a careful order:
- An QApplication is created
- qt4reactor is imported and installed into Twisted
- The main window is created
- Finally, the singleton Twisted reactor (which is actually a qt4reactor, since that's the one we've installed) is run
Note that there's no app.exec_() here, contrary to what you'd expect from a PyQt program. Since both PyQt and Twisted are based on event loops (in app.exec_() and reactor.run(), respectively), one of them should drive the other. The Twisted way is to let the reactor drive (hence we only call reactor.run() here). Inside its implementation, qt4reactor takes care of running an event loop in a way that dispatches events to both Twisted and PyQt.
Some additional notes:
- The PyQt main window no longer needs a timer to query results from the client thread. When SocketClientFactory is created in create_client, some methods are passed to it as callbacks. These callbacks will be invoked when interesting events happen.
- Even though Twisted's reactor is a global singleton object, it's good practice to pass it around in the application, instead of importing it from Twisted in multiple places. Here, SampleGUIClientWindow accepts the reactor object in its constructor and uses it later.
- Twisted's reactor keeps running until explicitly stopped. The user of a GUI expects the program to exit then the GUI is closed, so I call reactor.stop() in the main window's closeEvent.
This is it. Once set up, Twisted integrates quite nicely with PyQt. Since both the GUI framework and Twisted are based on the concept of an event loop with callbacks, this is a natural symbiosis.
A final note: I'm very far from being a Twisted expert. In fact, this is the first time I really use it, so may be doing things not in the most idiomatic or optimal way. If you can recommend a better way to implement some parts of this sample, I'll be very happy to hear about it.