The way forward with Python on Qt 5 (with a bit of history)

As you have probably heard on Twitter from the official Jolla account, the first Jolla will ship with Wayland. In that discussion, some worries are brought up about Python support with Qt 5. Here are my personal thoughts of how I see mobile Python development moving forward with the new technology. So first some background information:

If you want to write Python applications on a mobile platform, you usually need some way to interface with the native graphical toolkit. On Maemo 4 and Maemo 5, this was done via PyGTK and some additional bindings for the Hildon UI elements. PyGTK is licensed under the terms of the GNU LGPL, so you could use it to develop open and closed source applications.

When MeeGo 1.2 Harmattan (the system running on the N9) came up, the toolkit with which third party application developers wrote their applications changed from Gtk to Qt (or to be more precise, QML). Just like pure Gtk didn't provide all the UI elements for mobile (and therefore Hildon was used on top of Gtk), pure QML provided very little, and so Qt Quick Components were used for the Harmattan user interface (that includes things like buttons, toolbars, menus, dialogs, etc...). These could only be used from QML, so you had to use QML if you wanted your application to look anything like native (ignoring the MeeGo Touch Framework here, as it wasn't really used for third party apps, with the notable exception of sociality-mtf).

With Qt and QML being the new toolkit of choice, Python developers needed a different "bridge" to go from the Python world to the toolkit world. PyQt already existed for some time then, it had two license options: GNU GPL (basically requiring all your code that uses PyQt to also become GPL'd) and commercial. As the Wikipedia article of PySide says, after Nokia failed to reach an agreement with Riverbank Computing (the developers of PyQt) to change the license to LGPL, they started their own Python bindings named "PySide", which are licensed under the terms of the GNU LGPL. There's a wiki page with PySide-PyQt differences on the Qt Wiki.

As MeeGo at Nokia was ramped down, the core team of PySide had less and less time to spend on improving it. PySide is still kept up to date (the latest release is from July 9, 2013), and it already supports Python 3, but nobody has yet ported it to Qt 5. The most recent post I found about PySide and Qt 5 from one of the core developers was from November 2011. For using Python with Qt 5, there are two possibilities at the moment: Use PyQt under the terms of the GNU GPL, or get a commercial PyQt license. Another option to get LGPL'd bindings would be to add support for Qt 5 to PySide, but that needs somebody to adapt PySide to work with Qt 5.

But let's step back a little now. In my experience with writing Python applications for mobile operating systems (starting with Maemo 4 all the way to MeeGo 1.2 Harmattan), there are really two parts: The frontend, which is specified in terms of whatever the toolkit of the day happens to be, and the backend, which (in case of Python applications) is written in Python.

In Qt 5 / QtQuick, the user interface language is QML, which includes a declarative language and JavaScript for scripting. Writing QtQuick applications with Python doesn't mean replacing JavaScript, it means replacing the "backend" - code that would usually be written in C++ and take care of storage and communication. In fact, as QML grew more powerful over the years, you had to "go down to the C++ level" (or Python in our case) less often. And compared to imperative user interfaces (where you still go through your tree of UI widgets with code and then set some button text to some value), in declarative user interfaces the backend really only provides services to the frontend (that button gets the label from the backend in this way, and when the data in the backend changes, so does the button's text).

So, what we really need is a way to provide services to the QML UI by using Python. There's no need for having access to all classes in all Qt modules - even in Qt4 / PySide times, the classes that I used most often were QApplication (for the event loop), QDeclarativeView (for displaying the QML UI) and QObject (for providing signals and slots to interface the backend to the view). Things like access to the contacts database or GPS location can usually be done directly from QML (and displayed there) - it might be that you don't even need to have that data in your backend (and if you do, the UI can for example send the phone number of the contact "down to" the Python backend for further processing).

But before I'll tell you where I think mobile Python development should be heading, let's bring up yet another disadvantage of using PySide (probably applies to PyQt as well) for mobile applications: Startup time and responsiveness. Startup time is slow, because PySide libraries are big (they are - at least on the N9 - bigger than the Qt libraries they are binding), and the dynamic linker has to resolve all Qt symbols at load time instead of just the ones you end up using. Instead of seeing the QML UI right away, your application first has to load up the Python interpreter, then load up the PySide modules and other modules and then finally load and display the QML UI. And because of Python's Global Interpreter Lock, function calls from QML to Python will make the UI block and not be totally responsive, even if you are using threads.

So, in my opinion, mobile Python applications with Qt 5 have to be fast to load, lightweight and responsive. Fast to load can be achieved by making sure the libraries that get loaded are small, and that ideally the QML UI can already be displayed before the Python interpreter is even loaded and/or initialized. Lightweight can be achieved by not binding all the Qt classes, but only the ones you really need for creating QML applications. And responsive can be achieved by making sure that the interface between QML and Python is asynchronous, so the UI never blocks even if Python is working hard in the background to fulfill the requests of the user interface.

I currently have a prototype running on Python 3.3 (Python 2.6 and 2.7 is also supported still, but for me, this is a great opportunity to migrate some of my old code to Python 3, and new code should be written in Python 3, because, as you know, Python 2 sucks) and Qt 5 (I've just ported it to Qt 5 today, but it runs just as well on Qt 4 - with all the advantages like startup time and asynchronousness still being valid). In some unscientific tests that I carried out a few weeks ago, I've brought down startup time of gPodder on the N9 down from ~ 12 seconds (using Python 2 and PySide) to ~ 3 seconds (using Python 3.3 and the lightweight "PyOtherSide" approach). No code release yet, as this is tailored towards my gPodder experiment right now and has many things hardcoded, but once I clean it up (maybe as a QML plugin) and define the API in more detail, I plan to release it.

In the mean time, here are some (relatively old) videos of PyOtherSide running with Python 3 and Qt 4 on MeeGo Harmattan, Blackberry 10 and Android (so yes, it is portable).

Why I love the N9 and things I didn't know about WP7

With last week's Nokia World and today's hands-on event in Vienna, I got a chance to try out the WP7 devices and ask questions about them. Here's some insights that I want to share with you, because they might not be obvious if you haven't tried WP7 before (I haven't until last week, and these things were not obvious to me). I'm comparing the Lumia 800 with the N9 here, because they look very similar from the outside.

As the outer shell is basically the same (or in other words: all hardware features that they fitted into the N9's form factor could just as well be fitted into the Lumia 800 form factor), these things are software limitations that are in WP7 right now (doesn't mean that these will stay in the future). Most basic things that the N9 can do (and the WP7 devices can't yet) can even be done by the N900 and Symbian devices. It's also written from a pure end user experience perspective - I've talked about SDK availability earlier.

• The Lumia 800 has a camera hardware button - the N9 does not. I would really like the N9 to have a hardware camera button (what's the progress on getting one of the volume keys to act as shutter button?)
• If you turn off the Lumia 800, wake-up alarms won't work - they do work on the N9, i.e. with an alarm set and the N9 turned off, the N9 will turn itself on and ring the alarm
• If you receive a call from a number that's not on your phone book, you can't directly add it to your phone book - you have to copy the number, then go to the contacts, create a new contact and paste the number there - on the N9, there's a button for this - and it even allows you to "Add new" or "Merge"
• The N9 has NFC - the Lumia 800 does not (WP7 does not support NFC at the moment)
• The N9 has TV-out - the Lumia 800 does not (on the other hand, some Symbian devices have HDMI out in addition to TV-out - that would have been nice for the N9 as well ;)
• The N9 can be used in USB Mass Storage Mode - the Lumia 800 can not
• The N9 shows the time (+missed calls/new mails/SMS) on its low-power standby screen (like the N8 and E7) thanks to the AMOLED screen - the Lumia 800 has the same screen technology, but (due to WP7 not supporting it, as I've been told) there's no low-power standby screen (the lock screen when the device is reactivated does have more information, though)
• You can send and receive files via Bluetooth on the N9 - I've been told that you can't do this with the Lumia 800 (Update: According to Allan, there is an app for transferring contacts from an old phone to the new one - no info about arbitrary files yet)
• The N9's marvelous virtual keyboard (an open source project, by the way) has very good tactile feedback - on WP7, the only feedback you can get is an audible one (although a long-time WP7 user told me that the virtual keyboard there is very good, even without tactile feedback)
• Swipe. It really makes a difference

What's the point you ask? These are the things that let the N9 really shine compared to its WP7 "successor" (I don't talk about openness, SSH-into-ability, QML, hackability, Python goodness and other things here - that's something that developers like and really, really care about) for end users (it obviously also ignores apps, services, "ecosystems", but I'd argue that the things that I personally need are available on the N9 as well).

It also shows that when a vendor controls both hardware and software, they might be able to put together a more interesting HW/SW mix compared to the situation where someone else controls the software. And it really lets me look forward to the Next Billion(tm) and QML there. But for now, let's enjoy the N9 and create some cool apps (and games) for it, like a FM Radio GUI for javispedro's fmrx app or gPodder 3 (coming soon) maybe?

Comparing Mobile OS SDK availability by platform

Coming back from Nokia World in London earlier this week (thanks to Nokia Developer for inviting me), I've been thinking about the SDK availability for different mobile operating systems given a specific Desktop platform. While leaving out all the other criteria (openness, libreness, licensing, UX, device capabilities, programming languages, toolkits, data formats, annual costs for the SDK/developer account, store rules, target audience, revenue splitting, advertising/in-app purchase options, coolness, etc..) developers can choose their mobile OS by, I want to highlight a specific aspect: The availability of an SDK for a given Desktop operating system.

First up, I'm only talking about native apps here (with "native" being anything that's not just some HTML+JS zipped up or served via the web, i.e. native usually means you need to have some form of compiler, even if it targets a VM). If you are into "web apps", chances are that you don't need an SDK to get started (even though one might help). I'll look at iOS and Android (because these are the strong mainstream OSes today), Maemo/MeeGo (because it's my platform of choice right now), Symbian (because I'm targeting it too with Qt) and WP7 (because that's what MS and Nokia want to succeed). When I write "MeeGo" I mean MeeGo 1.2 Harmattan, which technically is more like a "Maemo 6". Think "the OS that the N9 runs". I have a bit of experience with Android development, no experience at all with iOS or WP7, some experience with Symbian (through Qt) and arguably lots of experience with Maemo/MeeGo (yay!).

Why is that important? First up, if you are a Linux or OS X user, chances are that you don't have a Windows installation, and installing Windows will cost you money (for the Windows license), time (because you have to set it up) and space (because you have to dedicate a partition/VM image for it). If you don't have Apple hardware, getting an OS X installation means purchasing Apple hardware (ignoring Hackintoshes here), which is again costly. At least you would need to purchase OS X and dedicate a partition/VM for it, plus the time it needs to set it up.

Why OS X and Windows? As far as I know, if you want to develop for iOS, you have to use XCode, and that is only available on Mac OS X. Similarly, if you want to develop for WP7, you have to use the Windows Phone SDK, which is only available on Windows (>= Vista according to the website, so your XP install might not help there).

Now, let's look at Android, MeeGo and Symbian. Android's SDK is available for Windows, Linux (x86 and amd64) and Mac OS X. You can compile your apps on all these platforms (using your system's javac + tooling provided by the SDK). For C/C++ Android apps, the NDK is also available for all three platforms. For Qt-based OSes (Symbian and Maemo/MeeGo), the Qt SDK itself is available for Windows, Linux (x86 and amd64) and Mac OS X (64-bit). Using the Remote Compiler (which uses compilers set up on a server farm at Nokia, and you need a Nokia Developer account to use it) you can compile Symbian binaries on OS X, Linux and Windows, although the locally-installable compilers for Symbian are only available on Windows (at least that was the case 6 months ago). For Maemo and MeeGo, cross-compilers exist natively for all supported platforms of the Qt SDK, so without using Remote Compiler, you can build and package your Qt apps for MeeGo on Windows, Linux (x86 and amd64) and OS X.

Now, people can argue that one can set up dual-boot or virtual machines to support all OSes, but that's not the point. The point is that if the SDK is available on all Desktop platforms (note that this is not the same as SDK targetting all mobile platforms), developers can retain their choice of Desktop OS on which they develop on, and are not forced to use OS X or Windows for development of apps for the corresponding mobile platform (I also understand the reason why these companies only provide the SDK for their own Desktop platform, but that is not a good reason from a developer's point of view).

I hope that the Qt SDK will continue to support Windows, Mac OS and Linux for any mobile target platforms that it supports - be it ones named after winds or not - so developers have a choice of development platform.

In other news, gPodder 2.950.15 has been uploaded to Nokia Store (still waiting in QA) which fixes video playback and streaming, so grab the update on your N950/N9 when it becomes available :)