Running a graphical window program via SSH on a remote machine (with GPU hardware acceleration)

Note 1: Even though it’s mid-2018, this post is still about the X Window System. Things still are in the transition phase towards Wayland, and things might get better or different over time.

Note 2: This post is not about displaying a graphical window of a program running on a remote machine on the local machine (like VNC or X forwarding). It is about running a remote program and displaying its graphical window on the remote machine itself, as if it had been directly started by a user sitting in front of the remote display. One obvious use case for the solution to this problem would be a remote graphics rendering farm, where programs must make use of the GPU hardware acceleration of the machine they’re running on.

Note that graphical programs started via Xvfb or via X login sessions on fake/software displays (started by some VNC servers) will not use GPU hardware acceleration. The project VirtualGL might be a viable solution too, but I haven’t looked into that yet.

Some experiments on localhost

I’m going to explore the behavior of localhost relative to our problem first. You’ll  need to be logged in to an X graphical environment with monitor attached.

The trivial case: No SSH login session

Running a local program with a graphical window from a local terminal on a local machine is trivial when you are logged into the graphical environment: For example, in a terminal, simply type glxgears and it will run and display with GPU hardware acceleration.

With SSH login session to the same user

Things become a bit more interesting when you use SSH to connect to your current user on localhost. Let’s say your local username is “me”. Try

It will output:

This can be fixed by setting the DISPLAY variable to the same value that is set for the non-SSH session:

Glxgears will run at this point.

With SSH login session to another user

Things become even more interesting when you SSH into some other local user on localhost, called “other” below.

You will get the message:

Trying to export DISPLAY as before won’t help us now:

You will receive the message:

This is now a permission problem. There are two solutions for it:

Solution 1: Relax permissions vIA XHOST PROGRAM

To allow non-networked connections to the X server, you can run (as user “me” which is currently using the X environment):

Then DISPLAY=:0 glxgears will start working as user “other”.

For security reasons, you should undo what you just did:

Settings via xhost are not permanent across reboots.

Solution 2: via Xauthority file

If you don’t want or can’t use the xhost program, there is a second way (which I like better because it only involves files and file permissions):

User “me” has an environment variable  env | grep XAUTHORITY

(I’m using the gdm display manager. The path could be different in your case.)

This file contains a secret which is readable only for user “me”, for security reasons. As a quick test, make this file available world-readable in /tmp:

Then, as user “other”:

Glxgears will run again.

To make sure that we are using hardware acceleration, run glxinfo:

This prints for me:

Make sure you remove  /tmp/xauthority_me after this test.

Note that the Xauthority file is different after each reboot. But it should be trivial to make it available to other users in a secure way if done properly.

Application on remote machine

If you were able to make things work on the local machine, the same steps should work on a remote machine, too. To clarify, the remote machine needs:

  • A real X login session active (you will likely need to set up auto-login in your display manager if the machine is not accessible).
  • A real monitor attached. Modern graphics cards and/or BIOSes simply shut down the GPU to save power when there is no real device attached to the HDMI port. This is is not Linux or driver specific. Instead of real monitors, you probably want to use “HDMI emulator” hardware plugs – they are cheap-ish and small. Otherwise, the graphical window might not even get painted into the graphics memory. The usual symptom is a black screen when using VNC.


If you SSH-login into the remote machine, as the user that is currently logged in to the X graphical environment, you can just set the DISPLAY environment variable when running a program, and the program should show on the screen.

If you SSH-login into the remote machine, as a user that is not currently logged in to the X graphical environment, but some other user is, you can set both DISPLAY and XAUTHORITY environment variables as explained further above, and the program should show up on the screen.

Related Links

How to digitize old VHS videos with an EasyCAP UTV007 USB converter on Linux

VHS is dead. If you don’t have a functioning VHS player any more, your only option is to buy second-hand devices. But if you still have old, valuable VHS videos (e.g. family videos) you should digitize them today, as long as there are still working VHS players around.

Our goal is to feed the audio/video (AV) signals coming out of an old VHS player into an EasyCAP UTV007 USB video grabber, which can receive 3 RCA cables (yellow for Composite Video, white for left channel audio, red for right channel audio).

EasyCAP UTV007 USB video grabber


VHS players usually have a SCART output which lucklily carries all the needed signals.

SCART connector

Via a Multi AV SCART adapter you can output the AV signals into three separate RCA cables (male-to-male), and from there into the EasyCap video grabber. If your adapter should have an input/output switch, set it to “output”.

Multi AV Adapter outputting 3 RCA connectors (yellow for Composite Video, white for left channel audio, red for right channel audio)

The EasyCAP USB converter uses a UTV007 chip, which is supported by Linux out-of-the-box. (Who said that installing drivers is a pain in Linux???) After plugging the converter into an USB slot, you should get two additional devices:

  1. A video device called “usbtv”
  2. A sound card called “USBTV007 Video Grabber [EasyCAP] Analog Stereo”

Too see if you have the video device, run v4l2-ctl --list-devices . It will output something like:

To see if you have the audio device, run

It will output something like:

To quickly test if you are getting any video, use a webcam application of your choice (e.g. “cheese“) and select “usbtv” as video source under “Preferences”. Note that this will only get video, but no audio.

We will use GStreamer to grab video and audio separately, and mux them together into a container format.

Install GStreamer

To install GStreamer on Debian-based distributions (like Ubuntu), run

Test video with GStreamer

Now, test if you can grab the video with GStreamer. This will read the video from /dev/video0 (device name from v4l2-ctl --list-devices above) and directly output in a window:

Test audio with GStreamer

Now, test if you can grab the audio with GStreamer. This will read the audio from the ALSA soundcard ID hw:3 (this ID comes from the output of pactl list above) and output it to PulseAudio (should go to your currently selected speakers/headphones):

Convert audio and video into a file

If both audio and video tested OK separately, we now can grab them both at the same time, mux them into a container format, and output it to a file /tmp/vhs.mkv. I’m choosing Matroska .mkv containing H264 video and Ogg Vorbis audio:

Record some video and then press Ctrl+C. The file /tmp/vhs.mkv should now have audio and video.

It would be nice if we could see the video as we are recording it, so that we know when it ends. The command below will do this:

You also can re-encode the video by running it through ffmpeg:

You can adjust the video and audio bitrate depending on the type and length of video so that your file will not be too large. The nice side-effect is that the coarser the video encoding, the more of the fine-grained noise in the VHS video is smoothed out.

Voila! You now should be able to record and archive all your old family videos for posterity!

Digitization of VHS video with Gstreamer.


How to install yubikey-manager on Debian

yubikey-manager is a Python application requiring some dependencies for it to be installed from the Python repositories, because it is not yet in the official Debian package repository. Here is how:

Here is the main commandline utility:

Zero Client: Boot kernel and root filesystem from network with a Raspberry Pi2 or Pi3

Boot your Raspberry Pi from nothing but an Ethernet cable

A Zero Client is a computer that has nothing on its permanent storage but a bootloader. Rather, it loads everything from the network.

With the method presented in this article, you will be able to boot a Raspberry Pi into a full Debian OS with nothing more on the SD card other than the Raspberry firmware files and the u-boot bootloader on a FAT file system. The Linux kernel and the actual OS will be served over the local ethernet network.

We will only focus on the Raspberry Pi 3, but the instructions should work with minor adaptations also on a Pi 2.

The following instructions assume that you have already built…

  1. a full root file system for the Raspberry
  2. a u-boot binary, and
  3. a Linux kernel

… based on my previous blog post. Thus, you should already have the following directory structure:

We will do all the work inside of the ~/workspace directory.

Preparation of the SD card

You will only need a small SD card with a FAT filesystem on it. The actual storage of files in the running OS will be transparently done over the network. Mount the filesystem on /mnt/sdcard and do the following:

Copy firmware

Copy u-boot bootloader

Create config.txt

config.txt is the configuration file read by the Raspberry firmware blobs. Most importantly, it tells the firmware what kernel to load. “Kernel” is a misleading term here, since we will boot u-boot rather than the kernel.

Create /mnt/sdcard/config.txt with the following contents:


Make an universal boot script for the u-boot bootloader

To achieve maximum flexibility — to avoid the repetitive dance of manually removing the SD card, copying files to it, and re-inserting it — we will make an universal u-boot startup script that does nothing else than loading yet another u-boot script from the network. This way, there is nothing specific about the to-be-loaded Kernel or OS on the SD card at all.

Create a file boot.scr.mkimage  with the following contents:

Replace the server IP with the actual static IP of your server. Note that this script does nothing else other than loading yet another script called netboot-${serial#}.scr  from the server. serial# is the serial number which u-boot extracts from the Raspberry Pi hardware. This is usually the ethernet network device HW address. This way, you can have separate startup scripts for several Raspberry Pi’s if you have more than one. To keep the setup simple, set the file name to something predictable.

Compile the script into an u-boot readable image:

Copy boot.scr to the SD card:

The SD card preparation is complete at this point. We will now focus on the serving of the files necessary for boot.

Preparation of the file server

Do all of the following as ‘root’ user on a regular PC running Debian 9 (“Stretch”). This PC will act as the “server”.  This server will serve the files necessary to network-boot the Raspberry.

The directory /srv/tftp will hold …

  • an u-boot start script file
  • the kernel uImage file
  • and the binary device tree file.

… to be served by a TFTP server.

The directory /srv/rootfs_rpi3 will hold our entire root file system to be served by a NFS server:

You will find installation instructions of both TFTP and NFS servers further down.


Serve the root file system

Let’s copy the pre-built root file system into the directory from where it will be served by the NFS server:

(notice the slash at the end of the source directory)


Fix the root file system for network booting

Edit  /srv/rootfs_rpi3/etc/fstab  and comment out all lines. We don’t need to mount anything from the SD card.

When network-booting the Linux kernel, the kernel will configure the network device for us (either with a static IP or DHCP). Any userspace programs attempting to re-configure the network device will cause problems, i.e. a loss of conncection to the NFS server. Thus, we need to prevent systemd-networkd from managing the Ethernet device. Make the device unmanaged by removing the folowing ethernet configuration file:

If you don’t do that, you’ll get the following kernel message during boot:

That is because systemd has shut down and then re-started the ethernet device. Apparently NFS transfers are sensitive to that.

In case you want to log into the chroot to make additional changes that can only be done from within (e.g. running systemctl scripts etc.), you can do:


Serve Kernel uImage

In this step, we create a Linux kernel uImage that can be directly read by the u-boot bootloader. We read Image.gz directly from the Kernel source directory, and output it into the /srv/tftp directory where a TFTP server will serve it to the Raspberry:


Serve device tree binary

The u-boot bootloader will also need to load the device tree binary and pass it to the Linux kernel, so copy that too into the /srv/tftp directory.


Serve secondary u-boot script loading the kernel

Create a file netboot-rpi3.scr.mkimage with the following contents:

Replace the server IP with the static IP of your server PC. Then compile this script into an u-boot readable image and output it directly to the /srv/tftp directory:

Make sure that the filename of the .scr file matches with whatever file name you’ve set in the universal .scr script that we’ve prepared further above.


Install a NFS server

The NFS server will serve the root file system to the Raspberry and provide transparent storage.

Edit /etc/exports and add:

To apply the changed ‘exports’ configuration, run

Useful to know about the NFS server:

You can restart the NFS server by running service nfs-kernel-server restart

Configuration files are /etc/default/nfs-kernel-server  and /etc/default/nfs-common


Test NFS server

If you want to be sure that the NFS server works correctly, do the following on another PC:

Mount the root file system (fix the static IP for your server):



Install a TFTP server

To install:

After installation, check if the TFTP server is running:

This command will tell you the default serving directory (/srv/tftp):

Here is another command that tells you if the TFTP server is listening:

To get help about this server: man tftpd


If you want to be sure that the TFTP server works correctly, do the following on another PC:

Then see if the server serves the Linux kernel we’ve installed before:

You now should have a local copy of the linux-rpi3.uImage file.



If you’ve done all of the above correctly, you can insert the prepared SD card into your Raspberry Pi and reboot it. The following will happen:

  1. The Raspberry Pi GPU will load the firmware blobs from the SD card.
  2. The firmware blobs will boot the image specified in config.txt. In our case, this is the u-boot binary on the SD card.
  3. The u-boot bootloader will boot.
  4. The u-boot bootloader loads and runs the universal boot.scr script from the SD card.
  5. The boot.scr downloads the specified secondary boot script from the network and runs it.
  6. The secondary boot script …
    • downloads the device tree binary from the network and loads it into memory.
    • downloads the Linux kernel from the network and loads it into memory
    • passes the device tree binary to the kernel, and boots the kernel
  7. the Linux kernel will bring up the ethernet device, connect to the NFS server, and load the regular OS from there.

Many things can go wrong in this rather long sequence, so if you run into trouble, check the Raspberry boot messages output on an attached screen or serial console, and the log files of the NFS and TFTP servers on your server PC.



How to turn the Raspberry Pi into a Gateway to mobile phone internet

Your DSL internet connection is too slow? Want to set up an improvised office? You do not want to pay for a DSL internet plan when you already have a fast 4G mobile plan? If yes to one of the above, it is quite easy to configure a Raspberry Pi to share one mobile internet connection to an Ethernet network.

raspberry pi mobile gateway topology
Turning a Raspberry Pi into a Gateway to mobile internet (Image license CC BY-SA 3.0)

Strictly speaking, you don’t have to use a Raspberry Pi to do this. A laptop or desktop computer with any Operating System would work too, but the Raspberry is so small and consumes only 2-3 W of electrical power, and is so cool (quite literally!), so will will make use of this awesomeness!



The following step-by-step guide is based on a pure Debian 9 (“Stretch”) distribution with a mainline/vanilla/unpatched Linux kernel built according to my previous blog post:

Raspberry Pi2 and Pi3 running pure Debian 9 (“Stretch”) and the Linux Mainline/Vanilla Kernel

  • We will not focus on the Raspbian OS nor on any other distribution, because documentation for these other setups exists in abundance.
  • You should not have a graphical interface installed. GUIs also install the NetworkManager service for systemd (Debian package “network-manager”), and I have not tested how NetworkManager interacts with the methods presented below. In addition, a bare-bone system is the preferred choice because it saves RAM and CPU resources.
  • In any case, you should attach a keyboard and screen to the Raspberry because you may temporarily lose network connectivity during the setup.
  • You also need a smart phone with an internet plan, supporting USB tethering. I have only tested recent Android based smartphones. Keep in mind during the following steps that, with most smart phones, you need to re-enable USB tethering after reboots or USB cable reconnects.



  • Computers in the LAN will be able to set the Raspberry Pi’s static IP address as internet Gateway and DNS server.
  • The Raspberry Pi will prefer a smart phone connection (tethered USB) to forward traffic.
  • If the smart phone is disconnected, the Rasbperry Pi will automatically fall back to an already existing gateway if present (i.e. a DSL modem)


Step 1: Install a DNS server

This ensures that cached DNS lookups are very fast when a DNS query has already been fetched.

Tell “bind” to use Google’s public DNS servers (they are good). Edit /etc/bind/named.conf.options and change the “forward” block to:

Restart “bind”:


Step 2: Configure a static IP address for the Ethernet adapter

If you already have a DHCP server running in your local network (we will use the subnet in this guide), give the Raspberry Pi a free static IP address in this existing subnet, e.g.

If you don’t have an existing DHCP server running in your local network, we will set one up on the Raspberry (see Step 8 below).

In both cases, we will give our Rasberry the static IP address Using systemd, change the config file of your ethernet connection /etc/systemd/network/


If your LAN already has an internet gateway, e.g. a DSL modem with address, add the following (optional) section to the same config file:

The large positive integer value of “Metric” ensures that other configured gateways with a lower Metric will be preferred. This will come in handy in the next step where the smart phone will be our preferred gateway with a Metric value of 1024.

Now reboot the Raspberry or run systemctl restart systemd-networkd.  You may lose network connectivity at this point if you are logged in via ssh.

Now, check that networkctl status eth0 matches our wanted static IP address:

Next, check the output of route -n (the kernel routing table). It should show:

If you have added the optional  [Route] section, you should also see the following as first line, which is our current default route to the internet:



Step 3: Set the smart phone connection as gateway

Plug in your phone’s USB cable into one of the Raspberry’s USB connectors. Then turn on USB tethering in the Settings UI of your smart phone.

Run networkctl. You should see the following entry amongst the other network connections (notice “off” and “unmanaged”).


To have the “systemd-networkd” service manage the “usb0” network device, create a file /etc/systemd/network/ with the following contents:

To apply this config file, run systemctl restart systemd-networkd .  After a few seconds,  networkctl should output (notice the “routable” and “configured” parts):

You also can check networkctl status usb0  to see the dynamic IP address obtained from the DHCP server on the smart phone. For Android phones this is usually in the subnet 42.

Next, check the output of route -n. Now, the phone connection “usb0” should be on the top of the list thanks to the lower metric of 1024:


Step 4: Check internet connectivity

With this routing table, we already can connect to the internet via the smart phone. To make sure that we are routed via the smart phone, we will ask the Linux kernel which gateway it would take first for traffic. ip route get  should ouput the IP address of the smart phone (, subnet 42):

Let’s ping Google’s server a few times: ping  to see if we have an actual working route to the internet:

The answer: Yes!

Check phone’s DNS server


Now let’s check if the phone’s DNS server is working. Type  dig (install Debian package “dnsutils” if not yet installed), and make sure that you’ve got an “ANSWER SECTION”:

Note that the response came from the phone’s IP. So, “systemd” has correctly configured the phone’s IP address as DNS server for the Raspberry (that information came from the phone’s DHCP server).

Run  dig again. This time the result should be cached and returned much faster (just 1ms):

Check local DNS server

Type  dig @localhost

Note that this time, the response came from the “bind” DNS server which we have installed in Step 1. It, in turn, forwards queries via the phone connection. This server will be used for all requests via Ethernet.

Step 5: Turn on IP protocol forwarding for the Linux kernel

By default, this feature is turned off. Check the current status of this feature:

sysctl -a | grep net\.ipv4\.ip_forward  will output:

To permanently set this variable to 1, create /etc/sysctl.d/30-ipforward.conf and add the following:

Reload all settings by typing  sysctl --system. Now, and also after a reboot, the “ip_forward” variable should stay enabled.


Step 6: Turn on Network address translation (NAT) aka. “Masquerading” between Ethernet and USB Smart Phone network links

Create a shell script  /usr/bin/ with the following contents and make it executable ( chmod a+x):

This will masquerade IP packets coming in through the Ethernet adapter as if they were coming from the Raspberry itself, forward them to the USB smart phone connection, and the incoming answers (from remote servers) will be re-written and forwarded back to whereever in the LAN they came from. That is the central purpose of the problem we’re trying to solve in this tutorial.

Run this script. Check the output of iptables -L -n -v:


To run this shell script at system boot, right after the network links have been brought up, create the following systemd service file:

Add the following:



Step 7: Test the Raspberry Gateway!

On another machine in your LAN (can be Linux, Windows or Mac), configure the Ethernet connection manually. Set the following:

  • Static IP Address: (or any other freely available address on this subnet)
  • Gateway:
  • DNS:

Then run traceroute  on that other machine. Truncated output:

The route is correctly resolved. First traffic goes to the Raspberry Pi, then to the smart phone, and from there to the internet.

If you can’t run traceroute on that other machine, using a regular browser to browse the internet should work at this point!


Step 8: Running a DHCP server on the Raspberry




This tutorial may seem long, but the commands are few, and with a bit of practice you can turn your Raspberry Pi into a mobile phone Gateway in 10 minutes to enjoy faster 4G internet when your other modems are too slow.

Reading Raspberry Pi chip temperature with mainline Linux kernel

This tutorial is based on a previous article where we installed pure Debian 9 with a recent mainline/vanilla Linux kernel on a Raspberry Pi, and so differs from what would be done on a Raspbian Distribution with a Raspbian kernel. In this article, we will read the Raspberry Pi chip temperature. Here is my previous article:

Raspberry Pi 2 running pure Debian 9 (“Stretch”) and the Linux Mainline/Vanilla Kernel

As of mainline Linux Kernel 4.9.0-rc3, the sysfs entry /sys/class/thermal for the Broadcom BCM283x chip found on Raspberry Pi’s is empty. You can apply the following patch to Linux kernel 4.9.0-rc3, even though it will soon be superfluous because it seems that currently there is ongoing work by Linux Kernel developers to add in the missing functionalities.

The patch which will get you the standard sysfs temperature node which you can read like this:

This is the chip temperature in thousandths of degrees cenigrade, i.e. 40.084 degrees Celsius.

The following patch will give you a new entry in the mainline Kernel config, under Drivers -> SoC -> BCM -> Raspberry Pi thermal sysfs driver, which you have to enable, then recompile your kernel.

I have actually submitted this patch to a Linux kernel developer, but the process to get code into the Linux kernel is quite elaborate, and he said that they are already working on it, so I let it drop and decided to write this blog post instead.


Setting I2C bus speed on a Raspberry Pi via Device Tree

This tutorial is based on my previous article where we installed pure Debian 9 with a recent mainline/vanilla Linux kernel, and so differs from what would be done on a Raspbian Distribution with a Raspbian kernel. In this article, we will set the I2C bus speed on a Raspberry Pi. Here is my previous article:

Raspberry Pi 2 running pure Debian 9 (“Stretch”) and the Linux Mainline/Vanilla Kernel

Device Trees

The I2C bus on the Broadcom BCM283x chips found on Raspberry Pi’s is well and directly supported by the mainline/vanilla Linux kernel. Since with the Raspberry Pi we’re dealing with a System on a Chip (SoC), and not a regular PC, the hardware is configured with so-called device trees, which is a low-level description of the chip hardware compiled from text into binary format.

The rpi23-gen-image script mentioned in my previous tutorial installs the binary device tree into /boot/firmware/bcm2836-rpi-2-b.dtb. The U-Boot bootloader can read this file and pass it to the Linux kernel which interprets it and enables all the mentioned features in it.

The clock frequency for the I2C bus is configured in this .dtb file, and the default is 100kHz. There is a tool which allows you to inspect the .dtb file, outputting regular text. With this tool you also can make changes to the device configuration. Nowadays, this is the proper way to configure low-level devices on SoC’s!


Read the device tree

This will output the decoded device tree as text. Regarding I2C, you will find i2c@-entries like this:

Change the device tree

The clock-frequency value is what we want to change. The value is a raw binary unsigned 32-bit int stored big-endian, unreadable for humans.

But you can use another tool fdtget to read just this value decoded:

This will output 100000.

In my case, I wanted to set the I2C bus to the slowest frequency, to compensate for long cable lengths. I found that one of the lowest supported I2C clock frequencies is 4kHz. With fdtput you can set the clock-frequency property for each i2c device (there are 3 on the RPi):

And that’s it. You have to reboot for these settings to take effect.

I used an oscilloscope to verify that the SCL pin of the GPIO of the RPi indeed toggled with 4kHz, and it did!

Raspberry Pi2 and Pi3 running pure Debian and the Linux Mainline/Vanilla Kernel

Update 2017 Feb 25: I have updated the step-by-step instructions based on the suggested fixes and improvements contained in the reader comments. I also have copied the step-by-step instructions from this blog post to the file hosted on From now on I will update the instructions only on github, so expect that the instructions in this blog post will grow slightly out of date.

Update 2017 Mar 4: 64 bit kernel and Debian OS now works on the RPi3.

Update 2018 May: Debian 10 “Buster” works, including wireless LAN on RPi3



Gallium graphics drivers for Raspberry Pi’s VC4 chip are now fully supported by Linux Mainline

This has been a long way coming. In February 2014, Broadcom announced that they would release the formerly closed-source drivers for the VideoCore IV (VC4) GPU of their BCM283x family of System-on-a-chip (SoC), powering Raspberry Pi’s.

To make a long story short, Eric Anholt started porting the Open Source drivers, as documented by this presentation early 2015, and contributed code to the Linux (Mainline) Kernel, libdrm, Mesa, and I’m sure it was a long and painful work. But the results are worth it. A picture says more than 1000 words:

Debian 9 ("Stretch") running on a Raspberry Pi 2, powered by Linux 4.9.0-rc3 Mainline/Vanilla Kernel. Notable in this image: Graphics driver is "Gallium" running on the VC4 GPU of the Broadcom 2836 system-on-a-chip (SOC). Glxgears runs with 60 FPS and consumes very little CPU. I2C interface is recognized.
Debian 9 (“Stretch”) running on my Raspberry Pi 2 (and 3), powered by Linux 4.9.0-rc3 Mainline/Vanilla Kernel. Notable in this image: Graphics driver is recognized as “Gallium” running on the VC4 GPU of the Broadcom 2836 system-on-a-chip (SOC). The glxgears benchmark runs with 60 FPS (the vsync of the monitor) and consumes very little CPU. Even the Raspberry I2C interface is recognized by the Linux Mainline Kernel.

To emphasize the point: It is no longer necessary to run specialized distributions (like Raspbian) or Linux kernels (like the Raspbian flavor) in order to have the Raspberry Pi well supported. And this is good news. Debian is a well established and maintained standard Distribution. And even though the Raspberry Pi is not powerful enough for the professional desktop user, it is powerful enough for the casual desktop user, and it is very small and cheap, which opens up a whole lot of possibities for new real-world applications.

I ran an additional test: Gnome even runs on Wayland (modern replacement for the X Window System) on the Raspberry Pi 2 (and 3):

Gnome on Wayland on Raspberry Pi 2
Gnome on Wayland on Raspberry Pi 2






It still is not a matter of a one-click installer to reproduce these results, you need some experience when you run into barriers. But it has gotten a whole lot easier. Github user drtyhlpr thankfully published the script rpi23-gen-image that can create a standard Debian distribution for the Raspberry Pi that can simply be copied to a SD card.

I have created a fork of this script to use the official Linux kernel instead of the Raspberry flavor one. Above screenshots are taken from a system that I’ve created with this script. My fork is developed into a slightly different direction:

  • Only official Debian releases 9 (“Stretch”) and newer are supported.
  • Only the official/mainline/vanilla Linux kernel is supported (not the raspberry flavor kernel).
  • The Linux kernel must be pre-cross-compiled on the PC running this script (instructions below).
  • Only U-Boot booting is supported.
  • The U-Boot sources must be pre-downloaded and pre-cross-compiled on the PC running this script (instructions below).
  • An apt caching proxy server must be installed to save bandwidth (instructions below).
  • The installation of the system to an SD card is done by simple copying or rsyncing, rather than creating, shrinking and expanding file system images.
  • The FBTURBO option is removed in favor or the working VC4 OpenGL drivers of the mainline Linux kernel.

All of these simplifications are aimed at higher bootstrapping speed and maintainability of the script. For example, we want to avoid testing of all of the following combinations:

RPi2 with u-boot, with official kernel
RPi2 without u-boot, with official kernel
RPi2 with u-boot, with raspberry kernel
RPi2 without u-boot, with raspberry kernel
RPi3 with u-boot, with official kernel
RPi3 without u-boot, with official kernel
RPi3 with u-boot, with raspberry kernel
RPi3 without u-boot, with raspberry kernel

Thus, the script only supports:

RPi2 with u-boot with official kernel
RPi3 with u-boot with official kernel

RPi2 (setting RPI_MODEL=2) is well supported. It will run the arm architecture of Debian, and a 32-bit kernel. You should get very good results, see my related blog posts:

The newer RPi3 (setting RPI_MODEL=3) is supported too. It will run the arm64 architecture of Debian, and a 64-bit kernel. The support of this board by the Linux kernel will very likely improve over time.

In general, this script is EXPERIMENTAL. I do not provide ISO file system images. It is better to master the process rather than to rely on precompiled images. In this sense, use this project only for educational purposes.

How to do it

Basically, we will deboostrap a minimal Debian 9 (“Stretch”) system for the Raspberry on a regular PC running also Debian 9 (“Stretch”). Then we copy that system onto a SD card, then boot it on the Raspberry.

We will work with the following directories:

Set up your working directory:

Do the following steps as root user.


Set up caching for apt

This way, you won’t have to re-download hundreds of megabytes of Debian packages from the Debian server every time you run the rpi23-gen-image script.

Check its status page:


Install dependencies

The following list of Debian packages must be installed on the build system because they are essentially required for the bootstrapping process.

For a RPi2, you also need:

For a RPi3, you also need:

Kernel compilation

Get the latest Linux mainline kernel. This is a very large download, about 2GB. (For a smaller download of about 90 MB, consider downloading the latest stable kernel as .tar.xz from

Confirmed working revision (approx. version 4.10, Feb 2017): 60e8d3e11645a1b9c4197d9786df3894332c1685

Working configuration files for this Linux kernel revision are included in this repository. (working-rpi2-linux-config.txt and working-rpi3-linux-config.txt).

If you want to generate the default .config file that is also working on the Raspberry, execute

For a RPi2:

For a RPi3:

Whichever .config file you have at this point, if you want to get more control as to what is enabled in the kernel, you can run the graphical configuration tool at this point:

For a RPi2:

For a RPi3:

Before compiling the kernel, back up your .config file so that you don’t lose it after the next make mrproper:

Compiling the kernel

Clean the sources:

Optionally, copy your previously backed up .config:

Find out how many CPU cores you have to speed up compilation:

Run the compilation on all CPU cores. This takes about 10 minutes on a modern PC:

For a RPi2:

For a RPi3:

Verify that you have the required kernel image.

For a RPi2 this is:

For a RPi3 this is:


U-Boot bootloader compilation

Confirmed working revision: b24cf8540a85a9bf97975aadd6a7542f166c78a3

Let’s increase the maximum kernel image size from the default (8 MB) to 64 MB. This way, u-boot will be able to boot even larger kernels. Edit ./u-boot/include/configs/rpi.h  and add above the very last line (directly above “#endif”):

Find out how many CPU cores you have to speed up compilation:

Compile for a RPi model 2 (32 bits):

Compile for a RPi model 3 (64 bits):

Verify that you have the required bootloader file:

Pre-download Raspberry firmware

The Raspberry Pi still needs some binary proprietary blobs for booting. Get them:

Confirmed working revision: bf5201e9682bf36370bc31d26b37fd4d84e1cfca

Build the system!

This is where you call the script.

For example:

You may want to modify the variables according to the section “Command-line parameters” below.

The file in my github repostory contains a working example.

Install the system on a SD card

Insert a SD card into the card reader of your host PC. You’ll need two partitions on it. I’ll leave as an exercise for the reader the creation of a partition table according to the following output of fdisk for a 64GB card:

The following commands will erase all contents of the SD card and install the system (copy via rsync) on the SD card:

Note about SD cards: Cheap (or sometimes even professional) SD cards can be weird at times. I’ve repeatedly noticed corrupt/truncated files even after proper rsync and proper umount on different brand new SD cards. TODO: Add a method to verify all file checksums after rsync.

Try booting the Raspberry

Insert the SD card into the Raspberry Pi, and if everything went well, you should see a console-based login prompt on the screen. Login with the login details you’ve passed into the script (USER_NAME and PASSWORD).

Alternatively, if you have included “avahi-daemon” in your APT_INCLUDES, you don’t need a screen and keyboard and can simply log in via SSH from another computer, even without knowing the Rasberry’s dynamic/DHCP IP address (replace “hostname” and “username” with what you have set as USER_NAME and HOSTNAME above):

Finishing touches directly on the Raspberry

Remember to change usernames, passwords, and SSH keys!

Check uber-low RAM usage

Running top shows that the freshly booted system uses only 23 MB out of the availabl 1GB RAM! Confirmed for both RPi2 and RPi3.

Network Time Synchronization

The Raspberry doesn’t have a real time clock. But the default systemd conveniently syncs time from the network. Check the output of timedatectl. Confirmed working for both RPi2 and RPi3.

Hardware Random Number Generator

The working device node is available at /dev/hwrng. Confirmed working for both RPi2 and RPi3.

I2C Bus

Also try I2C support:

Confirmed working for both RPi2 and RPi3.

Test onboard LEDs

As of the kernel revision referenced above, this only works on the RPi2. The RPi3 has only the red PWR LED on all the time, but otherwise is working fine.

By default, the green onboard LED of the RPi blinks in a heartbeat pattern according to the system load (this is done by kernel feature LEDS_TRIGGER_HEARTBEAT).

To use the green ACT LED as an indicator for disc access, execute:

To toggle the red PWR LED:

Or use the red PWR LED as heartbeat indicator (kernel option for this must be enabled):

Notes about systemd

systemd now replaces decades-old low-level system administration tools. Here is a quick cheat sheet:

Reboot machine:

Halt machine (this actually turns off the RPi):

Show all networking interfaces:

Show status of the Ethernet adapter:

Show status of the local DNS caching client:

Install GUI

Successfully tested on the RPi2 and RPI3.

If you want to install a graphical user interface, I would suggest the light-weight LXDE window manager. Gnome is still too massive to run even on a GPU-accelerated Raspberry.

Reboot, and you should be greeted by the LightDM greeter screen!

Test GPU acceleration via VC4 kernel driver

Successfully tested on the RPi2 and RPI3.

Glxinfo should output:

Kernel compilation directly on the Rasberry

Only successfully tested on the RPi2. Not yet tested on the RPI3.

In case you want to compile and deploy another Mainline Linux kernel directly on the Raspberry, proceed as described above, but you don’t need the ARCH and CROSS_COMPILE flags. Instead, you need the -fno-pic compiler flag for modules. The following is just the compilation step (configuration and installation omitted):

Follow-up articles

Setting I2C bus speed on a Raspberry Pi via Device Tree

Reading Raspberry Pi chip temperature with mainline Linux kernel

Zero Client: Boot kernel and root filesystem from network with a Raspberry Pi2 or Pi3


Adventures with various Segfaults due to defective RAM


If you get various segfaults on your Linux server, like these:




etc. etc., then, no, your system is not suddenly crazy. Nor are you. It is highly likely that you RAM is defective. You should reboot your server and run the  RAM test from your boot manager (Grub always has such a test) to see if it can detect faulty RAM.

If you are operating a server that you can’t reboot because you can’t tolerate downtime, there is an excellent tool called memtester , which is a memory test for a running system. It is part of the Debian distribution, installit with apt-get install memtester  Check top to see how much free RAM there is available. Say you have 10GB RAM free, then ask memterst to test 8GB of it (so that 2GB are remaining free for the running system to operate). In my case, memtester indeed detected faults.

I ran

It outputted stuff like this:

So, when I replaced the RAM, the Segfaults stopped. You can run memtester  regularly to make sure the RAM is okay. Healty RAM is a very crucial part of your successful hosting operation!

In my case however, the segfaults corrupted MySQL tables, which I had to clean up. All’s well that ends well!


Dual Monitor (Multi seat) setup with Displaylink USB Montors

Note: I wrote this tutorial in 2012. It is terribly out of date and meanwhile probably partly or entirely wrong. Nevertheless I decided to re-publish it in case it is useful for others. I will likely not be able to answer any questions about the subject.

This tutorial shows you a working setup for operating 2 monitors (one conventional one connected to the normal VGA output of your graphics card and one USB monitor with a Displaylink chip), demonstrated on Ubuntu 12.10 and Debian GNU/Linux 7 (Wheezy).


First, I should mention that there is a lot of movement in working with Displaylink based USB monitors in GNU/Linux. Almost all documentation in the internet is either outdated or does not work, despite great interest in running many seats with just one computer.

Earlier tutorials in the internet concentrate on the  xserver-xorg-video-displaylink package for the X Window System. To my great surprise, it as been removed from Ubuntu in August 2012 because the “displaylink KMS driver in the quantal kernel and x-x-v-modesetting replaces -displaylink”. After pulling some threads for an hour or so, it turned out that there is work going on to add displaylink support directly into the Kernel — this new support seems to be different from the already existing capability of the kernel to create a framebuffer device node like  /dev/fb1. This DRM kernel module is called  udl(announcement here and here), but the version at the point of this writing is 0.0.1 and, for now, it seems to do nothing more than to register and deregister Displaylink chips when plugged in and out.

If you research further, you will see that a new service manager called systemd is being developed as better replacement for existing startup/service managers, and that the  udl Kernel module is supposted to work with  systemd. Apparently it should be possible to have a single X11 server also ‘serve’ USB displays, so that you can share your desktop across several screens in true hotplug fashion.

This sounds very cool, but since all of that is in an extremely early stage, we can assume that a working plug and play for USB monitors will happen only a few years in the future, so going this route is a dead end, at least for now, and when you’re not a hardcore Kernel Module / X Windows developer that has a lot of experience in driving microchips.

In researching the alternatives, I found a working solution in running a separate X instance for the USB monitor. This second instance (apart from the first one that is ‘there’ by default) can either be started from a script — or from a Display Manager that supports Multi Seat setups.

In trying various configurations for X, it turned out that it is possible to make X use the framebuffer device node (like  /dev/fb1) directly, even without the now obsolete xserver-xorg-video-displaylink package, by using the driver “fbdev” instead of the obsolete driver “displaylink”. How exactly this is set up, see the instructions below. In any case, you need to run 2 separate instances of X in parallel.

Once I knew that X worked with the USB monitor, the next step was to research Display Managers to find out methods how to start this second X instance. I have looked into LightDM and the Gnome Display Manager gdm3.

To my dismay, I found out that LightDM, while advertised as very small and lean, almost has no useful documentation, and I spent several hours restarting lightdm to find out working settings in trial-and-error. Even though I found working settings for LightDM, those still seem to be extremely brittle and subject to bugs over bugs in LightDM. The advantage of lightdm over gdm is that it supports a Multi Seat setup, whereas newer versions (!) of gdm outright do not support multi seats any more. The subjectively perceived ‘brittleness’ of LightDM is such, that it will start only Unity whenever the slightest thing goes wrong in its configuration. Once Unity is started (by mistake), LightDM will remember Unity in a handful of configuration files and it becomes very hard again to get rid of Unity.

So, below we’ll present 2 solutions:

  1. LightDM multi-seat setup in Ubuntu 12.10, where two X servers are spawned from LightDM, running 2 instaces of the X Window System, with two different system users.
  2. Manual start of a second X server in Debian Wheezy, because gdm3 does not allow for Multi Seat configuration

Ubuntu 12.10 with LightDM and Window Managers “Unity” and “Fvwm”

Ubuntu 12.10 already prefers the new  udl Kernel module over the  udlfbframebuffer module, which means that you won’t get the “green screen of success” on Displaylink based USB monitors during system boot. Instead, you get the repeating test patterns, but we need the green screen for this method to work. So, we have to blacklist  udl and un-blacklist  udlfb:

  1. add  blacklist udl to  /etc/modprobe.d/blacklist-custom.conf
  2. comment out  blacklist udlfb from  /etc/modprobe.d/blacklist-framebuffer.conf

When you reboot, the USB monitor should be green.

In the course of my experimentation, I found that running “Unity” on the USB Displaylink monitor had severe distortions of color and window geometry, which rendered “Unity” useless for my purpose. So I chose a simpler and faster Window Manager as a workaround. I decided for “Fvwm”, which you can install with

Next, we need to create the X configuration for the second X instance. We will call the second X instance with the explicit path to the configuration file, so this does not need to go into  /etc/X11/xorg.conf.d, and you also don’t need to generate a working  xorg.conf first. Modern distributions come without this file anyway, and X autodetects everything. Simply create the file  /root/displaylink.conf with the following contents:


Change  /dev/fb1 to the device node that is created for you automatically by the Kernel when you plug in the USB monitor. Also, change the resulution “800×480” to the actual resolution you monitor has.

Now, in order to run two different Window Managers, we need to create 2 system users (the main user will have set “Unity” for the main monitor, and the second user will use “Fvwm” on the USB monitor). You cannot have both with just 1 user, we tried it and it does not work. Below, the main user is called “my_user1”, the second user “my_user2”. Create the user “my_user2” in the Settings dialog in Unity and set up a password. Of course, “my_user1” and “my_user2” are just examples, so make sure you use the real names in the configuration file below.

Next, change  /etc/lightdm/lightdm.conf to enable Multi Seats:


The keys  autologin-session and  user-session should, in theory (but only in theory) determine which Window Manager to start. The values “Fvwm” and “ubuntu” correspond to the  .desktop files in the directory  /usr/share/xsessions. Now, here is an explanation of why LightDM is so brittle. Thanks to long and painful reverse-engineering, I found out that the keys  user-session and  autologin-session are outright ignored when there are other ‘magic’ files present in the file system, which ‘cache’ these settings. The files in question are  /var/cache/lightdm/dmrc/my_user1.dmrc and  /home/my_user1/.dmrc where the contents are:

AND the file  /var/lib/AccountsService/users/my_user1 which has the content

Changing the values of  user-session and  autologin-session in  lightdm.conf will be useless, since it seems that the values of ALL 3 files must match so that lightdm does not default to “Unity”. Once it defaults to “Unity”, it will add “ubuntu” into all of these 3 files. Since we want “my_user2” to use “Fvwm”, we have to add the string “Fvwm” to all of those 3 files AND  lightdm.conf. If you now reboot, in theory, you should be auto-logged in in both X sessions, and you should have “Unity” on one screen and “Fvwm” on the USB monitor.

If you want to ‘autostart’ a few programs together with “Fvwm”, create a file  /home/my_user2/.xsessionrc and make it  chmod u+x. Add a few test programs:

Here is the proof that it worked for me:

Window Managers Unity and Fvwm running on two separate Xorg servers, spawned by LightDM Multi Seat configuration, with 7 inch Displaylink USB screen

If you need to configure screen blanking and keyboard/mouse, see sections below.

Debian GNU/Linux 7 (Wheezy) with gdm display manager

Since the Gnome Display Manager  gdm does not support Multi Seat configuration in its newer versions, we have to start the second X server manually. Also, the new  udl kernel driver is not yet active (as opposed to Ubuntu 12.10), so we don’t have to do the blacklisting as described in the previous section of this tutorial.

First, create  /root/displaylink.conf with the exact same contents as posted in the previous section of this tutorial.

You can call X directly only as superuser, since it needs raw access to hardware. So, execute the following command as superuser to test:

This should start Gnome on the USB screen. Press Ctrl + C to quit X, since it does not detach to the background. To choose “Fvwm” as a faster and smaller Window Manager alternative, install Fvwm …

then add “fvwm” to the above command:

It is not very flexible or elegant to add the command “fvwm” directly into the X startup line. We will copy it to a file called  /root/startupprograms.shinstead, so that we can add other startup programs later. Create this file and make it  chmod u+x. I am going to use my USB screen as a fullscreen web terminal, so I also start Chromium in kiosk mode. My  /root/ looks like this:

Of course, the X startup command now becomes

Now we need to add this command as a startup script called  displaylink (or any other name) in  /etc/init.d, so that we don’t have to run it manually, and that it will start during boot time:

Add the following to a file called  displaylink:

This script must be executable:

It must be symlinked to the Debian runlevel 2:

This creates a symlink in  /etc/rc2.d/Sxxdisplaylink, where xx is a number automatically determined by  update-rc.d.

Now if you reboot, you will get 2 working terminals: Gnome on the main monitor, and the fullscreen Chromium running on Fvwm on the USB monitor. Here is the proof:

Window Managers Gnome3 and Fvwm running on two separate Xorg servers


Disabling mouse and keyboard for the USB Monitor

Your mouse and keyboard will be enabled for both screens, which is undesirable. Disable keyboard and mouse in the X configuration file we’ve created above,  /root/displaylink.conf, by adding the following into the  Section "ServerLayout":

To move the mouse cursor to the bottom right corner (so that it is barely visible), add to  /root/ after installing  apt-get install xdotool:

xdotool mousemove <span class="hljs-number">10000</span> <span class="hljs-number">10000</span>
Before you start a browser that accesses a web application on localhost, you have to add a delay with  sleep 10, otherwise Apache won’t be running yet at this point of time.

If you run into problems, please contact Red (E) Tools Ltd. personally in our Support Forum.

Prevent screen blanks

In a Point of Sale setting, blanking or turning off the screen is not welcome. To keep displays on, you can either use the Energy settings in your display manager, but on our test system, in Gnome3, the maximum on-period of the monitor is 1 hour, and it would blank the screen anyway. You may need to override this manually, for both monitors. To keep your USB monitor turned on always, add the following commands to our startup file  /root/

To keep your main monitor turned on with Gnome, create a file in the home directory of your normal system user:

Make this file executable with  chmod a+x Then, add it as a Gnome startup program: In the Gnome Activities menu, enter “startup” as search term and start up this helper utility.

After you have rebooted, check in a terminal if the settings have been taken. Run  xset q and check that DPMS is disabled.