HTML5 + JavaScript + CSS3 RGBA video overlays on top of live GStreamer video pipelines

GStreamer comes with a number of plugins that allow rendering of text and/or graphics overlays on top of video: rsvgoverlay, subtitleoverlay, textoverlay, cairooverlay, gdkpixbufoverlay, opencvtextoverlay, etc. However, some of these plugins often allow only static graphics and text, and often do not approach the flexibility and power of dedicated video post-processing software products.

“noweffects” (a play on the name of a popular video post-processing software) is a proof-of-concept of leveraging the power of a modern HTML5 + JavaScript + CSS3 web browser engine to render high-quality, programmable, alpha-aware, animated, vector- and bitmap based content, which is then rendered into an RGBA raw video stream, which can then be transferred via some kind of IPC method to separate GStreamer processeses, where it can be composited with other content via GStreamers regular compositor or videomixer plugins.

Qt was chosen for its ease of integration of modern WebKit (QtWebKit) and GStreamer (qt-gstreamer), and its ability to render widgets to RGBA images. The QMainWindow widget is rendered in regular intervals to QImages in RGBA format, then inserted into a GStreamer pipeline via the appsrc plugin. This pipeline simply uses udpsink to multicast the raw video RTP packets on localhost to allow for multiple ‘subscribers’. A second GStreamer pipleline can then use udpsrc and apply the overlay.

Proof-of-concept code available at: https://github.com/michaelfranzl/noweffects

The following demonstration video was generated with “noweffects”: A website (showing CSS3 animations), rendered to an RGBA video via QtWebKit, then overlaid on top of a video test pattern in a separate GStreamer process.

“Open Source” does not imply “less secure”

Sometimes programmers hesitate to make their software open source because they think that revelation of the source code would allow attackers to ‘hack it’.

Certainly there are specific cases where this is true, but not as a general rule.

In my opinion, if inspection of the source code allows an attacker to ‘hack it’, then the programmer has done it wrong. Security primarily comes from writing secure algorithms, independent of their open source nature.

OpenSSL is a case in point: It is open source, but nevertheless it powers HTTPS all over the internet. “But,” you say, “it is only secure because its code is kinda obscure.” Well, no: Cryptographically secure algorithms exhibit very astonishing properties. For example, the One-time pad encryption technique is extremely simple and exhibits “Perfect secrecy” which is defined by Wikipedia as follows:

One-time pads are “information-theoretically secure” in that the encrypted message (i.e., the ciphertext) provides no information about the original message to a cryptanalyst (except the maximum possible length of the message). This is a very strong notion of security first developed during WWII by Claude Shannon and proved, mathematically, to be true for the one-time pad by Shannon about the same time. His result was published in the Bell Labs Technical Journal in 1949. Properly used, one-time pads are secure in this sense even against adversaries with infinite computational power.

Claude Shannon proved, using information theory considerations, that the one-time pad has a property he termed perfect secrecy; that is, the ciphertext C gives absolutely no additional information about the plaintext. This is because, given a truly random key which is used only once, a ciphertext can be translated into any plaintext of the same length, and all are equally likely.

Take for example the following simple implementation of the One-time pad (via XOR) in Ruby (which took me just a couple of minutes to write):

This code is open source, but it nevertheless exhibits the property of perfect (i.e. 100%) security “even against adversaries with infinite computational power” — given that the key is never submitted over insecure channels.

Sure, the One-time pad is not practical, and one could probably exploit weaknesses in Ruby or the underlying operating system. But that is not the point. The point is that, given proper implementation of software, it can be made open source without compromising its security.

To contrast this with an example of (bad) source code which should not be made public because it only creates a false sense of security:

Here, ((msgraw[n] + 7) ^ 99) is equivalent to a hard-coded secret. Sure, the obscured message, when transmitted over a public network, may look random. But the algorithm could easily be reverse-engineered by cryptoanalysis. Also, if the souce code were revealed, it would be trivial to decode past and future messages.

Conclusion

“Open Source” does not imply “insecure”. Security comes from secure — not secret — algorithms (which of course includes the freedom of bugs). What counts as “secure” is defined mathematically, and “mathematics (and in extension, physics) can’t be bribed.” It is not easy to come up with such algorithms, but it is possible, and there are many successful examples.

Needless to say, not every little piece of code should be made open source — ideally programmers will only publish generally useful and readable software which they intend to maintain, but that is a subject for another blog post.

WooCommerce Shipping Plugin “External Fetch”

I just wrote the following shipping plugin for WooCommerce because existing plugins would not cover the case I’m working at. It is available on my github account under the permissive MIT license.

https://github.com/michaelfranzl/woocommerce-shipping-external-fetch

woocommerce-shipping-external-fetch

Shipping Plugin for WooCommerce which HTTP PUTs the cart contents in JSON format to an external web service specified by protocol/host/port/URI, and receives the calculated shipping offer (costs, labels etc.) as JSON from that webservice.

Useful when the shipping calculation complexity exceeds the capabilities of other shipping calculation plugins.

The external shipping calculation application is, of course, business specific and thus not included in this repository. However, an example of a JSON request and reponse is shown below.

When the webservice is not reachable, not responsive, or returns HTTP status codes other than 200 (i.e. when it experiences a server error), this plugin offers free shipping to the customer, since technical problems are the ‘fault’ of the store owner and should not prevent a customer from completing an order. A setting to configure this behavior is not included, but it would be easy to add.

This plugin supports WooCommerce shipping zones. So, in theory you could have different web services dedicated to different shipping zones.

To add the plugin to an existing shipping zone:

  1. Go to WooCommerce -> Settings -> Shipping
  2. Click on “Manage shipping methods” below a Zone
  3. Click “Add shipping method” button
  4. Select “External Fetch” and click “Add shipping method” button
  5. Configure the Plugin by clicking on “Edit”
  6. Customize “Method title” and “Method description” and set the “JSON API Endpoint” to, for example  http://localhost:4040/calculate

At this point you can add a product to the cart, set up a simple webserver listening at port 4040 on the same machine, and receive/send JSON as shown in the section “Examples” further below.

This plugin is for DEVELOPERS only and will likely remain in an ALPHA state.

License

Copyright 2017 Michael Franzl

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Examples

Cart contents in JSON format as sent by this plugin:

 

Response in JSON format as expected by this plugin:

This plugin will add the  description values in a  <div class="shipping_rate_description" below each selectable shipping rate entry on the checkout page to give the customer a better idea about the shipping method.

You can put a message into  cart_no_shipping_available_html which will be added to the default WooCommerce message when there are no shipping methods available.

 

no.php – Transparent reverse proxy written in PHP that allows you to not have to write PHP any more

This little project will probably be my only contribution to the world of PHP.

The code is at https://github.com/michaelfranzl/no.php

This short, single-file, 80-line PHP script is a simple and fully transparent HTTP(S) reverse proxy written in PHP that allows you to never have to use PHP again for a new project, if you feel so inclined, for example if you are forced to host on a fully 3rd-party-managed server where you can’t do more than run PHP and upload files via FTP. The PHP script simply reads all requests from a browser pointed to it, forwards them (via PHP’s curl library) to a web application listening at another URL (e.g. on a more powerful, more secure, more private, or more capable server in a different data center), and returns the responses transparently and unmodified.

Supports:

  • Regular and XMLHttpRequests (AJAX)
  • All HTTP headers without discrimination
  • GET and POST verbs
  • Content types (HTTP payload) without discrimination
  • Redirects (internal redirects are rewritten to relative URIs)

Does not support (or not tested):

  • HTTP verbs other than GET and POST (but these are usually emulated anyway)
  • HTTP greater than version 1.1 (e.g. reusable connections)
  • Upgrade to websocket (persistent connections)
  • Multipart content type

Usage illustrated by the standard example

You have a non-PHP web application (called the “backend”) listening on https://myapp.backend.com:3000 but due to constraints you must make it available on a shared hosting server called https://example.com/subdir which only supports PHP and can’t be configured at all. On latter server, Apache (or Nginx, doesn’t matter) will usually do the following:

  1. If a URI points to a .php file, this file will be interpreted
  2. If a URI points to a file that is not existing, a 404 status will be returned.

Using no.php, to accomodate the second case, all URIs of the proxied web app (including static files) must be appended to the URI https://example.com/subdir/no.php. For example:

If your backend app supports that extra /subdir/no.php prefix to all paths, you are all set and ready to use no.php. Then:

  1. Simply copy no.php into the subdir directory of example.com
  2. Change $backend_url in no.php to "https://myapp.backend.com:3000"
  3. Point a browser to https://example.com/subdir/no.php

Project status

Experimental. Use only if you know what you are doing.

OpenGL programming in Python: pyglpainter

This was a recent hobby programming project of mine for use in a CNC application, using Python and OpenGL. The source code is available at https://github.com/michaelfranzl/pyglpainter .Simple OpenGL output using pyglpainter library

This Python module provides the class PainterWidget, extending PyQt5’s QGLWidget class with boilerplate code neccessary for applications which want to build a classical orthagnoal 3D world in which the user can interactively navigate with the mouse via the classical (and expected) Pan-Zoom-Rotate paradigm implemented via a virtual trackball (using quaternions for rotations).

This class is especially useful for technical visualizations in 3D space. It provides a simple Python API to draw raw OpenGL primitives (LINES, LINE_STRIP, TRIANGLES, etc.) as well as a number of useful composite primitives rendered by this class itself ( Grid, Star, CoordSystem, Text, etc., see files in classes/items). As a bonus, all objects/items can either be drawn as real 3D world entities which optionally support “billboard” mode (fully camera-aligned or arbitrary- axis aligned), or as a 2D overlay.

It uses the “modern”, shader-based, OpenGL API rather than the deprecated “fixed pipeline” and was developed for Python version 3 and Qt version 5.

Model, View and Projection matrices are calculated on the CPU, and then utilized in the GPU.

Qt has been chosen not only because it provides the GL environment but also vector, matrix and quaternion math. A port of this Python code into native Qt C++ is therefore trivial.

Look at example.py, part of this project, to see how this class can be used. If you need more functionality, consider subclassing.

Most of the time, calls to item_create() are enough to build a 3D world with interesting objects in it (the name for these objects here is “items”). Items can be rendered with different shaders.

This project was originally created for a CNC application, but then extracted from this application and made multi-purpose. The author believes it contains the simplest and shortest code to quickly utilize the basic and raw powers of OpenGL. To keep code simple and short, the project was optimized for technical, line- and triangle based primitives, not the realism that game engines strive for. The simple shaders included in this project will draw aliased lines and the output therefore will look more like computer graphics of the 80’s. But “modern” OpenGL offloads many things into shaders anyway.

This class can either be used for teaching purposes, experimentation, or as a visualization backend for production-class applications.

Mouse Navigation

Left Button drag left/right/up/down: Rotate camera left/right/up/down

Middle Button drag left/right/up/down: Move camera left/right/up/down

Wheel rotate up/down: Move camera ahead/back

Right Button drag up/down: Move camera ahead/back (same as wheel)

The FOV (Field of View) is held constant. “Zooming” is rather moving the camera forward alongs its look axis, which is more natural than changing the FOV of the camera. Even cameras in movies and TV series nowadays very, very rarely zoom.