This is the first post to this example blog. To add new posts, just add files to the posts/ subdirectory, or use the web form.

Blog via IkiWiki

(Blog software announcement 2019-1)

Today I set up a first test blog using ikiwiki.

Maybe I can even edit this from the web.

For reference, find a list of related articles embedded here:

Blog software announcements 2019-n

1. Blog via IkiWiki - Setup of blog (software)
2. Modifying IkiWiki - Local modifications to blog software
3. ikiwiki upgraded to version in upcoming Debian 10

A test from mobile!

Hello, there.

• It's week-end.
• The thing is finally working.
• It's raining outside, but there are predictions of snow - at slow beginning of summer... :-S

... and a few minutes-to-hours later, it's shining bright! Though the month of April is over, isn't it?

Feed experiments

Recently, I've been experimenting with (e.g. RSS or Atom) "feeds"; it's quite a nice technology which I've largely ignored while consuming. (I've had problems getting feeds to work on the "Joomla!" CMS including the photos from a gallery plug-in -- only the processing directives for the plug-in turned up -- but that's rather on the producing side.)

With this technology, the computer/program can process items (articles, comic strips, or also just any kind of repeating or automated information that may be useful to someone regularly?) from a web-site piece-by-piece (instead of web page-wise). This enables different possibilities, from sending as emails (ugh! want to get away from emails, not get even more of them!) to live-showing as Desktop notifications; but the most popular application maybe is the "Feed Reader".

Currently, I'm experimenting with the "Feedbro" Firefox web-browser extension on the Desktop/Notebook/PC, and the "Feeder" Android-App from the f-droid repository of free and open-source software, on mobile. I'm trying to have only few/lightweight feeds on the mobile app, as it's easy to get overwhelmed with information while using a feed reader, and may be impossible (or undesirable) to catch up, after missing out for a few days (as also suggested by a friend who tried using an RSS reader multiple times but ended up never looking at it again, due to this problem).

I'm not sure if I can keep up with what I've currently configured in the Desktop web browser, especially as some automated news items keep coming up again and again; though this may be a problem particular to Feedbro which I'm using there.

Here are some possibly-interesting feeds I've collected for the experimenting:

• Humour:
  • https://m.xkcd.com/rss.xml (from https://xkcd.com/)
• Debian:
  • "Planet Debian", a blog aggregator of Debian Developers: https://planet.debian.org/rss20.xml (see https://planet.debian.org/, linked from the Debian website)
  • "Debian micronews RSS Feed" https://micronews.debian.org/feeds/feed.rss (see https://micronews.debian.org/ / via the Debian blog from the Debian website)
  • "Debian News" https://www.debian.org/News/news.en; be sure to append .en to the advertised .../news URL to always get the untranslated / english original news, even if your browser configuration asks for a different language by default. (from the Debian website)
  • "Debian Security Advisories (summaries)" https://www.debian.org/security/dsa-long (from the Debian website, too)
• Oh dear, there are no other concrete resources left that I'd have liked to give as examples... (after eliminating what I'd not recommend to try (again))

• Some more suggestions:

  • Log in to your GitHub profile and subscribe to your "private feed"; it should be something like: https://github.com/ACCOUNTNAME.private.atom?token=...

  • Subscribe to your own website/blog/... (if it got a feed) to learn about, e.g., comments in a timely manner. Maybe also the sites you're an admin for.

  • If you care about a specific Debian package's evolution, perhaps due to having contributed to it, you can subscribe to its Debian package news,
    e.g., https://tracker.debian.org/pkg/PACKAGENAME/rss
    (via https://tracker.debian.org/pkg/PACKAGENAME).

    Be warned, though, that at least Feedbro (see above) sees the frequent automatic updates to automatically generated "action items" as a completely new post, all the time. It might therefore help to add an early, non-fallthrough rule to automatically mark as read everything whose article URL starts with https://tracker.debian.org/action-items/, or alternatively match against specific action-items instances by number.

  • Maybe subscribe to your favourite (or most recently discovered) Free Software project's news feed. It may be interesting, it may also be annoying, though ...

  • Or subscribe to the (e.g. GitHub) commits feed of your friend's long forgotten project; maybe it'll keep you up-to-date when/if it should become active again, or you'll have a nice test for how, e.g., Feedbro displays hundreds/thousands of days of inactivity in the feed stats...

This post has moved here.

Modifying IkiWiki

(Blog software announcement 2019-2)

It's incredible how flexible IkiWiki is.

For changing the page output, you can create a templates directory and copy your ikiwiki installation's page.tmpl to there - that's it! This will override the template and you can start making changes to it. (That's how I got the additional CTIME/MTIME lines added, where most of the time only one of them was provided so it felt for me as if information was missing when looking only at a single page...)

For changing code (at least as long as it's part of a plugin), you can set a libdir in the config, set up the required subdirectories IkiWiki/Plugin, copy your ikiwiki installation's comments.pm to there - and it'll override the installed plugin, so you can start to make live code changes.

Local avatars in comments

--- IkiWiki/Plugin/comments-20190509-1-ikiwiki_3_20141016_4_deb8u1.pm.bak  2017-01-11 19:18:52.000000000 +0100
+++ IkiWiki/Plugin/comments.pm 2019-05-10 04:25:09.364000000 +0200
@@ -641,8 +641,20 @@
    my $user=shift;
    return undef unless defined $user;
 
    my $avatar;
+
+  # Try using a locally hosted avatar, first.
+  if ($user !~ m#^(?:\.|\.\.|.*/.*)$#) {
+      foreach my $testuri (
+          '/avatar/'.$user.'.png',
+          '/avatar/'.$user.'.jpg'
+      ) {
+              return $testuri if -f $config{destdir}.$testuri;
+      }
+  }
+
+  # Only then embed external resources.
    eval q{use Libravatar::URL};
    if (! $@) {
        my $oiduser = eval { IkiWiki::openiduser($user) };
        my $https=defined $config{url} && $config{url}=~/^https:/;

View raw file

For reference, find a list of related articles embedded here:

Blog software announcements 2019-n

1. Blog via IkiWiki - Setup of blog (software)
2. Modifying IkiWiki - Local modifications to blog software
3. ikiwiki upgraded to version in upcoming Debian 10

ARM legacy system alongside arm64 replacement

Table of Contents

• General situation
• More specific situation
• The deprecated ARM instructions
  • Research
• Running mono and the syscall filter
  • The problem
• Conclusion

General situation

For some reason or another, you might have a legacy system that should theoretically be replaced by a newer system, especially when changing hardware/architecture anyway -- but you want to keep it running all the same. Or even need to, as it might be infrastructure for other hosts (e.g., needed to build Debian packages for them), which a newer system couldn't accomplish by itself.

Luckily, we're living in the future already, and can use many kinds of different technology to let the old system live on in a new one -- be it virtual machines (VMs; accelerated by hardware support like Linux KVM, or plain qemu without acceleration enabled), Linux Containers (LXC; the standalone one or that one provided by libvirt …), systemd-nspawn (chroot on steroids; possibly used via systemd-machined and machinectl), or anything else that you can imagine.

More specific situation

Years ago, I put my previous, 32-bit home server into an LXC container on the new, 64-bit system. Except for that it's still running although I wanted to retire it one day, it's working great. This was on amd64 (aka x86_64, or today simply "PC", as in Personal Computer) hardware.

So now I wanted to do mostly the same thing with a Raspbian armhf (meant for running on Raspberry Pi) development environment (32-bit) on a "new" RPi 3B+, which got Debian (the real thing) buster (upcoming Debian 10) arm64 (64-bit) running on it. One thing, I was astonished that it could do such a thing, being ARM-based and not PC, and all. Another thing, it was not such a smooth ride as on PC hardware!

The deprecated ARM instructions

So I set up the outer system based on a preview image (created by Gunnar Wolf), and copied the previous development environment to /var/lib/machines/devenv-raspbian. cd to there, do a quick test:

root@devenv-arm64:/var/lib/machines/devenv-raspbian# chroot . bin/bash
root@devenv-arm64:/# ls
bin  boot  dev  etc  [...]
root@devenv-arm64:/#

Some things to note here:

1. It works.

2. It spams the dmesg/journal/syslog with messages such as:

    Apr 25 14:17:38 devenv-arm64 kernel: "bash" (2612) uses deprecated CP15 Barrier instruction at 0xf7e88b50
    Apr 25 14:17:38 devenv-arm64 kernel: "bash" (2612) uses deprecated CP15 Barrier instruction at 0xf7e88b50
    Apr 25 14:17:38 devenv-arm64 kernel: "bash" (2612) uses deprecated CP15 Barrier instruction at 0xf7e88b50
    [...]
    Apr 25 14:17:39 devenv-arm64 kernel: "bash" (2612) uses deprecated setend instruction at 0xf7e666f4
    Apr 25 14:17:39 devenv-arm64 kernel: "bash" (2612) uses deprecated setend instruction at 0xf7e66ca4
    [...]
    Apr 25 14:18:05 devenv-arm64 kernel: cp15barrier_handler: 143 callbacks suppressed
    Apr 25 14:18:05 devenv-arm64 kernel: "bash" (2612) uses deprecated CP15 Barrier instruction at 0xf7d032b8
    Apr 25 14:18:05 devenv-arm64 kernel: "bash" (2612) uses deprecated CP15 Barrier instruction at 0xf7d03394
    Apr 25 14:18:05 devenv-arm64 kernel: "bash" (2612) uses deprecated setend instruction at 0xf7e666f4
    Apr 25 14:18:05 devenv-arm64 kernel: "bash" (2612) uses deprecated setend instruction at 0xf7e66bd8
    [...]
    Apr 25 14:18:17 devenv-arm64 kernel: "ls" (2613) uses deprecated CP15 Barrier instruction at 0xf7cbfb50
    Apr 25 14:18:17 devenv-arm64 kernel: "ls" (2613) uses deprecated CP15 Barrier instruction at 0xf7cbfb50
    Apr 25 14:18:17 devenv-arm64 kernel: compat_setend_handler: 2 callbacks suppressed
    Apr 25 14:18:17 devenv-arm64 kernel: "bash" (2612) uses deprecated setend instruction at 0xf7e666f4
    Apr 25 14:18:17 devenv-arm64 kernel: "bash" (2612) uses deprecated setend instruction at 0xf7e66ca4
    [...]
   

Point 2 nearly overshadowed point 1, here. Luckily, it was all easily fixed by doing:

root@devenv-arm64:~# echo 2 >/proc/sys/abi/setend
root@devenv-arm64:~# echo 2 >/proc/sys/abi/cp15_barrier

Apr 25 15:03:19 devenv-arm64 kernel: Removed setend emulation handler
Apr 25 15:03:19 devenv-arm64 kernel: Enabled setend support
Apr 25 15:03:40 devenv-arm64 kernel: Removed cp15_barrier emulation handler
Apr 25 15:03:40 devenv-arm64 kernel: Enabled cp15_barrier support

As you may guess from the dmesg messages, this didn't just silence the warnings, but did something different: It actually turned off the in-kernel support for those deprecated ARM instructions, and moved responsibility for supporting the functionality to the bare hardware. Luckily, our Raspberry Pi 3B+ is backwards-compatible to running an ARM userland compiled for the more limited Raspberry Pi 1, and still has the deprecated instructions implemented even in aarch64 mode running a 32bit personality. But if we would have liked to run an arbitrary 32bit ARM userland on an arbitrary 64bit ARM CPU running in 64bit mode, this could have been a show-stopper, here.

Research

Some details of my research into this weird situation: Initial suggestion for using sysctls abi.* found, given by someone at linaro in 2017; reference to previous discussion from 2014 made by someone at Arm Ltd in the same thread in 2017; proposed timeline for ARM instruction deprecation by someone else at Arm Ltd in 2014. There, it says how it's all meant to work together for the goal of finally getting rid of certain instructions.

As a side note, those emulation warnings are coming up for other people doing slightly similar things (though in a much more professional way) as well; e.g., directhex wrangling with automated build infrastructure. In section "When is a superset not a superset", it says:

CP15 memory barrier emulation is slow. My friend Vince Sanders, who helped with some of this analysis, suggested a cost of order 1000 cycles per emulated call. How many was I looking at? According to dmesg, about a million per second.

I guess that number has been taken by looking at the result of the emulation warning dmesg rate-limiting. Let's hope it wasn't millions of log lines, instead …

Running mono and the syscall filter

Having just a chroot would be lame, nowadays; as you can easily boot another operating system (as LXC or systemd-nspawn or even docker container, as long as it's a Linux distribution and/or copes with using the same kernel as the host system; or, otherwise, in a virtual machine, e.g. based on qemu and kvm). For this situation, where I thought the guest system would cope with the newer, foreign-architecture kernel of the host system with the deprecated ARM instructions put out of our way, I was going to use systemd-nspawn container.

So for our previous /var/lib/machines/devenv-raspbian, I set up a companion configuration file /etc/systemd/nspawn/devenv-raspbian.nspawn (for unsetting some systemd-nspawn defaults when used from systemd-machined, and to set up networking), which apparently already got used when I cd'd to the machine(/container) root directory and issued a simple:

root@devenv-arm64:/var/lib/machines/devenv-raspbian# systemd-nspawn
root@devenv-raspbian:~#

Again, some things to note:

1. It works.
2. This time, we're not inside a simple chroot, but in a container already, with the uts namespace unshared and the hostname set according to the container name. (No tricks with /etc/debian_chroot and a PS1 which uses it needed!)
3. It even picked up our inner root's home directory, and ended up there instead of in the container root directory.

With this running I tried the next step of booting into the container operating system. For the most part, it was a simple:

root@devenv-arm64:~# machinectl start devenv-raspbian

This gave the guest's boot messages in the host's journal. When it's ready, a simple machinectl without arguments, from an unprivileged user, says what OS/version the container OS is, and what IP address is being used -- an information which is readily available as it's a container, not a VM.

When all works out well, you can enable the container for start on next (host) boot:

root@devenv-arm64:~# machinectl enable devenv-raspbian

So far, so good.

The problem

After some time I realized everything mono (.NET) was instantly failing at program start. Initial thoughts were on maybe another missing instruction, at the hardware/kernel level; that wasn't the case, though. So I ran the thing through strace.. and noticed loads of cacheflush() failing with EPERM. After some fruitless web search into the Linux source, it appeared to me as if this could be part of some restriction placed upon us by the systemd-nspawn containering. -- A quick test running the csharp interactive C# shell just inside a chroot instead of a systemd-nspawn container succeeded: (Transcript from brain memory.)

root@devenv-arm64:/var/lib/machines/devenv-raspbian# chroot . bin/bash
root@devenv-arm64:/# csharp
[some message about mono needing /proc]
root@devenv-arm64:/# mount -t proc proc proc
root@devenv-arm64:/# csharp
[some assertion not met, seems to explode much as before]
root@devenv-arm64:/# linux32 csharp
[works]
root@devenv-arm64:/# umount proc  # Don't forget; or systemd-nspawn
                                  # later will error out when started
                                  # from "machinectl start devenv-raspbian".

So it was able to work, just wouldn't with systemd-nspawn.

Finally, I found out about systemd-nspawn having an automatic system-call-filter implemented as a white-list. So this was where the cacheflush() EPERM was coming from! Additionally to forcing the personality to "arm" (32-bit) in the .nspawn file mentioned above, we need to put cacheflush into the syscall-filter white-list. To avoid trial-and-error until the minimum necessary would be found, and as our containerization was for the old and new system to coexist -- and not to de-root the system being used as container --, I assumed that cacheflush was missing from the white-list due to the fact that it's being ARM-private, and simply put every ARM-private syscall there that I could find in an armhf header file. (Most probably what I was looking at was asm/unistd.h from the Linux kernel sources.) Specifically, that were: breakpoint cacheflush usr26 usr32 set_tls. (TODO: Report as issue / wishlist bug on systemd and link that here?)

Power down the container, start it again (remember to umount manually mounted proc first), et voila: mono runtime-based software from the old development environment was running, too!

Conclusion

It is possible to run an older Raspberry Pi compatible OS as a container inside a newer such OS on a newer hardware version of the Raspberry Pi. It's just unfortunate to seem impossible in practice at first; with both the deprecated ARM instructions filling up the journal, and mono runtime-based software of the older OS not running looking like a show-stopper. These problems can be overcome, though, which I hope to be able to communicate with this blog post.

streamserver-cvn runs on my smartphone!

As of 2019-05-11, streamserver-cvn, my personal project to help a friend fill in a missing part in his Desktop streaming, now runs on my smartphone, too! (In the GNU/Linux environment inside the Termux Android App, that is...)

As I've already written up the details as project documentation, please see there; or visit the master README.md and/or master BUILDING.md directly.

The essence is to use "qmake QMAKE_CXXFLAGS+=--target=armv7l-linux-android ..." for building, as the __ARM_ARCH defaults to 4, otherwise, while qt5 only supports 5-8.

A nice trick to get MPEG-TS (MPEG Transport Stream) compatible input for streamserver-cvn-cli is to use "mpv ... --o=- --of=mpegts" (ships via the default repos of the Termux App), possibly piped into it. (For getting something that actually plays on Kodi on the Raspberry Pi, it may need much more flags, though. What flags exactly is left as an exercise to the reader. ...)

Problems

The idea was to be able to stream some smartphone photos to the TV (via Kodi on the Raspberry Pi, as suggested above), though. This doesn't seem to work (yet?) as mpv seemed to set a frame rate of 24000 fps... It seems there is a /1001 missing, somewhere, but when I used some flags to set the time base to "25000/1000", each image was taking around 20-25 s, but Kodi hung on the stream, anyway. Perhaps I should have tried to set "1000/1000"... But can't test that, at the moment. It would be possible to try again using ffmpeg, too, instead.

ikiwiki upgraded

(Blog software announcement 2019-3)

I've finally upgraded ikiwiki from what was in Debian 8, to what is expected to be in upcoming Debian 10!

Sadly (or reassuringly), that isn't a giant leap forward as the time frame involved might have suggested; it brings nice things, though, such as basic HTML5 by default, plus mobile responsive layout based on it... That means I can finally read my blog in my smartphone's Firefox, and expect something readable. (That's why I'm typing the base text of this on my smartphone, but the webform hasn't got much responsiveness added, or I'm doing something wrong...)

(Created Sun 19 May 2019 01:59:51 CEST, published around Sun 19 May 2019 17:04:04 CEST.)

For reference, find a list of related articles embedded here:

Blog software announcements 2019-n

1. Blog via IkiWiki - Setup of blog (software)
2. Modifying IkiWiki - Local modifications to blog software
3. ikiwiki upgraded to version in upcoming Debian 10

TIL: git history visualisation via gource

Today I learned there is a tool out there, or rather part of Debian already¹, which helps getting an intuitive understanding of a git repository's history, by means of generating an interactive visualisation! (This can be exported as a video, too, but I didn't test that.)

Thanks, Simon, for pointing out gource exists!

gource on canvon-blog

Letting gource run on this blog's git history was particularly interesting (N.B.: the git repos isn't published (yet?), though); I saw multiple entities "fighting it out" between each-other, as pages were added by my git identity and tag pages created by the wiki's. See the (in comparison: boring) end result here:

gource on streamserver-cvn

Another way I've tested gource is by letting it run on my streamserver-cvn project's git repos; this was a bit boring, though, as there is only one author (myself), most of the time...

Operating gource

It seems as though you can just cd into your worktree (N.B.: bare repository doesn't work as of gource 0.44-1+b3 (Debian), whether running with GIT_DIR=. or not.) and run gource, without any additional command-line arguments, and expect it running.

A few points to note, though:

• Seeking seems to be possible by hovering with the mouse pointer over the lower part of the window/screen so that a seekbar appears, then click where you want to go; as of gource 0.44-1+b3 (Debian; again), it'll start from there, though, so will arrive at a different result than when left running from the beginning of git history...

• Pressing q or Ctrl-Q won't quit the application, but Esc does.

(Created Mon 20 May 2019 19:51:19 CEST, published around Tue 21 May 2019 00:58:42 CEST.)