Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
User Journal

Journal: 9 - systemD: Interfaces last longer than code 2

Journal by phantomfive

This entry discusses the benefits of good interfaces, then gives a (overly simple) metric for how to recognize them. Then it ties it back to systemd.

The 'unix way' is built around solid interfaces. The actual code to Unix is mostly gone (unless you use SCO or something), but the interface lives on with multiple (mostly compatible) implementations. The interface long outlasted the code. (NB: Unix hasn't always had good interfaces in every place, Unix isn't perfect.)

Another example of a good interface is the Berkeley networking API. It's not user-friendly, but as a low-level 'plumbing' API, it is extremely flexible and effective. It spread and now every OS has an implementation. (That is, every OS more complex than some stuff at http://wiki.osdev.org ). The original code is gone, but the interface will be around for a long, long time.

There are plenty of examples of interfaces in Unix. It doesn't have to be an API.

If you consider a programming language to be an interface between a computer and a human, then think of the C compiler. The original C compiler is no longer in use, but the interface was designed clearly enough that even intentionally bad IOCC code from 30 years ago still compiles.

Cron is another interface, as are the basic CLI commands like ls, mv, ps, grep, etc (which have now taken over almost everywhere....the interfaces, not the original code). The pipe is a powerful interface that seems obvious now, but wasn't at the time......the pipe makes it possible to take the output from any program and send it as input to any other program. That is impressive.

"Robert Metcalf [the inventor of Ethernet] says that if something comes along to replace Ethernet, it will be called Ethernet', so therefore Ethernet will never die. Unix has already undergone several such transformations." -- Ken Thompson

How do you recognize a good API? It's tough, but one thing is sure, a good interface allows easy swapping of components. If it doesn't allow easy swapping of components, it's not a good interface.

"Linux has always been multiple components that you can chose which one suits you best - whether its vi or emacs, gnome or kde, sendmail or postfix, apache or nginx, etc. This is a good thing, where you can swap out component A for B for any reason, and keeps the project competing with each other to get better and better." -- gbjbaanb.

When your component can be easily replaced, because the interfaces are good, it can only compete on its technical merits. That is how you will recognize the best interfaces, because when they are evaluated and put into use by skilled programmers, the best ones will come to the fore.

Time to wrap this back to systemd.

Systemd has gained traction as a init engine because the "unit file" interface provides benefits that a lot of people like. If the "unit file" interface proves durable, then it will last longer than the code. Interfaces can be immortal, code can not.

Throughout systemd there is a lack of understanding of proper interfaces. Making the GUI depend on a particular init system is a particularly obvious example of poor design, but the code was written from a 'code first' perspective rather than an 'interface first' perspective.

Lennart Poettering is a fine programmer. (His code is readable, and let's be honest, anyone who can hack on the Linux kernel has skill. Furthermore he is passionate about it). I don't think he really understands interfaces, though. When he does, his code will move up to next level.

User Journal

Journal: 8 - Does Gnome depend on systemd, or just logind? 7

Journal by phantomfive

The previous entry investigated Gnome and logind, and found that practically speaking, Gnome does depend on logind. There are some alternatives, but they are not complete or well-maintained.

This entry discusses the dependency between logind and systemd.

Logind is deeply embedded in systemd. Here is a list of logind features, which seem like a grab-bag of vaguely related requirements. If anything, their relation seems to be they are goals of GnomeOS. Maybe Poettering looked at that list and tried to implement as many as possible to convince Gnome to adopt his software.

At a minimum we can say the name 'logind' is a poor choice for a name, since it does much more than that: the name doesn't describe what the product does.

Logind depends on systemd (for example) for calling unit scripts to shutdown or put the computer in sleep mode. In logind-dbus.c function execute_shutdown_or_sleep() you can see an example. Systemd also seems to be required for detecting hot-plugged devices and probably for other stuff. Note also there is a clear dependency on DBUS.

Logind is described as a tiny daemon. That's a lie, the thing has 15491 lines, with the largest file having 3023 lines (3023 lines is large but not necessarily bad).

As always, if you find any errors, please let me know. I'm trying to find the truth, not start fights.

User Journal

Journal: 7 - Why does Gimp depend on systemd? 2

Journal by phantomfive

This post is flotsam related to some dependencies. If you note any mistakes, please let me know, I'm collecting information.

Gimp doesn't depend on systemd as far as I can tell. Gimp runs on Windows. It does depend on libgtk, whose dependency list is available.

There are reports that Gimp depends on systemd, but that seems to be an artifact of a package dependency tree on one system, not that Gimp actually needs systemd.

What about Gnome? Does Gnome depend on systemd? The answer is yes....., although there are alternatives. Systembsd, systemd-shim, and LoginKit.

LoginKit doesn't seem to be complete, like some files were missing from commit or something. The README is attractive, though. It seems to only be attempting to provide services for logind.

Systembsd provides hostenamed, localed, logind, and timedated.

Systemd-shim provides c-group services, some unit file services, power/sleep/reboot services, and ntpdate services.

Neither code base has been updated in four months, and the code in neither one looks particularly well organized.

I suspect there are other things systemd provides that Gnome relies on, but those aren't immediately apparent. I'm not sure where to look to figure that out.

Before logind, Gnome used ConsoleKit to provide login support. Fascinatingly, the commit to remove support for ConsoleKit was made by Florien Mullner.

A lot of this stuff depends directly on DBus. In my opinion that is a mistake; the communication mechanism should be separate from the API interface.

In the next journal entry I will investigate how deeply libsystemd is tied into the init system, or whether it is truly modular.

User Journal

Journal: Systemd (or, how to make portable software) 10

Journal by phantomfive

In this post, Lennart explains that he doesn't know how to write portable C code. He says:

[Making systemd portable] would turn every second line of systemd into #ifdefs.

The purpose of my post here is to explain how to write portable code, without #ifdefs on every other line. If that happens, you are doing it the hard way.

An easier way is to create an OS independent layer. Call it OSI (since no one is using that acronym anymore, right?). Create a header file that declares all your functions, then create an implementation file for each platform. Every time you find a piece of functionality that is different on different platforms, throw it into that header file.

For example, if you need to take a screenshot, it will be different on every platform you find. In the header file, create a function OSI_Screenshot(). Then in your OSI_Windows.c file, add an implementation for windows. In your OSI_Linux.c, add a screenshot implementation for Linux. In your OSI_OSX.c, add your implementation for OSX. This technique can also work for extremely incompatible platforms, with OSI_printf(), or OSI_malloc(), etc. If you use your compiler correctly, you can compile it without any extra overhead from the function call.

An example of this technique can be found in Android (look at sysdeps.h and sysdeps_win32.c), demonstrating that this technique works. They don't have #ifdefs scattered throughout their code. The Android example is tougher than it needs to be, because Android added the compatibility layer after the code was written. If they had started at the beginning, the code would have been much simpler.

The lack of knowledge (about how to write portable code) in the systemd team is causing them to make bad decisions, because the alternative seems too hard. For example:

I have no plans porting it to other kernels, and I will not merge any such patches........Quite frankly, I'd like to question [cross-platform compatibility]. In the light of GNOME OS I think we need to ask ourselves the question if we do ourselves any good if we continue to support all kinds of kernels that simply cannot keep up with Linux anymore.

Those who don't understand the failures of the past are destined to repeat them. Plenty of vendors try to focus on a single platform, and their work disappeared, to be replaced by something that was cross-platform compatible, and usually better work. The discipline required to keep things portable results in better code (there's a quote from Mythical Man Month that fits here but I'm too lazy to look it up). The Gnome developers understand the importance of portability but that's a story for another post.

User Journal

Journal: Systemd's solution to large init scripts 1

Journal by phantomfive

(Note: When you write code, you're making a UI for programmers. Learn to do it well.)

Below you can see a traditional unix init script. It's long, but if you're familiar with shell-script you can figure out what is going on. There is a lot of redundancy here, most init scripts have a switch that runs an option based on the first command-line parameter, for example. One solution is to put common code in a function, but Poettering decided to use config files.

Let's examine the general tradeoff between putting code in a function, and using config files (a form of declarative programming). Config files are fine as long as there aren't too many special cases. If there are too many special cases, you end up with so many options and keywords that it would have been easier to just use a scripting language.

The good side is systemd saves a lot of typing. Way down at the bottom, is a unit file for the same init script. It's clearly shorter, and easier to type.

The bad side is it has arcane keywords which are are not discoverable merely by looking at the file. This is a pattern that repeats itself over and over in systemd, things are easier if you know how to do them, but the system itself is inscrutable without arcane knowledge.

Ideal systems fulfill the requirements while making it easy for those who want to dig deeper. The system opens like the petals of a rose.

#!/bin/bash
# Starts the abrt daemon
#
# chkconfig: 35 82 16
# description: Daemon to detect crashing apps
# processname: abrtd
### BEGIN INIT INFO
# Provides: abrt
# Required-Start: $syslog $local_fs
# Required-Stop: $syslog $local_fs
# Default-Stop: 0 1 2 6
# Default-Start: 3 5
# Short-Description: start and stop abrt daemon
# Description: Listen to and dispatch crash events
### END INIT INFO
 
# Source function library.
. /etc/rc.d/init.d/functions
ABRT_BIN="/usr/sbin/abrtd"
LOCK="/var/lock/subsys/abrtd"
OLD_LOCK="/var/lock/subsys/abrt"
RETVAL=0
 
#
# Set these variables if you are behind proxy
#
#export http_proxy=
#export https_proxy=
 
#
# See how we were called.
#
 
check() {
    # Check that we're a privileged user
    [ "`id -u`" = 0 ] || exit 4
 
    # Check if abrt is executable
    test -x $ABRT_BIN || exit 5
}
 
start() {
 
    check
 
    # Check if it is already running
    if [ ! -f $LOCK ] && [ ! -f $OLD_LOCK ]; then
        echo -n $"Starting abrt daemon: "
        daemon $ABRT_BIN
        RETVAL=$?
        [ $RETVAL -eq 0 ] && touch $LOCK
        echo
    fi
    return $RETVAL
}
 
stop() {
 
    check
 
    echo -n $"Stopping abrt daemon: "
    killproc $ABRT_BIN
    RETVAL=$?
    [ $RETVAL -eq 0 ] && rm -f $LOCK
    [ $RETVAL -eq 0 ] && rm -f $OLD_LOCK
    echo
    return $RETVAL
}
 
restart() {
    stop
    start
}
 
reload() {
    restart
}
 
case "$1" in
start)
    start
;;
stop)
    stop
;;
reload)
    reload
;;
force-reload)
    echo "$0: Unimplemented feature."
    RETVAL=3
;;
restart)
    restart
;;
condrestart)
    if [ -f $LOCK ]; then
        restart
    fi
    # update from older version
    if [ -f $OLD_LOCK ]; then
        restart
    fi
;;
status)
    status abrtd
    RETVAL=$?
;;
*)
    echo $"Usage: $0 {start|stop|status|restart|condrestart|reload|force-reload}"
    RETVAL=2
esac
 
exit $RETVAL

-----------------------------------------------------------------------------------------

[Unit]
Description=Daemon to detect crashing apps
After=syslog.target
 
[Service]
ExecStart=/usr/sbin/abrtd
Type=forking
 
[Install]
WantedBy=multi-user.target

Reference to the examples.

User Journal

Journal: systemd - Why did Debian Adopt it? 8

Journal by phantomfive

There's a Debian debate page, but it's disappointing and everything systemd does is listed with equal value. Thanks to Russ Albery for making a much more balanced assessment, explaining what he likes. The short answer to the question is: SystemD makes things much easier for people writing init scripts. It wasn't about cgroups, or speed, or login managers, it was about writing easy init scripts.

Here are the major complaints he has with the traditional startup system:

* Lack of integration with kernel-level events to properly order startup.
* No mechanism for process monitoring and restarting beyond inittab.
* Heavy reliance on shell scripting rather than declarative syntax.
* A fork and exit with PID file model for daemon startup.

He furthermore points out these problems with startup scripts:

The model of fork and exit without clear synchronization points is inherently racy, the boot model encoded into sysvinit doesn't reflect a modern system boot, and maintaining large and complex init scripts as conffiles has been painful for years. Nearly every init script, including the ones in my own packages, have various edge-case bugs or problems because it's very hard to write robust service startup in shell, even with the excellent helper programs and shell libraries that Debian has available.

Those are the main things that systemd fixes for a distro builder, and probably why so many distros have switched to systemd, because it was built for them.

User Journal

Journal: systemd - A descendent of Apple's LaunchD 1

Journal by phantomfive

Systemd is a descendent of Apple's launchd, Lennart suggested the world watch this movie to learn about it, so why? What is good about launchd?

OSX has an excellent inter-processing message system, part of the mach kernel (which they got from CMU). Because of this, many times processes talk to each other through the standard messaging queues. If you use a library to launch a browser window, or query wifi strength, it will communicate through a message queue (although the library takes care of the details).

The interesting thing is that launchd can open a message queue before the recipient is running. So launchd doesn't launch services until they are needed, and if they aren't needed, they never get launched. The benefit is that resources are preserved, things don't get launched until someone sends a message.

The benefit here is dependency resolution. Instead of forcing a deep calculation of what depends on what, or forcing the server script to declare everything it depends on, you can say "manage all these thousand services!" and only the few that are needed will be used, when they are needed.

Another thing I think Lennart copied from launchd is the declarative config files. You don't need to write a shell script, you merely need to indicate what should be launched, and maybe give it configuration options like "relaunch on failure" or "launch completely before receiving message." Of course, some people like config files, others prefer scripts.....that's not an argument I want to get into here, I'm merely pointing out things that seem to have come from launchd. (

What is bad:

Launchd is utterly undiscoverable. It would have been nice of them to put a README in /etc/rc/ or something saying, "hey, all your startup scripts are gone, look elsewhere."

Launchd has a complex hierarchy of directories that it searches for startup scripts. They have justifications for why, and a reason behind each one.......but justifications and reasons are not the same as good design. Every bad design decision ever made had a justification and a reason.

User Journal

Journal: Systemd - Efficiency and Pike's Rules of Programming 4

Journal by phantomfive

(DISCLAIMER: These are my notes and might be wrong.)

Lennart Poettering has never heard of Rob Pike's rules of programming. Let's examine why:

He said, "On my system the scripts in /etc/init.d call grep at least 77 times.... that has to be incredibly slow...., and should be rewritten in C."

You can time it yourself, with this command:

time for i in {1..77}; do grep user /etc/passwd > /dev/null; done

On my machine it takes 0.229 seconds.

Lennart didn't get the speed increase he wanted. From his benchmark report, "booting up with Upstart takes 27s, with systemd we reach 24s." This result isn't too bad: when I've made a similar mistake of writing code before timing anything, ignoring Pike's rules for programming, once I actually slowed the system down.

If you don't measure, you can't optimize.

For entertainment and enlightenment, here is a quote by Ken Thompson talking vaguely about Pike's rules for programming:

Early Unix programmers became good at modularity because they had to be. An OS is one of the most complicated pieces of code around. If it is not well structured, it will fall apart. There were a couple of early failures at building Unix that were scrapped. One can blame the early (structureless) C for this, but basically it was because the OS was too complicated to write. We needed both refinements in tools (like C structures) and good practice in using them (like Rob Pike's rules for programming) before we could tame that complexity.

User Journal

Journal: SystemD: The Beginning 7

Journal by phantomfive

DISCLAIMER: THIS CODE REVIEW IS A LONG WORK IN PROGRESS, I COULD BE COMPLETELY WRONG IN ANYTHING I SAY.

To do a proper code review, you need to understand the purpose of the code, what all the stakeholders want. From my own perspective, init scripts work fine, but since Unix companies keep trying to create new init systems, they must have different needs than I do.

Here's a list of the stakeholders. I need to figure out what their goals are.
1) System admins.
2) Desktop users.
3) Distro builders.
5) Android (if systemd turns out to be good enough).
4) Programmers

My suspicion is that systemd has taken over because it makes things easier for 3.

At its core, Unix is a system for programmers. What other system comes with a compiler and multiple interpreters by default? Bash is so much more useable than DOS, or even powershell (yeah, go ahead and flame me but I'm right, Powershell doesn't even have < working). Unix was designed by programmers and for programmers.

The reason I'm talking about it is the traditional init process is incredibly discoverable. All you have to do is look in the /etc/rc directories, and you can figure out how your system boots. You can see it all. For a programmer it's easy. Poking around in the system to understand it is one thing that makes Unix great (and what I like about Slackware: the whole thing is lovingly crafted).

So that describes the approach I am taking to code review, and to the init process. Hopefully Systemd is an improvement.

Dead? No excuse for laying off work.

Working...