I thought I had made it easy to understand, but let me repeat and rephrase; if you for some reason have a user case for using text logs and therefore use rsyslog or similar, you get a slightly more complicated system since journald is now part of the logging process; but, you also get extra functionality, since journald can provide logging info that stand alone syslog systems can't.

Or to rephrase yet another time; you don't just get the old functionality back by combining systemd and syslog, you get more functionality.

But please, don't use systemd if you don't want to, I don't care at all, as long as you systemd haters fixes your own problems like ConsoleKit and udev, instead of attacking everyone else for not doing the work you ought to do.

I don't understand what you mean by "doesn't feed logs back upstream". Rsyslog or similar behave just as they always have.

You get a more complex system in order to get new features, in this case; journald+syslog=more logging info than syslog alone.

The above solution is only necessary for crazy legacy setups and log sinks, and people who doesn't yet know how good the journal actually is.

I don't think there are real user cases beyond that. Text logs are simply obsolete on Linux now, since the journal is so superior and can do everything that old text logs do, and much, much more on top of that.

journald=heaps more logging info than old stand alone syslog.
journald=still more logging info than journald+syslog.

Thinking that text logs was the best solution, turned out just to be a wrong concept. The gains of choosing a slightly more complex binary format, turned out to vastly more than the tiny problem of needing a special reader.

So drop your worries and try out systemd; give it a good workout, I think you end up liking it.

journald delivers ascii logs in real time, you just have to use a syslog daemon like you always have anyway.

It would be a stupid function duplication if systemd's journald also could partially act as syslog daemon. While outputting text is easy enough, you would also have to have rate-limiting, log-rotation etc. And it still wouldn't replace a classic syslog daemon because it couldn't be used as a log sink, and once you start to add log sink features, you end up with database drivers in the code and all sorts of filters.

The number one design goal for "journald" was "simplicity". Adding syslog features like text output is contrary to this. (unfortunately many people request such features, but the systemd developers have said no every time).

journald is made to be used in conjunction with syslog and actually enhances it because of its features, the journald simply works as a helper process for syslog if desired. Nothing is taken away as things were before systemd, real time text logs is possible. Nobody is forced to use the binary log files when using systemd.

Those who don't need text log files for legacy reasons, or a log sink, can skip the syslog daemon and rejoice in the power of a structured and indexed log file and just use journald. I think over time, this is what most people will, once they "get" the journal: IMHO, it really is logging done right on Linux, something people have tried to improve with only limited success the last decades.

I don't think it is really possible to make a good hybrid log-file, where some parts are a classic text dump, while the meta data and index is in binary. I think it would a complex and fragile solution. It would be much easier let the classic text tools like "grep" learn to read to binary journal directly (jgrep?). But it really isn't needed; a journal reader like "journalctl" combined with piping solves all such problems, so using the classic text tools on the binary journald is quite possible to do in real time.

Here is an example below: lets assume it is a bash script run by cron every 1 hour: it searches the journal for messages of log level "error" that contains the string "softreset".
"journalctl -p err | grep softreset"

As you can see, it isn't a problem using "grep" or "tee" or "sed" or any other classic text GNU tool together with the journal, to do real time inquiries or using them in bash scripts etc.

Because if you want first class service and process management, you need first class logging, and this is exactly what journald does.

You could never do what journald does with classic rsyslog, like living and logging while in initramfs, and then jump over to the main system when the root-fs was mounted, and continue logging while the system booted.

Anyway, that would also restrict systemd logging to a certain syslog daemon. The present solution is a general solution that works with all syslog implementation like syslog-ng and rsyslog etc.

The awesome structured and indexed log file format has a stable API and structure, so it is quite easy for other syslog implementations to read and write in that format. I believe rsyslog have already implemented a reader and some other journald features. So if they want, they too can have all the advantages of rich metadata.

Logging isn't done by or in PID1. It is done by the "journald" daemon that have its own pid. I must say, that whoever telling you about how systemd is designed and works, doesn't have a clue what they are talking about. I strongly suggest you check out the systemd homepage for some basic information about systemd.
http://www.freedesktop.org/wik...

People are hard to satisfy: some complain that systemd replaces daemons, and now you complain about it _doesn't_ replace syslog for writing text logs?

People who wants simple text-logs can still use rsyslog or whatever they have always used, and have "journald" forward all the messages to it. That way rsyslog can gain some of the systemd benefits like early boot logging, though not them all (rich meta data like monotonic stamps and whatnot). All in all you would be better off than simply using SysVinit and pure rsyslog.

Even without syslog, it is trivial to convert systemd's journal to text. You can even export it as JSON to preserve the rich meta data.

I really think the systemd developers did their homework very well with the design of the logging system. The more I use it, the more I like it. There is no going back for me to plain text logs.

The point is that most user processes doesn't communicate directly with the kernel, but with a cgroup manager using a higher level API, like systemd or cgmanager. So while the kernel works fine, the "communication link" or the mediator between processes and the kernel would be severed if the cgroup manager in PID2 died. That is highly likely to lead to an unstable system.

Anyway, looking at the design of systemd, you will see that it actually runs very little code in PID1, it just communicate via PID1 to the other helper processes. This enables PID1, the init part, to have total control and supervision of all processes. Making e.g. the cgroup manager run in PID2, wouldn't gain anything, but increase complexity, since there would need to be intense communication between PID1 and PID2 in order for the init system to use cgroup to track all processes.

I think the whole PID1 debate is based, not on hard facts or good analysis, but on a hatred against systemd. People seems to rely on random hate blogs on the net for what systemd is, instead of examining the systemd project for themselves.

Systemd's PID1 is actually rather small, especially compared to what functions it provides. Moving some core bits out of PID1 and place them in PID2 will just increase complexity with little if any gain.

The reality is, that the systemd designers made some good choices when making systemd. Personally I think it is hopeless for all the simple script based init-systems to have real feature parity with systemd, unless they make some of the same design decisions.

logind isn't a feature with "systemd" (the daemon), but with systemd (the project). "logind" is a consumer of the systemd-daemon's internal API however. That means you can use cgroups and other kernel and systemd features in user sessions. This is how eg. secure root-less X.org is possible with systemd.

I think many peoples idea what init is, have been clouded by the fact that traditional Linux init systems were so primitive. Certified Unix'en like Solaris and Mac OSX have abandoned crude Linux like init systems years ago.

Booting and initialization of a system is quite complex on modern OS's, so doing it by modules that aren't coordinated and aren't developed with the other modules in mind, really limits what the OS can do. Having modules like systemd's, that are designed to talk to each other, all other processes and the kernel, and is developed in a coherent manner, really can remove some old limits on how Linux works. Basic things like conferring "namespaces" and "capabilities" from the kernel to a service, just by adding a simple keyword in the service config file, shows the potential. But multi-seat computing, stateless booting and "zero config" boots are either realized already or being worked upon. With kdbus in the kernel, secure OS containers from basically unmodified Linux distros, will become a reality.

I also think it is good, that systemd now will reduce Linux fragmentation at some level at least, and freeing distro maintainers and developers from a lot of non-fun work like maintaining and debugging init-scripts, while making cross distro collaboration on e.g. extra hardened "Unit" files (service configs) possible.

Here is a youtube video where Lennart Poettering talks about why systemd goes beyond a simple init system:
https://www.youtube.com/watch?...

I strongly recommend reading as many relevant sections from this site:
http://www.freedesktop.org/wik...

There are also some design documents (like this old one about the journal)
https://docs.google.com/docume...

And of course more youtube talks from Fosdem etc:
https://www.youtube.com/watch?...

systemd's logging facilities are superior to old syslog in several ways. Fx. systemd and journald lives in initramfs while the system is booting, so it can collect logging info from before the root fs is even mounted. Since systemd actually have knowledge about mount points and files systems, it can also delay to the very last moment the things needed for journald to write to the log.

There is also kernel guarantee that the daemons/pid's/programs that appear in the log are the real ones since systemd have total control and supervision of all processes. So if "lp0: printer on fire" appears in the log, you will know whether it is a prank or a real message.

The structure and index makes it possible to store lots of interesting meta-data, like monotonic time-stamps, GUI, PID, command line, marks from where every boot started and ended ("journalctl -b -p err" shows all loglevel "error" messages generated last boot only). Using monotonic timestamps to compare two boot sequences on perhaps two similar machines, is quite interesting and very easy to do.

It also allows for multi-language log support, supplementary help files that can explain what the error message means, suggest how to solve the problem, etc.

Here is a list of fields in the journal. Because of the index, the journal have real time knowledge about what is written in the fields, so there is tab completion and extremely powerful sorting available:
http://www.freedesktop.org/sof...

If you care about logging with systemd, take a look a the original design document behind journal
https://docs.google.com/docume...
Notice how simplicity is design goal number 1.

If you intend to remain (a paid) Linux sysadmin in the future now where all major distro are starting to convert to systemd, you should really study systemd's journal.
It is much easier to "get" the power of a indexed and structured log file by trying it, than describing it. Fedora 20/CentOS 7 are reasonable choices to learn it on.

check out systemd's "nspawn" too: OS containers really are the future.

And that is part of the problem, that it is presented as a init vs init issue.

It has long since grown past that. Systemd, as a package, now holds udev, journald, its own cron replacement, a network manager, dhcp, ntp, inetd, etc etc etc.

There are several inaccuracies on that list; systemd has timers that may be used as a cron substitute, but really is a different design, so therefore cron can happily co-exist, and nobody forces anybody to use the systemd solution. Systemd doesn't have a replacement for a NTP server, it just has a simple sNTP client. Like the systemd dhcp client, it was made with a special user case in mind, namely OS containers, where existing solutions simply where too slow and noisy (when starting 1000 OS containers at the same time, it really matters if each dhcp client connection takes 60 seconds or 3). AFAIK, the dhcp code is mostly a library that can be ripped out of the systemd source tree.
Nobody forces anybody to use them; they are alternatives, not replacements.

Again, systemd doesn't have an "inetd" server. Systemd uses socket activation as pioneered by inetd, but there are important differences. Lennart has several blog pages about it, here are a couple: http://0pointer.de/blog/projec...
http://0pointer.de/blog/projec...

systemd can use "inetd" prepared services, but inetd may happily co-exist together with systemd, and inetd provides services that systemd doesn't provide such as TCPMUX and RPC services.

The journal was a part of systemd early on, because in order to have great service management, you need the best logging possible.

But the crowning achievement may well be logind, that more and more DEs are getting a dependency on.

And logind can only function with systemd as the init, full stop.

logind was part of systemd early on. It has never been a separate project. It has never been meant to work outside systemd. Systemd opponents seems to have this bizarre love-hate relationship with systemd code; they curse it and the open source developers who make it, claiming the code is bad etc. etc. But at the same time they lust for the code, claiming systemd should be split up in smaller modules so they can use it anyway.

It has been a seriously mistake for the systemd opponents that they have become so fixated on systemd's "logind". Why should they care about how systemd solves user session problems? Upstream projects like Gnome and KDE have warned about for years, that an _alternative_ to logind/ConsoleKit should be developed.

Upstream projects like Gnome and KDE becomes dependent on systemd's login, because it solves real problems, like no other maintained program does. If there was an alternative API, from a program with similar functionality, they would happily support that. But no-one seems to care enough to start developing such a program.

It has been said many, many times; the systemd opponents should start to solve their own problems that lie in front of them, instead of wasting breath on trash-talking systemd.
More code, less ranting.

First, the "kludges" you are talking about, is a minimal distro agnostic compatibility layer, so that upstreams projects like XFCE can finally get basic system information without searching 20 places just to display e.g. the distro version it is running on top of.

It is one of the many examples on how systemd solves real world problems that helps other Linux developers, which is why systemd is so popular among developers. It is quite shameful that SysVinit (and similar init systems) distros haven't done anything even as basic as that.

The "systemBSD" project can't really be ported to Linux, and the crucial part, "logind" doesn't work yet and may never do. Trying to clone a version of systemd's "logind" is also a the wrong thing to do: trying to present it self as a systemd logind for upstream projects, but without supplying the necessary functions will just mean errors and crashes. A "logind" using the systemd API isn't what upstream projects like Gnome and KDE etc. have asked for the last couple of years; they want an _alternative_ API that support at least the basic features that logind or or ConsoleKit have.

To sum up: what upstream projects wants are an _alternative_ to logind/ConsoleKit, that works and are maintained. It is the most minimal request they can formulate, and is essential for providing modern DE functionality to non-systemd distros.

But even that request are ignored by the systemd opponents, they prefer to trash-talk systemd instead of solving their own problems.

Systemd is brilliantly designed and engineered, which is why it has become so popular. Its backwards compatibility is top notch; there was no flag day where every service either was a systemd service, or it wouldn't function.

They studied "launchd" and Solaris "SMF" and other Unix init systems to make an init system that was better in everything, in every detail, than what was available for Linux. SysVinit OTHO, is a notoriously bad design, where code and config options are mixed together in free form executable config files. Systemd separates code from daemon config options, so the latter are in structured text files that are easy to parse for both humans and machines.

The systemd developers really cares about details too, so it has the best bash-completion I have ever seen. It is a joy to work with the systemd CLI tools.

Vague criticisms about systemd's design really isn't much of an argument. Systemd actually works extremely well in the real world.

If PID2 is responsible for critical features like eg. cgroups which affects all running processes, including PID1, then it won't make a difference.

I do really think that the systemd designers have actually done their homework quite well when they started out. PID1 is quite small and only contain what is needed, everything else runs as helper processes.

Besides that, systemd can supervise itself (PID1) by using a watchdog, so the system can react if PID1 doesn't answer the watchdog "ping".
http://0pointer.de/blog/projec...

Do you really think it is a serious argument is that you could re-implement "logind" as a bash script? We are talking serious hardcore system stuff here, which is why no-one have made an alternative to "logind" or "ConsoleKit" despite upstream projects have pleaded for such a program for several years.

You are seriously misinformed on how systemd works and what it can do:
It uses kernel namespaces and capabilities to protect the system; this is on top of SELinux etc.

Here is a general overview:
http://0pointer.de/blog/projec...

Here are some of the config options for the daemons you can use. See "CapabilityBoundingSet=" for one way of using kernel "capabilities":
http://0pointer.de/public/syst...

There are so many freaking cool security features in systemd. As time goes by, developers, distro maintainer, and systemd administrators, can add more and more options to the running processes, like "NoNewPrivileges=" to prevent privilege escalation, or "ProtectHome=" to prevent malware and exploited processes from stealing info from /home, even if they otherwise had permission to read in home.

All this great new stuff can be turned on and used by adding a simple keyword to a structured text file. As time goes by, systemd distros will become ever more hardened.

Well, it does need to be handled somewhere; if you want features, you will get complexity, it is that simple. But as explained, the features and complexity isn't running in PID1; PID1 (systemd) is just a hub for relaying those features to other processes.

I really think so much of the systemd opponents talk about "Unix way" and "PID1" should be simple, is hand waving to gloss over the fact that the non-systemd distros have no feature parity with systemd to speak of; SysVinit is crude and no one in their right mind would design a init system these days with executable config files. Service configuration files should be non-executable text only.

General and vague criticism against systemd really doesn't convince anybody. Anyway, the Linux community have spoken with a large majority of Linux distros using systemd in the future.

If SysVinit systems really have all the features of systemd, just much better because they are simpler, you would expect a "SysVinit" boom in the future with lots of developers and users.

Personally, I think the systemd opponents are too concerned with negative campaigns against systemd, that they entirely forget to code any alternatives, so I predict ever more distros like Slackware abandoning script based init systems; they simply don't have an alternative.

The journal is primarily append based, so it designed to be very robust against corruption, the log is also rotated whenever a corruption is detected. So usually the corruption you see are a single line that was malformed when writing because e.g. the daemon died unexpectedly; this happens all the time with text based logs, but because they have no inbuilt integrity checks like systemd, it doesn't get discovered.

So in most cases you don't really loose any log info, and the rest of the log remains fully readable.
I am sure that there are subtle bugs in systemd that end up causing file corruption, or enhances the risk of corruption when the systems suddenly shuts down. But I also think that the bugs can and will be fixed, so I really don't see it as a long term problem. The many advantages by having an indexed and structured logfiles far outweighs the theoretically small chance of file corruption on modern file systems.

I don't think it is practically possible separate text logs from the metadata and get the same functionality as systemd's journal. AFAIK, the journal is more or less just a textfile+index anyway. Try running "strings" on a journal, it will show a lot of the logging info.

You can't seriously claim that systemd provides nothing that can't be done by script based init systems, shell scripts and existing daemons, logind is just one example (ConsoleKit is a dead project with no alternative).

But it would be an interesting project to make a Linux SysVinit distro that tried get feature parity with systemd, so that daemons could utilize the kernel "namespaces" and "capabilities" to massively strengthen security, or have a total supervision chain of all processes, including PID1 etc. There is so much good stuff in systemd that the SysVinit-like init distros could clone.

Isn't that argument just trying to make a virtue out of the fact, that SysVinit and the like, are totally crude and primitive init systems that are unable to anything much of interest?

All the analyses I have seen shows that moving crucial processes into PID2, just makes everything more fragile and opens up security holes.

I think that there are actually very good design reasons for why systemd is designed like it is.
It only runs one process as PID1, the daemon "systemd" which is rather small. This daemon however, is capable of "talking" with with several other processes, which gives it many advantages, since it can act as a mediator between kernel features like cgroups and "capabilities" and the processes it controls.

Sometimes new stuff is actually much better than then old stuff. I was skeptical about binary logs until I actually tried it. The advantages of a indexed journal is overwhelmingly positive. "journalctl" is an extremely powerful logfilter exactly because of the indexed and structured logs.
Systemd's journal also collects all logs in the same place, so no need to use "last" to read the binary "wtmp" log files, or locate .xsessionerrors and what not; everything goes into the journal.

Also, all the usual text tools like grep, tee, sed etc. all works with the journal by the standard Unix concept of piping. "journalctl" simply enhances the Unix text tools.

Give "journalctl" a serious spin someday; you may like it.