Catch up on stories from the past week (and beyond) at the Slashdot story archive


Forgot your password?
For the out-of-band Slashdot experience (mostly headlines), follow us on Twitter, or Facebook. ×

Comment: Re:Interesting (Score 1) 37 37

The conservatives are just getting their panties in a bunch because they can no longer demand "moral" state backed censorship of the net.

Hyperbolic claims are about SPAM not being allowed to be blocked, are just sad political attempts to institute state sponsored censorship through the backdoor.

Looking at V. Ford's homepage just says she unsurprisingly supports companies right to unlimited spamming of customers, something that the new rules also forbid.

Comment: Re:Interesting (Score 1) 37 37

Of course SPAM filtering is allowed under the new rules.

Maybe there is an EU member state where SPAM/UCE is legal, but in all EU countries I know of, it is illegal and therefore can be blocked. Not only is the act of spamming illegal, but since they almost always use criminal methods like mass hacking, that fact too is reason for blocking all such SPAM.

ISP default content filtering or state mandated content filtering of legal stuff will be prohibited. So the UK state dictated "opt out" filtering will be illegal, while Symantec's etc. individually enabled and user controlled content filtering will be allowed.

You sound like you have been badly informed by some punter site.

Comment: Interesting (Score 2) 37 37

At first glance the new net neutrality rules looks very good. The "non-blocking" rules seems to make default and opt-out ISP porn-filtering illegal. If a site/service isn't illegal in one way or another, the ISP must not block it.

Sure, the ISP can offer various content filters as an opt in solution, leaving any such decisions to the individual where such decisions belongs, but the ISP can't "opt in" everybody, nor can any member state make any such filters mandatory.

From a free speech perspective that is a huge win on top of the network traffic net-neutrality rules.

I can't see anywhere if these rules are only binding within the EU, or if it is legal to block/throttle non-illegal sites and traffic from sites outside the EU. Does anybody know?


Comment: Re: you're a total ponce (Score 1) 128 128

You mean one developer, who's known for throwing shit together in a crufty way, managed to convince three people at the "PNELV" by either boring them shitless or throwing a tantrum.

No, I mean many many kernel developers, including the guy who maintain all long term stable Linux kernels for the Linux-foundation. Basically the second guy besides Linus Torvalds that the LF employ.

I have seen zero kernel developers backing any other Linux init-system. In fact, the Linux developers seems to actually flee from the rather toxic systemd-hater camp that you and your juvenile behaviour are stellar examples of.

Seriously, who would ever work in a project with a poisonous guy like you?

Comment: Re:They will care, probably sooner than they think (Score 2) 128 128

[snip: about "journalctl -b -1 -p err"]

You are cherry-picking the one thing that isn't logged by most syslog daemons by default,, in a disingenious attempt to show that syslog is "worse", even though it is off by default because it is of little use. If we cared AT ALL to have the "log level" information, it would be logged.

I chose the example because it has proven really useful to me. The example will quickly show any serious error that may have cause a system to fail. Being able to filter out all boot-sessions that aren't relevant is really useful. Being able to see all serious errors on a system at a glance is really useful too. Being able to easily combine such queries into one is pure gold.

But there is so much more that syslog doesn't log. This brings on another fundamental problem with text logs; they are hard to parse for machines due to their lack of structures, and they become hard to parse for humans too if they contain too much information.

Monotonic and micro-precision timestamps are great, but they foul up the readability of syslog textfiles simply because they make the lines longer. So basically syslog can't put the same amount of logging info in the log files, not for direct technical reasons, but because the log file format is inadequate.

The systemd journal on the other hand, can easily be extended with ever more fields as needs arise. And it can do so without breaking userland!

Because another problem with the unstructured syslog text logs are that they have no programmatic API or even "labels" for each part of a logging entry. That means the very structure of the log entries has become a sort of API, so changing that structure by adding more information, and thousands of userland log-watcher scripts that rely on "cut" simply breaks down.

The discussions of the many limitations of syslog,

Fine. Then solve the problem where it should be solved, and add this to /etc/syslog. You systemd apparatchik like editing non-script-based config files, right?

Many of the problems can't be solved by adding features to syslog. If you want "metal-to-metal" logging you just got to design something like journald for so many technical reasons, including that the Linux kernel only accept one owner of /dev/log.

But as said, the most fundamental problems are the total lack of coordination between stuff in Linux. It is almost impossible to improve some things because of that. The Rsyslog team have fought valiantly over the years, but the sheer lethargy and no formal coordination means changes are hard to impossible.

The systemd team solved this problem in the most elegant way possible; they made a new logger that were 100% backwards compatible with syslog, but at the same time introduced radical new features. The end-users could user whatever option that suited them best, and userland didn't have to change a line of code, while still benefiting from the new features.
This way all the systemd Linux distros and userland programs can slowly migrate to using the new journald logging API. No "flag day" problems!

# probably already in the config
source src { system(); internal(); }

# here's your damn filter
filter f_err_only { level( "error"}; };

# pre-filtered log output
destination err_only_log { file("/var/log/err_only_messages"); };

# link the filter to a destination
log { source(src); filter(f_err_only); destination(err_only_log); };

Now you can read those messages only using "less". You DID know that syslog has very flexible log routing and filtering capabilities, right?

I think the above examples greatly illustrate several problems with syslog text-files.
But first; despite spanning several lines, it isn't even remotely close to what "journalctl -b -1 -p err" does. It isn't an ad hoc query either, so it doesn't have any useful information from before it was set up.
It doesn't filter the errors to a particular boot, nor does it (AFAIK) show errors above "error".

It also introduce yet another log-file. On some systems you will find perhaps +20 such log files scattered all over the system. Madness I say. The reason for the splitting is of course, that it is damn hard to extract similar information with ad hoc queries. Especially newbies doesn't stand a chance.

With systemd's journal, you get a single view of _all_ log files on the entire system, including that pesky ".xsession-errors" that hides in /~user and only grows since it isn't log rotated.

And is so trivial to extract the information you want that no splitting is needed.

claiming that regular expressions are easy is laughable.

If regex is too hard, you might as well give up now. Regex is only hard if you abuse it badly, which is true for any programming language. This is just trolling at this point.

regex _is_ hard. This is partly because it is so powerful and generic. But using it is also hard because there are both regular _and_ extended regular expressions, _and_ maddening variations of both in awk/grep besides the Posix ones etc.

If it was easy to use regex on syslog text files there would never be a need to split them up.
Most Linux users these days don't know regex, so they use vi/less to read the logs and perhaps use grep if they are advanced users.

Oh, and thanks for admitting you are an inexperience n00b. You may have been using linux since the early slackware days, but didn't seem to learn much.

No, read again. What I said was that it should be easy to replicate the "journalctl -b -1 -p err" query since you claim it was n00b stuff. You utterly failed to do so.

As for your "challenge", I have yet to see any systemd apparatchik rise to the challenge to prove that systemd isn't an unmaintainable monolithic mess, by showing how to replace (NOT CHAIN) journald with syslog-ng or indeed run any of the systemd components in isolation.

You see, the genial thing about journald is that it is totally compatible with syslog-ng and all other syslog(3) implementations, and it actually enhances them by getting early boot logging info they can't obtain otherwise. All legacy userland logwatcher and log-analysing software still work when using journald that way.

It is possible to replace journald with a similar implementation, so in theory the syslog-ng could do that. But the functionality would have to be extremely similar (API etc). So they would basically just have another journald replica, and what would be the purpose of that?

Also, remember that journald isn't a log-sink while syslog-ng and Rsyslog are. So while Rsyslog isn't needed as a local system logger on ordinary machines, it still have a good purpose.

Regarding the monolithic claims. First, many parts of systemd are actually designed to have independent implementations with stable API's and what not. Here is a table of which parts and some known alternative implementations:

Also, a lot of the systemd stuff is in shared libraries so it is trivial to make alternative implementations, or in case of the networking stack, use it on non-systemd distros too. (it is basically "connman" turned into a library for many functions).

Finally, there are very few core dependencies for systemd; basically: the systemd daemon, udev and journald. And even the latter two can be ripped out (see minimal builds) for use in embedded systems.
Everything else is optional and can be substituted with any other Linux solution.

That some of the tools like "journalctl" should work outside a systemd context is a crazy requirement. They are systemd specific tools, just like the ext4 file system tools are ext4 specific.

The whole "monolithic" claim is hugely overblown and all stems from some people that felt entitled to leach parts of the systemd code so they didn't have to do the work themselves.

Comment: Re: you're a total ponce (Score 2) 128 128

Systemd will fail in the long run. Systemd lovers are just like the windows fanatics of yester year. My company has thousands of Linux systems guess what none of them are moving to a distro the uses systemd and never will. We will still keep making money. And will have a freedom of choice. So go suck it systems lovers.

Why should I care that you don't use a systemd distro? If you are making money on Linux, great. If you are using eg. Slackware to do so, hey, that is great too. I respect mr. Volkerding and his way of making a distro.

I like freedom of choice and I think systemd provides exactly that. Even if you don't like it, you benefit from the fact that there now are several udev-implementations (before there was just udev and the limited mdev) and several ConsoleKit/systemd-logind implementations (before systemd there was only CK).

But apparently the freedom of choice doesn't include the right to choose systemd.

That is a major problem with the behaviour of the anti-systemd camp; they won't accept that highly skilled Linux developers (including Kernel developers) and experienced distro and system maintainers, thinks that systemd is superior to whatever else out there, and therefore chooses to build their distro around it.

This lack of accepting other peoples freedom of choice is why you are trolling a Debian thread, even though you don't use the distro and claim you never will.

So think about what you are actually doing before saying "freedom of choice" again.

Comment: Re:They will care, probably sooner than they think (Score 1) 128 128

Such as? I've never had any problems.

I adore the power of using sql queries on logs.

How does journalctl fare in terms of having a trigger set up to automatically do things with logs when they're inserted?

The classic problem was speed. A DB was fine for storing and analysing, but could be severe bottleneck for log sinks.
It isn't so great for local system logging either since DB's tend to appear rather late in the boot sequence.

With journald you can have system logging before the rootfs is even mounted, and since both systemd and journald can pivot back to initramfs after the rootfs has been unmounted, you can potentially have logging after that.

These days it seems that people are using a mixture of DB's and heavily data massaged text logs (JSON etc) combined with a special index (Splunk etc).

There will never a single log storage solution for all, since the tools chosen will reflect the problems trying to be solved. For some it is auditing that is important, for others it is analysing website usage.

But since systemds journal is structured and field based, it is very easy to convert its log entries to standard JSON etc that consistently fits other storage means. A huge improvement over the many different syslog implementations.

Re: triggers. You don't use "journalctl" for that, but the API/libraries/language bindings. systemd provides a really good and well documented framework for making triggers, but don't provide them as such. You can have instantly triggered events on file systems that supports it.
"man sd_journal_get_events" documents the C API:

But there are also Ruby, Python, Go bindings etc. All in all I think systemd/journald is a major upgrade when it comes to log watchers; stable API, field based filtering, many language bindings, instant notifications etc.

Comment: Re:They will care, probably sooner than they think (Score 1) 128 128

Being able to do a "journalctl -b -1 -p err" is so much better than faffing around with grep and regex. (the line shows all log entries from the previous boot with the syslog severity level "error" and above, try that with grep!).

just playing around with "journalctl" for 10 minutes convinced me wholly.

So you've never tried using rsyslog to log to a database then?

Yes, so I know the many limitations of doing so. It is of course of interest to know that Rainer started the Rsyslog project exactly to overcome the many problems with the old syslog(3) interface, especially the problems with flat file text logs.

I have much respect for the Rsyslog developers work and it certainly weren't their fault they couldn't fix all the problems they set out to solve. So I am glad that Rsyslog now can read/write systemd journal files, meaning that it can be used as a log sink (at least for simpler cases) without regressing to using flat text log files.

Comment: Re:They will care, probably sooner than they think (Score 1) 128 128

Being able to do a "journalctl -b -1 -p err" is so much better than faffing around with grep and regex.

That statement alone shows the core problem with the systemd evangelists: you don' t want to learn generic tools, and instead want to use single-purpose, monolithic[1] apps.

"systemtctl" is a first class classic Linux/Unix tools that works together with all the standard Linux tools like sed, awk, grep, less, tee and what not.

It isn't interactive, it isn't chatty on success, can be piped etc, just like all the good ole GNU stuff. Sure, it can read binary text files like the journal, but so can "last" that is required to be Posix compliant (used to read the utmp/wtmp binary log files). And what about zgrep?

The systemd journal file format (basically an appended text file with non-standard delimeters+index) is fully documented and have excellent language bindings, making it trivial to make eg. Python scripts that read/writes directly into the journal. So it is trivial to make a "jgrep" or a "jless" that can directly read the journal if that is what you want.

The point is that "field" based logging is superior to the unstructured text dumps that syslog makes.

The point of what you call "faffing around with grep and regex" is that those tools are not difficult to use, and once you are familiar with the basics you now have a general tool you can apply to any unexpected situation.

Come on, claiming that regular expressions are easy is laughable. The old joke "I had problem. I decided to solve it with regex. Now I have two problems" is still true.

Not talking about that people should know the difference between regular and extended expressions, and that each and every tool uses its own maddening variation. Basically, if you don't use regular expressions as part of your everyday job, you are unable to use it without heavy man-page consulting for just the basic syntax.

I seriously doubt that any more than a tiny fraction of Linux users are able to whip out an even moderately complex regex. And that means the vast majority of Linux users can't filter their logs to any serious degree. It is so sad to see how people are literally trawling through the logs, reading them as a book in order to find problems. People are even using "vi" to do it, yikes.
systemd's journal basically solves this problem for good for Linux newbies. It is fun to see how productive people can become with "journactl" after a 10 minute introduction.

For power users, the field based approach is immensely powerful too. If you remember how Rsyslog started a decade ago, it was exactly to overcome the many limitations of flat file text logs.

syslog severity level "error" and above

I have never even remotely needed to filter events by syslog(3)'s "level" bits - it's not a very reliable filter, as app can be inconsistent in what LOG_* flag they use.

It is a superb way to filter from both "that boot only" and "this severity only". One reason why you probably haven't tried it is because this is very hard to do using generic tools using flat file syslog text logs.

Filtering on the facility (source) or time is far more useful. If you find that particular command to be useful, then great - which would be a good example of how use-cases can vary a *lot* depending on what you're doing.

Even this is something journald does so much better than syslog; There is kernel guarantee that the source is always is always what it claims to be. Same with what program generated the log entry. It can even distinguish between two instances of the same program running simultaneously since each and every log entry is stamped with a unique id (and session ID, and machine ID so you can trail logs even if the hostname/IP changes or across multiple machines and OS containers).

As for time based filtering: I just love "journalctl --since -10m" (show everything that happened from 10 minutes ago) or "journalctl --since -3w -u smartd.service".

As for filtering per boot. If it had been easy to with syslog, I would have done so many times. Being able to only see the relevant boot is great, but it is also really powerful to compare two different boots using monotonic timestamps.

try that with grep!

Listen, if you want lessons on how to use basic unix tools, there are many available on the web. For now, what you're obviously missing is that you would use sed for range filtering, not grep. do the line-range filtering. You simply use two regex in the form sed -n '/start_line_pattern/,/end_line_pattern/ p'.

Then, once you have a useful query built with the standard tools, you save it in a 2 line shells script. Seriously, do you think we actually type this stuff out verbosely every time we want to search a logfile? Have you evne *used* a CLI? This is n00b level stuff.

Actually I am old enough to have installed Slackware from 40 floppies (never again). Yes I have the O'Reilly books on sed, awk, bash etc.

But I see how you skipped my challenge, should be trivial since you claim it is CLI n00b stuff.

The point is that I am not the only one who thought that Linux logging have been a mess from the beginning. I had high hope when Rsyslog started, and they did fix several glaring errors, but in the end they failed because of the total lack of coordination in the Linux world. You can't really improve syslog, because then it wouldn't be syslog, and then userland don't support it. This circular dependency have been holding Linux back in several key areas.

The discussions of the many limitations of syslog, SysVinit, and X and Linux goes back to the previous century. It was just extremely hard problems to solve, so too many people have been confusing the longevity of SysVinit, syslog(3) and Xfree/ with any inherent quality. They all served their purpose in their time, but have been obsolete for at least a decade an a half. With systemd and Wayland, several of those systemtic problems have been solved for good, and that is something I really appreciate as a long time Linux user.

With some exaggeration, I think that using regex tools to do even basic log filtering is the perfect "Turing tar pit": "in which everything is possible but nothing of interest is easy.

Don't get me wrong, I like regex tools. Being able to utilise them is a major reason for why I like Linux so much, and the knowledge have of such tools have proven very useful many times outside Linux admin. But regex quickly becomes really unwieldy looking much like line noise. The main problem isn't the regex tools, but that syslog log files have such a loose structure. Having field based filtering just makes everything so much more powerful and easier.

Comment: Re:No, that's not possible. (Score 1) 128 128

Systemd is terribly unstable and rolls state back for trivial reasons. Those of us in charge of data centers see the issues, your laptop or home pc is not a viable model of reality.

What "state" are you talking about. This is Linux/Unix, not a transactional OS. systemd doesn't perform "roll backs". What trivial reason? Maybe that could explain what you are trying to say. It really sounds like you have no personal experience with systemd nor have read anything about it.

Comment: Re:They will care, probably sooner than they think (Score 2) 128 128

You are talking bullshit here, we've been testing systemd for 6 months and it is unstable, rolls the system back to start state for trivial reasons. Not to mention needing all kinds of shims for the parts that haven't even been written yet so it could "function" in a current system. What a bunch of badly designed over complicated garbage

There are no "shims" necessary to run systemd. On Debian there is "logind-shim", but no one should use that instead of the proper systemd-logind.

Comments like that, and your bizarre claim of " rolls the system back to start state" makes me suspect that you don't have actual personal understanding and experience of systemd management.

Sure, truly understanding systemd require some serious study, something way too many have neglected, thinking they could just wing it when time came. The payback for the time spend studying is that systemd also have some serious cool technology that is far superior to everything else in Linux/Unix land.

Comment: Re:you're a total ponce (Score 1, Interesting) 128 128

You are just talking bullshit here. I have been using systemd for +4 years now and it has been rock stable.

On a VM on your mom's MacBook that you take down to Starbuck's in a record bag slung over the crossbar of your fixie.

It is exactly because the systemd-hater camp apparently consist of technical illiterates like you, that they have lost each and every technical argument on all major distros. Ad hominem attacks and poisonous threats and trolling systemd threads are all you can do. Almost all volunteer developers have left the non-systemd camp because of its toxic atmosphere where attacking open source developers and users are as normal as breathing air.

Think about it; the anti-systemd faction couldn't even muster 5 Debian developers to sponsor a GR bill to overturn the technical committees decision of making systemd default init.

The negative, hate-driven anti-systemd campaign have resulted in that 100% of all commercial general Linux distros and most of the community driven distros are behind systemd and are supporting it. Talk about a losing campaign strategy.

Comment: Re:No, that's not possible. (Score 0) 128 128

No, it's not possible for systemd to have a bug. Our very own Peter H.S. told us that systemd is "rock stable". "Rock stable" software systems don't have bugs! And Peter H.S. has a 5-digit Slashdot UID! Clearly he knows what he's talking about, and couldn't possibly be wrong.

Yes, I know what I am talking about since I both use systemd and have read the systemd documentation. This is unlike you and the rest of the toxic systemd-haters who only repeat the party line as regurgitated by loony blogs, but hasn't bothered reading even the systemd man-pages.

systemd really is rock solid and amazingly secure by default. The fact that it uses "namespaces", "cgroups" and "capabilities" to protect long running system daemons, is far superior to what any other non-systemd distro have to offer.

While the systemd-haters are still wasting time trolling systemd-threads, their non-systemd distros slowly whither away by complete lack of development. But then again, does anybody actually care?

Comment: Re:They will care, probably sooner than they think (Score 1) 128 128

The journal is brilliant, anyone who says it isn't needs their head examined.

Yes, the journal really is brilliant. It just solves so many decade old logging Linux problems, and make hard stuff very easy.

I was sceptical about binary log files when I first heard about it, but just playing around with "journalctl" for 10 minutes convinced me wholly.

If a 6600 used paper tape instead of core memory, it would use up tape at about 30 miles/second. -- Grishman, Assembly Language Programming