Forgot your password?

Comment: Re:complex application example (Score 1) 161

by Mr Thinly Sliced (#47514761) Attached to: Linux Needs Resource Management For Complex Workloads

You're welcome - I hope you get it sorted out.

The only other thing I'd mention - you perhaps noticed I kept saying "threads like.." and "with regular threads" because it's basically introduced a number of single points of failure. Due to the lack of back channel or retransmission, things can go silently wrong without notice (network cable failure etc). In an ideal world you'd double up on some of that infrastructure and networking.

I know you need to get something up and running, but it's perhaps something to bear in mind for a later iteration.

Comment: I will not stop saying this, I won't,... (Score 1) 172

by AbRASiON (#47513917) Attached to: Firefox 31 Released


Christ lord almighty, this program has gone to the shitter and it's killing me, PLEASE focus in that damned order.

First I thought it was me or my machine, because I'm an extremely heavy tab user, but I'm seeing it on my other machines too. Yes, I run a heck of a lot of tabs in some sessions (probably over 100 right now) however Firefox has run over 100 tabs for me for the best part of 8+ years.
When I'm researching I have 8 tabs open, it's what I do. I middle click the link to open in the background (tab mix plus â(TM)¥) and then I read the current tab, switch instantly to the next one, no back buttons and clicking the next link, I've already clicked it, it's waiting for me.

I just want the damn thing to be stable, I want them to STOP fiddling with the UI, I want them to STOP adding features. I've got 8gb of RAM use it (properly!) if you need it - but maintain stability and performance.
I'm almost using Chrome at the moment and I can't stand Chromes, Apple style "our way or the highway" on stupid changes.
FF with Tabs Menu, Xmarks, Tab Mix Plus, Lastpass is a joy to use when it's being reliable but god help me this is killing me.

Comment: Re:what environments allow USB boot? (Score 2) 132

by Noryungi (#47508613) Attached to: Exodus Intelligence Details Zero-Day Vulnerabilities In Tails OS

Anything that has a USB port, really.

Essentially, anything that is run by NGOs or individuals.

Sure, in a corporate or governmental/military environment, USB ports are usually a big ''no no'' but some of use like them USB gadgets.

(Yes, before anyone ask, there has been infiltration through contaminated USB drives and keys ''abandoned'' in strategic locations...)

Comment: Re:complex application example (Score 4, Informative) 161

by Mr Thinly Sliced (#47493435) Attached to: Linux Needs Resource Management For Complex Workloads

First - the problem with python is that because it's a VM you've got a whole lot of baggage in that process out of your control (mutexes, mallocs, stalls for housekeeping).

Basically you've got a strict timing guarantee dictated by the fact that you have incoming UDP packets you can't afford to drop.

As such, you need a process sat on that incoming socket that doesn't block and can't be interrupted.

The way you do that is to use a realtime kernel and dedicate a CPU using process affinity to a realtime receiver thread. Make sure that the only IRQ interrupt mapped to that CPU is the dedicated network card. (Note: I say realtime receiver thread, but in fact it's just a high priority callback down stack from the IRQ interrupt).

This realtime receiver thread should be a "complete" realtime thread - no malloc, no mutexes. Passing messages out of these realtime threads should be done via non-blocking ring buffers to high (regular) priority threads who are in charge of posting to something like zeromq.

Depending on your deadlines, you can make it fully non-blocking but you'll need to dedicate a CPU to spin lock checking that ring buffer for new messages. Second option is that you calculate your upper bound on ring buffer fill and poll it every now and then. You can use semaphores to signal between the threads but you'll need to make that other thread realtime too to avoid a possible priority inversion situation.

> how do you then ensure that the process receiving the incoming UDP messages is high enough priority to make sure that the packets are definitely, definitely received

As mentioned, dedicate a CPU mask everything else off from it and make the IRQ point to it.

> what support from the linux kernel is there to ensure that this happens

With a realtime thread the only other thing that could interrupt it would be another realtime priority thread - but you should make sure that situation doesn't occur.

> is there a system call which makes sure that data received on a UDP socket *guarantees* that the process receiving it is woken up as an absolute priority over and above all else

Yes, IRQ mapping to the dedicated CPU with a realtime receiver thread.

> the message queue destination has to have locking otherwise it will be corrupted. what happens if the message queue that you wish to send the UDP packet to is locked by a *lower* priority process

You might get away with having the realtime receiver thread do the zeromq message push (for example) but the "real" way to do this would be lock-free ring buffers and another thread being the consumer of that.

> what support in the linux kernel is there to get the lower priority process to have its priority temporarily increased until it lets go of the message queue on which the higher-priority task is critically dependent

You want to avoid this. Use lockfree structures for correctness - or you may discover that having the realtime receiver thread do the post is "good enough" for your message volumes.

> to the best of my knowledge the linux kernel has absolutely no support for these kinds of very important re-prioritisation requirements

No offense, but Linux has support for this kind of scenario, you're just a little confused about how you go about it. Priority inversion means you don't want to do it this way on _any_ operating system, not just Linux.

Comment: Re:complex application example (Score 5, Insightful) 161

by Mr Thinly Sliced (#47493195) Attached to: Linux Needs Resource Management For Complex Workloads

> the first ones used threads, semaphores through python's multiprocessing.Pipe implementation.

I stopped reading when I came across this.

Honestly - why are people trying to do things that need guarantees with python?

The fact you have strict timing guarantees means you should be using a realtime kernel and realtime threads with a dedicated network card and dedicated processes on IRQs for that card.

Take the incoming messages from UDP and post them on a message bus should be step one so that you don't lose them.

Optimism is the content of small men in high places. -- F. Scott Fitzgerald, "The Crack Up"