Nice work you condescending twat, that phone list is from three years ago.

why don't you grow a sack?

"Phoronix Forums. You will never find a more wretched hive of scum and villainy. We must be cautious."
- Obi-Wan Kenobi

You're welcome - I hope you get it sorted out.

The only other thing I'd mention - you perhaps noticed I kept saying "threads like.." and "with regular threads" because it's basically introduced a number of single points of failure. Due to the lack of back channel or retransmission, things can go silently wrong without notice (network cable failure etc). In an ideal world you'd double up on some of that infrastructure and networking.

I know you need to get something up and running, but it's perhaps something to bear in mind for a later iteration.

I wonder would could be used to hold the soil in place? If there was only a way, something to hold it all together...

1STABILITY
2PERFORMANCE
3FEATURES
4USER INTERFACE FUCKING FIDDLING

Christ lord almighty, this program has gone to the shitter and it's killing me, PLEASE focus in that damned order.

First I thought it was me or my machine, because I'm an extremely heavy tab user, but I'm seeing it on my other machines too. Yes, I run a heck of a lot of tabs in some sessions (probably over 100 right now) however Firefox has run over 100 tabs for me for the best part of 8+ years.
When I'm researching I have 8 tabs open, it's what I do. I middle click the link to open in the background (tab mix plus â(TM)¥) and then I read the current tab, switch instantly to the next one, no back buttons and clicking the next link, I've already clicked it, it's waiting for me.

I just want the damn thing to be stable, I want them to STOP fiddling with the UI, I want them to STOP adding features. I've got 8gb of RAM use it (properly!) if you need it - but maintain stability and performance.
I'm almost using Chrome at the moment and I can't stand Chromes, Apple style "our way or the highway" on stupid changes.
FF with Tabs Menu, Xmarks, Tab Mix Plus, Lastpass is a joy to use when it's being reliable but god help me this is killing me.

Wow, was that clever.

Amen, brother.

(And don't forget the French!)

I think you forgot "FCUK NSA" somewhere in that NSA food... Or is it "FSCK GCHQ''?

Anything that has a USB port, really.

Essentially, anything that is run by NGOs or individuals.

Sure, in a corporate or governmental/military environment, USB ports are usually a big ''no no'' but some of use like them USB gadgets.

(Yes, before anyone ask, there has been infiltration through contaminated USB drives and keys ''abandoned'' in strategic locations...)

No, not extortion against Tails - extortion of money from the NSA or whoever else their ''clients'' are.

I am sure a lot of TLAs right now are salivating -- unless they have discovered these vulnerabilities before Exodus. In which case, silence can be golden, indeed.

FWIW I agree vis-a-vis using UDP for a business critical thing. I'd want exemption from responsiblity for any missed packets purely due to the infrastructure in between.

First - the problem with python is that because it's a VM you've got a whole lot of baggage in that process out of your control (mutexes, mallocs, stalls for housekeeping).

Basically you've got a strict timing guarantee dictated by the fact that you have incoming UDP packets you can't afford to drop.

As such, you need a process sat on that incoming socket that doesn't block and can't be interrupted.

The way you do that is to use a realtime kernel and dedicate a CPU using process affinity to a realtime receiver thread. Make sure that the only IRQ interrupt mapped to that CPU is the dedicated network card. (Note: I say realtime receiver thread, but in fact it's just a high priority callback down stack from the IRQ interrupt).

This realtime receiver thread should be a "complete" realtime thread - no malloc, no mutexes. Passing messages out of these realtime threads should be done via non-blocking ring buffers to high (regular) priority threads who are in charge of posting to something like zeromq.

Depending on your deadlines, you can make it fully non-blocking but you'll need to dedicate a CPU to spin lock checking that ring buffer for new messages. Second option is that you calculate your upper bound on ring buffer fill and poll it every now and then. You can use semaphores to signal between the threads but you'll need to make that other thread realtime too to avoid a possible priority inversion situation.

> how do you then ensure that the process receiving the incoming UDP messages is high enough priority to make sure that the packets are definitely, definitely received

As mentioned, dedicate a CPU mask everything else off from it and make the IRQ point to it.

> what support from the linux kernel is there to ensure that this happens

With a realtime thread the only other thing that could interrupt it would be another realtime priority thread - but you should make sure that situation doesn't occur.

> is there a system call which makes sure that data received on a UDP socket *guarantees* that the process receiving it is woken up as an absolute priority over and above all else

Yes, IRQ mapping to the dedicated CPU with a realtime receiver thread.

> the message queue destination has to have locking otherwise it will be corrupted. what happens if the message queue that you wish to send the UDP packet to is locked by a *lower* priority process

You might get away with having the realtime receiver thread do the zeromq message push (for example) but the "real" way to do this would be lock-free ring buffers and another thread being the consumer of that.

> what support in the linux kernel is there to get the lower priority process to have its priority temporarily increased until it lets go of the message queue on which the higher-priority task is critically dependent

You want to avoid this. Use lockfree structures for correctness - or you may discover that having the realtime receiver thread do the post is "good enough" for your message volumes.

> to the best of my knowledge the linux kernel has absolutely no support for these kinds of very important re-prioritisation requirements

No offense, but Linux has support for this kind of scenario, you're just a little confused about how you go about it. Priority inversion means you don't want to do it this way on _any_ operating system, not just Linux.

> the first ones used threads, semaphores through python's multiprocessing.Pipe implementation.

I stopped reading when I came across this.

Honestly - why are people trying to do things that need guarantees with python?

The fact you have strict timing guarantees means you should be using a realtime kernel and realtime threads with a dedicated network card and dedicated processes on IRQs for that card.

Take the incoming messages from UDP and post them on a message bus should be step one so that you don't lose them.

Works for most people, do some reading and get back to me.