Comment Re:Scala (Score 1) 315
Then you're dong it wrong. For the class of problems I'm interested in each process needs an input queue, and the ability to detect (somehow...there are several plausible means so I'm not choosing) what other processes are around and how to write to their input queues. And you need to be able to examine your input queue to tell whether there's a waiting message without blocking. All fairly straightforwards. You don't synchronize the processes, each one runs as far as it can with the inputs it has available and then waits for additional input. What is queued should be messages as short as possible, but that's true whenever you copy an array. And the queue should hold either a deep-copy of whatever is being exchanged or a reference to an immutable instance. No scheduling, per se, except that you might want to be able to adjust the priority at which the processes execute.
FWIW, I'm currently implementing something that operates in this way, and most of the tools I'm using to do this are excessively slow BECAUSE they are capable of a lot more than I'm asking them to do. I'm only planning on having around 8-16 processes because I've only got about 8 processors. I expect that most of the processes will keep busy all the time without needing new inputs. The messages that I'm planning on passing all have the form (action, key, value), and for my purposes key will always be either a string or an integer, and value will be an array of stuff. The kind of stuff will vary depending on what kind of process is receiving the input and what action is to be preformed. (Which is why I really don't want a static type.) Generally, however, it will begin with a few numbers, then a few (usually 4) arrays of structures(without internal pointers) and then possibly a string. This kind of thing is quite simple to handle in a language with dynamic types, and a real pain in languages with static types.
Do note that this means that most of the messages will be longer than is optimal, and that the length will not be consistent. It's the kind of thing that marshall, pickel, yaml, or json and handle trivially. No class serialization needed.
This means that each process totally controlls it's internal synchronization without external conflicts. Thread synchronization is not a problem. Scaling is trivial. Efficiency...well, I'm not so sure of that. I need to set things up so that most processing happens without IPC, and I'm not sure how possible that will be. I may need to go all out and find or build an even simpler IPC mechanism. (I think what I'm currently planning on has TCP/IP sockets burried within the implementation. I'm using localhost, so that probably gets translated into UNIX domain sockets, but even that may not be as fast as possible. OTOH, I don't want the input queues to be bounded by a pre-determined amount of RAM unless I must.)