Forgot your password?

Comment: COM (MSRPC), Objective-C/J and Software Libre (Score 2) 43

by lkcl (#47917061) Attached to: Industry-Based ToDo Alliance Wants To Guide FOSS Development

in looking at why both apple and microsoft have been overwhelmingly successful i came to the conclusion that it is because both companies are using dynamic object-orientated paradigms that can allow components from disparate programming languages to be accessible at runtime. COM is the reason why, after 20 years, you can find a random Active-X component written two decades ago, plug it into a modern windows computer and it will *work*.

Objective-C is the OO concept taken to the extreme: it's actually built-in to the programming language. COM is a bit more sensible: it's a series of rules (based ultimately on the flattening of data structures into a stream that can be sent over a socket, or via shared memory) which may be implemented in userspace: the c++ implementation has some classes whilst the c implementation has macros, but ultimately you could implement COM in any programming language you cared to.

the first amazing thing about COM (which is based on MSRPC which in turn was originally the OpenGroup's BSD-licensed DCE/RPC source code) is that because it is on top of DCE/RPC (ok MSRPC) you have version-control at the interface layer. the second amazing thing is that they have "co-classes" meaning that an "object" may be "merged" with another (multiple inheritance). when you combine this with the version-control capabilities of DCERPC/MSRPC you get not only binary-interoperability between client and server regardless of how many revisions there are to an API but also you can use co-classes to create "optional parameters" (by combining a function with 3 parameters in one IDL file with another same-named function with 4 parameters in another IDL file, 5 in another and so on).

the thing is that:

a) to create such infrastructure in the first place takes a hell of a lot of vision, committment and guts.

b) to mandate the use of such infrastructure, for the good of the company, the users, and the developers, also takes a lot of committment and guts. when people actually knew what COM was it was *very* unpopular, but unfortunately at the time things like python-comtypes (which makes COM so transparent it has the *opposite* problem - that of being so easy that programmers go "what's all the fuss about???" and don't realise quite how powerful what they are doing really is)

both microsoft and apple were - are - companies where it was possible to make such top-down decisions and say "This Is The Way It's Gonna Go Down".

now let's take a look at the GNU/Linux community.

the GNU/Linux community does have XPIDL and XPCOM, written by the Mozilla Foundation. XPCOM is "based on" COM. XPCOM has a registry. it has the same API, the same macros, and it even has an IDL compiler (XPIDL). however what it *does not* have is co-classes. co-classes are the absolute, absolute bed-rock of COM and because XPCOM does not have co-classes there have been TEN YEARS of complaints from developers - mostly java developers but also c++ developers - attempting to use Mozilla technology (embedding Gecko is the usual one) and being driven UP THE F******G WALL by binary ABI incompatibility on pretty much every single damn release of the mozilla binaries. one single change to an IDL file results, sadly, in a broken system for these third party developers.

the GNU/Linux community does have CORBA, thanks to Olivetti Labs who released their implementation of CORBA some time back in 1997. CORBA was the competitor to COM, and it was nowhere near as good. Gnome adopted it... but nobody else did.

the GNU/Linux community does have an RPC mechanism in KDE. its first implementation is known famously for having been written in 20 minutes. not much more needs to be said.

the GNU/Linux community does have gobject. gobject is, after nearly fifteen years, beginning to get introspection, and this is beginning to bubble up to the dynamic programming languages such as python. gobject does not have interface revision control.

the GNU/Linux community does actually have a (near full) implementation of MSRPC and COM: it's part of the Wine Project. the project named TangramCOM did make an attempt to separate COM from Wine: if it had succeeded it would be maintained as a cut-down fork of the Wine Project. The Wine Project developer's answer - if you ask - to making a GNU/Linux application use COM is that you should convert it to a Wine (i.e. a Win32) application. this is not very satisfactory.

in other words, the GNU/Linux community has a set of individuals who are completely discoordinated, getting on with the very important task - and i mean that absolutely genuinely - the very important task of maintaining the code for which they are responsible.

the problems that they deal with are *not* those of coordinating - at a top level - with *other projects*.

now, whilst this "Alliance" may wish to "guide" the development of the GNU/Linux community, ultimately it comes down to money. do these companies have the guts to say - in a nice way of course - "here's a wad of cash, this is a list of tasks, any takers?"

but, also, does this "Alliance" have the guts to ask "what is actually needed"? would it be nice, for example, rather than them saying "this is what you need to do, now get on with it", which would pretty much guarantee to have no takers at all, would it be nice for them to actually get onto various mailing lists (hundreds if necessary) and actually canvas the developers in the software libre world, to ask them "hey, we have $NNN million available, we'd like to coordinate something that's cross-project that would make a difference, and we'd like *you* to tell *us* what you think is the best way to spend that money".

where the kinds of ideas floated around could be something as big and ambitious as "converting both KDE and Gnome to use the same runtime-capable object-orientated RPC mechanism so that both desktops work nicely together and one set of configuration tools from one desktop environment could actually be used to manage the other... even over a network with severely limited bandwidth [1]".

or, another idea would be: ensure that things like heartbleed never happen again, because the people responsible for the code - on which these and many companies are making MILLIONS - are actually being PAID.

but the primary question that immediately needs answering: is this group of companies acting genuinely altruistically, or are they self-serving? an immediate read of the web site, at face value, it does actually look like they are genuine.

however, time will tell. we'll see when they actually start interacting with software libre developers rather than just being a web site that doesn't even have a public mailing list.

[1] i mention that because the last time i suggested this idea people said "what's wrong with using X11?? problem solved... so what are you talking about?? i'm talking about binary-compatible APIs that stem ultimately from IDL files". *sigh*...

Comment: define "customer" (Score 4, Informative) 287

by lkcl (#47887995) Attached to: German Court: Google Must Stop Ignoring Customer E-mails

from what i understand of the definition of "customer", a "customer" means "someone who is paying for a service". here, there's no payment involved, therefore there is no contract of sale. i would imagine that it's fairly safe to say that we're most definitely *not* quotes customers of google quotes.

if on the other hand these individuals are actually _paying_ google for service and are not receiving a response, _then_ i could understand.

Comment: Re:Where to draw the line (Score 1) 326

by lkcl (#47850635) Attached to: Stallman Does Slides -- and Brevity -- For TEDx

there is a beautiful tale which i will share with you, which helps to explain why what Dr Stallman is doing is so important:

"the reasonable man adapts himself to the world. the unreasonable man adapts the world to himself. therefore, all progress depends on the unreasonable man".

now, if it wasn't for Dr Stallman, the average pathological corporation (see the first few minutes of the documentary "The Corporation") would take whatever it could get (and you only have to look at the 98% endemic GPL violations on android smartphones and tablets to see the consequences of non-GPL software such as android)

so if it wasn't for Dr Stallman sticking to his principles, you would probably be using a computer that crashes 10 to 15 times a day for anything but the most mundane of tasks, and was entirely outside of your control.

Comment: Re:so why is intel's 14nm haswell still at 3.5 wat (Score 1) 161

by lkcl (#47798903) Attached to: Research Shows RISC vs. CISC Doesn't Matter

You seem to be conveniently ignoring Intel's Atom and Quark lines. They're all x86 and none of them has a TDP larger than 3w.

i'm not. intel's quark line - the one i saw announced on here last year - tops out at 400mhz. it has... nothing in the way of interfaces that can be taken seriously. it doesn't even have RGB/TTL video out. however if you are right about the latest intel atom being 3w, then now i am interested! so i am very grateful for you pointing this out, i will go check.

Comment: Re:so why is intel's 14nm haswell still at 3.5 wat (Score 1) 161

by lkcl (#47798897) Attached to: Research Shows RISC vs. CISC Doesn't Matter

Here is your answer, the A20 is freakishly slow compared to anything Intel would put their name on.

Granted, you can build a tablet to do specific tasks (like decoding video codecs) around a really slow processor and some special-purpose DSPs. But perhaps the companies in that business aren't making enough profit to interest Intel.

interestingly that assumption - that allwinner is not making enough profit - is completely wrong. allwinner is now one of _the_ dominant tablet SoC manufacturers in the world. their first revision (the A10, which was a Cortex A8) actually caused a major recession in the electronics industry when it first came out, as it was only $7.50 compared to the nearest competitor at around $11 to $12. everyone *not* using the A10 at the time was left holding worthless components; contracts for supply were reneged on; the change was so quick that many factories and design houses simply went out of business.

the volumes that allwinner are shipping are simply enormous, and, along with rockchip, their nearest competitor, the tablet market is completely and utterly overwhelmingly dominated by processors of the type that you describe as "built to do specific tasks".

those "specific tasks" include "running the android OS at a pace that's good enough for the overwhelming majority of end-users".

in short, intel has a long *long* way to go before they can even remotely consider that they have a processor that can be taken seriously in this very large market, both in terms of price and also in terms of performance.

what is particularly interesting about the comment that you make is that it would seem that intel really does, just as you do, believe that "a really slow processor and some special-purpose DSPs" simply is... not enough. and, contrary to that belief, it can be quite clearly seen by the total dominance of allwinner and rockchip that "a really slow processor and some special-purpose DSPs" really *is* enough.

one of the reasons for that is because if you look at the market you find that you need:

* audio and video CODEC processing. this can be handled by a special-purpose DSP. some of these are now handling 3D 4096-bit-wide screens.

* 3D graphics. these are handled by licensing a whole range of hard macros (special-purpose DSPs) that come with proprietary libraries implementing OpenGL ES 2.0. they're good enough, and some of them are getting _really_ good.

* an (as you put it) "really slow processor" - although if you look at allwinner's latest processor the A80 it can hardly be called "slow", it's an 8 core monster - which covers the running of the general OS.

overall these processors are graded according to price: $5 will get you something dreadful but "good enough", $20 will get you something that's complete overkill for a tablet.

and you know what? the $7 1.2ghz dual-core ARM Cortex A7 Allwinner A20 is, when it's put with 2gb of RAM, actually extremely quick. i tested out 1gb of RAM running debian GNU/Linux: i fired up xrdp and i had *five* rdesktop sessions running OpenOffice and Firefox on it, onto my laptop. it didn't fall over, and it wasn't dreadfully slow.

so i think you, just like intel, are completely and entirely missing the point. and in intel's case, that means entirely missing out on a *huge* market segment.

Comment: so why is intel's 14nm haswell still at 3.5 watts? (Score 0, Troll) 161

by lkcl (#47773819) Attached to: Research Shows RISC vs. CISC Doesn't Matter

ok, so the effect of RISC vs CISC has absolutely *no* relation to power, right? so why in god's green earth is, for example, the allwinner a20 1.2ghz processor - which is still in 40nm btw - maxing out at 2.5 watts and delivering great 1080p video, reasonable 3D graphics and so on - yet intel is having to go to 14nm and, even at 14nm they STILL can't release a processor that, if you run it in a very limited configuration, is STILL listed as 3.5 watts??

there's a quad-core rockchip 28nm SoC. maximum (actual) top power consumption: below 3.0 watts. intel's haswell tablet SoC is 20nm: it's 4.5 watts "Scenario" Design Power i.e. if you only run certain apps in certain ways it *might* keep below 4.5 watts.

i really _really_ want to know why it is that intel cannot deliver an SoC that has an absolute peak limit of 2.5 watts.


Climate Scientist Pioneer Talks About the Furture of Geoengineering 140

Posted by samzenpus
from the it's-getting-hot-in-here dept.
First time accepted submitter merbs writes At the first major climate engineering conference, Stanford climatologist Ken Caldeira explains how and why we might come to live on a geoengineered planet, how the field is rapidly growing (and why that's dangerous), and what the odds are that humans will try to hijack the Earth's thermostat. From the article: "For years, Dr. Ken Caldeira's interest in planet hacking made him a curious outlier in his field. A highly respected atmospheric scientist, he also describes himself as a 'reluctant advocate' of researching solar geoengineering—that is, large-scale efforts to artificially manage the amount of sunlight entering the atmosphere, in order to cool off the globe."

Comment: Re:complex application example (Score 4, Informative) 161

by lkcl (#47493359) Attached to: Linux Needs Resource Management For Complex Workloads

> the first ones used threads, semaphores through python's multiprocessing.Pipe implementation.

I stopped reading when I came across this.

Honestly - why are people trying to do things that need guarantees with python?

because we have an extremely limited amount of time as an additional requirement, and we can always rewrite critical portions or later the entire application in c once we have delivered a working system that means that the client can get some money in and can therefore stay in business.

also i worked with david and we benchmarked python-lmdb after adding in support for looped sequential "append" mode and got a staggering performance metric of 900,000 100-byte key/value pairs, and a sequential read performance of 2.5 MILLION records. the equivalent c benchmark is only around double those numbers. we don't *need* the dramatic performance increase that c would bring if right now, at this exact phase of the project, we are targetting something that is 1/10th to 1/5th the performance of c.

so if we want to provide the client with a product *at all*, we go with python.

but one thing that i haven't pointed out is that i am an experienced linux python and c programmer, having been the lead developer of samba tng back from 1997 to 2000. i simpy transferred all of the tricks that i know involving while-loops around non-blocking sockets and so on over to python. ... and none of them helped. if you get 0.5% of the required performance in python, it's so far off the mark that you know something is drastically wrong. converting the exact same program to c is not going to help.

The fact you have strict timing guarantees means you should be using a realtime kernel and realtime threads with a dedicated network card and dedicated processes on IRQs for that card.

we don't have anything like that [strict timing guarantees] - not for the data itself. the data comes in on a 15 second delay (from the external source that we do not have control over) so a few extra seconds delay is not going to hurt.

so although we need the real-time response to handle the incoming data, we _don't_ need the real-time capability beyond that point.

Take the incoming messages from UDP and post them on a message bus should be step one so that you don't lose them.

.... you know, i think this is extremely sensible advice (which i have heard from other sources) so it is good to have that confirmed... my concerns are as follows:


* how do you then ensure that the process receiving the incoming UDP messages is high enough priority to make sure that the packets are definitely, definitely received?

* what support from the linux kernel is there to ensure that this happens?

* is there a system call which makes sure that data received on a UDP socket *guarantees* that the process receiving it is woken up as an absolute priority over and above all else?

* the message queue destination has to have locking otherwise it will be corrupted. what happens if the message queue that you wish to send the UDP packet to is locked by a *lower* priority process?

* what support in the linux kernel is there to get the lower priority process to have its priority temporarily increased until it lets go of the message queue on which the higher-priority task is critically dependent?

this is exactly the kind of thing that is entirely missing from the linux kernel. temporary automatic re-prioritisation was something that was added to solaris by sun microsystems quite some time ago.

to the best of my knowledge the linux kernel has absolutely no support for these kinds of very important re-prioritisation requirements.

Comment: complex application example (Score 4, Insightful) 161

by lkcl (#47492919) Attached to: Linux Needs Resource Management For Complex Workloads

i am running into exactly this problem on my current contract. here is the scenario:

* UDP traffic (an external requirement that cannot be influenced) comes in
* the UDP traffic contains multiple data packets (call them "jobs") each of which requires minimal decoding and processing
* each "job" must be farmed out to *multiple* scripts (for example, 15 is not unreasonable)
* the responses from each job running on each script must be collated then post-processed.

so there is a huge fan-out where jobs (approximately 60 bytes) are coming in at a rate of 1,000 to 2,000 per second; those are being multiplied up by a factor of 15 (to 15,000 to 30,000 per second, each taking very little time in and of themselves), and the responses - all 15 to 30 thousand - must be in-order before being post-processed.

so, the first implementation is in a single process, and we just about achieve the target of 1,000 jobs but only about 10 scripts per job.

anything _above_ that rate and the UDP buffers overflow and there is no way to know if the data has been dropped. the data is *not* repeated, and there is no back-communication channel.

the second implementation uses a parallel dispatcher. i went through half a dozen different implementations.

the first ones used threads, semaphores through python's multiprocessing.Pipe implementation. the performance was beyond dreadful, it was deeply alarming. after a few seconds performance would drop to zero. strace investigations showed that at heavy load the OS call futex was maxed out near 100%.

next came replacement of multiprocessing.Pipe with unix socket pairs and threads with processes, so as to regain proper control over signals, sending of data and so on. early variants of that would run absolutely fine up to some arbitrarry limit then performance would plummet to around 1% or less, sometimes remaining there and sometimes recovering.

next came replacement of select with epoll, and the addition of edge-triggered events. after considerable bug-fixing a reliable implementation was created. testing began, and the CPU load slowly cranked up towards the maximum possible across all 4 cores.

the performance metrics came out *WORSE* than the single-process variant. investigations began and showed a number of things:

1) even though it is 60 bytes per job the pre-processing required to make the decision about which process to send the job were so great that the dispatcher process was becoming severely overloaded

2) each process was spending approximately 5 to 10% of its time doing actual work and NINETY PERCENT of its time waiting in epoll for incoming work.

this is unlike any other "normal" client-server architecture i've ever seen before. it is much more like the mainframe "job processing" that the article describes, and the linux OS simply cannot cope.

i would have used POSIX shared memory Queues but the implementation sucks: it is not possible to identify the shared memory blocks after they have been created so that they may be deleted. i checked the linux kernel source: there is no "directory listing" function supplied and i have no idea how you would even mount the IPC subsystem in order to list what's been created, anyway.

i gave serious consideration to using the python LMDB bindings because they provide an easy API on top of memory-mapped shared memory with copy-on-write semantics. early attempts at that gave dreadful performance: i have not investigated fully why that is: it _should_ work extremely well because of the copy-on-write semantics.

we also gave serious consideration to just taking a file, memory-mapping it and then appending job data to it, then using the mmap'd file for spin-locking to indicate when the job is being processed.

all of these crazy implementations i basically have absolutely no confidence in the linux kernel nor the GNU/Linux POSIX-compliant implementation of the OS on top - i have no confidence that it can handle the load.

so i would be very interested to hear from anyone who has had to design similar architectures, and how they dealt with it.

Comment: legal ramifications of identity verification (Score 1) 238

by lkcl (#47464365) Attached to: Pseudonyms Now Allowed On Google+

i think one of two things happened, here. first is that it might have finally sunk in to google that even just *claiming* to have properly verified user identities leaves them open to lawsuits should they fail to have properly carried out the verification checks that other users *believe* they have carried out. every other service people *know* that you don't trust the username: for a service to claim that they have truly verified the identity of the individual behind the username is reprehensibly irresponsible.

second is that they simply weren't getting enough people, so have quotes opened up the doors quotes.

Comment: Re:Hardware is hard (Score 2) 71

by lkcl (#47364551) Attached to: Improv Project, Vivaldi Tablet Officially Dead

Read "hard" as "Expensive as Hell"

That is part of it yes. It requires a wide range of differently experienced people: low level software, high level software, circuit design, assembly, layout, component sourcing, factory liasion, DFt, Manufacturing etc.

Then you need to get them all to work together. And you have to pay them.

... ynow... one of the reasons i came up with the idea to design mass-volume hardware that would be eco and libre friendly was because, after having developed the experience to deal with both low-level software and high-level software, and having done some circuit design at both school and university, i figured that the rest should not be too hard to learn... or manage.

  you wanna know the absolute toughest part [apart from managing people?] it's the component sourcing. maan, is that tough. if you want a laugh [out of sheer horror, not because it was actually funny] look up the story on how long it took to find a decently-priced mid-mount micro HDMI type D [8 months].

  so anyway, i set out to find people with the prerequisite skills that i *didn't* have, offered them a chance to participate and profit. the list of people who have helped and then fallen by the wayside... i... well.... i want to succeed at this so that i can give them something in return for what they did.

Comment: Re:Would it kill you to hint at what Improv is (wa (Score 3, Informative) 71

by lkcl (#47362691) Attached to: Improv Project, Vivaldi Tablet Officially Dead

If only there was some way to get more information, perhaps with a sort of "link" of some kind to a more detailed description.

here is the [old] specification of the [revision 1] CPU Card:

the current revision 2 which i am looking for factories to produce (RFQs sent out already) we will try with 2gb of RAM. this is just a component change not a layout change so chances of success are high.

here is the [old] specification of the Micro-Engineering Board:

that was our "minimal test rig" which helped verify the interfaces on the first CPU Cards (and will help verify the next ones as well, with no further financial outlay needed. ever. ok, that would be true if i hadn't taken the opportunity to change the spec before we go properly live with it!! you only get one shot at designing a decade-long standard.... i'd rather get it right)

this will be the basis of the planned crowd-funding campaign: it's more of a micro-desktop PC:

the micro-desktop chassis is very basic: VGA, 2x USB, Ethernet, Power In (5.5 to 21V DC). all the other interfaces are on the CPU Card (USB-OTG, Micro-HDMI, Micro-SD). however unlike the Micro-Engineering Board, the power is done with a view to the average end-user (as is the VGA connector which means 2 independent screens, straight out the box).

does that help answer the question?

Comment: Re:What was desirable about it? (Score 3, Interesting) 71

by lkcl (#47362605) Attached to: Improv Project, Vivaldi Tablet Officially Dead

Open hardware sounds cool, but as others have noted, good hardware design is both difficult and expensive. Considering how rapidly the components advance (CPU/SoC, I/O, displays, etc.),

aaaah gotcha! that's the _whole_ reason why i designed the long-term modular standards, so that products *can* be split around the arms race of CPU/SoC on the one hand and battery life / display etc. on the other.

and the factory that we are in touch with (the big one), they _love_ this concept, because the one thing that you might not be aware of is that even the big guys cannot react fast enough nowadays.

imagine what it would mean to them to be able to buy HUGE numbers of CPUs (and related components), drop them into a little module that they KNOW is going to work across every single product that conforms to the long-term standard. in 6 months time there will be a faster SoC, more memory, less power, but that's ok, because *right now* they can get better discounts on the SoC that's available *now*.

on the other side of the interface, imagine what it would mean to them that they could buy the exact same components for a base unit for well... three to five years (or until something better came along or some component went end-of-life)?

it took them a while, but they _loved_ the idea. the problem is: as a PRC State-Sponsored company they are *prohibited* from doing anything other than following the rules... i can't tell you what those rules are: they're confidential, but it meant that we had to find other... creative ways to get the designs made.

We're in a world where a first generation Nexus 7 tablet sells for $140 or less. At Walmart.

yeah. now that prices are dropping, just like the PC price wars, the profits are becoming so small that the manufacturers are getting alarmed (or just dropping out of the market entirely). those people are now looking for something else. they're willing to try something that might get them a profit. what should we tell them?

anyway: thank you for your post, darylb, it provides a very useful starting point for some of the key insights i want to get across to people.

1 + 1 = 3, for large values of 1.