Do you have to write java.math.BigDecimal in English or is there a Japanese version of the library? Do you have to know English in order to program? Do some English programming constructs seem immensely stupid to you? Just wondering since I speak English as my native language and it occurs to me that this is not true of everyone in the world and yet we all seem to run software based on the same sets of source code files. If those files are in C/C++ and so on then they have a lot of English in them even if it is just "if", "for", and "while" or MFC calls. Does this hinder software development at all?

I think I just invented a new technology but now is the worst time in my life to work on it. Somebody open source this puppy so we can all work on it.

I clearly don't get the jBPM... I'm definately out of my depth.

Where are the books on jBPM? Where are they? Is anyone else using this thing?

Wait, wait, let me google this before the site gets slashdotted so I can tag it in my wiki entry!

I've not been around /. much the last year or so because I've been busy working on a jBPM/Rules engine based product. Because of that I've been on planet Java for the last six months. I just got my Beryl desktop running this week and just about missed that whole fad. My Windows and Mac co-workers are jealous of my Beryl fire and wobble effects.

One thing that I noticed is that there really is a "planet Java" and a "planet Linux" and so on. Each sub-culture of the greater computer geek culture is strangely oblivious to the other. I suspect that there is a "planet Windows" and some of the newer more entertaining tech blogs reflect this. DailyWTF aka Worse Than Failure and Coding Horror reflect this. Many examples are Microsoft platform centric. It could be because VB, C#, and Windows programs in general are just more prone to stupidity... or because "planet Windows" is closed source so it's novel to have the ability to scrutinize code and laugh at it.

Has anyone seen Unix/Linux/BSD code that commits the same faux pas as the code posted to WTF or Coding Horror? What normally happens on "planet Linux" when code faux pas is committed? I suspect we end up in formally peer reviewing each other and chiding the WTFer into correcting their mistake.

Haven't had to say that in about two or three months. I guess things are slowing down around here. Anything in the last three to six months that "changes everything" that I should know about?

So we've covered SOAP and Domain Specific Languages. Now, I'd like to talk about data. Well, specifically relational databases. If you recall or can find someone old enough and sane enough to recall when Codd came down from the mountain with the first three normal forms carved on stone tablets it was quite an earth shattering development.

Up until that point data had been structured in "networks" or hierarchies of nodes in some fashion. The idea of a Relational Database that would allow for data entities to be queried and restructured in any given way was just shocking. It meant that not only could programs share data between them, but programs could use data in new and interesting ways never envisioned by the original system designer.

This introduced a whole level of unpredictability to the data world. Now you could have tables being joined in all sorts of crazy ways. There was more than one path to any given node... and that node might end up joined into any given report. Very very powerful stuff.

Meanwhile, elsewhere in the world other folks were wrestling with a different problem. Mainly that computer systems got really big. And you'll recall from my SOAP rant under "crazy thing number 1" that various strategies were devised for getting code into manageable chunks that could be reused or called on by other code. One such strategy is Object Orientation another is Aspect Orientation. It is no coincidence that Service Orientation and Object Orientation match up well. Both schools of thought are born out of trying to solve the same basic problems.

A block of code represents a block of utility. To leverage that utility in the largest number of places either you must be able to move the block around or move the data around to it. SOA moves the data to the block of utility... OOP makes it so you can move the utility. If you have a utility that cuts across concerns (violates object structure) you can use an AOP or Aspect Oriented Programming technique to cut across a system outside of type. For example all your objects could have a logging aspect attached to their toString method, one copy of the logging code could enable all objects properly annotated through out the system.

So then a block of data represents what? Well, if we really want to think long and hard about it your MP3 player is useless unless it has an MP3 to play. So it goes for our programs too. If there is no data to act upon there is no utility for the code no matter how good it may be.

Relational Databases lend structure to data that exists independent of the particular program. It is true that the Relational Database Model needs a System to run in... but that system is very generic. The Relational structure may also be radically different than any real-world useful structure requiring work to be done before it is in an appropriate form.

The problem is... nobody builds only OOP systems or only Relational Databases. Virtually every system that deals with significant amounts of data ends up being partially OOP and partially RDBMS. But, thinking in DB and thinking in OOP are fundamentally different.

In OOP we think of generalizations and abstractions to create interfaces and inheritance hierarchies. In the RDBMS we are looking to reduce repetition in the data and to adhere to those magical normal forms which have the effect of creating for us a data structure that can be used in a multitude of purposes. But is also ironically completely useless until it is transformed.

For example a naive Database table might hold an address in one table and have columns like line1, line2, city, state, zip and that would be just fine for small data sets. If you have two addresses in the same city, however, the city name will be repeated in the address table. For example if there were two addresses in your database in Springfield, Ohio the word Springfield would appear twice in the address table. In Relational Databases that is a big no-no. What if you needed the two letter abbreviation of Ohio to be printed for some outputs and the full name for others?

A more normalized table might have line1,line2,zip in the Address table and zip, stateCode, city in the Location table, and stateCode, stateName in the State table. But even that design violates normalization rules because there is a line1 and line2 column in one table. Still, this more normal form happens to show how now that we have broken up the addres into three tables you don't have any one address in one place... you have to restructure the data to create an address suitable for putting on a mailing label.

How would the Address object look? An array of lines and a pointer to a city, state, and zip. The Object isn't worried over any particular rules about the data. It is worried over what the Address is... not what it is made of. The Address is a form of contact information. So our Address object might be a child of a ContactInfo object that has its own properties. In object land all the relevant data is in one happy place.

How do we reconcile the DB model with the OO model?

If you are a Ruby true believer you use the Active Record design pattern. The Active Record design pattern basically says that one table equals one object. Done. That means that either you morph your OO design to fit directly over the DB design or you morph your DB to match your objects. Or a little of both.

That compromise leads to bad Object design and bad Table design. You end up with denormalized tables and screwy objects where inheritance loses a lot of its power because you can't add new fields to the objects that aren't already in the tables. Not only that but you also end up with objects that don't represent ... say ... and address anymore.

To solve this problem folks devised complex systems that allow for object persistence in database tables. These tools became the Object Relational Mapping tools we all know and hate today. Many of these tools simply added a new layer to work out object to table mappings in... and that layer tended to be XML.

Yes our friend XML gets used a lot. It is the universal data format after all. XML could lay out that Object A's id field matched Table B's autoIncrement column and so on. This repetition of information... this mapping activity... is sometimes lovingly called the XML sit-up. You do them over and over again and they are supposed to be good for you. Yeah right.

As XML use has grown it became important to start imposing structure on XML. A grammar to the language if you will. It is sort of like how in English you have words... "out, cat, put, now" ...but if you don't obey the English grammar you can't make a sentence ... like: "Put out the cat now" ... so XML got grammar. The first stab at this was the DTD but lately we've gotten the Xml Schema Definition or XSD.

Now table structures are called Schemas and this should probably lead into my point somehow. The Relation Database described by Codd way back in the 1980s was structure for data... XML with XSD is structure for data. XSD can describe relationships... Relational Databases describe relationships... what's going on here?

No small wonder the Database folks have been creating "XML Databases" that use XML and XML definitions as the basis for a relational engine. And that is almost crazy by itself. One fine day some young programmer will wake up in a world with a truly universal data format and a universal way to query and structure it... nearly crazy... but I've not gotten to crazy yet.

XSD can be used to define an object model... that is it can be used to design classes much like UML. *oh snap*

If you use a tool like XJC for Java you can have the tool generate annotated classes from an XML definition. If you are lucky enough to work in an EJB3 shop you can annotate the self same generated classes to create a table structure that is not necessarily the same as the object structure... the tables can be normalized independently of the object structure via the annotation or aspect oriented tags in the code.

If you are using annotated code, you can generate XSD from the code and Tables from the code. Right now the code is the center of a beefy submarine sandwich of juicy fresh functionality. Now you can create two models of data from one well annotated source.

I know this works because I've done it. I've created systems of objects that map to Tables or XML with XSD on the fly. Both forms of data structures roll just fine for my super smart heavily annotated objects. It's mighty sweet.

But, one day soon something even more amazing is likely to happen. The XSD aware XML Database that you could use XJC on to generate Java code. Or go the other way and use annotated code with super XJC to create an XSD aware XML Database.

And that's just crazy.

Any one else experienced that altered state of consciousness brought about by intense coding?

More crazy things coming... must resume code trance now.

Last time we covered the fall of CORBA and the rise of SOAP. And that was crazy thing number 1. Let's talk about crazy thing number 2.

Back in the day, when I was a toddler and there was no microprocessor monopoly there was more than one binary language in the world. It was a pain to learn these binary languages... folks covered these up with assembly languages that were mnemonic names for bit codes that a simple program could swap over to binary. Assembly was great and you could cross-compile it to various processors as needed.

Eventually, folks started writing more software than one person could hold in their noggin' more code than could be re-written in a weekend and the problem was to re-use code and to share it with other programmers. Higher level languages were developed to help us out with this problem.

C was one such language. And the fact that we had a language like C at all was awesome. Now we started to see software ecosystems. Different programs performed different jobs inside a framework provided by a new thing called an operating system. To solve this problem and to try and match human thinking better we developed new techniques for coding. We were blessed with Object Oriented Programming.

So many programmers were in love with C that C++ was born. C++ was the next incarnation of C and it had objects. Code reuse and layers of abstraction would save us. These layers would mean you could ignore certain facets of your computer at certain times. You could work in your happy little object and not worry about the disk drive spin speed or network latency.

Now the forces that created crazy thing number 1: SOAP are at play in this play ground too. Data doesn't flow between different machines easily because the bits don't lay in memory the same. And, you have to compile your code for every CPU you want it to run on. So somebody said. Hey, why not make one universal byte code and write an interpreter to handle that byte code turning it into native binary just in time?

Great. Java was born. It ran on a virtual machine. The virtual machine saved you from having to learn fifty different CPU, binary, OS, and other software ecosystems. Write code once compile to Java byte code and run it everywhere.

As it turns out language companies had been cheating in a similar way for years prior. Microsoft didn't write an entire compiler for every OS their languages ran on, they compiled to a byte code then they wrote a byte code assembler that took their special byte codes the last mile to CPU native binary. Every language compiler would be written to turn high-level code to byte code... then port the byte code assembler to every platform. After Java became popular Microsoft just shifted their existing architecture a little and you get .Net ... which again shows that Microsoft has brains they didn't even know they had ... but I digress, this particular thing has very little to do with crazy thing number 2.

There's something to notice here. We've spent all this time and energy on a single general purpose programming language. Most programmer's burned out their whole careers in a few of these general purpose languages on various platforms. All this even though we had ecosystems of software.

Back in old skool unix hackerdom the philosophy of having a whole mess of little tools and binding them together had accidentally lead to some interesting tools. Among those tools were things like sed and awk. These were text processing tools with their own languages. They used regular expressions which are exceedingly cryptic to the neophyte and take weeks to master but are exceedingly powerful.

In the rarefied air of database land, Structured Query Languages came to exist solely for the purpose of manipulating data in a relational structure. These were easy to learn, hard to master, query languages that could do a great deal with structured data. In fact DBA's were born out of the need to master this new way of dealing with data... a fact that plays into just how crazy crazy thing number 2 is...

Think back to the early 1990s and how a super spiffy windows program in 1995 might go together. You would write your program in C++ it would use C++ to run queries, it would use C++ to call the MFC to create windows and a GUI... it would use C++ to talk on the network or use C++ running queries to update tables that would act as message queues.

Consider the cool 1997 Java program would use Java to run SQL, Java to draw AWT, and Java to talk on the network.

Consider the cool 1993 Unix haxor program. It would use C to run SQL on a database, Sed and Awk to chop up data, and a CLI or a web interface to show the data. You might even shoot the data to a shell, sed, awk, and postscript printer driver.

Which of these things is not like the others?

Aside from the fact that the Unix program even in 1997 would have the crappiest UI of all three approaches the Unix program is doing something crazy. It's using different language at different levels each to their own strength.

There are real horrific problems with the Unix haxor's program to be sure. His UI sucks. His various languages are undisciplined and cryptic to anyone schooled on all but that particular level and language. But, he's using a federation of simple tools together to get a very complicated job done in fact each language is good at only one thing... they are...

Domain Specific Languages

Just as SOAP begins to free your mind from being locked into a single OS, computer, geographic processing location, or even programming language ... Domain Specific Languages begin to free your mind from procedural chains. Different problems lend themselves well to different languages and different descriptions.

Crazy thing number 2 is found in the classification of the following things: ECMA Script, SQL, XML, XUL, PDL, BPEL, Rules, XSLT, ... and so on. These are quirky limited languages that provide powerful constructs in their domains. These languages can't do the job of a C or a Java but they can do their specific job very very well.

SOAP helps open the door to using different languages for different tasks. However, the utility and power of DSL (Domain Specific Languages) can make very difficult thoughts easy to express.

Today your spiffy application would live on the web. It would use C#, Java, or some other traditional language to talk to a database through a broker that has a configuration or DSL like HQL. The flow of the program would be dictated by a BPEL script. Logical assertions handled by a Rules language. Your traditional language would then send out XML via SOAP and talk to a web front end described in XHTML with widgets written in a child of ECMA Script and transform the data (perhaps in client) with a data transformation language like XSLT.

If one goal of programming languages it to provide structure to human thought (so it can be compiled for different platforms, reused, shared with others) who says we all think the same way? Who says procedural thinking encapsulates all of human thought? Why not have different languages to cover different domains of thought?

Just as German has terms like Schadenfreude that don't really translate into English well, SQL does things that C can't do well either. Sure, you can get the same meaning because you ultimately translate SQL into C but "SELECT * FROM Clue WHERE User = ?" encapsulates the thought so much better. Schadenfreude... damage joy, mischief enjoyment, whatever... you get the idea.

Layout is covered really well by XHTML, ECMA Script covers UI actions well, C covers OS well, SOAP handles data interchange well, Java and C# cover procedural how-to-do-things programs well, Work flows can tie objects together into a process, Rules handle the what-and-why of logic better than procedures.

Tie the DSL together with a uniform interface and mix them together to solve problems. We have finally gotten away from the one-language-fits-all trap. Put DSL with SOAP in your blender and press blend.

SOA.

Service Oriented Architecture

I know you kids out there have been running around getting your eye-brows pierced and tatting your bodies with XML, but to us old fogies this SOA thing is quite surprizing. The idea that you can have a common data interchange language that is shared by all services and then mash-up the data into new applications... why that's the original promise of OOP! But, this time it will work.

Back in the day we had a problem. Not all machines put bits in the right order. And, the right order was "big-endian" order. Just the way you ... the human... reads his bits when he reads his alpha numeric cereal in the morning. The big bits go left, the little bits go right. But some of our machines were "little-endian" these evil little _personal_ computers were build with low byte left, high byte right. This was done apparently because when you pulled bytes from their memory registers you used a pop operation and when you finished with the bytes you used a push operation.

That meant that if you wanted to preserve endian order on data elements larger than word length (or one byte) you had to shuffle memory somewhere since you can't push and pop multiple items between two stacks and preserve their order without a third sorting stack. So the little PC's saved themselves lots of clock cycles by just letting bytes sit in memory in what ever order they came of the CPU in... which turns out to be upside down.

There's no problem if you have homogeneous data. For example an image that has all pixel data in it. All the data elements are going to be the same. You could have a header or waste pixel that should be FFEF but if the data gets scrambled it will read EFFF and now you know to unscramble the bits. But what if you have a complex type?

If you have an Object or a Struct you may have an int then a char with five characters, then a float, then a double. Now then, the bits might look like this on a big endian machine: CD30313233343500FFEC32ECFFEC3210123C3210 and like this on a little endian machine: CD30313233343500EC32ECFF10323C121032ECFF ... no problem right? Just flip those bits around and get 30CD32313433003532ECFFEC3210123C3210FFEC ... wait that doesn't look right does it?

  big : CD 30 31 32 33 34 35 00 FFEC32EC FFEC3210123C3210
  little: CD 30 31 32 33 34 35 00 EC32ECFF 10323C121032ECFF

Bits are only flipped around on the little endian machine in the float and the double. That's because the float and the double are primitive data entities over one byte in size. (we're assuming a one byte word size) So when they get put onto stack they are going in one byte at a time in reverse order... and only the float and the double have this problem.

Don't forget we even have the potential for different programming languages on different computers over a network to need to send and receive these bits. So what's the answer? Someone tried enforcing "network" byte order. Someone tried to create an entire framework. Something named CORBA came into being.

CORBA was amazing it had all this messaging over-head and would marshall and unmarshall data through these things called IDL that would describe a universal bit pattern for an object. Wow. You would use the IDL to hang C structs, Java object or whatever off of a CORBA core. And it all worked over the network transparent to the programmer. So why aren't we all CORBA programmers today?

CORBA sucks. Yes I know that the real reason CORBA failed is political. The standards body produced paper instead of product and just assumed that real world programmers would read the papers and say: "wow, I should write that!" but CORBA sucks fundamentally. CORBA sucks because you end up writing the same thing over and over in each language you need to use that damn IDL in. And if you change an IDL in one place you change it everywhere... that sucks. CORBA sucks because its data output isn't human readable... and it sure isn't machine readable unless you have all the right CORBA bits running.

But there's hope.

Notice that the int and the characters are completely unperterbed. Wouldn't it be nice if we just represented all data as strings turning them into more complex types when needed?

The old skool 'nix hackers wrote everything to output text and operate on text. Everything was a file. Including nothing. Text didn't get perterbed by getting shot through the network, munched by a little endian machine, then shot at a big endian solaris box. So why not do CORBA all in text?

Bandwidth.

But today in the 21st century we have lots of bandwidth and our software all have a universal zip and unzip compression that can be turned on at both ends.

Enter SOA. SOA is the same basic idea except we use XML as our universal data format. XML is composed of nice 8 bit single byte values. No machine on earth will screw up the pretty 8 bit streams. The problems are parsing overhead and a common standard. These are solved by the defacto SOAP standard (thank you Microsoft) and the fact that parsing XML is no big deal for your gigahertz processor.

So old skool Unix wisdom hits 21st century CPU horse power and bandwidth and you have a solution that is both human readable, machine parsable, and hardware agnostic. SOA, SOAP, and XML aren't crazy new... they evolved from old skool unix hacker wisdom and corporate standards body governance working together. That's why SOA will really work this time. No really. It's just 1960's techniques with 21st century know how and everybody on the band wagon.

And that's just crazy thing number 1 I've got a few more crazy things to introduce... and then we'll put them in a blender with a frog and see what we get. You with me?

Those of you that have read my journal for a while know that I've had jobs in C, Perl, and even Ada. The great irony of my career is that I was actually a much stronger C++ candidate straight out of University. Not just C++ but MS Visual C++. I did a brain bench right after graduating and had a perfect score in OOP.

So I was a very very strong MS/OOP/C++ candidate but there were three jobs on the table and the Perl job paid thirty percent more than the C++ jobs. So I took the Perl job and thus began my days as a Perl/Linux/C hacker. And, also a great lesson in the fact that the higher paying job may not be the better one.

In the last couple of years at my current job, we've begun to realize that simple CRUD practices won't get us by anymore. If we want our company to have real advancement we would have to provide our business with more sophisticated software. Trading Perl for PHP or JSP would only be re-arranging deck chairs on a sinking ship.

We needed to make the leap ahead. We needed to move beyond just fetching data and throwing it at batch jobs. And so I began digging into the cutting edge of computer technology. Past the hype and past the commercial interests several important things have begun to happen.

The world of computing is about to change again.

They used to say you were a geek in "larval stage" if you pulled more than one all nighter in a row. It was a sign that you were gestating some major hackage. I've noted that "larval stage" symptoms have recurred at different times in my life. Each time I learn a new programming language I kind of geek-out for a while, up late, up early, forget to eat, forget about a lot of things while I hyper focus on learning the new technology in front of me. I'm coming out of the longest "larval stage" I've ever had. I spent over a year on it. And, in hind-sight I wonder why it took so much intensity? It never seems like that big a deal after I've learned the thing...

... really? did ya?

... write a complete journal entry?