Slashdot Log In
The Amazon Technology Platform
Posted by
ScuttleMonkey
on Wed May 17, 2006 04:24 AM
from the under-the-covers dept.
from the under-the-covers dept.
Don420 writes "Jim Gray has an interview with Amazon CTO Werner Vogels for ACM Queue. It is filled with a lot of details about the Amazon architecture that we have not seen before: 'If you hit the Amazon.com gateway page, the application calls more than 100 services to collect data and construct the page for you.' But also quite a strong statements about developing software at Amazon: 'Developers of our services can use any tools they see fit to build their services. [...] Whatever tools are necessary, we provide them, and then get the hell out of the way of the developers so that they can do their jobs. [...] Developers are like artists; they produce their best work if they have the freedom to do so, but they need good tools.'"
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
100 Services ? (Score:4, Insightful)
and this a good thing ?
Re:100 Services ? (Score:5, Informative)
(http://www.isights.org/)
Major web sites are just a "little" more complex than your typical iWeb home page...
Unlimited budget (Score:4, Funny)
(http://wfmh.org.pl/carlos/)
Sounds (almost) too good to be true... (Score:5, Interesting)
Cat got your tongue? (Score:2, Funny)
(http://www.theinternetisboring.net/)
Thanks (Score:1, Interesting)
Or maybe I got you wrong and you meant that developers are like artists: Poor, starving, living for their work and only valued once they are no longer available.
Re:Thanks (Score:4, Insightful)
(http://www.themanpages.net/ | Last Journal: Tuesday September 06 2005, @03:45PM)
Move away from the coasts. I make $75,000/year working 40 hour weeks. I'm not on-call, have flex hours, get 3 weeks of vacation, and unlimited sick time. Quit working for IT sweat shops. Move somewhere where family time is valued and it's impossible to hire people unless you are willing to give them that flexibility. I've been through four employers in the St. Louis area and been able to land jobs with a deal akin to this one at all four. Developers are "poor"? No. Elementary school teachers are "poor." Starting salary for a developer in a low-watt market is close to $40K without a degree. That's not "poor." That's not starving, and that's not living for their work, unless by "living for their work" you mean that you're expected to show up on time and do your job.
But they need good tools (Score:4, Informative)
(http://wyoguide.sf.net/)
For users the answer is easy, they simply switch to something different, but for developer it's not. You usually first have to get a lot of knowledge which needs time. But one does never get more time!
So developers have to think in advance sometimes several years. This means constantly be on the edge of the available knowledge. Tools can certainly help but nothing prevents you from getting the knowledge in advance.
O. Wyss
Artists? (Score:5, Funny)
Tools are all well and good... (Score:4, Interesting)
The transition from the monolithic Obidos to the current SOA makes me wonder how exactly that part of the system works. Though it's not (that I could see) explictly stated, it certainly seems that adding scalability was a long and painful process. Planning for future developments like this is something that developers tend to be much better at than managers - so I wonder whether the developers didn't think about including scalability hooks in their initial efforts, whether they decided (back in the early days) that it wasn't worth it, or whether they wanted to but were told not to bother.
All said, I do applaud the public stance that Vogels is taking in his attitude. If more CxOs shared it, we'd likely have beeter-designed systems all over the shop. You hire the developer because (s)he's good at developing - so let them go to it!
Good Tools? (Score:1, Insightful)
100 services you may never use (Score:2)
No wonder it takes so long to load
i wish... (Score:1)
gee, i wish my boss(es) was/were like that
just so we didn't have to end up doing
half-assed jobs with half-assed resources
ending up in half-assed products...
(and maybe half-assed revenue?)
oh well...
My question is (Score:5, Interesting)
I wonder how they avoid the maintenance nightmare which is having multiple application components done using various obscure technologies/tools and the person that did it leaving the company and somebody else having to maintain/extend those application components?
Do they standardize their build tools, require good documentation on the service implementations or just overwork the poor sods that have to do maintenance to death?
CTO speak (Score:2)
(http://www.achacha.org/)
1 click == 100 services (Score:1)
(http://www.opusnet.demon.co.uk/)
Recruiting interview? (Score:2, Interesting)
This interview, while I'm sure sincere on the part of the CTO, comes across as a recruiting pitch. Obvious fallacies:
"Developers themselves know best which tools make them most productive and which tools are right for the job."
This sort of development mishmash depends on the developers never leaving (which most do after 2-3 years). Maintenance is, at best, nightmarish and leads to a patchy (with apologies to Apache) mess. FWIW, most developers seem to jump into coding right away with no thought for architecture or design.
"Whatever tools are necessary, we provide them, and then get the hell out of the way of the developers so that they can do their jobs."
Hmm, so the developers manage themselves. What a great job being a manager must be there.
"Developers are like artists; they produce their best work if they have the freedom to do so"
In my experience, most developers are nothing like artists and more closely resemble petulant, undisciplined children. Often they ignore the most basic principles of good software development (like version control, automated build and test suites, documentation, etc.) because "those are boring".
"I think part of the chaotic nature--the emerging nature--of Amazon's platform is that there are many tools available"
Wow, CTO speak at its finest to explain the disorganized nature of the organization.
"As a result of this principle, we have many support tools that are of a self-help nature."
See my point above about documentation being boring.
Comments I've heard from people who work at Amazon:
Your mileage may vary.....
Re:Recruiting interview? (Score:4, Interesting)
1. Low pay - it's a great place for budding "artists" fresh out of school to build "experience" that has to be unlearned at a more organized shop.
Not at all. I'm making 15% more here than I did at HP last year. I make more than devs with equal experience at MS. The pay is pretty good.
2. Dot-Com mentality - Lots of excuses for your desk being a door on two filing cabinets as well as the lack of organization.
THe door desk is aq culture thing. THere's a whole story behind it. Think of it like a tradition a sports team has. In the end, it works well- its not pretty, but its sturdy and does its job.
3. Turnover rate - as soon as people get experience, they leave for better paying jobs; Amazon is *always* hiring.
Amazon is also growing- we have more developers than the year before. Yeah, there's turnover. Welcome to the IT industry- its rare for anyone to work at 1 place for more than 3-4 years. A lot of it is people staying just long enough to vest their stock grant (thats right, our sign on bonus is a grant, not an option) and then leave for another bonus elsewhere. Moneywise its the smart thing to do at any company.
The next wave: Service Oriented Architectures (Score:3, Informative)
The article is impressive in hearing how Amazon successfully migrated from their legacy platform to a SOA. They may become a real contender in this emerging market, considering that they already have the user base and are quickly maturing a powerful platform. The other major contenders are Rearden Commerce and Salesforce.
Rearden Commerce, the company I work for, has developed a very pure SOA. They are currently targetting enterprise customers in order to gain the critical mass and user adoption necessary to succeed, which can be very difficult for a startup working in the consumer market. Their goal is to provide a web-based personal assistent that you can use to book plane tickets, dinning, etc. and all coordinated with your peers and working with your calendar and notification preferences (email, SMS, voice). It looks as if Amazon is on a similar path, so it will be interesting to see what happens in the next few years.
After the critical mass and the base platform are available the next big issue is getting 3rd party developers on the platform. That's something that everyone seems to be working on, which is why we're seeing so many AJAX and other toolkits emerging from companies like Yahoo, Google, and Zimbra. Imagine another company's product integrating just as neatly with Gmail as Google Calendar, yet staying very decoupled. That's part of the promise, and is the next big hurdle for the SOA leaders even though their platforms are still quite fresh and new.
Slightly OT: Amazon compatibility with Squid proxy (Score:1)
We have worked with Astaro support, and they have narrowed down the problem and sent the information to Amazon, but there has been no response yet.
In researching the issues, I have found posts all the way back to 1999 regarding incompatibilities with Squid and Amazon. Both ends have worked on the problem, but there has not been a 100% successful fix.
The issue only seems to affect far less than 5% of squid users, and even then it's sometimes inconsistent. So, it's almost impossible to fix because it's so hard to demonstrate. We can repeat the problem consistently, but it is the first case Astaro has had, and their systems are pretty much identical across the board, so it might even be a hardware compatibility issue with specific NICs. Who knows.
Problem description [squid-cache.org]
Amazon employee post [squid-cache.org]
If anybody has had problems in the past and was successful in resolving them, please reply!
Thanks,
Brian
artiste (Score:1)
if only all IT managers knew this.
best quote from this fluff article (Score:2)
Re:I wish they would spend more time on thier supp (Score:1)
(http://www.netsoc.tcd.ie/~sjf)
Netcraft confirms it (Score:3, Interesting)
Yes. [netcraft.com]
Re:I wish they would spend more time on thier supp (Score:2)
(http://richardstanford.com/ | Last Journal: Monday April 05 2004, @06:03PM)
Re:After 25 Years, I Finally Figured It Out (Score:2)
Anecdotal, but... (Score:2)
In other occasion, I ordered a book by priority shipping and more than a week later I hadn't received it. I sent them an email and they answered that there had been a robbery at an airport warehouse, they sent me another book by priority shipping and, since I hadn't received the first book in time, they charged me only the price for standard shipping. The difference was reinbursed by the credit card company, it was discounted from my next CC payment.
they no longer sell electronics to Ireland, although they will send to N.I.
I don't live there, but, AFAIK the United Kingdom of Great Britain and Northern Ireland is not the same country as Ireland, although, according to what I have seen quite often in the news in the last 35 years or so, there are people who are not satisfied with that. Amazon's policy varies from country to country, the legislation in Ireland must present some difficulties for shipping electronics there.
Re:Much use of Haskell and SML. (Score:4, Interesting)
(http://slashdot.org/)
Re:Much use of Haskell and SML. (Score:2)