Please create an account to participate in the Slashdot moderation system


Forgot your password?

+ - Test shows big data text analysis inconsistent, inaccurate->

Submitted by DillyTonto
DillyTonto (793644) writes "The 'state of the art' in big-data unstructured data (text) analysis turns out to use a method of categorizing words and documents that, when tested, offered different results for the same data in 20% of the time and was flat wrong another 10%, according to an analysis by researchers at Northwestern. Researchers offered a more accurate method, but only as an example of how to use community detection algorithms to improve on the leading method (LDA). Meanwhile, a certain percentage of answers from all those big data installations will continue to be flat wrong until they're re-run, which will make them wrong in a different way."
Link to Original Source

+ - 42% of Database Specialists Struggle to Manage NoSQL Solutions

Submitted by RaDag
RaDag (3455045) writes "New Forrester research found many database pros struggle with the NoSQL solutions deployed in their environments and 52% admit they are unable to prevent developers from deploying NoSQL databases on their own. Bottom line is the majority — 78% -want one database that can handle all data types. With relational database technologies like Postgres having evolved to do just that, the report urges database pros to consider examining these solutions instead of deploying multiple, specialized databases."

+ - What Makes a Great Software Developer?->

Submitted by Nerval's Lobster
Nerval's Lobster (2598977) writes "What does it take to become a great—or even just a good—software developer? According to developer Michael O. Church’s posting on Quora (later posted on LifeHacker), it's a long list: great developers are unafraid to learn on the job, manage their careers aggressively, know the politics of software development (which he refers to as 'CS666'), avoid long days when feasible, and can tell fads from technologies that actually endure... and those are just a few of his points. Over at Salsita Software’s corporate blog, meanwhile, CEO and founder Matthew Gertner boils it all down to a single point: experienced programmers and developers know when to slow down. What do you think separates the great developers from the not-so-fantastic ones?"
Link to Original Source

+ - Patent Trolls and Trial Lawyers->

Submitted by Jayson
Jayson (2343) writes "Steve Malanga takes down patent trolls and trial lawyers. Automated Transactions to the EFF to Judge Posner to sewing machines in a (very) long form article.

In fact, the opposition ran deeper than the trial bar, threatening future patent reform. Flaws in our patent system, which the distinguished appellate judge and law professor Richard Posner dubs “dysfunctional,” have transformed the technology market, making ceaseless litigation lucrative not only for Automated and patent trolls like it, but for others, too.


Link to Original Source

Comment: Blame the greenies (Score 1, Insightful) 118

by Jayson (#37011448) Attached to: L.A. Artist Contemplates Future Traffic Flow, With Hot Wheels

It has the environmentalists' dream for years now for high-density living with mostly public transportation. What they call sprawl and fight tooth and nail against is what most of us call personal space, a yard for children, and a nice house. They would rather cram everybody in the smallest area possible and hope we all convert to riding buses because the traffic congestion is so bad. Urban planning classes pump out more and more people with this same view every year, so I can only imagine it getting worse.

Comment: Re:I am an HFT programmer (Score 1) 791

by Jayson (#36936992) Attached to: How and Why Wall Street Programmers Earn Top Salaries

FYI, I'm not the original poster, but another HFT developer. That is the only >$1mm loss I've ever had, but have had smaller, and it wasn't exactly for what some consider HFT, but for a trend prediction system. It was for an overseas illiquid market across a couple instruments and we had a bug elsewhere that didn't count our position properly.

Comment: Re:I am an HFT programmer (Score 2) 791

by Jayson (#36936962) Attached to: How and Why Wall Street Programmers Earn Top Salaries

To some extend yes, but the order management and routing systems have checks and there are circuit breakers to prevent total trainwrecks. It is a cost/benefit thing. How likely am I to lose $10k immediatey verus make $50k on the market close? One very important skill is in being able to estimate how likely you think you are right.

I wouldn't do this for huge programs or where the lost can be gigantic or I couldn't evaluate the risk.

Trying to cut corners, I did lose $1.6 million one day because I had a bug in my code. A lot of people have these stories when working on some high-risk projects (something HFT place usually try to stay away from). I learned from my mistake and more than made up for it a week later.

Comment: Re:J/MW? (Score 1) 410

by Jayson (#36907230) Attached to: Solar Energy Is the Fastest Growing Industry In the US

That is actually absolute advantage, and people confuse it with comparative advantage all the time.

In comparative advantage, you don't need to be the best at producing anything. You can actually be the worst at producing everything.

With comparative advantage, the weaker producer it taking advantage of the fact that the better producer is much better at producing A than B, even if you are worse at producing both, so you produce B.

That is, no matter how terribly inefficient you are at producing A and B, it still makes sense to trade with you.

Of course, this has nothing to do with how terribly inefficient solar power is.

Comment: "Just give them spoons" (Score 1) 410

by Jayson (#36907158) Attached to: Solar Energy Is the Fastest Growing Industry In the US

As related by Mark Calabria of CATO:

Prof. Friedman visited China in the early 1960s and was taken by a government official to see a public works project. Chinese workers were building a canal. Friedman was struck by seeing everyone digging the canal with shovels. Friedman asked the official, "why no heavy earth-moving equipment?" The official said, "oh, this is a jobs program." So Friedman then says to the official, "then why don't you just give them spoons instead of shovels to create even more jobs?"

Comment: Re:Ron Paul 2012 (Score 1) 499

by Jayson (#36843168) Attached to: Fed Audit's Initial Report Reveals Trillions in Secret Loans

You have it backwards. The dollar has booms or busts and it shows up immediately in the price of gold as gold falls or rises, respectively.

Prices are the result of 2 supply curves and 2 demand curves: the supply and demand for an item and the supply and demand for dollars (more generally, the unit that item is denominated in). Gold's supply and demand curves are fairly static. You cannot simply look at price fluctuations and say therefore the value of gold is rising and falling since you are then ignoring the value of the dollar.

Making things a little more difficult, most currencies move together since we are in a global economy with global trade with central banks that follow similar methodologies. If gold is soaring, but the EUR/USD exchange rate isn't moving too much, it simply means the EUR and USD are inflating together.

The way you see this is by looking at the effects of inflation. This is why so many (poor Keynesian) economists get it so very wrong. They see and flat EUR/USD and assume no inflation and then get really confused by it so they play games with pricing metrics to make it magically disappear. Higher prices cased by inflation will slowing trickle though the economy starting in commodities, moving through the economy finally impacting durable goods and wages on the other extreme (also, this is one of the reason the core CPI and GDP deflator are often poor measure of inflation because they are many years backwards looking).

This also leads to the confusing about "cost push" inflation. The full story isn't that higher oil prices lead to higher goods costs, it is that excess liquidity flows into commodities first and then that liquidity ripples through the economies as contracts renew. All else being equal, if you help the money supply constant and oil prices went up, goods in the rest of the economy would fall. That relative price movement is the market signal to produce more oil. Inflating currencies ruin this pricing signal by showing false signs of scarcity.

Comment: Re:Expanding Money Supply isn't Necessary (Score 1) 499

by Jayson (#36840894) Attached to: Fed Audit's Initial Report Reveals Trillions in Secret Loans

Expanding the money supply isn't necessary for economic growth - what happens instead is that as the economy grows, prices of everything fall, which is a reasonable response to having more stuff around for people to buy.

Deflation is a bad thing. Of the three main functions of currency, a stable unit of measurement is one of them. Falling prices because of lower demand or too much supply is one thing. Those are pricing signals. But if supply and demand do not change, the price shouldn't change. It would be like constantly redefining the inch every year to be larger and larger. Contracts to pay $100 for something then have to content with "well $100 from what year?" That is the same problem you have with inflation.

Comment: Re:Ron Paul 2012 (Score 2) 499

by Jayson (#36840840) Attached to: Fed Audit's Initial Report Reveals Trillions in Secret Loans

So, bright eyes, what would happen if a gold-backed currency could not find more gold to back it?

You are not understanding how a gold standard is operated. It doesn't mean that the amount of money is tied to an amount of gold. It means that the dollar is tied to the price of gold.

For example, right now you would set the dollar to target $1500 per ounce of gold. As gold rose, dollars would be drained from the economy. As gold fell below $1500, dollars would be injected into the economy. We don't need to monkey with the tax rates either (actually, that would be an absolutely terrible idea). You would use open market operations like they do now. Too many dollars (inflation) - sell bonds and extinguish the dollars. Too few dollars (deflation) - buy bonds with newly printed dollars.

People often get this really wrong and assume that a gold standard implies money has to grow at the rate of gold when that isn't true at all. The central bank doesn't even need to own any gold at all to make it work since you are just doing a price target. People also often confuse inflation and deflation with rising and falling prices. They are not the same thing. For example, inflation will put upward price pressure that will result in rising prices - ALL ELSE BEING EQUAL. But scarcity or high demand can also cause rising prices, but that isn't inflation - it is a pricing signal unrelated to inflation.

Comment: Re:Ron Paul 2012 (Score 1) 499

by Jayson (#36840712) Attached to: Fed Audit's Initial Report Reveals Trillions in Secret Loans

Unfortunately, when the population and economy expand, the money supply has to expand with it. When administered responsibly (I.E. not just printing more money to cover debts), a paper money supply can be controlled much more finely. The gold supply expands in a fairly unpredictable way, controlled by how fast mining can be done, which can be completely unrelated to current economic conditions.

Actually the above ground gold stock expands at a fairly steady rate. It has a low industrial demand that tends to match its production. This is one of the reasons gold's value is fairly stable, the most stable over a very long time of anything we know of.

To have a gold standard doesn't mean you need to actually have every dollar backed by gold or use gold directly. It is called a gold target. We say that a dollar is to be worth 1/1500th of an ounce of gold and the money supply is expanded and contracted. This maintains its role of a unit of measurement.

It isn't rocket science or new as to how to make a gold standard work. It is just that politicians like to be in control and use inflation as a tax (Carter's budgets hid out year deficits by predicting inflation pushing people into higher tax brackets it was so institutionalized).

Programmers used to batch environments may find it hard to live without giant listings; we would find it hard to use them. -- D.M. Ritchie