Nerval's Lobster writes: "At this year’s SXSW conference, Stephen Wolfram—most famous in tech circles as the chief designer of the Mathematica software platform, as well as the Wolfram Alpha “computation knowledge engine”—demonstrated his upcoming Programming Cloud, and indicated he was developing a mobile platform for engineering and mathematical applications based on the Wolfram programming language built for Mathematica. He also talked more broadly about the future of Wolfram Alpha, which he said will become more anticipatory of peoples’ queries. “People generally don’t understand all the things that Wolfram Alpha can do,” Wolfram told the audience. His researchers are also working on a system modeler tool, which will allow researchers to simulate complex devices with tens of thousands of components; in theory, you could even use such a platform for 3D printing. Wolfram also wants to set Wolfram Alpha loose on documents, with the ability to apply complex calculations to, say, company spreadsheets. “A whole bunch of things that I’ve been working on for 30 years are converging in a very nice way,” he said."
Nerval's Lobster writes: "Nate Silver feels a little odd about his fame. That’s not to say that he hasn’t worked to get to his enviable position. Thanks to his savvy with predictive models, and the huge readership platform provided by The New York Times hosting his FiveThirtyEight blog, he managed to forecast the most recent presidential election results in all 50 states. His accuracy transformed him into a rare breed: a statistician with a household name. But onstage at this year’s SXSW conference, Silver termed his fame “strange” and “out of proportion,” and described his model as little more than averaging the state and national polls, spiced a bit with his algorithms. “It bothered me that this was such a big deal,” he told the audience. In politics, he added, most of the statistical analysis being conducted simply isn’t good, which lets someone like him stand out; same as in baseball, where he made his start in predictive modeling. In fields with better analytics, the competition for someone like him would be much fiercer. He also talked about, despite a flood of data (and the tools to analyze it) in the modern world, we still face huge problems when it comes to actually understanding and using that data. “You have a gap between what we think we know and what we really know,” he said. “We tend to be oversensitive to random fluctuations in the data and mistake the fluctuations for real relationships.”"
Nerval's Lobster writes: "Of late, there’s been a number of crossovers between technology and entertainment, including the rash of creative directors at brands like Polaroid (Lady Gaga), Intel (Will.i.am) and BlackBerry (Alicia Keys). It’s a much rarer thing, though, for rock stars to invest in infrastructure, rather than serve as the “face” of a brand. But that's exactly what three members of the rock band Live, which sold 20 million albums in their '90s heyday, are doing as their second act: investing in a company that plans on building a 100-Gbit fiber link across Pennsylvania to four data centers. That company, United Fiber and Data (originally known as United Federal Data), will build out a network between New York City and Ashburn, Virginia—providing a low-latency data pipeline that connects offices in Virginia to data centers owned by Wall Street. Supposedly in the name of security, the network will avoid the traditional I-95 corridor, a more direct route used by many other networks."
Nerval's Lobster writes: "Computer games are big business, with millions of players and billions of dollars in revenue every year. But that popularity puts game studios in a tough spot, especially when it comes to mobile games that need to serve their players a constant stream of updates and rewards. That pressure is leading to an interesting phenomenon: while IT companies that create more "serious" software (i.e., productivity apps, business tools, etc.) are often viewed as cutting edge, it might be game developers actually doing the most innovative stuff when it comes to analytics, cloud and high-performance computing, and so on. Broken Bulb Studios, Hothead Games, and some other studios (along with some hosting companies) talk about how they've built their platforms to handle immense (and fluctuating) demand from gamers."
Nerval's Lobster writes: "The Apache Hadoop open-source framework specializes in running data applications on large hardware clusters, making it a particular favorite among firms such as Facebook and IBM with a lot of backend infrastructure (and a whole ton of data) to manage. So it’d be hard to blame Intel for jumping into this particular arena. The chipmaker has produced its own distribution for Apache Hadoop, apparently built “from the silicon up” to efficiently access and crunch massive datasets. The distribution takes advantage of Intel’s work in hardware, backed by the Intel Advanced Encryption Standard (AES) Instructions (Intel AES-NI) in the Intel Xeon processor. Intel also claims that a specialized Hadoop distribution riding on its hardware can analyze data at superior speeds—namely, one terabyte of data can be processed in seven minutes, versus hours for some other systems. The company faces a lot of competition in an arena crowded with other Hadoop players, but that won't stop it from trying to throw its muscle around."
Nerval's Lobster writes: "If you live in Hawaii, congratulations: according to a new study by researchers at the University of Vermont, you live in the happiest state in the union—at least as far as Twitter sentiment is concerned. (Hat tip to The Atlantic for posting about the research.) The researchers—affiliated with the University’s Department of Mathematics & Statistics, Complex Systems Center, Computational Story Lab, and Advanced Computing Core—collected 10 million geo-tagged Tweets from 373 urban areas across the United States in 2011 and ran them through a system designed to tag each on a scale from 1 (sad) to 9 (happy). According to the study, the five happiest states include Hawaii, Maine, Nevada, Utah and Vermont; the five saddest are Louisiana, Mississippi, Maryland, Delaware and Georgia. In general, the West and Northeast seemed much happier than the Mid-Atlantic and South—with the exception of Florida, which shaded “happier” than many of the surrounding states. While the researchers admitted their study’s limitations, there are certainly a lot of opportunities for refining the model: for example, if Hawaii’s status as a vacation state affects its rate of “happy” Tweets, or if incorporating languages other than English into the dataset would affect the ultimate results."
Nerval's Lobster writes: "Decades after the space race pitted the United States against Russia, a new race has emerged: the race to map the human brain. The New York Times reported Feb. 18 that the Obama administration is gearing up to announce the Brain Activity Map project, an effort to map an active human brain that could give new insight into how neurons interact with each other, providing new avenues of research for diseases such as Alzheimer’s. The U.S. will apparently pit itself against a collection of European research agencies that have announced similar projects. The U.S. effort, however, will apparently involve U.S. businesses, which would naturally benefit from the high-profile nature of the effort; in theory, the latter could also apply the resulting discoveries to their own computing efforts. The Times reported that representatives from Google, Microsoft, and Qualcomm met with government representatives at the California Institute of Technology to try and figure out whether or not there are sufficient computing resources to process the vast amounts of data that the experiments are expected to produce, or whether new ones would need to be built."
Nerval's Lobster writes: "Whatever your actual feelings about football and this weekend's Super Bowl, you have to admire Wolfram Alpha's willingness to crunch any dataset and see what it can find. The self-billed “computational knowledge engine” has analyzed the historical data for both teams involved in this Sunday’s Super Bowl XLVII. Its conclusion? The San Francisco 49ers and Baltimore Ravens are "annoyingly similar" when it comes to numbers, although some players stand out as potential game changers — if the math plays out right."
Nerval's Lobster writes: "A presidential campaign is many things to many people: a reason to hope in the future, a wellspring of jokes and debate fodder, an annoyance to tune out, a chance to participate in the civic process. But for a couple dozen software engineers and developers involved over the past two years in President Obama’s re-election effort, a campaign was something entirely different: a billion-dollar tech startup with an eighteen-month lifespan and a mandate to ship code under extreme pressure. Speaking to a New York City audience, some of Obama for America’s leading tech people—those involved in the all-important Dashboard and Narwhal projects, as well as fundraising and DevOps—characterized the experience as “insane,” filled with unending problems and the knowledge that, at the end of the whole process, nearly everything they worked on would likely end up tossed away. This is the story of what happened, and how technologies on a massive scale can make or break campaigns."
Nerval's Lobster writes: "Back in April 2010, the Library of Congress agreed to archive four years’ worth of public Tweets. Even by the standards of the nation’s most famous research library, the goal was an ambitious one. The librarians needed to build a sustainable system for receiving and preserving an enormous number of Tweets, then organize that dataset by date. At the time, Twitter also agreed to provide future public Tweets to the Library under the same terms, meaning any system would need the ability to scale up to epic size. The resulting archive is around 300 TB in size. But there’s still a huge challenge: the Library needs to make that huge dataset accessible to researchers in a way they can actually use. Right now, even a single query of the 2006-2010 archive takes as many as 24 hours to execute, which limits researchers’ ability to do work in a timely way."
Nerval's Lobster writes: "Can data-analytics software win a Super Bowl? That’s what the Buffalo Bills are betting on: the NFL team will create an analytics department to crunch player data, building on a model already well established in professional baseball and basketball. “We are going to create and establish a very robust football analytics operation that we layer into our entire operation moving forward,” Buffalo Bills president Russ Brandon recently told The Buffalo News. “That’s something that’s very important to me and the future of the franchise.” The increased use of analytics in other sports, he added, led him to make the decision: “We’ve seen it in the NBA. We’ve seen it more in baseball. It’s starting to spruce its head a little bit in football, and I feel we’re missing the target if we don’t invest in that area of our operation, and we will.”"
Nerval's Lobster writes: "What's the most optimal path for Santa (if he actually, you know, existed) to travel around the world on Christmas Eve? The answer is a variation of the classic "Traveling Salesman Problem." According to a history of the problem compiled by Georgia Tech, the traveling salesman problem (or “TSP”) dates back to the 1800s, when Irish mathematician Sir William Rowan Hamilton challenged friends to navigate 20 points in the shortest path using only a series of specified connections. But it wasn’t until the first decades of the 20th century when statisticians began to tackle the challenge in a more systematic way. The World TSP Problem involves 1,904,711-city instance of locations throughout the world, with a most-optimized route of 7,515,778.188 kilometers (4,670,090 miles). The "Santa problem" would be even more complex — hope those reindeer are well-fed."
Nerval's Lobster writes: "Facebook is letting users vote on changes to its Data Use Policy and Statement of Rights and Responsibilities (Facebook users can vote via this link). The company will also host a live Webcast to answer questions at 9:30 AM PST. One section of Facebook’s revamped policies insists that the network can share information with its family of companies. This apparently applies to Instagram, the photo-sharing service acquired by Facebook earlier this year. Under the terms of the provision, Facebook can store “Instagram’s server logs and administrative records in a way that is more efficient than maintaining totally separate storage systems.” Facebook is also clarifying its language surrounding affiliates, as well. As long as Facebook continues to exist in its current form, these debates over its privacy rules will almost certainly continue to crop up on a semi-regular basis. The challenge for Facebook executives is how to best maintain that delicate dance between their need for revenue, advertising firms’ desire for effective marketing campaigns, and users’ rights to privacy. They run a corporation—but at moments, it also starts to resemble a messy democracy."
Nerval's Lobster writes: "On Election Day, as millions of people headed to the polls to vote, Google’s Consumer Surveys team pushed out a survey with questions about the election. Google managed to collect some 10,000 samples between 2 AM PST and 11 PM PST, or the equivalent of one data point every 10 seconds. “Now that we’ve collected the dataset, we want to provide it to the community to see what interesting analyses, insights, and visualizations you can discover,” reads a note on the Consumer Surveys team’s Webpage. “This is a quest for the data scientists, engineers, statisticians, and political scientists out here.” It’s a contest of sorts: whomever submits what Google views as the “most insightful” visualization or analysis of data will earn “one coupon towards free research using Google Consumer Surveys,” which isn’t exactly a new Wii U, but could interest those who work with datasets on a regular basis. The deadline for submissions is Dec. 20; the actual results of the survey are available here."