Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×

Comment Re:Please specify a better scenario (Score 1) 272

Instead of "sharding" (split customers across multiple copies of the database) you should try a NoSQL solution to handle the flood of writes as the first layer. Then an recurring process can query the data in your NoSQL object store (by timestamp) and aggregate it into an SQL database for reporting. You could archive those processed entries, or wait until they get old, to another object store for your "data warehouse" -- basically just an archive in case you need to do different aggregate reporting in the future (depending on storage size of course).

I must ask, do you really need to store each full piece of information written by these clients at such a high volume?

Depending on your use of the data, you could even just store the results in memory for X hours/minutes, and then aggregate-process that and write the results to your SQL DB. A single DB with many application servers would be fine in this condition, with writes every X hours/minutes. (You are probably already flat-file logging the incoming requests; that is an archive if you *really* need to go back.) If you cannot afford memory loss if an app server dies, solutions like EhCache (java) will persist the memory to disk, in case of hardware/software failure.

Comment Re:Use PostgreSQL (Score 1) 272

Was your 5000 tps using normal insert/update/delete statements or using the COPY statement? (I guess it's a form of batching: meaning, you issue large copy statements instead of many insert statements, if your application can data that way.)

Also, was your hstore experience with 9.3+ or what version(s) had problems?

Comment Crowd Funding (Score 1) 480

Do you see crowd funding as a means to economically provide for the development of Free Software and even Free Hardware?

Have any small projects grown to a critical mass where only a few funding rounds bootstrapped them into having a sustainable Libre/Free product?

Privacy

Supreme Court Ruling Relaxes Warrant Requirements For Home Searches 500

cold fjord writes with news that the Supreme Court has expanded the ability of police officers to search a home without needing a warrant, quoting the LA Times: "Police officers may enter and search a home without a warrant as long as one occupant consents, even if another resident has previously objected, the Supreme Court ruled Tuesday ... The 6-3 ruling ... gives authorities more leeway to search homes without obtaining a warrant, even when there is no emergency. The majority ... said police need not take the time to get a magistrate's approval before entering a home in such cases. But dissenters ... warned that the decision would erode protections against warrantless home searches." In this case, one person objected to the search and was arrested followed by the police returning and receiving the consent of the remaining occupant.
The Internet

German Chancellor Proposes European Communications Network 197

An anonymous reader sends word that German Chancellor Angela Merkel wants to build a European communication network to keep data transmission away from the United States. She plans to discuss the issue with French President Francois Hollande. "Merkel said in her weekly podcast that she disapproved of companies such as Google and Facebook basing their operations in countries with low levels of data protection while being active in countries such as Germany with high data protection. 'We'll talk with France about how we can maintain a high level of data protection,' Merkel said. 'Above all, we'll talk about European providers that offer security for our citizens, so that one shouldn't have to send emails and other information across the Atlantic. Rather, one could build up a communication network inside Europe.' Hollande's office confirmed that the governments had been discussing the matter and said Paris agreed with Berlin's proposals."

Slashdot Top Deals

IN MY OPINION anyone interested in improving himself should not rule out becoming pure energy. -- Jack Handley, The New Mexican, 1988.

Working...