rockmuelle writes: I work in a 'Big Data' space (genome sequencing) and routinely operate on tera-scale data sets in a high-performance computing environment (high-memory (64-200GB) nodes, 10 GigE/IB networks, peta-scale high-performance stroage systems). However, the more people I chat with professionaly on the topic, the more I realize everyone has a different definition of what consitutites big data and what the best solutions for working with large data are. If you term yourself a 'big data' user, what do you consider 'big data'? Do you measure data in mega, giga, tera, peta-bytes? What is a typical data set you work with? What are the main algorithms you use for analysis? What turn-around times are typical for analyses? What infrastructure software do you use? What system achitectures work best for your problem (and which have you tried that don't work well?)?
rockmuelle writes: "We are pleased to announce the latest release of CorePy, now with full support for x86
processors (32 and 64-bit) and an Open Source license.
CorePy is a Python package for developing assembly-level applications
on x86, Cell BE and PowerPC processors. Its simple APIs enable the
creation of complex, high-performance applications that take advantage
of advanced processor features usually inaccessible from high-level
scripting languages, including multiple cores and vector instruction
sets (SSE, VMX, SPU). Based on an advanced run-time system, CorePy
lets developers build and execute assembly-level programs
interactively from the Python command prompt or embed them directly in
Python applications. CorePy is available under a standard BSD