Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
What's the story with these ads on Slashdot? Check out our new blog post to find out. ×

Submission + - Raw Data Collection Software? 1

An anonymous reader writes: I have been looking for a tool (hopefully open source) that would allow some more intelligent management of data collection. Even better would include some data processing. Specifically I am looking to manage and chronicle raw data collected on hardware (in my case engines) and create a timeline of events (engine shut down, engine repair, parts swap, etc) allow for commenting and searching, etc Is there a good tool for this? I haven't been able to find one. Thanks for the help!

Currently, we just take each file and dump it into a share drive. It's not easily searchable and there is no good means for batch processing it. Data gets lost all the time and multiple groups collect data from the same engine. If we have a question about a particular behavior, we can't search these, we rely on someone remembering about a test and where they put it. PLEASE HELP!
This discussion was created for logged-in users only, but now has been archived. No new comments can be posted.

Raw Data Collection Software?

Comments Filter:
  • by rwa2 (4391) *
    Well, this is kinda abusing a logfile collection utility, but give Logstash [logstash.net] + Kibana a try. It's very fast, and sounds perfect for the kinds of "events" you're looking at:

    howto [vmdoh.com]

    You could also try paying for something like Splunk, which is a bit slower, but gives you a much richer analysis & graphing toolset (though it's frustratingly difficult to do some things that you'd think should be simple, like plotting multiple searches on a timeline so you can do correlations.)

    Now if you want to graph lot

"The Avis WIZARD decides if you get to drive a car. Your head won't touch the pillow of a Sheraton unless their computer says it's okay." -- Arthur Miller

Working...