karniv0re's Journal: Adventures in Data Science

I knocked out a little bit of homework today. I've been doing my assignments in LaTeX (specifically ShareLaTeX), which is yet another skill I can kind of get for free while doing things I need to do. My handwriting is atrocious anyway, so I might as well. Plus, I'm in grad school. I should act like it. I'm going at a rate of about 2-3 hours per problem. It's rough. But I'm making progress.

In my spare time, I've been listening to Partially Derivative, a podcast about data science. I'm starting from episode 1, which is about a year old. They talked about this project, 1 CSV, 30 stories, where a dude takes one massive CSV (3.2 GB) and does some data science on it and tries to get 30 stories out of it. Well, a year later, it looks like he stopped a little bit short at 21, but good effort! I'm going through it and trying to see how he did it, starting with bootstrapping (the data munging part). Already, I'm seeing where the troubles arise.

We've gotta munge this text before we can do anything with it. And as we've all seen, human-entered data is never clean, particularly when dealing with plain text.

So I'm currently playing with that just trying to get to the point he starts at. It seems that Data Tools has changed a bit since his blog, so I'm currently trying to get it to work the way it is now.

I'm also working on a work-related DS project that involves a probability model. I'm not sure if I've got the math chops yet after just 3 weeks back in class, but I'm getting some ideas. I think that's helpful to get a real-world scenario to drive home the theory.

I also started reading Antifragile. Reeeeaaaalllyyyy liking the concept of this book. But anyway, gotta cut this one short. More later this week!