Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!


Forgot your password?
Check out the new SourceForge HTML5 internet speed test! No Flash necessary and runs on all devices. ×
User Journal

Journal Journal: Miscellany Across the Spectrum

I don't have anything terribly specific to write about, but lots of little things.

Looking back nearly two years ago provides a pretty good picture of where I'm at. This tidbit from November of 2014 provides some insight:

After re-reading my last journal entry, man. This year has sucked. Shit was bleak.

That was from a journal entry titled Things Are Looking Up. But things never went up. I never got that architect position, and no more than a week later, the lady fell sick with Meniere's, which has been the centerpiece of our lives ever since.

Since then, it's been a war of attrition to see whether we can outlast our problems or if our problems outlast us.

I did end up going back to school, and I'm still in school. It's been a mixed bag of good and bad. I'm glad I'm learning things, and I'm glad I'm progressing toward a degree. I hate how much stress it causes me and how much time it eats up. I'm getting super out of shape because I don't have time to workout. I haven't been to the gym in probably about a year. I still run when I can, but it's not enough to counteract the booze and comfort food. I have cut back a bit on drinking. I don't keep bourbon around anymore. My regular mix of a couple beers and a couple manhattans from 2015 was slowly killing me. So I'm down to around 2-4 beers a night. Not great, but better.
I actually don't like being drunk anymore. I associate it with bad times now. So now, I prefer to be sober but still drink beers. It's a balance I guess. I expect that as I get older, and stress goes down (hopefully!), I'll drink even less.

But 2014 - 2018 (possibly even into 2019) maybe the most stressful years of my life. I feel like I can't relax until the Lady is done with school, boards passed, job acquired, and has been working for at least a year. In a way, I'm kind of glad 2016 is nearly over. I put this year on the calendar as a year to "get through." There were no incentives this year to speak of. None to look forward to.

I moved back over the the Glass Palace, giving up my awesome old cube and nice place in the company for a more anonymous role. Although it wasn't a great role there either. But at least I could focus on school. Now I've got a project due AND two classes. It's killing me. If I make it through this semester, I'm not sure what other challenges you could throw at me that would be tougher. I mean, I shouldn't say that. I thought I was being challenged a year ago with my Probability Models class. But Bayesian Stats + Data Science + work project + girlfriend with health issues is pretty much maxing me out.

Do I even need to mention how terrible the election has been? But I can't look away. It's such a fucking trainwreck. It also makes me question if I'm really a good person. I've done plenty I'm not proud of. Who really is good anyway? We're all sacks of waste waiting to die and fertilize the soil.

Maybe I'll list some good news. I did talk with my old boss and he's interested in bringing me on his team. He runs the Data Science team here. So that would be some great experience for me, and a definite change of pace. I'm going to talk with our AVP here pretty soon about that and my final school project. I want to work on ETAs. I think that'll be a great project. And next semester I'm taking a Time Series Analysis class, which will help in that analysis.

It looks like I will be out of debt by (revision number 920442) bonus time in 2018. So a little over a year to go. We'll see what pops up along the way. Kind of annoying that I'll be 35 just getting out of debt. Ridiculous, actually. But this is where we're at.

I have been toying with the idea of getting a PhD but I really don't know. Seems like an awful lot of effort for those letters. But I know it'd be huge for me. I'll probably continue to ponder it through 2017. Leaving it on the table.

But for now, I have a mobile app to build. I have roughly two weeks to put the finishing touches on the code. Time to earn my bonus.

User Journal

Journal Journal: Growing in Data Science

I'm taking two classes this semester. Stepping it up from the single-class semesters. I think it'll be ok though. I'm in Intro to Data Science and Bayesian Statistics. Man, a year ago, I was so excited to be in a DS class. And now, here I am, and it feels like bush league. It's because I've been studying this stuff so intensely for the last year, doing things the right way, no shortcuts, doing my own research, listening to DS podcasts, etc. I'm not terribly surprised when our professor is talking about reproducible research using Knitr. Got it, dude. I've been rolling with RMarkdown for at least a semester, and LaTeX even longer.

And despite how it sounds, I'm trying NOT to be a know-it-all. But it's hard. Because I've all the things we're learning in this class are things I've been doing on my own in other classes. So I'm just trying to keep my mouth shut and try to learn a thing or two. And I have! In learning R, I've skipped over a lot of the programming language aspects of it. I read somewhere, R is meant to be learned in tandem with statistics. So that's what I've been doing. And in doing so, I've skipped a lot of the fundamental programming language stuff that I would normally learn when learning any other language. So now's a good time to start picking that stuff up.

But to the title of this post, we've got a semester-long project we have to work on, and on a long run today, I was thinking about project ideas. Then I got to thinking about why it's so hard to come up with good DS projects. Here is my attempt to explain this, which can be used as a catalyst to explain why data science will never be fully automated, at least until we have artificial creativity (still a long way from reality).

1) There are two high level purposes of machine learning - classification and prediction.

So far so good. I think we can all agree on this. We're either trying to put things in buckets or guess what the next value is going to be. Of course it gets a lot more complicated when you dive in, but at the high level, that's it.

2) There are 5 types of predictions we can make (I came up with these off the top of my head while on a run, so don't take this as gospel).
2a) Natural phenomena - weather, stock market, global economies.
2b) Human phenomena - Baseball players hitting home runs, football teams winning the Super Bowl, how much I will weigh next year, will you click this ad, will you buy this product
2c) Social phenomena - Presidential elections, who you might want to follow on Twitter, Data Scientist wages
2d) Games of chance - Poker, Blackjack, dice, flipping coins
2e) Games of strategy - Literally any board game with multiple players

(There are almost certainly others, but this is seriously all I can think of and it makes sense to me)

Furthermore, there are three stages to data science (ONCE AGAIN, MAKING THIS UP AS I GO):

I) Asking questions, designing the problem
II) Modeling the problem, delivering results
III) Making decisions on those results

I am coming to the conclusion that data science doesn't spend enough time on I and III. It focuses so much on II, the technical side, that it forgets to even teach us to ask questions or tell anyone why the model is useful. And maybe in many scenarios, the business teams, managers, VPs and CEOs perform I and/or III. But I am somehow in a position where I need to prove why DS is useful. All the technical knowledge in the world ain't gonna make that happen. I need to be able to apply it. So really, I and III are about advertising ourselves, somewhat. Machine Learning without purpose is just a cool parlor trick. But use it to answer a really a good question and then perform some action to make life better, and you're a superhero.

So anyway, that was my key insights today as I hauled in 8 miles. If I'm way off on this shit, let me know in the comments.

User Journal

Journal Journal: Learning Data Science at t_1

Nearly a year ago today (give or take a few hours), I formally began this path toward Data Science. We'll call that t_0.

Back then, I had the faintest idea of what Data Science was. It seemed to me to be a collection of tools and programming languages. Like, oh, if I know R, or MapReduce/Hadoop/NoSQL, I'm a data scientist. Well, I'm happy to report, in the year that I've been studying Data Science, I know enough to tell you that it is a lot more than that. I am now on my fifth probability and/or statistics class. I've had one Big Data class (that focused on NoSQL and Hadoop) and I can say, it was probably the least of my concerns. Data Science is not just some extension of being a software engineer. Sure, being a Software Engineer will make a lot of things easier, but being a data scientist is an extension of being a statistician who incidentally has some software engineering skills.

I will still say, the biggest help to me throughout grad school thus far has been my professional background. I see all these kids going straight through from undergrad to grad, and they're still just students. There's really no professionalism in them. I've hit straight As or A+s in my first 4 classes, and I ascribe that entirely to having a strong professional background. I approach school like I do my job. I'm not trying to just pass the tests, but rather completely understand the subject.

So today, a year later (let's call this t_1), I am feeling smarter. But not by any means ready. If I were to be hired as a data scientist today, I would probably be in an intro level job. And I wouldn't blame someone for putting me there. I'm basically starting from scratch (with a little software engineering background to help me out). I am hoping I can sneak in some actual real-world work before I graduate to give me a foothold though.

I may have a little consulting work in the healthcare field, thanks to the Lady. I will definitely be able to help her with her Capstone project, but may also be able to help out one of her professors with data he collected on recertification of CRNAs and whether it is effective to do so. I'm kind of itching to do something like that. It's out of my realm of expertise, so it'd be cool to get healthcare data experience. Do I have time? Eh, I think I could make time probably.

My current work project is somewhat coming along. After hitting the point where I decided it all needs rewritten, things started moving faster. We just finished up a demo to our customer, and they approved. I think this is going to go nicely. Time to get the backend services knocked out.

Alright, well anyway. Intro to Data Science and Bayesian Statistics this semester. Two classes. But I think Intro to DS will be a piece of cake. Bayesian, though... We'll see. I want to keep my 4.0. Gonna have to keep pushing.

User Journal

Journal Journal: Big Data, Distribution-Free Stats, and Mobile Apps

Just a quickie before I start my next summer class. I just finished Big Data a few weeks ago. It was really a piece of cake, but also kind of awesome.

Remembering my entry from September 10th, 2015 Dear Gournal. The reason I was so confused is that these are all so new, and classifying them requires a good deal of insight. Well, my Big Data class gave me that, so at least I know what each of them are and what they do. Solid win.

With the mobile app, I at least have a plan. A plan is better than no plan.

And now I need to go to my Distribution-Free Stats class. Should be pretty straightforward, but my teacher seems all over the place. We'll see.

User Journal

Journal Journal: More Cordova Hell

By now, I am desperate.

This fucking app has advanced exactly 0 measurable units in the last 3 weeks. I have tried a lot of things, but I always get stopped a few steps in by more shit breaking. This is insanity.

I'm going to continue documenting my struggle, in the hopes that I figure something out and am able to articulate why this has taken so long.

Since the app I have is completely fucked as-is, I need to start small and add in. I will start with a clean slate. A sample app. My goal, by the end of this, is to have a working app that can take
a picture with my phone's camera. Doesn't have to do much else.

I will start with this sample app:

        git clone https://github.com/gizur/cordova-camera-example

Obeying the readme.md, I started in the directory and did

        npm install

        cd www

sh.exe": cd: www: No such file or directory
Great. I'll assume he means www-src and move on.

        cd www-src
        bower install

sh.exe": bower: command not found

        npm install -g bower

        bower install

bower retry Request to https://bower.herokuapp.com/packages/remote failed with ECONNRESET, retrying in 1.6s ...

Googled grunt ECONNRESET: https://github.com/bower/bower/issues/467

Googled "where is .gruntrc", because honestly, I don't even know where npm is putting shit. Find out it's in AppData\Roaming\npm\node_modules\bower

Settled on home directory.

Trying to update .gruntrc.
Error: Windows Explorer tells me I can't do that and that I must type a filename.
Find out, I can't do that in Explorer and have to do it on the command line.

        mv bowerrc.txt .bowerrc

Added to .bowerrc:

    "directory": "library",
    "registry": "http://bower.herokuapp.com",

        bower install
        cd ..

        cordova plugin add cordova-plugin-console

Error: Current working directory is not a Cordova-based project.

And at this point, I'm realizing this isn't going to work.

Starting over.


        cordova platform add android --save
        cordova build android --verbose

Copy platform\android\build\outputs\apk\android-debug.apk to phone and try to install.

Error: App not installed

Deleted old app and tried again. App installed.

Now I'm back to a starting point.

User Journal

Journal Journal: Mobile Hell

I've been trying to write a mobile app for about 5 years. What a fucking joke. Nobody even uses apps anymore.

I mean, sure, we use a few. But what was the last app you downloaded that you regularly use? Probably Facebook or Twitter. Probably not Billy Joe's Bait Shack app. You just go to his fucking website.

Anyway, my company wants an app. It kind of makes sense, and I can certainly do some cool things. But I'm inheriting an app that was already half-assed, and now I have to make it work. I am running
into so many problems.

So here's the specs:

It's an Apache Cordova app with some native parts to Android and iOS. The only other technologies are Backbone.js and Require.js. But it's also about 4 years old and age has not treated it well. And
I'm finding that the tutorials for Cordova apps are shit. So I'm going to try to follow one and see how far I get before I lose my shit.

I am here: https://ccoenraets.github.io/cordova-tutorial/create-cordova-project.html. Please follow along.

I already have Cordova, so I'll update:

        $ npm update -g cordova

That worked.

Ok, now I'll create my sample project:

        $ mkdir workshop
        $ cordova create workshop com.blah.workshop Workshop

        Error: Cannot find module 'config-chain'

Uhh. Ok.

        $ npm install -g config-chain
        $ cordova create workshop com.blah.workshop Workshop

        Error: Cannot find module 'umask'

Ok, I see where this is going.

        $ npm install -g umask
        $ npm install -g npmlog
        $ npm install -g uid-number
        $ npm install -g which
        $ npm install -g npm-registry-client
        $ npm install -g chownr
        $ npm install -g dezalgo
        $ npm install -g npm-cache-filename
        $ npm install -g char-spinner

        Creating a new cordova project.

Finally! Now I can create my Android platform.

        $ cordova platforms add android

        Error: Failed to fetch platform android
        Probably this is either a connection problem, or platform spec is incorrect.
        Check your connection and platform name/version/URL.
        Error: Cannot find module 'path-is-inside'

OMFG. More.

        $ npm install -g path-is-inside
        $ npm install -g fs-vacuum
        $ npm install -g async-some
        $ npm install -g fs-write-stream-atomic
        $ npm install -g fstream-npm
        $ npm install -g sha
        $ npm install -g normalize-git-url
        $ npm install -g realize-package-specifier
        $ cordova platforms add android

Finally that's done. Now we can move on with the tutorial.

        $ cordova plugin add org.apache.cordova.device
        Error: Registry returned 404 for GET on https://registry.npmjs.org/org.apache.cordova.console

You have got to be fucking kidding me. Thankfully, Andre had a useful comment:

For point 8, replace:

cordova plugin add org.apache.cordova.device
cordova plugin add org.apache.cordova.console


cordova plugin add cordova-plugin-device
cordova plugin add cordova-plugin-console

That worked.

        $ cordova build android --verbose

Now, I SHOULD be able to run this directly on my plugged in mobile device. I mean, it shows up in chrome://inspect/#devices.

        $ adb devices

No devices show up. It says I probably need the USB drivers, but I supposedly have them. So I guess I'll have to copy the file to my phone manually.

But you know what? That works!!!! Thank the fuck Christ. That gives me a Hello World. Now for the hard part...

User Journal

Journal Journal: Idea For a Bar

I was on the bus on my way to school and I thought of this great idea for a bar. So I'm going to write it down here so someday when I'm rich enough, I can make it happen.

I'd call it The Library, or something similar, and it would be almost a literal library.

The inside would be similar to the New York Public Library, with green lamps on desks. There would be actual books there, but like the old looking ones.

Here's the best part: The basement (cause there has to be a basement) would be the quiet section for people who actually want to study. We'd have coffee, and of course beer, wine, and cocktails. And here's the other kicker - I would have a broad range of prices. Cheap PBR and whiskey for the budget conscious, and higher end cocktails for people who want to feel fancy.

Oh, and there should be a "call" button for the waitresses so they're not always coming down and bothering you. (Why don't we have these already?!)

I would use data science to select the right building, market, handle the finances, etc. I would need a partner on this, someone who can run a bar. I just want to finance it and make my dream a reality. But I guess I need some money first. hrm. I'll come back to this when I have some of that.

User Journal

Journal Journal: Gets Worse

At least it seems like things are getting worse.
My weekend started off with annoyances by our architect on Friday. He was so goddamn worried about how we were going to format our code. He's contacted me on at least 6 different occasions every time it was brought up. He's so worried I'm going to throw some crazy book of rules at him. For fucks sake. I just want it to be formattable and not looking like shit.

And then another fuckhead at the company I'm consulting for made some stupid comment about me being a Millennial when I said I didn't have a car. What the fuck does that even mean? That we're breaking from the Baby Boomer and Gen X trends of destroying our planet and making better decisions about how we consume? Fuck you asshole. Of course he drives a Benz SUV. What a cock.

So that went with me into my weekend. Then I spent the day sharing the apartment as study space with my ever-increasingly disgruntled girlfriend. Her shit is getting tough, so now she's starting to crack. I can't fucking deal with this. Today, I started the day off with her calling me because she got pulled over by a cop because she was driving in the left lane on the interstate, but there were people in the right lane so she couldn't pull over. He flashed his lights at her and she didn't know he just wanted her to get out of the way, so she pulled on to the left shoulder, which is illegal. So he ticketed her for driving in the left lane and asked her where she learned to drive. Thanks asshole, for setting my day off wrong. Fuck the police if they're going to be unprofessional assholes.

Then she failed her patho-phys test and her teacher called out her Meniere's in front of the class. So I've been trying to assuage her feelings of uselessness once again and telling her that quitting is not an option. Fuck me. I don't know how many times I've had to do this. It felt like 2015 was this, and only this, over and over. But for a while, I felt like we were coming out of it. But I guess it's always going to be there, rearing its ugly head. Fuck all this.

I know, there's not much tech stuff here. Sometimes I just need a journal to rant in. I'm trying to figure out a bunch of shitty unit tests that aren't unit tests. They're all integration tests, and not even half of them pass. So I'm also dealing with that. There. Tech.

Alright, I'm out. Just wanted to bitch.

User Journal

Journal Journal: Quick-ish Rant

I feel the need to rant.

I am drunk, and this time it's not for "just because!".

We have a new architect (who is not me) and he has been spraying his scent everywhere to stake his claim. Outright, I can't say it's bad. He's ushering in TDD (which I am iffy about, given our limited time to market), collaboration (because he's remote), and some poorly thrown jabs at me, that I assume were meant to be humorous but as I already mentioned, they fell flat on the floor.

I don't know how to feel. On the one hand (the pro-YAY, NEW ARCHITECT THAT IS NOT GOING TO FUCK THIS UP hand), I'm ok with it. Ya know, it can't be worse than how it's been. We got rid of our worst asshole. So what am I worried about? Well, he's not at all cognizant of our current environment. And a large part of me says "OOOhhhh, so you're basically a big pussy who can't accept change!? OooooOhhhhhHHHHH!!!! FUCK YOU." Thanks, dick-me. Back atcha. But really, dick-me has a point.

It's hurting my sensitive little feewings a bit, and it's disrupting my day-to-day habit. So that's bothering me. It also kind of bothers me that this dude rolls in without giving much fuck to what we've already built. Maybe that's what's driving me the most nuts. C'mon, man. You can't roll into the Taj Mahal and say, "Wow. This needs redone. Let's start here..." which is pretty much how I feel this is going.

But I'm in a pretty precarious spot. I am trying to get through this Master's program, so I need things stable. I can't be rocking boats, and obviously, there aren't many boats on my side of the navy. So I guess I need to bite my lip. It's really not all bad, he has good experience, and honestly, I think in any other situation, I would value it a lot.

I just hate the feeling of a storming general coming in and telling me what to do. Because that feels like the "president" has decided that I am not fit to command and would be better suited to sit on the sidelines and follow orders. That's where my frustrations lie.

And probably will for a while.

Ok, drunk guy going to bed to get bossed around tomorrow by some dude he could probably be. Whatever.

User Journal

Journal Journal: You Can Chat in Hell

Back in late November of 2015, I was tasked with coming up with an instant messenger solution that our partners could use to communicate with our operators. That sounds pretty simple. Everyone uses IM and has for over 20 years. And Slack and HipChat are massively popular.

Here's where they start throwing wrenches into my plan. Any third party server is not allowed. Law does not want our conversations stored on third party servers, and security doesn't either. So that scratched off my obvious answer. We use Sametime as a company, and there is an external option for that, but security also denied that stating that then an external user would have visibility to the entire company and could potentially IM, say, our CEO. Ok, so there goes that too.

You know where I'm going next with this, right? Roll our own XMPP server, of course!

Ahh, XMPP. I have had a fondness for it since Google Talk came out. An open protocol that seems to work pretty well and has a lot of users. Ok, so we have a starting point. Now what.

Well, the next step was to figure out how easy it would be to host a server. We basically have 3 options. There's ejabberd, an Erlang-based server that's been around the longest. I like it, but we don't have a lot of Erlang developers (which is sad, because I also like Erlang. So it goes). There's Prosody, which is kind of a KISS model written in Lua. I liked it too, but it wasn't very Enterprisey. Then there's Openfire. Java-based? Check. Plugin architecture? Check. Mature? Very!

Openfire looked very intriguing when I started looking at it. Of course, being enterprisey, it's a lot heavier than something like Prosody. Which means it has a steeper learning curve. But you take the good with the bad.

So I pitched my idea to Legal. They said we have to archive messages for a certain time period, which I can do with this server. It can archive to Oracle. Perfect. Then I talked with security. My initial solution was to use Strophe.js to connect directly to Openfire. They put the kabosh on that when they saw that users would be unauthenticated in our DMZ. I walked away from that meeting a little perplexed. After educating myself on our architecture a little more, I dug around for a SiteMinder module. There was none. But, there is a webchat interface called Fastpath. It allows you to run helpdesk operations. That's my ticket. Because then they would be operating within our standing operating procedures, just like any other app.

This thing is ugly, and hasn't been given a good enhancement in about 10 years. So now I take that challenge on. More to come on that.

Once I overcame the architectural hurdles, then I had to start working with our infrastructure teams. This required an ungodly amount of requests. Oracle, server planning, application IDs, Linux guys, Apache rules, change management... I think I worked with just about every team in the company to get this done.

I've made a lot of progress though over the last month and I'm ready to go to production next week. I AM STOKED. I've been saving a cigar for just such an occasion. But next comes the hard part. I will need to start developing on this decade-old code. I have a user-request to add group chat, which is not at all what this thing is meant for. But even before that, I need to fix security issues and add SiteMinder integration. I don't think any of this is going to be a walk in the park. On the other hand though, it does sound fun! And I'm giving back to open source, which I love doing.

I'm omitting a lot of the trials and tribulations I went through when setting this all up, but I'll probably just put that in some internal documentation.

I'm just ready for a cigar.

User Journal

Journal Journal: The Streaming Data of My Consciousness

First of all, why is the "Write in Journal" button at the goddamn nether reaches of Slashdot? Are you trying to get me to not write my thoughts here? Too fucking bad. I've been doing it for over a decade, so deal with it.

This post will be largely drunken and angry and in a stream of consciousness style because I am drinking, angry, and don't care. I mean, most of my posts are stream-o-consciousness posts, but I am not always drunk. I'm actually not drunk... yet. I'm working on it. Drinking a LeatherheadRed which is quite delicious, and I took a shot of Evan Williams right when I got home because, well, that's where I'm at.

The Lady asked for more money to buy school books today. I don't think it would bother me as much if she just said, "heyyyy, I might need $500 for books. Can you swing that?" Rather than, "So I might need money for books." Me: "How much." Her: "I just hate how they do this. I hate not contributing. blah blah blah." None of that helps. In fact, it just makes me angrier. Just tell me what you need so I can figure it out. Right now, I'm still living on 401(k) loan money. The stupidest loan I've taken out, but what choice did I have. It is keeping me off of credit cards, but is in effect, decreasing my retirement significantly, and tying me to the company that I would like to have the option to leave.

This goddamn company, man. Stock is dropping. They're predicting another shitty year in 2016. For FUCKS SAKE. THIS IS NOT THE TIME. Everything else seems like it's falling apart. And what bothers me probably most right now is that I'm having trouble on my last homework in Probability Models. We're doing Renewal Theory, and shit got hard. All out of nowhere, everything was like, "Hey! We heard you've been doing well! We're here to fuck that up for you!" I have to turn it in tomorrow. I could turn it in and take a hit on it, which would keep me at a low A, high B, but for one thing, I want the A. And another, at least one of the problems on this homework is on our final, which is worth a shitton, so I HAVE to understand it regardless. So that, on top of realizing I spent over $600 on chameleons that we had to take to the Humane Society because the cats kept fucking with them.

Just came back from a break. I checked Facebook. That was a mistake. Read some bullshit about Nassim Taleb hating on GMOs. Goddamnit, nobody is good. Everyone has a part of them I will hate. Maybe that's just life. I'm sure there's things you'd hate about me (least of all my writing). I guess that just gives me more of a stance to take while reading Antifragile. The book overall is quite good and inspiring, but some of it must be taken with a grain of salt, or from a certain context. Like when he says that nothing good comes from universities, but rather from tinkering and hobbyists. Well, that's true for the big things (think the automobile, Microsoft, Facebook, etc), it's not true in general. But that's not what Taleb deals in. He deals in Black Swans - unforeseen groundbreaking events - good and bad, in which case, scientists being wrong about GMOs (lots and lots of scientists... lots) would be a Black Swan. Still, he puts the burden of proof on the scientists to prove that he is wrong. And that is not good skepticism or science. If you're bringing the skepticism, you better have science on your side. Unfortunately for Taleb, it is not. Can't get behind him on that one. Also, he resorts to name calling. That's pretty sad for a well-respected best selling author and academic. :/

Well anyway. My sister is badgering me to do her statistics homework. I partially want to to see if I can do it without remembering much from Prob & Stats I, but I also know I have a lot of shit to do and she's not learning anything if I do it for her. So I told her I'd give her an hour for questions. I have my own final to do! I'm genuinely stressed about this now. I want to do well. I want an A. For the first time in my academic career, an A matters to me. Demonstrable understanding of a topic matters to me. I want to be able to say, Yes, I took Probability Models. I can show you how to do Renewal Theory.

Alright, I'm gonna watch some 'flix and hit the sack. Peace. Hope you enjoyed the stream.

User Journal

Journal Journal: As the Path is Revealed, the Map Gets Bigger

I've been on the Data Science road for about 5 months. I initially became intrigued by the idea of Data Science on January 5th, 2015. This came about when I inquired about starting my master's degree in mathematics and was informed about a concentration in Data Science. "What is Data Science?" I wondered. So I started looking into it.

Here's my initial thought progression over time:

"I have no idea what that is"

"Sure looks trendy!"

"I bet this is one of those things everybody and their dog will want to do but without having the math chops to really be good... just like developers."

"Ooohhhhhh, look at all these tools! Hey! I've heard of a lot of these! Hadoop! CouchDB! MongoDB! Ummm, Spark? Dremmel? Spanner? Voldemort?! Where does this list end?!?!?! This must be Data Science!"

"Oh, so Data Science is an umbrella term. Underneath that is Predictive Analytics, Machine Learning, Data Engineering, Data Architecture, Computer Vision, Natural Language Processing [list goes on]. Ok, so we're back to Computer Science."

Me now: "Why don't they just call it Data-centric Computer Science?" "Because it's not catchy enough and it wouldn't pay as well." Oh yeah.

So we've come full circle. It's always been Computer Science. Some of us just took more math classes. All of those tools I mentioned in my post Dear Gournal have as much to do with data science as Matlab has to do with Mathematics. You wouldn't say Matlab is Mathematics. You would say Matlab is a Mathematical tool. In the same way, all of those technologies are Data Science tools, but they are not Data Science. I'm glad I'm realizing that now.

I'm almost through with my Probability Models class. I am somehow riding on a low A, and hope to finish strong, but the latest lessons on Poisson Processes and Renewal Theory are clouding my head. Still, it's been a very good class. Now I know what actuaries do! And I know I do not want to be one. Still, probability will never not be useful. It is at the core of what I want to do. AI, for example, is heavily based on prob. DS which is heavily stats based, is inherently prob based as well. Predictive Analytics, for example, would be impossible without probability theory. I hope to bring some of that to the table. Next semester, I'm taking Prob & Stat II. I think this class prepared me pretty well for it.

But while I now realize what Data Science is more or less, it only makes me realize how much I don't know. It's not as simple as learning a few tools and technologies. It's about learning the fundamentals of statistical analysis and probability, and then the things that build on that, like machine learning and predictive analytics. I'm excited and scared at the same time. It's terrifying if you try to eat the elephant in one bite. So I'm trying to take it a byte at a time, starting with the toe.

In my spare time, I'm working on a sports database, and a statistical analysis of a fantasy prognosticator. I've finally got the database together, and now I'm working toward the guru analysis. It's taken a little over a month, and I'm guessing will take another month to finish up, but I'm proud of it nonetheless. It's stupid but has been a fun exercise in Data Science.

Not much has changed at work except that I now have a project, finally. I am tasked with bringing Instant Messaging to the company. I mean, we already have IM, but it's internal only. This has to be internal AND external. So I'm rolling with XMPP. At least I hope. I have a meeting with our security team tomorrow to discuss the feasibility (security-wise, not technical). I think it's going to be a fun project. I'm planning on rolling with OpenFire. Should be pretty straightforward.

Anyway, that's it for now. Gonna go read more Anti-Fragile.

User Journal

Journal Journal: Year of the Black Swan

I'm reading Anti-Fragile, a book by Nassim Taleb, who also wrote a book called the Black Swan. I haven't read that one, and I'm thinking I should now, although he talks a lot about Black Swans in Anti-Fragile. A Black Swan is a disaster of levels previously unseen. Things like the 2008 financial meltdown, the Fukushima nuclear disaster, or Hurricane Katrina. In most Black Swan cases, people are only prepared for what they have seen, or what lies along the mean of the normal distribution. They don't plan on outliers, even though outliers are what destroy lives.

I thought 2013 was a bad year, because I couldn't do a lot of the things I wanted to do because I had six weddings to be in, which essentially drained my vacation time and my bank account. I am a major opponent of weddings because of that year. Fuck your wedding. But more on topic, I didn't think 2014 could be that bad, but knew that if it was, I could endure it. But I made the mistake in the assumption that 2013 was the worst it could get. 2014 proved to be even worse due in large part to my girlfriend moving in and bringing her horrible attitude at the time with her (that was due almost entirely to her job situation). It was a year fraught with sickness, strife over a troublesome dog that she did not want to let go of, an apartment complex under construction for 7 months, complete with an unbearable drilling and jackhammering into the walls that made sleeping impossible for her, a few near-breakups, and more sickness.

And then when things started to clear up - she got into anesthesia school, we got rid of the dog, they finally finished construction - another Black Swan. She was diagnosed with Meniere's Disease. Something that could be devastating for not only her career but her life. We've been living with it, and managing it, and she got into school. But as the Black Swan principle dictates, the worst is yet to come.

Now she is failing school. If she can't find a way to pull her grades up, the $100,000+ gamble will have gone to the house and I possibility of getting out of debt in a swift few years on good anesthesia pay will have been but a fantasy, and I'll be paying off debt well into my 40s.

This is a lesson I will take with me for the rest of my life. The Black Swan is very real. And the fragility that I have subjected myself to, has made everything so much worse. I took out a $25k Upstart loan last year. I'm already through that money. So I took out a $20k 401(k) loan. That made things even worse. I can't quit my job (or lose it!) because I will owe that money immediately back. On top of it, I've stopped contributing to my retirement fund. Again, fragile.

I'm in such a bad position, and it is mostly because of this high stakes gamble. If she can't pull through, then we are more or less fucked. I still don't know what to do. I don't really have a choice anyway. I am trapped in this fragile nightmare, at the whims of the Black Swan. Yesterday, she got a nail in her tire, which I took into the repair shop, and of course, they didn't have her size. So, to avoid her having to ride with her classmates (she can't stand being around them), I rented her a car. I don't love spending that money, which essentially doubled the cost of the tire, but more so, I didn't opt for the insurance or adding a driver. My Black Swan mind scolded me the entire drive home. A wrecked car, with her driving would be the biggest disaster yet, which is the definition of a Black Swan. Yet I didn't stop to correct it before driving away. And this is how the Black Swan thrives. It preys on those who are not robust or anti-fragile. And so, until I return it hopefully later today, I have to hope that that gamble does not result in a disaster.

And then I have to worry about the bigger impending Black Swan. The fact that she has to get a 91% on her final in order to pass her anesthesia class. We are probably just fucked.

User Journal

Journal Journal: Sick of Being With the Sick

This won't be a very tech-y post. I just need to vent. I thought, maybe, JUST MAYBE, the end of 2014 would be the end of all of our problems. But then 2015 comes along and it turns out, it is trying to one-up 2014. I'm talking about illness. My girlfriend has somehow, since 2014, went from being completely healthy, to being one of the most fragile people I have ever known.

Let's break down the numbers.

2014-01-02: Gastroenteritis - ER visit
2014-03-02: Tamarind allergy discovered - ER visit
2014-05-11: Sick (cold or something)
2014-06-01: Sick (cold or something)
2014-08-21: Tamarind allergy from Mexican food - ER visit
2014-08-22: Bad reaction to steroids from Tamarind allergy - ER visit
2014-08-24: Still having reactions from the steroids, mistook it for thyroid problems - ER visit
2014-11-20: Began having Meniere's episodes, though didn't know what it was
2014-11-21: Began having Meniere's episodes, though didn't know what it was
2014-11-22: Began having Meniere's episodes, though didn't know what it was
2014-11-23: Began having Meniere's episodes, though didn't know what it was
2014-11-24: Began having Meniere's episodes, though didn't know what it was
2014-11-25: Began having Meniere's episodes, though didn't know what it was
2014-11-26: Began having Meniere's episodes, though didn't know what it was
2014-11-27: Began having Meniere's episodes, though didn't know what it was
2014-11-28: Began having Meniere's episodes, though didn't know what it was
2014-11-29: Began having Meniere's episodes, though didn't know what it was
2015-02-12: Meniere's episode
2015-04-11: Meniere's episode
2015-05-08: TMJ
2015-05-15: Swollen lymph nodes
2015-05-16: Swollen lymph nodes
2015-05-17: Swollen lymph nodes
2015-05-18: Swollen lymph nodes - ER visit
2015-06-01: Cold
2015-06-11: Meniere's episode
2015-06-25: Meniere's episode
2015-07-05: Meniere's episode
2015-07-09: Meniere's episode
2015-07-18: Meniere's episode and stomach ache
2015-07-19: Stomach ache
2015-07-26: Meniere's episode
2015-07-31: Meniere's episode
2015-08-02: Meniere's episode
2015-08-03: Meniere's episode
2015-08-09: Meniere's episode
2015-08-12: Meniere's episode
2015-08-14: Meniere's episode
2015-08-18: Meniere's episode and cold
2015-09-11: Meniere's episode
2015-09-12: Meniere's episode
2015-09-13: Meniere's episode and stomach ache
2015-09-14: Stomach ache - ER visit
2015-09-20: Meniere's episode
2015-10-06: Meniere's episode
2015-10-11: Stomach ache
2015-10-12: Stomach ache
2015-10-13: Stomach ache
2015-10-14: Stomach ache
2015-10-15: Stomach ache

Those are just the days I documented. There are countless others where she just didn't feel good because of Meniere's or whatever, and times she didn't even tell me she wasn't feeling good. But from just those days alone, since January 1st, 2014, she has been ill 7.5% of the time (see, at least I'm including a little data science in here). That is a horrible batting average. That's averaging out to be almost a month of illness per year. I'd be stoked if I got that much vacation time! And it is terrible quality of life. And just when she was adjusting to the Meniere's and starting to figure out how to somewhat control it, she gets hit with this GI thing. Of course the doctor's don't want to even try to figure out what it might be. They just mumble something about IBS and walk out. Fuck our health care system. I think we should start calling it a "health don't-care" system. They only things they seem to be concerned with is passing out antibiotics, operating on people for things that could have been prevented in the first place, and fixing erectile dysfunction, which, God knows, is the world's biggest problem. For fuck's sake.

The stress this brings on me is causing me health problems too. My hair has thinned out a lot in the last 2 years (though, some of that can be blamed on my line of work), and a gigantic cold sore just reared its ugly head. And lord only knows what I'm doing to my liver to cope with the stress. I don't know what I need. I briefly fantasized about buying a one-way plane ticket to the Virgin Islands and just disappearing. Why there? I dunno, first place I thought of that was far away and didn't need a Visa. Maybe I could be homeless for a while. Start with some manual labor, working my way up to owning my own business, and bang lots of tourists. And then the cold reality of my life slaps that fantasy out of my head. Plus, it would just make her life harder, and I would never want that. She deserves so much better than what she is getting.

I don't know what to do at this point. I'm running out of positive affirmations. For myself included.

Sorry for the personal garbage. I thought it might feel good to just get out in words, but I'm not sure if it is helping or just confirming what I already thought. Anyway. Off to a stupid meeting where more people will probably just piss me off.

User Journal

Journal Journal: I Hope the World Burns

There's a scene from Nickelodeon's Salute Your Shorts I remember where Donkeylips and Spud were assigned to trash pickup duty and Donkeylips wasn't going to make weight to go to a wrestling tournament or something, and it all boiled down to him probably not getting to go eat some delicious lobster. I don't know why I remember this so vividly, but I guess it has always stuck in the back of my mind. In that scene, the trash bag rips open and all the trash they had collected starts blowing all over and Donkeylips says, "Garbage! My life is garbage!"

I feel like that lately.

This entire year has been nothing but rejections and watching my career sink. It started with not getting the architect job. I have since considered this blessing in disguise (not the paycheck part), as the action of that rejection caused an equal but opposite reaction in the opposite direction of Data Science. Now, while it is cool that I was spurred in that direction, there's literally nothing here helping me get there.

Earlier this year I felt like I needed a breadwinner of an idea. Something to cement my bonus. That idea was the automation of one of our accessorial billing line items, which I estimated could have brought in around $600k per year. I caught the ear of the guy who could have set that project in motion and then... nothing. I don't know what happened. If there was fear or trepidation in pulling a (slightly) bold move. But for whatever reason, the project never got initiated.

With that out the window, I was given the task of integrating systems with our partners over at R (names changed to protect the guilty). Since they suck at thier jobs and life, I knew this probably wasn't going to actually happen anytime soon, but I needed something to make the project pop. So I thought I would build out our entire portal as a B2B Hypermedia REST API. That is pretty much done and it is just sitting there waiting to move forward, and what's worse, I don't think anyone even cares. No one. I haven't advertised it yet, because nobody even seems to care that I'm literally doing nothing.

Today, I got my mind really hooked on this problem of predictive analysis. Since I'm in a Probability Models class, I successfully modeled one of our processes as a Poisson Process. I know that Asshat is already working ("working") on this with our Data Science team, but they're going to take supposedly months to get some proprietary 3rd party system up to do forecasting, and I know that I can have a crude but fairly effective model up in a week or less. So I took it to the big boss. But my pitch must have been off, because he just kinda said, "Cool!" and dismissed it like it was a neat little chart. FUCK. What do I have to do around here to get some important, thoughtful work?! Who's dick do I have to suck?!

So here we are, on the brink of the 4th quarter and I have nothing to show for this year. Unbelievable.

I literally don't even need to come in. I could have stayed home for 90% of this year. That's how far I've advanced my career in 2015.

My influence here is fading too. Our new architect doesn't clue me in on anything he's doing (but has made good friends with the other developers), my boss was let go, so now power-hungry Asshat has all the cool projects, my new boss doesn't really have a nose for strategic development, since his specialty is in improving and maintaining legacy systems, nor does he know how to do battle with Asshat. And as a symptom of all this, I'm starting to care less. Fuck it. If nobody is going to give me work or let me run with projects I think would be cool, I'll just do nothing. I'll do homework. I'll leave at 2:30. I'll learn things I want to learn, like R and Python.

I'm so frustrated. And I can't leave, because of my debt situation. Fuck this. I'm going for a run. Then I'll come back, but all I'm going to do is read my 100 pages of Markov Chains of class.

Slashdot Top Deals

How come everyone's going so slow if it's called rush hour?