Do a CRC ...AND... an MD5, and use a database. I built an internal file management system for one company that was (at the time) handling only about 1.75 TB of files. I was unpleasantly surprised at the number of duplicate CRCs I got on non-duplicate files. Using two different types of signatures, plus the file sizes alleviated that problem. Once you have it in a database it is also a bit easier to decide which one you want to keep and/or where to keep it by using some query/column value tricks. Flagging the values as duplicates is also trivial. My product was server based with a web services interface to access the files, so I also threw in a compressing/encrypting feature with access by their original path hitting a keyed server file in the back end. It was amazing to me how much faster server file access was when I did endpoint (both server and client) zipping before transfer. I also used the CRC, MD5, and length to avoid unnecessary downloads to begin with.

Another option (not available for Windows, as far as I've seen) is to use a file system that simply dedups each sector, which buys A LOT more disk space than a simple file-level dedup.

You've got to remember that using FOSS software doesn't mean that you aren't going to have an expense for this. One of the downsides of FOSS is that it is generally software that "scratches the itch" of those willing to develop code for it. It doesn't mean that the software is lower quality, just that they may not have covered everything you need. Also, it may not be the easiest thing to install. If you aren't a Linux geek, or you now don't have time to be since you are running a restaurant, make sure you have some competent local support lined up. Proper install, setup, and security is important and can't just be swept under the rug.

Also, another somewhat obvious suggestion is to make sure you can line up an accountant that is familiar (or willing to become so) with the software you choose for the books. If you find one that actually uses some FOSS, they would have better advice on what packages to use, since they are more familiar with the accounting/regulations side of things.

Be aware that regional corporate and finance laws may be different than those of the software developers'. Commercial software has a general business requirement to keep up with those and supply the necessary patches. In absence of the commercial incentive to "not get sued over missing a patch" you will need to make sure that you have that covered. A few dollars of support to a local programmer (in conjunction with the aforementioned accountant to keep things moving in the right direction) will keep you out of the legal ditches as well as ACTUALLY support FOSS software.

In general, there is a price to pay for freedom. There always has been. If you want software that isn't locked up by greedy or laconic software corporations, you can't be greedy either. You still need to pay for the expertise to keep things on track and actually support the free environment that you wish to take advantage of. Costs are still there. They just shift. If you go in with open eyes, it won't shock you. It's still worth the investment. It just takes a slightly thicker skin to (hopefully) get a slightly cheaper and more customized outcome.

I can see it now. 500 petabytes stored on a postage stamp housed in a device the size of an overstuffed, large suitcase. It has geek written all over it! I must have one.!!!

There are those of us who learn and visualize what is around us from a purely conceptual viewpoint. There are others who view things from the point of view of the formulas behind them. Both approaches are correct, just different. One NEEDS to understand "why" while the other NEEDS to understand "how". It is simply how we are wired. It's the dichotomy of the two that have allowed us to reach the heights that we have. This is not an argument that should be ventured, "who is right?" The best option is to understand our own propensities and explore our individual strengths for learning and growth while also reaching across the chasm to those of the different stripe to build that whole picture. It is that collaboration that, more quickly, takes us from theoretical to applied. We can avoid the "religious" disputes of our predecessors by understanding this. A Physicist uses math to find an explanation for what he already sees. A Mathematician discovers physical truths through his math.

Economies of scale is a proven strategy. Virgin Galactic is already selling at $200K a pop. Their idea is to sell at that price for as long as they can in order to drive up demand and drive down costs, thus price. The more we go, the cheaper we go. Though it may never be "cheap" it will definitely get cheaper. VG has already proven that there is a significant market at $200K for just a limited, very short (couple of hours, I think) LEO trip. The approach is genius.

There is still an enormous market for commercial LEO, GSO, and Lagrange hardware. Leveraging private desire for space drives down R&D costs and general launch costs for those commercial ventures. Eventually, the same strategy WILL BE USED for an orbital hotel as well as a lunar hotel to drive costs down for a commercial moon mining base, a necessary ingredient for manned exploration deeper into our own solar system. Just look at the number of people who recently volunteered for a one-way Mars mission (when it was only a suggestion from someone at NASA). Most of those would be willing to pay for part of the ride just to get a chance to attempt to establish a colony there.

Our history is rich with the pioneering spirit (human kind, not just America). Look at the large number of documented failures where all members were lost trying to establish a "European foothold" on North America. This isn't just about a thrill ride. OK, so maybe it is for a lot of people. It's really about humanity reaching out and expanding who we are. It's built into our DNA. When all is said and done, humanity WILL BE a multi-planet species. It may be delayed by random circumstances (recessions, wars, stupid politics, etc...) but the drive will never really be contained. Eventually we will find ourselves spread across the solar system. Eventually the galaxy.

I applaud the guys doing what it takes to make space accessible to everyone (VG, SpaceX, and others). Once the populace is used to being able to reach space, though costly, politicians and circumstances will never be able to take that away from us.

Personally, I'm trying to build enough of a financial base to fund (or help fund) a commercial moon base. Programming isn't cutting it so I've started writing. Look me up on G+. Maybe we can hit this mark together a bit quicker.

Dude, I out rank you by several years. You mind is flexible. It can be retrained to function in ways you have never dreamed before... at any age. It takes a willingness to attack the inflexibility with novel approaches. Listen to different types of music. Work on making music. If you aren't a math wiz dive into Khan Academy and stretch an analytic approach to thinking (go from basic to advanced math as fast as you can.) Run your life different. Start writing books for fun. Learn a new language. (Chinese is a great brain jump starter!)

The point is, you have to choose the level of flexibility for your brain. Remapping patterns (thinking behaviors and assumptions), on a regular basis, teaches your neural network flexibility. It changes the wiring of the neural pathways from "think as a coder" to "think as a learner/creator". Even within the narrow confines of "being a programmer", moving from OOP/imperative programming to functional programming, requires a great deal of brain flexibility. Once you start opening up your ability to learn in new ways, however, you will find that you are no longer limited to a programming (or development manager) career path.

You may be 40 but you are probably not even half over with your life. Keep you brain (and body) active and changing. It will insure an interesting LIFE instead of a limited CAREER.

Gook luck grasshopper.

Those who do keep the binaries, partly, because the tool sets change. A patch to the compiler can change the resulting binary. If you have a specific binary in the field that was compiled with one version, you may not be able to duplicate a bug using a binary from a newly patched compiler.

Granted, I don't recommend using a VCS to store all binaries along with the source. That is just asking for trouble. I do, however, store a single "release package" of binaries and other data that relate to a specific source code branch for each officially released version.

This historical reference helps when traditional debugging doesn't work. Sometimes the bug was in the compiler. If you can't isolate the problem to the binary vs. the source code, it can be a nightmare.

It would probably be more efficient to use a traditional file store but using a VCS, if done simply, can be very convenient.

Finish your math degree. Go as far as you can go on that. (Doctorate, even...) If you can do the math you can outpace most programmers at their own game. Most programmers, as it is, were not trained in college to program. I was trained in Chemistry.

Basic programming may not look like math but almost anything worthwhile being programmed uses math extensively (even when the programmer doesn't realize it!). Understanding the best way to accomplish something mathematically will make shorter work of programming it. Math may not be "the end all" but it's definitely an essential skill.

Programming languages and concepts are much easier to learn than the math so finish there first. Then get a "math job" that also needs some programming on the side. You'll be able to do the switch easy enough.

It's actually pretty fun. I already transcribed one piece. It's running a bit slow though. When they said "crowd sourcing", I don't think they meant "slashdot crowd sourcing".

Honestly, you may never be able to convince a budget pusher that testing will save money. What they see is a financial constraint that they may not be able to get around. As a coder, however, you can make YOURSELF more valuable by learning test methodology as a way of life. It has changed the way I think and code. My code is cleaner. I now deliver finished and stable product much faster.

First, let's look at the types of testing you may encounter. The most important, and easiest to introduce, is unit testing. Unit testing effects your approach to building clean, reusable modules. The second is regression testing, which is basically just the practice of keeping all your unit tests and running them ALL every time you add a new module or change an existing one. This keeps your bug fixes or functionality from "regressing". The next, if needed, is stress testing. This is important if you have an app that deals with a lot of data or a lot of user traffic. It helps you determine is there might be an issue with the chosen platform, database engine, even your network "pipes" clogging traffic. The last, and hardest to implement, is UI testing. That route usually requires a set of libraries that compile into your app. There aren't a lot of good ones out there, but the are some new stars rising in the field.

Let's get to the most important one now, unit testing. Unit testing, for a lot of languages can be implemented free and incrementally, which is how I started. If you are running C#, look into NUnit or even Microsoft's testing suite, built into pro versions of VS 2005 or greater. If C++, look at CppUnit. For Java, there is JUnit. If you are using SmallTalk (not very likely) there is the granddaddy of unit testing, SUnit, from which the rest derived. Even Flex and Flash from Adobe has FlexUnit. These are all free (or included in the development tools.) There are other projects out there for other languages but you'll have to research that yourself.

If you have never done unit testing get a book or three on Test Driven Development. It's a pretty large bite to chew, but give it time. This is more about improving your career than fixing a single problem. The basic idea is that, when you start solving a programming problem, break it down into small chunks and write a set of tests that will prove the code for each small chunk. Keep your classes small and focused. This will improve not only testability, but reusability, and inheritability (if those are even words). Don't tie UI code directly to the functionality behind it. This improves testability, but it also makes it easier to change out UI components when a wrong UI decision is corrected. For instance, if you program in a progress bar but the customer wanted a fuel gauge look, the functional code can service the different looks without changing.

Even with unit tested code you will still find bugs. The trick here is to make sure that 1) the classes are small and focused (mentioned earlier), 2) the class (and relevant unit tests) are well documented so that there is no guessing as to why you did what you did, and 3) the unit tests are well organized. When you run into a bug, sometimes just because of bad data in the real world, build a new test that duplicates the condition and fails (throws an error or somesuch), then modify the class(es) to handle the condition. Run that test again to make sure it worked. Then, lastly, run the full suite of tests that you built to see if you brake anything else with your change. This last test (regression testing) is the magic that makes unit testing shine.

I know that this post doesn't directly answer your question, but sometimes, a little understanding goes a long way to helping you ask the right question instead of guessing. Currently, I write about twice as much code as I used to. Half of it is unit tests (maybe more). The quality of my code, however, has greatly improved. I am also able to deliver cleaner code faster. Just like project management, where, if done right, you plan as much as you can up front and the project takes half the time, programming with testing as a part of the design will eventually) speed up your code delivery. It seems counter-intuitive until you have experienced it for yourself. Trust me, I put it off for twenty years before I "converted".

For those not too young to remember, Bill Clinton's reelection was guaranteed by the Democrats using the same tactics to get Bob Dole on the Republican ticket. This tactic is distasteful no matter who tries it. What would be intensely hilarious to me would be for Palin to win in 2012 after such meddling. Here's a warning to those Democrats who would stoop to such tactics: She has more support than you think. A true grass roots push coupled with a group of idiots voting outside their party would be enough to tip the pot. How 'bout instead of being dishonest, people, try to find a better candidate than Obama. If he can't win without subterfuge, he doesn't belong on the ticket. Though legal, this is no more morally sound than the Watergate breakins during the Nixon administration.

Not been keeping up with the current healthcare insurance bill, have we?

The Challenger disaster was a personal tragedy for many of us. The shuttles represented the resurgence of hope that we were experiencing after 2 decades of societal insanity (60s and 70s). If you're too young to remember or not from the US, the whole nation mourned for quite a while. Having a video that you took of the incident would be akin to keeping a memento of a loved one that you just lost, a personal reminder of what was lost.

You forgot. We had a "season change" back in 2004 when those 3 hurricanes blew through. Their paths crossed over each other to make a nice asterisk right on top of where I live.

I've had very similar experiences with pianos. I'm by no means a master. I don't even consider myself proficient, but the opportunities that I have had to play a really excellent piano made things totally different. From the weight of the keys and their responsiveness to my touch to the acoustic quality and shape of the box surrounding the harp, everything sounded and felt different. I've only been "lost" playing piano twice in my life. Once was on a high end traditional full grand (can't remember the make). The other time was, to my surprise, on an simple Yamaha full keyboard (a high-end electric one with only 2 "voices"). It felt and sounded as good as the grand to me. I was shocked that an electronic device could really produce that sound and have such a velvety feel. It was a dream to play.

If instruments with that level of quality were accessible to everyone I believe that the amount of truly inspiring musical compositions would begin to soar. My piano play began to wane after those experiences simply because everything else left me flat. When you truly enjoy the experience of playing a quality instrument and the instrument itself is not fighting against you in the production of a really beautiful sound your creativity level goes way up.

With the studies out that have definitively proven the link between playing music (including singing) and the increase in abilities in language, math, and science aptitude I begin to wonder if more common access to high quality instruments would help to improve the math, science, and language problems we now have in the education system. I would certainly help reduce the amount of pop-noise that pervades our society today.