Managing Site Growth? 37
markmcb asks: "I started a web site about two years ago. When it began it was simple. The code was 75% hacked, and administration was easy. However, the times they are a changin'. Now I get hundreds of thousands of hits and have a steady flow of new users. I'm noticing an ever-increasing gap in terms of my site's popularity and its technological progression. Specifically, I have all sorts of 'XYZ for Beginners' books that are no longer of use to me. Even the so-called non-beginner publications seem to only scratch the surface of running a site. As problems get more complex, trying to Google every situation/issue I have with site administration has become less useful as well. I'm finding things like writing optimal code, configuring servers for high-volume, balancing ad income vs. server costs, and maximizing the efficiency from my moderation team have all become issues and that aren't addressed most books. What is the best way for a low-income, non-professional, but enthusiastic web designer/administrator like myself to manage site growth as it leaves the realm of just-for-fun?"
Hire a professional (or become one) (Score:5, Insightful)
"Scalable" and "customized" are two things that when put together simply require a professional. And quite a lot of people calling themselves "professional" can't handle it, either.
Now, by "professional" I don't necessarily mean a degreed guy who makes at least $X thousand a year with Y years of experience. What I mean is, you're stepping into the domain where you can't hardly acquire the experience and skills necessary with anything less than full dedication usually brought on by having a job in the relevant domain.
There is, however, one other possibility for you to consider. If you analyze your needs and the available packages for your type of website, you may find that you can drop the "customized" aspect of it, if you can find a project close enough to your needs to require only minimal customization, perhaps even no actual code customization. Then you just need to import the data, and you will presumably have satisfied yourself that this package can meet your performance needs.
If the website you are referring to is the "OmniNerd" site you have a link to, then I would imagine this should be feasible. There are a lot of "news" packages, free and otherwise, and at least on first blush I don't see anything particularly unique about it. It looks an awful lot like slash, although from what I've heard that's not the easiest thing to customize. (slash hackers feel free to comment.)
Really, there's no excuse nowadays to start a new web framework from scratch, and your first impulse if your hack-job is starting to come apart at the seams should be to change to one of the umpteen bajillion tested, performant frameworks. Depending on your skill levels, which you did anything but talk up, you may even be missing basic pieces like caching, which is pretty important on a site like that. Non-professionals should not attempt to write website caching routines. Unless you want to go insane. (It's not that it's hard to write... it's that it's hard to get correct, and debugging cache problems are close to sheer hell.)
Depends upon the platform (Score:4, Insightful)
Open source technology tends to be more scalable and solid, but even there, a bad choice stifles your progress. If money is no object, I guess you can always scale up, but the commercial platforms often have their admins spending more time patching and maintaining the status quo than progressing. The bigger question is: Did you do your homework when you initially designed the system? If you're stuck, that's likely the problem.
If you have a choice to redesign or redeploy your site, what you need to do is ask yourself, not whether or not the technology you're familiar with can do what you want, but instead, are you using the right technology to do what you want?
This is a great example of what acadmia teaches yo (Score:2, Insightful)
However, even from day one, I could see they were trying to teach us something else, and not just how to program. This wasnt clear or evident to anyone else who didnt know how to program in a compiled language, because the workload of simply understanding how to program really did washout the assignment goals. Our assignments - based on the BlueJ work (http://www.bluej.org) - taught us how to compartmentalize classes, and structures. They made us think of more inventive ways to decouple classes. Taught use to introduce and extend classes using aditional abstract datatypes rather than rewrite the base class.
Applying what I learned to Perl has not helped comprehensively, but it has increased my awareness of types and the structure of larger projects. Even a simple RPG can have a few hundred ADTs/structures.
What I feel is important for me to share, however, is that there is no final solution to programing and stepping into the complex. Sometimes it's easier and quicker to isolate peices of code for what they are - functionaly complete - and not get bogged down trying to improve something that doesnt really need it. Focus on the way things are coupled - whether you use a single big import/include/use statement, or dynamically link things. It pays off in the long run to have a better execution structure (Perl's POE anyone?!) than programmatic correctness in function-orientated areas.
Practice. Practice. Practice. Don't worry about things too much. Starting again, even twice, is not a failure. *You* learn the better ways with time. Don't be afaid to write down absolute crap on paper - even if its block diagrams, code fragments or unrolled loops. You need total awareness of your task (aka design spec) until you see the 'best way' and produce tight code.
(nowadays, lmost everything I begin requires a little thought, but the moment I start writing code, I'm doing it correctly, making use of language features and code conventions. experience... you'll get there in time!)
Matt
Tips from the trenches (Score:3, Insightful)