Cross-Platform Company Storage Architecture? 60
Eric^2 asks: "My company is preparing to implement a major network storage upgrade, and I'd like to get some ideas from Slashdot about what devices should be considered, and hopefully some experiences with some of the offerings that are available. What types of storage are you using and what would you recommend?"
"We are currently using approximately 2TB of storage space, and will need to expand to over 10TB in the next two to three years. We have a mix of Windows, Mac OS X, and Linux clients and servers. All of our authentication is presently done through an Active Directory. If possible, we would like to centralize all of the storage into a single namespace, such as OpenAFS or DFS. Anything we purchase will have to be under maintenance contract for hardware such as failed drives or controllers. Ideally, whatever system we choose would allow us to purchase both high-speed SCSI spindles for our transactional needs and lower-speed SATA high capacity drives for our archival storage needs."
NetApp (Score:4, Informative)
-Randy
Re:NetApp (Score:2, Informative)
If it's cheap, then Netapp might not qualify... :)
What about technologies - NAS? Host-attached? Gateway/NAS? Grids?
Other companies/products to consider:
EMC [emc.com] (The Celerra is a nice product)
Onstor [onstor.com] Bobcat
HP [hp.com]
IBM [ibm.com]
Hitachi [hds.com]
Panasas [panasas.com]
Exanet [exanet.com]
Yotta Yotta [yottayotta.com]
StoreAge [storeage.com]
If you want basic raid devices look at Infortrend [infortrend.com]/Transtec [transtec.co.uk]. Their S-ATA offerings now support RAID-6 and are dirt cheap.
Re:NetApp (Score:4, Informative)
NetApp is by far my choice but if I need to get a cheaper device, Dell PowerVaults are generally adequate unless you are looking for highly transactional file activies (we've run into the file lock problem on the device occasionally if we process files, in our case image processing). To avoid the file lock though, we process locally on our UNIX boxes and transfer the results to the PowerVault. One limitation we find on the PowerVault (and likely it affects all MS Win32 based file systems) is case sensitivity. I believe you "can" change to allow differentiation of files based on case but Microsoft doesn't recommend it.
Re:NetApp (Score:2)
The Filers perform outstandingly and do everything they're touted to do (no vaporware yet!). The machines themselves, however, are nothing compared to the service f
Re:NetApp (Score:2)
Everything just works, their boxes are incredibly stable. Ours are pushing 900 days of uptime, with zero service interruptions during that time.
We've had one hard disk fail, I got a call from NetApp support while at lunch and I literally had to argue with them to get them to just drop ship the drive and not send a tech along to replace it.
NetApp is a fantastic product, and really offers surprisingly go
Re:NetApp (Score:1)
Consider Apple's XServe RAID and XSan (Score:4, Insightful)
XSan is really the deal of the century - you can build a full-blown StorNext system starting with ADIC's software, but that approach can be exceptionally expensive. Instead, start with XSan (which is a functional but slightly stripped version of StorNext) and then use ADIC's much less expensive StorNext FX client licenses for each non OS X server that needs to join.
Redundancy can be everywhere. Start with a pair of redundant XServes as metadata controllers. Add a pair of redundant SAN switches. Apple's FiberChannel HBAs are all dual channel, as are the XServe RAID cabinets. For any non-Apple hardware, buy dual channel QLogic HBAs.
Apple provides a variety of maintenance contracts for all their hardware, as does QLogic. ADIC and Apple provide support maintenance agreements for the software. The only missing piece of your equation is SCSI-based storage. But since this whole system is entirely standards-based, all you need to do is find a favorite vendor of SCSI FiberChannel cabinets and drop a few into your SAN and then partition them accordingly, right along with all the SATA storage.
It's a beautiful system, and a raging bargain compared to every other comparable solution I've investigated.
Re:Consider Apple's XServe RAID and XSan (Score:1)
"In the post production worl
Re:Consider Apple's XServe RAID and XSan (Score:1)
Mostly for the RAID Manager app, which, if you open it up and pull out the
See: http://www.alienraid.org/ [alienraid.org]
Venti (Score:4, Informative)
Venti is block level and, as such, coalesces identical blocks, a bit like LZW, so backing up 100 Windows machines doesn't take up 100x the disk space of backing up 1 windows machine.
http://cm.bell-labs.com/sys/doc/venti.html [bell-labs.com]
http://cm.bell-labs.com/magic/man2html/8/venti [bell-labs.com]
http://swtch.com/plan9port/man/man8/venti.html [swtch.com]
http://swtch.com/plan9port/man/man8/vbackup.html [swtch.com]
Sean Quinlan (one of the 2 Venti inventors) moved from Bell Labs by Google.
08:56-10:13
News for nerds, stuff that matters
Post Comment
Database maintenance is currently taking place. Some items such as comment posting and moderation are currently unavailable.
MySQL r0x0rs
Application ? (Score:2, Insightful)
Re:Application ? (Score:1)
I'm not looking for a *cheap* solution, per se, although price will be factored into the decision. What I'm looking
Re:create by yourself? (Score:4)
Re:create by yourself? (Score:2)
NetApp (Score:1, Informative)
it depends... (Score:5, Informative)
SAN - block level data access to storage. Good for databases; low client counts (because SAN ports are expensive relative to ethernet) - but with high IO demands. EMC are good, but pricey - a low to mid end Clariion would probably be the right range to aim at.
NAS - file level data access to storage. Good for situations where there are many clients connecting, and their IO demands are not excessive. Netapps filers are very good at this (if youy can find information on their new OS (10GX) then it's VERY interesting. ILM use them in their render farms.
iSCSI - a blend of the best of both, but it's still looked upon as an emerging technology. You get (or did) free iSCSI licenses with netapps filers.
O'Reilly have a good book on this. "Using SAN's and NAS" which is vendor agnostic http://www.oreilly.com/catalog/sansnas/index.html [oreilly.com]
Re:it depends... (Score:2, Informative)
Re:it depends... (Score:2)
Also, don't forget about backup. Sure, you're protected by RAID, but with the more advanced systems, you can send a single command to the storage system and make a copy of everything within the array to allow you to recover from user errors (or virus dest
Samba, Dfs, and NFS (Score:3, Interesting)
This likely won't work in all environments, however. Our data is divided into thousands of discrete and manageable chunks stored in individual directories, so stitching it together via an automated process is relatively simple. Part of the job of the scripts mentioned above is to "rebalance" these chunks (move them from server to server) to prevent any one volume from becoming full. If your data "components" are large, or if your data is too active to move regularly, this won't work.
It's the poor man's cluster, and there are better solutions out there, but it works extremely well in our case.
If I were starting over from scratch (Score:2)
2) a pair of IBM pSeries boxes that can be DLPAR'd
3) Put all storage, including boot disks, on the SAN. All servers boot off of the SAN within the pSeries hardware. All servers have a failover DLPAR ready to go on the second pSeries box.
4) Run Linux on all of the DLPAR's. The storage servers would be running OpenAFS.
OpenAFS client is well supported on *NIX & Windows... it's a mature and actively developed platform. Very secure
etherdrive (Score:3, Interesting)
http://www.coraid.com/ [coraid.com]
How we skin a cat... (Score:5, Interesting)
We protect 3 Terabytes per night from 250 remote servers with a backup strategy using RSYNC. These include both Windows and Netware servers. Our centralized backup file server is a single Dell PowerEdge 2850 with dual Xeon CPUs which runs OpenSuse 10 and has a combination of both Dell Powervault RAID SCSI enclosures and LaCie Big Disk USB External drives attached. Using a fast server with an OS that we can tune gives us incredible multistream-capable throughput for network storage. Think about the speed required folks, 3 Terabytes in 12 hours from 250 hosts at 75 sites. (Well RSYNC means we don't send all the data, but still!
Then, each day, we back up the Linux box using a Windows server installed on a Dell Optiplex workstation box with a tape jukebox attached and running CA ArcServe. That way we get a daily snapshot to tape allowing us to do a scheduled rotation.
This means we are following the Golden Rule of Backups, which applies no matter how much data you back up, which is this: Always have 2 separate backup copies of important data. And it's better if they are different types of media. And with SANS and NAS solutions redundancy is critical. These acronyms should be called AIOB which stands for 'All In One Basket'
RSYNC has done what no commercial software seemed to be able to do: give us a good working backup system for our enterprise. It uses very efficient synchronization and compression algorithms to move the changes from our distributed servers. If you want this rig to do backups too I recommend considering it. Here's a link to the RSYNC Project:
http://rsync.samba.org/ [samba.org]
Here's the Novell RSYNC forum:
http://forge.novell.com/modules/xfmod/newsportal/
And here's a good resource for RSYNC on Windows:
http://art.wilderness.org.au/software/help_cygwin
Here are two more good RSYNC Windows links:
http://www.itefix.no/phpws/index.php?module=pagem
http://www.nasbackup.com/ [nasbackup.com]
The NASBackup Project is a neat Open Source effort to make a gui-based RSYNC client for Windows. It works very well.
More info: RSYNC uses an algorithm that only sends the changes in the file systems. This algorithm is so efficient that i can even get down to only sending the changed blocks in an individual file without having to send the whole file. It works very well for us even over DSL/Cable speed connections. You want to optimize your entire I/O schema including all network layers as well as the way you read, write, and cache file and database operations on all connected hosts.
I hope this little bit of info helps you.
FreeNAS.org (Score:2, Informative)
SAN (Score:2)
We use SCSI 320s for our main file servers and databases and use UltraATA storage for disk to disk backups and other non-highspeed required storage.
iSCSI from LeftHand or EqualLogic (Score:2)
Nothing from EMC, HP, or the other big boys came close in terms of functionality and scalability at this low end of the market. EqualLogic and LeftHa
This is too important to rely on /. nonsense (Score:2)
Google's Appliance? Storage is cheap. (Score:4, Interesting)
Given an effective search, you can store the information on anything. That means you can deploy many cheap and fast servers close to the source of information creation, and have that information available everywhere. With 250 GB drives going for $50, you could have all 10TB of storage taken care of twice for $4,000.
Re:Google's Appliance? Storage is cheap. (Score:2)
Isilon (Score:1)
Just went through this... (Score:1)
What we ended up doing is getting a SATA2 SAN that supports 5 simultaneous connections over gig copper. We connect to it over iSCSI. We have:
- M
What is with all the NAS? (Score:1)
great product (Score:1)
I'm relocating to US from beginning of June, and I will be available under:
Krzysztof (Kristof) Franek
CEO & President
Open-E, Inc.
2694 Middlefield Rd, Suite A
Redwood City, CA 94063
Open-E develops innovative software products for cost-effective Network Attached Storage (NAS) and iSCSI solutions. It is with great excitement to announce that we are growing globally with our products and talented team members.
http://www.open-e.com/ [open-e.com]
ibrix (Score:1)
Their site seems to be down now, but google for them and you will see articles.
HTH, i'm no storage / SAN expert.
reliability, raid, updates, security, monitoring (Score:2)
Re:reliability, raid, updates, security, monitorin (Score:2)
You mentioned needing a support contract. What happens if the system goes down? Does the company go out of business after a few hours of downtime? How about a day?
No, but we are out some money. We could probably survive one days outage, and our existing NetApp FAS250 has next-day service on it. I'm happy with the NetApp, but want to do my homework on this next storage device just like I did three years ago.
Looks like there is enough advice on vendors and hardware spec
Debian updates checker, apt-update (Score:2)
My script in its current form will email security-related update notifications as they arrive, and other upgrades are only reported on Mondays. Some day, I'll write
From a Sun Employee... (Score:2)
OK. Up front:
<disclaimer>
I work at Sun. I do not speak for Sun in any way, however. None of what I'm saying is priviledged, or otherwise not publicly available.
</disclaimer>
A lot of what is right for you depends on your exact setup. Given your description, I'm assuming you are primarily concerned with file serving for clients, with the possibility of needing to centralize some primary storage for DB or similar app-specific servers. I'm also going to assume you are single-site (given the rela
Re:From a Sun Employee... (Score:2)
I've been following ZFS for the last few months (since the November slashdot post about the open source release) and am very impressed with its' capabilities. That, combined with the binary-ready build of AFS on Solaris, and this is a very attractive solution that I'll look into.
Talk to a Sun sales rep
Re:From a Sun Employee... (Score:2)
Honestly,just go with the 800 number. I can't say who your Acct Rep would be these days.
Also, I'm not 100% sure that OpenAFS is covered by Sun. I'm pretty sure it is, but... Everything else definitely is covered, though, and at the minimum, you'll get help from a Sun SysEng if you're looking at OpenAFS. (with a Gold Contract, SysEng's respond quite fast. ;-)
-Erik
SAN/NAS with multiple tiers of performance (Pillar (Score:1)
We were in
Recommendation for NetApp (Score:1)
We tried Snap Servers, Sun SPARC boxes with FC disk arrays, Linux boxes, various RAID
Consider this - there is lots being left out (Score:1)
1) It doesn't matter if you use SCSI or FC - if it's connected via iSCSI that will be your bottle neck. 2gb / 4gb Fiber Channel will give you the perf that iSCSI won't. Eiter way spend the money on good HBA's, and get a spare
2) If it's not something you're already fimilar with, you should spend the money and get a box from EMC and let them deal with it. Yes, some times its worth spending the money on. They show up to swap out any "issue" component, and th
Do you care about Unix-side security at all? (Score:1)
Netapp (Score:3, Interesting)
o Data OnTap OS
o NAS and iSCSI
o Optional FC interface (yes, NAS, iSCSI and FC in one device)
o "simplified" web interface
o Based on FAS250/270
o $5000 entry level price
o Scalable to 12TB
Presumably the products will launch some time in June.