Ask Slashdot: NFS on Free OSes Substandard? 107
Yet another fearless member of Clan
Anonymous Coward wrote in with this intriguing
issue: "I am trying to convince
my company to move off of Digital Unix and
Sun OS to either FreeBSD or Linux as our
primary server platform. The main argument
I am getting is the NFS client performance
on these free OSes is much worse than that
of Solaris or DU. Can anyone give any
recent data on relative NFS performance on
these platforms?"
Re:NFS performance (Score:1)
NFS (Score:1)
The network is partly shared 100BaseT, partly switched 100BaseT.
Some of these workstations have multiple processors and are heavily used for big jobs with a lot of I/O to the NFS filesystem. Other systems are every now and then abused by students.
At the beginning, I had some filesystem consistency problems, 4 or 5 in a year. Since I have installed Redhat 5.* (almost one year ago) there were absolutely no problems at all.
On the other hand, a Solaris server sharing some home directories with other two other Sun/Solaris computers gave me troubles about once a month. In the end the owners switched to PCs running Linux.
So, as robustness, my experience shows Linux is superior at this moment.
As performance, I don't have any useful data.
The systems work fine, I didn't see any delay to access a file, all the users are happy, so I didn't make tests.
Re:Open Source solutions (Score:1)
> it was usable. NFS is relatively easy to
> configure and use, but it needs NIS database
> for user authentication.
Actually, NFS does
NFS and Linux (Score:1)
NFS daemon every 15 mins from cron. I noticed that 2.2.7 has NFSv3 support. Hopefully by the time we come to 2.2.30+ and the knfsd stabilizes, we'll have a solid NFS server (perhaps 6 mo from now). Meanwhile, evaluate for yourself whether you have the luxuxry of explaining to colleagues or boss that things will most probably get better very soon. If you don't, leave it to the those who have the clout in their offices who can stand by Linux.
On the other hand, Linux is an excellent Samba server in my experience. Push for it, if you see a need. Once the confidence builds, push for Linux as NFS server. Hopefully, Linux NFS should be stable by then.
Ramana
Re:Another Question?? (Score:1)
I havent really tested for the purpose of getting hardcore numbers, but IMHO the new linux server beats the pants off the SPARC, and basically due to the huge increase in horsepower under the hood. I recently tried coping over about 3 gigs of files, using the Sun as a NFS server, and it was chunking them across pretty good, although it loaded up to about 21 load average and spawned about 10 nfsd procs. I did the same thing, this time as with the linux server as the nfs server. It only had one nfsd proc running, and it made it to about, oh,
Of course, this doesnt really mean much, as the old server was being used by users at the time (although no more than about 3 of em) and the new was not.
In other tests I have done, i copied over a 600 meg cd image from this new server to a linux box, and while the server didnt break a sweat at about 1 load average, the nfs process was comsuming about 30% of the cpu time.
For us, linux is the only way to go, as a department with no more than a 5 grand a year equipment budget, we cant afford to upgrade our Suns at all, and for a fraction of the price of a new Ultra, we have a pretty awesome machine.
Hope this helps somewhat
-hamster
FreeBsd & Linux NFS performance (Score:2)
Linux showed a 14% NFS performance penalty over local disk access, and FreeBSD showed a 55% NFS performance penalty.
If anyone cares about the test specs, send me email at mka@ieee.org
Re:Lacking: NFSv3 (Score:3)
http://www.linuxhq.com/
2.2.x vs. 2.0.x (Score:3)
It is unfortunately not NFSv3 yet, though a fair amount of the features have been implemented. And there are still a few lingering bugs from what I've heard, though I personally have yet to run into a problem with it.
The raw preformance I've seen is around 40% faster on a single client, which is corroborated by RedHat's experience. They also claim over 400% improvement in multi-client environment.
Probably the most important considerations is WHAT ARE YOUR NEEDS?
If you have extremely large traffic requirements (read large number of clients or large files) or if you absolutely need NFSv3 compliance (for 64-bit file handles, etc), then don't use Linux or *BSD.
If, on the other hand, you are handling a couple dozen clients with low-to-middling NFS requirements, save yourself a boatload of money and use a Redhat server.
But even if the free Unices don't make sense for your NFS servers, by all means recommend them for other tasks - mail server, web server, database server, router, etc, etc.
Open Source solutions (Score:5)
Then, there is Linux. The 2.0 kernel suffers from the userland-only nfsd implementation which has a real impact on the speed on especially fat pipes (>100MBit/sec). The interaction between userland/kernel demands many context changes and data copying between the two areas decreasing the overall speed and increasing the server load.
Linux 2.1/2.2 uses a kernel nfs implementation which is currently under heavy development and as such overall reliability cannot be foreseen. It still suffers from problems with using bigger read/write blocksizes. But HJLu (I think he is working on that) and the other contributors are doing a great job, so this area will improve over time.
If you want to run a big network with many clients (300+), you should currently go with a commercial OS such as Solaris (I don't know anything about HP-UX' NFS performance/reliability) and run it on vendor hardware (yes, I'm conservative). At the current stage of open source implementation of NFS, it would only discredit open source and yourself as a open source advocate, if you would suggest to use open source software for running a huge network. You can easily go with Linux or FreeBSD, if you want to build a rather small network (I have a client with ~70 networked stations depending on a FreeBSD 3.0 server) and don't need a really scalable solution.
solaris vs. linux nfs times (Score:5)
I use the following commands to write and read a file, respectively (see the NFS howto):
(1) time dd if=/dev/zero of=/opt/stuff/testfile bs=16k count=4096
(2) time dd if=/opt/stuff/testfile of=/dev/null bs=16k
One of the sparc clients has a line like this in its
linuxServer:/opt/stuff -
A linux client has the following line in it's
linuxServer:/opt/stuff
This is a typical (I say typical because I'm substituting the average times)
out put from (1) on the sparc client:
4096+0 records in
4096+0 records out
real 0m20.90s
user 0m0.24s
sys 0m2.49s
And for (2):
4096+0 records in
4096+0 records out
real 0m0.69s
user 0m0.04s
sys 0m0.62s
For the linux client, (1):
4096+0 records in
4096+0 records out
0.01user 2.05system 0:21.00elapsed 9%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (89major+15minor)pagefaults 0swaps
and (2):
4096+0 records in
4096+0 records out
0.00user 1.49system 0:36.13elapsed 4%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (96major+15minor)pagefaults 0swaps
For (2), the sparc is significantly faster.
If I change the rsize and wsize of the sparc client to 4096 each, the sparc
client will crash nfsd on the linux server.
We are just a small group of developers who happen to be linux enthusiasts, so we configured this setup ourselves. In short, we don't claim to be masters at configuring unix networks.
The sparc client is an ultra 5, the linux client is an 450MHZ HP Vectra (p.o.s.), and the linux server is a 333MHZ Dell Dimension.
Re:Hey, wait, tell us about CODA (Score:1)
Re:FreeBSD NFS (Score:1)
NFS performance (Score:2)
Linux growing in popularity (Score:1)
Looking at the numbers reported on The Internet Operating System Counter , it's fairly obvious that a lot of people are migrating to Linux.
I published an analysis "Growing Internet Presence" at which notes that during the past 3 months, the number of web servers (www, ftp, mail) grew a bit over 27% -- but Linux increased by 39%. (The Mac OS was the only other OS to grow faster than the market, while Windoze was a bit behind the market.)
That doesn't demonstrate performance, but does show that a lot of servers the used to be Windows or other Unix variants are now going Linux. With 31.3% of all servers using Linux, more experts are choosing it than any other OS.
Dan Knight, Mac Advocate dknight@reformed.net
Low End Mac
the iMac channel
"In view of the fact that God limited the intelligence of man,
it seems unfair that He did not also limit his stupidity."
- Konrad Adenauer
NFS currently better on commercial unix (Score:1)
We've got IRIX, Solaris, and linux running as clients and servers. IRIX and Solaris support NFS version 3. Linux currently supports only version 2. It has been my experience that NFSv3 helped performance quite a bit. Our mail server supports NFSv3. We noticed that mail user agents such as pine were running much slower on NFSv2 clients such as linux that it was on NFSv3 clients. Turns out that reading mail is a very write intensive operation! NFSv3 optimizes writes. There are of course other ways to optimize writes on a server mostly involving saving written data to memory in a RAID controller.
Another feature of Solaris is commands like nfsstat which can help you determine what's going on with your NFS servers and clients.
When I was your age we didn't have NFS. We had to feed stacks of computer cards into card readers which sometimes jammed sending you to a key punch to retype the 80 byte data card. Yep NFS seems pretty fast to me so be happy you at least got NFSv2.
Solaris sucks, let's use linux! (Score:2)
If it works, why do you want to change it? Surely your loyalty to Linux/FreeBSD isn't clouding your judgement now, is it?
Why reinvent the wheel?
ash
CODA, what it is (Score:1)
My big complaint was it needed a partition or at least a swap file-esque section for the share. So you can not just drop files in and expect it to work, you must know the size ahead of time.
Plus side is that it does disconnected access, so a laptop which edits a file in the share and then reconnects will copy over the new version.
Linux NFS sucks matter out of black holes (Score:1)
Network filesystems (Score:1)
FWIW, I'd try to run an AFS based filesystem like CODA. I'm not sure if there are any free DFS implementations for linux but I know you can get CODA up and running. I'm not sure how it performs relative to other networking filesystems but it is actively developed and I'm sure they are aiming at providing a high performance solution. Development is open. There are some ways to tune NFS but unless you specifically need it for legacy support or something I'd go with CODA.
Re:NFS is a Sun thing (Score:1)
Another Question?? (Score:3)
Doesn't the Hardware itself play a large role in the NFS server? I can't see an NFS server needing massive CPU power, but I can draw some lines to Memory I/O bandwidth, SCSI systems, and Network Interface devices. When any hardware componant is "weak" it could potentially effect the preformance some percentage, right?
So, comparing a Sparc w/ Solaris to a x86 w/ Linux/FreeBSD just makes me think your actually comparing a lot more than just OS's, and I would want to know the detailed specs on the systems being compared.
Or can someone somehow prove to me that the software is the over-riding influencing factor, and the hardware doesn't matter?
Linux 2.0 client kills IRIX 6.4 server (Score:1)
Nick
Re:Weird (Score:1)
Sun is the reference standard (Score:3)
Recent versions of HP-UX seem to have borrowed a lot of Sun technology (an update to 10.20 gave NFSv3, Sun's autofs, and ONC+ in one fell swoop, and 11.00 incorporates all these also). I work at an all HP shop right now, and I've have to say it works OK, though we don't make heavy use of NFS (no shared home dirs, SW builds on NFS filesystems, etc.), just some light data sharing.
As for FreeBSD, I only have a 3.0-CURRENT box current as of Jan. or so, and a 2.2.8 box, so I don't have firsthand experience, but reading the mailing lists, significant progress has been made on general NFS stability and functionality (e.g. NFS over TCP). I don't have a Linux box, so I can't comment there.
Mike.
Re:Linux 2.0 client kills IRIX 6.4 server (Score:1)
The Problem went away with Linux 2.2
IRIX 6.4 works fine here, but IRIX 6.3 as server and Linux (2.2.2 at the moment) as client locks up directories on the client machine. But this might be due to the use of autofs...
Lacking: NFSv3 (Score:3)
Of course, people are working on fixing this...
WARNING: Linux NFS v3 patches... (Score:1)
Of course, if you're willing to hack a little, load 'em up (especially if you have a variety of OS's and versions on NFS server) and report bugs to the kernel list.
Rumba (Score:1)
V2, V3, HW/SW costs, etc. (Score:1)
How can that be, that FreeBSD outperformed SOLARIS (2.6, patched), well, probably because the FreeBSD system has a CMD Ultra-Daytona cacheing (64M) SCSI RAID array versus 4 SCSI drives on the Ultra 5 (both systems using Adaptech 3940-ish cards). Now, some may call this cheating, but the cost of the FreeBSD HW/SW setup was only about $400 more than the SUN, and it has about 4G more space. (and we bought the Suns at edu discount, and the disks/controller 3rd party)
Now I admit that I don't know how stable this system will be under heavy use, it's been mostly just me for the past month or so. And Linux not having quite as 'finished' as v3 implementation or Journaling or similar FS (I'm also using softupdates under FreeBSD) may even be a few months behind FreeBSD, though it typically catches up quick when it falls behind in an area. So if you need a large, high-performance NFS server right now, lay out the cash for a NetApp. But if you are looking for something medium-performance/size and/or planning for a fall deployment, select a quality hardware cacheing RAID subsystem and I suspect FreeBSD and/or Linux will be able to easily outperform a similarly priced chunk of SUN hardware.
Also, of course, I've glossed over the essentially free Solaris/Intel, partly because I haven't had the resources to experiment with it, and partly because the last time I did (> 1year past), I was underwhelmed...
knfsd vs. luserland nfsd (Score:3)
linux's forte really is as a desktop unix (my friggin' P5-100 is _so_ much more responsive under X on the console than the ultra it isn't funny) and i think even the performance hit of the userland nfsd is outweighed by the performance gains in other respects (mostly X and file caching). it's also true that linux comes with a lot more software prepackaged whereas with most commericial unices you have to spend a week digging up and compiling such basic stuff as perl, python, or even bash.
until knfsd shakes out a bit more i probably wouldn't want to use linux as a really hardcore NFS server (multiple hundreds of clients, heavy load, etc.), but in my experience it's fine in more modest environments and as an NFS client with HP or Sun NFS servers.
tim
Re:NFS performance (Score:2)
to post performance numbers, say from a specific
number of operations between a Netapp filer and
a sun, lintel, linux-alpha, dec-alpha, and winnt
box, but I don't have the extra
report as FUD... We love linux to pieces but the
NFS performance has been a showstopper for us.
Tuning it isn't the solution; NFS writes are slow.
No fud here -- it's a certainty, not uncertainty,
no doubt about it at all.
Not much hope here... (Score:3)
It seems like most of the responses lean
toward "not using NFS" or "using something
else (CODA, AFS)" but apparently we are still
lacking in the NFS department.
Unfortunately for linux in at least one place
I know, this is terrible. What if (your shop) wants
to use Network Appliance for your storage solution? All of a sudden you have an argument
against linux based strictly on a technical merit -- NFS Performace. Not good. (When you get a
Netapp filer talking coda, call me!) Even dyed-in-the-wool linux advocates in my company are
forced to bite the bullet and use other platforms
because linux isn't suitable to task for NFS with lots of writes.
NFS itself is not to blame, after all Digital Unix
performs well in this context. Even a commercial
NFS implementation would be okay for a solution.
Poor NFS performance has been a problem with linux
for too many years now. I keep waiting for the
problems to magically go away, but I guess they
aren't going to. I've studied filesystems but still don't think I can fix this...
Use knfsd in linux v2.2.x kernels (Score:1)
The server was/is a P133 with IDE disks on a 10baseT network. (Yes, I know... pretty weak server...) When writing ~50 MB from two clients (in this case, Sun ultra 1's) the userland NFS server would hang. I am sure the problem would be worse on a 100baseT network.
Switching to knfsd fixed the problem. AFAIK, the NFS server on solaris and the other big commercial flavors of unix are kernel level servers.
Re:FreeBSD & Linux NFS performance (Score:1)
I don't think -STABLE has the latest NFS patches that make it stable. In -CURRENT, I
hear NFS has been almost perfected (in the kernel.)
NFS needs NIS??? (Score:1)
Re:NFS is a Sun thing (Score:1)
The good news is that we ordered a Sun box for our NFS server. It should be in "any day now".
About NIS+, I had to enable "NIS compatibility" on the Sun NIS+ server to allow Linux NIS clients to work with it, and it seems to work out of the box on RH 5.2, anyway.
Re:NFS is one of Linux's weak spots (Score:1)
A related question... (Score:1)
I thought I remember reading that Sun had a product to do this, I can't remember the name, though.
Re:Hey, wait, tell us about CODA (Score:1)
A disconnected client still acts like it's connected to the CODA system, and you can make changes to CODA files and they will be resynced when you connect back to the network.
At least this is what I gather from reading about it, I haven't used it myself yet.
Re:NFS (Score:1)
It's likely that Linux NFS was designed to work with Linux NFS, it doesn't work well with others.
I have Samba, it's only a server (Score:1)
(Sure smbclient allows access, but it doesn't allow shares to be mounted either)
Re:Cost & Y2k (Score:1)
NFS is one of Linux's weak spots (Score:2)
Also try mounting a Linux export from a Sun system and watch the Sun complain.
This is with the userspace NFS server, I haven't tried the new kernelspace one yet.
Why is it necessary to migrate from Sun and Digital anyway?
Re:NFS performance (Score:1)
Re:Linux NFS (Score:1)
Now Transarc has decided to take over this development on their own, and just finally released their first AFS client for Linux. Course, me being without money, I've not had a chance to use it yet... but i'd imagine it works... course, my university has been having a lot of trouble with transarc lately...
oh, and i've played with Arla. that was a while back tho.. it might be time now to look into it again...
Re:Linux NFS (Score:1)
thanks!
Linux NFS (Score:3)
Since I don't yet have access to Transarcs AFS client for Linux 2.2, I am using NFS to mount AFS off another machine that is capable of AFS. Working out authentication was a pain (luckily somebody did most the work for me) and it still isn't trustworthy. I login remotely to machines that are really on AFS if I have to do anything more than just a quick edit...
but among the odd things i've noticed: files I don't have permission to just don't show up. This is really annoyingish, when say, I accidently open a file with a stupid mode and it suddenly disapears (spent a good while debugging programs tonight before i thought to log into a remote machine and voila... there was my file, mode 000) I don't know for sure, though, if this is Linux's fault or NFS in general...
anyway, to sum up, I'd say stick with something else for NFS for now (I like Slowlaris... it was my first UNIX)... Linux still has a ways to go, methinks...
Re:FreeBSD NFS (Score:1)
10000+0 records in
10000+0 records out
10240000 bytes transferred in 0.653652 secs (15665829 bytes/sec)
% time cp bigfile
cp bigfile
% time cp bigfile
cp bigfile
% time cp bigfile
cp bigfile
this is with 3.1 client & server over a v3 tcp mount, 100baseT, full duplex.
Re:knfsd seems like admitting defeat (Score:1)
cjs
Re:Sun is the reference standard (Score:1)
Sun NFS is probably the best NFS there is. My point is that people have been calling it "stable" and "rock solid" even when it had very serious bugs and performance problems, and I'm not at all convinced that those aren't still lurking around (I simply gave up using NFS for anything serious).
knfsd seems like admitting defeat (Score:1)
Re:NFS performance (Score:2)
You don't need very complex test setups to measure those differences--simply read and write a bunch of big files with "dd". If you want to do it simultaneously from several clients, there are some simple, free tools that let you execute the same command on multiple systems in parallel.
NIS+ (Score:1)
Thanks
Scott
Scott
C{E,F,O,T}O
sboss dot net
email: scott@sboss.net
Re:NIS+ (Score:1)
Scott
C{E,F,O,T}O
sboss dot net
email: scott@sboss.net
NIS not needed for some PC NFS clients (Score:1)
Cost & Y2k (Score:1)
That's what someone said earlier, I disclaim any knowledge of knowing whether or not it's true.
On a different note, Sun boxes come with a real paucity of software, as far as I can tell. O fcourse, most of my experience is with older sun boxes, not newer ones, so I don't know if they do include all the utilities that I'm used to with Linux (such as compiler for more than 5 languages, make, bash, perl, etc.)
Note: I'm not advocating blindly replacing everything with Linux. It'll take a while before Linux is ready to be put on 64 CPU sun boxes (though I hear that there are people working on High end SMP stuff).
Re:Hey, wait, tell us about CODA (Score:1)
You can also preload, that is, if youre going somewhere with your laptop and know youre going to use emacs you can do some magic to make sure all the necessary files are cached.
Clear conscience is nothing but an expensive form of luxury
Re:Linux growing in popularity (Score:1)
Think about it, If you argument is "everybody's using..",or "everybody moving to....", 99% of the time your sentence will end with windows. Maybe your career too. Unless of course, you're a manager ;-)
-earl
Nhfsstone: NFS benchmark (Score:1)
2.9. Nhfsstone
Benchmark intended to measure the performance of file servers that follow the NFS protocol. The work in this area continued within the LADDIS group and finally within SPEC. The SPEC benchmark 097.LADDIS (SFS benchmark suite, see separate FAQ file on SPEC) is intended to replace Nhfsstone, it is superior to Nhfsstone in several aspects (multi-client capability, less client sensitivity).
Re:knfsd seems like admitting defeat (Score:1)
Overall, I do feel linux is very weak on NFS. It gets hit from too many directions. Being NFSv2 and userland exacerbate problems of NFS itself and problems with it being a "young" implementation.
But, I do have hope. And I think it will get there soon. Now that linux is being seriously thought of as a server I think NFS will be pushed along.
Careful - wait for CODA (Score:1)
Apparently there are still soft spots and non-standard bits floating around in there.
He does send problems and patchlets in to knfsd, but has no patience with the userspace nfsd.
Remember that the Titanic Linux network often got stuck when the (SGI) NFS server fell over. (I severely doubt that particular problem is still around).
Personally I'm waiting for CODA - NFS isn't the prettiest creature in any light, or on anyones platform.
Re:Linux NFS (Score:2)
About two years ago I tried the Transarc Linux AFS client. It worked but was a real pain in the a**. It was a kernel module distributed in binary form only, and always for a very old kernel (which, incidentally, didn't support all of my hardware). Perhaps this has changed.
-Tom
Re:NFS performance (Score:4)
This is documented right at the top of the nfs man page, and makes a world of difference. My group at work has a very similar situation to yours (most shares served by Digital Unix but adding more Linux boxes every day), and NFS was definitely a problem until we fixed this.
Div.
--
But my grandest creation,
As history will tell,
Was Firefrorefiddle,
Linux not too good but getting better (Score:2)
2.0 Linux had a reasonable, but not brilliant client - certainly slower than the better commercial unices. It didn't do locking (at all), but was pretty stable (we had some occaisional problems on SPARC, but none on intel). If you didn't need locking then it worked fine (the other performance benefits of Linux outweighed the NFS degredation).
2.2 Linux is meant to be much better performance and locking is getting there... but its currently flakey. However I haven't used it seriously yet so ignore me on post 2.0 NFS
Re:FreeBSD & Linux NFS performance (Score:1)
I would not go so far as to say that NFS in -CURRENT was perfected but it does look good with Matt's latest patches. I really must review them properly and try to help get them committed into -CURRENT...
Re:Linux/OSF Performance (Score:1)
Some NFS servers can change this behaviour to improve performance but I would not recommend it since it can cause data loss if the server reboots unexpectedly. For FreeBSD, the command 'sysctl -w vfs.nfs.async=1' will improve the performance of NFSv2 clients.
Re:Lacking: NFSv3 (Score:2)
The two stage write-commit protocol of NFSv3 is necessary to get any kind of performance out of NFS without gross hacks at the server and which violate the protocol semantics.
FreeBSD NFS (Score:3)
I am biased since I work on the FreeBSD kernel and in the past have been involved with fixing and optimising the NFS client but I also use NFSv3 on a regular basis and see excellent performance with 100baseTX (I haven't measured performance for about a year but I seem to remember multiple Mb/sec write performance).
I can't comment on Linux performance since I have only tested RedHat 5.2 (which has terrible NFS performance, IMHO) and I believe that many improvements have been made in the 2.2.x kernel series.
Re:Linux NFS server/IRIX client problems (Score:1)
Perhaps I'll try a later Linux kernel again - though that's sort of a pain due to LILO being dumb about IDE and SCSI drives :P
As for the 5.3 IRIX problems, it could help by using the -32bitclients option on your IRIX NFS server - however, not having more information on your setup, that's only an educated guess (from having seen it before)
Linux NFS server/IRIX client problems (Score:2)
My network was setup originally with an SGI Challenge S box as an NFS server. The client machines were a combination of PCs running Linux or Solaris, and a couple of SGI Indys. With this setup, there was little to no problems with NFS.
However, I moved a whole bunch of stuff over to a Linux server (some home directories) and I got hit with problems on the client side hard.
The Linux clients talk to the Linux NFS server fine, but clients like the Indy take a real dislike to it. Even forcing the Indy to use NFSv2 and trying static mounts over automount/autofs, any process on the Indy that tries to use NFS to the Linux server just hangs.
No matter what I've tried, I can't seem to fix the problem. I've tried the Linux kernel implementation of nfs and the userspace versions. Same problem.
My advice; stick with commercial versions of NFS for the time being, ie those that come with Solaris and IRIX (especially since NFS comes packaged with 6.5 :)
Re:NFS performance (Score:1)
The #1 reason for this is that a dedicated filer isn't even _allowed_ to do other work. The #2 reason is that the dedicated filer is configured to a CPU/memory/bus/disk balance appropriate for file service.
When you peek under the covers, though, neither the hardware nor the software is really anything special or unique. A reasonably intelligent person who knows something about performance measurement and tuning would generally have little trouble duplicating the functionality and performance of a Netapp (or any of their competitors) using commidity parts and software...for about 1/3 the price.
Re: Solaris sucks, let's use linux! (Score:1)
Re:Open Source solutions (Score:1)
Re:Linux as NFS client (Score:1)
making your benchmarks 'bogus'.
Where do I find the 'bonnie' benchmark s/w?
Some NFS Benchmarks I'd done recently (Score:2)
They aren't a complete test of NFS performance
by any means but illustrate that a Network
Appliance (F720) can match local disk performance
and trounce a SUN at NFS server performance.
I would add that though I use linux as an NFS
server at home for my 4 machine network with
few problems, my empirical experience is that
it is not ready for high-load mision critical
NFS server applications.
My opinion is that a Network Appliance's
clever write caching technology reduces the
NFSv2 write penalty dramtically and increases
linux client useability. We run 100+ fast
linux clients and 20+Suns and I know that this
setup is best server ($$$ and performance)
by a Netapp rather than ANY UNIX solution.
This first one shows raw sustained NFS
performance by using dd to read or write a
100MByte file over a switched full duplex
100MBit ethernet between a P-II 333 (3COM 905)
and Redhat 5.2 and a NetApp720.
In summary I can achieve approximately:
33Mbits/sec NFS write performance to the network appliance and
62MBits/sec NFS read performance from the network appliance.
XXX.8x8.com 28: time dd if=/dev/zero of=/mnt/ianb/testfile bs=16k
count=6250
6250+0 records in
6250+0 records out
0.050u 3.740s 0:25.24 15.0% 0+0k 0+0io 96pf+0w
XXX.8x8.com 29: time dd if=/dev/zero of=/mnt/ianb/testfile bs=16k
count=6250
6250+0 records in
6250+0 records out
0.040u 3.840s 0:23.13 16.7% 0+0k 0+0io 95pf+0w
XXX.8x8.com 30: time dd if=/mnt/ianb/testfile of=/dev/null bs=16k
6250+0 records in
6250+0 records out
0.040u 2.690s 0:14.60 18.6% 0+0k 0+0io 99pf+0w
XXX.8x8.com 31: time dd if=/mnt/ianb/testfile of=/dev/null bs=16k
6250+0 records in
6250+0 records out
0.030u 3.150s 0:12.58 25.2% 0+0k 0+0io 114pf+0w
XXX.8x8.com 32: ls -al
-rw-r--r-- 1 ianb users 102400000 Mar 11 10:29
/mnt/ianb/testfile
This 2nd benchmark is a GCC compilation of
8000 lines of C in many files. It illustrates
both the dramtic differences between local
disk/netappNFS/solarisNFS and the benefits
of using gcc and make options to improve
efficiency by running parallel compiles
(which uses idle CPU that is lost whilst the
OS waits for the remote RPC's to complete in an
unparallelized compile) and by using
interprocess comuncation insted of files in
/tmp to communicate between different
compile stages. Conclusion using 100Mbit
network, compile performance approaches local
disk performance.
NADS BUILD (GCC) ON SUN NFS DISK (10Mb/s net)
6.480u 1.460s 0:38.36 20.6% 0+0k 0+0io 12596pf+0w
NADS BUILD (GCC) ON SUN NFS DISK (100Mb/s net)
6.480u 1.110s 0:29.69 25.5% 0+0k 0+0io 12596pf+0w
NADS BUILD (GCC) ON SUN NFS DISK (10Mb/s net)
WITH GCC -PIPE (NO
6.890u 1.770s 0:33.17 26.1% 0+0k 0+0io 12597pf+0w
NADS BUILD (GCC) ON SUN NFS DISK (100Mb/s net)
WITH GCC -PIPE (NO
6.660u 1.280s 0:25.22 31.4% 0+0k 0+0io 11736pf+0w
NADS BUILD (GCC) ON NETAPP NFS DISK (10Mb/s net)
6.650u 1.230s 0:17.67 44.5% 0+0k 0+0io 12596pf+0w
NADS BUILD (GCC) ON NETAPP NFS DISK (100Mb/s net)
6.490u 1.250s 0:09.35 82.7% 0+0k 0+0io 12596pf+0w
NADS BUILD (GCC) ON NETAPP NFS DISK (10Mb/s net)
WITH GCC -PIPE (NO
6.940u 1.430s 0:13.93 60.0% 0+0k 0+0io 11741pf+0w
NADS BUILD (GCC) ON NETAPP NFS DISK (100Mb/s net)
WITH GCC -PIPE (NO
6.830u 1.140s 0:08.77 90.8% 0+0k 0+0io 11736pf+0w
NADS BUILD (GCC) ON LOCAL DISK
5.730u 1.020s 0:11.31 59.6% 0+0k 0+0io 11069pf+0w
NADS BUILD (GCC) ON LOCAL DISK
WITH GCC -PIPE (NO
5.700u 1.040s 0:09.71 69.4% 0+0k 0+0io 10271pf+0w
NADS BUILD (GCC) ON LOCAL DISK
WITH GCC -PIPE (NO
6.000u 1.100s 0:08.19 86.6% 0+0k 0+0io 10280pf+0w
Re:Lacking: NFSv3 (Score:1)
Re:NFS performance (Score:1)
dedicated network-attached storage like a
Network Appliance filer for NFS or CIFS.
BTW, Dell has one coming out soon, I hear.
Maybe the price will be better.
Re:FreeBSD & Linux NFS performance (Score:2)
According to RFC1094, NFSv2 servers must commit data to nonvolatile storage before acknowledging that a block write request has completed successfully. Given the relatively small size of NFS blocks (8K or less for NFSv2), forcing a separate write/sync/acknowledge cycle for each block that is written can result in relatively poor NFS write performance.
There are several solutions to this problem. Linux (and some commercial UNIX variants) acknowledge NFS write operations without forcing a disk write and sync for each individual block. This is obviously somewhat more dangerous, but yields much better NFS write performance. Some vendors offer hardware add-ins (such as the PrestoServ board) that provide nonvolatile storage that can be written to faster than a disk.
NFS version 3 has much better support for asynchronous writes without resorting to such hacks. Hopefully, Linux NFS3 will be usable soon.
If you are willing to live with the risk of data not necessarily being written to disk on the server when clients think that it has been, you can force FreeBSD to acknowledge NFS writes asynchronously using the command
/sbin/sysctl -w vfs.nfs.async=1
Retrying the benchmark after configuring FreeBSD to act more like Linux may yield different results.
Re:A related question... (SUN Samba software) (Score:1)
It's a handy URL to have if you use Solaris a lot.
John.