Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×

Nice Performance Tuning For UNIX 206

Posted by CmdrTaco from the never-nice-enough dept.
Professor writes "Be 'nice' to your computers and examine some general guidelines for tuning server performance. A computer is like an employee who does tasks for you -- it's a good idea to keep from overburdening them. Keep this from happening by using the UNIX 'nice' command."
This discussion has been archived. No new comments can be posted.

Nice Performance Tuning For UNIX More

Nice Performance Tuning For UNIX

Comments Filter:

  • Scheduling Priority is for sissys (Score:5, Funny)

    by SatanicPuppy ( 611928 ) <Satanicpuppy.gmail@com> on Wednesday April 12, 2006 @11:15AM (#15113782) Journal
    Set 'em all to -19, and let the best program win! If they don't have to fight each other for CPU cycles they will grow up weak and feeble.

    • Have a DVD-ripping death match! (Score:5, Funny)

      by MoxFulder ( 159829 ) on Wednesday April 12, 2006 @11:22AM (#15113846) Homepage
      Now here's a fun thing to do: rip a DVD with several programs at once, all at nice -19 at you suggest, and encode it to XVid and DivX and Theora all at once for good measure.

      Watch the DVD drive churn and seek and gasp!
      Watch the encoders fight for CPU time with top open in a terminal window!
      Run some unnecessary I/O-bound process like updatedb in the background so that the hard drive can get in on the thrashing!

      Wheeee! What fun...
      • Do you complain about the lousy framerate you get playing your FPS while the rest of this is happening?
      • This brings up a good point. CPU isn't the only limited resource. I've often ran applications that saturate IO busses or network interfaces, or eat up a ton of memory, but only used a fraction of CPU time.

        In the first case, a large simple parsing app or file compression/decompression can saturate an IO bus. While performing such operations, other user-sensitive tasks like opening an application or checking mail. Granted most users don't do a lot of IO-flooding apps, but what's more prevalent are network-flo
        • Yes, that's a very good point. It would be nice if it was possible to throttle other resources in a uniform fashion.

          With network bandwidth it's quite application-specific. For example I can tell Azureus (uber-featureful bittorrent client) to only use a certain amount of upload bandwidth. But I can't do this with Gaim, and when I'm sending someone a big file too fast, my web browsing grinds to a halt. It'd be handy to have a uniform way to restrict the bandwidth of apps.

          Storage device accesses would be a
        • Well QoS has existed forever in most posix operating systems out there for network interfaces. Theres your network QoS.

          Now I wonder if without using virtualization, on the command line, you could limit memory available to a process. One of the big 3 UNIXen out there probably has that.
    • If they're lucky, they'll get placed on the game grid, where they'll use futuristic "light cycles" and weird jai-lai pelotas to battle for supremacy.
    • i use nice a lot, and also renice afterwards when something goes kindof out of hands, too bad the powernowd needs to be disabled to let the machine run at full speed but process still at a nice priority ...

      is there a similar option for kernel modules ? some kind of scheduling trick ? the vmplayer from vmware is quite nice, but with it's kernel module it totally annihilates the rest of your linux, and renice'ing the vmware's process itself doesn't really have an effect, since the kernel module is wh
      • I hardly ever use nice...Usually only to de-prioritize some job that isn't playing well with others, so you're nice-knowledge is bound to be superior to my own.
         
        • I haven't used it recently too but at one point, I used it quite often. Back in the mid to late 90's when ESRI's ArcInfo software ran on UNIX systems, I was used to do a fair amount of GIS processing on datasets. Generally I was pulling together tiled data into a single dataset or doing some spatial data checks that took a while so I would either lower other users priorities or increase mine (I also was the SA for most of these jobs too). Most of the other users on the system were pretty much just data c
        • "Usually only to de-prioritize some job that isn't playing well with others"

          Once, while in college, I was sitting in the Solaris lab wondering why every workstation was slow as molasses. It quickly became apparent that an over-enthusiastic EE major was using every workstation in a cluster to do his bidding. He was not being 'nice' at all :(

    • Set 'em all to -19, and let the best program win! If they don't have to fight each other for CPU cycles they will grow up weak and feeble.

      Very funny. Its like when I'm talking to people that believe in heaven and hell, I just say "Kill 'em all and let God sort them out!"

      But back on the topic. Err, I'm new to UNIX/Linux, I've only been using it since 1993, but nice and getpriority() setpriority() and scheduling have been around for a _long_ time.

      Debian launches services via the start-stop-daemon program th
      • I don't know, I mean, it's kind of nice for some reasons, though I agree, nice is ancient. There is so much in Unix though, that I don't mind occasional articles about semi-obscure commands, and nice is obscure for a lot of people...I can't count the number of times I've had people bring up "irreconcilable" job scheduling to me, that can easily be solved with nice...So I think refresher articles are a good thing; occasionally I learn something.

        • I don't mind refresher stuff either, but UNIX was designed from the beginning to be multi-user and multi-tasking. The top command has a nice column. The manpage for top says:

          The nice value of the task. A negative nice value means higher priority, whereas a positive nice value means lower priority. Zero in this field simply means priority will not be adjusted in determining a task's dispatchability.

          nice, and renice are common commands for dealing with running processes. top can renice a process as w
      • I'm new to UNIX/Linux, I've only been using it since 1993....

        Considering that half of the /. population wasn't even reading in 1993, I think that allows you to drop your "new" classification.
        • I'm new to UNIX/Linux, I've only been using it since 1993....

          Considering that half of the /. population wasn't even reading in 1993, I think that allows you to drop your "new" classification.

          I consider myself pretty "new" to the UNIX/Linux world, too, and I've been using some variation since 1993. I'll stop considering myself new when I can remember every command ;)
          • I'll stop considering myself new when I can remember every command ;)


            By that standard everyone is a *nix newbie.
          • I consider myself pretty "new" to the UNIX/Linux world, too, and I've been using some variation since 1993. I'll stop considering myself new when I can remember every command ;)

            While we are on the topic, is there a market for a slashdot style site that is more geared for computer professionals?

            Slashdot has a very smart userbase, and I would not abandon slashdot, but I would like to get away from the highschool and undergrad "know it all" people. Articles like this one are way below anyone who does Linux or
            • For Linux news, LWN [lwn.net]; for general tech/sci news, Technocrat [technocrat.net].

              Slashdot has a *lot* more users than either though. Although some times it can seem otherwise, the good comments can show though... you just need to browse at +4 and ignore anything posted = 25 minutes after a story is posted. :)
              • Slashdot has a *lot* more users than either though. Although some times it can seem otherwise, the good comments can show though... you just need to browse at +4 and ignore anything posted = 25 minutes after a story is posted. :)

                Hey, I browse at +4 already. 90% of my foes are those MMLM people with "free" iPods in their sigs. That has gotten rid of most of the college kids. I'm a subscriber and give bonus points to friends, friend of friends, interesting, informative, yada yada. I've been reading slashd

  • First Post? (Score:5, Funny)

    by bje2 ( 533276 ) on Wednesday April 12, 2006 @11:18AM (#15113814)
    nice -19 -n doFirstPost

    i'll surely get first post this way!!!

  • Next up: "man nice" "man man" "man mount" ? (Score:5, Insightful)

    by Gothmolly ( 148874 ) on Wednesday April 12, 2006 @11:19AM (#15113828)
    Performance tuning means that IO and other resources are sufficient to run tasks. The 'nice' command isn't that. 'nice' lets you run jobs whose complete time can vary, since you can put them on the bottom of the list.

    Performance tuning is fiddling with /proc, and matching file and FS parameters with your page size.

    This is a non-article.

    • Re:Next up: "man nice" "man man" "man mount" ? (Score:5, Funny)

      by eln ( 21727 ) on Wednesday April 12, 2006 @11:23AM (#15113854)
      Whoa whoa whoa, slow down. First I learn there's a "nice" command, and now you're telling me there's a "man" command too? This is way too much information for one day.
    • One of the things I don't like about nice is that people often use it to baby weak code. Some joker writes the world most inefficient Perl script, and then, because it takes forever to run, uses nice to give it a higher priority than it deserves, thus cutting it's runtime down to an acceptable time, rather than just fixing the damn code. I hardly ever see anyone use it nicely (har har), setting a non-critical job to run at a lower priority.

      So I'm always suspicious when I see nice used. I really prefer to just schedule more intelligently, so things don't overlap as much, and running daemons that hog cycles 24/7 on their own hardware.
      • Some joker writes the world most inefficient Perl script, and then, because it takes forever to run, uses nice to give it a higher priority than it deserves, thus cutting it's runtime down to an acceptable time, rather than just fixing the damn code.

        Well, this joker has root access, because only root can increase a priority of a process.

        No system I run with multiple users would have such an incompetent person that is allowed to have root access. In fact, when a user writes some broken perl script that burn
      • I use it on my own system all the time. I've got a dual processor Mac, but once in a while I have too many things running in the background (encoding a movie, analyzing my data in Matlab, etc) and want to keep manipulating some images in the foreground, so I set Quicktime and Matlab and whatever else to -19 priority. They still finish, but when I need the processor to open/close/resize/apply a filter to my image, it takes it and then lets the rest work on the past stuff. I find that it can greatly increa
        • I think you're off by a factor of -1.

          Negative numbers (down to -19) have the highest priority, and positive numbers have the lowest priority. Yes, it's backwards and nonsensical and I can only assume that some programmer somewhere is having a good laugh at everyone who's ever been confused by it.

          I personally haven't ever used "nice" on my Mac -- most of the stuff I launch from the commandline isn't that processor-intensive. I do use "renice" a lot, which modifies the priority of a process that's already run
          • It's counter-intuitive, but the shell command would be 'nice -19 program' to run it in really nice mode.

            To run a program in hog mode, you would go 'nice --19 runmenow'

            Thus, his '-19' reference makes complete sense.

            -- at least that's how it works for the 'nice' binary, and bash. csh, on the other hand, uses 'nice +19' to be nice and 'nice -19' (as root) to be nasty. -- One of the many things that csh does weird that weaned me off of it. (but let's walk away from bash/csh flame wars, shall we?)

      • I renice 10 Folding@home so that the computer is actually useable when I need it. When I'm out, it can do what it wants !
    • From what I recall, the O(1) scheduler in Linux doesn't properly interact with nice anyway (i.e. the difference between a extreme ends of the nice scale is very small). One of the design requirements for the FreeBSD ULE scheduler was that the nice scale should follow a curve which is steep at both ends.
      • If the idea of the scheduler is that the system administrator has to hand-tune all their processes with 'nice' to balance interactivity, CPU-hogging, etc., then yes, the Linux scheduler does a terrible job. However, for an untuned workload of dozens of nice = 0 processes, it works great, though.

        Having said that, I do nice down my "top" processes and set the delay to at least 5 seconds. This seems to give a better idea of what's running a lot as top then uses less resources.

        $0.02USD,
        -l

  • Here's a tip. (Score:5, Funny)

    by Anonymous Coward on Wednesday April 12, 2006 @11:20AM (#15113835)
    When using 'nice' try the -666 option to enable the evil bit. Hilarity will ensue.
    • When I saw 666, the evil bit and Hilarity, I immediately thought Hillary (Clinton)???
    • Hey, that really works!
      calum@foo ~ $ nice -666 bash
      .... Entering bash666 .....

      666 > ls
      Bad command, Satan.
      666 > help

      Purge
      Drown
      Kill
      Maim
      Loot
      Rape
      Exfo liate

      help help for more minions.
      666 > ^D
      calum@foo ~ $ whoami
      calum
      calum@foo ~ $

  • I got my server tuned (Score:2, Funny)

    by Anonymous Coward
    You don't need the nice utility.

    Some "R" stickers along the side, a huge wing on top of the box, and replacing the sound card with a fart can is all you need to tune your server.

  • Bad teacher, no cookie! (Score:5, Insightful)

    by Fubar420 ( 701126 ) on Wednesday April 12, 2006 @11:24AM (#15113872)
    FTFA: "In fact, only the ps command was running when I generated this list. Most tasks are designed to do what they need to do quickly and then exit or sleep."

    Of course, because all other processes, at the instant PS was running, were blocking on the CPU. In other words, on a uniprocessor system, you can only have one process running at a time, and in the case of a process that reports the state of other processes, its only THAT PROCESS THAT WILL APPEAR RUNNING...

    Go play in /proc/self for a while.

    • Too bad I dont have mod points, +1 Insightful to you!

    • Re:Bad teacher, no cookie! (Score:5, Insightful)

      by alexhs ( 877055 ) on Wednesday April 12, 2006 @11:47AM (#15114067) Homepage Journal
      Of course, on a uniprocessor system only one task can run at a time. However, (GNU) ps and (GNU) top report R as Running or Ready to Run, so there may be more than one at any time. Average of runnable tasks count get you the load.

      Nota : I put the GNU because BSD equivalent behaves differently, I just checked and I get details like "select" where I guess I would only get "Sleep" on GNU/Linux.

    • Re:Bad teacher, no cookie! (Score:4, Insightful)

      by ultranova ( 717540 ) on Wednesday April 12, 2006 @01:26PM (#15114853)

      Of course, because all other processes, at the instant PS was running, were blocking on the CPU. In other words, on a uniprocessor system, you can only have one process running at a time, and in the case of a process that reports the state of other processes, its only THAT PROCESS THAT WILL APPEAR RUNNING...

      Not true. "Running" as a process status doesn't mean that the process is burning CPU time right now, it means that it is in the runqueue and will round-robin with all the other running processes for the CPU time.

      The opposite state, sleeping, means that the process is not currently in the runqueue because it is waiting for something - for a disk read to conclude, for some data from the network, for your keypress, or simply that some time has elapsed.

      I repeat: a "running" process is one that is in the runqueue, not neccessarily being run by the CPU at this particular microsecond.

      • Although the author of TFA could have been a lot more clear, in his defense he does explain this (admittedly in a roundabout way):

        The scheduler maintains a number of internal tables to manage the context of every running task in the system. It also manages resources using a pair of queues known as the run queue and the sleep queue. The run queue is where tasks that have all of their resources available are held. The sleep queue is for tasks that are waiting for one or more resources to be available.

        He cou

  • Worst. Advice. Ever. (Score:4, Informative)

    by V. Mole ( 9567 ) on Wednesday April 12, 2006 @11:26AM (#15113887) Homepage

    Ah, yes, the extremely bad idea of running updatedb at low priority surfaces again. Then, instead of finishing during the early morning hours, it lasts all day, interferring with real work. Yes, this is what really happens: we tried this quite a while ago in Debian, and it's a Bad Idea(tm). What happens, IIRC, is that updatedb gets CPU so rarely that other tasks end up flushing the file buffers, and updatedb has to re-read the disk, over and over.

    If the problem is that your system isn't on all the time, and anacron is running updatedb when you log in, then just disable updatedb. You probably never use 'locate' anyway.

    • Yep. We had that problem as well with some big file servers and NFS. It turned out that pdflush was set to flush data to disk once every five seconds if filled, and once every 30 seconds if stale. This was KILLING our NFS performance. Of course, once we found the real performance tuning points in /proc/sys/vm, we were home free on that. Using nice as a performance tuner of a system is just silly. nice can be used to tune the performance of programs interacting with each other to some extent, but when it com

    • Re:Worst. Advice. Ever. (Score:4, Informative)

      by gowen ( 141411 ) <gwowen@gmail.com> on Wednesday April 12, 2006 @11:48AM (#15114082) Homepage Journal
      Then, instead of finishing during the early morning hours, it lasts all day, interferring with real work.
      If there are no other processes competing for resources, niced and non-niced processes will complete in approximately the same time. If your niced late-night updatedb is taking forever, its because you've chosen to run other processes overnight as well. And if your updatedb runs quickly, then the other processes will "interfere with real work".

      In short : nice doesn't change the total amount of time your processes take (or, at least, not by very much), it just changes which one finishes first.
      • In short : nice doesn't change the total amount of time your processes take

        No. In the instant case, the problem was that the system was 'thrashing'. Useful data was being flushed, and it had to be reloaded from disk again and again. That causes the system to chew up extra resources and slows things down. In a case like that, increasing the priority of updatedb would have it get it's work done quickly during low priority times and then get out of the way. (of course, if you're on a server, you probably

      • In short : nice doesn't change the total amount of time your processes take (or, at least, not by very much), it just changes which one finishes first.

        Actually it does change the total amount of time. The issue in question here is about disk caching. The problem was that, if you run updatedb niced, then it doesn't get to run as often. In fact, it can run so infrequently that between one timeslice and the next, the disk cache has been replaced by other stuff, because of all the processes that ran in between
        • The issue in question here is about disk caching. The problem was that, if you run updatedb niced

          One of the funniest things I ever saw as a sysadmin was updatedb running down an arch repository... on a database server.
    • Is there any reason to use slocate/locate rather than have beagled index everything on your system?

      I'm genuinely asking. I have no idea. But do you really need to use locate when you can use beagle-query? [antezeta.com]
    • Agreed. Use the old newspool trick: `updatedb` runs like greased lightening on a partition mounted with `noatime`. ~1 min for 40 GB. All those accesstime updates eat most of the wall time. I mount all partitions NOATIME in /etc/fstab unless I _really_ need atime.

  • Wow! Amazing what constitutes for an article on Slashdot.
    "nice" is but a small subset of performance tools, and is the least likely command I would run on a system.
    "nice" is fine when you have a series of batch jobs, and you want a specific job to recieve more CPU than another. That's a very small subset of realworld situations.
    • I just don't get the whole submission thing here sometimes. My submission on the value of tr got rejected!

      • I agree.

        I mean, I know they say you shouldn't gripe about these things, but so many unix(-alike) users don't know the full power of programs like true and yes.

        What if I was learning shell scripting and didn't know how to define the truth condition? What would I do? What would you do?? What would Brian Boitano do???

  • There are two keys to efficient server tuning. One, build the server with the capability to run at least 3 times as much as you need right now. Then kill -9 any process that slows your watching the p0rn filter go nuts during lunch and breaks. Who really needs a freaking relational database engine anyway? The accounting department can kiss off, you have blackmail to collect.
  • Which one of you basement runt been treating UNIX so badly that someone had to post an article on being "nice" to UNIX? Someone here deserves a beating with a network cable.

  • Repetitive tasks and tuning... (Score:5, Interesting)

    by Dr. Zowie ( 109983 ) <slashdot@defores t . org> on Wednesday April 12, 2006 @11:45AM (#15114053)
    I used to work at NASA/GSFC [nasa.gov], and one of the workstations there sat all day running periodic housekeeping tasks from cron -- parsing telemetry, handling command load updates, etc. The problem was that every once in a while something would stall and the next batch of cron jobs would launch before the first ones completed. Instant snowballing death would ensue as nothing completed and the load average would soar into the hundreds as cron maniacally, stupidly spawned more and more processes into the poor overloaded workstation.

    There are several relevant tools available now but then I wrote my own - a perl script called "qproc" that would queue up jobs for execution, kill them if they hung too long, and refrain from launching multiple copies of the same job at the same time.

    Until I got hit by that, I never thought about the fact that cron is very dangerous to use on a production server. But it is -- if cron tasks use a non-infinitesimal part of the computer, you have to take steps to prevent the same marching-broomsticks failure mode.

    • This is an old problem, and it's not one that can be solved perfectly in the scheduler, due to race conditions (what if the admin was doing something at the wrong moment?). Fortunately, it can be solved perfectly in the application being run. Just use a pair of flock()ed lock files to ensure that (a) no more than one copy of the process is running at any one time, and (b) no more than one copy is waiting to run at any one time.

      Algorithm:

      - try to get the 'run' lock. if we get the run lock, then run t

  • Off topic: Zombies (Score:4, Interesting)

    by smcdow ( 114828 ) on Wednesday April 12, 2006 @12:03PM (#15114212) Homepage
    As someone who just had to deal with a machine becoming inoperable because the process table filled up with thousands of zombies because 3rd party software wasn't reaping its children.....

    Please, for the love of all that's holy and Unixy, teach yourselves what a SIGCHLD is and how to use wait(2).

  • Does anyone know why lower numbers give higher priority? Isn't it more intuitive to use higher numbers for higher priority? I'm curious as to why it was done this way...
    • Probably because it's always been that way. IRQs are prioritized with lower numbers getting higher priority (except for the peskiness that ensues with 8-15 being tied back to IRQ 2). Someone probably decided a long time ago that it made sense (do 1, then 2, then 3, etc.) and it's stuck. But signed bits may play into it as well.
    • Have you ever finished first in life? First in line is next to go.
    • It's because they changed the metaphor from "having priority" to "being nice." If a process is more nice, it lets other processes use CPU time first. If it is less nice, it hogs it all for itself.

      It's confusing if you are used to thinking in terms of priority, but it makes sense after the first few times you see it.
    • Think of "priority" as a commodity to be divided up among all the processes, and the amount of priority a particular process gets is its nice value divided into priority. The smaller the divisor assigned to a process, the more CPU time it gets.

      priority / nice = amount of CPU time.
    • It's the positive viewpoint; the glass is half-full.

  • Ric Romero? (Score:2, Funny)

    by cashman73 ( 855518 )
    Is Ric Romero [go.com] taking notes on this article?
  • nice article.
  • The only use for nice I've ever been forced to implement was renicing processes based on the job submitter's rank in the company. (This was a CPU-intensive render/simulation farm.)

    Well, there was time some bozo's log rotation script filled up var with log copies - he copied every file every day, even copies of copies, until the filesystem ran out of inodes. THere were too many files to delete with "rm *", so I did something like "for n in 0 1 2 3 4 5 6 7 8 9 ; do nice -n $n rm ${n}*; done". I don't think it

  • Jesus, what kind of a commie pinko are you? (Score:3, Funny)

    by RomulusNR ( 29439 ) on Wednesday April 12, 2006 @02:00PM (#15115098) Homepage
    A computer is like an employee who does tasks for you -- it's a good idea to keep from overburdening them.

    Don't I already pay this hunk of junk enough in electricity and bandwidth to keep it from slacking off? If it isn't willing to constantly pull more weight, for the greater success, what am I keeping it on for?

  • Nice is Nasty (Score:4, Interesting)

    by redelm ( 54142 ) on Wednesday April 12, 2006 @02:48PM (#15115466) Homepage
    Seriously, nice is not a good performance tuning tool in most cases. All it does is shorten the timeslice, inducing more cache misses. The work remains the same or goes up due to cache reloads.

    A good [Linux] scheduler already does process restart immediately upon unblocking to reduce latency.

    But an overloaded box is still overloaded. The question is how you want it to fail. In what direction? A little cron job watching load with a shedding/restart list probably does better.

  • nice for the Mac geeks (Score:4, Informative)

    by Been on TV ( 886187 ) on Wednesday April 12, 2006 @02:54PM (#15115514) Homepage

    For Mac OS X users trying out the commands in the article, you need to type the following to get the list of processes as shown in the article:

    ps -ax -o pid,state,nice,command | less -5

    Also, on a standard Mac OS X system, the updatedb command to update the locate database is run by cron from the 500.weekly script located in /etc/periodic/weekly/.

  • nice -5 apachectl start (Score:3, Interesting)

    by Spazmania ( 174582 ) on Wednesday April 12, 2006 @03:48PM (#15115904) Homepage
    Its also very helpful to run Apache at nice level 5. That way when the CGI programs go nuts and the load shoots up to 200 the sshd process won't get starved for CPU time and you can still log in to fix the server.

Slashdot Top Deals

Two can Live as Cheaply as One for Half as Long. -- Howard Kandel

Close