Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
DEAL: For $25 - Add A Second Phone Number To Your Smartphone for life! Use promo code SLASHDOT25. Also, Slashdot's Facebook page has a chat bot now. Message it for stories and more. Check out the new SourceForge HTML5 Internet speed test! ×
User Journal

Journal Journal: Biology Help Desk: Volume 3^3 5

As requested by the world's greatest masked mystery person, Anonymous Coward, it may or may not be time for yet another biology help desk thread, after a surprisingly long hiatus of about four years. Feel free to contribute both questions and answers.

User Journal

Journal Journal: Linux PXE server P2V: Debian Testing + Vmware Workstation 11

Keeping older hardware useful:
P2V and V2P with an old Debian Testing PXE server and Vmware Workstation 11
By Dave Bechtel / Kingneutron

MARCH 2017 â" so I've had an old PXE server (an âoeASUS S-pressoâ - Pentium-4, 32-bit portable box) sitting around for the last few years, and got a wild hair to do a P2V on it â" make sure it still works right and test an upgrade of Debian Testing to modern standards. Last time this box was booted was November 2013 â" and it was provisioned with Dreamlinux 5 back in January of 2012.

--Thankfully, the SATA drive and CMOS battery have survived with apparently no ill effects, since the box has been moved around through a couple of house changes with no special storage arrangements â" it's basically been unpowered sitting in a corner.

( My unit has a PCI slot on the right that has a Gig Ethernet card in it, and an unused AGP video card slot on the left. Motherboard only supports 100Mbit Ether. )

--Rather than using the antiquated PS/2 ports, I used an adapter to combine a USB keyboard and mouse into (1) USB port.

STEP 1: BACKUP the physical PC

--This box is so old, I had to install fsarchiver (my preferred backup) manually. Incidentally, it's also heavy as hell with a DVD drive and a standard SATA hard drive, so removing the DVD and converting to an SSD (or laptop drive) to save weight would be recommended if I wanted to haul it around.

(Physical box, bash shell, as root)
# apt-get update; apt-get install fsarchiver

# ddir=/mnt/extra
# outfile=spresso-p2v-backup-20170302.fsarchive.fsa
# rootdev=/dev/sda3

# time fsarchiver -o -A -z 1 -j 2 savefs \
    $ddir/$outfile \
    $rootdev

# cd $ddir
# fsarchiver archinfo $outfile 2> flist--$outfile.txt

--I then copied âoespresso-p2v-backup-20170302.fsarchive.fsaâ to my ZFS server (method omitted), since we will need it later for the restore.

--Also, since all of my Squid cache and ISO files for PXE are on /mnt/extra, I made a tar backup of those to my ZFS server.

# cd /mnt/extra
# time tar czpf /mnt/bkpspace/spresso-mnt-extra-bkp-20170302.tgz *

-------------

STEP 2: Setup the P2V VM in Vmware Workstation 11 and restore the backup

--I tried as much as possible to duplicate the physical box, but in this case I already had a P2V VM of my daily workstation (Ubuntu 14.04) that I wasn't using, so adding a 3rd virtual drive to the existing VM and re-using the existing GRUB saved me some steps. Running ' update-grub ' on the Ubuntu side after restoring enabled me to boot my restored environment.

VM details:

RAM: 1.5GB
Processors: 1
SCSI1: 23.5GB (this is the Ubuntu drive)
SCSI2: 20GB ( this is /home and swap )
SCSI3: added as new and then Expanded from 10GB to 60GB (after running out of space)

Net1: Bridged (gets DHCP address from my LAN)
Net2: Started out as Host-only and ended up as LAN Segment â" this enabled the âoeclientâ bare VM to boot over the network.

--I used Systemrescuecd to boot the VM, issued ' startx ' and used ' gparted ' to make 2 partitions that fairly closely matched my physical layout.

PROTIP: In hindsight, I should have made the root partition around 15-20GB because of all the package upgrades that needed to be downloaded. (I ran out of space on root once during the upgrade and had to issue an ' apt-get clean ' to free up space.)

sdc3: 10GB ext4 (restored root) â" make yours bigger.
sdc4: 50GB ext4 (/mnt/extra)

PROTIP: ' tune2fs -m1 /dev/sdc3 ' -- this will reduce the reserved space for ext4 and give you some extra usable free space.

--Now, since I had plenty of space to work with, I copied my fsarchive backup file from my ZFS server to the 50GB VM partition.

(still in Systemrescuecd, in the VM)

# mkdir /mnt/tmp
# mount /dev/sdc4 /mnt/tmp
# cd /mnt/tmp

# ifconfig # Make note of my IP address
# nc -l -p 32100 |tar xpvf -
##Netcat - Listen on port 32100 and untar

(Now on my ZFS server)
# tar cpf - spresso-p2v-backup-20170302.fsarchive.fsa |nc -w 5 192.168.1.95 32100
## (tar to stdout and stream a copy of the backup file to the VM's IP address)

--This is basically a quick-and-dirty way to transfer files over the network without resorting to FTP or slow SSH file copies â" the fsarchive backup file ended up being around 3GB and transferred in less than a minute over Gig Ethernet.

--Now to restore the backup:

(still in Systemrescuecd, in the VM)
# time fsarchiver restfs spresso*.fsa id=0,dest=/dev/sdc4

--And that's half the battle right there. Now we just need to make some changes to the restored /etc/fstab so it will boot. ( If we were doing a full migration, we would also need to adjust things like /etc/network/interfaces , /etc/rc.local , /etc/hostname , /etc/hosts )

--For completeness, here's more info on how to do a Linux bare-metal backup and restore:
http://crunchbang.org/forums/viewtopic.php?id=24268

--So now I ' reboot ' into the VM's already-installed Ubuntu 14.04 and edit my Dreamlinux /etc/fstab.

# mkdir /mnt/tmp
# mount /dev/sdc3 /mnt/tmp

# screen -aAO # PROTIP: GNU screen is invaluable for switching between virtual terminal windows

# fdisk -l # check out our disks
# blkid # get partition labels and UUIDs /dev/sdb2: LABEL="swapb" UUID="f0eb7148-4ff2-4eeb-a82c-349d384a5255" TYPE="swap" PARTUUID="ff1ca075-02" /dev/sdc3: LABEL="root" UUID="38f1d4be-3293-4272-ab79-4ad76cbd5a36" TYPE="ext4" PARTUUID="3a3b65f3-03" /dev/sdc4: LABEL="extra" UUID="603a61fc-4436-4cb0-baac-ef9170754228" TYPE="ext4" PARTUUID="3a3b65f3-04"

--NOTE that fsarchiver will by default restore the same UUID and filesystem label, so no worries.

( Now Hit Ctrl-A, then c to create a new Screen )

# cd /mnt/tmp/etc
# jstar fstab # use your own editor here, I happen to like Wordstar key bindings ;-)

--Now we can switch between those 2 virtual terminal windows and even copy/paste text without using the mouse. See ' man screen ' for more details.

--To make a long story short, I added or verified the following to SpressoVM's /etc/fstab to enable my existing swap partition and double-check that the root filesystem would be mounted as expected.

LABEL=swapb none swap sw,pri=2 0 0
LABEL=extra /mnt/extra ext4 defaults,noatime,rw 0 2

--While I'm here, I also restored the /mnt/extra files from their tar backup as well.

# umount /mnt/tmp # we're done with restored root

# mount /dev/sdc4 /mnt/tmp
# cd /mnt/tmp
# nc -l -p 32100 | tar xzpvf -

( then on my ZFS Server )

# cd /mnt/bkpspace; time cat spresso-mnt-extra-bkp-20170302.tgz |nc -w 5 192.168.1.95 32100

( now back in the VM )

# update-grub # make sure ubuntu knows how to boot dreamlinux
# reboot

--And that's pretty much it. After that, to make a long story short, I went through several cycles of
' apt-get upgrade ' and ' apt-get dist-upgrade ' making VM Snapshots along the way, and had to make allowances for files that were provided in more than one package.

--I also upgraded the kernel from 3.10-3-686-pae to linux-image-4.9.0-1-686. The full saga is documented with errors and fixes, so email me if you want to know more (but it's a pretty long read.)

# apt-cache search 4.9.0 |awk '{print $1}'

# apt-get install linux-headers-4.9.0-1-686 linux-headers-4.9.0-1-686-pae linux-headers-4.9.0-1-all \
linux-headers-4.9.0-1-common linux-support-4.9.0-1 linux-image-4.9.0-1-686-pae

STEP 3: Test PXE booting with another VM using a dedicated network segment

--The end result of all this: I created a âoeblankâ VM with the capability to boot from network (needed to modify the VM's BIOS for this) and after switching the 2nd network adapter to âoeLAN segmentâ I successfully booted the VM from PXE! Mission Accomplished!
--Now, since I've done all the heavy lifting in the VM and my original box is still working the same as it was (but running old software) I can use pretty much the same procedure to do a V2P (Virtual to Physical) to a spare 500GB laptop drive instead of having to repeat the upgrade all over again.

--For brevity, these are the instructions I include in my bkpsys-2fsarchive script:

# time fsarchiver restfs backup-root-sda1--ubuntu1404*.fsa id=0,dest=/dev/sdf1
Statistics for filesystem 0
* files successfully processed:....regfiles=159387, directories=25579, symlinks=49276, hardlinks=25, specials=108
* files with errors:...............regfiles=0, directories=0, symlinks=0, hardlinks=0, specials=0
real 4m26.116s
( 3.9GB )

# mkdir /mnt/tmp2
# mount /dev/sdf1 /mnt/tmp2

# grub-install --root-directory=/mnt/tmp2 /dev/sdf
# mount -o bind /dev /mnt/tmp2/dev; mount -o bind /proc /mnt/tmp2/proc; mount -o bind /sys /mnt/tmp2/sys
# chroot /mnt/tmp2 /bin/bash
# update-grub
[[
Generating grub configuration file ...
Warning: Setting GRUB_TIMEOUT to a non-zero value when GRUB_HIDDEN_TIMEOUT is set is no longer supported.
Found linux image: /boot/vmlinuz-4.2.0-36-generic
Found initrd image: /boot/initrd.img-4.2.0-36-generic
Found linux image: /boot/vmlinuz-3.19.0-25-generic
Found initrd image: /boot/initrd.img-3.19.0-25-generic
Found memtest86+ image: /boot/memtest86+.elf
Found memtest86+ image: /boot/memtest86+.bin
Found Ubuntu 14.04.4 LTS (14.04) on /dev/sda1
done
]]

# grub-install /dev/sdf # from chroot

^D
# umount -a /mnt/tmp2/*

# DON'T FORGET TO COPY /home and adjust fstab for swap / home / squid BEFORE booting new drive!
# also adjust etc/network/interfaces , etc/rc.local , etc/hostname , etc/hosts

--Once I had Knoppix up and running in the âoeblankâ VM, I used ' gparted ' to make a 768MB Swap partition and used the rest as a âoedataâ ext4 partition.

--NOTE that while the PXE boxes are running a 32-bit processor and environment, CLIENT boxes can boot a 64-bit kernel and environment as long as the CLIENT processor is capable.

STEP 4: Reverse the process and upgrade the physical box

--I haven't done the âoeV2Pâ part yet, but that's not a high priority at this point since I have everything pretty much the way I like it right now.

User Journal

Journal Journal: HOWTO safe rm in Linux 1

--HOWTO safe rm on Linux (at least):

- Interactive:
o Use Midnight Commander. Works from a text terminal, no GUI needed.
Insert to mark files/dirs, F8 to delete. Never had a misfire, even as root.

- Script (for the paranoid, and want a log):
IF you have a known dir " destdir=/tmp/blah " with a subdir " /tmp/blah/1 "
$ cd $destdir && cd 1 && cd .. && rm -rv 1/* |tee /tmp/rm.txt

- REALLY safe rm, with find:
# find multiple (known) names of files > 40 days old in a given directory and delete them
bkpath="/mnt/bkpdrive"
pathh="$bkpath/work/bkpsys-laptop-p2400-thinkpad-xubuntu-14-04-LTS--64--sda7"
cd $pathh && \
        find $pathh/* \( -name "bkp*gz" -o -name "bkp*bz2" -name "bkp*lzop" -o -name "flist*" \) -type f -mtime +40 -exec /bin/rm -v {} \;

--If you need to delete multiple levels of subdirectories, or dotfiles -- do it interactively, and use MC.

User Journal

Journal Journal: Biology Help Desk: Volume Seven 19

Hey, remember these? I do. Vaguely. But vaguely isn't good enough, so here's another one. Since I've discovered that the secret to understanding machine learning problems well enough to implement them is mostly a matter of strategically procrastinating, it seems only fair that I should run one of these. So before cross-entropy actually starts making sense to me, what would you like to know?

User Journal

Journal Journal: Biology Help Desk: Volume 6 36

It's probably a bad idea to run one of these while I'm slogging through the project bulge of my last semester of undergrad, but let's take a stab at it anyway. Have at it! Clever trolls welcome!

User Journal

Journal Journal: Biology Help Desk: Volume 2n+1, n=2 34

I've gotten a couple of requests now for another one of these, so here one is. Please, bring to me your curiosities and questions about the strange and mysterious biological sciences. I can probably answer (or research answers to) most questions. (Also, if you keep missing these, I've decided I'm going to vaguely hint at them in mysterious ways on Twitter (@rhet0rica) from now on. I guess even Twitter has to have some utility.)

User Journal

Journal Journal: Biology Help Desk: Volume 4 22

Bring me your curiosity! As before, if this expires and you want to ask a question, just slip me a comment (or an e-mail) and I'll put up another one of these. No question is too trivial; no thinly-veiled troll too transparent! (I'm going to regret saying that, I'm sure.)

User Journal

Journal Journal: Biology Help Desk: Volume 3 10

I've received another request for one of these, so I figured that I would put it up. Since they only last about two weeks, please feel free to track me down and ask for a new one if you ever have a question!

User Journal

Journal Journal: Biology Help Desk Again 25

I'm really not sure how long these last as comment-on-able, so here's another one. I'll try to be a little more vigilant in making sure there's always a journal open for asking questions.

A little more elucidation: I'm in the fourth year of a combined bioinformatics/medical informatics degree. Most of my semester is from the CS curriculum, but I have gotten through biochemistry, genetics, molecular biology, physiology, and organic chemistry courses. (Mostly in the presence of pre-med students.) I've also taken genomics-specific courses and worked in a molecular biology lab studying C. elegans, and a medical lab studying Autism.

So ask away!

User Journal

Journal Journal: Current mood: Pensive -- 2011 1

" See the man with the lonely eyes
Oh Take his hand, you'll be surprised " -- Supertramp

http://www.youtube.com/watch?v=hGI6qQHFlPk

Give a little bit
  Give a little bit of your love to me
  Give a little bit
  I'll give a little bit of my love to you
  There's so much that we need to share
  Send a smile and show you care

  I'll give a little bit
  I'll give a little bit of my love to you
  So give a little bit
  Give a little bit of your time to me

  See the man with the lonely eyes
  Take his hand, you'll be surprised

  Give a little bit
  Give a little bit of your love to me
  I'll give a little bit of my love for you
  Now's the time that we need to share
  So find yourself, we're on our way back home

  Going home
  Don't you need to feel at home?
  Oh yeah, we gotta sing

=======

--Pensive, but trying to stay Positive
/ Looking for work

User Journal

Journal Journal: Biology Help Desk 4

Thanks to the power of my silly sig, I've been getting a lot of biology questions lately. Most of these are fun to answer, but occasionally they pop up in totally inappropriate threads just out of the blue. Since Slashdot supports commenting on journal posts, it seemed like the best thing to do would be to make one and encourage people to ask here instead. So do that!

And I really will phrase things as car and computer analogies when possible. Although computer analogies are way more common.

For clarification, I've taken physiology and genetics courses, and can answer most geeky things about the human body and fundamental biology. I don't know much about pharmacology or ecology (because it is very, very, dry), and I am not a doctor. But I'll try to answer those questions, too.

User Journal

Journal Journal: Saving Graces - 2010

So a couple of weeks ago I was truly ready to call it quits, and start learning how to live with being single again for a while.

Sometimes, God steps in where you weren't expecting, and really does something to save your ass.

Things are better now. She's doing better than I am, these days. We're even looking at going to a different church.

Such fierce loyalty, such commitment... The full-on realizations, soul-searching, and changes on her side lately have been incredible. It wasn't what I was really expecting, that night -- but having gone through something similar myself a while ago, it's definitely something I value - and respect.

All I can do is remember to be humble, and thankful, and try to be a better (and Godly) man, for both our sakes... she deserves nothing less.

User Journal

Journal Journal: This is Spring? - 2010

My house is cold, even though I have had the heat on for about a week and a half... This is the coldest d--n Spring in memory.

Announcements

Journal Journal: I has a sad - 2010 1

--Last night, I said something that hurt somebody I really care about.

--Needless to say I did not get any sleep, and had a horrible day wracked with guilt and shame - wishing I could take it back and make things OK again... We were actually having a pretty good time until I caught Teh Stupid.

--Looking back on it, I am appalled not just at my own words, but also the way I said them. I thought I was "being honest", but I was really being insensitive (and blinded to it by my own callousness.)

--I kept checking my phone for texts that never came, and feeling low. Spent most of the day thinking about it with a big pain in my heart, and keeping the tears back.

- I couldn't get this song out of my head all day:
http://www.youtube.com/watch?v=xoW3bqnr7tw

--She is one of the best things in my life, and I acted the fool.

--Honey, Six -- if you're reading this, please forgive me - I really am sorry.

Slashdot Top Deals

What is worth doing is worth the trouble of asking somebody to do.

Working...