Here's how I ran my PhD simulations on EC2:
- The AMI downloads a manifest file at startup.
- The manifest has one record per line, two fields per record: the s3 URL of a
- The AMI then runs a shell script (/etc/run.sh) that's been put there by a manifest entry
Shell scripts upload new files to s3 (e.g.,
Other shell scripts stopped/started experiments on these VMs.
Other shell scripts shut down the VMs when I'm done.
The scripts did little more than scan the appropriate machine list from the ec2 tools and ssh into them with a specific command.
At the end, I had some of the experiment-specific scripts quickly have git clone/pull in files I was changing quickly per experiment.
All of it worked really well for me. Nothing fancier than the ec2 command-line tools, bash, ssh, & git necessary.