Skip to content

All In Tech

Providing IT Tutorials, Scripts, and Much More!!

Archive

Category: Operating Systems

Distributive computing has been around for a long time and there are many different ways of doing it. Today i will be looking at the Message Passing Interface (MPI) standard and will be using the MPICH2 sources provided from http://www.mcs.anl.gov/research/projects/mpich2/

On such a cluster you can run any application that has been designed for MPI, for some benchmarks i ran John The Ripper. (you can get the MPI version here: http://www.bindshell.net/tools/johntheripper/john-1.7.3.1-all-2-mpi8.tar.gz)

Results of the benchmark

No MPI:

Benchmarking: Traditional DES [128/128 BS SSE2]… DONE
Many salts: 1986K c/s real, 1982K c/s virtual

MPI (4 cores over 2 machines)

Benchmarking: Traditional DES [128/128 BS SSE2]… DONE
Many salts: 5151K c/s real, 5151K c/s virtual

As you can see its about a 3 fold improvement.

I have decided to go with Ubuntu for the base system as I’m most familiar with this distro.

Configure the Head Node:

Download the necessary packages

sudo apt-get install nfs-kernel-server openssh-server build-essential

Create the share for NFS

sudo mkdir /mirror
sudo mkdir /mirror/mpiu

run the following to add “/mirror *(rw,sync)” to /etc/exports

echo /mirror *(rw,sync) >> /etc/exports

restart NFS

sudo /etc/init.d/nfs-kernel-server restart

Create a user for the cluster

sudo useradd -d /mirror/mpiu
sudo passwd mpiu
su mpiu
sudo chown mpiu /mirror

Now we setup SSH

ssh-keygen -t dsa

*Don’t set a passphrase

Add the key to Authoirzed keys

cd ~/.ssh

cat id_dsa.pub >> authorized_keys

Download MPICH2

wget http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/1.0.7/mpich2-1.0.7.tar.gz

mkdir /mirror/mpich2

tar -xvf mpich2-1.0.7.tar.gz

cd mpich2-1.0.7

./configure --prefix=/mirror/mpich2

make

sudo make install

Now add the following lines to /mirror/mpiu/.bashrc

export PATH=/mirror/mpich2/bin:$PATH

export PATH

LD_LIBRARY_PATH="/mirror/mpich2/lib:$LD_LIBRARY_PATH"

export LD_LIBRARY_PATH

Now edit /etc/environment and add the following

/mirror/mpich2/bin

Add the hostnames of the machines which are in the cluster into ~/mpd.hosts (currently there will be none if this is your server)

And then run the following commands

 echo secretword=password >> ~/.mpd.conf
sudo chmod 600 ~/.mpd.conf

And there you go, a MPI server running on ubuntu 8.04

A Ubuntu MPI Cluster (PART 2 – client setup) (coming soon!)

7 people like this post.

If your like me and start up a lot of applications that do not have shortcuts in your start menu or on your desktop you may find this tip useful.

In both Vista & XP you can configure Application Paths. Lets say as an example that I want to start notepad.exe by just typing in note.exe or even npd.exe I can edit the registry to make it allow this. This also is handy if you have some programs that you use often but are not in your Start Menu or Desktop such as PuTTy.

What you need to do is open up regedit with Administrative privileges. The location you need to navigate to is:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths

If I highlight App Paths in the left pane and right click in the right pane and select New -> Key I can type in what I want to type to run the application. In this case I’ll use npd.exe. Once the key is created all that is needed is to edit the Default value with the location and name of the exe, so c:\windows\notepad.exe. Now whenever I type npd.exe into the search box on Vista or the Run box on XP it will bring up Notepad for me.

Warning: Before changing any aspect of your registry it is a good idea to do a full backup of your registry in case anything goes wrong.

A quick update: An open source app has been created to do this for you http://sourceforge.net/projects/apppaths/

Be the first to like.

Having recently upgraded to Vista I decided it was high time to look at separating my user and program files from my Operating System. So I shifted around some files and formatted my D: drive in preparation for moving the files. I played with a few different methods for moving the user files including updating their Profile Path using the Users & Groups section in Computer Management but despite it working I felt this wasn’t what I was looking for so started looking at using Junctions to redirect my files instead. Ultimately through the use of a DOS Prompt and junctions I’ve moved all my user & program files off to my D: Drive. Here are the steps and information involved.

Windows Vista already uses several junction points which need to be replaced with our own ones. I have listed them here along with where they point to:

In the Root C:
Documents & Settings -> Users

In each User directory:
Application Data -> AppData\Roaming
Cookies -> AppData\Roaming\Microsoft\Windows\Cookies
Local Settings -> AppData\Local
My Documents -> Documents
NetHood -> AppData\Roaming\Microsoft\Windows\Network Shortcuts
PrintHood -> AppData\Roaming\Microsoft\Windows\Printer Shortcuts
Recent -> AppData\Roaming\Microsoft\Windows\Recent
SendTo -> AppData\Roaming\Microsoft\Windows\SendTo
Start Menu -> AppData\Roaming\Microsoft\Windows\Start Menu
Templates -> AppData\Roaming\Microsoft\Windows\Templates

All users have the same Junctions in their user directories, including Default. There are two other Junctions created in the Users directory, these are:
All Users -> ProgramData
Default User -> Default

ProgramData is located is a directory in root C:.

As you can see there are numerous links to recreate. The reason why they are used is to provide backward compatibility with software designed explicitly for Windows XP that may have hardcoded the Documents & Settings directory or other directories within each User Profile. As a result you could try without them and any programs explicitly designed for Windows Vista should work perfectly as well as the majority of Windows XP programs but you may still run into issues. As its fairly simple to recreate the links its worthwhile doing.

The program that is used to create a link is called mklink and it is included with Windows Vista so no extra downloads are required for it.

Usage:Creates a symbolic link.

MKLINK [[/D] | [/H] | [/J]] Link Target

/D Creates a directory symbolic link. Default is a file
symbolic link.
/H Creates a hard link instead of a symbolic link.
/J Creates a Directory Junction.
Link specifies the new symbolic link name.

Target specifies the path (relative or absolute) that the new link refers to.

The majority of the folders we will be linking we will be using Directory Junction’s. This is roughly equivalent to a Hard Link. The usage is pretty easy to follow and if you have any trouble you’ll get a better idea of it in the nitty gritty of moving the files around.

Okay before rebooting what we’ll do is create a batch file which does everything for us. This is a lot easier than typing it all in at the command prompt and allows us to check easily for any missing parts. So open notepad.exe and we’ll get on with it. Just remember to replace any drive letters and other path issues that need to be different for your system.

The first part of the file is just creating the destination directories and then copying the files:

mkdir D:\Users
robocopy /XJ /MIR "C:\Users" "D:\Users"
mkdir D:\ProgramData
robocopy /XJ /MIR "C:\ProgramData" "D:\ProgramData"

If you want to move your Program Files directory as well then include this bit:
mkdir "D:\Program Files"
robocopy /XJ /MIR "C:\Program Files" "D:\Program Files"

Next we delete the original files off the C: Drive


rmdir /S /Q "C:\Users"
rmdir /S /Q "C:\Program Files"
rmdir /S /Q "C:\ProgramData"
rmdir /S /Q "C:\Documents & Settings"

Next we will recreate the user links:
mklink /J "D:\Users\UserName\Application Data" "D:\Users\UserName\AppData\Roaming"
mklink /J "D:\Users\UserName\Cookies" "D:\Users\UserName\AppData\Roaming\Microsoft\Windows\Cookies"
mklink /J "D:\Users\UserName\Local Settings" "D:\Users\UserName\AppData\Local"
mklink /J "D:\Users\UserName\My Documents" "D:\Users\UserName\Documents"
mklink /J "D:\Users\UserName\NetHood" "D:\Users\UserName\AppData\Roaming\Microsoft\Windows\Network Shortcuts"
mklink /J "D:\Users\UserName\PrintHood" "D:\Users\UserName\AppData\Roaming\Microsoft\Windows\Printer Shortcuts"
mklink /J "D:\Users\UserName\Recent" "D:\Users\UserName\AppData\Roaming\Microsoft\Windows\Recent"
mklink /J "D:\Users\UserName\SendTo" "D:\Users\UserName\AppData\Roaming\Microsoft\Windows\SendTo"
mklink /J "D:\Users\UserName\Start Menu" "D:\Users\UserName\AppData\Roaming\Microsoft\Windows\Start Menu"
mklink /J "D:\Users\UserName\Templates" "D:\Users\UserName\AppData\Roaming\Microsoft\Windows\Templates"
mklink /J "D:\Users\UserName\AppData\Local\Application Data" "D:\Users\UserName\AppData\Local"
mklink /J "D:\Users\UserName\AppData\Local\History" "D:\Users\UserName\AppData\Local\Microsoft\Windows\History"
mklink /J "D:\Users\UserName\AppData\Local\Temporary Internet Files" D:\Users\UserName\AppData\Local\Microsoft\Windows\Temporary Internet Files"
mklink /J "D:\Users\UserName\Documents\My Music" "D:\Users\UserName\Music"
mklink /J "D:\Users\UserName\Documents\My Pictures" "D:\Users\UserName\Pictures"
mklink /J "D:\Users\UserName\Documents\My Videos" "D:\Users\UserName\Videos"

Replace Username with whatever the username you are working on is, just use Find & Replace in Notepad to fix it up. You need to use that piece of code for all users that you have added to your system. The code for Default is as follows:

mklink /J "D:\Users\Default\Application Data" "D:\Users\Default\AppData\Roaming"
mklink /J "D:\Users\Default\Cookies" "D:\Users\Default\AppData\Roaming\Microsoft\Windows\Cookies"
mklink /J "D:\Users\Default\Local Settings" "D:\Users\Default\AppData\Local"
mklink /J "D:\Users\Default\My Documents" "D:\Users\Default\Documents"
mklink /J "D:\Users\Default\NetHood" "D:\Users\Default\AppData\Roaming\Microsoft\Windows\Network Shortcuts"
mklink /J "D:\Users\Default\PrintHood" "D:\Users\Default\AppData\Roaming\Microsoft\Windows\Printer Shortcuts"
mklink /J "D:\Users\Default\Recent" "D:\Users\Default\AppData\Roaming\Microsoft\Windows\Recent"
mklink /J "D:\Users\Default\SendTo" "D:\Users\Default\AppData\Roaming\Microsoft\Windows\SendTo"
mklink /J "D:\Users\Default\Start Menu" "D:\Users\Default\AppData\Roaming\Microsoft\Windows\Start Menu"
mklink /J "D:\Users\Default\Templates" "D:\Users\Default\AppData\Roaming\Microsoft\Windows\Templates"
mklink /J "D:\Users\Default\AppData\Local\Application Data" "D:\Users\Default\AppData\Local"
mklink /J "D:\Users\Default\AppData\Local\History" "D:\Users\Default\AppData\Local\Microsoft\Windows\History"
mklink /J "D:\Users\Default\AppData\Local\Temporary Internet Files" D:\Users\Default\AppData\Local\Microsoft\Windows\Temporary Internet Files"
mklink /J "D:\Users\Default\Documents\My Music" "D:\Users\Default\Music"
mklink /J "D:\Users\Default\Documents\My Pictures" "D:\Users\Default\Pictures"
mklink /J "D:\Users\Default\Documents\My Videos" "D:\Users\Default\Videos"

We also have the Public Directory to fix up links for:

mklink /J "D:\Users\Public\Documents\My Music" "D:\Users\Public\Music"
mklink /J "D:\Users\Public\Documents\My Pictures" "D:\Users\Public\Pictures"
mklink /J "D:\Users\Public\Documents\My Videos" "D:\Users\Public\Videos"

Now in the \Users directory we have some links to create as well:

mklink /D "D:\Users\All Users" "D:\ProgramData"
mklink /J "D:\Users\Default User" "D:\Users\Default"

And here are the other links for the root C:

mklink /J "C:\Documents and Settings" "D:\Users"
mklink /J "C:\Users" "D:\Users"
mklink /J "C:\ProgramData" "D:\ProgramData"

To include Program Files add this piece of code in as well:

mklink /J "C:\Program Files" "D:\Program Files"

That is the whole batch file completed. Now before you head off to run it make sure you double & triple check your paths, if you have made a mistake it could result in the loss of your personal data and/or program files. As an example of something that occurred while I was testing this. I had Vista already running in a VM so I booted that to test it, created a new drive which was assigned the driveletter E: which was fine so I modified my batch file copy all files to the E drive. Unfortunately unbeknowest to me when I restarted to get into the Command Prompt it reassigned my new drive to drive D so as a result it tried to copy all my user files to D: which naturally failed, the rmdir /Q /S makes it remove the relevant directories without any user interaction so before I knew it all the files were gone meaning I had to reinstall Windows Vista before running another test, with destination drive set to D:.

To run this anyway you just need to boot from your Windows Vista CD. Select to Repair a installation, select your Windows Vista installation and finally select Command Prompt. Then just navigate to wherever you stored it and run it by typing in the name.

There are a few issues with this method. One of the biggest is the way Windows Vista handles links still. Just like Windows XP it includes them in drive space usage checks. This can be quite annoying as ideally you want to shrink down the Windows Vista drive to a much smaller space since it doesn’t have your user files anymore and only Windows Vista files and Program Files(if you didn’t move them as well). This means that it will keep bringing up the message warning you of low disk space. Unfortunately with Windows Vista’s low disk space warnings there doesn’t appear to be a way to prevent it from checking a single drive. Rather its a all or nothing approach. To disable it you can follow the instructions here. Before doing so it may be a good idea to check this one out the warnings here.

Stay tuned for a proper script to ask questions and do it all for you on the fly.

A word of caution regarding moving your Program Files and ProgramData directories. Windows update will not function properly with them on a separate drive despite the use of hard junction points to link them to the new locations. Whilst there is a fix it is a time consuming one and requires you to make changes for every update that it can be applied to since it can only be applied to updates that require you to restart your computer. For more information you can search for the error code 0×80070011.

1 person likes this post.