The University of Southampton is the latest university to join the UK National Grid Service with affiliate status.  The Microsoft Institute for High Performance Computing at the University will be providing access to the Microsoft Compute Cluster Service through a Globus gateway, the first instance of a Windows cluster being incorporated into a production level Grid.  Prof. Simon Cox, Director of the Institute said “This is an exciting opportunity to offer, for the first time, Microsoft Windows Compute Cluster Server on the National Grid Service, where it will interoperate with other resources”.

 

A large proportion of the UK research community primarily use Windows based systems or are more familiar with Windows than other operating systems.  The implementation of the Globus Gateway will allow the NGS to be accessed by a much larger number of researchers who will be able to carry out research under a more familiar operating system. 

 

The CCS Globus Gateway provides access to a cluster under the Windows x64 platform using the normal Globus client tools and Grid security mechanism, therefore makes it possible to share the Windows based computational resources with a wider user base in the e-Science community.

 

About the University of Southampton

• The Microsoft Institute for High Performance Computing at the University of Southampton is part of the University's School of Engineering Sciences, which offers expertise in engineering disciplines that underpin every aspect of 21st century life. Evolving from its traditional engineering roots, the School continues to expand into new, exciting research areas. From orthopaedic implant modelling and modern fuel cell technologies, to enhancing satellite broadcast technology and improving the aerodynamics of Formula One racing cars and hydrodynamics of America's Cup yachts, the School is always at the leading edge of engineering research. www.mihpc.net
• The University of Southampton is a leading UK teaching and research institution with a global reputation for leading-edge research and scholarship.  It is one of the UK's top 10 research universities, offering first-rate opportunities and facilities for study and research across a wide range of subjects in humanities, health, science and engineering.  The University has around 20,000 students and over 5000 staff.  Its annual turnover is in the region of £310 million. www.southampton.ac.uk

We released the MOM Management Pack for Windows CCS back in November last year. You can download the pack at http://www.microsoft.com/downloads/details.aspx?FamilyId=53AE834A-7369-4655-82E5-1C2563CEC677&displaylang=en. You can also download the pack guide at http://www.microsoft.com/mom/techinfo/productdoc/default.mspx#E2BAC.

Anybody has been using the pack? Do you have any suggestions, comments, etc? We would love to hear from you  and do the best we can to update the pack to suit your needs.

 

 

In todolist’s “Manage Windows Firewall” wizard, there are two options: Enable Windows Firewall (WFW), and disable WFW. Given all different network topologies and services running on the nodes, doesn’t this control seem to be too simple?

 

CCS philosophy is that firewall configurations should to be secure and work (does not break applications) for most of cases by default. That’s why the wizard doesn’t provide much options to confuse you. Let me explain what CCS does for WFW under the hood.

 

Before network topology is configurated

 

Head node’s WFW is turned on. By default Windows will enable firewall for every network connection at this moment. Head node firewall is configured as such:

• Firewall notifications are disabled
• Remote administration exceptions are enabled
• Port 5969 is opened for client tools to connect to scheduler
• Port 6729 is opened for management services on compute node to talk to management service on the head node
• Port 9892 is opened for compute node’s management service to connect to SDM service on head node
• File and printer service and remote desktop service are enabled

 

If a Compute node is installed using Todolist “automatic add nodes” function,  WFW is enabled during RIS.

 

Regardless how a compute node is set up, once it is added to a cluster, management service on compute node will contact its counterpart on head node and pull out configuration from SDM service. The configuration at this moment is to turn on firewall for the machine (and thus for every connection) with this profile:

• Firewall notifications are disabled
• Remote administration exceptions are enabled
• Port 6729 is opened for head node to talk to compute node’s management service
• remote desktop service are enabled

 

After network topology is configurated

WFW will be disabled on private network and MPI network if they are defined so applications on compute nodes could communicate with each other.

 

If the cluster has only public network, WFW will still be enabled for that network but few more exceptions will be added to the profile:

For head node

• port 5970 will be opened so compute node could talk to scheduler

On a compute node

• Port 1856 will be opened so scheduler could connect to compute node’s node manager
• Port 8677 will be opened for root MPI services (smpd) so MPIExec could contact them
• Smpd.exe in CCP’s bin directory is added as an application exception because it uses dynamic ports

 

After the cluster is configurated

 

CCS will enforce WFW settings to follow rules described above. For example, if an admin disables firewall on one node’s public network interface manually without using todolist firewall wizard, WFW will be enabled automatically again on that interface once CCS finds out about the change; or if port 5969 is removed from head node’s exception list, it will be automatically added. However CCS only manages those networks defined in todolist, for any other network connection, admin could turn firewall on or off and CCS will not care. Similarly, admins could add more exceptions (open new ports, add new authorized applications) and CCS will not override the settings.

 

Of course, admin could has disable WFW using the wizard. Once this option is selected, CCS will disable every node’s WFW at machine level thus firewall is effectively turned off on networks. This is dangerous but could be useful sometimes, especially in a public network only topology.

 

Key take-away

 

1. Before network topology is defined, firewall will be enabled by default on all network connections with very few port opened, applications including CCS tools could fail at this stage. Always define topology before trying to run any application.
2. WFW is disabled on private and MPI network once they are defined, so make sure those network are really internal.
3. CCS internal traffic will be directed to private network as long as it is defined. For MPI traffic, CCS will make sure it uses MPI network if it’s defined, otherwise it will use private network if it exists. So in a topology where there are more than one network connection, serial or MPI applications should just work. If an non-MPI application need to contact other nodes in a cluster, it may fail if it happens to use the public network connection.
4. In a public network only topology, a lot of applications might not work with default configuration. Administrators may need to manually enable those applications on every node’s WFW profile or disable firewall using manage firewall wizard with caution.

In light of a recent blog post by Erez suggesting some QFE's/Hotfixes that you may want to deploy to your cluster and how to do so with Clusrun, I thought I'd chime in with some details on how you can deploy hotfixes directly to your RIS images.

The following method works for almost all Windows hotfixes (including service packs!) and allows you to add Windows patches to:

• Your existing RIS images
• Windows CD's (You'll need to copy the CD to your hard drive, integrate fixes, and then re-burn the CD)
• Network Windows installation shares

This magic can be achieved with the "/integrate" command-line flag.  Simply download and run the executable with the /integrate flag, specifying the location of the image you would like to upgrade.  For example:

WindowsServer2003-KB910481-x64-ENU.exe /integrate:"D:\RemoteInstall\Setup\English\Images\WindowsCCE"

This will go through the image and update all of the necessary binaries.  Any machines you image from then on will start from this base state.

Pretty neat, eh?

 

Compute Cluster Pack (CCP) RIS is great for deployment of compute nodes if the base image contains the necessary drivers.  If not, then you have to manually add the drivers and updates to the base image.  CCP RIS also automatically inserts the CCP node services into the base image which enables a RIS’ed node to join the cluster unattended. 

 

If it appears burdensome to manually configuring the base RIS image with a number of drivers and updates, then consider using Standard RIS instead.  Standard RIS is included as a ‘Windows Component’ within each Windows Server installation.

 

Standard RIS also includes the RIPREP.EXE utility (\\MyRisServer\reminst\admin\amd64\riprep.exe) which enables you to easily capture a ‘Golden Image’.  The general procedure for capturing a fully configured base image is as follows:

 

1. Install the base OS to a target machine.
2. Install all drivers.
3. Install all Windows Updates.
4. Copy any user data files (e.g. to directory c:\MyData).
5. Copy the CCP setup files to c:\CCP.
6. Enable ‘Remote Desktop’ via the ‘My Computer – Properties’ dialog.
7. Disable the Windows Firewall (assuming an isolated cluster LAN).
8. Install and/or configure any other aspects of the server.
9. From the new server, run \\myserver\reminst\admin\amd64\riprep.exe.  This will capture the ‘golden image’. 
10. Deploy the new image to your cluster nodes using a network PXE boot and Standard RIS.
11. Configure the head-node using the Compute Cluster Pack setup program.
12. Configure the compute-nodes using the ‘setupccp.vbs’ script included within the downloads section of the IT Professionals files folder (http://windowshpc.net/files/17/scripting/default.aspx).  This script mimics the unattended 'add node' functionality included with CCP RIS.

Be sure to read the ‘deploy.chm’ file included within the ‘deploy.cab’ file on the second CD of the Windows Server 2003 R2 installation media.  The cab file also includes the ‘SetupManager’ utility which is useful in configuring a fully unattended installation answer file.

 

 

August 1: Today Microsoft announced the general availability of Microsoft Windows Compute Cluster Server 2003, the company’s first product designed specifically for high-performance computing (HPC). With Windows Compute Cluster Server 2003, Microsoft aims to make it easier to create, integrate and operate HPC clusters within organizations, thereby expanding the technology beyond traditional supercomputing centers by bringing the value of computational clusters within reach of more people.

To understand the impact of today’s milestone, PressPass convened a roundtable of customers who have been test driving Microsoft Windows Compute Cluster Server 2003 in demanding applications, including biomedical research and scientific modeling. Providing their insight are:

•	Ron Elber, professor of computer science at Cornell University
•	John Michalakes, senior software engineer at the National Center for Atmospheric Research (NCAR) in Boulder, Colo.
•	Matt Wortman, director of computational biology and IT at the Genome Research Institute, University of Cincinnati More...

FYI...You may have recently received email telling you that GotDotNet deleted the CCS Codegallery. This is true! I just wanted to let you know that this will not impact this community site. I had sent email to all CodeGallery members in early May telling them that the CodeGallery would be deleted as we moved to WindowsHPC.NET (this community).

Regards,

Dennis