Hi All, ################################ # Action ################################# Please could you try to find the computational needs (or even just "would like"s) of projects that your group would put forward. These include: - Computational load (how many CPUs for how long ... yes I know that this is hard) - Storage: How much data would you like the students to be able to access from their compute nodes? - Software (Matlab, mathematica, root,...) - unusual (non-windows) environments So Adrian, Mike and I met this afternoon to discuss this and there are a number of different solutions for different scales of problems. I think that we can stitch these together into a very good solution for a wide variety of projects. So what we discussed were: - The condor pool is still there however the interface is through a web page. I am going to try this out over the next few days and talk to ICT about how we could best use it for different projects in physics. - Storage. With the possibility of people proposing "Big Data" style projects we thought that there might be a case for buying some physics specific storage for such projects and making it visible from the condor nodes. We discussed scales of 50 to 100 TB sort of scale. P{roject supervisors would be able to give access to their students only if they so wished. - Non- windows environments. There are redhat VMs installed on the machines down stairs, but for anything more specific Imperial has a contract with MS azure and it we can set up projects requiring specific environments in VMs machines on this cloud. I will be given access over the next couple of days and will have a look around. - If projects require MPI computing we can create accounts on the college HPC for them. Clearly some of these things are (almost) free and others are not. So it is important to find out what we need and what we would use if we had it. best, david On 14/02/15 18:10, David Colling wrote:

Hi Julia,

(adding the computing list and Kenny to aid communications).

Yes, this is something that has come from time to time since I have been involved in the computing committee. At one stage we almost bought a dedicated (small) set of servers for this purpose but then decided not to for some (good) reason that I cannot remember. Looking at the machines downstairs I can see that they are actually running condor (which makes them a useful resource). However I think that this is not enough by itself - obviously no documentation is a problem but there is more than that. I can see situations where access to considerables amount of data might be useful etc or to specific environments with specifically licensed software. I can see a number of ways of dealing with this (although they may require a modest outlay - hence Kenny's inclusion ;-) ).

I think that the best approach might be to organise a meeting where we can decide this once and for and provide a good working environment. I will set up a doodle poll (although probably not until Monday).

Adrian, perhaps you and could meet up earlier to discuss the options?

Best, david

On 13/02/15 14:39, Julia Sedgbeer wrote:

...

Hi Dave, Ray,

I'm emailing you in your capacity as members of the departmental computing ctte.

Tim has asked about computing facilities for projects (see email below). This comes up every year; some projects use research group facilities, some depend on students having their own laptops. Is there a way to make better use of the computing power of the teaching cluster for projects, e.g. a batch system (assuming that there currently isn't one) or similar?

Cheers, Julia

---------- Forwarded message ---------- Date: Fri, 13 Feb 2015 08:23:28 +0000 From: "Evans, Tim S" <t.evans@imperial.ac.uk> To: "Sedgbeer, Julia K" <j.sedgbeer@imperial.ac.uk> Subject: RE: MSci Proposals Dear Julia (cc'ing the departmental list to help communications),

...

Dear Julia,

Do you know if there is any specific dept/college support for student project computing needs? Or who to ask about this? Some of my Erasmus Bsc students may need some extra power. There was a system I made a note of but that doesn't seem to exist (see below).

Thanks

Tim

Old system:- Every year students ask about how they can use cluster computers for long running experiments/projects and we've been working to provide a solution to this. Condor is an open source package which is designed to queue up jobs and submit them to computers when they're not busy; it's been used on Linux by DoC for some time but we now have it available for Windows users. Any student who wants to run code which they've compiled or a .m file from Matlab can submit a job via Condor using the website https://icwincondor1.cc.ic.ac.uk/condor.A couple of key points are that the job must run non-interactively and must not try to access the user's H: drive (or any other specific location - the code should assume that all files are in the current directory)

_______________________________________________ Physics-Departmental-Computing mailing list Physics-Departmental-Computing@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/physics-departmental-computing