Minutes of WFCAM Science Archive meeting: 11 July 2003 ------------------------------------------------------ ------------------------------------------------------ Present: IAB, NCH, MAR, PMW Apologies: JMS, RGM, ETWS, HMG, AL Actions discharged: ------------------- ACTION: IAB to check the SSA HTM code into the CVS under SSA/src/scos2htm/... ACTION: PMW to communicate SRIF1 wish list to Arthur Trew. ACTION: IAB to recheck the scos2htm codes into the appropriate module in the CVS repository, and NCH to move the WSA ones into the attic. ACTION: IAB to check CU codes into the CVS (under WSA!) as they are being developed. ACTION: NCH to talk to Guy Rixon at next week's NeSC meeting re: gridFTP and further transfer tests. - all done and discharged (thanks to all concerned!) Actions partly discharged but continuing: ----------------------------------------- ACTION: PMW to draft an action plan in the light of the review panel report. ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. - continuing; thanks for progressing these. Actions carried forward from 27/06/03 meeting: ---------------------------------------------- ACTION: PMW to liaise with AL to make sure that Compute Support are made aware, from the highest "official" level, of our requirements, re. 100 Gbyte/day transfer from CASU, and to impress on them that the 1 Gbit/s FW should be their highest priority. - CONTINUES Specific points and new actions: -------------------------------- Project management: PMW reported the outcome of the quarterly VDFS managment group meeting at CASU on the 4th July. The main issues were: i) table of deliverables to be explicitly included in the quarterly report; ii) include a contingency in the forward plan in case of unexpected high priority work (eg. NCH & PMW working on the VEGA/VDFS continuation e-science bid); iii) revise the forward quarterly plan to breakdown large tasks into smaller units so it is easier to see progress with the 10/50/100% completion granularity; 4) keep documentation up-to-date. NCH reported that a second issue of the ICD has been produced that includes new information on FITS files; this will be distributed once CASU have had a chance to comment on the changes. The team agreed that the philosophy of keeping the docs up-to-date was a good one, but that it was neceassarily a low priority task and care must be taken to not waste time making a large number of small changes to the documentation; rather changes should be accumulated and incorporated together at appropriate times. NCH is keeping a master doc set with all required changes noted for this purpose. Hardware: HMG reported (via NCH) that the last batch of hardware from the first Eclipse order was expected next week, at which point the first of the catalogue servers can be set up. PMW reported that the SRIF1 underspend opportunity has resulted in all of the items on our wish list (except UPS units) having been approved by the University and we have placed orders for additional storage and a server (note that this money had to be spent immediately, but the additional storage be invaluable for the purposes of setting up the prototype server for the SSA since staging/mirroring/indexing storage overheads have likely been underestimated by NCH). There has been a request from MJI at CASU to test out LTO 1 & 2 compatibility using the ATC's recently acquired summit hardware (LTO-1) and our own system (LTO-2) - this should be tested when the systems are set up. ACTION: NCH to ask AJV/DWL about when/where the ATC/JAC LTO-1s are likely to be set up. Software: IAB reported: "Python scripts implementing cu2, cu3, cu4, and cu13 are in the CVS. I have been testing the CASU image stacking software and they seem to work well. I have written a Python binding to the stacking routine so that it is possible to call this directly from the script without using a "system" fork. I am currently weighing up the pros and cons of the two approaches." NCH noted the requirements of some curation tasks in terms of client/server interactions in the light of looking at the likely heavy traffic for tasks such as source merging in the archive - it is necessary to revisit the software architecture to allow for minimising network traffic between the Unix client side (where all curation tasks are to be launched from) and the server side. ACTION: NCH & IAB to meet offline to resolve unix-client / windows- server connectivity. MAR reported: "This week I've been working on the form driven interface to the SSA. I've also been looking at some of the stuff that cropped up during the chat with Jim Gray namely the Skyserver web access. They import their web log into a SQL database table on a daily basis, they also record the runtime and no of rows return of every query. I can see how most of this is achieved but it's not clear that the personal skyserver contains all the relevant code (not that it would all be the same as they use an IIS/windows/asp server). Jim also mentioned that all their web queries aree run in a wrapper which is basically a stored procedure that checks a few things in the SQL statement mainly that there is a SELECT clause and no funny calls. He admitted it was a fairly basic attempt to prevent attack and or newbie mistakes but it looks easy to implement so will look into it further. We also chatted about limiting data returned and estimating query runtimes, this is still an area for concern and not really resolved though they've had no complaints about their 1000 row, 90 sec limit but they also have unlimited access for "special" users." Networking: NCH reported a useful chat with Guy Rixon concerning the way forward with testing gridFTP with 1 Gb/s connectivity at the end stations. Guy has communicated details of the server and port at the Cambridge end that has the required hardware & software (this has been communicated to ETWS) but has requested that we hold off any large scale tests until after the IAU GA in Sydney at which AstroGrid demos are being done using the same server. The plan is do the next tests once our own WSA Gb/s Unix web server is set up - this will be done over the next couple of weeks so that tests can be set up immediately that the Cambridge end server is free of IAU demo traffic. Miscellaneous: ACTION: NCH to type up and circulate these minutes. DONM: 10am, Friday 18th July, plate library.