...
Linux Computing Cluster
Six HP dual QuadTen SuperMicro 24-core Xeon Z800 U628 2U rackmount workstations (two with Intel® Xeon® E5530 2.40 GHz 8MB/1066 QC CPU, four with Intel® Xeon® E5630 2.53 GHz 12MB/1066 QC CPU) each with 24GB RAM, 1.5 with Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, and v4 @ 2.4oGHz 35MB Cache,2400MT/s) each with 384GB RAM, 20 TB of local hard disk storage, NVidia Quadro FX3800 graphics, are clustered with Sun Grid Engine as a 48240-core IBIC managed condominium computing cluster.
Two additional HP dual Quad-core Xeon Z800 workstations (Intel® Xeon® E5630 2.53 GHz 12MB/1066 QC CPU) each with 24GB RAM, 1.5 TB of local hard disk storage, NVidia Quadro FX3800 graphics,
these workstations are dedicated to individual developers and project groups.
One final HP dual Quad-core Xeon Z800 workstation (Intel® Xeon® E5630 2.53 GHz 12MB/1066 QC CPU) with 72GB RAM, 9.0 TB hard disk space, NVidia Quadro FX3800 graphics, is dedicated to , and is equipped for internal development and memory-intensive processing. , and capable of parallel pipeline on large datasets.
All resources are linked with 1 10 Gb Ethernet, with 10 Gb bandwidth connectivity to two Network Attached Storage (NAS) systems through a dedicated 24 port DLink switch.
...
The computing cluster workstations support IBIC developers and research scientists developing software and designing/scripting analysis pipelines.
Four Dell PowerEdge servers (each with dual Quad-core Intel® Xeon® E5430 2.66 GHz 12MB/1333) are available for shared projects with the Chaovalitwongse lab.
In addition, a 36-core (AMD Opteron Processor 4180) “condominium cluster” running Sun Grid Engine, administered by IBIC for the Seattle Longitudinal Study, is available for shared use.
Desktop Workstations
Seven Dell Precision T1500 workstations Quad-core Intel® Core™ i5-760 (2.80 GHz 8MB with 8GB RAM, 1.5 TB of local hard disk storage, NVIDIA Quadro FX380 graphics), fourteen Dell Precision T1600 workstations Quad-core Intel® Xeon® E3-1225 workstations (3.10 GHz, 6M with 8GB RAMEight Dell Precision T7820 Tower Workstation Intel® Xeon® Bronze 3204 (8.25 MB cache, 6 cores, 6 threads, up to 1.90 GHz, 85 W) with 16GB DDR4 2933MHz RDIMM ECC Memory, 1.5 TB of local hard disk storage, NVIDIA Quadro NVS300), and one Dell Precision T1650 workstation Quad-Core Intel® Xeon® E3-1225 v2 (3.2GHz, 8M, with 8GB RAM, 1.0 TB of local hard disk storage, NVIDIA Quadro 410), Quadro P620 graphics), all with dual 24 inch LCD monitors, are used at the desktop by individual researchers developing scripts and analysis pipelines.
...
“ivan” Workstation (Interoperable and Virtualized Analysis of Neuroimages)
One Dell Precision T1600 Quad-core Intel® Xeon® E3-1225 workstation (3.10 GHz, 6M with 8GB RAMPrecision T7820 Tower Workstation Intel® Xeon® Bronze 3204 (8.25 MB cache, 6 cores, 6 threads, up to 1.90 GHz, 85 W) with 16GB DDR4 2933MHz RDIMM ECC Memory, 1.5 TB of local hard disk storage, NVIDIA Quadro P620 graphics) , is available for non-core affiliates of IBIC. It is the preferred platform for converting Enhanced DICOM to NIFTI, and runs the common IBIC workstation image processing application software.
Central Virtualized EnvironmentStorage
IBIC has a virtual desktop interface (VDI) server system implementing virtualization of an interoperable image processing environment (IVAN: Interoperable and Virtualized Analysis for Neuroimaging). The hardware is comprised of three Sun Fire X4170 (two with 8 cores / 8GB RAM, one with 16 cores and 24 GB RAM), and three Sun Fire X4270 servers (each 16 cores with 48 GB RAM).
Two of the servers run Ubuntu Linux 10.04 and the common IBIC workstation image processing application software. Users are authenticated via a web browser Session Manager and presented with the Ulteo Open Virtual Desktop (http://www.ulteo.com). The system is authenticated with UW netid using Shibboleth. UW netids are sponsored by IBIC for users external to the University of Washington.
Virtualization reduces load on IT personnel, ensures efficient use of computing resources, and facilitates dissemination. The overriding advantage is the meta-issue of standardization and tracking of the processing environment, the related issues of stability of the build across investigators, labs, and contexts, and "version control" of the environment.
Storage
PACSsoft, a DICOM image archive, and Bioscribe, an overarching research metadata database, manage image data sets related to individual participants and to specified research studies as well as any text-based and graphical data (such as medical histories, medication information, physiological measurements, etc.) associated with the study. Bioscribe allows complete
XNAT is an open-source eXtensible Neuroimaging Archive Toolkit developed by the NRG at Washington University in Saint Louis. This imaging informatics platform provides complete storage, integrity, data security, and HIPAA compliance of research data from the various projects. This powerful system is designed to support multicenter research projects where the central coordinating site would be located at UW. The system has a 4 TB capacity.
XNAT is an open-source eXtensible Neuroimaging Archive Toolkit developed by the NRG at Washington University in Saint Louis. This imaging informatics platform provides the functionality of the combination PACSsoft and Bioscribe systems and adds hundreds research projects, and support for pipeline processing. In addition XNAT has a large and growing user community. DISC XNAT is hosted on an HP ProLiant DL 380p Generation 8 server with 20TB of storage (12.7 TB RAID 6). The XNAT system is the successor to PACSsoft / Bioscribe and is being used for all ongoing and new studies.
SuperMicro Storeform R514v7 server (2 x 8-core Intel Xeon Scalable Silver 4208, 2.1GHz, 2400 MT/s 85W) with RAID-6 40TB of usable storage.
NAS: High-performance network attached storage is provided by two Sun model 7310 NAS controllers each with a J4400 SAS ARRAY of 24 X1TB x 1TB 7200RPM SATA HDDs. The disk arrays are configured as Single Parity Mirrored. This NAS system has privileged bandwidth to the compute cluster (10Gbit).
NAS_II: IBIC also also a secondary has an extensible network attached storage system (Thecus N16000) providing 16 x 3TB in a RAID6 provided by a SuperMicro Rackform U627.v7 (Xeon Gold 6128)controller with 4 Storform DS4-44 4U JBOD Chassis, SAS ARRAY of 36 x 12TB 3.0 12.0Gb/s 7200RPM HDDs (total of 176 slots available), providing up to 1.2 PB storage capacity in a ZFS RAIDz2 configuration. This system has 10 20 Gb bandwidth to the IBIC compute clusters and the UW network.
IBIC also has access to a 50 Tb capacity extensible central data repository in the Department of Radiology, University District Building, funded by an NIH S10 grant (PI: Minoshima).
PACSsoft and Bioscribe,
The DISC-XNAT, the Confluence IBIC wiki server, and the MR lab scheduling tool are backed up nightly using Tivoli Storage Manager via a campuswide campus-wide service.
Software resources
...
Web presence
The IBIC web site is is www.ibic.washington.edu. IBIC-wide web resources are collectively managed with enterprise wiki software (Atlassian Confluence). The URL is is www.ibic.washington.edu/wiki. Confluence was chosen for a what-you-see-is-what-you-get quality that minimizes barriers to the user.
...
IBIC has a wide range of image processing software platforms for functional, diffusion and structural image processing, which are maintained in a standardized Ubuntu Debian 10.04 10 operating environment disc image across all IBIC desktop workstations.
...
The IBIC cluster nodes are running the newest version of image processing software on a standardized Ubuntu 12Debian 10.04 10 operating environment disk image (“Evolution”).
The image processing software tools include FSL, SPM8, FreeSurfer, BioImage Suite, AIR, Brains3, tal_programs, MRIcron, ANTS, R, and many others, as well as general purpose scientific analysis software: MATLAB, and Python libraries. Custom software is also described below.
...