HR: 16:15h
AN: S42J-02 INVITED     [PDF]
TI: Large Scale Parallel 3D Simulation of Regional Wave Propagation Using the Earth Simulator
AU: * Furumura, T
EM: furumura@eri.u-tokyo.ac.jp
AF: Earthquake Research Institute, University of Tokyo, 1-1-1 Yayoi, Bunkyo-ku, 113-0032 Japan
AB: This paper presents an efficient parallel code for seismic wave propagation in 3D heterogeneous structures developed for implementation on the Earth Simulator (5120 CPUs, 40 TFLOPS)_@at the JAMSTEC Yokohama Institute, a high-performance vector parallel system suitable for large-scale simulations. The equations of motion for the 3D wavefield are solved using a higher-order (8,16, 32 etc.) staggered-grid finite-difference method (FDM) in the horizontal (x,y) directions and a conventional fourth-order FDM in the vertical (z) direction. Compared to traditional Fourier pseudospectral method (PSM), the higher-order FDM achieves very good performance on vector processors as well as on the latest high-performance microprocessors (Intel Pentium 4, Itanium 2 etc.). The parallel computing is based on partition of the computational domain, with each subregion assigned to a node of the Earth Simulator. Message passing interface (MPI) inter-node communication is employed for data exchange between subregions. Small-scale heterogeneities such as low-velocity sedimentary basins are accounted for in large-scale models by adopting a multi-grid approach that combines a coarse mesh model with embedded finer mesh model. An accurate interpolation procedure, based on the fast Fourier transform (FFT), is used to combine the wavefield in the different grids. The results of application of the multi-grid, parallel FDM code on the Earth Simulator for modeling strong ground motions from recent large earthquakes are also presented. Events such as the 1993 Kushiro (Mj7.8) and 2000 Tottori-ken Seibu (Mj7.3) earthquakes were simulated using 3D structural models of northern and western Japan. The subsurface structures in Japan were derived by combining data from a number of reflection and refraction experiments, Bouguer anomaly data, and travel-time tomography studies of P and S waves. The scale of the 3D model is about 500 km by 1000 km by 350 km, which is divided into grid intervals of 0.5 to 1 km. The simulation required 128 to 364 Gb of computation memory, and computation took 1 to 2 h using 256 to 1408 processors of the Earth Simulator. Assuming a minimum shear wave velocity of Vs = 1.7 km/s, the modeling is capable of treating high-frequency seismic wave propagations of over 1 to 2 Hz. The volume rendering technique was employed to illuminate the 3D wavefield, and a set of snapshots was combined into a video sequence. The high-resolution 3D simulations for frequencies over 1 Hz provide a good representation of wave propagation in Japan during the large earthquakes. The computer simulation also matches the observations by the dense seismic array (K-Net and KiK-net, over 1700 stations) well, demonstrating the effectiveness of the simulation model. The combined studies of high-resolution computer simulation and dense seismic observation can therefore be expected to be highly valuable in understanding the complex seismic behavior associated with heterogeneities in the subsurface structure, and for predicting the pattern of ground motions expected for future earthquake scenarios.
DE: 0902 Computational methods, seismic
DE: 7212 Earthquake ground motions and engineering
DE: 7223 Seismic hazard assessment and prediction
DE: 7260 Theory and modeling
SC: Seismology [S]
MN: 2003 Fall Meeting