Naoshi Hirata, Tsuruoka Hiroshi , Danijel Schorlemmer 9th International Workshop on Statistical Seismology Five-year Japanese earthquake predictability experiment with multiple runs since 2009 including the 2011 Tohoku-oki earthquake I am reporting CSEP japan progress report since 2009 including 2011 Tohoku-oki Earthquake. Naoshi Hirata, Tsuruoka Hiroshi , Danijel Schorlemmer 2015/6/17 Statsei9
Outline Japanese CSEP experiment: classes and regions Models Results 3-1. One day test for periods before and after the 2011 Tohoku-oki event (All Japan) 2011/03/01-2011/03/31 3-2 Three month test before and after the 2011 Tohoku-oki 2011/02/01-2013/08/01 All Japan Discussion Conclusions 2015/6/17 Statsei9
1. Japanese CSEP testing 2009/11/1- (Collaboratory for the Study of Earthquake Predictability) Test region (Nanjo et al., 2011; Tsuruoka et al., 2012) AllJapan 0.1°x 0.1° depth ≤ 100km Test class 1-day 4≤M≤9 3-month 4≤M≤9 1-year 5≤M≤9 3-year 5≤M≤9 Mainland 0.1°x 0.1° depth ≤ 30km Evaluation significance level2.5% Kanto 0.05°x 0.05° depth ≤ 100km test L Likelihood of distributions of magnitude, number and place CL Conditional L-test N Event number M Magnitude distribution S Spatial distribution W/T Pair-wise comparison between forecasts I briefly mention the CSEP-Japan Rule or Protocol. We start this experiment from November 2009. CSEP-Japan has 4 testing class and 3 testing regions. We evaluate earthquake models forecasting by using CSEP standard methods such as L, N, M and S-Tests. Nanjo, K. Z. H. Tsuruoka, N. Hirata, and T. H. Jordan, Overview of the first earthquake forecast testing experiment in Japan, Earth Planets Space, 63, 3, 159-169, 2011. Tsuruoka, H., N. Hirata, D. Schorlemmer, F. Euchner, K. Z. Nanjo, and T. H. Jordan, 2012, CSEP Testing Center and the first results of the earthquake forecast testing experiment in Japan, Earth Planets Space, Vol. 64 (No. 8), pp. 661-671, 2012 2015/6/17 Statsei9
2. Models submitted to CSEP-Japan 1Y 3Y TOTAL 2009/11/01- 11 25 31 24 91 2012/08/01- 45 53 160 This is the models that CSEP-Japan testcenr have in our servers. Numbers od applied forecasting models for each class for each period. 2015/6/17 Statsei9
Daily forecast testing class 2012/08/01- Model AllJapan Mainland Kanto 1 ERS 2 ETES 3 ETAS HISTETAS5PA1011 HISTETAS5PA 4 HISTETAS7PA1011 HISTETAS7PA 5 ERS1111 6 ETAS1105 ETES1111 7 HISTETAS5PA1208 8 ETES111 HISTETAS7PA1208 9 HISTETAS5PA1205 ETAS1208 10 HISTETAS7PA1205 11 TOTAL Names of daily forecasts applied to each region started from August 2012. AllJapan +ETAS1208 Mainland +HISTEAS5PA1208, HISTETAS7PA1208, ETAS1208 Kanto +ETAS1208 TOTAL=31 2015/6/17 Statsei9
3 months testing class 12th round (2012/08/01-2012/11-01) Model1 AllJapan Mainland Kanto 1 HISTETAS5PA EEPAS 2 HISTETAS7PA PPE 3 MARFS RI10K 4 MARFSTA RI30K 5 RI50K 6 RI100K 7 TRIPLES1011 8 HISTETAS5PA1011 9 HISTETAS7PA1011 10 T1Q1208 HISTETAS5PA1205 11 T2Q1208 HISTETAS7PA1102 HISTETAS7PA1205 12 T4Q1208 ETAS1208 13 14 15 HISTETAS5PA1208 16 HISTETAS7PA1208 17 DelonayPoisson 18 TOTAL Names of models tested for 12th round of the 3 months forecasts applied to each region . AllJapan -T1Q1205, T2Q1205, T4Q1205 +T1Q1208, T2Q1208, T4Q1208, ETAS1208 Mainland -T1Q1205, T2Q1205, T4Q1205 +T1Q1208, T2Q1208, T4Q1208, HISTETAS5PA1208, HISTETAS7PA1208, DelonayPoisson, ETAS1208 Kanto +ETAS1208 TOTAL=45 2015/6/17 Statsei9
3. Results 3-1. One day test for periods before and after the 2011 Tohoku-oki event (All Japan) 2011/03/01-2011/03/31 3-2 Three month testing before and after the 2011 Tohoku-oki : 2011/02/01-2013/08/01 All Japan 2015/6/17 Statsei9
3-1. One day test for aftershocks of the 2011 Tohoku-oki event Test region:AllJapan 2011/03/01-2011/03/31 CSEP-Japan target M>=4.0 This is the similar report of 1-day forecast including the seismicity before and after Tohoku earthquake to those published by Nanjo and others in Geophysical J. International, but the data is based on the final edition of the JMA catalog. １日の結果は，楠城さんの一日論文のサマリーだけをリストしました． 2015/6/17 Statsei9
2011/03/01 ERS : Falcone et al. (2010) ETES: Falcone et al. (2010) ETAS：Zhuang（2011） HISTETAS：Ogata(2011) 2015/6/17 Statsei9
2011/03/03 2015/6/17 Statsei9
2011/03/05 2015/6/17 Statsei9
2011/03/07 2015/6/17 Statsei9
2011/03/08 2015/6/17 Statsei9
2011/03/09 2015/6/17 Statsei9
2011/03/10 2015/6/17 Statsei9
2011/03/11 2015/6/17 Statsei9
2011/03/12 2015/6/17 Statsei9
2011/03/13 2015/6/17 Statsei9
2011/03/14 2015/6/17 Statsei9
2011/03/15 2015/6/17 Statsei9
2011/03/20 2015/6/17 Statsei9
2011/03/25 2015/6/17 Statsei9
2011/03/30 2015/6/17 Statsei9
Predicted vs Observation M>=4.0 ETES Observed ETESの地震予測数は，実際の地震発生数と近いので，ETASの次によかったと考えられる． 2015/6/17 Statsei9
Loglikelihood/#event (L-test) イベントあたりの対数尤度をそのままプロットした図である．トータルでベストのモデルは， ETASである． Daily log-likelihood scores of the five forecasts. BEST: ETAS (blue) 2015/6/17 Statsei9
Ranking of Information Gain (Space) Reference: uniform model 1 ETAS 2.461 2 ERS 2.332 3 ETES 2.179 4 HISTETAS5PA 2.123 5 HISTETAS7PA 1.927 次に，L-Testではなくて，空間分布のS-Testについて，一様空間期待値モデルを基準とした 情報利得値を比べてみると，これもETASがトップであった．次は，ERSがはいった． This table shows that the ETAS forecast attained the best score. 2015/6/17 Statsei9
Information Gain Refrence:ETAS ETASを基準として他のモデルをプロットしてみると，ETASの次にERSがよいということがわかる． 2015/6/17 Statsei9
Summary of 1-day results during March 2011 including mainshock, foreshocks and aftershocks All models fail on the most active days. (ex. 3/9, 3/11, 3/22) ETES model provides consistent predictions for the total numbers. ETAS model has a good performance of Probability Gain(=Information Gain) of Space the other models tend to under-estimate the numbers. 2011年3月の1日予測のまとめである． 2015/6/17 Statsei9
3-2 Three month testing before and after the 2011 Tohoku-oki : 2011/02/01-2013/08/01 (6-15 rounds) All Japan 次にAllJapanの3ヶ月予測結果を示す． モデルは，HISTETAS5PA, HISTETAS7PA, RI10K, MARFS, MARFSTA, HISTETAS5PA1205, HISTETAS7PA1205の7モデルを 評価した． 2015/6/17 Statsei9
Forecasted (HISTETAS7PA): 6th round 2011-02-01 to 2011-05-01 2015/6/17 Statsei9
Observed: 6th round 2011-02-01 to 2011-05-01 2015/6/17 Statsei9
Observation vs Forecast この図は，実際の地震数と7モデルによる予測数を示している． HISTETAS{5,7}PA1205は，予測地震数は，観測数を追随しているが， HISTETAS5PA, HISTETAS7PA, RI10Kは，追随していないのがわかる． 2015/6/17 Statsei9
Ranking of L-test LL/#event MODEL Mean LL/#event 1 HISTETAS7PA1205 -6.502 2 HISTETAS5PA1205 -6.617 3 RI10K -6.953 4 HISTETAS5PA -6.954 5 MARFS -7.052 6 MARFSTA -7.056 7 HISTETAS7PA -7.066 まずは，L-Testの同様のランクを示す． HISTETAS7PA1205がトップであった． This table shows that the ETAS forecast attained the best score. 2015/6/17 Statsei9
AllJapan 3month (L-test) LL/#event 先ほどの図をそのままプロットしたものである．HISTETAS7PA1205 と HISTETAS5PA1205は， 6-10ラウンドまでは，事後予測になるが，高いパフォーマンスを示している． 2015/6/17 Statsei9
AllJapan 3month L-test LL/#event reference: RI10K HISTETAS5PA1205, HISTETAS7PA1205は，RI10Kよりもパフォーマンスがよく， その他は，RI10Kよりも全体として劣っていることがわかる． 2015/6/17 Statsei9
Ranking of Information Gain (Space) reference: uniform model Mean LL/#event 1 HISTETAS7PA1205 1.747 2 HISTETAS5PA1205 1.725 3 RI10K 1.380 4 HISTETAS5PA 1.372 5 HISTETAS7PA 1.260 6 MARFS 1.254 7 MARFSTA 1.251 S-testによる評価で，空間一様モデルを基準とした情報利得でのランキングである． これもHISTETAS7PA1205が十婦であった． This table shows that the ETAS forecast attained the best score. 2015/6/17 Statsei9
Information Gain (S-test) Reference: uniform model 前の表を図示したものである．HISTETAS{5PA,7PA}1205が高いパフォーマンスをしめしていることがわかる． 2015/6/17 Statsei9
Information Gain (S-test) Reference: RI10K 前の図をRI10Kを基準としたものである．6ラウンド以降において，RI10Kよりも，HISTETAS5PA, HISTETAS7PA, MARFS, MARFSTAのパフォーマンスが劣っていることがわかる． 2015/6/17 Statsei9
4. Discussion After the 2011 Tohoku-oki EQ Many model underestimate forecasted number of Eq after the 2011 Tohoku-oki EQ Time variant component is also needed even for 3 for all testing regions → Forecasting number of EQ is important Space-test is effective for selecting models ３／１１以降は，3ヶ月予測テストクラスにおいても，時間依存の要素を考慮したモデル化が必要である． We will discuss the requirement for models after the 2011 tohoku-oki earthquake 2015/6/17 Statsei9
5. Conclusions After Tohoku-oki earthquake, model need time variant component even for 3 month for all testing regions. Models may be evaluated by not L-test but S-Test after Tohoku-oki earthquake Predictability of forecasting number is more important after Tohoku-oki earthquake than before. We summarize the results in the following. 2015/6/17 Statsei9
Ongoing Plans Real-Time Forecasts based on Automated Catalogs Comprehensive and Topical Evaluation of Continuous Time Forecasts (cf., Ogata et al. 2013, BSSA): Retrospective and Prospective Real-Time Forecasts based on Automated Catalogs Kanto 3D testing region experiment - Time independent and dependent CSEP-japan has several ongoing plans, which include -- 2015/6/17 Statsei9
CSEP(2D) vs Uraban Project(3D) 2D (CSEP) 3D(Urban Project) regions CSEP’s Kanto region Lon138.5-141.5 Lat34.5-37.0 Dep0-100km Resolutions 0.05°x0.05°x 100km (169881bins=3331x51x1) 0.05°x0.05°x5km (3060000bins=3000x51x20) 0.05° 0.05° 5 km We add depth axis in the Kanto testing region. This has first been examined by 3D-RI model, and also considering 3D ETAS models. For the former, you can see Yokoi-san’s poster for the SCEC meeting. One of the points is that 3D models performed better than the 2D models. 2015/6/17 Statsei9
Thanks for attention! 2015/6/17 Statsei9