Data Analysis: What Can Be Learned From the Past 50 Years

DATA ANALYSIS: What Can Be Learned From the Past 50 Years......Page 5
CONTENTS......Page 7
Preface......Page 13
1 What is Data Analysis?......Page 17
1.1 Tukey\'s 1962 paper......Page 19
1.2 The Path of Statistics......Page 21
2.1 Strategy in Data Analysis......Page 27
2.2 Philosophical issues......Page 29
2.2.1 On the theory of data analysis and its teaching......Page 30
2.2.2 Science and data analysis......Page 31
2.2.3 Economy of forces......Page 32
2.3 Issues of size......Page 33
2.4.1 Planning the data collection......Page 37
2.4.2 Choice of data and methods......Page 38
2.4.3 Systematic and random errors......Page 39
2.4.4 Strategic reserves......Page 40
2.4.5 Human factors......Page 41
2.5.1 Inspection......Page 42
2.5.2 Error checking......Page 43
2.5.5 Modeling and Model fitting......Page 46
2.5.6 Simulation......Page 47
2.5.9 Presentation of conclusions......Page 48
2.6.1 Ad hoc programming......Page 49
2.6.2 Graphics......Page 50
2.6.4 Creating and keeping order......Page 51
3 Massive Data Sets......Page 53
3.1 Introduction......Page 54
3.3 What is massive? A classification of size......Page 55
3.4.1 Human limitations: visualization......Page 56
3.4.3 Storage requirements......Page 57
3.4.4 Computational complexity......Page 58
3.5.1 Types of data......Page 59
3.5.3 On data organization......Page 60
3.5.4 Derived data sets......Page 61
3.6 Data base management and related issues......Page 62
3.6.1 Data archiving......Page 64
3.7.1 Planning the data collection......Page 65
3.7.4 Initial data checking......Page 66
3.7.6 The final product: presentation of arguments and conclusions......Page 67
3.8 Examples and some thoughts on strategy......Page 68
3.9 Volume reduction......Page 71
3.10 Supercomputers and software challenges......Page 72
3.10.2 General Purpose Data Analysis and Supercomputers......Page 73
3.10.3 Languages, Programming Environments and Databased Prototyping......Page 74
3.11 Summary of conclusions......Page 75
4 Languages for Data Analysis......Page 77
4.1 Goals and purposes......Page 78
4.2.1 Natural languages......Page 80
4.2.2 Batch languages......Page 81
4.2.3 Immediate languages......Page 83
4.2.4 Language and literature......Page 84
4.2.5 Object orientation and related structural issues......Page 85
4.2.6 Extremism and compromises, slogans and reality......Page 87
4.2.7 Some conclusions......Page 89
4.3 Interface issues......Page 90
4.3.1 The command line interface......Page 91
4.3.2 The menu interface......Page 94
4.3.3 The batch interface and programming environments......Page 96
4.3.4 Some personal experiences......Page 97
4.4.1 On building blocks......Page 98
4.4.3 On notation......Page 99
4.4.4 Book-keeping problems......Page 100
4.5 Requirements for a general purpose immediate language......Page 101
5.1 Models......Page 105
5.2 Bayesian modeling......Page 108
5.3 Mathematical statistics and approximate models......Page 110
5.4 Statistical significance and physical relevance......Page 112
5.5 Judicious use of a wrong model......Page 113
5.6 Composite models......Page 114
5.7 Modeling the length of day......Page 115
5.8 The role of simulation......Page 127
5.9 Summary of conclusions......Page 128
6 Pitfalls......Page 129
6.1 Simpson\'s paradox......Page 130
6.2 Missing data......Page 132
6.2.1 The Case of the Babylonian Lunar Six......Page 134
6.2.2 X-ray crystallography......Page 142
6.3 Regression of Y on X or of X on Y?......Page 145
7 Create order in data......Page 149
7.1 General considerations......Page 150
7.2 Principal component methods......Page 151
7.2.1 Principal component methods: Jury data......Page 153
7.3.2 Multidimensional scaling: a synthetic example......Page 161
7.4.1 Correspondence analysis: the method......Page 163
7.4.2 Kültepe eponyms......Page 164
7.4.3 Further examples: marketing and Shakespearean plays......Page 172
7.5 Multidimensional scaling vs. Correspondence analysis......Page 176
7.5.1 Hodson\'s grave data......Page 178
7.5.2 Plato data......Page 184
8 More case studies......Page 193
8.1 A nutshell example......Page 194
8.2 Shape invariant modeling......Page 198
8.3 Comparison of point configurations......Page 200
8.3.1 The cyclodecane conformation......Page 202
8.3.2 The Thomson problem......Page 205
8.4 Notes on numerical optimization......Page 206
References......Page 211
Index......Page 221