├── demo.png ├── .gitattributes ├── demo_sorted_by_category.png ├── LICENSE ├── README.md └── spec_curve_demo.do /demo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hhsievertsen/speccurve/HEAD/demo.png -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | # Auto detect text files and perform LF normalization 2 | * text=auto 3 | -------------------------------------------------------------------------------- /demo_sorted_by_category.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hhsievertsen/speccurve/HEAD/demo_sorted_by_category.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Hans Henrik Sievertsen 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # speccurve 2 | Specification curve in Stata 3 | 4 | 5 | Hi, 6 | 7 | the file "spec_curve_demo.do" shows how you can create a specification curve (like the ones below) in Stata. The example uses the auto dataset in Stata. 8 | 9 | 10 | ### Version 1: sorted by coefficient size 11 | 12 | ![Demo](demo.png) 13 | 14 | 15 | ### Version 2: sorted by category (Brian Heseung Kim implementation) 16 | ![Demo](demo_sorted_by_category.png) 17 | 18 | 19 | #### Use 20 | 21 | * I created a small function to store estimates, but created the chart manually based on the stored estimates. 22 | * Brian Heseung Kim implemented the option to sort by category. To activate that just remove the "sort beta". 23 | 24 | #### License 25 | You are very welcome to use the code, change it, suggest improvements etc. You do not need to give me credit, but be kind to someone today. 26 | 27 | 28 | #### Credits 29 | I was inspired to create this chart by Peter Eibich (https://www.demogr.mpg.de/en/institute/staff_directory_1899/peter_eibich_3594.htm) who had a similar chart in a presentation. So any credit should go to him 30 | 31 | There is a lot of material on this out there that is much more sophisticated. See for example the work of Uri Simonsohn (number 37): http://urisohn.com/ 32 | 33 | #### For R users 34 | 35 | See this brilliant package: https://github.com/AakaashRao/starbility 36 | 37 | #### For Stata users 38 | See this brilliant function: https://github.com/martin-andresen/speccurve 39 | 40 | 41 | Best, 42 | Hans 43 | 44 | 45 | -------------------------------------------------------------------------------- /spec_curve_demo.do: -------------------------------------------------------------------------------- 1 | * specification curve demo by Hans H. Sievertsen h.h.sievertsen@bristol.ac.uk, 29/10-2019 2 | 3 | clear 4 | cap program drop specchart 5 | program specchart 6 | syntax varlist, [replace] spec(string) 7 | * save current data 8 | tempfile temp 9 | save "`temp'",replace 10 | *dataset to store estimates 11 | if "`replace'"!=""{ 12 | clear 13 | gen beta=. 14 | gen se=. 15 | gen spec_id=. 16 | gen u95=. 17 | gen u90=. 18 | gen l95=. 19 | gen l90=. 20 | save "estimates.dta",replace 21 | } 22 | else{ 23 | * load dataset 24 | use "estimates.dta",clear 25 | } 26 | * add observation 27 | local obs=_N+1 28 | set obs `obs' 29 | replace spec_id=`obs' if _n==`obs' 30 | * store estimates 31 | replace beta =_b[`varlist'] if spec_id==`obs' 32 | replace se=_se[`varlist'] if spec_id==`obs' 33 | replace u95=beta+invt(e(df_r),0.975)*se if spec_id==`obs' 34 | replace u90=beta+invt(e(df_r),0.95)*se if spec_id==`obs' 35 | replace l95=beta-invt(e(df_r),0.975)*se if spec_id==`obs' 36 | replace l90=beta-invt(e(df_r),0.95)*se if spec_id==`obs' 37 | * store specification 38 | foreach s in `spec'{ 39 | cap gen `s'=1 if spec_id==`obs' 40 | cap replace `s'=1 if spec_id==`obs' 41 | } 42 | save "estimates.dta",replace 43 | * restore dataset 44 | use `temp',clear 45 | end 46 | 47 | 48 | * run regressions 49 | clear 50 | sysuse auto 51 | 52 | * main spec 53 | regress price weight 54 | specchart weight,spec(main sample`i' linear) replace 55 | 56 | * alternative specs 57 | * loop over sample size 58 | forval i=50(2)74{ 59 | clear 60 | sysuse auto 61 | keep if _n<`i' 62 | * no covars 63 | * no covars + linear mpg 64 | qui: regress price weight mpg 65 | specchart weight,spec(covars linear sample`i') 66 | 67 | * no covars + quadratric mpg 68 | qui: regress price weight c.mpg##c.mpg 69 | specchart weight,spec(covars quadratic sample`i') 70 | 71 | * no covars + cubic mpg 72 | qui: regress price weight c.mpg##c.mpg##c.mpg 73 | specchart weight,spec(covars cubic sample`i') 74 | 75 | * no covars + quartic mpg 76 | qui: regress price weight c.mpg##c.mpg##c.mpg##c.mpg 77 | specchart weight,spec(covars quartic sample`i') 78 | 79 | * with covars 80 | * w. covars + linear mpg 81 | qui: regress price weight length turn trunk displacement 82 | specchart weight,spec( linear sample`i') 83 | 84 | * w. covars + quadratric mpg 85 | qui: regress price weight c.mpg##c.mpg length turn trunk displacement 86 | specchart weight,spec( quadratic sample`i') 87 | 88 | * w. covars + cubic mpg 89 | qui: regress price weight c.mpg##c.mpg##c.mpg length turn trunk displacement 90 | specchart weight,spec( cubic sample`i') 91 | 92 | * w. covars + mpg length 93 | qui: regress price weight c.mpg##c.mpg##c.mpg##c.mpg length turn trunk displacement 94 | specchart weight,spec( quartic sample`i') 95 | } 96 | 97 | * create chart 98 | use "estimates.dta",clear 99 | * drop duplicates 100 | duplicates drop covars cubic linear quadratic quartic sample*, force 101 | /* sort specification by category */ 102 | gsort -quartic -cubic -quadratic -linear -covars /// 103 | -sample74 -sample72 -sample70 -sample68 -sample66 /// 104 | -sample64 -sample62 -sample60 -sample58 -sample56 /// 105 | -sample54 -sample52 -sample50, mfirst 106 | /* sort estimates by coefificent size, uncomment to activate sort by category */ 107 | sort beta 108 | * rank 109 | gen rank=_n 110 | * gen indicators and scatters 111 | local scoff=" " 112 | local scon=" " 113 | local ind=-1.5 114 | foreach var in covars linear quadratic cubic quartic { 115 | cap gen i_`var'=`ind' 116 | local ind=`ind'-0.4 117 | local scoff="`scoff' (scatter i_`var' rank,msize(vsmall) mcolor(gs10))" 118 | local scon="`scon' (scatter i_`var' rank if `var'==1,msize(vsmall) mcolor(black))" 119 | } 120 | * samples 121 | local ind=`ind'-0.6 122 | forval i=50(2)74{ 123 | cap gen i_sample`i'=`ind' 124 | local ind=`ind'-0.4 125 | local scoff="`scoff' (scatter i_sample`i' rank,msize(vsmall) mcolor(gs10))" 126 | local scon="`scon' (scatter i_sample`i' rank if sample`i'==1,msize(vsmall) mcolor(black))" 127 | } 128 | 129 | 130 | * plot 131 | tw (scatter beta rank if main==1, mcolor(blue) msymbol(D) msize(small)) /// main spec 132 | (rbar u95 l95 rank, fcolor(gs12) lcolor(gs12) lwidth(none)) /// 95% CI 133 | (rbar u90 l90 rank, fcolor(gs6) lcolor(gs16) lwidth(none)) /// 90% CI 134 | (scatter beta rank, mcolor(black) msymbol(D) msize(small)) /// point estimates 135 | `scoff' `scon' /// indicators for spec 136 | (scatter beta rank if main==1, mcolor(blue) msymbol(D) msize(small)) /// main spec 137 | (scatter i_sample74 rank if main==1,msize(vsmall) mcolor(blue)) /// 138 | (scatter i_linear rank if main==1,msize(vsmall) mcolor(blue)) /// 139 | ,legend (order(1 "Main spec." 4 "Point estimate" 2 "95% CI" 3 "90% CI") region(lcolor(white)) /// 140 | pos(12) ring(1) rows(1) size(vsmall) symysize(small) symxsize(small)) /// 141 | xtitle(" ") ytitle(" ") /// 142 | yscale(noline) xscale(noline) ylab(0(2)9,noticks nogrid angle(horizontal)) xlab("", noticks) /// 143 | graphregion (fcolor(white) lcolor(white)) plotregion(fcolor(white) lcolor(white)) 144 | 145 | * now add stuff to the y axis 146 | gr_edit .yaxis1.add_ticks -1. `"Specification "', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 147 | gr_edit .yaxis1.add_ticks -1.5 `"Covariates"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 148 | gr_edit .yaxis1.add_ticks -1.9 `"Linear"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 149 | gr_edit .yaxis1.add_ticks -2.3 `"Quadratic"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 150 | gr_edit .yaxis1.add_ticks -2.7 `"Cubic"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 151 | gr_edit .yaxis1.add_ticks -3.1 `"Quartic"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 152 | 153 | gr_edit .yaxis1.add_ticks -3.6 `"Sample "', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 154 | gr_edit .yaxis1.add_ticks -4.1 `"<=50"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 155 | gr_edit .yaxis1.add_ticks -4.5 `"<=52"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 156 | gr_edit .yaxis1.add_ticks -4.9 `"<=54"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 157 | gr_edit .yaxis1.add_ticks -5.3 `"<=56"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 158 | gr_edit .yaxis1.add_ticks -5.7 `"<=58"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 159 | gr_edit .yaxis1.add_ticks -6.1 `"<=60"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 160 | gr_edit .yaxis1.add_ticks -6.5 `"<=62"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 161 | gr_edit .yaxis1.add_ticks -6.9 `"<=64"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 162 | gr_edit .yaxis1.add_ticks -7.3 `"<=66"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 163 | gr_edit .yaxis1.add_ticks -7.7 `"<=68"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 164 | gr_edit .yaxis1.add_ticks -8.1 `"<=70"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 165 | gr_edit .yaxis1.add_ticks -8.5 `"<=72"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 166 | gr_edit .yaxis1.add_ticks -8.9 `"<=74"', custom tickset(major) editstyle(tickstyle(textstyle(size(vsmall))) ) 167 | 168 | gr_edit .yaxis1.add_ticks 9 `"Coefficient"', custom tickset(major) editstyle(tickstyle(textstyle(size(small))) ) 169 | --------------------------------------------------------------------------------