This massive open online course (MOOC) was conceived during a call for proposals made by the executive of the University of Cape Town (UCT) in 2013. At that time, the continent of Africa was devoid of any tertiary education content on the large, international MOOC platforms. As leading ranked university on the continent, it was perhaps seen as the duty of the university to contribute to this field.
\n", 145 | "More so, Groote Schuur Hospital (GSH) is home of the clinical services of the Faculty of Health Sciences at UCT.Even after nearly 50 years, it is still well-known for the first human heart transplant, and therefore was inevitably going to play its role.
\n", 146 | "This course, was therefore proposed and designed in the Department of Surgery at GSH. As recipient of the 2014 Open Education Consortium Individual Educator of the Year award for the development of open educational resources, I personally felt both compelled and hugely excited to bring this course to life.
" 147 | ] 148 | }, 149 | { 150 | "cell_type": "markdown", 151 | "metadata": {}, 152 | "source": [ 153 | "## Audience" 154 | ] 155 | }, 156 | { 157 | "cell_type": "markdown", 158 | "metadata": {}, 159 | "source": [ 160 | "In this second decade of the twentieth century, technology has awarded us the gift of new forms of teaching and learning. As an outstanding example, we have the MOOC, which you are currently partaking in. The first term, massive, provides, in itself, a massive challenge. In the traditional classroom, teacher and student are bound by many common ideals, mindsets and goals, which are all well ring-fenced in an individual facility.
\n", 161 | "In contrast to this, the MOOC has to be designed to fulfill many needs as well as be applicable in many scenarios. Herein, , can also lie a strength. In the previous lecture, reference was made to the fact that both novice researchers (in a first-world setting) as well as potential researchers (in under-resourced areas) could find benefit from this course. This lecture will elaborate on the intended audience.
\n", 162 | "It is a well-established fact that a large proportion of healthcare research is conducted in the first-world. Resources are, to some extent, widely available. These resources include funding, which forms part of a self-perpetuating and well-maintained ecosystem. More research ensures continued funding, which in turn, allows for the continuation of research.
\n", 163 | "With the abundance in research there comes experienced researchers. These individuals and groups are well placed to establish a hierarchy for the training of new researchers. Time, effort and money are all available, not only to teach newcomers, but also to market the idea of conducting research.
\n", 164 | "More financial resources also bring more personnel. Healthcare provision is understood to stand on three pillars: service delivery, teaching, and research. With a larger pool of human resources, more time is available to individuals to concentrate on research and to induct newcomers into the field.
\n", 165 | "However, with this excellence in research, there comes competition in the workplace. Those who wish to embark on healthcare research have a lot to live up to. The expectations are indeed enormous and the bar is set extremely high. A subset of the intended audience for this course then, is those who intend to get a head-start in this process. No doubt, most of them will have resources available at their home institutions, but as with most online courses, the intent is not to replace, but to augment and provide yet another learning and teaching opportunity.
\n", 166 | "The healthcare research spectrum is wide indeed, and leaning towards the opposite side is the 'under-resourced' setting. In many cases none or very little research activity is noted. Experienced individuals are scarce and many of them are lost to better equipped facilities and countries.
\n", 167 | "Monetary resources are much more difficult to come by in these areas. This has the effect of reducing the personnel available to work in healthcare, taking those involved away from research and concentrating on service delivery and, to a lesser, extent, teaching. Perhaps under appreciated is the fact that no culture exists to market research under these circumstances. This point has been raised twice now, with the explicit aim of emphasizing the need to sell the prospect of becoming involved in research. In many instances service delivery becomes the only way for individuals to attain job security and thereofore an income.
\n", 168 | "All of this is compounded by the burden of disease which is prevalent in many of these settings. Long delays ensue and the natural progression of disease is seen to a much larger extent than in the first-world. Not only do patients present late in the course of their disease, but the disease profile as such, does not reflect that of the first-world. With the latter engaged in the research of diseases of relative importance to them. Hence the emphasis is not always placed on diseases that affect a large proposition of humanity.
\n", 169 | "Lastly, addressing specifically the individual for whom this course was developed, consideration must be given to the modern view-point of interdisciplinary and integrated healthcare. A great many individuals and groups are involved in modern healthcare delivery, irrespective of the setting. From laboratory scientist to healthcare manager; from medical specialist, to assistant nurse; and from pharmacist to physiotherapist, every role-player will find some benefit to this course.
" 170 | ] 171 | }, 172 | { 173 | "cell_type": "markdown", 174 | "metadata": {}, 175 | "source": [ 176 | "## Aim" 177 | ] 178 | }, 179 | { 180 | "cell_type": "markdown", 181 | "metadata": {}, 182 | "source": [ 183 | "The aim of this course is quite simply to provide a structural foundation for the first steps in healthcare research. A foundation which is structured such that it is independent of where in the spectrum of the audience a participant might be and independent of the caregiver group he or she may represent.
\n", 184 | "In order to contribute effectively to the effort of healthcare research, this structural foundation requires some essential ingredients. This ranges from the obvious (which involves at least a basic understanding of statistics), to more practical issues.
\n", 185 | "To commence any research, a research question must exist. This involves the identification of a problem that requires a solution or, at least, an answer. It is important to understand how to identify these questions. An essential step in formulating a proper question is the need to identify other research that has been conducted in this field. The ability to write a literature review is a skill that all healthcare providers should possess.
\n", 186 | "From research questions which arise, the need to gather data evolves. This is not a trivial task and requires deep thought. Not only are the logistics important, but the actual data to be collected must contain within it, the solution to the research question. Data should be collectable in an accurate and reproducible way, whist simultaneously not being duplicate.
\n", 187 | "Collecting data in the setting of healthcare very often involves personal patient information and it is absolutely essential to understand the ethical ways of obtaining, storing, and using this data.
\n", 188 | "The actual vehicle of data capturing must be understood. In this modern era, many devices and software tools not only allow for accurate data capturing, but allow for the analysis of the data. It is important to have at least some understanding of the tools available as well as be comfortable in the use of some of them.
\n", 189 | "Once the data is analyzed it becomes important to share the results with peers and the wider community. A skill essential in this regard is the writing of a manuscript for presentation or submission either in the form of an abstract or a full-text article.
\n", 190 | "This course, then, aims to take you through this process.
" 191 | ] 192 | }, 193 | { 194 | "cell_type": "markdown", 195 | "metadata": {}, 196 | "source": [ 197 | "# Motivation for this lecture series" 198 | ] 199 | }, 200 | { 201 | "cell_type": "markdown", 202 | "metadata": {}, 203 | "source": [ 204 | "Welcome to this series on conducting healthcare research.
\n", 205 | "Whilst the main study material for this course is provided in the form of video lectures, it is my aim to provide you, the participants, with as many educational resources as possible. It is clear that some learners prefer the ability to watch video lectures and find maximum pedagogical benefit from this type of resource, rich in visual and auditory input, whilst others benefit most from the written word. For most of us, our personal preference lies somewhere in between. I trust, then, that this text will be of some use.
\n", 206 | "To further enrich the learning experience, the text is provided within an active coding environment. You could thus choose to simply stick to the written word, or you can partake in the actual analysis of data using code.
\n", 207 | "Although we can in no way compare to research performed in the hard sciences, such as particle physics, we have made huge strides by the introduction of evidence-based research, clinical management, and teaching." 208 | ] 209 | }, 210 | { 211 | "cell_type": "markdown", 212 | "metadata": {}, 213 | "source": [ 214 | "## Healthcare as a science" 215 | ] 216 | }, 217 | { 218 | "cell_type": "markdown", 219 | "metadata": {}, 220 | "source": [ 221 | "Research in the health sciences is taking the first steps towards becoming a true scientific endeavor, such as physics. Many of us are old enough to, with some fondness, remember the days of the preeminent statesman-like professor, on whose words we hung. Eminence-based medicine has a rich and prolific history, indeed.
\n", 222 | "As previously mentioned, when considering physics, it is important to take cognizance of the long lead-time it has over healthcare research. The strides made in the former field began at the start of the twentieth century, when Rutherford’s postgraduate students first noted evidence for the nuclear form of the atom. Bohr gave us the structure of electron orbits based on the work of Planck and Einstein, from which grew our fundamental understanding (or lack thereof) of the quantum world.
\n", 223 | "Not only does physics have a much longer research history, which has lead to a deep and wonderful understanding of the inner workings of our universe, but it also benefits from the size of available data to work with. We need only consider the work done at the Large Hadron Collider in Switzerland. The shear volume of data is so large, that most of it has to be discarded. Even so, they have to make use of distributed computing, with interconnected computers all over the world, to store and evaluate data.
\n", 224 | "Contrast this to the sample sizes seen, on average, in a typical healthcare research paper. We are bound by arbitrary values of two standard deviations as a measure of significance, whilst our learned colleagues in physics can set the bar orders of magnitude higher.
\n", 225 | "Turning full-circle we need to remember that our understanding of disease is still very primitive. We design studies by trying to isolate a characteristic to use in comparing various groups. It is a serious matter to consider the fact that these characteristics are seldom ‘singular’. If we knew more we would probably discover a large and rich causal structure belying our characteristic of choice and if we could design studies to investigate these, our patient care would improve.
\n", 226 | "Consider the following scenario: A researcher looks into breast carcinoma and groups patients according to stage. An intervention is shown to have a significant difference between a set of groups. These groups were constructed from the presence of tumor size and nodal and distal spread. In the analysis, statistical outliers will invariably be noted. Patients whose reactions to the intervention stand in contrast to that of their group. Alas, they are often lost in our analysis.
\n", 227 | "Yet, we have enough knowledge now to know that tumor size and disease spread has a deeper causal layer, involving genetics as an example. The rich interaction between genetic mutations in tumor genes and the patient’s defense system, with resulting sensitivity to chemotherapeutic agents are much more important to investigate. For this, though, we need adequate sample sizes, something physicists will always have over us.
\n", 228 | "Akin to their efforts however,we are metaphorically, digging deeper. In order for us to continue this process, it is important to widen the research capabilities of those involved in healthcare. This can be achieved in a variety of ways. Considering those of us who are to take up positions in healthcare in well-resourced environments, it is perhaps prudent to provide courses such as this to those who wish to get get a head-start in the process before being formally introduced to the topic during their early education. For others, perhaps in more austere circumstances, the luxury of well-funded research units with experienced colleagues is not available. Workers in these cricumstances, though, are usually blessed with riches in available clinical data. They might be in the position to solve many of our outstanding research questions if provided with the tools to commence their own research.
\n", 229 | "One of the major steps towards improving our science then, is to enable all healtcare workers to understand data analysis. This is the true motivation for this course. It is my sincerest wish then, that someone out there will find value in this course as they embark on a career in healthcare research that will benefit us all.
" 230 | ] 231 | }, 232 | { 233 | "cell_type": "code", 234 | "execution_count": null, 235 | "metadata": { 236 | "collapsed": false 237 | }, 238 | "outputs": [], 239 | "source": [] 240 | } 241 | ], 242 | "metadata": { 243 | "kernelspec": { 244 | "display_name": "Python 3", 245 | "language": "python", 246 | "name": "python3" 247 | }, 248 | "language_info": { 249 | "codemirror_mode": { 250 | "name": "ipython", 251 | "version": 3 252 | }, 253 | "file_extension": ".py", 254 | "mimetype": "text/x-python", 255 | "name": "python", 256 | "nbconvert_exporter": "python", 257 | "pygments_lexer": "ipython3", 258 | "version": "3.4.3" 259 | } 260 | }, 261 | "nbformat": 4, 262 | "nbformat_minor": 0 263 | } 264 | -------------------------------------------------------------------------------- /Chapter04_Introduction_to_the_IPython_Notebook_II.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": false 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "from warnings import filterwarnings\n", 12 | "from IPython.core.display import HTML\n", 13 | "from IPython.display import Image\n", 14 | "\n", 15 | "filterwarnings('ignore')" 16 | ] 17 | }, 18 | { 19 | "cell_type": "markdown", 20 | "metadata": {}, 21 | "source": [ 22 | "# IPython Notebook" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": {}, 28 | "source": [ 29 | "Fernando Pérez, Brian E. Granger, IPython: A System for Interactive Scientific Computing, Computing in Science and Engineering, vol. 9, no. 3, pp. 21-29, May/June 2007, doi:10.1109/MCSE.2007.53. URL: http://ipython.org
" 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "metadata": {}, 35 | "source": [ 36 | "## Learning more" 37 | ] 38 | }, 39 | { 40 | "cell_type": "markdown", 41 | "metadata": {}, 42 | "source": [ 43 | "You can learn a lot more about *IPython* from there website: http://www.ipython.org
" 44 | ] 45 | }, 46 | { 47 | "cell_type": "code", 48 | "execution_count": 2, 49 | "metadata": { 50 | "collapsed": true 51 | }, 52 | "outputs": [ 53 | { 54 | "data": { 55 | "text/html": [ 56 | "" 57 | ], 58 | "text/plain": [ 59 | "This lecture is devoted to the topic of study design. One of the most important factors in both reading and evaluating published literature and in designing your own research is a thorough understanding of the variety in study types. Indeed, the choice of study type has a direct bearing on the findings, its importance and relevance.
\n", 145 | "As with most scientific endeavors, the classification of study type is varied and different schemes exist to classify designs. One logical approach is to divide designs into two main groups: *observational* and *experimental*. In observational studies, subjects or variables are merely observed and in experimental studies, an intervention is performed. From this basic difference a scheme to different study design can follow.
\n", 146 | "I'll conclude with two study types that fall outside of the chosen classification system: *meta-analysis* and *reviews*.\n", 147 | "
" 148 | ] 149 | }, 150 | { 151 | "cell_type": "markdown", 152 | "metadata": {}, 153 | "source": [ 154 | "## Classification" 155 | ] 156 | }, 157 | { 158 | "cell_type": "markdown", 159 | "metadata": {}, 160 | "source": [ 161 | "Observational studies consist of four main types: *case-series*, *case-control*, *cross-sectional*, and *cohort* studies.
\n", 162 | "Experimental studies are much easier to identify, as they directly state the form of intervention, whether it be the administration of a drug or the performance of procedure. In the case where experimental studies involve humans as test subjects, the term *clinical trial* is more commonly used. Experimental trials are further divided depending on the type of control that is used for comparison: *trials with independent concurrent controls*, *trials with self-controls* (dependent groups), *trials with external controls*, and *uncontrolled trials*.
" 163 | ] 164 | }, 165 | { 166 | "cell_type": "markdown", 167 | "metadata": {}, 168 | "source": [ 169 | "## Case series" 170 | ] 171 | }, 172 | { 173 | "cell_type": "markdown", 174 | "metadata": {}, 175 | "source": [ 176 | "This is perhaps the simplest of all study types and reports a simple descriptive account of a characteristic observed in a group of subjects. A good example would be a *clinical audit*, merely measuring various variables of a particular subject group over a defined period.
\n", 177 | "No research hypothesis exists and these studies are usually used to identify interesting observations for more detailed research or future planning.
\n", 178 | "Note that the actual data collection (for this and other types) can be performed either in *retrospect*, where we take the files and hunt for values for chosen variables, or in *prospect*, where we decide beforehand what variables we want documented and collect these during the management of every patient.
" 179 | ] 180 | }, 181 | { 182 | "cell_type": "markdown", 183 | "metadata": {}, 184 | "source": [ 185 | "## Case-control series" 186 | ] 187 | }, 188 | { 189 | "cell_type": "markdown", 190 | "metadata": {}, 191 | "source": [ 192 | "A case-control study starts with the presence (or absence) of an outcome and looks rearwards in time comparing subjects with and without the particular outcome in an attempt to find risk factors or characteristics that differ between the groups. It is important to note that the two groups are identified at the start of the study and data points on various variables are then collected retrospectively.
\n", 193 | "Do not confuse this with the terms prospective and retrospective. Let me explain. Nothing prevents you from setting up a database for any particular group or disease or whatever in your department or unit. You may have no plans yet for any research. What you are looking for is a set of variables, which you want completed for every patient. This type of data collection is done prospectively. On the other hand, you might have not had this forethought and simply go to the free-form notes in the patient files. This data collection is done retrospectively.
\n", 194 | "Back to case-control serieds. A good example would be to look at a group of patients who has had a particular procedure, with two varying outcomes. The fact that both groups had the same intervention differentiates this from an experimental study. Individuals with one outcome are separated from those with another outcome and data is collected on decided variables. The measurements are compared, to look for any differences.
\n", 195 | "The main disadvantage of this type of study is a concept termed confounding. It refers to a fact that a specific causative agent or parameter is difficult to identify. A good illustrative example would be that of the intake of meat in patients who suffer a myocardial infarction. It might be found that these patients have a high level of meat intake, but this is no proof that the meat was the causative agent. Meat-eaters might also have a high incidence of smoking, drinking, poor exercise regimes, and many others.
\n", 196 | "In a *nested case-control study*, cases and controls are taken from a prospective cohort study. The cases develop a certain effect and the controls don’t.
" 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "## Cross-sectional series" 204 | ] 205 | }, 206 | { 207 | "cell_type": "markdown", 208 | "metadata": {}, 209 | "source": [ 210 | "Many types of studies can be classified as cross-sectional and these include surveys, epidemiological studies, and prevalence studies. The outstanding identifying factor in the heterogenous groups is that an observation was made at a particular point in time, as opposed to over a period of time. To complicate matters ever so slightly, this type of study can be incorporated in other designs. It can, for example, be used as part of case-control or cohort studies.
\n", 211 | "A common use for cross-sectional studies is investigating correlation, whether between cause and effect or accuracy of diagnostic techniques, to name but two examples. As with case-control studies, cross-sectional studies also suffer from confounding and when cause and effect are under investigation, it is often impossible to prove that the cause actually is that and indeed came before the effect.
" 212 | ] 213 | }, 214 | { 215 | "cell_type": "markdown", 216 | "metadata": {}, 217 | "source": [ 218 | "## Cohort series" 219 | ] 220 | }, 221 | { 222 | "cell_type": "markdown", 223 | "metadata": {}, 224 | "source": [ 225 | "Here, it is important to define a *cohort* as a group of individuals (with a common trait such as a disease or risk factor) who remain part of that group over an extended time period. Subjects with the specific characteristic is identified at the start of the study and followed with measurement of various variables in a prospective manner. The time period does not necessarily have to be that long. Some prospective follow-ups after an intervention can last for a few days, for instance after a surgical procedure, termed outcome assessment.
\n", 226 | "Cohort and case-control studies are differentiated by their direction of inquiry, with the former always collecting data from a starting point, moving forward in time and the latter starting from a point in time and collecting data points that was noted before that point.
" 227 | ] 228 | }, 229 | { 230 | "cell_type": "markdown", 231 | "metadata": {}, 232 | "source": [ 233 | "## Trials with independent concurrent controls" 234 | ] 235 | }, 236 | { 237 | "cell_type": "markdown", 238 | "metadata": {}, 239 | "source": [ 240 | "Moving on to experimental trials, this first type includes trials that group patient who have and have not had an intervention. These are usually termed the experimental and the control groups. An attempt is made to have no other differences between the groups other than the intervention. This is usually impossible with common differences in age, gender, body habitus, and many others present. Subgroups can then usually be formed so as to keep the differences to a non-clinically significant level.
\n", 241 | "What separates this type of study from other experimental studies is the fact that data points are collected on both groups at the same time, i.e., they run concurrently.
\n", 242 | "When both the subject and the observer are unaware of which group the subject belongs to, the term *double-blinded* is used. If only the subject is unaware, it is termed a *blinded* study.
\n", 243 | "The pinnacle of clinical research is usually seen to be the randomized controlled trial, that is grouped under this heading of independent concurrent trials, with the double-blinded form being the pinnacle of medical research. It is often the strongest form of evidence to prove causation.
" 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "## Trials with self-controls" 251 | ] 252 | }, 253 | { 254 | "cell_type": "markdown", 255 | "metadata": {}, 256 | "source": [ 257 | "Here, the test subjects form their own controls. These studies are similar to cohort studies, expect for the fact that an intervention took place.
\n", 258 | "The most elegant subtype here is the cross-over study. Two groups are formed each with their own intervention. Most commonly one group will receive a placebo. A period where no intervention takes place follows and then restarted, but interchanged between the individuals in the two groups.
" 259 | ] 260 | }, 261 | { 262 | "cell_type": "markdown", 263 | "metadata": {}, 264 | "source": [ 265 | "## Trials with external controls" 266 | ] 267 | }, 268 | { 269 | "cell_type": "markdown", 270 | "metadata": {}, 271 | "source": [ 272 | "An appropriate example would be the use of previously published groups as controls for analysing against a current intervention group. These are termed *historical controls*. It is commonly used in oncological research where an effective treatment does not yet exist.
\n", 273 | "The risk exists in the choice of the control group, a choice that may influence and exacerbate the findings of the outcome of the statistical analysis.
" 274 | ] 275 | }, 276 | { 277 | "cell_type": "markdown", 278 | "metadata": {}, 279 | "source": [ 280 | "## Uncontrolled trials" 281 | ] 282 | }, 283 | { 284 | "cell_type": "markdown", 285 | "metadata": {}, 286 | "source": [ 287 | "In these studies an intervention takes place, but there are no controls. The hypothesis is that there will be varying outcomes and reasons for these can be elucidated from the data. No attempt is made to evaluate the intervention itself, as it is not being compared to either a placebo or an alternative form of intervention.
" 288 | ] 289 | }, 290 | { 291 | "cell_type": "markdown", 292 | "metadata": {}, 293 | "source": [ 294 | "## Meta-analysis" 295 | ] 296 | }, 297 | { 298 | "cell_type": "markdown", 299 | "metadata": {}, 300 | "source": [ 301 | "A *meta-analysis* uses pre-existing research and combines their results to permit an overall conclusion. It is especially helpful if the individual studies that make up the analysis are of inadequate size or power.
" 302 | ] 303 | }, 304 | { 305 | "cell_type": "markdown", 306 | "metadata": {}, 307 | "source": [ 308 | "## Reviews" 309 | ] 310 | }, 311 | { 312 | "cell_type": "markdown", 313 | "metadata": {}, 314 | "source": [ 315 | "As in the case of a meta-analysis various papers are included based on a particular factor. The discussion around this amalgamation, though, remains qualitative and no quantitative assessment and summary of findings are included.
" 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "execution_count": null, 321 | "metadata": { 322 | "collapsed": false 323 | }, 324 | "outputs": [], 325 | "source": [] 326 | } 327 | ], 328 | "metadata": { 329 | "kernelspec": { 330 | "display_name": "Python 3", 331 | "language": "python", 332 | "name": "python3" 333 | }, 334 | "language_info": { 335 | "codemirror_mode": { 336 | "name": "ipython", 337 | "version": 3 338 | }, 339 | "file_extension": ".py", 340 | "mimetype": "text/x-python", 341 | "name": "python", 342 | "nbconvert_exporter": "python", 343 | "pygments_lexer": "ipython3", 344 | "version": "3.4.3" 345 | } 346 | }, 347 | "nbformat": 4, 348 | "nbformat_minor": 0 349 | } 350 | -------------------------------------------------------------------------------- /Chapter09_Data_types.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "Setting up a fancy stylesheet" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": { 14 | "collapsed": false 15 | }, 16 | "outputs": [ 17 | { 18 | "data": { 19 | "text/html": [ 20 | "\n", 21 | "\n", 22 | "\n", 23 | "\n", 24 | "\n", 25 | "\n", 26 | "\n", 109 | "\n" 110 | ], 111 | "text/plain": [ 112 | "In order to conduct a study, information is captured in a spreadsheet or in a database. Examples of this include demographic data such as patient gender and age, or specific medical laboratory values such as leukocyte count. Each of these are referred to as variables. Variables can be divided into two main groups, called *categorical* and *numerical* and the start of this chapter deals with this form of classification. It greatly influences the type of statistical analysis that can be performed.
\n", 145 | "Later in the chapter, I introduce another classification system. Again, there are two main types, named *discrete* and *continuous*.
" 146 | ] 147 | }, 148 | { 149 | "cell_type": "markdown", 150 | "metadata": {}, 151 | "source": [ 152 | "## Categorical and numerical data types" 153 | ] 154 | }, 155 | { 156 | "cell_type": "markdown", 157 | "metadata": {}, 158 | "source": [ 159 | "*Categorical data* is data that either represents words, or if numbers, have no mathematical value. Cathegorical data is further divided into *nominal* and *ordinal* data types.
\n", 160 | "
A good example of the nominal categorical data type is disease names. Disease names can be grouped and counted, but no mathematical procedure can be performed on the names themselves.
\n", 161 | "Ordinal categorical data types also represent words, but in this instance, they can be ranked. The Likert scale, used for surveys, is a good example. The subjects completing the survey may be asked to rate their opinion to a statement by selecting from a list either: strongly disagree, disagree, neither agree nor disagree, agree, or finally, strongly agree. There is definitely some order to this categorical type. It can even be expressed as numbers, for instance subjects can be asked to rate how much they agree with a statement, with the choices being one to 10. It should be clear that although numbers can be used, no mathematical operation can be performed with this data. It would be pointless to suggest that a subject who chooses four, agrees twice as much as one who chooses two. It is incorrect to calculate the mean or standard deviation from this data!
\n", 162 | "Numerical data is likewise divided into two sub-types. They are *interval* and *ratio*. As the names imply, both of these consist of numeric values only. Degrees Celsius is good example of interval numeric data. There is a clear and consistent difference between values, such as 37.1 and 36.5 deg C. It is distinguished from the ratio numerical data type, though in that no zero exists. It might be true that zero deg C exists, but it is not a true zero. Only the Kelvin temperature scale has a true zero. It is thus wrong to suggest that 20 deg C is twice as warm as 10 deg C! \n", 163 | "Any variable with a true zero such as leukocyte count, blood pressure, and many others, are ratio numeric data types." 164 | ] 165 | }, 166 | { 167 | "cell_type": "markdown", 168 | "metadata": {}, 169 | "source": [ 170 | "## Discrete and continuous data types" 171 | ] 172 | }, 173 | { 174 | "cell_type": "markdown", 175 | "metadata": {}, 176 | "source": [ 177 | "
Another way to look at data variables, is to make the distinction between discrete and continuous types. Data variables are said to be discrete when the actual values are not sub-divisible. A smallest unit of increment exists and cannot be divided any further. It is useful to introduce the concept of discrete data types by way of examples. The simplest non-medical example of a discrete data type is the flip of a coin. The values can only be heads or tails. Another famous example is the rolling of a die. We will take a look at the probabilities associated with the ouctomes of the rolling of dice in a later chapter.
\n", 178 | "With only two outcomes, the sample space of the flip of a coin is said to be *binomial*, a type of discrete variables. Examples from medical research would include the presence of a complication. If a patient has undergone a procedure or received some form of treatment, a complication of that procedure or treatment may or may not occur. Only one of two outcomes is possible, hence the distribution of the possible outcomes, called the frequency distribution, is binomial. Please be careful, though. It is not because the values are integers that make them discrete. Some variables with decimals can also be discrete, as long as they have a finite smallest increment between them that is no longer divisible. Thruth be told, it is sometimes somewhat arbitrary!
\n", 179 | "For continuous data types there are no gaps in between values. This might take a while to get use to. Consider leukocyte count, which when expressed in cells per liter, has a range in the order of *109*. Now, although each cell is a whole and cannot be subdivided and still leave a complete cell, the shear numbers that are involved in clinical practice, make “the gaps” almost infinitely divisible and leukocyte count is considered to be continuous.\n", 180 | "Even money can be difficult to come to terms with as a continuous variable. A single cent cannot be further divided. It must be understood, though, that when financial institutions work with large sums of money and have to calculate percentages of interest, fraction of cents are involved. They cannot simply disappear. Eventually a lot of money will disappear and that’s not good. Therefor, money is also a continuous variable.
" 181 | ] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "execution_count": null, 186 | "metadata": { 187 | "collapsed": false 188 | }, 189 | "outputs": [], 190 | "source": [] 191 | } 192 | ], 193 | "metadata": { 194 | "kernelspec": { 195 | "display_name": "Python 3", 196 | "language": "python", 197 | "name": "python3" 198 | }, 199 | "language_info": { 200 | "codemirror_mode": { 201 | "name": "ipython", 202 | "version": 3 203 | }, 204 | "file_extension": ".py", 205 | "mimetype": "text/x-python", 206 | "name": "python", 207 | "nbconvert_exporter": "python", 208 | "pygments_lexer": "ipython3", 209 | "version": "3.4.3" 210 | } 211 | }, 212 | "nbformat": 4, 213 | "nbformat_minor": 0 214 | } 215 | -------------------------------------------------------------------------------- /Chapter11_Measures_of_central_tendency_dispersion.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "Setting up a fancy stylesheet" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": { 14 | "collapsed": false 15 | }, 16 | "outputs": [ 17 | { 18 | "data": { 19 | "text/html": [ 20 | "\n", 21 | "\n", 22 | "\n", 23 | "\n", 24 | "\n", 25 | "\n", 26 | "\n", 109 | "\n" 110 | ], 111 | "text/plain": [ 112 | "| \n", 400 | " | File | \n", 401 | "Age | \n", 402 | "Gender | \n", 403 | "Delay | \n", 404 | "Stay | \n", 405 | "ICU | \n", 406 | "RVD | \n", 407 | "CD4 | \n", 408 | "HR | \n", 409 | "Temp | \n", 410 | "CRP | \n", 411 | "WCC | \n", 412 | "HB | \n", 413 | "Rupture | \n", 414 | "Histo | \n", 415 | "Comp | \n", 416 | "MASS | \n", 417 | "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", 422 | "1 | \n", 423 | "38 | \n", 424 | "Female | \n", 425 | "3 | \n", 426 | "6 | \n", 427 | "No | \n", 428 | "No | \n", 429 | "NaN | \n", 430 | "97 | \n", 431 | "35.2 | \n", 432 | "NaN | \n", 433 | "10.49 | \n", 434 | "10.4 | \n", 435 | "No | \n", 436 | "Yes | \n", 437 | "Yes | \n", 438 | "5 | \n", 439 | "
| 1 | \n", 442 | "2 | \n", 443 | "32 | \n", 444 | "Male | \n", 445 | "6 | \n", 446 | "10 | \n", 447 | "No | \n", 448 | "Yes | \n", 449 | "57 | \n", 450 | "109 | \n", 451 | "38.8 | \n", 452 | "45.3 | \n", 453 | "7.08 | \n", 454 | "19.8 | \n", 455 | "No | \n", 456 | "No | \n", 457 | "Yes | \n", 458 | "8 | \n", 459 | "
| 2 | \n", 462 | "3 | \n", 463 | "19 | \n", 464 | "Female | \n", 465 | "1 | \n", 466 | "16 | \n", 467 | "No | \n", 468 | "No | \n", 469 | "NaN | \n", 470 | "120 | \n", 471 | "36.3 | \n", 472 | "10.7 | \n", 473 | "13.00 | \n", 474 | "8.7 | \n", 475 | "No | \n", 476 | "No | \n", 477 | "No | \n", 478 | "3 | \n", 479 | "
| \n", 177 | " | File | \n", 178 | "Age | \n", 179 | "Gender | \n", 180 | "Delay | \n", 181 | "Stay | \n", 182 | "ICU | \n", 183 | "RVD | \n", 184 | "CD4 | \n", 185 | "HR | \n", 186 | "Temp | \n", 187 | "CRP | \n", 188 | "WCC | \n", 189 | "HB | \n", 190 | "Rupture | \n", 191 | "Histo | \n", 192 | "Comp | \n", 193 | "MASS | \n", 194 | "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", 199 | "1 | \n", 200 | "38 | \n", 201 | "Female | \n", 202 | "3 | \n", 203 | "6 | \n", 204 | "No | \n", 205 | "No | \n", 206 | "NaN | \n", 207 | "97 | \n", 208 | "35.2 | \n", 209 | "NaN | \n", 210 | "10.49 | \n", 211 | "10.4 | \n", 212 | "No | \n", 213 | "Yes | \n", 214 | "Yes | \n", 215 | "5 | \n", 216 | "
| 1 | \n", 219 | "2 | \n", 220 | "32 | \n", 221 | "Male | \n", 222 | "6 | \n", 223 | "10 | \n", 224 | "No | \n", 225 | "Yes | \n", 226 | "57 | \n", 227 | "109 | \n", 228 | "38.8 | \n", 229 | "45.3 | \n", 230 | "7.08 | \n", 231 | "19.8 | \n", 232 | "No | \n", 233 | "No | \n", 234 | "Yes | \n", 235 | "8 | \n", 236 | "
| 2 | \n", 239 | "3 | \n", 240 | "19 | \n", 241 | "Female | \n", 242 | "1 | \n", 243 | "16 | \n", 244 | "No | \n", 245 | "No | \n", 246 | "NaN | \n", 247 | "120 | \n", 248 | "36.3 | \n", 249 | "10.7 | \n", 250 | "13.00 | \n", 251 | "8.7 | \n", 252 | "No | \n", 253 | "No | \n", 254 | "No | \n", 255 | "3 | \n", 256 | "
When conducting research by scrutinizing a sample set of individuals taken from a larger population, we create statistics. We then use these statistics to infer parameter values for the greater population. It can be used to guide patient treatment and management, by inferring that our results are applicable to that larger (patient) population.
\n", 302 | "However, there is no way that we can say with absolute confidence that our statistic is exactly what would to be found in the general population. Instead we can place an arbitrary confidence around our test statistics, and infer that we are sure that the real (larger population) parameters are within that arbitrary confidence that we placed in our statistic(s).
" 303 | ] 304 | }, 305 | { 306 | "cell_type": "markdown", 307 | "metadata": {}, 308 | "source": [ 309 | "## Confidence interval around mean of a sample variable set" 310 | ] 311 | }, 312 | { 313 | "cell_type": "markdown", 314 | "metadata": {}, 315 | "source": [ 316 | "Most textbooks begin with the chapter on confidence intervals (CI's) by using an example where the population mean and standard deviation is known. This is almost never the case in healthcare research, so let's have a look at what is commonly observed.
\n", 317 | "Most of the time we are stuck with a set of variable values from our sample set and we can calculate the mean, standard deviation and standard error of that set.
\n", 318 | "In our example dataset we might look at the white cell count (WCC) on admission. We could divide our sample patients into two groups, say those with and without retroviral disease.
\n", 319 | "In the code below we make two new DataFrames, with each DataFrame object attached to a new computer variable.
" 320 | ] 321 | }, 322 | { 323 | "cell_type": "code", 324 | "execution_count": 4, 325 | "metadata": { 326 | "collapsed": false 327 | }, 328 | "outputs": [], 329 | "source": [ 330 | "RVD_neg = data[data['RVD'] == 'No'] # Only including the rows in the original DataFrame that have a Yes in the RVD column\n", 331 | "RVD_neg_appx = RVD_neg[RVD_neg['Histo'] == 'Yes'] # Extracting only those who actually had appendicitis\n", 332 | "RVD_pos = data[data['RVD'] == 'Yes'] # Doing the same but with No in the RVD column\n", 333 | "RVD_pos_appx = RVD_pos[RVD_pos['Histo'] == 'Yes'] # Extracting only those who actually had appendicitis" 334 | ] 335 | }, 336 | { 337 | "cell_type": "markdown", 338 | "metadata": {}, 339 | "source": [ 340 | "
Let's look at the first rows in each new DataFrame using the *.head()* function. Leaving the argument blank will result in a default of *5* rows.
" 341 | ] 342 | }, 343 | { 344 | "cell_type": "code", 345 | "execution_count": 5, 346 | "metadata": { 347 | "collapsed": false 348 | }, 349 | "outputs": [ 350 | { 351 | "data": { 352 | "text/html": [ 353 | "| \n", 358 | " | File | \n", 359 | "Age | \n", 360 | "Gender | \n", 361 | "Delay | \n", 362 | "Stay | \n", 363 | "ICU | \n", 364 | "RVD | \n", 365 | "CD4 | \n", 366 | "HR | \n", 367 | "Temp | \n", 368 | "CRP | \n", 369 | "WCC | \n", 370 | "HB | \n", 371 | "Rupture | \n", 372 | "Histo | \n", 373 | "Comp | \n", 374 | "MASS | \n", 375 | "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", 380 | "1 | \n", 381 | "38 | \n", 382 | "Female | \n", 383 | "3 | \n", 384 | "6 | \n", 385 | "No | \n", 386 | "No | \n", 387 | "NaN | \n", 388 | "97 | \n", 389 | "35.2 | \n", 390 | "NaN | \n", 391 | "10.49 | \n", 392 | "10.4 | \n", 393 | "No | \n", 394 | "Yes | \n", 395 | "Yes | \n", 396 | "5 | \n", 397 | "
| 7 | \n", 400 | "8 | \n", 401 | "19 | \n", 402 | "Male | \n", 403 | "2 | \n", 404 | "13 | \n", 405 | "No | \n", 406 | "No | \n", 407 | "NaN | \n", 408 | "102 | \n", 409 | "38.5 | \n", 410 | "224.6 | \n", 411 | "8.99 | \n", 412 | "15.5 | \n", 413 | "No | \n", 414 | "Yes | \n", 415 | "Yes | \n", 416 | "5 | \n", 417 | "
| 8 | \n", 420 | "9 | \n", 421 | "46 | \n", 422 | "Male | \n", 423 | "2 | \n", 424 | "9 | \n", 425 | "No | \n", 426 | "No | \n", 427 | "NaN | \n", 428 | "107 | \n", 429 | "36.2 | \n", 430 | "385.1 | \n", 431 | "10.26 | \n", 432 | "15.5 | \n", 433 | "No | \n", 434 | "Yes | \n", 435 | "No | \n", 436 | "1 | \n", 437 | "
| 9 | \n", 440 | "10 | \n", 441 | "19 | \n", 442 | "Male | \n", 443 | "7 | \n", 444 | "12 | \n", 445 | "Yes | \n", 446 | "No | \n", 447 | "NaN | \n", 448 | "105 | \n", 449 | "36.6 | \n", 450 | "123 | \n", 451 | "22.19 | \n", 452 | "13.7 | \n", 453 | "No | \n", 454 | "Yes | \n", 455 | "Yes | \n", 456 | "5 | \n", 457 | "
| 10 | \n", 460 | "11 | \n", 461 | "33 | \n", 462 | "Male | \n", 463 | "2 | \n", 464 | "14 | \n", 465 | "No | \n", 466 | "No | \n", 467 | "NaN | \n", 468 | "104 | \n", 469 | "36.8 | \n", 470 | "NaN | \n", 471 | "16.20 | \n", 472 | "12.7 | \n", 473 | "No | \n", 474 | "Yes | \n", 475 | "No | \n", 476 | "9 | \n", 477 | "
| \n", 522 | " | File | \n", 523 | "Age | \n", 524 | "Gender | \n", 525 | "Delay | \n", 526 | "Stay | \n", 527 | "ICU | \n", 528 | "RVD | \n", 529 | "CD4 | \n", 530 | "HR | \n", 531 | "Temp | \n", 532 | "CRP | \n", 533 | "WCC | \n", 534 | "HB | \n", 535 | "Rupture | \n", 536 | "Histo | \n", 537 | "Comp | \n", 538 | "MASS | \n", 539 | "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 4 | \n", 544 | "5 | \n", 545 | "28 | \n", 546 | "Female | \n", 547 | "3 | \n", 548 | "3 | \n", 549 | "No | \n", 550 | "Yes | \n", 551 | "491 | \n", 552 | "115 | \n", 553 | "37.1 | \n", 554 | "51.6 | \n", 555 | "21.98 | \n", 556 | "13.4 | \n", 557 | "No | \n", 558 | "Yes | \n", 559 | "No | \n", 560 | "7 | \n", 561 | "
| 6 | \n", 564 | "7 | \n", 565 | "36 | \n", 566 | "Male | \n", 567 | "6 | \n", 568 | "14 | \n", 569 | "No | \n", 570 | "Yes | \n", 571 | "203 | \n", 572 | "95 | \n", 573 | "37.5 | \n", 574 | "NaN | \n", 575 | "16.21 | \n", 576 | "17.7 | \n", 577 | "Yes | \n", 578 | "Yes | \n", 579 | "Yes | \n", 580 | "7 | \n", 581 | "
| 19 | \n", 584 | "20 | \n", 585 | "33 | \n", 586 | "Female | \n", 587 | "1 | \n", 588 | "19 | \n", 589 | "No | \n", 590 | "Yes | \n", 591 | "203 | \n", 592 | "123 | \n", 593 | "37.5 | \n", 594 | "46.5 | \n", 595 | "8.00 | \n", 596 | "10.4 | \n", 597 | "No | \n", 598 | "Yes | \n", 599 | "Yes | \n", 600 | "8 | \n", 601 | "
| 31 | \n", 604 | "32 | \n", 605 | "39 | \n", 606 | "Male | \n", 607 | "2 | \n", 608 | "10 | \n", 609 | "No | \n", 610 | "Yes | \n", 611 | "102 | \n", 612 | "100 | \n", 613 | "36.8 | \n", 614 | "208 | \n", 615 | "20.43 | \n", 616 | "12.9 | \n", 617 | "Yes | \n", 618 | "Yes | \n", 619 | "No | \n", 620 | "1 | \n", 621 | "
| 32 | \n", 624 | "33 | \n", 625 | "33 | \n", 626 | "Male | \n", 627 | "0 | \n", 628 | "21 | \n", 629 | "No | \n", 630 | "Yes | \n", 631 | "266 | \n", 632 | "100 | \n", 633 | "39.7 | \n", 634 | "194.1 | \n", 635 | "7.31 | \n", 636 | "10.7 | \n", 637 | "Yes | \n", 638 | "Yes | \n", 639 | "Yes | \n", 640 | "3 | \n", 641 | "
The easiest way to calculate the confidence intervals is using the *scipy.stats* function *bayes_mvs*. It actually gives nine results, three rows of three values each. The first row gives the mean and then the lower as well as the upper bound of the stated confidence interval.
\n", 683 | "The second row does the same but for the variance. The last row calculates the values for the standard deviation.
\n", 684 | "Let's first describe our two DataFrames then calculate the CI's.
" 685 | ] 686 | }, 687 | { 688 | "cell_type": "code", 689 | "execution_count": 7, 690 | "metadata": { 691 | "collapsed": false 692 | }, 693 | "outputs": [ 694 | { 695 | "data": { 696 | "text/plain": [ 697 | "count 78.000000\n", 698 | "mean 14.044103\n", 699 | "std 4.347939\n", 700 | "min 6.610000\n", 701 | "25% 11.270000\n", 702 | "50% 13.655000\n", 703 | "75% 16.305000\n", 704 | "max 26.400000\n", 705 | "Name: WCC, dtype: float64" 706 | ] 707 | }, 708 | "execution_count": 7, 709 | "metadata": {}, 710 | "output_type": "execute_result" 711 | } 712 | ], 713 | "source": [ 714 | "RVD_neg_appx['WCC'].describe() # Describing the WCC column in the RVD_neg DataFrame" 715 | ] 716 | }, 717 | { 718 | "cell_type": "code", 719 | "execution_count": 8, 720 | "metadata": { 721 | "collapsed": false 722 | }, 723 | "outputs": [ 724 | { 725 | "data": { 726 | "text/plain": [ 727 | "count 40.000000\n", 728 | "mean 15.773250\n", 729 | "std 4.934148\n", 730 | "min 2.950000\n", 731 | "25% 12.475000\n", 732 | "50% 15.575000\n", 733 | "75% 19.920000\n", 734 | "max 24.890000\n", 735 | "Name: WCC, dtype: float64" 736 | ] 737 | }, 738 | "execution_count": 8, 739 | "metadata": {}, 740 | "output_type": "execute_result" 741 | } 742 | ], 743 | "source": [ 744 | "RVD_pos_appx['WCC'].describe() # Describing the WCC column in the RVD_pos DataFrame" 745 | ] 746 | }, 747 | { 748 | "cell_type": "markdown", 749 | "metadata": {}, 750 | "source": [ 751 | "Now for the CI's. The *bayes_mvs()* function takes two arguments, the first is the values themselves and the second is the required confidence interval. In this example I want the 95% CI's. I also make sure to discard all non-numerical values.
" 752 | ] 753 | }, 754 | { 755 | "cell_type": "code", 756 | "execution_count": 9, 757 | "metadata": { 758 | "collapsed": false 759 | }, 760 | "outputs": [ 761 | { 762 | "data": { 763 | "text/plain": [ 764 | "(Mean(statistic=14.044102564102564, minmax=(13.063793818374444, 15.024411309830684)),\n", 765 | " Variance(statistic=19.408694495726504, minmax=(14.110883384251707, 26.648893927036038)),\n", 766 | " Std_dev(statistic=4.3908697719337093, minmax=(3.7564455785025959, 5.1622566700074142)))" 767 | ] 768 | }, 769 | "execution_count": 9, 770 | "metadata": {}, 771 | "output_type": "execute_result" 772 | } 773 | ], 774 | "source": [ 775 | "bayes_mvs(RVD_neg_appx['WCC'].dropna(), 0.95)" 776 | ] 777 | }, 778 | { 779 | "cell_type": "code", 780 | "execution_count": 10, 781 | "metadata": { 782 | "collapsed": false 783 | }, 784 | "outputs": [ 785 | { 786 | "data": { 787 | "text/plain": [ 788 | "(Mean(statistic=15.773250000000001, minmax=(14.195232891628422, 17.35126710837158)),\n", 789 | " Variance(statistic=25.661807500000002, minmax=(16.336646621396056, 40.140096800828552)),\n", 790 | " Std_dev(statistic=5.0316399443801929, minmax=(4.041861776631662, 6.33562126399839)))" 791 | ] 792 | }, 793 | "execution_count": 10, 794 | "metadata": {}, 795 | "output_type": "execute_result" 796 | } 797 | ], 798 | "source": [ 799 | "bayes_mvs(RVD_pos_appx['WCC'].dropna(), 0.95)" 800 | ] 801 | }, 802 | { 803 | "cell_type": "markdown", 804 | "metadata": {}, 805 | "source": [ 806 | "Let's put down in words what we have found from the calculations above. We could state in a paper that there were 78 patients in the RVD negative group with acute appendicitis. They had a mean WCC of 14.04 on admission, with a 95% confidence interval of 13.06 to 15.02. There were 40 RVD positive patients with an average admission WCC of 15.77 (95% CI 14.20 to 17.35). (Note the mathematical rounding).
\n", 807 | "However, what do we mean by this 95% CI? Well, by our statistical anlaysis, we found a mean WCC of 15.77 in RVD positive patients. If want want to infer this on the larger population we will have to say that according to our analysis we think that the true population admission WCC value out there for all RVD positive patients is between two values. We can state those two values (lower and upper) with a set level of confidence. Here, we are 95% confident that in the larger population out there, we think the true value lies between 14.20 and 17.35.
" 808 | ] 809 | }, 810 | { 811 | "cell_type": "markdown", 812 | "metadata": {}, 813 | "source": [ 814 | "Let's drop our CI to 80%. What do you expect to happen? Well, the lower and upper values will be closer to the mean, making us less confident that our *prediction* (inference) is correct. Let's see:
" 815 | ] 816 | }, 817 | { 818 | "cell_type": "code", 819 | "execution_count": 11, 820 | "metadata": { 821 | "collapsed": false 822 | }, 823 | "outputs": [ 824 | { 825 | "data": { 826 | "text/plain": [ 827 | "(Mean(statistic=15.773250000000001, minmax=(14.756206821106424, 16.790293178893577)),\n", 828 | " Variance(statistic=25.661807500000002, minmax=(18.742423588894685, 33.674780516226335)),\n", 829 | " Std_dev(statistic=5.0316399443801929, minmax=(4.329252081929936, 5.8029975457711798)))" 830 | ] 831 | }, 832 | "execution_count": 11, 833 | "metadata": {}, 834 | "output_type": "execute_result" 835 | } 836 | ], 837 | "source": [ 838 | "bayes_mvs(RVD_pos_appx['WCC'], 0.8)" 839 | ] 840 | }, 841 | { 842 | "cell_type": "markdown", 843 | "metadata": {}, 844 | "source": [ 845 | "Indeed, the values are now in a much narrower range of 14.76 to 16.79. We are much less confident about this, though!" 846 | ] 847 | }, 848 | { 849 | "cell_type": "markdown", 850 | "metadata": {}, 851 | "source": [ 852 | "## What lies behind these calculations" 853 | ] 854 | }, 855 | { 856 | "cell_type": "markdown", 857 | "metadata": {}, 858 | "source": [ 859 | "Let's start with the equation:
\n", 860 | "$$ \\overline { x } \\pm t{ s }_{ SE } $$\n", 861 | "Now *t* is the t-statistic. The t-statistic is calculated (or read from a table) and depends on the value of confidence desired (actually alpha value) and the degrees of freedom.
\n", 862 | "The *sSE* is the standard error of the mean or...
\n", 863 | "$$ \\frac { s }{ \\sqrt { n } } $$\n", 864 | "...the standard deviation over the square root of the sample size.
\n", 865 | "Given the degrees of freedom and alpha value, a t-distribution is calculated. It looks like a normal distribution and is different for any given parameters. Therefore, when we state a confidence interval, we calculate what middle area will be covered by that confidence, i.e. we'll cover 0.95 of the total area under the curve and calculate where on the x-axis these cut-offs fall (the t-statistic). This we multiply by the standard error of the mean, and subtract as well as add to the mean to get the lower and upper bounds of the required CI. Neat!
" 866 | ] 867 | }, 868 | { 869 | "cell_type": "markdown", 870 | "metadata": {}, 871 | "source": [ 872 | "## Bootstrapping a confidence interval" 873 | ] 874 | }, 875 | { 876 | "cell_type": "markdown", 877 | "metadata": {}, 878 | "source": [ 879 | "There is another method to calculate the CI. It involves bootstrapping or resampling. We are interested in infereing population parameters from statistics. We would like to say that the real population parameter is between such and such a value and we are confident about this *guess* at a given level.
\n", 880 | "The only knowledge we have about the population, though, is our sample set. Actually we have nothing else. What we good do is take random samples from our sample set. We need to take exactely the same number of random samples from our sample set. In order not to have identical resamples, we allow a given sample value taken to be *thrown back* into the pool and have a chance of being resampled again, and again. We can do thos thousanfs of times. With the dawn of the age of fast computers, this is no problem. Now we have as big a sample set as we want. The hope is that this is a good refelction of the population. From it we can form a distribution curve and calculate the CIs.
" 881 | ] 882 | }, 883 | { 884 | "cell_type": "code", 885 | "execution_count": 12, 886 | "metadata": { 887 | "collapsed": false 888 | }, 889 | "outputs": [ 890 | { 891 | "data": { 892 | "text/plain": [ 893 | "array([ 14.23725, 17.23825])" 894 | ] 895 | }, 896 | "execution_count": 12, 897 | "metadata": {}, 898 | "output_type": "execute_result" 899 | } 900 | ], 901 | "source": [ 902 | "bs.ci(RVD_pos_appx['WCC'].dropna(), alpha = 0.05, n_samples = 10000)\n", 903 | "# Note that we state the alpha value directly and here I use 10000 resamples" 904 | ] 905 | }, 906 | { 907 | "cell_type": "markdown", 908 | "metadata": {}, 909 | "source": [ 910 | "I can now given the CIs as the values in the array above. Note, it will be different everytime you do it!
" 911 | ] 912 | }, 913 | { 914 | "cell_type": "code", 915 | "execution_count": null, 916 | "metadata": { 917 | "collapsed": false 918 | }, 919 | "outputs": [], 920 | "source": [] 921 | } 922 | ], 923 | "metadata": { 924 | "kernelspec": { 925 | "display_name": "Python 3", 926 | "language": "python", 927 | "name": "python3" 928 | }, 929 | "language_info": { 930 | "codemirror_mode": { 931 | "name": "ipython", 932 | "version": 3 933 | }, 934 | "file_extension": ".py", 935 | "mimetype": "text/x-python", 936 | "name": "python", 937 | "nbconvert_exporter": "python", 938 | "pygments_lexer": "ipython3", 939 | "version": "3.4.3" 940 | } 941 | }, 942 | "nbformat": 4, 943 | "nbformat_minor": 0 944 | } 945 | -------------------------------------------------------------------------------- /Chapter19_Comparing_categorical_data.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "Setting up a fancy stylesheet" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": { 14 | "collapsed": false 15 | }, 16 | "outputs": [ 17 | { 18 | "data": { 19 | "text/html": [ 20 | "\n", 21 | "\n", 22 | "\n", 23 | "\n", 24 | "\n", 25 | "\n", 26 | "\n", 109 | "\n" 110 | ], 111 | "text/plain": [ 112 | "In the previous chapter we created two groups. These groups were categorical in nature. Group A was those without histological evidence of appendicitis and group B included those with evidence of appendicitis. Furthermore, these two groups are nominal. There is no order to with and without appendicitis.
\n", 175 | "The variable we examined was white cell count. This represented continuous data of the ratio numerical type. Now, let's construct two groups, once again of nominal ordinal type, but this time the data variable is also categorical.
" 176 | ] 177 | }, 178 | { 179 | "cell_type": "markdown", 180 | "metadata": {}, 181 | "source": [ 182 | "## Importing and examining the dataset" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "execution_count": 3, 188 | "metadata": { 189 | "collapsed": false 190 | }, 191 | "outputs": [ 192 | { 193 | "data": { 194 | "text/html": [ 195 | "| \n", 200 | " | File | \n", 201 | "Age | \n", 202 | "Gender | \n", 203 | "Delay | \n", 204 | "Stay | \n", 205 | "ICU | \n", 206 | "RVD | \n", 207 | "CD4 | \n", 208 | "HR | \n", 209 | "Temp | \n", 210 | "CRP | \n", 211 | "WCC | \n", 212 | "HB | \n", 213 | "Rupture | \n", 214 | "Histo | \n", 215 | "Comp | \n", 216 | "MASS | \n", 217 | "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", 222 | "1 | \n", 223 | "38 | \n", 224 | "Female | \n", 225 | "3 | \n", 226 | "6 | \n", 227 | "No | \n", 228 | "No | \n", 229 | "NaN | \n", 230 | "97 | \n", 231 | "35.2 | \n", 232 | "NaN | \n", 233 | "10.49 | \n", 234 | "10.4 | \n", 235 | "No | \n", 236 | "Yes | \n", 237 | "Yes | \n", 238 | "5 | \n", 239 | "
| 1 | \n", 242 | "2 | \n", 243 | "32 | \n", 244 | "Male | \n", 245 | "6 | \n", 246 | "10 | \n", 247 | "No | \n", 248 | "Yes | \n", 249 | "57 | \n", 250 | "109 | \n", 251 | "38.8 | \n", 252 | "45.3 | \n", 253 | "7.08 | \n", 254 | "19.8 | \n", 255 | "No | \n", 256 | "No | \n", 257 | "Yes | \n", 258 | "8 | \n", 259 | "
| 2 | \n", 262 | "3 | \n", 263 | "19 | \n", 264 | "Female | \n", 265 | "1 | \n", 266 | "16 | \n", 267 | "No | \n", 268 | "No | \n", 269 | "NaN | \n", 270 | "120 | \n", 271 | "36.3 | \n", 272 | "10.7 | \n", 273 | "13.00 | \n", 274 | "8.7 | \n", 275 | "No | \n", 276 | "No | \n", 277 | "No | \n", 278 | "3 | \n", 279 | "
| 3 | \n", 282 | "4 | \n", 283 | "20 | \n", 284 | "Female | \n", 285 | "2 | \n", 286 | "9 | \n", 287 | "No | \n", 288 | "Yes | \n", 289 | "NaN | \n", 290 | "120 | \n", 291 | "35.7 | \n", 292 | "77.8 | \n", 293 | "4.45 | \n", 294 | "8.8 | \n", 295 | "No | \n", 296 | "No | \n", 297 | "No | \n", 298 | "0 | \n", 299 | "
| 4 | \n", 302 | "5 | \n", 303 | "28 | \n", 304 | "Female | \n", 305 | "3 | \n", 306 | "3 | \n", 307 | "No | \n", 308 | "Yes | \n", 309 | "491 | \n", 310 | "115 | \n", 311 | "37.1 | \n", 312 | "51.6 | \n", 313 | "21.98 | \n", 314 | "13.4 | \n", 315 | "No | \n", 316 | "Yes | \n", 317 | "No | \n", 318 | "7 | \n", 319 | "
For this example I want to know if there is a difference in the incidence of histologically proven appendicitis between those with and without retroviral disease (RVD).
\n", 362 | "Think about it for a moment. I will have two groups: Group A without appendicitis and Group B with. Within each of these I will have two groups: Group I without RVD and Group II with. Form this I can create a little table called a *contingency table*. In this example the table will have two rows and two columns. For a Χ*2* test, the table can have more rows and more columns.
\n", 363 | "In order to do this test, we will have to get the values to fill in our column, manually.
" 364 | ] 365 | }, 366 | { 367 | "cell_type": "code", 368 | "execution_count": 4, 369 | "metadata": { 370 | "collapsed": false 371 | }, 372 | "outputs": [ 373 | { 374 | "data": { 375 | "text/plain": [ 376 | "Histo RVD\n", 377 | "No No 16\n", 378 | " Yes 14\n", 379 | "Yes No 80\n", 380 | " Yes 40\n", 381 | "dtype: int64" 382 | ] 383 | }, 384 | "execution_count": 4, 385 | "metadata": {}, 386 | "output_type": "execute_result" 387 | } 388 | ], 389 | "source": [ 390 | "histo_group = data.groupby(data['Histo'])\n", 391 | "histo_group['RVD'].value_counts()" 392 | ] 393 | }, 394 | { 395 | "cell_type": "markdown", 396 | "metadata": {}, 397 | "source": [ 398 | "Here, I have made use of the powerful *.groupby()* function. It splits my DataFrame into parts according to the values found in the specified column (here I chose **Histo**). I attached the new (split) DataFrame to a computer variable name *histo_group*.
\n", 399 | "I then moved on and asked the software to give me the value counts for the **RVD** column of this new DataFrame. Note how the results tell me that the DataFrame is split by **Histo** and then gives me a breakdown of results found in the requested column. It found values for *yes* and *No* and told me how many of each it found.
" 400 | ] 401 | }, 402 | { 403 | "cell_type": "markdown", 404 | "metadata": {}, 405 | "source": [ 406 | "### Creating the contingency table (matrix)" 407 | ] 408 | }, 409 | { 410 | "cell_type": "markdown", 411 | "metadata": {}, 412 | "source": [ 413 | "Now we have to construct our little two-by-two matrix. Here I will use *numpy*. Note the use of square brackets.
" 414 | ] 415 | }, 416 | { 417 | "cell_type": "code", 418 | "execution_count": 5, 419 | "metadata": { 420 | "collapsed": false 421 | }, 422 | "outputs": [ 423 | { 424 | "data": { 425 | "text/plain": [ 426 | "array([[16, 14],\n", 427 | " [80, 40]])" 428 | ] 429 | }, 430 | "execution_count": 5, 431 | "metadata": {}, 432 | "output_type": "execute_result" 433 | } 434 | ], 435 | "source": [ 436 | "histo_RVD_observed = np.array([[16, 14], [80, 40]])\n", 437 | "histo_RVD_observed" 438 | ] 439 | }, 440 | { 441 | "cell_type": "markdown", 442 | "metadata": {}, 443 | "source": [ 444 | "From the *scipy.stats* library I am going to use the *chi2_contingency()* function. It takes a single argument (my observed table above) and returns for values, hence my use of four computer variable names before the equal sign. They are:\n", 445 | "* The Χ*2* value\n", 446 | "* The *p*-value\n", 447 | "* The degrees of freedom\n", 448 | "* The expected table" 449 | ] 450 | }, 451 | { 452 | "cell_type": "code", 453 | "execution_count": 6, 454 | "metadata": { 455 | "collapsed": false 456 | }, 457 | "outputs": [], 458 | "source": [ 459 | "chi_val, p, df, expected = stats.chi2_contingency(histo_RVD_observed)" 460 | ] 461 | }, 462 | { 463 | "cell_type": "markdown", 464 | "metadata": {}, 465 | "source": [ 466 | "
Let's print each of these to the screen.
" 467 | ] 468 | }, 469 | { 470 | "cell_type": "code", 471 | "execution_count": 7, 472 | "metadata": { 473 | "collapsed": false 474 | }, 475 | "outputs": [ 476 | { 477 | "data": { 478 | "text/plain": [ 479 | "1.3183593750000002" 480 | ] 481 | }, 482 | "execution_count": 7, 483 | "metadata": {}, 484 | "output_type": "execute_result" 485 | } 486 | ], 487 | "source": [ 488 | "chi_val" 489 | ] 490 | }, 491 | { 492 | "cell_type": "code", 493 | "execution_count": 8, 494 | "metadata": { 495 | "collapsed": false 496 | }, 497 | "outputs": [ 498 | { 499 | "data": { 500 | "text/plain": [ 501 | "0.25088670393543944" 502 | ] 503 | }, 504 | "execution_count": 8, 505 | "metadata": {}, 506 | "output_type": "execute_result" 507 | } 508 | ], 509 | "source": [ 510 | "p" 511 | ] 512 | }, 513 | { 514 | "cell_type": "code", 515 | "execution_count": 9, 516 | "metadata": { 517 | "collapsed": false 518 | }, 519 | "outputs": [ 520 | { 521 | "data": { 522 | "text/plain": [ 523 | "1" 524 | ] 525 | }, 526 | "execution_count": 9, 527 | "metadata": {}, 528 | "output_type": "execute_result" 529 | } 530 | ], 531 | "source": [ 532 | "df" 533 | ] 534 | }, 535 | { 536 | "cell_type": "code", 537 | "execution_count": 10, 538 | "metadata": { 539 | "collapsed": false 540 | }, 541 | "outputs": [ 542 | { 543 | "data": { 544 | "text/plain": [ 545 | "array([[ 19.2, 10.8],\n", 546 | " [ 76.8, 43.2]])" 547 | ] 548 | }, 549 | "execution_count": 10, 550 | "metadata": {}, 551 | "output_type": "execute_result" 552 | } 553 | ], 554 | "source": [ 555 | "expected" 556 | ] 557 | }, 558 | { 559 | "cell_type": "markdown", 560 | "metadata": {}, 561 | "source": [ 562 | "So, our *p*-value was more than *0.05* and we can say that there was no difference in the rate of RVD between those with and without appendicitis.
\n", 563 | "
The expected table is quite interesting. It calculates what we would have expected given the total number in each group.
" 564 | ] 565 | }, 566 | { 567 | "cell_type": "code", 568 | "execution_count": null, 569 | "metadata": { 570 | "collapsed": false 571 | }, 572 | "outputs": [], 573 | "source": [] 574 | } 575 | ], 576 | "metadata": { 577 | "kernelspec": { 578 | "display_name": "Python 3", 579 | "language": "python", 580 | "name": "python3" 581 | }, 582 | "language_info": { 583 | "codemirror_mode": { 584 | "name": "ipython", 585 | "version": 3 586 | }, 587 | "file_extension": ".py", 588 | "mimetype": "text/x-python", 589 | "name": "python", 590 | "nbconvert_exporter": "python", 591 | "pygments_lexer": "ipython3", 592 | "version": "3.4.3" 593 | } 594 | }, 595 | "nbformat": 4, 596 | "nbformat_minor": 0 597 | } 598 | -------------------------------------------------------------------------------- /Chapter21_Sensitivity_specificity_risks_rates_odds.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "Setting up a fancy stylesheet" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": { 14 | "collapsed": false 15 | }, 16 | "outputs": [ 17 | { 18 | "data": { 19 | "text/html": [ 20 | "\n", 21 | "\n", 22 | "\n", 23 | "\n", 24 | "\n", 25 | "\n", 26 | "\n", 109 | "\n" 110 | ], 111 | "text/plain": [ 112 | "When considering the use of special investigations or even clinical findings, it is important to understand how accurate these are and how this impacts on their use and the clinical decisions we make that depend on them.
\n", 159 | "
Whereas the sensitivity and specificity are independent of the prevalence of a condition, predictive values are and incidence profoundly affects the use of their reported values. Fortunately, there are equations that can translate reported predictive values to a relevant prevalence.
\n", 160 | "
Predictive values make use of a form of Bayesian statistics. Bayesian inference makes use of past knowledge (as gained from research) to predict, in essence, the future.
\n", 161 | "In this chapter I will look into the calculations and use of sensitivity, specificity, predictive values, incidence, cumulative risk, prevalence and some ratios." 162 | ] 163 | }, 164 | { 165 | "cell_type": "markdown", 166 | "metadata": {}, 167 | "source": [ 168 | "# Sensitivity and specificity" 169 | ] 170 | }, 171 | { 172 | "cell_type": "markdown", 173 | "metadata": {}, 174 | "source": [ 175 | "## Introduction" 176 | ] 177 | }, 178 | { 179 | "cell_type": "markdown", 180 | "metadata": {}, 181 | "source": [ 182 | "
Let's start by short explanations for sensitivity and specificity in the context of healthcare. *Sensitivity* expresses the probability that a test will return a positive result given the existence of a (disease) condition. In contrast to this, *specificity* expresses the probability that the same test will return a negative result given the absence of that disease condition.
" 183 | ] 184 | }, 185 | { 186 | "cell_type": "markdown", 187 | "metadata": {}, 188 | "source": [ 189 | "## Calculating sensitivity and specifcity" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "### A simple example" 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "
Consider a research study in which, for instance, 1000 volunteers are divided into two groups. In group one, a specific disease is absolutely shown to be present. A good example of this would be tissue taken for histological examination and proved to be diseased. This histological examination will be taken as the gold standard. In group two, this disease is absolutely known to be absent.
\n", 204 | "Study participants are then investigated with a new test. Results for this test are either *positive* or *negative*. Four results follow from this.\n", 205 | "* Group with disease\n", 206 | " * A positive result\n", 207 | " * A negative result\n", 208 | "* Group B\n", 209 | " * A positive result\n", 210 | " * A negative result\n", 211 | " \n", 212 | "
We use specific words to desrcibe these results. A positive result in a participant with the disease is termed a *true positive*. A negative result in a participant with the disease is a *false negative*.
\n", 213 | "A negative result in a participant without the disese is a *true negative* and a positive in this group would be a *false positive*.
\n", 214 | "*Sensitivity* is then a simple ratio between the true positive incidence and the total number of participants with the disease. *Specificity* is the ratio between the true negative rate and the number of participants without the disease.
\n", 215 | "Sensitivity and specificity should be used and understood from the point of view of the decision to use a test in the work-up of a condition. If a test has a high sensitivity it will diagnose a high percentage of patient who have the disease. Likewise, if a test has a high specificity it will indicate a large percentage of patients who do not have that disease.
\n", 216 | "Note also, how crucial it is to have a gold standard to divide the groups with.
" 217 | ] 218 | }, 219 | { 220 | "cell_type": "markdown", 221 | "metadata": {}, 222 | "source": [ 223 | "In this simple example of 1000 participants let's suppose that 10 individuals have the disease and an overwhelming 990 do not. Of the 10 with the disease, nine tests positive (true positive) and tests negative (false negative). Of the 990 without the disease, 90 test positive (false positive) and 900 test negative (true negative).
\n", 224 | "Sensitivity (*Sn*) is a simple fraction with true positive (*PT*) in the numerator and the sum of the true positives (*PT*) and the false negatives (*NF*) (in other words, all (10) with the disease) in the denominator.
\n", 225 | "Specificity (*Sp*) has true negative (*NT*) in the numerator and the sum of the true negative (*NT*) and false positive (*PF*) (sum of all without the disease) in the denominator.
" 226 | ] 227 | }, 228 | { 229 | "cell_type": "markdown", 230 | "metadata": {}, 231 | "source": [ 232 | "$$ { S }_{ N }=\\frac { { P }_{ T } }{ { P }_{ T }+{ F }_{ N } } $$" 233 | ] 234 | }, 235 | { 236 | "cell_type": "markdown", 237 | "metadata": {}, 238 | "source": [ 239 | "$$ {{S}_{P} = \\frac {{N}_{T}}{{{N}_{T}} + {{P}_{F}}}} $$" 240 | ] 241 | }, 242 | { 243 | "cell_type": "markdown", 244 | "metadata": {}, 245 | "source": [ 246 | "Our code could look something like this:
" 247 | ] 248 | }, 249 | { 250 | "cell_type": "code", 251 | "execution_count": 3, 252 | "metadata": { 253 | "collapsed": false 254 | }, 255 | "outputs": [], 256 | "source": [ 257 | "true_positive = 9 # Create a computer variable called true_positive and give it the value 9\n", 258 | "false_negative = 1 # True and False are python code words, so don't use these as computer\n", 259 | "# variable names\n", 260 | "false_positive = 90\n", 261 | "true_negative = 900\n", 262 | "\n", 263 | "sensitivity = true_positive / (true_positive + false_negative) # A simple equation\n", 264 | "specificity = true_negative / (true_negative + false_positive) # Note to add denominator values\n", 265 | "# in brackets to make the algebra (order of arithmetic execution) work out" 266 | ] 267 | }, 268 | { 269 | "cell_type": "code", 270 | "execution_count": 4, 271 | "metadata": { 272 | "collapsed": false 273 | }, 274 | "outputs": [ 275 | { 276 | "name": "stdout", 277 | "output_type": "stream", 278 | "text": [ 279 | "The sensitivity is: 90.0 %\n" 280 | ] 281 | } 282 | ], 283 | "source": [ 284 | "print('The sensitivity is: ', (sensitivity * 100), '%')\n", 285 | "# Using the python 3.x print statement\n", 286 | "# Note the use of quotation marks for literal text\n", 287 | "# Multiplying the values with 100 to get percentage" 288 | ] 289 | }, 290 | { 291 | "cell_type": "code", 292 | "execution_count": 5, 293 | "metadata": { 294 | "collapsed": false 295 | }, 296 | "outputs": [ 297 | { 298 | "name": "stdout", 299 | "output_type": "stream", 300 | "text": [ 301 | "The specificity is: 90.9090909090909 %\n" 302 | ] 303 | } 304 | ], 305 | "source": [ 306 | "print('The specificity is: ', (specificity * 100), '%')" 307 | ] 308 | }, 309 | { 310 | "cell_type": "markdown", 311 | "metadata": {}, 312 | "source": [ 313 | "We could no say that this new test will pick up 90% of those with the disease with a positive result and 90% of those without, with a negative result.
" 314 | ] 315 | }, 316 | { 317 | "cell_type": "markdown", 318 | "metadata": {}, 319 | "source": [ 320 | "### Example using our mock data" 321 | ] 322 | }, 323 | { 324 | "cell_type": "markdown", 325 | "metadata": {}, 326 | "source": [ 327 | "
We'll do another example importing our mock data.
\n", 328 | "I won't use the powerful * pandas .groupby()* function, instead doing a laborious construction of new *DataFrame* objects, to make it easy to follow along.
" 329 | ] 330 | }, 331 | { 332 | "cell_type": "code", 333 | "execution_count": 6, 334 | "metadata": { 335 | "collapsed": false 336 | }, 337 | "outputs": [ 338 | { 339 | "data": { 340 | "text/html": [ 341 | "| \n", 346 | " | File | \n", 347 | "Age | \n", 348 | "Gender | \n", 349 | "Delay | \n", 350 | "Stay | \n", 351 | "ICU | \n", 352 | "RVD | \n", 353 | "CD4 | \n", 354 | "HR | \n", 355 | "Temp | \n", 356 | "CRP | \n", 357 | "WCC | \n", 358 | "HB | \n", 359 | "Rupture | \n", 360 | "Histo | \n", 361 | "Comp | \n", 362 | "MASS | \n", 363 | "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", 368 | "1 | \n", 369 | "38 | \n", 370 | "Female | \n", 371 | "3 | \n", 372 | "6 | \n", 373 | "No | \n", 374 | "No | \n", 375 | "NaN | \n", 376 | "97 | \n", 377 | "35.2 | \n", 378 | "NaN | \n", 379 | "10.49 | \n", 380 | "10.4 | \n", 381 | "No | \n", 382 | "Yes | \n", 383 | "Yes | \n", 384 | "5 | \n", 385 | "
| 1 | \n", 388 | "2 | \n", 389 | "32 | \n", 390 | "Male | \n", 391 | "6 | \n", 392 | "10 | \n", 393 | "No | \n", 394 | "Yes | \n", 395 | "57 | \n", 396 | "109 | \n", 397 | "38.8 | \n", 398 | "45.3 | \n", 399 | "7.08 | \n", 400 | "19.8 | \n", 401 | "No | \n", 402 | "No | \n", 403 | "Yes | \n", 404 | "8 | \n", 405 | "
| 2 | \n", 408 | "3 | \n", 409 | "19 | \n", 410 | "Female | \n", 411 | "1 | \n", 412 | "16 | \n", 413 | "No | \n", 414 | "No | \n", 415 | "NaN | \n", 416 | "120 | \n", 417 | "36.3 | \n", 418 | "10.7 | \n", 419 | "13.00 | \n", 420 | "8.7 | \n", 421 | "No | \n", 422 | "No | \n", 423 | "No | \n", 424 | "3 | \n", 425 | "
We will divide our data set into those with and without histologically proven appendicitis. The histological evaluation is our gold standard. We will now look at using a white cell count (WCC) of 12 or more as a positive result.
" 458 | ] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "execution_count": 7, 463 | "metadata": { 464 | "collapsed": false 465 | }, 466 | "outputs": [], 467 | "source": [ 468 | "appen_pos = data[data['Histo'] == 'Yes'] # A new DataFrame including only those with\n", 469 | "# histologically proven appendicitis\n", 470 | "appen_neg = data[data['Histo'] == 'No']\n", 471 | "\n", 472 | "tp_all = appen_pos[appen_pos['WCC'] >= 12] # A new DataFrame object from the appen_pos\n", 473 | "# DataFrame with a WCC of 12 or more as a positive result, hence true positive\n", 474 | "fn_all = appen_pos[appen_pos['WCC'] < 12]\n", 475 | "\n", 476 | "tn_all = appen_neg[appen_neg['WCC'] < 12]\n", 477 | "fp_all = appen_neg[appen_neg['WCC'] >= 12]" 478 | ] 479 | }, 480 | { 481 | "cell_type": "code", 482 | "execution_count": 8, 483 | "metadata": { 484 | "collapsed": false 485 | }, 486 | "outputs": [ 487 | { 488 | "data": { 489 | "text/plain": [ 490 | "count 87.000000\n", 491 | "mean 16.507471\n", 492 | "std 3.729035\n", 493 | "min 12.000000\n", 494 | "25% 13.605000\n", 495 | "50% 15.650000\n", 496 | "75% 17.720000\n", 497 | "max 26.400000\n", 498 | "Name: WCC, dtype: float64" 499 | ] 500 | }, 501 | "execution_count": 8, 502 | "metadata": {}, 503 | "output_type": "execute_result" 504 | } 505 | ], 506 | "source": [ 507 | "tp_all['WCC'].dropna().describe() # Just interested in the counts here" 508 | ] 509 | }, 510 | { 511 | "cell_type": "code", 512 | "execution_count": 9, 513 | "metadata": { 514 | "collapsed": false 515 | }, 516 | "outputs": [ 517 | { 518 | "data": { 519 | "text/plain": [ 520 | "count 31.000000\n", 521 | "mean 9.361935\n", 522 | "std 2.010896\n", 523 | "min 2.950000\n", 524 | "25% 8.195000\n", 525 | "50% 9.390000\n", 526 | "75% 10.650000\n", 527 | "max 11.990000\n", 528 | "Name: WCC, dtype: float64" 529 | ] 530 | }, 531 | "execution_count": 9, 532 | "metadata": {}, 533 | "output_type": "execute_result" 534 | } 535 | ], 536 | "source": [ 537 | "fn_all['WCC'].dropna().describe()" 538 | ] 539 | }, 540 | { 541 | "cell_type": "code", 542 | "execution_count": 10, 543 | "metadata": { 544 | "collapsed": false 545 | }, 546 | "outputs": [ 547 | { 548 | "data": { 549 | "text/plain": [ 550 | "count 18.000000\n", 551 | "mean 8.375000\n", 552 | "std 2.481184\n", 553 | "min 4.220000\n", 554 | "25% 6.795000\n", 555 | "50% 8.155000\n", 556 | "75% 10.542500\n", 557 | "max 11.700000\n", 558 | "Name: WCC, dtype: float64" 559 | ] 560 | }, 561 | "execution_count": 10, 562 | "metadata": {}, 563 | "output_type": "execute_result" 564 | } 565 | ], 566 | "source": [ 567 | "tn_all['WCC'].dropna().describe()" 568 | ] 569 | }, 570 | { 571 | "cell_type": "code", 572 | "execution_count": 11, 573 | "metadata": { 574 | "collapsed": false 575 | }, 576 | "outputs": [ 577 | { 578 | "data": { 579 | "text/plain": [ 580 | "count 11.000000\n", 581 | "mean 16.730909\n", 582 | "std 3.835021\n", 583 | "min 13.000000\n", 584 | "25% 14.155000\n", 585 | "50% 15.970000\n", 586 | "75% 17.780000\n", 587 | "max 26.190000\n", 588 | "Name: WCC, dtype: float64" 589 | ] 590 | }, 591 | "execution_count": 11, 592 | "metadata": {}, 593 | "output_type": "execute_result" 594 | } 595 | ], 596 | "source": [ 597 | "fp_all['WCC'].dropna().describe()" 598 | ] 599 | }, 600 | { 601 | "cell_type": "code", 602 | "execution_count": 12, 603 | "metadata": { 604 | "collapsed": false 605 | }, 606 | "outputs": [], 607 | "source": [ 608 | "tp = 87\n", 609 | "fn = 31\n", 610 | "tn = 18\n", 611 | "fp = 11\n", 612 | "\n", 613 | "sens = tp / (tp + fn)\n", 614 | "spec = tn / (tn + fp)" 615 | ] 616 | }, 617 | { 618 | "cell_type": "code", 619 | "execution_count": 13, 620 | "metadata": { 621 | "collapsed": false 622 | }, 623 | "outputs": [ 624 | { 625 | "name": "stdout", 626 | "output_type": "stream", 627 | "text": [ 628 | "The sensitivity of a raised WCC in appendicitis is: 73.72881355932203 %\n" 629 | ] 630 | } 631 | ], 632 | "source": [ 633 | "print('The sensitivity of a raised WCC in appendicitis is: ', (sens * 100), '%')" 634 | ] 635 | }, 636 | { 637 | "cell_type": "code", 638 | "execution_count": 14, 639 | "metadata": { 640 | "collapsed": false 641 | }, 642 | "outputs": [ 643 | { 644 | "name": "stdout", 645 | "output_type": "stream", 646 | "text": [ 647 | "The specificity of a raised WCC in appendicitis is: 62.06896551724138 %\n" 648 | ] 649 | } 650 | ], 651 | "source": [ 652 | "print('The specificity of a raised WCC in appendicitis is: ', (spec * 100), '%')" 653 | ] 654 | }, 655 | { 656 | "cell_type": "markdown", 657 | "metadata": {}, 658 | "source": [ 659 | "
There we have it. A sensitivity and specificity value for a raised WCC in our data set.
" 660 | ] 661 | }, 662 | { 663 | "cell_type": "markdown", 664 | "metadata": {}, 665 | "source": [ 666 | "# Predicitive values" 667 | ] 668 | }, 669 | { 670 | "cell_type": "markdown", 671 | "metadata": {}, 672 | "source": [ 673 | "## Introduction" 674 | ] 675 | }, 676 | { 677 | "cell_type": "markdown", 678 | "metadata": {}, 679 | "source": [ 680 | "Predictive values make use of a form of Bayesian statistics. Bayesian inference makes use of past knowledge (as gained from research) to predict, in essence, the future.
" 681 | ] 682 | }, 683 | { 684 | "cell_type": "markdown", 685 | "metadata": {}, 686 | "source": [ 687 | "In contrast to sensitivity and specificity, predictive values should be viewed from the point of view that the test was already performed. The values can either be expressed as a *positive* predictive value, which expresses the probability that a patient has a disease once the test result was positive, or as a *negative* predictive value, which expresses the probability that the patient does not have the disease, once the test result is negative.
\n", 688 | "Predictive values should be viewed only once a clear indication of the prevalence of the disease in the particular study is given, as this greatly influences the result. In order for predictive values to be used properly, they have to be converted to reflect the prevalence of a disease in the patients seen by a practicing clinician, in his or her setting.
" 689 | ] 690 | }, 691 | { 692 | "cell_type": "markdown", 693 | "metadata": {}, 694 | "source": [ 695 | "## Calculating some values" 696 | ] 697 | }, 698 | { 699 | "cell_type": "markdown", 700 | "metadata": {}, 701 | "source": [ 702 | "Considering the first research study example above, in which (very importantly and for our purposes here), participants were recruited randomly and thus accurately reflect the true prevalence of a disease (in which case the predictive values will be accurate), calculating the predictive values is quite simple.
\n", 703 | "The positive predictive (*PP*) value is the ratio between the true positive results and the sum of the true positive and false negative rates. It expresses the probability that a patient will have the disease in the case of a a positive test result.
\n", 704 | "The negative predictive (*PN*) value is the ratio between the true negative rate and the sum of the true negative and false negative rates. It expresses the probability that a patient does not have the disease, in the case of a negative test result.
" 705 | ] 706 | }, 707 | { 708 | "cell_type": "markdown", 709 | "metadata": {}, 710 | "source": [ 711 | "$$ {P}_{P} = \\frac {{P}_{T}}{{{P}_{T}} + {{F}_{P}}} $$" 712 | ] 713 | }, 714 | { 715 | "cell_type": "markdown", 716 | "metadata": {}, 717 | "source": [ 718 | "$$ {P}_{N} = \\frac {{N}_{T}}{{{N}_{T}} + {{F}_{N}}} $$" 719 | ] 720 | }, 721 | { 722 | "cell_type": "markdown", 723 | "metadata": {}, 724 | "source": [ 725 | "So, for our first example above we have:
" 726 | ] 727 | }, 728 | { 729 | "cell_type": "code", 730 | "execution_count": 15, 731 | "metadata": { 732 | "collapsed": false 733 | }, 734 | "outputs": [], 735 | "source": [ 736 | "ppv = true_positive / (true_positive + false_positive) # Positive predicitive value\n", 737 | "npv = true_negative / (true_negative + false_negative) # Negative predicitive value" 738 | ] 739 | }, 740 | { 741 | "cell_type": "code", 742 | "execution_count": 16, 743 | "metadata": { 744 | "collapsed": false 745 | }, 746 | "outputs": [ 747 | { 748 | "name": "stdout", 749 | "output_type": "stream", 750 | "text": [ 751 | "The positive predicitive value is 9.090909090909092 %\n", 752 | "The negative predicitive value is 99.88901220865705 %\n" 753 | ] 754 | } 755 | ], 756 | "source": [ 757 | "print('The positive predicitive value is ', (ppv * 100), '%')\n", 758 | "print('The negative predicitive value is ', (npv * 100), '%')" 759 | ] 760 | }, 761 | { 762 | "cell_type": "markdown", 763 | "metadata": {}, 764 | "source": [ 765 | "Just look how low the positive predicitive value is. The chance of actually having the disease when the test results comes back positive is only 9%. It should be clear that this is highly dependent on the incidince of the disease in our study population. Here we only had 10 out of 1000 patients with the disease (a 1% incidence).
\n", 766 | "If your research sample does not accurately refelct the population the predictive values can be highly skewed and not representative of the population at large.
" 767 | ] 768 | }, 769 | { 770 | "cell_type": "markdown", 771 | "metadata": {}, 772 | "source": [ 773 | "For instance, in our appendicitis study, with a WCC of less than 12 as a negative result, the negative predicitive value would only be:
" 774 | ] 775 | }, 776 | { 777 | "cell_type": "code", 778 | "execution_count": 17, 779 | "metadata": { 780 | "collapsed": false 781 | }, 782 | "outputs": [ 783 | { 784 | "name": "stdout", 785 | "output_type": "stream", 786 | "text": [ 787 | "The negative predicitive value of a normal WCC (the chance of not having appendicitis given a negative result is 36.734693877551024 %\n" 788 | ] 789 | } 790 | ], 791 | "source": [ 792 | "print('The negative predicitive value of a normal WCC (the chance of not having appendicitis given a negative result is', (tn * 100 / (tn + fn)), '%')" 793 | ] 794 | }, 795 | { 796 | "cell_type": "markdown", 797 | "metadata": {}, 798 | "source": [ 799 | "That makes no sense!
" 800 | ] 801 | }, 802 | { 803 | "cell_type": "code", 804 | "execution_count": 18, 805 | "metadata": { 806 | "collapsed": false 807 | }, 808 | "outputs": [ 809 | { 810 | "name": "stdout", 811 | "output_type": "stream", 812 | "text": [ 813 | "The positive predicitive value is : 88.77551020408163 %\n" 814 | ] 815 | } 816 | ], 817 | "source": [ 818 | "print('The positive predicitive value is :', (tp * 100 / (tp + fp)), '%')" 819 | ] 820 | }, 821 | { 822 | "cell_type": "markdown", 823 | "metadata": {}, 824 | "source": [ 825 | "## Correcting a positive predictive value (PP)" 826 | ] 827 | }, 828 | { 829 | "cell_type": "markdown", 830 | "metadata": {}, 831 | "source": [ 832 | "In order to convert positive predictive values to a population group with a different prevalence, first consider the pre-test odds (*OP*). That is the ratio between having the disease and not having the disease in the group under consideration. This is simply done by dividing the prevalence (*Pv*) by one minus the prevalence, with the prevalence being a simple fraction of those with the disease (*d*) over the total number in the study (*n*).
" 833 | ] 834 | }, 835 | { 836 | "cell_type": "markdown", 837 | "metadata": {}, 838 | "source": [ 839 | "$$ {{P}_{v} = \\frac {d}{n}} $$" 840 | ] 841 | }, 842 | { 843 | "cell_type": "markdown", 844 | "metadata": {}, 845 | "source": [ 846 | "$$ {{O}_{P} = \\frac {{P}_{v}}{1 - {{P}_{v}}}} $$" 847 | ] 848 | }, 849 | { 850 | "cell_type": "markdown", 851 | "metadata": {}, 852 | "source": [ 853 | "This result has to be multiplied by the likelihood ratio (*RL*). This is the sensitivity divided by 1 minus the specificity.
" 854 | ] 855 | }, 856 | { 857 | "cell_type": "markdown", 858 | "metadata": {}, 859 | "source": [ 860 | "$$ {{R}_{L} = \\frac {{S}_{N}}{1 - {{S}_{N}}}} $$" 861 | ] 862 | }, 863 | { 864 | "cell_type": "markdown", 865 | "metadata": {}, 866 | "source": [ 867 | "Now we get the post-test odds *OT* (multiplying the pre-test odds and the likelihood ratio):
" 868 | ] 869 | }, 870 | { 871 | "cell_type": "markdown", 872 | "metadata": {}, 873 | "source": [ 874 | "$$ {{O}_{O} = {{{O}_{P}} \\times {{R}_{L}}}} $$" 875 | ] 876 | }, 877 | { 878 | "cell_type": "markdown", 879 | "metadata": {}, 880 | "source": [ 881 | "Now we calculutate the new positive predictive value (*PN*), which is the post-test odds divided by one minus the post-test odds:
" 882 | ] 883 | }, 884 | { 885 | "cell_type": "markdown", 886 | "metadata": {}, 887 | "source": [ 888 | "$$ {{P}_{N} = {\\frac {{O}_{O}}{1 - {O}_{O}}}} $$" 889 | ] 890 | }, 891 | { 892 | "cell_type": "markdown", 893 | "metadata": {}, 894 | "source": [ 895 | "Remember that in our test example above the positive predictive value was 9.1%. Let's do that again but consider a population in which the prevalence of appendicitis is only 0.5% (0.005).
" 896 | ] 897 | }, 898 | { 899 | "cell_type": "code", 900 | "execution_count": 19, 901 | "metadata": { 902 | "collapsed": false 903 | }, 904 | "outputs": [ 905 | { 906 | "name": "stdout", 907 | "output_type": "stream", 908 | "text": [ 909 | "The positive predicitive value in a population with a prevalence of only 0.5% is: 3.5726018396846264 %\n" 910 | ] 911 | } 912 | ], 913 | "source": [ 914 | "pre_test_odds = 0.005 / (1 - 0.005)\n", 915 | "likelihood_ratio = sens / (1 - sensitivity) # Remember sensitivity was our computer variable for sensitivity\n", 916 | "# in the test example above\n", 917 | "post_test_odds = pre_test_odds * likelihood_ratio\n", 918 | "new_PPV = post_test_odds / (1 + post_test_odds)\n", 919 | "\n", 920 | "print('The positive predicitive value in a population with a prevalence of only 0.5% is: ', new_PPV * 100, '%')" 921 | ] 922 | }, 923 | { 924 | "cell_type": "markdown", 925 | "metadata": {}, 926 | "source": [ 927 | "# Incidence, cumulative risk, and prevalenceWhile major parts of these lectures are devoted to statistical analysis and comparisons between groups, it is important to consider and measure the absolute and relative frequencies with which parameters in a study present. In this section I will commence with a description of the various methods of expressing the frequency of a parameter before moving on to comparing frequencies to each other in the form of differences.
\n", 942 | "There are three main measures of frequency: incidence (incidence rate), cumulative risk (cumulative incidence) and finally prevalence. There are two main differences, namely a difference in rates and a cumulative difference in risk.
" 943 | ] 944 | }, 945 | { 946 | "cell_type": "markdown", 947 | "metadata": {}, 948 | "source": [ 949 | "## The incidence rate" 950 | ] 951 | }, 952 | { 953 | "cell_type": "markdown", 954 | "metadata": {}, 955 | "source": [ 956 | "The word rate suggests time involvement and that is indeed the case. It measures the magnitude of occurrence over time and as such can only be used in studies that follow patients over time, such as cohort studies and randomized trials.
\n", 957 | "The incidence (*RI*) is usually expressed in terms of events per person per year. This would indicate a simple equation, with division of events by person-years, with events as *en*, sample size as *n* and average follow-up as *t* with a bar (line) over it.
" 958 | ] 959 | }, 960 | { 961 | "cell_type": "markdown", 962 | "metadata": {}, 963 | "source": [ 964 | "$$ {{R}_{I} = \\frac {{e}_{n}}{n \\times \\overline{t}}} $$" 965 | ] 966 | }, 967 | { 968 | "cell_type": "markdown", 969 | "metadata": {}, 970 | "source": [ 971 | "So if five events occur in 100 patients, followed-up for on average one year would have an incidence rate of 5 per 100 patients per year.
" 972 | ] 973 | }, 974 | { 975 | "cell_type": "markdown", 976 | "metadata": {}, 977 | "source": [ 978 | "## Cumulative risk" 979 | ] 980 | }, 981 | { 982 | "cell_type": "markdown", 983 | "metadata": {}, 984 | "source": [ 985 | "Cumulative risk is also known as cumulative incidence (*IC*). It expresses the number of events as a percentage of a larger total. This equation makes use of the number of events per person involved in the study. It is less powerful than the incidence rate and due to the fact that time is not included in the calculation, it can change dramatically with shorting or extending of the study period. Following a given sample size over a longer period, for instance, might results in more events occurring. Likewise a shorter follow-up may falsely show a smaller cumulative risk, as not enough time might have passed for events to occur.
" 986 | ] 987 | }, 988 | { 989 | "cell_type": "markdown", 990 | "metadata": {}, 991 | "source": [ 992 | "$$ {{I}_{C} = \\frac {{e}_{n}}{n}} $$" 993 | ] 994 | }, 995 | { 996 | "cell_type": "markdown", 997 | "metadata": {}, 998 | "source": [ 999 | "## Prevalence" 1000 | ] 1001 | }, 1002 | { 1003 | "cell_type": "markdown", 1004 | "metadata": {}, 1005 | "source": [ 1006 | "Prevalence gives the percentage of occurrence of an event at any point in time. The calculation is a simple ratio of events and number involved (which, bar the context, is calculated in the same way as cumulative risk). Multiplying the ratio by 100%, expresses the result in percentage.
" 1007 | ] 1008 | }, 1009 | { 1010 | "cell_type": "markdown", 1011 | "metadata": {}, 1012 | "source": [ 1013 | "## Difference in rates" 1014 | ] 1015 | }, 1016 | { 1017 | "cell_type": "markdown", 1018 | "metadata": {}, 1019 | "source": [ 1020 | "Calculating the difference in the rates between two events is the most accurate method of expressing outcome. It is simply a subtraction of one rate from another. If the incidence rate of a complication using drug A is 2.1 events per 100 people per year and the incidence rate of the same complication while taking drug B is 4.5 events per 100 people per year, then the difference is simply a subtraction of one value from the other, here resulting in a difference rate of 2.4 events per 100 people per year.
" 1021 | ] 1022 | }, 1023 | { 1024 | "cell_type": "markdown", 1025 | "metadata": {}, 1026 | "source": [ 1027 | "The difference rate can also be used to calculate the very useful and often-quoted value, **number needed to treat**. The number needed to treat is the reciprocal of the difference rate. Depending on specific circumstances this value would represent the number of patients required to suffer an event.
" 1028 | ] 1029 | }, 1030 | { 1031 | "cell_type": "markdown", 1032 | "metadata": {}, 1033 | "source": [ 1034 | "## Difference in cumulative risk" 1035 | ] 1036 | }, 1037 | { 1038 | "cell_type": "markdown", 1039 | "metadata": {}, 1040 | "source": [ 1041 | "As with the difference rate, the difference in cumulative risk is merely a subtraction of one cumulative risk from another, resulting in a value expressed in the same units as the original cumulative risks. As an example, if the cumulative risk of a particular adverse event using drug A is 3.8% and the cumulative risk for the same adverse event suing drug B is 1%, then drug B a 2.8% decrease in the cumulative risk for that adverse event.
" 1042 | ] 1043 | }, 1044 | { 1045 | "cell_type": "markdown", 1046 | "metadata": {}, 1047 | "source": [ 1048 | "# Rate, risk and odds ratios" 1049 | ] 1050 | }, 1051 | { 1052 | "cell_type": "markdown", 1053 | "metadata": {}, 1054 | "source": [ 1055 | "## Rate ratios" 1056 | ] 1057 | }, 1058 | { 1059 | "cell_type": "markdown", 1060 | "metadata": {}, 1061 | "source": [ 1062 | "In the prior sections that dealt with differences, I performed simple subtraction. Ratios imply *division* and the rate ratio is the ratio of incidence rates, i.e. dividing one incidence rate by another. These divisions result in dimensionless values that require interpretation.\n", 1063 | "The *hazards ratio* is a special kind of rate ratio and looks at instantaneous incidence rates, calculated using Cox regression.
\n", 1064 | "The rate ratio can take one of three sets of values, the first being exactly *1.0*. This would require the incidence rates of two events to be equal. It is called the null value and is interpreted as no difference. A value of less than *1.0* would imply a protective effect as the causative agent in the numerator lowers the rate and a value of more than *1.0* is seen as harmful and indicates an increase in risk.
\n", 1065 | "The rate ratio is usually expressed in a very particular manner in the literature and requires some explanation. Suppose that drug A causes 2.1 events per 100 patients per year and drug B causes 4.5 events per 100 patients per year. Dividing 2.1 by 4.5 results in 0.46. Drug A can therefore be seen as having a protective effect, in as much as the value is less than *1.0*. By subtracting 0.46 from *1.0* the resultant 0.54 can be expressed as a percentage, 54%. This is the value that is usually expressed in the literature and would be used by stating that drug A reduces the rate of the particular adverse event by 54%.
\n", 1066 | "If the rate ratio was more than *1.0*, *1.0* is subtracted from the rate ratio value. As an example, the rate ratio might be 2.4. Subtracting 2.4 from *1.0* leaves 1.4. This will be expressed as an increase in the incidence rate of an event by 140%.
" 1067 | ] 1068 | }, 1069 | { 1070 | "cell_type": "markdown", 1071 | "metadata": {}, 1072 | "source": [ 1073 | "## Risk ratio" 1074 | ] 1075 | }, 1076 | { 1077 | "cell_type": "markdown", 1078 | "metadata": {}, 1079 | "source": [ 1080 | "The risk ratio can be expressed as the division of two cumulative risk values or two prevalence values. As with the rate ratio the resultant value can be subtracted from *1.0* (or *1.0* subtracted from it) and multiplied by 100%, so as to express the value as a decrease or an increase in risk or prevalence.
" 1081 | ] 1082 | }, 1083 | { 1084 | "cell_type": "markdown", 1085 | "metadata": {}, 1086 | "source": [ 1087 | "## Odds ratio" 1088 | ] 1089 | }, 1090 | { 1091 | "cell_type": "markdown", 1092 | "metadata": {}, 1093 | "source": [ 1094 | "Odds is indeed a very peculiar term and must always be interpreted with caution and in view of how common or rare an event is. Whereas rates and risk result in the probability of likelihood of occurrence (change), the odds are just that, the odds of an occurrence.
\n", 1095 | "It is useful for two reasons. Firstly, when dealing with case-control studies, it is the only valid measure of relative risk. Remember that in a case-control study the number of cases and controls are chosen beforehand. This choice commands an absolute effect on rates and risks and is set by the study design. Secondly, it is usually calculated by logistic regression (and then expressed as adjusted odds ratio as opposed to unadjusted odds ratio). Using logistic regression can compensate for confounding.
\n", 1096 | "Odds are the ratio of the probability of an event divided by the probability of the event not occurring. If the probability fo an event occuring is a half (0.5), then the probability of it not occuring is 0.5. This gives you odds of 1. Let's do another example. If the probability of an event occuring is 0.75, then the probability of it not occuring is 0.25 (1.0 - 0.75), giving odds of 3 to 1 or 3.
\n", 1097 | "For rare events, arbitrarily chosen as equal to or less than 10%, risks and odds are close in value, but when events are more common, the odds are much higher than the risk.
" 1098 | ] 1099 | }, 1100 | { 1101 | "cell_type": "markdown", 1102 | "metadata": {}, 1103 | "source": [ 1104 | "### An example" 1105 | ] 1106 | }, 1107 | { 1108 | "cell_type": "markdown", 1109 | "metadata": {}, 1110 | "source": [ 1111 | "The concept of odds is best explained by an example. Consider two parameters. The first has three values (it is categorical): low, moderate, and high, and the second is expressed as the prevalence of that parameter with respect to patients in the specific group of parameter 1.
\n", 1112 | "Let's say that for the low group the prevalence of parameter 2 is 0.9%, for the moderate group it is 2.3% and for the high group it is 3.0%.
\n", 1113 | "the risk ratio in the low group is reference (0.9% / 0.9%) = *1.0*. For the moderate group it is (2.3% / 0.9%) 2.56 and for the high group it is 3.33.
\n", 1114 | "Now, let's do the odds ratio. remember it is the ratio of the probability of something occuring and it not occuring. So for the low group it will be 0.9% divided by 99.1% (note I am expressing values in percent here), which is 0.0091. This is our reference. The odds of parameter 2 in the moderate group is 2.3% and therefore 97.7% against. The odds ratio for parameter 2 in this groups is then 2.3% / 97.7%, which is 0.0235. Dividing this by our reference 0.0091 gives an odds ratio of 2.58. For the high group we have odds for parameter 2 of 3%, thus odds (probability) against of 97%, with an odds ratio of (3 % / 97%) 0.031. This divided by our refernce (0.0091) gives an odds ratio of 3.41.
" 1115 | ] 1116 | }, 1117 | { 1118 | "cell_type": "markdown", 1119 | "metadata": {}, 1120 | "source": [ 1121 | "Again, we intepret this being above or below 1.00 and convert to percentage. So, 2.58 becomes (2.58 - 1.00) 1.58, which is 158%. The odds ratio of 3.41 becomes 241%. That is an increase in odds of parameter 2 in the high group over the low group of 241%.
" 1122 | ] 1123 | }, 1124 | { 1125 | "cell_type": "markdown", 1126 | "metadata": {}, 1127 | "source": [ 1128 | "The 10% rule applies here too. Be very carful when reading the literature. Note when parameters are expressed as risk and odds ratios. One more example will make it crystal clear.
" 1129 | ] 1130 | }, 1131 | { 1132 | "cell_type": "code", 1133 | "execution_count": 20, 1134 | "metadata": { 1135 | "collapsed": false 1136 | }, 1137 | "outputs": [], 1138 | "source": [ 1139 | "data_odds_ratio = pd.read_csv('Odds_ratio.csv')" 1140 | ] 1141 | }, 1142 | { 1143 | "cell_type": "code", 1144 | "execution_count": 21, 1145 | "metadata": { 1146 | "collapsed": false 1147 | }, 1148 | "outputs": [ 1149 | { 1150 | "data": { 1151 | "text/html": [ 1152 | "| \n", 1157 | " | Param_1 | \n", 1158 | "preval_param_2 | \n", 1159 | "risk_ratio | \n", 1160 | "meaning_risk_ratio | \n", 1161 | "odds_param_2 | \n", 1162 | "odds_ratio | \n", 1163 | "meaning_odds_ratio | \n", 1164 | "
|---|---|---|---|---|---|---|---|
| 0 | \n", 1169 | "low | \n", 1170 | "0.2 | \n", 1171 | "ref | \n", 1172 | "NaN | \n", 1173 | "0.25 | \n", 1174 | "NaN | \n", 1175 | "NaN | \n", 1176 | "
| 1 | \n", 1179 | "moderate | \n", 1180 | "0.6 | \n", 1181 | "3 | \n", 1182 | "200 | \n", 1183 | "1.50 | \n", 1184 | "6 | \n", 1185 | "500 | \n", 1186 | "
| 2 | \n", 1189 | "high | \n", 1190 | "0.9 | \n", 1191 | "4.5 | \n", 1192 | "350 | \n", 1193 | "9.00 | \n", 1194 | "36 | \n", 1195 | "3500 | \n", 1196 | "
I've imported another *csv* spreadsheet file called *Odds_ratio.csv* and show the only three rows. Here we have a much higher prevalence of an event (parameter 2) in the moderate and high groups. Note that for the high group there is a 350% increase ratio risk, but a 3500% odds ratio increase. There are examples in the literature where risk and odds values are used intermittently and without reading carefully, your impression of results can be represented in a verty skew manner.
" 1227 | ] 1228 | }, 1229 | { 1230 | "cell_type": "markdown", 1231 | "metadata": {}, 1232 | "source": [ 1233 | "For a prevalence of about 10% risk and odds ratios are very similar. With higher and higher prevalence, these become very different!
" 1234 | ] 1235 | }, 1236 | { 1237 | "cell_type": "code", 1238 | "execution_count": null, 1239 | "metadata": { 1240 | "collapsed": false 1241 | }, 1242 | "outputs": [], 1243 | "source": [] 1244 | } 1245 | ], 1246 | "metadata": { 1247 | "kernelspec": { 1248 | "display_name": "Python 3", 1249 | "language": "python", 1250 | "name": "python3" 1251 | }, 1252 | "language_info": { 1253 | "codemirror_mode": { 1254 | "name": "ipython", 1255 | "version": 3 1256 | }, 1257 | "file_extension": ".py", 1258 | "mimetype": "text/x-python", 1259 | "name": "python", 1260 | "nbconvert_exporter": "python", 1261 | "pygments_lexer": "ipython3", 1262 | "version": "3.4.3" 1263 | } 1264 | }, 1265 | "nbformat": 4, 1266 | "nbformat_minor": 0 1267 | } 1268 | -------------------------------------------------------------------------------- /IPython_Notebook_Toolbar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/juanklopper/Python_for_Medical_Statistics/b9f62c416f016c4850e02b6da74392e2a307f1f8/IPython_Notebook_Toolbar.png -------------------------------------------------------------------------------- /Launcher.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/juanklopper/Python_for_Medical_Statistics/b9f62c416f016c4850e02b6da74392e2a307f1f8/Launcher.png -------------------------------------------------------------------------------- /MOOC_Mock.csv: -------------------------------------------------------------------------------- 1 | File,Age,Gender,Delay,Stay,ICU,RVD,CD4,HR,Temp,CRP,WCC,HB,Rupture,Histo,Comp,MASS 2 | 1,38,Female,3,6,No,No,NA,97,35.2,NA,10.49,10.4,No,Yes,Yes,5 3 | 2,32,Male,6,10,No,Yes,57,109,38.8,45.3,7.08,19.8,No,No,Yes,8 4 | 3,19,Female,1,16,No,No,NA,120,36.3,10.7,13,8.7,No,No,No,3 5 | 4,20,Female,2,9,No,Yes,NA,120,35.7,77.8,4.45,8.8,No,No,No,0 6 | 5,28,Female,3,3,No,Yes,491,115,37.1,51.6,21.98,13.4,No,Yes,No,7 7 | 6,28,Female,2,13,No,Yes,312,133,38.3,92.5,10.34,9.9,No,No,Yes,6 8 | 7,36,Male,6,14,No,Yes,203,95,37.5,NA,16.21,17.7,Yes,Yes,Yes,7 9 | 8,19,Male,2,13,No,No,NA,102,38.5,224.6,8.99,15.5,No,Yes,Yes,5 10 | 9,46,Male,2,9,No,No,NA,107,36.2,385.1,10.26,15.5,No,Yes,No,1 11 | 10,19,Male,7,12,Yes,No,NA,105,36.6,123,22.19,13.7,No,Yes,Yes,5 12 | 11,33,Male,2,14,No,No,NA,104,36.8,NA,16.2,12.7,No,Yes,No,9 13 | 12,27,Female,6,7,No,No,NA,129,38.3,10.7,13.6,17.6,No,No,Yes,6 14 | 13,22,Male,1,11,No,No,NA,87,37.1,122,6.76,13.8,No,Yes,Yes,2 15 | 14,25,Male,1,6,No,No,NA,85,36.7,NA,13.2,17.8,No,Yes,No,9 16 | 15,19,Female,1,2,No,No,NA,100,36.6,6.6,8.39,16.2,No,Yes,Yes,4 17 | 16,35,Male,4,15,No,No,NA,108,39.3,14,10.6,20.2,No,Yes,Yes,8 18 | 17,25,Male,2,14,No,No,NA,130,35.9,5.4,13.96,19.2,No,Yes,No,4 19 | 18,23,Female,3,18,No,No,NA,98,37,92.5,12,10.8,No,Yes,No,2 20 | 19,64,Male,0,6,No,No,NA,129,35.2,33,15.1,17.9,No,Yes,Yes,7 21 | 20,33,Female,1,19,No,Yes,203,123,37.5,46.5,8,10.4,No,Yes,Yes,8 22 | 21,55,Male,5,10,No,No,NA,87,35.8,NA,NA,NA,No,No,Yes,3 23 | 22,22,Male,4,7,No,No,NA,100,38.5,91.8,19.24,12.8,No,No,Yes,6 24 | 23,33,Female,5,19,No,No,NA,98,37,178.3,26.19,10.7,No,No,Yes,0 25 | 24,43,Male,7,11,No,Yes,202,95,36.1,34,20,12,No,No,Yes,1 26 | 25,23,Male,5,5,No,No,NA,116,38.8,NA,8.9,18.1,No,Yes,No,2 27 | 26,23,Male,1,11,No,No,NA,91,38.3,167.5,14.81,16.6,No,Yes,Yes,0 28 | 27,32,Male,1,17,No,No,NA,103,37.9,285,12.66,20,Yes,Yes,Yes,6 29 | 28,43,Female,3,2,No,No,NA,116,37.9,NA,18.9,11.1,No,Yes,Yes,5 30 | 29,20,Female,6,9,No,Yes,290,82,38.1,Yes,10.1,13.6,No,No,No,8 31 | 30,20,Male,5,11,No,No,NA,80,37.2,23,9.92,19.9,No,Yes,No,5 32 | 31,20,Female,6,20,No,No,NA,103,37.2,NA,12.6,15.6,Yes,Yes,Yes,1 33 | 32,39,Male,2,10,No,Yes,102,100,36.8,208,20.43,12.9,Yes,Yes,No,1 34 | 33,33,Male,0,21,No,Yes,266,100,39.7,194.1,7.31,10.7,Yes,Yes,Yes,3 35 | 34,51,Female,1,17,Yes,No,NA,121,37.6,51,15.8,15.1,Yes,Yes,Yes,6 36 | 35,35,Male,1,12,No,No,NA,84,36.6,Yes,18.63,15.1,Yes,Yes,Yes,5 37 | 36,26,Male,3,15,No,No,NA,131,36.6,44.9,16.6,11.3,Yes,Yes,No,0 38 | 37,30,Male,4,18,No,No,NA,129,36.5,8.3,10.32,18.7,Yes,Yes,Yes,7 39 | 38,19,Male,0,10,No,Yes,300,94,36.4,111,15.1,16.6,Yes,Yes,No,4 40 | 39,20,Male,3,17,No,No,NA,140,38,Yes,12.6,10.1,Yes,Yes,Yes,8 41 | 40,21,Male,3,10,No,No,NA,108,36.9,38.5,10.61,14,Yes,No,No,1 42 | 41,22,Male,4,9,No,No,NA,80,38.6,79.5,17.2,13.5,Yes,Yes,No,1 43 | 42,23,Female,1,5,No,Yes,400,103,39.8,11,23.55,15.3,Yes,Yes,No,5 44 | 43,30,Female,4,2,Yes,No,NA,138,39.3,22,7.59,19.9,Yes,Yes,Yes,4 45 | 44,27,Female,6,14,No,Yes,1129,93,37.2,102.6,8.51,11.4,Yes,No,Yes,0 46 | 45,23,Male,1,9,No,No,NA,118,37.7,280.7,21.16,10,Yes,Yes,Yes,5 47 | 46,23,Male,6,4,No,No,NA,107,36.5,13.9,22.5,17.9,Yes,Yes,No,5 48 | 47,20,Male,5,21,No,Yes,400,112,37.1,NA,24.89,18.2,Yes,Yes,No,2 49 | 48,46,Female,4,9,No,Yes,500,87,37.5,12,15.21,8.7,Yes,No,Yes,7 50 | 49,43,Female,4,6,No,Yes,600,111,36.5,12.8,13.14,17.2,Yes,Yes,No,2 51 | 50,19,Male,6,19,No,Yes,700,88,37.2,95.1,20.28,10.2,Yes,Yes,No,3 52 | 51,20,Female,6,12,No,Yes,800,102,37.4,20.5,13.59,19.4,Yes,Yes,Yes,6 53 | 52,25,Female,5,17,No,Yes,900,93,38,222,19.8,19.1,Yes,Yes,No,1 54 | 53,64,Female,0,15,No,Yes,1000,130,38.5,NA,17.5,13.9,Yes,Yes,Yes,6 55 | 54,26,Female,7,10,No,Yes,303,122,37.5,32,11.5,11.2,Yes,Yes,No,5 56 | 55,27,Female,3,15,No,Yes,405,90,35.9,NA,20.7,10.1,No,Yes,Yes,6 57 | 56,23,Male,1,5,No,Yes,NA,108,35.9,NA,15.65,13,No,Yes,No,9 58 | 57,29,Male,3,9,No,Yes,NA,93,36.2,123,11.99,18.8,No,Yes,No,1 59 | 58,28,Female,7,19,No,Yes,232,95,36.8,21.7,16.38,11.1,No,Yes,Yes,6 60 | 59,29,Male,7,3,No,Yes,NA,122,35.1,Yes,16.2,15.1,No,No,Yes,7 61 | 60,25,Female,3,14,Yes,Yes,NA,85,37.1,295.8,10.94,11.2,Yes,No,Yes,8 62 | 61,37,Male,0,20,No,Yes,654,115,37.1,Yes,7.21,9.9,No,No,Yes,4 63 | 62,47,Male,0,12,No,Yes,NA,82,37.7,NA,16.32,11.7,No,No,No,9 64 | 63,21,Female,5,14,No,Yes,NA,101,36.4,123,12.5,17.3,Yes,Yes,No,8 65 | 64,28,Male,3,11,No,Yes,234,99,39,38.7,15.48,13.7,No,Yes,No,1 66 | 65,34,Female,3,9,No,Yes,NA,135,35.3,15.7,13.78,17.8,No,No,No,5 67 | 66,27,Male,2,4,No,Yes,NA,138,36.8,89.7,15.18,13.4,No,Yes,Yes,2 68 | 67,35,Male,2,5,No,Yes,754,82,36.1,25.3,12.14,16,No,Yes,Yes,6 69 | 68,35,Male,1,6,No,Yes,231,139,36.2,123,7.8,17.4,No,No,Yes,6 70 | 69,45,Female,3,13,No,Yes,1011,96,37,398.9,17.35,11,No,Yes,No,9 71 | 70,42,Male,3,13,No,Yes,234,117,35.9,9.9,12.15,17.2,No,Yes,No,5 72 | 71,20,Male,3,5,No,No,NA,100,37.7,132,14.23,12.5,No,Yes,No,6 73 | 72,22,Male,4,3,No,No,NA,136,36.9,NA,15.1,17.5,No,Yes,No,7 74 | 73,24,Male,0,17,No,No,NA,113,36.6,123,11.35,13.4,No,No,No,9 75 | 74,43,Male,3,9,No,No,NA,130,37.7,NA,26.4,11.2,No,Yes,Yes,6 76 | 75,22,Male,1,21,No,No,NA,92,37.4,NA,11.8,19.9,No,Yes,Yes,6 77 | 76,23,Male,6,17,No,No,NA,128,39.1,314.1,22.1,14.5,Yes,Yes,No,1 78 | 77,24,Female,6,7,No,No,NA,138,38,123,15.86,10,No,Yes,Yes,6 79 | 78,21,Male,3,14,No,No,NA,101,36.2,Yes,4.22,17.1,No,No,No,4 80 | 79,21,Female,0,4,No,No,NA,112,39.5,NA,16.1,16.2,Yes,Yes,Yes,7 81 | 80,48,Female,3,21,No,No,NA,110,36.4,123,9.39,18.2,No,Yes,No,3 82 | 81,28,Female,7,15,No,No,NA,82,36.5,44,8.67,8.2,No,Yes,No,8 83 | 82,22,Male,0,2,No,No,NA,102,37,66,13.1,14.6,Yes,Yes,Yes,8 84 | 83,23,Male,2,6,No,No,NA,99,37.2,77,12.2,16.2,No,Yes,Yes,8 85 | 84,27,Male,1,3,No,No,NA,86,36.7,88,6.61,15.2,No,Yes,No,2 86 | 85,34,Male,5,13,No,No,NA,94,37.2,99,10.7,18.8,Yes,Yes,No,2 87 | 86,33,Male,4,5,No,No,NA,121,37.6,69.1,14.32,19.5,Yes,Yes,No,3 88 | 87,32,Male,6,19,No,Yes,177,108,39.9,244.7,15.65,9.1,Yes,Yes,No,6 89 | 88,20,Male,5,6,No,No,NA,99,36.3,NA,8.97,18.5,Yes,Yes,No,3 90 | 89,20,Male,0,14,No,No,NA,109,36.2,Yes,12.2,9.5,Yes,Yes,Yes,9 91 | 90,24,Female,1,13,No,No,NA,81,38.3,NA,10.1,17.4,Yes,No,Yes,8 92 | 91,21,Male,7,2,No,No,NA,135,38.5,NA,24.14,10.8,Yes,Yes,No,1 93 | 92,26,Male,3,17,No,No,NA,134,37.6,34,13.69,10.4,Yes,Yes,Yes,7 94 | 93,20,Male,7,2,No,No,NA,95,35.7,Yes,10.08,8,No,Yes,No,5 95 | 94,33,Female,6,4,No,No,NA,138,36.7,NA,14.62,9,No,Yes,No,9 96 | 95,30,Male,4,16,No,No,NA,127,35.2,Yes,16.93,15,No,Yes,Yes,9 97 | 96,44,Male,0,7,No,No,NA,140,37.3,NA,9.26,18.8,No,Yes,No,5 98 | 97,36,Male,6,13,No,Yes,399,86,39.3,NA,20.86,15.6,No,Yes,Yes,6 99 | 98,38,Male,1,17,No,Yes,NA,140,36.4,44,10.17,11.9,No,Yes,Yes,7 100 | 99,24,Female,4,21,No,No,NA,125,36.9,NA,4.72,17.3,No,No,No,7 101 | 100,26,Male,2,5,No,Yes,NA,82,38.1,304.5,7.7,17.5,No,No,No,1 102 | 101,45,Male,0,3,No,No,NA,110,36.6,78,14.53,10.1,No,No,Yes,8 103 | 102,31,Male,6,2,No,Yes,46,133,36.1,104.6,2.95,8.6,No,Yes,No,2 104 | 103,21,Male,2,12,No,No,NA,92,35.7,234.2,16.34,14,Yes,Yes,No,3 105 | 104,36,Female,4,6,No,Yes,26,107,37.1,253.4,12.4,14.5,Yes,Yes,No,3 106 | 105,44,Female,3,15,No,No,NA,84,38.4,114.8,26.19,12.9,Yes,Yes,No,3 107 | 106,33,Male,1,4,No,No,NA,140,37.8,350,12.16,12.9,Yes,Yes,No,6 108 | 107,24,Female,4,4,No,No,NA,108,37.1,NA,6.31,10,No,No,No,3 109 | 108,43,Female,7,13,No,No,NA,129,36.9,25.8,6.7,13.9,No,No,Yes,8 110 | 109,22,Male,5,13,No,No,NA,125,37,129.1,11.7,10.9,No,No,No,0 111 | 110,21,Male,5,15,No,No,NA,126,36.4,61.5,15.1,17.2,No,Yes,No,5 112 | 111,39,Female,7,12,Yes,Yes,12,132,35.3,135.8,8.9,11.1,Yes,Yes,No,7 113 | 112,28,Male,2,13,No,No,NA,125,38.3,NA,13.62,NA,No,Yes,No,5 114 | 113,32,Male,0,14,No,Yes,NA,126,38.5,NA,12.9,NA,Yes,Yes,No,2 115 | 114,18,Female,4,16,No,Yes,152,90,38.6,245.4,14.57,NA,Yes,Yes,Yes,9 116 | 115,36,Male,7,5,No,No,NA,93,37.7,NA,13.1,11.4,No,Yes,No,7 117 | 116,25,Male,7,18,No,No,NA,136,37,56,11.13,11.2,No,Yes,No,4 118 | 117,54,Male,2,9,No,No,NA,96,38.5,67.7,12.63,19.9,No,Yes,No,5 119 | 118,33,Male,1,19,No,No,NA,89,38.7,289.9,16.51,19.9,Yes,Yes,Yes,4 120 | 119,61,Male,7,8,No,No,NA,116,37.9,NA,11.69,11.3,Yes,Yes,Yes,0 121 | 120,20,Male,3,18,No,Yes,207,87,38,NA,21.7,18,Yes,Yes,No,5 122 | 121,19,Male,0,8,No,No,NA,108,36.5,286.6,16.8,20.3,No,Yes,Yes,5 123 | 122,26,Male,6,8,No,No,NA,129,38.3,38.2,16.48,NA,No,Yes,Yes,5 124 | 123,19,Female,5,4,No,No,NA,100,35.8,NA,NA,NA,No,Yes,Yes,5 125 | 124,22,Female,2,7,No,No,NA,116,35.5,57.7,14.97,9.1,No,Yes,Yes,6 126 | 125,19,Male,1,9,No,No,NA,116,38.5,NA,10.91,7.9,No,No,No,7 127 | 126,18,Male,0,2,No,No,NA,103,37.3,277.9,11.97,16.1,No,Yes,Yes,5 128 | 127,23,Female,2,10,No,No,NA,105,39.5,NA,13.77,13.2,No,Yes,No,6 129 | 128,28,Male,3,21,No,No,NA,122,38.2,66.6,12.33,11,No,Yes,No,1 130 | 129,27,Female,6,16,No,No,NA,119,37.6,NA,15.45,NA,No,Yes,No,4 131 | 130,29,Male,1,5,Yes,No,NA,129,37.5,297.7,7,16.4,Yes,Yes,Yes,7 132 | 131,21,Male,2,6,No,No,NA,94,36.6,NA,16.64,14.7,Yes,Yes,No,5 133 | 132,31,Female,3,11,No,No,NA,134,36.1,162.5,14.5,18.2,Yes,Yes,No,1 134 | 133,24,Female,2,17,No,No,NA,104,38.6,NA,23.22,15.5,Yes,Yes,No,7 135 | 134,30,Male,6,16,No,No,NA,98,36.2,NA,14.1,8,No,Yes,No,8 136 | 135,31,Female,2,18,Yes,No,NA,133,39.2,290.9,17.51,20.9,No,Yes,No,5 137 | 136,31,Male,2,6,Yes,Yes,44,108,38.6,NA,12.69,12.3,No,Yes,No,6 138 | 137,28,Male,7,10,No,Yes,77,90,38.5,NA,22.7,19,No,Yes,Yes,8 139 | 138,45,Female,2,16,Yes,Yes,101,108,37.6,287.6,17.8,21.3,No,Yes,No,8 140 | 139,45,Male,6,9,Yes,Yes,300,127,37.1,39.2,17.48,NA,Yes,Yes,No,3 141 | 140,33,Female,0,3,No,No,NA,81,39.6,NA,NA,NA,Yes,Yes,No,1 142 | 141,28,Female,1,13,No,No,NA,102,37.2,58.7,15.97,10.1,Yes,No,Yes,6 143 | 142,67,Male,1,14,Yes,No,NA,108,40.2,NA,11.91,8.9,Yes,Yes,Yes,7 144 | 143,55,Male,4,13,No,No,NA,136,39.6,278.9,12.97,17.1,No,Yes,No,3 145 | 144,55,Female,0,9,No,No,NA,97,39.5,NA,14.77,14.2,No,Yes,Yes,8 146 | 145,25,Female,0,12,No,No,NA,109,38.6,67.6,13.33,12,No,Yes,Yes,1 147 | 146,31,Male,0,14,No,No,NA,120,38.1,NA,16.45,NA,No,Yes,Yes,7 148 | 147,45,Female,1,7,Yes,No,NA,120,40.6,298.7,8,17.4,No,Yes,Yes,4 149 | 148,54,Male,5,11,Yes,Yes,NA,115,38.2,NA,17.64,15.7,Yes,Yes,No,1 150 | 149,29,Male,4,6,No,Yes,45,133,41.2,163.5,15.5,19.2,Yes,Yes,Yes,9 151 | 150,55,Female,5,2,Yes,Yes,78,95,40.6,NA,24.22,16.5,Yes,Yes,No,4 152 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Python_for_Medical_Statistics 2 | YouTube course using IPython and Jupyter notebooks to do statistical analysis. 3 | -------------------------------------------------------------------------------- /Regression.csv: -------------------------------------------------------------------------------- 1 | PCT,CRP 2 | 2.6,32.6 3 | 9.8,88.2 4 | 4.9,44.6 5 | 7.7,84.2 6 | 2.3,14.3 7 | 9.7,106.8 8 | ,97.4 9 | 9.8,91.9 10 | 3.4,31.7 11 | 3.5,25 12 | 7,64.1 13 | 7.8,69.3 14 | 8.2,84.7 15 | 9.4, 16 | 5.4,45 17 | 8.1,77.9 18 | 5.5,55.8 19 | 4.7,51.4 20 | 5,59.3 21 | 8.6,92.3 22 | 5.9,54.5 23 | ,59 24 | 5.7,52.7 25 | 8.9,87 26 | 6.4,73.5 27 | 5.6,53.2 28 | 2.5,17.3 29 | 6.4,55.7 30 | 9.6,94 31 | -------------------------------------------------------------------------------- /Toolbar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/juanklopper/Python_for_Medical_Statistics/b9f62c416f016c4850e02b6da74392e2a307f1f8/Toolbar.png -------------------------------------------------------------------------------- /odds_ratio.csv: -------------------------------------------------------------------------------- 1 | Param_1,preval_param_2,risk_ratio,meaning_risk_ratio,odds_param_2,odds_ratio,meaning_odds_ratio 2 | low,0.2,ref,,0.25,, 3 | moderate,0.6,3,200,1.5,6,500 4 | high,0.9,4.5,350,9,36,3500 5 | -------------------------------------------------------------------------------- /p_graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/juanklopper/Python_for_Medical_Statistics/b9f62c416f016c4850e02b6da74392e2a307f1f8/p_graph.png -------------------------------------------------------------------------------- /style.css: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 90 | 91 | -------------------------------------------------------------------------------- /wcc_crp.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/juanklopper/Python_for_Medical_Statistics/b9f62c416f016c4850e02b6da74392e2a307f1f8/wcc_crp.xlsx --------------------------------------------------------------------------------