I touched on this a little bit in the blog about a good research article and upon a suggestion from someone else decided to add a bit more about peer reviewed articles, what they are and why they are so important.
A peer reviewed journal article is basically what it sounds like, an article published in a journal that has been reviewed by a group of experts (peers) within your field of research.
The job of these experts is to analyze your your research, data and results to determine whether or not you've provided a sound study. This is an important process when it comes to research as these people are supposed to be determining whether or not you've made any mistakes in your research, done your stats wrong, were unethical, etc. If they determine you have a sound study they will then give it the thumbs up to be published.
Scientific journals are where these studies are going to be published, some are going to be better than others, but basically they will contain a set of their reviewed research papers for other people interested to read. These are not the same thing as magazine articles or articles published to the internet that rely on the writer to determine whether or not their research was sound.
Some journals you might see:
Journal of Medicinal Chemistry
Chemical Reviews
Journal of Psychiatry and Neuroscience
The American Journal of Clinical Nutrition
and more may be found here:
http://www.eurekalert.org/links.php?jrnl=A
While the peer review process is not fail safe and bad articles can get through they can at least ensure that an expert in the field other than the writer or researcher of the article has taken a look at the article and determined they approve of the research and findings.
Wednesday, October 7, 2009
Monday, October 5, 2009
Basics of a Good Research Article
Ok, so I ideally want to be doing one of these blogs a week, but somehow ended up really busy for a few weeks, I apologize.
This week we are exploring what makes a study good or bad. Scientific research papers can be pretty complicated to read and understand, but there are some basics you can use to determine if what you are reading has any weight. Keep in mind I am only using the very basics of what makes a good study, there are some things that can be tweaked and altered and your study will still be fine but for the sake of this blog we'll be using some quick and easy stuff to pick out.
First and foremost any research study you look at should be a peer-reviewed journal article. This means the article in question was looked at by a group of experts and peers in the field and determined to be an all around good study. This doesn't mean that the study is flawless, but it does help it hold weight in the real world.
The Hypothesis
This week we are exploring what makes a study good or bad. Scientific research papers can be pretty complicated to read and understand, but there are some basics you can use to determine if what you are reading has any weight. Keep in mind I am only using the very basics of what makes a good study, there are some things that can be tweaked and altered and your study will still be fine but for the sake of this blog we'll be using some quick and easy stuff to pick out.
First and foremost any research study you look at should be a peer-reviewed journal article. This means the article in question was looked at by a group of experts and peers in the field and determined to be an all around good study. This doesn't mean that the study is flawless, but it does help it hold weight in the real world.
The Hypothesis
I'll start out with one we are familiar with. The first thing you want to look at is whether the researcher chose a hypothesis that is testable. It is simply not testable at this point to state that you hypothesize that there is a planet inhabited by green people orbiting a distant star. As I have pointed out before there is simply no way to prove this.Typically when you are reading a research article a researcher will state what is known as the null hypothesis. The null hypothesis is the assumption that there is no relationship between your the variables you are testing and that any observed patterns are due merely to chance. This is important when it comes to statistical testing as a researcher will be attempting to show that there really is a difference between the data sets and they will be trying to reject or disprove the null hypothesis. The purpose of the null is to allow you to compare your hypothesis against something else. An example might be:
Hypothesis: Watering plants will help them to grow bigger.
Null Hypothesis: There will be no difference in growth of the plants due to water.
The researcher's study will attempt to determine whether or not they are able to reject the null (the plants indeed grew) or accept the null (whether the plants were watered or not made no difference).
Sample size
Hypothesis: Watering plants will help them to grow bigger.
Null Hypothesis: There will be no difference in growth of the plants due to water.
The researcher's study will attempt to determine whether or not they are able to reject the null (the plants indeed grew) or accept the null (whether the plants were watered or not made no difference).
Sample size
In general, the larger the sample a researcher is able to get the more data they will have on hand to work with and analyze. The number I use as a rule of thumb is a sample size of 30 or more. Below that, it becomes hard to determine whether or not your results can be generalized to the rest of the population. Studies will use smaller sample sizes quite frequently and effectively, but the study is usually an initial one which requires more research. Sometimes a larger sample is just not possible to find, especially if you are researching a relatively rare phenomenon. For general purposes though I would go with the 30 or more rule.
Placebos
Placebos
Whenever doing a study with human and animal participants it is important to have a placebo group; this is the group that is given no treatment and will help researchers be able to tell whether or not their results are due to chance. An example of this would be if you are testing the effectiveness of a headache pill. The pill given to one group will be the actual pill for headaches, the pill given to another group will be a sugar pill which should have no effect. Aside from this, both pills should look exactly the same. This is particularly important when dealing with medical research, as oftentimes patients will react to a treatment simply because it is a medical intervention and they expect that it will improve their condition.
Double Blind
Double Blind
We are very influenced by our surroundings, a lot of the time without even knowing it. In order to control for this in an experiment, researchers will use a double blind study. This means that the participants of the study don't know which group they are in (placebo or test group) and the researchers are also blind to what group the participants are in. This helps to ensure that the people carrying out the testing are not influencing their participants, and the participants are not influencing the study by knowing which group they are in.
P-value
P-value
This is a more confusing part of research studies- when you get to the statistics, how can you tell whether or not the study you are researching had a significant result? One way to determine this is in the P-value given for the study. The P-value is the probability that your results are due entirely to chance. Most researchers will use a value of 0.05 representing a 5% chance that the results were due to chance alone and not the effect/treatment you are studying. If a p-value comes up over 0.05 then you would fail to reject your null hypothesis (this means your study showed no effect) if it is below 0.05 then you can reject the null hypothesis (This means the effect in your study was not caused just by chance). This value can be adjusted, but as a general rule of thumb a researcher will use 0.05 or lower and what you should be on the look out for when reading a research article is that they've increased their p-value to greater than 0.05.
Meta-analysis
Meta-analysis
This isn't generally a part of one research study, but collects the data from many different research studies with similar hypotheses and combines them to see if there is a common trend. An example would be a meta-analysis study of homeopathy; a couple of studies may have shown a weak positive result, but the combination of many sets of data show a strong negative.
Now you should have a better idea of what to look for when you are checking out various information on the internet. I would however remember that I have only laid out the very basics of a research article, there is much more that goes into making a study good or not, so if you find yourself on the fence about an article I suggest finding a sciencey friend to look it over and see what they think!
Now you should have a better idea of what to look for when you are checking out various information on the internet. I would however remember that I have only laid out the very basics of a research article, there is much more that goes into making a study good or not, so if you find yourself on the fence about an article I suggest finding a sciencey friend to look it over and see what they think!
Monday, August 31, 2009
Theories and Hypotheses
This is the beginning of all science right here, I mean, how can you even begin to start an experiment, trial, test etc, without guessing what the outcome might be? This ultimately gives your experiment some direction.
Hypothesis-
As I explained last time, this is your best guess as to the outcome of your experiment. Was the light bulb out because it was bad? Was there some other reason? If you will recall, the primary rule of ALL hypotheses is that they MUST be testable and they must be able to be proven false. Its all well and good to hypothesize that there is a planet consisting entirely of tiny green men orbiting a distant star, but given current technology there's no way to test for it; you can't prove it to be true and you can't prove it false either. Ultimately you are at an impasse and have shown nothing.
Once you have completed your study your hypothesis can either then be used to modify a current theory, or in the creation of a new one.
Theories-
A typical theory will be the simplest explanation for a given set of data or a particular phenomenon. Like hypotheses, theories should be plausible and able to be tested and expanded on. Any scientist should be able to construct a study that will test a given theory. The purpose of a theory is to explain or predict observed phenomena or data, what started out as a well researched prediction (aka hypothesis), and now has the support of evidence and research to back it up. The more research and evidence you have which agrees with your theory, the stronger the support for it becomes. This is why the theory of evolution is one of the strongest theories out there– it has an overwhelming amount of scientific evidence and research which supports it. After over hundred years of research it is one of the most "complete" theories in existence.
Theory or fact?
Unlike what you may typically think of as the meaning of theory (an educated guess), scientific theories are formulated in order to explain verifiable observations (a.k.a. facts, ex: a ball will fall to the ground when you drop it). These facts are considered to be true under all conditions (ex: the sun rises in the east and sets in the west). Theories are then used to explain these relevant observed facts (how come the ball falls to the ground?). There typically will always be an instance in which a theory does not hold up so a scientist will never present his theory as a fact. An important thing to remember in science is that nothing will beyond a doubt under all conditions be proven to be true, only supported by the evidence. There will always be at least one condition in which your theory will fail to be true.
An example: You have stated that all birds can fly, you state this is a fact. You could study thousands of birds, all of which fly, never finding the one that does not, but there will always be the chance that you will find one that does not fly. All you would need in this instance is to find one bird who cannot fly (a kiwi or an ostrich for instance) and your fact is no longer true. However, if this were a theory, you would have the support of many birds that had been observed flying; your theory could be revised to state that most birds fly, with the exception of a minority which appear to have lost the ability.
Hypothesis-
As I explained last time, this is your best guess as to the outcome of your experiment. Was the light bulb out because it was bad? Was there some other reason? If you will recall, the primary rule of ALL hypotheses is that they MUST be testable and they must be able to be proven false. Its all well and good to hypothesize that there is a planet consisting entirely of tiny green men orbiting a distant star, but given current technology there's no way to test for it; you can't prove it to be true and you can't prove it false either. Ultimately you are at an impasse and have shown nothing.
Once you have completed your study your hypothesis can either then be used to modify a current theory, or in the creation of a new one.
Theories-
A typical theory will be the simplest explanation for a given set of data or a particular phenomenon. Like hypotheses, theories should be plausible and able to be tested and expanded on. Any scientist should be able to construct a study that will test a given theory. The purpose of a theory is to explain or predict observed phenomena or data, what started out as a well researched prediction (aka hypothesis), and now has the support of evidence and research to back it up. The more research and evidence you have which agrees with your theory, the stronger the support for it becomes. This is why the theory of evolution is one of the strongest theories out there– it has an overwhelming amount of scientific evidence and research which supports it. After over hundred years of research it is one of the most "complete" theories in existence.
Theory or fact?
Unlike what you may typically think of as the meaning of theory (an educated guess), scientific theories are formulated in order to explain verifiable observations (a.k.a. facts, ex: a ball will fall to the ground when you drop it). These facts are considered to be true under all conditions (ex: the sun rises in the east and sets in the west). Theories are then used to explain these relevant observed facts (how come the ball falls to the ground?). There typically will always be an instance in which a theory does not hold up so a scientist will never present his theory as a fact. An important thing to remember in science is that nothing will beyond a doubt under all conditions be proven to be true, only supported by the evidence. There will always be at least one condition in which your theory will fail to be true.
An example: You have stated that all birds can fly, you state this is a fact. You could study thousands of birds, all of which fly, never finding the one that does not, but there will always be the chance that you will find one that does not fly. All you would need in this instance is to find one bird who cannot fly (a kiwi or an ostrich for instance) and your fact is no longer true. However, if this were a theory, you would have the support of many birds that had been observed flying; your theory could be revised to state that most birds fly, with the exception of a minority which appear to have lost the ability.
Monday, August 24, 2009
The Scientific Method
While I would hope that most of us learned about this as part of your standard education (I learned it in middle school), I will review it in case you are a little rusty.
The scientific method is the set of steps each scientist needs to go through in the course of their research. You probably don't realize, but this is something we as humans do inherently quite often. Do you have a light bulb out? What are the possible reasons it could be out? Is it simply burned out? Is the electricity off? Is there a wire loose? You then decide that your light bulb is out because the bulb has burned out, your next step will be replacing the bulb to see if the light comes back on. Does that work? or if it doesn't will you go back and try another idea? If you have ever done that or something similar you have utilized the scientific method, it is the method in which we figure out the who, what, why, where and how of something.
The steps included in this method are as follows:
Observation- (You observe your light bulb is out)This has been the beginning of many a scientific study, by observing a phenomenon that you or someone else may not understand. Think about what you would like to know about this particular phenomenon. How did it happen? Why did it happen? Who or What made it happen? Where does this happen? Your next step will then be research, research, research.
Research-(You look up possible reasons a light bulb may have gone out) It may be that what you have observed has already been extensively researched, you can find the answers to the questions by reading a book or an article already written on the subject. The next thing any scientist will do when they find something that interests them is to find out everything they can, anything that has already been written. The most common way they will do this is to read through many peer reviewed journal articles. These are articles that have been read by a group of a researchers, peers and judged by them to be an adequate study. While other articles may carry useful information, peer reviewed journal articles will lay out a previous researchers study, they will show their hypothesis, testing methods, results and conclusions. Research that is not carried out adequately is not typically published in journals, while it is not a fail safe method of finding a good study it is the best method of looking at what other people have done and guaranteeing some form of quality control. Peer reviewed articles can also be useful in that they will likely be the most current research on a particular subject. It is important when looking through articles that you take note of when they have been published because this can affect the accuracy of their research (it may be that a study with a different or better result was posted later on). Once you have found many articles on the subject you are interested in you can then form a hypothesis.
Hypothesis-(You decide your light bulb has burned out) This is the next step in your study, this is what you think will be the outcome of your study, the reason you think your observed phenomena is happening. The number one important thing about your hypothesis is that it MUST be falsifiable. This is why you see many scientists becoming frustrated over the argument about whether there is a god or not. There is no way to prove there is no god, the hypothesis is unable to be falsified, it cannot be tested. I will go more into hypotheses and theories in my next blog. Once you have your hypothesis to work from, the next step will be testing that hypothesis.
Testing your Hypothesis-(You try replacing your light bulb) This is where you craft a study that will allow you to test your hypothesis. I have enough to say on testing conditions that I will be creating a separate blog about it, there are stringent designs for the proper way to test a hypothesis which will hopefully allow you to come out with a reasonable result. These involve things such as sample size, placebos, randomization, double-blind studies, etc. The way in which you test your hypothesis will depend on the hypothesis in question, it is important to design a study that has accurate results. Once you are done with your testing portion you will then move on to collecting and analyzing your data.
Collecting/Analyzing Data-(Did the light come back on?) Hopefully you've been collecting data all along while doing your research, but it will be important to organize it in such a way that you can run your statistical analysis on it without any issues. This is where you determine whether or not your hypothesis is supported or rejected, if your results support your hypothesis then yay! move on to your next step of trying to get your results published or making them known! If your hypothesis is not supported you can either publish the results of the study concluding that your research did not support your hypothesis or you can go back to the hypothesis and rework it to start the study anew. You cannot go back and alter your hypothesis so that it supports your results. You may also take a look at how your study was done to determine whether or not there was something in the testing methods that caused you to get the results that you did, was your sample size too small? was your sample size not random enough? did you do your statistics wrong (this is a common one)? or was there something else? I would encourage researchers to point out when they find they do not come up with a supported hypothesis as then we more able to define a direction in which to head for further study of a subject. In any case, once you have concluded your analysis of the data your next step will be reporting the results.
Reporting your Results-(You tell the rest of your household you've got the lights back on!) The best way to give your study credence is to have your results published in (you guessed it) a peer reviewed journal article. In addition to specific guidelines for testing there are also specific guidelines for writing up a research paper. You will have to discuss the research you did prior to your study, your hypothesis, the methods of research you used and your results and conclusions for the study. This allows anyone who reads your research to be able to determine whether or not your study was a good one. chances are if a researcher refuses to release any of their data, though they report results, then their study is probably bad. The mark of a good study is that you are able to replicate it, and if a researcher refuses to release their methods and research it will be impossible to replicate their study.
Next, we will go deeper into Hypotheses and Theories!
The scientific method is the set of steps each scientist needs to go through in the course of their research. You probably don't realize, but this is something we as humans do inherently quite often. Do you have a light bulb out? What are the possible reasons it could be out? Is it simply burned out? Is the electricity off? Is there a wire loose? You then decide that your light bulb is out because the bulb has burned out, your next step will be replacing the bulb to see if the light comes back on. Does that work? or if it doesn't will you go back and try another idea? If you have ever done that or something similar you have utilized the scientific method, it is the method in which we figure out the who, what, why, where and how of something.
The steps included in this method are as follows:
Observation- (You observe your light bulb is out)This has been the beginning of many a scientific study, by observing a phenomenon that you or someone else may not understand. Think about what you would like to know about this particular phenomenon. How did it happen? Why did it happen? Who or What made it happen? Where does this happen? Your next step will then be research, research, research.
Research-(You look up possible reasons a light bulb may have gone out) It may be that what you have observed has already been extensively researched, you can find the answers to the questions by reading a book or an article already written on the subject. The next thing any scientist will do when they find something that interests them is to find out everything they can, anything that has already been written. The most common way they will do this is to read through many peer reviewed journal articles. These are articles that have been read by a group of a researchers, peers and judged by them to be an adequate study. While other articles may carry useful information, peer reviewed journal articles will lay out a previous researchers study, they will show their hypothesis, testing methods, results and conclusions. Research that is not carried out adequately is not typically published in journals, while it is not a fail safe method of finding a good study it is the best method of looking at what other people have done and guaranteeing some form of quality control. Peer reviewed articles can also be useful in that they will likely be the most current research on a particular subject. It is important when looking through articles that you take note of when they have been published because this can affect the accuracy of their research (it may be that a study with a different or better result was posted later on). Once you have found many articles on the subject you are interested in you can then form a hypothesis.
Hypothesis-(You decide your light bulb has burned out) This is the next step in your study, this is what you think will be the outcome of your study, the reason you think your observed phenomena is happening. The number one important thing about your hypothesis is that it MUST be falsifiable. This is why you see many scientists becoming frustrated over the argument about whether there is a god or not. There is no way to prove there is no god, the hypothesis is unable to be falsified, it cannot be tested. I will go more into hypotheses and theories in my next blog. Once you have your hypothesis to work from, the next step will be testing that hypothesis.
Testing your Hypothesis-(You try replacing your light bulb) This is where you craft a study that will allow you to test your hypothesis. I have enough to say on testing conditions that I will be creating a separate blog about it, there are stringent designs for the proper way to test a hypothesis which will hopefully allow you to come out with a reasonable result. These involve things such as sample size, placebos, randomization, double-blind studies, etc. The way in which you test your hypothesis will depend on the hypothesis in question, it is important to design a study that has accurate results. Once you are done with your testing portion you will then move on to collecting and analyzing your data.
Collecting/Analyzing Data-(Did the light come back on?) Hopefully you've been collecting data all along while doing your research, but it will be important to organize it in such a way that you can run your statistical analysis on it without any issues. This is where you determine whether or not your hypothesis is supported or rejected, if your results support your hypothesis then yay! move on to your next step of trying to get your results published or making them known! If your hypothesis is not supported you can either publish the results of the study concluding that your research did not support your hypothesis or you can go back to the hypothesis and rework it to start the study anew. You cannot go back and alter your hypothesis so that it supports your results. You may also take a look at how your study was done to determine whether or not there was something in the testing methods that caused you to get the results that you did, was your sample size too small? was your sample size not random enough? did you do your statistics wrong (this is a common one)? or was there something else? I would encourage researchers to point out when they find they do not come up with a supported hypothesis as then we more able to define a direction in which to head for further study of a subject. In any case, once you have concluded your analysis of the data your next step will be reporting the results.
Reporting your Results-(You tell the rest of your household you've got the lights back on!) The best way to give your study credence is to have your results published in (you guessed it) a peer reviewed journal article. In addition to specific guidelines for testing there are also specific guidelines for writing up a research paper. You will have to discuss the research you did prior to your study, your hypothesis, the methods of research you used and your results and conclusions for the study. This allows anyone who reads your research to be able to determine whether or not your study was a good one. chances are if a researcher refuses to release any of their data, though they report results, then their study is probably bad. The mark of a good study is that you are able to replicate it, and if a researcher refuses to release their methods and research it will be impossible to replicate their study.
Next, we will go deeper into Hypotheses and Theories!
First Post!
we'll see how this takes off, it is entirely possible I will have this fabulous idea and then let it fall by the wayside.
In any case, the reason for creating this blog is to hopefully educate some people about the basics that come with science as well as the topics we see most frequently covered in the media. Typically I have noticed the mainstream media has a poor understanding of science in general and frequently gets its articles wrong. In order to help combat this, I will be covering things we are seeing in the news quite often alternative therapies, vaccines, medicines, drugs, mental illness, etc.
It is my hope that the more public understanding about these things the better we will be able to tell bad reporting from good or good studies from bad.
In any case, the reason for creating this blog is to hopefully educate some people about the basics that come with science as well as the topics we see most frequently covered in the media. Typically I have noticed the mainstream media has a poor understanding of science in general and frequently gets its articles wrong. In order to help combat this, I will be covering things we are seeing in the news quite often alternative therapies, vaccines, medicines, drugs, mental illness, etc.
It is my hope that the more public understanding about these things the better we will be able to tell bad reporting from good or good studies from bad.
Subscribe to:
Posts (Atom)
