Back

Research: To Trust or Not To Trust (this is a questions you must ask yourself)

Don't worry I will not be diving into the world of poetry or love stories. What I do want you, the reader, to take out of this is that there is lots of clear bad research out there.  In addition there is also research that seems great which has been cleverly disguised as sound data showing genuine evidence that is just as bad. These articles can be very difficult to spot, for both the average reader and an experienced researcher. At this point I'm sure you're saying, what in the world are you talking about? Please review the two links below. Then I will walk you through the behind the scenes of how research is executed.

The research process:

  1. A research topic is created. 
  2. A research team is constructed and the topic is refined. 
  3. The refined topic is now going to be put the a blue print application including how data will be collected, what data will be collected, how data will be and could be analyzed to answer research question that stem from the research topic and/or the research topic/question it self, and of course how the research will be funded and where it will potentially be published. 
  4. The blue prints are refined to clean up all potential loose ends or errors. 
  5. The research study is presented to the academic department that is most correctly related to the field of study. 
  6. The department chair looks over the research to refine all aspects of research so that the department is satisfied with the study. 
  7. The department chair works with the lead researcher to continue refining the research study then once satisfied will sign off for research to be sent to the universities board of research review. 
  8. The board will then review the research being proposed then provide any updates or points of clarification that are needed. 
  9. These updates or points of clarification are met by the head researcher than the research is re-submitted for approval. 
  10. The board will approve the research once the study passes examination. 
  11. Now there is an approved study that is ready to be executed (this process can take several months).
  12. The research team needs to be notified that the study has been approved. 
  13. Time to execute the research. 
  14. Data collection and data cleaning will begin (this process will take weeks to months based on the sample size). 
  15. Data will be analyzed. Then data will be interpenetrated to see what the results show. Findings are shared with the research team and analyzed as a team. 
  16. Once all parties are satisfied with the findings data is described and put into words to better understand. This is where answers are found or not. Also where data is described to reflect the findings no matter how much digging and analyzing was needed. 
  17. The findings are presented to the chair of the department then any re-analyzing is performed. 
  18. Once research is approved from start to finish the study is typed and analyzed back and forth from between the research team and the department chair until the study meets the expectations. 
  19. The study is then submitted to the journal for approval...again any edits are executed if needed. 
  20. The study is now published and ready for the public. 

 

Ok, now that you have seen the very brief, readers digest, version of research behind the scenes what are your thoughts? I can imagine you go lost somewhere in the mix of the back and forth at some point....and yes that truly is how this goes. So how in the world does this process let bad research through? Well that simple not all schools have standard policies and sometimes politics...money talks. Another struggle with research is that after all this work that takes months sometimes years, there are no findings. That's a long time and sometimes a lot of money spent just to show that there are no findings from the study. This is when most errors occur. Because the research team will dig and dig at data using multiple calculations, equations, and cross referencing data tools to find some sort of significance in the data that can correlate with the research question. All this analyzing eventually stretches data thin so that the true findings are so blurred that findings are formed more by the searching for findings than the findings themselves. This is where most good research goes wrong and is produced with poor findings. This is an example of p-smudging. 

 

The article you see below discusses a well known researcher from a well known university that has fallen prey to these research hacks. This is a case that is in the spotlight but is just one of hundreds of examples that are out there. 

 

The youtube video below highlights an article a bout low carb high fat diets that has circled the news, magazines, and research journals. This very beneficial to watch and helps to better understand why you should always do your homework before taking the findings of a study at face value before looking at the details yourself. 

Back