Workshops

Introduction

On your first day at a new lab or on a new project, you will undoubtedly be given research papers to read to familiarize yourself with the work that has already been done in the field. As you progress on a project, you will constantly consult the literature to explain the results you observe and to give your research direction. When trying out new techniques, you should inspect the methodology sections of papers to aid in the design of your experiment. In general, scientific literature will be a constant teacher and guide throughout your research journey. 

With any established field in STEM, you can find thousands of papers published on any subject. It is a challenge to learn to effectively comb through all this literature to find the few dozen papers that will be useful to help answer your specific question. This is further complicated by the fact that scientific papers can be incredibly dense with jargon and techniques you are not familiar with. This kind of technical writing can be confusing (and boring), making a scientist's greatest tool also their greatest dread. 

We have designed this workshop content to give you young scientists a few tips on how to navigate scientific literature. Follow along and keep these tips in mind as you progress with your project. 

How to Read Scientific Papers

As you work through the lessons below, remember that no one is naturally "good" at reading scientific literature! Furthermore, you aren't taught how to read scientific papers in school. The only way to become more efficient is to practice consistently. Don't be discouraged if you find this task difficult at first or if it takes you several hours to get through a paper. Sucking at something is the first step towards being sorta good at something.

Every few months or so, revisit the first paper you read and see how much more you understand compared to your first time reading that paper. This is a great to really view your progress and feel more confident in reading scientific papers.

Read the article linked below to see just how intimidating and overwhelming it feels to read scientific articles, even for established scientists.

To find a “good” paper for your research, you will have to sift through many “losers”.

As mentioned above, you can find thousands of papers on your topic. The majority of these are not going to be applicable to your specific question. A “good” paper can help explain some result you are seeing in an experiment, can explain the scientific concept you are dealing with, or can give new ideas for the direction of your research. Your job is to learn to assess the usefulness of a paper in the least amount of time possible. Tips for this:

1. What journal was this paper published in (importance of impact factor)? How long ago? How many citations does it have?
   1. Papers from top tier journals tend to be “good” examples of the current state of research in a field. A paper in a low tier journal can still be good. I would base this on how many times it has been cited. 
   2. A recently published paper is likely to be up to date in terms of the science and the methodology. Older papers can still be useful. For some fields or concepts, the bulk of the literature on them was published a long time ago. 
2. Read the Title and Abstract first
   1. This may be an obvious one. Pay attention to the vocabulary of the title, are the words related to your research? The abstract is a summary of the introduction, the scientific question, and the results. If the abstract is applicable to your project, then put the paper in the “good” pile…for now. 
3. Keep track of your papers.
   1. When you find a “good” paper, SAVE IT! Either bookmark it or download it. You might not have time to properly read it or might want to consult it later. It is tedious to try to find a paper again. 

4. For an example on how to comb through literature, watch the following video:
   
   [video:https://youtu.be/NugRVhBx_Uo?si=_wHHnIypndmi6dGs]

Chronological order is not the best order.

Reading a paper from start to finish can take a couple of hours. There are more efficient ways to understand the main idea of a paper and whether it will be useful for you. If a paper is useful, you will be reading it many times over. But for the first pass, an efficient skim can tell you if it’s worth more of your time. 

To skim a paper efficiently, it helps to be familiar with its structure. The usual format is Abstract, Introduction, Methods, Results, Discussion. 

Order of skimming: 

1. Title and Abstract (as explained above)
2. Conclusion. Are the results clearly stated? What do the authors believe they accomplished with this research? Is it applicable to your specific question? 
3. Figures and results. Focus on the main figures of the paper, usually those highly referenced in the conclusions. Is the data convincing? 
   1. At the beginning, you will likely be unfamiliar with what makes data convincing, this is something you will learn as you progress in your research. Is the methodology sound? Do they include the right controls? Have they thought of all confounding variables? 

To fully understand a paper, it takes time and multiple reads.

At this point you have determined that your paper is potentially useful based on the Title and Abstract. You have skimmed the paper and the results and conclusions are applicable to your research. Now you can read through the entire paper. 

1. Introductions are good for getting more background information. 
2. Take notes as you read. Particularly pay attention to main figures, the claims made about the results, and useful methodology. 
3. Summarize the results and assess the value of the paper. Is there information or experiments missing? What is the contribution of the paper to the field? 
4. Read the paper multiple times. 
5. Look at their references cited. This is a great source for additional literature on the topic. 

For more tips on how to get the most out of reading a paper, check out these articles: 

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7392212/

https://www.science.org/content/article/how-seriously-read-scientific-paper

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3687192/

Google Scholar is your best friend.

Google scholar is a great place to find papers from any field and any journal. You can filter papers by date, relevance, and type of article. Google scholar also gives you other important information such as how many times the paper was cited, the citation for that paper in different formats, and related papers. 

As part of a university, you have access to almost every paper/journal out there. Rarely, you may come across papers that can’t be accessed. If they are of great value for your research, there are resources in the University’s library to find access to the article. 

Take a look at this video that goes over all the features discussed for google scholar:

[video:https://youtu.be/J5ia2UoSr1o?si=xQUZWUwHVLYnyPt-]

Reviews can be great summaries of a topic in your field.

There are different types of scientific papers. Up until this point we have discussed the importance and structure of primary research papers. Review papers are a summary of the work done and findings collected in a particular topic of research. They can be great sources of background information and point you to the primary literature that led to the results summarized. 

Additionally, other types of literature you might come across are methodology papers (explain experimental techniques in detail), opinion pieces (offer good insight and future direction of study on a particular topic), a patent (describes a new invention and provides legal rights for the inventor), a dissertation (a PhD candidate’s thesis/research project), amongst others. 

Not every Journal is at the same level.

In every field there will be a top tier of journals. 

Journals are given an impact factor (IF) score, meant to be an indicator of the importance of the journal in the field. Impact factor can be calculated after a journal has had a minimum of 3 years of publications, meaning new journals will not have a score. The journal with the highest impact factor will be the one that has the highest average number of citations per paper within a 2 year period. 

For a more complete definition of impact factor. 

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4150161/

Often, a high IF is interpreted to mean that the journal has a more rigorous review process and therefore the “best” papers will be published in these journals. This is not an accurate assessment of IF scores. In fact, there are research groups that have built strong and well-known reputations and top tier journals are aware of which these are. Sometimes, what gets published and where it gets published depends highly on the reputation of the lab and the strength of the research may fall secondary. For this reason, you have to use your own judgment when it comes to evaluating the strength and usefulness of a paper, no matter what journal it comes from. 

There are two other sources for scientific literature that I want to touch upon. Preprint servers are databases where you can submit papers without peer-review. Traditionally, it can take a while to publish a paper in a science journal, because of the long review process. If you are a research group that is eager to publish, you might submit your paper to a preprint server while it is also in review at a journal. 

Typically, when a paper is submitted for review, it can either be accepted right away (very rare), rejected right away, or the researchers are given a list of corrections to complete in order for the paper to be published; these corrections can be quite extensive. There are some journals like eLife, which are taking a novel approach and publishing any paper that makes it to peer-review along with the assessment and public reviews of the paper. 

For more on eLife’s approach 

https://elifesciences.org/inside-elife/54d63486/elife-s-new-model-changing-the-way-you-share-your-research

Read a paper the way you get the most out of it.

Here, we have discussed some of the tricks to efficiently find science literature and assess its value to your project. However, these guidelines may not fit everyone’s reading/learning style. You will find that as you practice reading scientific literature, you will develop your own tricks for doing this. 

For the benefits of reading a paper daily, click on this link: 

https://www.nature.com/nature-index/news/this-scientist-read-research-academic-paper-every-day-what-she-learned

ASSIGNMENTS:

1. Find a paper that is applicable to your research project. For this exercise, pick a primary literature paper, not a review. 
2. After reading the paper, answer the following questions
   1. What is the context/background for this research?
   2. What are the authors trying to show?
   3. What do the authors claim to have found? 
   4. Choose one representative figure from the paper and explain the methodology used and results obtained. 

BONUS: Are you convinced by the data presented?

[video:https://youtu.be/m0Mcuoy_mhc?si=UhNi1cVYhS7OIqKO]

Welcome to the Resume and CV module! In this module, we’ll cover a nonextensive list of do’s & don'ts for CV/resume writing. This will include templates to download so you can begin writing your very own CV/resume! We’ll cover some tips and how to tailor your CV/resume for your audience.

Introduction:  Why should I have a resume, CV, or both???

Sometimes, we enjoy previews of what is to come in our life. That could come in the form of a movie preview, music preview, video game trailer, teaser for an art exhibit, and a magnitude of other exciting previews. Employers like to see some preview of what their potential employees could bring to the table and that is possible through your resume/CV! Often, your resume/CV is the first thing potential employers will see and could have an impact on getting an interview or not.

The Big Picture: Who will be reading my cv/resume??

When creating your cv/resume, you do have to keep in mind the layout of your document: Now, I’ll begin referring to cv & resume as separate entities. Resumes are typically for employment outside of academia, these jobs would request a resume before a CV. A CV is more common in academia, as this lays out an entire summary of your time in undergraduate and beyond. Graduate schools, conferences, and possible employers would ask for a CV. Resumes can be tailored for specific positions(civilian or federal job positions) while CV’s remain quite general. 

1. Resume
   1. What sort of position am I applying for?
      1. Resumes are tailored for civilian or federal positions.
   2. Layout
      1. It really depends on how much work experience you have.
      2. If you have more than 3-5 years of work experience, then begin with a chronological resume. Ex: begin with your most recent job. Typically do not want to span more than 10 years of work experience on a resume.
   3. Not a lot of work experience? That's okay!
      1. You’d probably want a functional resume, which highlights your skills and any objectives you may have!
   4. CV (curriculum vitae)
      1. Full summary of your academic career.
      2. Typically formatted in LaTeX.

 When should I make my resume/cv??

Best practice is to have both your resume and cv on hand. However, it’s best to create both sooner rather than alter. Once you have created either one, you can begin updating them as you gain more experience. 

Thoughts: Take a couple minutes to think about future positions you will be applying for. What does each position require? Cv, resume, or both? Is your position in the federal government? Or will you be applying for graduate school?

Templates for Resumes:

 As I’ve previously mentioned, resumes have various templates to them and should be tailored for the position that you are applying for. In addition, if you’re applying for a federal job then there is a separate template for that. Let's suppose, you do not have a lot of work experience and want to create a resume. In this case, you would want to create a functional resume! This is because a functional resume highlights your soft skills rather than your work history. 

Chronological Resume:

 Chronological resumes begin with your most recent employment to your least recent. In these cases, your work history should be in one career path(typically). You typically do not want to exceed 10 years of work experience when it comes to your resume.

Resource:  Click on the following link: chronological resume

This link will direct you to indeed which gives you a template for writing a chronological resume.

Comment: Resumes usually use bullet points when describing your work history. See the following link-Should You Use Paragraphs or Bullet Points on a Resume?

Functional Resume:

Functional resumes are resumes when you really do not have much work experience. This sort of resume will give you the opportunity to highlight the skills you have! There is really not much else to a functional resume! Unfortunately, many employers do not prefer functional resumes because it is assumed the employee is trying to hide something. Tread carefully when considering a functional resume.

Resource: indeed link with tips: functional resume

Chronological and Functional Resume:

Lets suppose you’re switching careers but you have some work experience. You can combine both a chronological and functional resume. This is useful in the event that you have developed some skills but want to switch careers. When formatting this type of resume, combine the formats above to get your resume together.

Federal Resumes:

Federal resumes do not come in any of the formats above. The federal government has a standard when it comes to hiring, so your resume has to reflect what the job posting is searching for. However, when it comes to explaining your work experience the federal resume should highlight all your accomplishments at your prior places of employment. Also, if you are regularly involved in community service then your resume should note that as well.

Resource: USAJOBS federal resume tips: federal tips

Thoughts:  What sort of job position will you be applying to soon? What kind of resume should you create(or format)?

Formatting Your  Resume:

  -Name and Contact information

• Name
• Phone number
• Professional email. Gmail.com or .edu emails are safe emails to use on a resume

-Self-summary

• Summary of what you’ve done throughout your work history. Can be about a paragraph long.

-Work history

• Chronological order (starting with the most recent) place of employment. Include what your role was during your time of employment and highlight any accomplishments.
• Can be bulleted or in paragraph form.

-Education

• Highest degree earned or in progress.

-Skills (hard or soft skills)

• Hard skill- Skills related to any technical areas such as computer, computer programming, trade skills, etc.
• Soft skill - communication skills, problem solving, leadership, customer service, etc etc

-Typing style and color

• Typically, you’d want your resume to be in black ink & in Times New Roman or Calibri formatting. This will make it easier for employers to read through your resume.
• You can type out your resume in any word processing program of your choice, but many templates live online and you can download them.

Resource: Indeed website to download a template for a resume: resume template

Curriculum vitae  (CV) :

Your CV is the equivalent of a resume but for your time during undergraduate and beyond. CV’s are somewhat formatted like a resume but you would include more information about the courses you have taken, research experience, or projects you have been involved in. Again, your CV will have small blurbs about your time in a particular research group or student involvement.

Now, CV’s are intended to be a full history outline of your time in research and academia. This is different from a resume, where resumes do not ask for work experience that is older than 10 years.

Resource: resource on how to write a CV-How to Write a CV in 2023: Examples, Tips, and Templates

Formatting your CV :

-Name and Contact information

• Name
• Phone number
• Professional email. Typically use your .edu email as a primary.

-Self-summary

• Summary of what you’ve done throughout your work history. Can be about a paragraph long.

-Work history

• Chronological order (starting with the most recent) place of employment. Include what your role was during your time of employment and highlight any accomplishments.
• This will mainly be your research experience or work experience while on campus.
• (you can separate research experience and work experience if need be.)

-Education

• Highest degree earned or in progress of one, mention GPA.

-Academic History

• Include any relevant coursework here. Example: If you’re a physics major, mention the upper division physics courses you have completed.

-Skills (hard or soft skills)

• Hard skill- Skills related to any technical areas such as computer, computer programming, trade skills, etc.
• Soft skill - communication skills, problem solving, leadership, customer service, etc etc

-Awards & Scholarships

• What awards or scholarships you have received and the year you received them.

-Published Works

• Include any publications you are an author on

-Typing style and color

• Same rules apply as with the resume but you can personalize a tad bit more when it comes to CV’s. Just as long as it easy to follow
• Typically, CV’s are written in LaTeX. This is a word processor that is commonly used in academia. A free LaTeX resource is overleaf( log in with your school email!).

Resource:

• Typing in LaTeX -> beginning with LaTeX(using overleaf)

• Various templates on overleaf

Assignments

1. Take a couple minutes to think about future positions you will be applying for. What does each position require? Cv, resume, or both? Is your position in the federal government? Or will you be applying for graduate school? Mention some comments of what you're thinking. 
2. Submit via Canvas and bring to workshop a current copy of your resume or CV for peer review. If you don't have a CV or Resume, start one and submit it to Canvas and bring it to the workshop.  

How to apply to REU (Research Experience for Undergraduates) and other research programs

It might seem early in your Uplift year to start thinking about future research experiences, yet a lot of research positions, particularly for the summer, have application deadlines in early January or February. There are ample opportunities for summer research positions or for post-graduation research programs. These can be found within your own university campus or from exterior sources.

In this module, we are emphasizing the NSF (National Science Foundation) REU programs. The NSF funds research sites across the nation, some even have an international research component. REU projects involve students in meaningful and funded research experiences. These opportunities are, unfortunately, only available for U.S. citizens, U.S. nationals, or U.S. permanent residents.

If you are an international student, refer to the “Alternative option for finding research opportunities” section.

Applications for REU are NOT available through the NSF website. Through the NSF website, you will find a list of actively funded research sites. Once you find a research site of interest, then you apply directly through that lab or research group.  

Here is a link to NSF available REU sites:

https://www.nsf.gov/crssprgm/reu/reu_search.jsp

Each site is going to have their own application requirements, so make sure you know what those specific requirements are. Most applications will consist of a personal essay or short-questions inquiring about your interest in the program and why you would be a good fit. Additional information like CVs, letters of recommendation or transcripts may be required.

A quick note on letters of recommendations: Most of the research programs you will apply to in the future will require letters of recommendation. There are couple things to consider when asking for a letter of rec:

1. How well do you know the person you are asking a recommendation from? Can they speak highly of you? Even if you did well in the class or project, if the person you want a recommendation from didn’t have a lot of interaction with you, the likely thing is that they will write a very generic letter.
2. It is better to have a letter from someone related to the field you are applying to. For example, if you are applying to a research lab in biological sciences, your Spanish teacher is not the best person to ask for a letter of rec, unless you have no other options and that Spanish teacher can write you a very good letter.
3. Don’t be shy and ask for the letter. Part of the job of a faculty member or a project leader is to help their researchers move on to great opportunities. When the time comes, don’t be afraid to reach out and ask for a letter.
4. When asking for a letter, consider providing your letter-writer with a document that outlines the experience you want them to talk about. If it’s your performance on a class project, include a link to the copy of the project so that the writer can refresh their memory on the strengths of your work. If there is anything else you want your writer to highlight, like relevant coursework or work experience, make sure they have that information as well. The easier you can make the process of writing the letter, the more time that your writer can spend focusing on the content, instead of trying to remember the work that you did together and what they should be highlighting in discussing your fit for the REU position.
5. Send constant reminders until the letter is submitted. It is your responsibility to have all application materials submitted on time. Faculty is constantly busy, and they will forget things unintentionally. Polite reminders will save you from being rejected from research opportunities just because you didn’t have all your materials in on time.

Alternative options for finding summer research opportunities:

1. Start on your campus.
   • Additional research funding opportunities like UROP.
   • Ask your current lab if they would have the space and the funding for you to continue over the summer.
   • Ask a faculty member you know if they would be interested in taking an undergraduate student researcher.
   • Research CU Faculty of interest and reach out to ask about the possibility of them taking an undergraduate student. This can be at any CU campus.
      
2. Sign up for newsletters from institutes or organizations that you would be interested in working with. Often, these newsletters have great positions that you can apply to. Other professional networking sites, like LinkedIn, can also be used for this purpose. Consider following or connecting with researchers in your area of interest.
3. Google search for summer research programs for undergraduates. Some of the resources I found from this search are below. There are more opportunities than this.

• https://reufinder.com/programs-for-international-students-daca-undocumented-students/
  • This website has REU programs broken down by year (freshmen, sophomore, junior, senior) and also has programs available to international students!
• https://www.pathwaystoscience.org/undergrads.aspx
  • In this website, you can search for research opportunities and scholarships in STEM
• https://www.apa.org/education-career/undergrad/research-opportunities
  • More research opportunities, specifically in psychology.
• https://www.cmu.edu/scs/s3d/reuse/
  • Software engineering research at Carnegie Mellon (Pittsburg, PA)
  • Internationals welcomed
  • Must graduate post November 2024.
• https://www.aamc.org/career-development/affinity-groups/great/summer-undergrad-research-programs
  • Opportunities in medical/clinical research

A note about funding:

All of the REU-NSF programs will be funded, meaning you usually get living accommodations and food paid for. Moreover, you usually receive a stipend for any additional expenses. Some of the other programs listed here are partially funded or not at all, meaning you would have to find exterior funds. There are many small scholarships that you can apply to fund your summer research. Some you can find through the pathwaytoscience link above. Additional scholarship opportunities can be found through google searches.

Assignment:
    Find 2-4 research opportunities that are of interest to you and that you might consider for your next research adventure. Whether that is post-graduation research or a summer research opportunity. Give a summary of the research project and why you are interested in it. Write down details of application deadline and requirements.

[video:https://youtu.be/nWmZoT5yAP8?si=eTmi_AdPWxQjL1AC]

Science Communication: Basics of science writing for project proposals

One of the most difficult jobs as a scientist is to communicate your research. You need to be capable of presenting your work in a manner that is comprehensible to audiences with different levels of knowledge of your field. More importantly, you have to persuade people that your research will answer an important question, necessary to help further the collective understanding of the field. There are many instances when you will have to do so, some are listed below.

1. When writing a research paper for publishing.
2. When presenting your work at conferences.
3. When applying for grants or other funding opportunities.

As a researcher, your day to day is consumed by the specifics of your project, trying to solve one small piece of the puzzle at a time. It is then a difficult task to step back and develop the full story. But this is an important exercise to not only communicate your research successfully, but also to help keep you organized and guide your experiments. What is the main question you are trying to solve, how are you going to solve it, and why is it important?

In this module, we will work on crafting the main story within the context of a project proposal.
In a project proposal, you will not only have to convince people that your research matters, but also be able to present a feasible research plan to answer your question. Later on, we will focus on how to communicate science effectively for presentations.

There are many formats for writing a research proposal, that will depend on where you are submitting to, but they all include the same foundational content.

Below we will present the main sections of a research proposal and things to keep in mind when tackling these sections:

1. Title

• A good title should be concise yet descriptive.
• A great title will peak the reader’s interest; a catchy title
• Your title will likely change many times as you develop your proposal.  

2. Abstract

• A brief summary of your proposal.
• Include the main question, the rationale of the study, the hypothesis (if any), and the main methods.
• An abstract should represent the proposal in its entirety and stand alone, meaning reader’s shouldn’t have to reference other parts of your proposal as they are reading the abstract.

3. Introduction

• The introduction serves two main goals: introduce the background for your project and establish the significance.
  • Sometimes, there is a separate section required called “Literature Review” where you will go into a lot more detail on what has already been published on your project.
• In this section you will be establishing what is known in the field and how your project will help fill in knowledge gaps.
• Intros should aim to be persuasive for the readers.
• Introductions should end by stating the hypothesis or line of inquiry the research will follow.

4. Aims and objectives

• Typically, your main goal or question is split into more descriptive aims.
• Each aim can be split into multiple objectives. Objectives typically refer to tools or systems you want to develop to study your question, specific information you are trying to acquire or validate, type of data you will be collecting and how you will analyze it, etc.
• Proposals typically have 2 or more aims
• Aims should not be dependent on one another, meaning if aim 1 should fail or proves to be too difficult, aim 2 should still be doable.

5. Research Design and Method

• This is the meat of your proposal
• Often grouped with the Aims and Objectives.
• This is a detailed outline of your experimental design and approach.
• Refer specific tools, data analyses, software, etc that you will employ to answer your question.
  • It can be tricky to know what specifics don’t need to be included. The experimental design specifics to include should reference the main methods.
  • Example: if I say I will be using quantitative PCR to validate the expression of a gene, I would not include what type of primers or what type of detection method (TaqMan, SYBR green, etc) will be used.
• Should refer to possible limitations and alternatives.
  • Reader’s want to know that you have really thought about your project and you have backup methods in case the primary ones described should fail.
  • Reader’s also want to know that you are realistic about your goals, so you should also talk about the limitations in general of your experimental design. What questions will be left to answer? This is also often described as “future directions”.

6. Timeline

• Part of presenting a feasible project is proposing a timeline.
• Here you will outline how long you think each aim or objective will take.
  • It would be great if science worked in a given timeline, but that is very unrealistic. There are many roadblocks that you will not be aware of until you actually start your project. Reader’s will not take timeline’s literally, they just want a best case scenario approximation for your project.

7. Citations
In the Science Literature Module, you were introduced to reference managers, like zotero. When you have a document with a lot of references, zotero not only creates the in text citations and bibliography for you, but it keeps all your sources organized such that you can constantly add or delete them.
Watch the video on using reference managers attached to this module.

8. Extra sections

• These are some commonly requested sections for project proposals not covered above
  • Ethical considerations: if working with animals or hazardous materials.
  • Training: if a researcher needs to be trained to work with certain materials or to operate machinery.
  • Budgets: Outlined costs of the project.
  • Letters of support: from other faculty members

9. General notes on formatting:

• Use 11-12 pnt font size
• Use a legible font type
• Leave space for figures
• Format the document such that it is easy to read
  • Readers have a lot of proposals to get through. They are likely to pay more attention to a proposal that is formatted in an effective way. For example, leave white spaces and format the section titles in a different way than the main text, such that individual sections are easily identified.

Resources:

Detailed guide of proposals (scroll down for common mistakes to avoid)
https://research.com/research/how-to-write-a-research-proposal

As researchers, we are constantly trying to uncover uncontroversial truths about the world around us. Yet, when asking scientists whether bias exists in research that can affect the validity of the results, the adamant answer is YES! The concept of bias is difficult for researchers at any stage to fully grasp. This makes bias difficult to identify and minimize, complicating our pursuits of absolute truth. Biases are so innate in human nature that they can be inadvertently introduced at any phase of an investigation. When this goes unchecked, it can lead science towards erroneous, or worse, dangerous conclusions.

As young scientists, it is important to be aware of the primary biases that exist in your field of study and be trained in efforts to minimize it. Open discussion of biases in experimental design, data analysis, and project conclusions with your coworkers and mentors is an essential step to identify and mitigate bias. For this workshop, we will introduce and discuss in groups some of the principal sources of bias.

Selection Bias: Refers to experimental mistakes that lead to inaccurate representation of the research sample. There are many examples of selection bias nicely outlined in this blog: https://www.questionpro.com/blog/selection-bias/. Highlighted below are some of the most common ones you will encounter:

1. Sampling bias: when our study group is not truly representative of the target population. This could be because of accessibility to certain subjects vs others or because sampling is skewed towards a certain commonality, not randomized.
2. Exclusion Bias: when you intentionally exclude some subgroups from the sample population.
3. Small Sample Size: Rare events can be difficult to capture or analyze systematically because they are unlikely to be in the sampling group.

• https://www.sciencedirect.com/science/article/pii/S1048984323000012
  • This is particularly challenging when trying to train a computational model to become predictive of all events.
• Small sample size reports large effect sizes.
  1. Fanelli, Daniele, Rodrigo Costas, and John PA Ioannidis. "Meta-assessment of bias in science." Proceedings of the National Academy of Sciences14 (2017): 3714-3719.
     1. "Just because small studies tend to give exaggerated results doesn’t mean we should stop doing them."
     2. Exchanging small studies for a more multi-team collaborative model. It’s too difficult to try to combine the results of all these small studies together and trying to find a model that fits all the independent conclusions.
     3. Including more expertise in projects limits individual bias.
     4. Large scale studies are not feasible for small independent groups to carry out.

A note on “The Pressure to Publish: Publish or Perish”

• In academia, what gives a lab capital is the reputation of its findings including: how significant the results are, in what journals they are published, how many times is the work was cited, etc.
• This creates pressure to be constantly publishing positive results in high impact journal, leading to:
  • Selectively reporting studies that “worked” (excluding e.g. null results) (example of confirmation bias)
  • Carrying out different statistics until results show significance (a form of “p-hacking”).
  • Deciding whether to collect additional data after checking to see whether the results were significant (a form of “p-hacking”)

Confirmation or Cognitive Bias: When you favor results that confirm your knowledge or hypothesis and disregard those that don’t.

• Example on Climate change: https://www.sciencedirect.com/science/article/pii/S0896627321006267

• Scenario 1: A scientist is studying a novel isoform of the IL2-receptor (IL2Rs). IL2 is an important molecule in immune cell signaling that leads to division of T cells (TCs) that eliminate pathogenic cells. The IL2 receptor is a transmembrane protein (spans the membrane) that has a binding domain that interacts with the IL2 molecule outside the cell and a signaling domain that interacts with proteins inside the cell to transmit the signal and results in cell division. The novel isoform IL2Rs is truncated compared to the normal receptor, is predicted to be soluble, and bind IL2. After reading the existing literature on the IL2 signaling receptor, the scientist hypothesizes that IL2Rs shuttles IL2 away from the cell, therefore minimizing signal potential and cell division. The scientist conducts growth experiments on TCs where she administers IL2 to one group and to the other she gives IL2 and purified IL2Rs. She sees that 70% of the time, treatment with IL2Rs results in less growth, supporting her hypothesis.
  • Explain how this scenario could lead to confirmation bias?

P-Hacking: Trying to increase the significance of your results by manipulating the statistics.

Resource: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002106&

The History of P

Fisher [20] introduced null hypothesis significance testing (NHST) to objectively separate interesting findings from background noise [21]. NHST is the most widely used data analysis method in most scientific disciplines [22,23]. The null hypothesis is typically a statement of no relationship between variables or no effect of an experimental manipulation. With NHST, one computes the probability (i.e., p) of finding an effect at least or more extreme than the observed finding if the null hypothesis is true [24 ,25].

The NHST approach uses an arbitrary cutoff value (usually 0.05). Findings with smaller p-values are described as “statistically significant” (“positive” findings), and the remainder as “nonsignificant” (“negative” findings). This arbitrary cutoff has led to the scientifically dubious practice of regarding “significant” findings as more valuable, reliable, and reproducible [24], thereby incentivizing various kinds of research bias.

The p-value is easily misinterpreted. For example, it is often equated with the strength of a relationship, but a tiny effect size can have very low p-values with a large enough sample size. Similarly, a low p-value does not mean that a finding is of major clinical or biological interest [28]. Many researchers have advocated abolishing NHST (e.g., [29,30]). However, others note that many of the problems with publication bias reoccur with other approaches, such as reporting effect sizes and their confidence intervals [31] or Bayesian credible intervals [32]. Publication biases are not a problem with p-values per se. They simply reflect the incentives to report strong (i.e., significant) effects.

Inflation bias, also known as “p-hacking” or “selective reporting,” is the misreporting of true effect sizes in published studies (Box 1). It occurs when researchers try out several statistical analyses and/or data eligibility specifications and then selectively report those that produce significant results [12,15]. Common practices that lead to p-hacking include: conducting analyses midway through experiments to decide whether to continue collecting data [15,16]; recording many response variables and deciding which to report postanalysis [16,17], deciding whether to include or drop outliers postanalyses [16], excluding, combining, or splitting treatment groups postanalysis [2], including or excluding covariates postanalysis [14], and stopping data exploration if an analysis yields a significant p-value [18,19 ].

When false positive results enter the literature they can be very persistent. In many fields, there is little incentive to replicate research [38]. Even when research is replicated, early positive studies often receive more attention than later negative ones. In addition, false positives can inspire investment in fruitless research programs, and even discredit entire fields [14,16].

Questionable practices in research are common:

• One survey study in 2018 of scientists in ecology and evolutionary biology showed a large proportion of researchers admit to questionable practices.
  • Cherry picking results (64%)
  • p-hacking (42%)
  • Fraser, H., Parker, T., Nakagawa, S., Barnett, A., & Fidler, F. (2018). Questionable research practices in ecology and evolution. PloS one, 13(7), e0200303

Elevator Pitches Outline

Length: 3min pitches

Start with your name and the lab you are working in.

Ex: Hi, mi name is Andrea Ordonez and I work for the Chuong Lab in the Molecular, Cellular, and Developmental biology department.

No notes allowed. This should be natural, like you're telling a friend about what you're studying.

Content:

1. What is the problem your research aims to solve?
   • In a broad way, introduce the problem you are trying to solve and why is it important.
   • Consider the knowledge level of your audience – in this case, your Uplift cohort, who all have a foundational base of science knowledge, but not particularly in your field.
   • Tips:
     • Center your intro on why as humans we should care about the problem you are trying to solve: does it affect our social, medical, or natural world?
     • Try to think of a one-liner to start your pitch that will grab people’s attention.
     • Be relatable and engaging – after all, elevator pitches are originally intended to get people to fund your idea.
        
2. How are you solving it?
   • Describe your general approach.
   • Avoid heavy jargon.
   • Minimize details.
   • Tips:
     • Start with the simplest way to explain it. Practice your pitch, if you have more time, then add more details.
     • Use simple language, avoid having to explain anything too difficult.
        
3. Bring it full circle.
   • Connect your previous ideas together to leave your audience with a full story and a reminder of the importance of your project.

More Tips:

• Your delivery is key – you want to have a good tempo, be concise, be clear, be natural, be exciting. A lot to ask for, but we can try! Practice helps.
• Write out a script.
  • Start by writing down the main goal of your research.
  • Bullet point the important points to hit.
• Practice the script to make it natural.
• Practice with a friend that has minimal understanding of what you do.
• Practice! Time yourself!