ii. Reproducibility

We strongly encourage you to use resources available about how to conduct reproducible science (for example Reproducibility for Everyone and using RRIDs). We will also have discussions in lab meetings about reproducibility. There is never a wrong time to talk about any concerns or new ways to improve the reproducibility of the science within the lab. Science is very difficult, and it is easy to make mistakes or assumptions. We want to limit both of these, but still expect them to happen often. In essence, we want to control every variable to the furthest extent feasible and to document the experimental procedures so that they can be repeated by other lab members and research groups. To ensure reproducibility within the lab, we try to develop standard protocols, which will be linked at the bottom of this section.

​

If there are deviations from these protocols, please note changes to that protocol and make it accessible to all lab members. In addition, standard protocols should not be changed unless there is sound reasoning and changes are made clear to everyone, including Dr. Evangelista. There are a variety of ways to ensure reproducibility of one's work, like showing a finding in a second system, asking a lab mate to conduct the same experiment and verify findings, including sufficient controls, verifying the reagents you are using, etc. The overall goal is to move science forward. Ensuring you document sufficient and detailed information and making this information available to everyone is essential. Keep in mind, you can also improve the reproducibility of your science, so never settle for your current approach!

iii. Collaborations

 

Collaborations are an essential aspect of science, but they will not work without clear and constant communication.

a. Intra-lab Collaboration

All lab members are welcome to help one another. In fact, intra-lab collaboration is greatly encouraged. Sometimes this will be formal in that Dr. Evangelista will actively ask for members to work together on a common goal/project. This can also happen informally or organically, though it is important to remember to communicate any plans with Dominic and to keep track of relative contributions. Keeping a spreadsheet (or similar) on a shared drive (e.g. Google Drive) noting who participated in a project can be a useful way to track how a project is coming together and who contributed to what extent. This also assures that all contributions to a collaboration are acknowledged correctly. It is also useful to set up formal meetings among people who are collaborating on a semi-regular basis. While there may be informal communication on a daily basis, it’s important to have formal meetings with the whole project team to review aims, problems, and future directions, so that there are not any surprises.

b. Inter-lab Collaboration

The ability to collaborate with labs all over the world is one of the great aspects of our job! If you find a lab that you think would be beneficial to work with and you/we can learn something from, bring this to Dominic’s attention. We will then discuss if we think it is the correct time to reach out (i.e. we may want to do more groundwork first) and the best approach for contacting the lab of interest. It’s important to remember that 1) the other lab may not want to collaborate and 2) after talking with the lab you may not think that it’s the best fit. Both outcomes are completely fine! Typically when collaborating with a lab you haven’t worked with before, it’s advantageous to come up with a small and inexpensive (in terms of time and monetary resources) experiment to determine if the project is worth pursuing. These initial experiments can allow one to understand how working with the other group will be and help to establish the project’s foundation. Ideally, collaborations are mutually beneficial. For example, keep in mind that reaching out to a group who has a technology that no one else in the world has, may seem to them like you are asking them to do experiments for you. It is therefore important to discuss expectations, contributions, authorship, and timeline early on in the collaboration, and to confirm this in writing. You don’t want another lab to put in a year's worth of work and then add them in the Acknowledgements section of a paper because that was your understanding, while they thought they were going to be co-authors. Also, here it is important to keep track of contributions in a shared document (e.g., a Google spreadsheet) throughout the project. As with intra-lab collaboration, formal discussions on a semi-regular basis are essential for updates on project progress. Keep in mind that some aspects of a project can be performed very rapidly with intense work, while others may take long periods of time with intermittent work. Both of these may end up being equal contributions.

​

iv. Mistakes

Make mistakes, it's okay! If you do not make mistakes, especially in a new lab, maybe you are not trying hard enough. However, mistakes can be minimized by being careful. Efficiency is important, but it is better to be slow when you are learning. Do not rush your work. Think about everything. Run through a procedure ahead of time to make sure everything is working and in place. To focus on the experiment at hand, it may be useful to organize/make reagents or label tubes the day before a big/new experiment.

​

Plan ahead to avoid mistakes. Determine how you can make an experiment easier to avoid common potential mistakes. Dominic and (senior) lab members can provide advice in this regard; you can learn from our prior mistakes. Double, and triple check, all your analyses and data. Even if your analysis is HUGE in scope, there are ways you can efficiently check your data and estimate it’s reliability.

​

Here’s an example. For one project (Evangelista, et al 2017) Dominic was doing a review of J.W.H. Rehn’s dissertation on cockroach wing evolution published in 1951. The aim was to see if any of the data in that dissertation was systematically informative (turns out…very little of it is). This document is 186 pages long. In order to do this, Dominic read through the text and turned 186 pages of textual character descriptions into a discrete morphological matrix. The resulting matrix was 50 rows long by 146 columns wide (i.e., ~7300 total morphological observations). Doing this one task took weeks or months. How did he check this data? He did not do the weeks long task for checking every single data point. No. Instead he randomly drew an arbitrary number of data points and checked those, and fixed any errors. At first a higher % of data was wrong (maybe ~10-20%). Eventually, the error rate was lower (10-5%). Once the the error rate was <5% of observations then he stopped checking for errors and accepted it as is.

​

Incorporate sanity checks. Ask others to look at your code or data. It is always better to catch a mistake immediately instead of months down the line! For the love of cockroaches, please ask Dominic “Hey can you check that this looks right to you?” He will help and maybe he will say “Yea this looks great. I agree with you”, “Oh I think you did this wrong. I am so happy you asked because now you can do this better”, or “I have no idea. If neither of us can figure this out maybe we should reevaluate this project’s feasibility. I am so glad we figured this out”. If we made a mistake, we admit it, then we correct it, and then we move on with minimal drama. What matters is how you deal with errors and what you learn from them. Furthermore, it is an excellent opportunity for you to identify the supportive people in the lab who will help you succeed during your research career. Utilize your environment, work hard, and ask lots of questions. Never hesitate to ask a question regarding experimental design, execution, or interpretation. Do not try to cover up or deny a mistake. If you have made an error in experimental execution, make sure to record it in your lab notebook in detail and bring it up at lab meetings so we can all learn from each other’s mistakes. If you do make a mistake, tell your collaborators if they have already seen the results, and especially if the paper is being written up, already submitted, or accepted.

​

v. Project management

If you are a post-doc or grad student, you are in charge of your own project so you must follow good rules of project management. The guidance in this section is good advice for anyone in the lab but for those of us managing projects, they are a must.

​

Physical and digital organization. Whether your data are physical (e.g., insect samples, DNA) or digital (e.g., photos, spreadsheets, genetic data) organization is hugely important. Even if your files are backed up, if you lose them, or can rarely find the right file, then what does it matter. Your project files (particularly if you are doing bioinformatic research) will become massive in scale in a very short period of time. For example, as of May 2023, Dominic’s OneDrive has 111,000 files in it. This includes a single project folder that, by itself, contains 39,928 files in 568 folders. Use a system to manage projects, of any scale (e.g, https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000424).

​

Sharing and backing up your files. Since you now have a folder with all your data and project files, that folder must be backed up on the cloud (UIUC uses OneDrive, so this is preferred) and preferably shared with your PI and possibly other collaborators. If you do not feel comfortable sharing the whole folder with your PI you should share the living version* of (a) the documents that contain your proposal, outline, methods and manuscript drafts, and (b) your raw data.

​

*by living version I mean the version you use, not a copy. So, if Dominic make comments/edits to the document you share with me, they should automatically show up in your version without him having to send you an email/Slack message.

​

Note taking. In your digital project folder, take extensive notes. Take notes as you do things, not in retrospect. In particular, take notes about (a) who contributed to your project, when, and how they did so, (b) your exact step-by-step process of performing your project data (even better, create a manual that describes what you’re doing as you go), and (c) explanations of data, figures, and data files. All of these things will eventually need to go into your finished product (your dissertation and resulting publications). (A) is important for research integrity and properly attributing people who assisted with coauthorship or acknowledged assistance. (B) Is extremely important and has many benefits. Keeping detailed methods will allow your project to be reproducible, clearly understood by others, and more publishable. Not only that but it will help you during the actual project. A piece of advice: if you do something once, you will almost certainly do it again (probably 3 times, or more) in the future. You will be very grateful to your past self if you write a clear coherent guide to your process. (C) is obviously important for understanding your results, but more-so it’s important for getting a nuanced understanding of your data. Were you equally confident about all the data you collected? Did you go into autopilot while doing any of your data collection and therefore you’re not sure what you ACTUALLY did? Which version of your data did you analyze for XYZ figures? This is all important to keep track of, because it will help you get a better understanding of things when you’re integrating everything at the end.

​

Data management. Data is academic currency. Therefore, if you lose your data you are bankrupt and may have your academic home foreclosed. There are many best practices when it comes to data. Here is a minimum we will ask for in the Roach Brain lab.

1. Backup your data twice. Once on the cloud and once on a physical hard-drive. Your physical backup should be separate from your working copy (i.e,it should not just be the version on your computer that you are working on.

2. Keep a copy of your unaltered, raw data. You will need to share this when your study is published, and you need to have this in case you make a mistake downstream.

3. Allow your data analyses to be replicable. The easiest way to do this is to analyze your data in a coding platform. R or Mathematica are preferred in the Roach Brain Lab, however python is acceptable. If you refuse to learn a coding language then you must simply make extremely detailed notes.

 

Here are some other suggested practices.

1. Use a relational database. Spreadsheets are messy, ineffective, and limited in their abilities. Relational databases are better for integrative datasets. As of May 2023 Dominic has resisted using relational databases but wants to change this.

2. Use GitHub to manage your code.

3. Use digital data repositories. For the most part, this is only required when you publish your study.

4. Make your data FAIR (https://www.go-fair.org/fair-principles/)

5. Make your data public. Most of the time this would not be a good suggestion, but it could lead to unexpected opportunities. For example, put everything on iNaturalist before you publish it. Maybe you’ll get help with your taxon identifications or insights about biogeography/ecology.

6. Google “best practices for data management in biology”. You’ll find a lot more suggestions.

​

vi. Literature search

Literature search should be the first thing you do for a project. Take notes as you do (see vii. Writing below). You are expected to be the lab’s expert on your project so your knowledge of the literature should reflect that. You will have to search the literature multiple times in multiple ways in order to find the most relevant papers for your topic. Here are some strategies, most of which you should use.

• Use Google scholar initially because it is quick and efficient.

• After you use Google scholar, do a more thorough series of searches in Web of Science (WoS) or another academic database search tool.

• Craft a search query in WoS using wildcards (*), multiple synonyms for the words you are looking for, and logical operators (OR, AND…)

• Read papers and find new papers based on the papers they cite within them. This is often the most useful step.

• If you notice one author has written a lot about the topic you’re writing on, then you should check out their Google Scholar profile, or their lab website.

• Read more papers and come up with new search terms to use in WoS and repeat the process.

• Come back a intermittently as your project progresses and repeat. Maybe you won’t repeat everything exhaustively, but you should at least do a quick search for new papers on your topic.

 

Finally, use a reference manager. Dominic uses EndNote. Mendelay is a free option. Make sure the platform you use includes simple CWYW (cite while your write) options for your word processor. A reference manager is required for all people writing papers in the lab.

​

vii. Writing

How to start writing. There is no right way to do this but here are some tips I use and some that I have heard (but maybe don’t always use).

• Pre-writing starts in the beginning phase of a research project. Your project probably started with an idea, probably a hypothesis. Then you must have done some background research. As you do your background research you will find ideas you believe are important to addressing your hypothesis. If an idea will justify your hypothesis, or give background information to better explain why you have that hypothesis then you will (eventually) want to cite that idea in your proposal, or the introduction to your publication. If the idea relates to your potential results or bigger picture, it might end up in the discussion section of your eventual publication. And so on. Write down these ideas and the citations. This can form the beginnings of the thing you’re writing.

• The first page of the proposal is everything. Follow the guide in the “Professor is In…” book. It goes something like this. (PLEASE UPDATE…I am writing this without the book in front of me. The first two paragraphs of your proposal should be a slightly expanded version of this…. “X is important because Y. But there’s a problem with X. This is a big issue because Z. But it turns out I can fix the problem with X, which prevents Z, and thus solves Y.” Then, structure the rest of your proposal around this reasoning, and expand with examples, figures, and more explanations.

• Writing methods for a paper. This should be done as you do your project. The process of writing your official methods section should be a process of rearticulating the notes you took while doing your project, adding citations, formatting information into tables, diagrams, or equations.

• Writing an abstract. Do this LAST. Use a structured abstract format in your first draft of your abstract.

• Start papers/dissertation chapters by writing an outline.

• Use “text styles” (i.e. format your headers) in your work processor. Trust me. They help so much at ALL stages of writing.

• Writing the introduction. Opinions about this differ. Some people say that the introduction should be written after everything except the abstract. Dr. Evangelista usually writes this before the results because it helps him better structure the logical flow of what the main results should be and their importance. Regardless, the introduction should contain both background information and synthesis of ideas that justifies your hypotheses/aims.

• Writing the results section. Simply state your results in a logical order. No citations.

• Writing a discussion section. Start with your hypothesis/aims and explain which of your results addresses them. Then, relate each of these to the literature. Other thoughts (caveats, errors, implications) should also be included in the discussion section. You should add an outline of these things as you are doing the other parts of your study.

 

There are entire books about scientific writing so this guide will say little more about it other than this. Prepare for writing to be a long process. Many of Dr. Evangelista’s papers were written over the course of 1 or more years. If you think you can write up a chapter of your thesis in 2 weeks, you are wrong. The quality of the paper you will have after 2 weeks of writing would likely be rejected from a journal without review (unless it’s a short, taxonomic paper). Prepare to go through at least 3 full drafts of your paper before it is even ready to be prepped for a journal submission. Each draft will likely take 3 weeks to a month to get through. You can optimize this process by using a reference manager, taking thorough notes, not arguing with your coauthors/editors, having a strong knowledge of the literature, and starting early.

viii. Other research practices

This section includes some starting points for people who are doing different types of things Roach Brainiacs do. If you see something missing, ask Dr. Evangelista.

​

Link to lab protocols folder

​

General reading recommendations

• To find out what Dominic does read his papers. Evangelista et al. 2019. Proceedings B.; or Evangelista et al. 2021. Systematic Entomology; of Evangelista et al. 2016. Palaeontologia Electronica. Find them on roachbrain.com/publications or Google Scholar.

• To get to know cockroaches: Bell, Roth, Nalepa 2007. Cockroach textbook (PDF available free online…or if you can’t find it ask Dominic)

• To get to know the specific organism you’re working on: read the most recent (and most narrow) published phylogeny of the group, and the most recent taxonomic study of that group. For more, look through the sources they cite. Also, ask Dominic to share the most important taxonomic paper for that group with you.

• To see a really good example of an organismal study: Sauquet, H., et al. (2017). "The ancestral flower of angiosperms and its early diversification." Nat Commun 8: 16047.

• To see a really good example of a integrated organismal/methods study: Borowiec, M. L., et al. (2015). "Extracting phylogenetic signal and accounting for bias in whole-genome data sets supports the Ctenophora as sister to remaining Metazoa." BMC Genomics 16: 987.

• To see a really good example of a methods paper: Dornburg, A., et al. (2018). "Optimal rates for phylogenetic inference and experimental design in the era of genome-scale datasets." Systematic Biology 68(1): 145-156.

• To see other really good papers paper about phylogenetics:

  • Townsend, J. P. (2007). "Profiling phylogenetic informativeness." Systematic Biology 56(2): 222-231.

  • Wägele, J. W. and C. Mayer (2007). "Visualizing differences in phylogenetic information content of alignments and distinction of three classes of long-branch effects." (147): 1-24.

• To get to know the software you’re using: Read their guidebook and go to the original citation for that software and read that paper.

​

 

 

 

 

 

 

​

​

​

​

​

​

​

​

​

​