So, in this module, we'll be looking at special considerations and things we need to think about as we're collecting data directly from research participants using questionnaires or surveys. So, the central principles will be, talking about those best practices as we've done before. But this time, again when we're collecting the data directly from participants. And as always, we'll be looking at the importance of doing this, doing it right, before we start the study collection processes So why use surveys? There's a lot of power in collecting data directly from research participants. obvious ones would include streamlining procedures having less people in the process of getting data direct from an individual. collecting it and pushing it onto paper and then having somebody transcribe it and load it into a database. There are, there are places in there that are inefficients, there are also places in there where you might make errors. So there's a lot of power in collecting that data directly from the participants. Other advantages would be that, you may collect data directly from the participants in an anonymous fashion, and in so doing you can elimanate some, some, some barriers to participation by, by individuals who might not otherwise want to be identified and, in, in giving information, in, in a, in a way that they could be identified. Using self-administered questionnaires and surveys you can eliminate interviewer bias. Individuals might, might answer a question different if they're not having to look a research coordinator in the eye as they're telling them information particularly around sensitive topics. finally it, it could increase the response rate and, and the validity of those responses. Again, especially in sensitive areas where, where people would be a little bit hesitant to answer maybe otherwise. however there are some difficulties. One of the difficulties is that if you're giving, administering a data collection instrument by its survey mode, where an individual has a piece of paper that they're filling out, or maybe they're looking at a website or any other modality which we may talk about over the next course of several slides. Then, the great thing is, you know, all of these things that we just mentioned, the difficult thing is that there is nobody there to answer a question if they don't understand. And so you have to be very, very careful, even more careful, in survey design to make sure that you are getting things upfront correctly so that, you know, there is no question or there is no ambiguity. When the, the individual taking that instrument or that survey is, is, is out there trying to fill it out. because worst case they won't answer or perhaps even worse, they'll answer and, and they'll not quite understand the question. And they'll provide something that's, that's in error. So, you have to be very, very careful there. I want to give a disclaimer that, although I've worked with a lot of surveys, I know a lot of survey experts. I've learned a lot of survey, a lot, a lot of what I'll present here, best practices or principles that I try to use. I've learned them from my own mistakes. But also I've learned a lot from poorly designed surveys that I've been asked to take, but I would not consider myself and would not present myself as an expert on this topic. What I do think we can cover here is sort of special considerations as, as we said when, when we are going to collect data this way then, then I think we, we've got some core principles that you can you can apply. But, you know, real surveyance from design testing invalidation is quite difficult, and, and probably covering all of the material there will take at least another entire course. So with that disclaimer, let me also remind you that this is a special considerations topic. We've done a lot of work prior to this, this module, talking about best practices. And putting, putting what we learn into practice in electronic data capture systems, et cetera. I would say that, given that this is a special consideration, all of those things that we've talked about up to this point they still apply. We're just going to be talking through a few extra things you might want to think about as you're collecting data, using instruments and surveys. The other thing I want to remind up front. Because it doesn't really fit anywhere else in the, in the lecture material that I've put together. But it's something that's so critical. And a lot of times, it can be overlooked in survey-only research. Is to remember that all human subjects research must be approved by the institutional review board. I've, I've seen that in, in some cases, researchers think that, because they're, they're doing survey work, because it's easy, you know? It's, they're, they're contacting directly, and they're not, sometimes, even seeing the patient face to face. That, that maybe there's some sort of different ruling around human subject protection in, in survey research. So we always try to remind people that even though it's survey-only research make sure that you're clearing things through the, through the IRB or whatever that human-subjects board is called in your particular county or location. I will say as well and at least in the United States, a lot of times if you can collect the data in a de-identified fashion using surveys that, that even though you still have to have IRB approval, you can you, you can many times qualify your research as minimal risk or even, even with except status. However, even with exempt status it's important and, and essential that you go through the IRB for this type of research. so really a couple of couple of things to mention around this, this very central concept. And we've covered this is several different ways before, talking about putting the time in upfront. But I've found especially end survey research that creating a good questionnaire or survey, actually is quite difficult. It takes some time and, and we'll go through the different steps of, of doing that the right way in the next several slides. But, but if you do it, then, then you get to the end of the study and the data are actually quite good. Assuming you've, you've posed a good research question and, and the data are quite easy to analyze and then you know, you're own to publish your results. On, on the flip side of that coin, simple questionnaires are easy to create. You know, it's, it's quite easy to throw a bunch of questions in and, and not give it a lot of thought, think you're getting to, to the right answer with your research question. Easy to roll those out very quickly, but but a lot of times what you find is that you, you take those shortcuts up front and, and you get to the end of the study and you've got data that's difficult or even impossible to analyze. And so again because you're not in there in the room with the, the volunteer. Because many times you're sending out many of these at once and you've got, kind of got one shot to get it right. It essential that you that you put good time in up front even though it's difficult and takes time, it'll save time and it'll certainly improve the quality of your study to do it right from the beginning. So, so with all of that in mind, I'm going to go through a process where, where we look at navigating the, the creation and administration of survey instruments. Using seven different domains starting with defining scope and ending with data analysis. In each of these different domains we've got one or two or sometimes more principles that, that we'll stress along the way. So the first of those core areas is defining the scope. So again it's a scientific study we're talking about. We want to make sure That, before we begin, we're defining specific aims. We also need to, just as with any scientific study, we want to make sure that we're defining the population of interest. Surveys are like any other study and it's important to define the necessary sample size. Make sure that we're right sizing that and not just going with convenience sample because, you know, we can contact a thousand people. It's important that, that we think about that. How many do we need? How many will we contact? What is our expected expected ratio of people that actually take the survey and complete it to those that we send. You know depending on the, the particular area and domain that you're working. Sometimes 10% is a great number. There are things that, that we'll talk about but, but you know, maybe, maybe increasing those. But it's, it's quite important to sort of right size the population for the study and think about all of that just as you would with any other study before you start. [COUGH] Marketing strategy. So we mentioned the 10% being a, you know, sometimes a great, maybe sometimes a bad percentage of participants actually going on to take surveys and complete a study. You can increase that by various methods. One of the things that we talked about. earlier, and we'll talk about it in the next bullet actually, is, is on automatizing data collection. You know, if you've got a, got a study where we know that things are going to be anonymous, people might, might be more interested, or perhaps less likely in, in your case for or in certain cases for participating in the survey. There are also ways to incentivize. You know, if you tell everyone that, that, you know, it's a ten minute job on your part. And, and we'll be awarding $50 gift cards to, to each participant. Chances are you'll get a pretty high ratio of, of respondents to those contacted. if you do, you, you know, if you do a random drawing of five random $50 gift cards for 100,000 sent or 10,000 sent. You know, obviously that's going to change it, but there are incentive and, and marketing strategies that you can you can and you need to think about before you start rolling things along. So consider existing instruments. And this, this is a very important topic we mentioned in the foundation best principles earlier, and that is, don't reinvent the wheel. so it's a review of the literature. Collect all instruments that are related in concept. If, if you can, use an existing instrument. with, without having to create your own survey to measure a particular topic or domain. Then it will save development time, especially in testing and validation. It'll allow comparison of results across studies. As long as other people are using that same instrument. And, in as well, you know, your, your research is going to be much easier to publish, because you're not inventing a method and then going off and, and doing your study. So, it might eliminate the need for a pre-study if, if you can reuse existing measurment sorry instruments as you're going along. finally even with existing instrument it's really important to, to, to look within the literature and where those where those validation studies are reported and make sure that anything you are using as, as a validated instrument to measure X that, that, that it was validated in the population that you're thinking about studying within the administration modality that you're studying in the environment. So in other words if we were looking at some validated instrument that looks at smoking in Alaskan individuals in nineteen that, that was validated in 1950. Maybe that's something we'd think hard about, before we decided well, that, that can be also used here at Vanderbuilt University and Nashville, Tennessee in a rural area, I'm sorry, an urban area. So make sure you are looking at the environment how those validations occurred and factoring that into your decision on whether an existing instrument is the one to choose for your particular study. So, we'll go on next into questionnaire planning and development, but we'll stop here for this particular segment.
