A survey is a method of gathering information from a number of individuals, known as a sample, in order to learn something about the larger population from which the sample is drawn.  Although surveys come in many forms, and serve a variety of purposes, they do share certain characteristics.  In order for the objectives of a survey to be met, the results must reliably project on the larger public, from which the sample is drawn.  A sample can be scientifically chosen so that each individual in a population has a known chance of selection.  This ensures that a sample is not selected haphazardly or uses only those eager to participate.  The sample size for a survey will depend on the degree of reliability necessary and how the results are to be used.  A properly selected sample should be able to reflect the various characteristics of a total population within a very small margin for error.  There are many surveys that study the total adult population but many others that focus on selected populations:  employees, academics, industry experts, computers users, or customers that use a particular product or service.

Some surveys focus on opinion and attitudes while others are more concerned with collecting factual information. Many surveys combine questions of both types.   A respondent could be asked what they heard or read about an issue, what they know about it, their opinion, how strongly they feel and why, interest in the issue, past experiences with it, and also certain demographic information which will help the survey analyst classify the responses (such as age, sex, marital status, occupation, and place of residence).  Questions can be open ended (“What does that make you think of?”) or closed (“Do you agree or disagree?”); they may ask the respondent to rate a product or a service on some kind of scale; they may ask for a ranking of various alternatives.  The questionnaire could be very brief – a few questions taking no longer than five minutes, or it could take a demanding hour or more of a respondent’s time.  A survey is usually rooted in situations where an individual or institution is confronted with an information need and no existing data will suffice. Once the information need has been identified and a determination made that existing data is inadequate, objectives are laid out for the investigation. These objectives should remain as specific, clear cut and unambiguous as possible.

Designing the questionnaire is a critical stage of the survey development process.  The questionnaire links the information need to the realized measurement.  Scaling techniques used for measurement can be comparative or non comparative.  If proper care and attention is not placed on clearly defining concepts and unambiguously phrasing questions, the resulting data is apt to contain serious biases.  Questions used in surveys must be refined to minimize interpretation problems and thus reduce measurement error.  If the respondent is unable to understand a question or fails to comprehend the question the way it was intended then the data is neither valid nor reliable.  A question must use language in a way that makes the intended observer’s meaning behind that question obvious.   In order to do this a survey must reflect an understanding of the population being sampled. For example, many people do not distinguish between robbery which requires the theft to be in the immediate presence of the victim and burglary which can involve breaking and entering without any confrontation.  The National Crime Survey, done by the Bureau of the Census, does not even mention the word “robbery” when it asks questions about robbery victimization.  Rather they ask several questions that use universally understood phrases, consistent with the operational definition of robbery, that when used together are able to capture the desired responses.

Keeping response errors and biases to a minimum factor heavily in designing a survey. How questions are interpreted by respondents must be carefully considered.  Also, you need to consider the length of the survey. If a questionnaire is too long than it can be burdensome to the respondent, inducing respondent fatigue that leads to response errors, refusals, incomplete questionnaires, and can contribute to higher non-response rates in subsequent surveys involving the same respondent.  Memory plays an important role when surveys deal with past events.  For most people the greater the demand a question places on memory, such as being asked to recall trivial details occurring on any given random date, the less accurate the responses and therefore the less reliable the survey data that is collected.  An appropriate choice of reference period should be made so that a respondent is not forced to report on events that happened too long ago.  Grouping similar questions together can reduce the cognitive burden a survey places on a respondent.  The validity of a survey can be put in jeopardy if questions are too sensitive, if they may prejudice the respondent, if they unduly invade a respondent’s privacy, and if the information that is sought is too difficult for a willing respondent to provide.

A web survey is one of five general ways research is typically conducted using surveys.  Surveys are also given using mail (the kind that requires postage), telephone, in-person interviews, and intercept surveys (respondents are “intercepted” as they pass through a heavily trafficked area).  The web survey most closely resembles the type of surveys which were traditionally mailed out.  Only this time you’re using email to send a link that takes you to an HTML form that is designed for completion and submission through the computer.   Information is collected quickly because as soon as your respondent has finished taking the survey, their responses are immediately submitted.  Web surveys can also be embedded on your web site using what is known as an Iframe.  An IFrame is an HTML element that makes it possible to embed an HTML document inside of another HTML document.   You can also create a Pop-Up survey on your web site so that when a site visitor visits a page the HTML form pops up from their web site.   If they have pop-up blockers enabled then this method will prevent them from taking your survey so you can also create a link to the survey which you can display on your web site.

The web survey is a cost effective way of administering a survey that allows you to collect large amounts of information without having to pay for interviewers, paper supplies or postage, and does not require separate data entry for responses to be processed.  The rise in web based surveys is due in no small measure to the increasingly widespread availability of computers.   Particularly in organizational or professional settings, the ability to receive a questionnaire and complete it at home or in the office on a computer is very convenient for most people.  You can expect to wait at least a few weeks for a questionnaire that is mailed out to a respondent to be returned.  A web survey allows you to rapidly collect data in a timely manner.   Information can be collected and processed in just a few days.    It also allows respondents ample time to carefully consider response selection and to enter in text for open ended questions.  If factual information is required then the respondent has enough time to consult their records.

You can increase response rate to web surveys by sending respondents a pre-notification of the intent of the survey.   This lets your respondents know what the survey is about before they invest any of their time into taking it.  You can also send follow-up reminders to respondents that have not completed the survey. This can be used in a situation where a respondent must complete the survey within a given date range.  Web surveys assume a minimal level of computer literacy so keeping your surveys simple can also help your response rate.  The technical nature of online surveys make them ideal for specialized or well defined populations that have access to an email account and/or a computer.  Coverage bias may result when targeting populations where computers or internet access are not widespread.  Determining an appropriate type of survey to administer to your respondents must take the population that is being measured into consideration.

People ask questions in order to learn what they wouldn’t otherwise know.  When a respondent answers a question on a survey they must rely on their comprehension to interpret what that question means.  Then during a process of memory retrieval they access relevant information for content to be included in their answer.  Judgment is used for comparing and evaluating ideas in a way that allows information to be synthesized.  During response selection an answer is selected and placed in the required response format. All stages of the cognitive process just described rely on the first stage which is question interpretation. For this reason questions used in surveys must be refined to minimize interpretation problems and thus reduce measurement error.  If the respondent is unable to understand a question or fails to comprehend the question the way it was intended then the data is neither valid nor reliable.  A question must use language in a way that makes the intended observer’s meaning behind that question obvious.   In order to do this a survey must reflect an understanding of the population being sampled. Before administering a survey you need to ask yourself whether your questions measure what you intend them to measure.  Is it valid?   Do your respondents interpret your questions the same way.  Is it reliable?  In order to ensure that the respondent is answering the question we think they are answering it’s important to make questions clear and precise.

Level of Wording

Question wording can affect the answers that are received.  The wording used in questions should be appropriate to the level of education and characteristics of the respondents.  A researcher must be aware of the respondent’s understanding of difficult words, colloquialisms, jargon, technical terms, and language proficiency when settling on an appropriate level of wording.  They also must be aware of words or phrases that are gender or ethnic sensitive.  If the language used is not the first language of all your respondents then the level of language proficiency of all your respondents must be carefully considered.   Level of wording is important because a respondent that is embarrassed to admit that they do not know something is likely to give any answer rather than admit they do not know.  You may also see a lot of neutral response choices such as  “do not know” or “no opinion.” Also, it can lead to a high rate of refusal to complete survey rate.   In general you should use wording that is simple, direct, and to the point.  A question that would not be appropriate to the level of wording of your average survey respondent could be:

“Did the attorney deliberately obfuscate the truth by inundating the jury with statements that were verbose and declamatory?”

An understanding of the population you are administering the survey to must be worked out in advance.   If you are administering a survey to attorneys you would use a different level of wording than you would if you were asking the same questions to a group of layman.

Ambiguous Questions

If there is more than one way to interpret a question than that question is ambiguous.  When a question is vague it has an imprecise range of application.  If a question uses vague wording  or is ambiguous it can lead respondents to interpret a question in a variety of ways.  An example of an ambiguous question might be:

“Who do you buy groceries for?”

It is not clear whether either you or any household pets are included in this question.   Is this asking who you usually buy groceries for?  Does this include anything you might pick up for your parents that you no longer live with?  A better way to ask this question would be:

“What is the total number of household occupants that you purchased groceries for in the past year?”

Ambiguity can also arise if a respondent is unclear about what type of question is being asked, the motive for the question being asked, the purpose for a question being asked, and when inferring motives behind a question that have no truth in reality.

Confusing Questions

A question is confusing if it leads to a feeling of uncertainty in the respondent when the intent or meaning is not clear.  Here is an example:

“Does it seem possible or impossible that the Cubs will win the World Series, or do you feel they will always find a way to lose?”

1_ It seems possible

2_ It seems impossible

Here the respondent may feel trapped into either agreeing that the Cubs will find a way to lose or saying that it is impossible for the Cubs to win the World Series.   A clearer way to ask this question would be:

“Does it seem possible that the Cubs will ever have a chance to win the World Series?”

1_It is possible that it could happen.

2_It is not possible that could happen.

Working Memory Overload

In order for a respondent to process a question it can not impose a cognitive burden on their working memory.   Working memory is short term memory used to hold information while the respondent is comprehending a question.  This can often happen when a question attempts to be too precise, by injecting complex syntax and embedded clauses,  in an attempt to clearly define a question.   Earlier we talked about vagueness and ambiguity.   Couldn’t you make our earlier question more precise by rewording it as:

“Including yourself, any pets you own, but not including any relatives or friends that do not live in your household for at least 320 days a year, how many household occupants did you buy groceries, by groceries we mean any item of food bought at the grocery store, for in the last year?”

The problem with asking questions like this is that you are going to tax your respondents memory and some of them will not even get what you are trying to say.  Individuals differ in how much their working memory can handle.  Questions that pose a burden to some may not pose a burden to others.  Avoid complex syntax in your survey questions.  This means that its grammatical composition can not be dense, structurally ambiguous, or not well formed syntactically.   Memory overload can occur with extremely long sentences, using too many logical operators (such as or, and, if-else, then), and quantifiers.    In order to make responses more precise it is better to ask a series of simpler questions.  The two major consequences of overworking memory are that items drop out of working memory or cognitive processing can slow down.  The respondent may simply drop some of the intended meaning of the question and they could spend a lot of time coming up with an incomplete interpretation of the question.

Double Barreled Questions

When a question introduces two or more issues with the expectation of a single response it is a double barreled question.   This inadvertently confuses the respondent and limits the useful data that you can collect.  The respondent is forced to answer two issues with a single response which they may agree with and disagree with respectively as in the following question:

“Is the big city a great place to experience culture and raise a family?”

A person may very well feel that a big city is a great place to experience culture because of its diversity but that doesn’t mean that they want to raise their family there for a variety of reasons.

Leading Questions

A presupposition is anything that is assumed to exist or be true and every question has a number of nouns or propositions that are presumed to be true.   Questions that contain a false presupposition are known as leading questions.   Under certain circumstances false presuppositions can cause respondents to make incorrect inferences about what occurred and to misremember events.  Imagine a cop asking the following question:

“At what point did you quit speeding when you saw the truck swerving onto oncoming traffic?”

Pretend you weren’t speeding.  There is evidence to suggest that asking leading questions like this can cause respondents to misremember what really happened.

Loaded Questions

In order to obtain accurate results a survey question should be neutral in order to welcome as many points of views as possible.  A loaded question contains an incriminating assumption if the respondent accepts it to be true.   An example of this can be found here:

“Do you believe that we should redistribute wealth by allowing tax breaks to expire?”

The loaded word here is “redistribute” as it is often associated with socialism.   The question is asked in such a way that a respondent may not feel comfortable giving an honest answer.   Injecting emotional language in surveys can bias survey results.

Pretesting

If you have a clear understanding of each survey question’s intent you can correct flaws in the survey instrument at the outset by pretesting your questions.  Common methods used in pretesting include expert reviews, forms appraisal, cognitive interviews, and the use of focus groups.

Keep the focus of each question on the specific attribute or phenomenon that you are measuring.  Collecting quality responses is highly dependent on the clarity of questions.   Keep your syntax simple so that your respondent is not easily confused.  Although you want your question to be precise you need to avoid using long, dependent clauses or compound, complex sentences as they can overwork your respondent’s memory and cause them to answer questions incorrectly.  Just as you should not ask a question that is relevant to the research you are conducting you should not use a word that is not relevant to the question you are asking.  Keep it brief. The longer a question is the more likely the respondent is to misinterpret it.  Focus on asking one question at a time.  Brevity can not come at the expense of the intended meaning of the question.  You are simply being succinct or in other words choosing the shortest way to pose a question.  Pretesting can allow you to further refine your questions so that the meaning you intend is not lost on your respondent.

Our Survey Software Hosted service has been upgraded to ActiveCampaign Survey Software version 2.0. Now all hosted users will be able to utilize the new features and updates available in the recent major update of Survey Software.

To learn more about Survey Software hosted visit this page: online survey service.

Yesterday marked the completion of the update to our online Survey Software solution, which included some template changes. The new interface for the application has been modified using the currently implemented CSS, and was changed to comply with the standard interfaces of our other products, mainly Help Desk Software and Email Marketing. We have also swapped the icons used in the Create A Survey process for dropdowns to provide an easier and more intuitive way of generating and distributing your surveys.

If you have any suggestions for development within iSalient, please email feedback@activecampaign.com
We also have a Development Tracking System for any bugs found within the application, along with the iSalient Forum.

If you haven’t taken a look at our downloadable templates, they are available here.

We have recently released a new license option for ActiveCampaign Survey Software; the Application Service Provider License.

With this new license option you are able to create sales plans, add clients, manage your clients, and more. This allows you to resell ActiveCampaign Survey Software from a local copy utilizing the sales and management features currently used for our hosted online survey solution.

The license cost for this type of license is a yearly fee starting at only $1,250/yr with plenty of room for profit.

Should you have any questions – just let us know at sales@activecampaign.com!

As of November 16, 2005 ActiveCampaign Survey Software “hit the shelves” with both Hosted and Owned online survey software. “What is the difference exactly?”, you may be thinking. The largest difference between the two available copies of the Survey Software are the options you have for using the Application along with an array of choices for purchasing. Our hosted solution will allow you to “pay-as-you-go” with monthly subscriptions, while our Owned Licenses are a one time fee that allows you to download and use the application on your own server. The Monthly/Hosted Plans include support for as long as you subscribe to the service, and the Owned Licenses include 6 months of support, and after the initial 6 months ends, the option to renew.

Please take a look at our Order Now page or Survey Software for the latest in the prices. We also have the Feature Tour, Benefits page, and many Screenshots available.

Two new question types have also been added to the Survey Creation process. Form sliders, either graphical or not, can be inserted into your surveys giving your users a faster, more enjoyable way to fill out your online form. Be sure to sign up for the Trial to see the sliders in action. All of the features that you will have available to you in the trial will also be available in the Hosted, and Owned copies of the Application.

If you have any sales questions or any comments, be sure to Contact us.