A survey is a method of gathering information from a number of individuals, known as a sample, in order to learn something about the larger population from which the sample is drawn.  Although surveys come in many forms, and serve a variety of purposes, they do share certain characteristics.  In order for the objectives of a survey to be met, the results must reliably project on the larger public, from which the sample is drawn.  A sample can be scientifically chosen so that each individual in a population has a known chance of selection.  This ensures that a sample is not selected haphazardly or uses only those eager to participate.  The sample size for a survey will depend on the degree of reliability necessary and how the results are to be used.  A properly selected sample should be able to reflect the various characteristics of a total population within a very small margin for error.  There are many surveys that study the total adult population but many others that focus on selected populations:  employees, academics, industry experts, computers users, or customers that use a particular product or service.

Some surveys focus on opinion and attitudes while others are more concerned with collecting factual information. Many surveys combine questions of both types.   A respondent could be asked what they heard or read about an issue, what they know about it, their opinion, how strongly they feel and why, interest in the issue, past experiences with it, and also certain demographic information which will help the survey analyst classify the responses (such as age, sex, marital status, occupation, and place of residence).  Questions can be open ended (”What does that make you think of?”) or closed (”Do you agree or disagree?”); they may ask the respondent to rate a product or a service on some kind of scale; they may ask for a ranking of various alternatives.  The questionnaire could be very brief – a few questions taking no longer than five minutes, or it could take a demanding hour or more of a respondent’s time.  A survey is usually rooted in situations where an individual or institution is confronted with an information need and no existing data will suffice. Once the information need has been identified and a determination made that existing data is inadequate, objectives are laid out for the investigation. These objectives should remain as specific, clear cut and unambiguous as possible.

Designing the questionnaire is a critical stage of the survey development process.  The questionnaire links the information need to the realized measurement.  Scaling techniques used for measurement can be comparative or non comparative.  If proper care and attention is not placed on clearly defining concepts and unambiguously phrasing questions, the resulting data is apt to contain serious biases.  Questions used in surveys must be refined to minimize interpretation problems and thus reduce measurement error.  If the respondent is unable to understand a question or fails to comprehend the question the way it was intended then the data is neither valid nor reliable.  A question must use language in a way that makes the intended observer’s meaning behind that question obvious.   In order to do this a survey must reflect an understanding of the population being sampled. For example, many people do not distinguish between robbery which requires the theft to be in the immediate presence of the victim and burglary which can involve breaking and entering without any confrontation.  The National Crime Survey, done by the Bureau of the Census, does not even mention the word “robbery” when it asks questions about robbery victimization.  Rather they ask several questions that use universally understood phrases, consistent with the operational definition of robbery, that when used together are able to capture the desired responses.

Keeping response errors and biases to a minimum factor heavily in designing a survey. How questions are interpreted by respondents must be carefully considered.  Also, you need to consider the length of the survey. If a questionnaire is too long than it can be burdensome to the respondent, inducing respondent fatigue that leads to response errors, refusals, incomplete questionnaires, and can contribute to higher non-response rates in subsequent surveys involving the same respondent.  Memory plays an important role when surveys deal with past events.  For most people the greater the demand a question places on memory, such as being asked to recall trivial details occurring on any given random date, the less accurate the responses and therefore the less reliable the survey data that is collected.  An appropriate choice of reference period should be made so that a respondent is not forced to report on events that happened too long ago.  Grouping similar questions together can reduce the cognitive burden a survey places on a respondent.  The validity of a survey can be put in jeopardy if questions are too sensitive, if they may prejudice the respondent, if they unduly invade a respondent’s privacy, and if the information that is sought is too difficult for a willing respondent to provide.

A web survey is one of five general ways research is typically conducted using surveys.  Surveys are also given using mail (the kind that requires postage), telephone, in-person interviews, and intercept surveys (respondents are “intercepted” as they pass through a heavily trafficked area).  The web survey most closely resembles the type of surveys which were traditionally mailed out.  Only this time you’re using email to send a link that takes you to an HTML form that is designed for completion and submission through the computer.   Information is collected quickly because as soon as your respondent has finished taking the survey, their responses are immediately submitted.  Web surveys can also be embedded on your web site using what is known as an Iframe.  An IFrame is an HTML element that makes it possible to embed an HTML document inside of another HTML document.   You can also create a Pop-Up survey on your web site so that when a site visitor visits a page the HTML form pops up from their web site.   If they have pop-up blockers enabled then this method will prevent them from taking your survey so you can also create a link to the survey which you can display on your web site.

The web survey is a cost effective way of administering a survey that allows you to collect large amounts of information without having to pay for interviewers, paper supplies or postage, and does not require separate data entry for responses to be processed.  The rise in web based surveys is due in no small measure to the increasingly widespread availability of computers.   Particularly in organizational or professional settings, the ability to receive a questionnaire and complete it at home or in the office on a computer is very convenient for most people.  You can expect to wait at least a few weeks for a questionnaire that is mailed out to a respondent to be returned.  A web survey allows you to rapidly collect data in a timely manner.   Information can be collected and processed in just a few days.    It also allows respondents ample time to carefully consider response selection and to enter in text for open ended questions.  If factual information is required then the respondent has enough time to consult their records.

You can increase response rate to web surveys by sending respondents a pre-notification of the intent of the survey.   This lets your respondents know what the survey is about before they invest any of their time into taking it.  You can also send follow-up reminders to respondents that have not completed the survey. This can be used in a situation where a respondent must complete the survey within a given date range.  Web surveys assume a minimal level of computer literacy so keeping your surveys simple can also help your response rate.  The technical nature of online surveys make them ideal for specialized or well defined populations that have access to an email account and/or a computer.  Coverage bias may result when targeting populations where computers or internet access are not widespread.  Determining an appropriate type of survey to administer to your respondents must take the population that is being measured into consideration.

Survey questions often ask respondents for autobiographical information.  The accuracy of responses to these questions is dependent on the respondent’s ability to recall memories.  Unless you have a condition known as hyperthymestic syndrome you do not have a superior autobiographical memory.  Neither do your survey respondents.  In fact there have only been three confirmed cases of individuals with hyperthymestic syndrome,  a name given by researchers at University of California-Irvine who used the greek for excessive (hyper) and remembering (thymesis).  One of the subjects researched went by the initials AJ until she used her real name, Jill Price, in a book published last May titled “The Woman Who Can’t Forget: The Extraordinary Story of Living with the Most Remarkable Memory Known to Science – A Memoir.”  She was originally the subject of a study published in the Journal Neurocase in 2006 that led to the discovery of two other cases: a Wisconsin man named Brad Williams and Rick Barron from Ohio.  What makes them extraordinary is their ability to recall specific events from their personal experience and the abnormal amount of time they spend thinking about it.  When you ask them about a random date they can describe events that occurred on that day,  tell you what the weather was like, and other trivial details that most people would not be able to recall.   For most people the greater the demand a question places on memory, such as being asked to recall trivial details occurring on any given random date, the less accurate the responses and therefore the less reliable the survey data that is collected.  Because recall is not reliable, respondents rely on inferences to fill in the blanks for any details that they are not able to recall.  Understanding the cognitive processes involved with memory in survey response can help you improve the reliability of your surveys.

In order to appreciate how remarkable an accurate memory is then we must first understand what goes into forming and retrieving a memory.  Memory is usually understood in general terms as an individual’s mental ability to store, retain, and recall information.  Memories are stored in nerve network patterns.  Each nerve cell in the brain connects with a thousand other nerve cells.  Memories are retained on the basis of whether these connections are strong or weak. When we learn to do something better such as play a musical instrument then these connections become stronger.  Information that comes from a respondent becomes more reliable when a respondent is exposed to the same information repeatedly.  When we learn to ignore something such as a the constant humming coming from a train near an apartment then these connections become weaker. Likewise the information that a survey respondent is likely to remember is not going to come from experiences they have consciously sought to ignore.  A nerve cell is also referred to as a neuron.  Memories are recalled when we activate a network of interconnected neurons.  Information comes to us through our sense of sight, smell, taste, hearing, and touch.  When we recall a memory we re-fire the same neural paths that we used to sense the original experience and in a way we recreate the event.  A question may ask about an event that was experienced by a respondent but it could also ask about a concept or an idea.  When memories come from concepts or ideas we extract the essence of sensed experience to form generalized concepts.  This means that actual events and the how they are perceived by the respondent’s senses have a physical basis even when questions are only relevant to a particular concept or idea.

In the movie “50 First Dates” Adam Sandler must win Drew Barrymore’s heart each day as if it were the first time they met because she suffers from memory loss as the result of a brain injury.  She is no longer able to encode, store, and retrieve new memories.  Every morning when she wakes up with no recollection of the previous day.  She is able to recall previously stored memories from before she suffered an injury but she is no longer able to create new memories based on recent events.  Short term memory allows you to remember recent events, experiences, and information while long term memory stores information so that it can be used again and again.  In order for Drew Barrymore to create new memories she must store information in a usable form in a process known as encoding.  Memory must then be stored for later use.  Much of these stored memories lie outside of our awareness until we need it. She must then be able bring the stored memories into conscious awareness.  No matter how hard Adam Sandler tried he could not get Drew Barrymore to remember him the next day because the brain injury she suffered did not allow memories to ever develop to the point where they were able to be encoded, stored, so they could be later retrieved.  When a question is asked in a survey it is assumed that any reference to past events are in a place where they can be brought into conscious awareness.  The stage model of memory holds that memory occurs in three stages: sensory memory, short term memory, and long term memory.

During the sensory memory stage we hold onto sensory information for fractions of a second before some of the data is moved over to short term memory. Only the data that catches the respondent’s attention is moved to short term memory.  Data that does not catch their attention is lost for good.  Questions that are not based on issues that are likely to attract a respondent’s attention initially may not have any basis for reference.  Short term memory is also known as working memory.   When a survey question is too long, contains too many embedded clauses or complex syntax, it can impose a cognitive burden on the respondent’s working memory.  Every respondent has a different level of working memory.   Working memory is generally assumed to be around 20-30 seconds.  Many short term memories are forgotten.  This is because short term memory is small. It can hold about 7 items at one time.  However when information is attended to then it can often be brought to the attention of working memory.

Remember when we discussed hyperthymestic syndrome.  One of the criteria established for this was an excessive amount of time thinking about past experiences.  Memories are therefore present in working memory even when they concern seemingly trivial events.  Unless your respondent is unusually obsessed with the question you are asking then this is not always going to be the case.  Particularly when you ask them a question that they may have not thought about for a while.  The accuracy of survey responses decreases over time.  If a question is relevant to a respondent then it will be easier for them to retrieve this information into working memory.  Therefore the questions that you ask your respondent are likely to increase in accuracy in relation to relevancy.  When memory passes into long term memory it exists on a preconsious or unconscious level.   While this information exists largely outside of our level of awareness it can still be brought into working memory fairly easily.  Long term memories can be used to make decisions, interact with others, and solve problems.  The processes involved with long term memory are the same processes that Drew Barrymore could no longer accomplish in “50 First Dates.”   The three processes involved are encoding, storing, and retrieval.

Finally, memories are arranged in clusters in which related information is grouped together so that it can then be parsed more easily.  When information is organized into categories it becomes easier to remember and therefore recall into working memory.  You might associate a coffee shop with hours spent studying, reading, and the intelligent or funny conversations you had while you were there. Over time similar events you experienced while at that coffee shop begin to get grouped together.   As time passes you are no longer able to separate what you think you experienced at the coffee shop from what you infer happened.  If someone asked you what you did at a coffee shop you have not been at for awhile then all your memories will blend in with other memories.  It might seem like you were either reading something or talking with someone you usually went there with.  However, certain events while you were at the coffee shop may trigger memories that distinguish one experience from another.  The time you spilled coffee on your laptop for example.  Including cues in survey questions can allow respondents to distinguish one event from another and improve the accuracy of their survey responses.  Another way to improve memory in surveys is by asking questions that surround a target event in order to bring your respondent back to the event’s original memory encoding. For example, you may be able to recall what you were doing at the coffee shop better if you are reminded that you slept through an exam the next day.  Understanding how memory is used to recall events can improve the accuracy of your survey by separating events to the extent that you are able to distinguish them.

With the recent release of our Installer for the Downloadable Edition of ActiveCampaign Survey Software, we have been receiving many extremely good comments from our users. The release is available here for the duration of the Beta Testing Period. We also have our Hosted Survey Software Solution available for free during our Beta Testing Period. You can sign up and try it for free by visiting the Trial Sign-Up Form.

Updates to the Hosted and Downloadable Editions of iSalient include a more streamline interface; as we have taken out the grapics from the top of the applications main interface and have replaced them with faster loading CSS styles. The Respondent Response Reporting screen has been overhauled, and now displays with the custom style that you have used for that particular survey. Users can enjoy a seemless process of taking online surveys, to viewing theirs and others results. Reporting has been revamped with options for three different colors allowing for more graphing customizations, along with Reporting Options that include Header and Footer addition, nine different Display Options, Active and Inactive Status, and Shared settings. The Enterprise Edition, although it is not available for testing yet, also has had some new development with the additon of our integration with Verisign, more subscriber management tools including customer perging options, and faster searching results.

Overall, our Beta period has been going steady, but we are always looking for more feedback, and for more Beta Testers. Its a great way to get a feel for the software before you decide to purchase, and to get any features into the application that you feel that you would use often and that are not included as of yet. Because this is a Beta Period, all of the submissions for features will be considered and the development turnaround for those features will be faster than normal.

We do have a few more steps to take before we will be releasing both the Hosted and the Downloadable Editions of iSalient, but the time for that release is coming up quick. So keep an eye in our Blog, as well as the Survey Software main page for survey software prices.

Have any questions about ActiveCampaign Survey Software?

If you are a current Beta Tester please use the form within the application (near the footer) to comment on your experience with the Hosted Solution. If you would like to be contacted about specific details on the features of the application, you can email me at derek@activecampaign.com, or if you would like to be contacted by our Sales Department, you can simply fill out our sales form here.

-Derek Entringer
iSalient Project Manager