A survey is a method of gathering information from a number of individuals, known as a sample, in order to learn something about the larger population from which the sample is drawn.  Although surveys come in many forms, and serve a variety of purposes, they do share certain characteristics.  In order for the objectives of a survey to be met, the results must reliably project on the larger public, from which the sample is drawn.  A sample can be scientifically chosen so that each individual in a population has a known chance of selection.  This ensures that a sample is not selected haphazardly or uses only those eager to participate.  The sample size for a survey will depend on the degree of reliability necessary and how the results are to be used.  A properly selected sample should be able to reflect the various characteristics of a total population within a very small margin for error.  There are many surveys that study the total adult population but many others that focus on selected populations:  employees, academics, industry experts, computers users, or customers that use a particular product or service.

Some surveys focus on opinion and attitudes while others are more concerned with collecting factual information. Many surveys combine questions of both types.   A respondent could be asked what they heard or read about an issue, what they know about it, their opinion, how strongly they feel and why, interest in the issue, past experiences with it, and also certain demographic information which will help the survey analyst classify the responses (such as age, sex, marital status, occupation, and place of residence).  Questions can be open ended (“What does that make you think of?”) or closed (“Do you agree or disagree?”); they may ask the respondent to rate a product or a service on some kind of scale; they may ask for a ranking of various alternatives.  The questionnaire could be very brief – a few questions taking no longer than five minutes, or it could take a demanding hour or more of a respondent’s time.  A survey is usually rooted in situations where an individual or institution is confronted with an information need and no existing data will suffice. Once the information need has been identified and a determination made that existing data is inadequate, objectives are laid out for the investigation. These objectives should remain as specific, clear cut and unambiguous as possible.

Designing the questionnaire is a critical stage of the survey development process.  The questionnaire links the information need to the realized measurement.  Scaling techniques used for measurement can be comparative or non comparative.  If proper care and attention is not placed on clearly defining concepts and unambiguously phrasing questions, the resulting data is apt to contain serious biases.  Questions used in surveys must be refined to minimize interpretation problems and thus reduce measurement error.  If the respondent is unable to understand a question or fails to comprehend the question the way it was intended then the data is neither valid nor reliable.  A question must use language in a way that makes the intended observer’s meaning behind that question obvious.   In order to do this a survey must reflect an understanding of the population being sampled. For example, many people do not distinguish between robbery which requires the theft to be in the immediate presence of the victim and burglary which can involve breaking and entering without any confrontation.  The National Crime Survey, done by the Bureau of the Census, does not even mention the word “robbery” when it asks questions about robbery victimization.  Rather they ask several questions that use universally understood phrases, consistent with the operational definition of robbery, that when used together are able to capture the desired responses.

Keeping response errors and biases to a minimum factor heavily in designing a survey. How questions are interpreted by respondents must be carefully considered.  Also, you need to consider the length of the survey. If a questionnaire is too long than it can be burdensome to the respondent, inducing respondent fatigue that leads to response errors, refusals, incomplete questionnaires, and can contribute to higher non-response rates in subsequent surveys involving the same respondent.  Memory plays an important role when surveys deal with past events.  For most people the greater the demand a question places on memory, such as being asked to recall trivial details occurring on any given random date, the less accurate the responses and therefore the less reliable the survey data that is collected.  An appropriate choice of reference period should be made so that a respondent is not forced to report on events that happened too long ago.  Grouping similar questions together can reduce the cognitive burden a survey places on a respondent.  The validity of a survey can be put in jeopardy if questions are too sensitive, if they may prejudice the respondent, if they unduly invade a respondent’s privacy, and if the information that is sought is too difficult for a willing respondent to provide.

A web survey is one of five general ways research is typically conducted using surveys.  Surveys are also given using mail (the kind that requires postage), telephone, in-person interviews, and intercept surveys (respondents are “intercepted” as they pass through a heavily trafficked area).  The web survey most closely resembles the type of surveys which were traditionally mailed out.  Only this time you’re using email to send a link that takes you to an HTML form that is designed for completion and submission through the computer.   Information is collected quickly because as soon as your respondent has finished taking the survey, their responses are immediately submitted.  Web surveys can also be embedded on your web site using what is known as an Iframe.  An IFrame is an HTML element that makes it possible to embed an HTML document inside of another HTML document.   You can also create a Pop-Up survey on your web site so that when a site visitor visits a page the HTML form pops up from their web site.   If they have pop-up blockers enabled then this method will prevent them from taking your survey so you can also create a link to the survey which you can display on your web site.

The web survey is a cost effective way of administering a survey that allows you to collect large amounts of information without having to pay for interviewers, paper supplies or postage, and does not require separate data entry for responses to be processed.  The rise in web based surveys is due in no small measure to the increasingly widespread availability of computers.   Particularly in organizational or professional settings, the ability to receive a questionnaire and complete it at home or in the office on a computer is very convenient for most people.  You can expect to wait at least a few weeks for a questionnaire that is mailed out to a respondent to be returned.  A web survey allows you to rapidly collect data in a timely manner.   Information can be collected and processed in just a few days.    It also allows respondents ample time to carefully consider response selection and to enter in text for open ended questions.  If factual information is required then the respondent has enough time to consult their records.

You can increase response rate to web surveys by sending respondents a pre-notification of the intent of the survey.   This lets your respondents know what the survey is about before they invest any of their time into taking it.  You can also send follow-up reminders to respondents that have not completed the survey. This can be used in a situation where a respondent must complete the survey within a given date range.  Web surveys assume a minimal level of computer literacy so keeping your surveys simple can also help your response rate.  The technical nature of online surveys make them ideal for specialized or well defined populations that have access to an email account and/or a computer.  Coverage bias may result when targeting populations where computers or internet access are not widespread.  Determining an appropriate type of survey to administer to your respondents must take the population that is being measured into consideration.

People ask questions in order to learn what they wouldn’t otherwise know.  When a respondent answers a question on a survey they must rely on their comprehension to interpret what that question means.  Then during a process of memory retrieval they access relevant information for content to be included in their answer.  Judgment is used for comparing and evaluating ideas in a way that allows information to be synthesized.  During response selection an answer is selected and placed in the required response format. All stages of the cognitive process just described rely on the first stage which is question interpretation. For this reason questions used in surveys must be refined to minimize interpretation problems and thus reduce measurement error.  If the respondent is unable to understand a question or fails to comprehend the question the way it was intended then the data is neither valid nor reliable.  A question must use language in a way that makes the intended observer’s meaning behind that question obvious.   In order to do this a survey must reflect an understanding of the population being sampled. Before administering a survey you need to ask yourself whether your questions measure what you intend them to measure.  Is it valid?   Do your respondents interpret your questions the same way.  Is it reliable?  In order to ensure that the respondent is answering the question we think they are answering it’s important to make questions clear and precise.

Level of Wording

Question wording can affect the answers that are received.  The wording used in questions should be appropriate to the level of education and characteristics of the respondents.  A researcher must be aware of the respondent’s understanding of difficult words, colloquialisms, jargon, technical terms, and language proficiency when settling on an appropriate level of wording.  They also must be aware of words or phrases that are gender or ethnic sensitive.  If the language used is not the first language of all your respondents then the level of language proficiency of all your respondents must be carefully considered.   Level of wording is important because a respondent that is embarrassed to admit that they do not know something is likely to give any answer rather than admit they do not know.  You may also see a lot of neutral response choices such as  “do not know” or “no opinion.” Also, it can lead to a high rate of refusal to complete survey rate.   In general you should use wording that is simple, direct, and to the point.  A question that would not be appropriate to the level of wording of your average survey respondent could be:

“Did the attorney deliberately obfuscate the truth by inundating the jury with statements that were verbose and declamatory?”

An understanding of the population you are administering the survey to must be worked out in advance.   If you are administering a survey to attorneys you would use a different level of wording than you would if you were asking the same questions to a group of layman.

Ambiguous Questions

If there is more than one way to interpret a question than that question is ambiguous.  When a question is vague it has an imprecise range of application.  If a question uses vague wording  or is ambiguous it can lead respondents to interpret a question in a variety of ways.  An example of an ambiguous question might be:

“Who do you buy groceries for?”

It is not clear whether either you or any household pets are included in this question.   Is this asking who you usually buy groceries for?  Does this include anything you might pick up for your parents that you no longer live with?  A better way to ask this question would be:

“What is the total number of household occupants that you purchased groceries for in the past year?”

Ambiguity can also arise if a respondent is unclear about what type of question is being asked, the motive for the question being asked, the purpose for a question being asked, and when inferring motives behind a question that have no truth in reality.

Confusing Questions

A question is confusing if it leads to a feeling of uncertainty in the respondent when the intent or meaning is not clear.  Here is an example:

“Does it seem possible or impossible that the Cubs will win the World Series, or do you feel they will always find a way to lose?”

1_ It seems possible

2_ It seems impossible

Here the respondent may feel trapped into either agreeing that the Cubs will find a way to lose or saying that it is impossible for the Cubs to win the World Series.   A clearer way to ask this question would be:

“Does it seem possible that the Cubs will ever have a chance to win the World Series?”

1_It is possible that it could happen.

2_It is not possible that could happen.

Working Memory Overload

In order for a respondent to process a question it can not impose a cognitive burden on their working memory.   Working memory is short term memory used to hold information while the respondent is comprehending a question.  This can often happen when a question attempts to be too precise, by injecting complex syntax and embedded clauses,  in an attempt to clearly define a question.   Earlier we talked about vagueness and ambiguity.   Couldn’t you make our earlier question more precise by rewording it as:

“Including yourself, any pets you own, but not including any relatives or friends that do not live in your household for at least 320 days a year, how many household occupants did you buy groceries, by groceries we mean any item of food bought at the grocery store, for in the last year?”

The problem with asking questions like this is that you are going to tax your respondents memory and some of them will not even get what you are trying to say.  Individuals differ in how much their working memory can handle.  Questions that pose a burden to some may not pose a burden to others.  Avoid complex syntax in your survey questions.  This means that its grammatical composition can not be dense, structurally ambiguous, or not well formed syntactically.   Memory overload can occur with extremely long sentences, using too many logical operators (such as or, and, if-else, then), and quantifiers.    In order to make responses more precise it is better to ask a series of simpler questions.  The two major consequences of overworking memory are that items drop out of working memory or cognitive processing can slow down.  The respondent may simply drop some of the intended meaning of the question and they could spend a lot of time coming up with an incomplete interpretation of the question.

Double Barreled Questions

When a question introduces two or more issues with the expectation of a single response it is a double barreled question.   This inadvertently confuses the respondent and limits the useful data that you can collect.  The respondent is forced to answer two issues with a single response which they may agree with and disagree with respectively as in the following question:

“Is the big city a great place to experience culture and raise a family?”

A person may very well feel that a big city is a great place to experience culture because of its diversity but that doesn’t mean that they want to raise their family there for a variety of reasons.

Leading Questions

A presupposition is anything that is assumed to exist or be true and every question has a number of nouns or propositions that are presumed to be true.   Questions that contain a false presupposition are known as leading questions.   Under certain circumstances false presuppositions can cause respondents to make incorrect inferences about what occurred and to misremember events.  Imagine a cop asking the following question:

“At what point did you quit speeding when you saw the truck swerving onto oncoming traffic?”

Pretend you weren’t speeding.  There is evidence to suggest that asking leading questions like this can cause respondents to misremember what really happened.

Loaded Questions

In order to obtain accurate results a survey question should be neutral in order to welcome as many points of views as possible.  A loaded question contains an incriminating assumption if the respondent accepts it to be true.   An example of this can be found here:

“Do you believe that we should redistribute wealth by allowing tax breaks to expire?”

The loaded word here is “redistribute” as it is often associated with socialism.   The question is asked in such a way that a respondent may not feel comfortable giving an honest answer.   Injecting emotional language in surveys can bias survey results.

Pretesting

If you have a clear understanding of each survey question’s intent you can correct flaws in the survey instrument at the outset by pretesting your questions.  Common methods used in pretesting include expert reviews, forms appraisal, cognitive interviews, and the use of focus groups.

Keep the focus of each question on the specific attribute or phenomenon that you are measuring.  Collecting quality responses is highly dependent on the clarity of questions.   Keep your syntax simple so that your respondent is not easily confused.  Although you want your question to be precise you need to avoid using long, dependent clauses or compound, complex sentences as they can overwork your respondent’s memory and cause them to answer questions incorrectly.  Just as you should not ask a question that is relevant to the research you are conducting you should not use a word that is not relevant to the question you are asking.  Keep it brief. The longer a question is the more likely the respondent is to misinterpret it.  Focus on asking one question at a time.  Brevity can not come at the expense of the intended meaning of the question.  You are simply being succinct or in other words choosing the shortest way to pose a question.  Pretesting can allow you to further refine your questions so that the meaning you intend is not lost on your respondent.

In order to control the future you must control the past.  While you cannot rewrite the past in order to take over the future you can use it to predict it.  The past allows you to learn more about your subscribers so that you can then target your email marketing approach toward them.  When it comes to understanding your customers there is no better predictor of future behavior than the past.   Purchasing behavior is a response to an object of interest.  If behavior is recent then its more likely to happen again.  Perhaps that object of interest is a song you recently heard when you were around an acquaintance.  You really like the song when it’s played but you do not know who its from.  You spend the entire next day trying to remember who the song was from.  You would purchase if you only knew.  You completely forget about the experience the next day.   As an object loses its recency interest fades over time.

However,  if a positive impression is initially formed it can lead to purchasing behavior later.   This is why its important to work on forming good impressions with everyone you meet.  You had a positive impression of your acquaintance because they had something to offer you in the form of good music.  This did not translate in a purchasing decision the first time around but let’s see what happens when you run into the same acquaintance again.  By the time you run into them you are no longer interested in finding out who played the song you liked.  You do however trust their opinion so you can say they have established credibility.  When you are around them this time you hear a different song that you are impressed with.   This time you get your acquaintance’s number in case you forget the name of the song when you go out to purchase it tomorrow.   You become friends with your acquaintance and the more time you spend with them the more likely you are to go out and purchase music.  Interest peaks when exposure is recent.  Your subscribers are more likely to purchase from you in the period immediately following a mailing.   The longer they do not respond to a mailing the less likely they are going to.  You have a window of interest but after that they will forget about you completely.  One way of measuring purchasing behavior is by measuring when the purchase was made.  Before you met this person you were not exposed to music so recency was low.   Now that you are friends recency has increased.

The frequency with which an action occurs in the past can be used to measure interest.  You are now purchasing more and more music with greater frequency as a result of your friend.  The frequency of your product purchases is directly proportional to the amount of time you spend with your friend.  One other way of thinking about frequency is how it relates to exposure.  Have you ever played a song that you really liked over and over again until you got sick of it?  If you’re frequently exposed to a song then it sticks in your head and this is a good thing or a bad thing depending on what you think of the song.   The fact that you’re voluntarily playing the song over and over again means you are interested in it.

Notice what is happening here.  When your friend was merely an acquaintance they did not force their music taste on you.  They gave you a good experience by exposing you to what they considered good music.  They established trust with you.  When you later ran into them they were able to repeat the positive experience.  Credibility began forming and this led to a purchase decision. As your relationship progressed your acquaintance became a friend.  You begin basing your purchasing decisions on their judgment.   You do this because they demonstrated reliability.  Which led to an increase in frequency of purchases.

A good newsletter does the same thing.   It exposes subscribers to something that’s worth having.  The newsletters you send should be of interest to your subscribers.   If you are able to demonstrate how your newsletter is of interest you will establish trust.  From there you establish credibility by repeating the positive experience without being too aggressive.   Your subscriber is an acquaintance that is just getting to know you.  When you are able to demonstrate reliability to your subscriber they will begin relying on your judgment when forming the basis of their decisions.

Now an emerging need for treating different customers differently is apparent.  You’re not treating some customers better than others.  You are treating them like people and using their past behavior to govern your future behavior toward them.  Which of your customers recently purchased from you?   How many times did they do it.  How much did they spend on individual purchases?   You are asking yourself questions in order to learn which of your subscribers are your best customers so that you can target your marketing approach to them.   In the process you are attempting to turn acquaintances into friends by showing respect for the relationship.  A healthy relationship grows because it is mutually beneficial to both parties.

Now let’s consider monetary value and what it means to be a best customer.  Some of my favorite albums of all time have been concept albums such as Wilco’s A Ghost Is Born, The Decemberists’ The Crane Wife, Radiohead’s OK Computer, and Pink Floyd’s The Dark Side of the Moon.  The beauty of a concept album is that you are not listening to it just for one song.  You have to purchase the entire album to appreciate the concept!  Every song is made fuller and richer by the songs that surround them.  They also have a powerful emotional component. Perhaps you’ve heard a song that you really like that led you to purchase an entire album.  Once you purchased the album that led you to purchase other albums from the band until you owned their entire music catalog. You then tried to turn your friends into fans of the music so that you wouldn’t have to enjoy it alone.  If you have then you would be that band’s best customer.  You would be one of the true fans that the band relies on to make a living.  Once you have a best customer you can market to them differently because they have demonstrated through their purchasing behavior that the relationship demands it.

You would market to a best customer for a band such as Pink Floyd differently than you would for a weaker customer.  A best customer may be interested in owning all of their albums in CD quality,  having all the original vinyl copies as well as digitally remastered CDs, box sets, concert recordings, T-shirts and posters, DVD footage of live concerts as well as documentaries, Nick Mason’s Personal History of Pink Floyd, and solo albums from the band members. Marketing all of these products would be of great interest to a devoted fan of Pink Floyd but would be of lesser interest to a fan who simply wants to have MP3 albums of their most seminal work.  Here you could use a newsletter to turn a casual fan into a best customer by giving them insider information on the band in order to strengthen the relationship that fan feels toward the band.  Meanwhile,  you could keep the devoted happy by offering them exclusive opportunities for material that would only be of interest to the band’s most ardent supporters.  One purpose for the newsletter would be to track live performances from surviving band members.   A best customer would not let money be an obstacle for going to see a live performances from David Gilmour or Roger Waters.  You could offer these subscribers the first chance to purchase the best seats available.  At the same time you could keep all your subscribers happy by offering all of them an opportunity to win free tickets.  The best part of having a best customer is the word of mouth advertising they provide.  A Floyd head will try to turn their friends into fans as well.   Perhaps some of these friends may later turn into best customers.

The marketing approach of your newsletter should address the needs of all your subscribers separately.   You are going to earn most of your revenue from your best customers.  So your challenge is to turn good customers into best customers and weak customers into good customers.

When I think of reliability I imagine always knowing what to expect.  If a person is able to produce the same quality work consistently then they are considered reliable.  You see it in sports all the time. Certain players have a knack for coming through in key situations no matter how late in the season or how worn down they are.   However, I can imagine few jobs that require more reliability than a surgeon.  Having an off day for them could prove disastrous. For a measure to be reliable it must demonstrate consistency as well as repeatability.  When carrying out research our results should be accurate across a range of measurements.  In surveys you would like to think that you would get the same response no matter what mood your respondent is in but that is not always the case.   A surgeon must deal with difficult situations while showing the same precision and reliability.  That is a quality to be admired but you can not always expect everyone to act like a surgeon at all times.  It’s also possible that your respondent won’t know what you mean when you ask them a certain question resulting in an answer that is entirely different from what you are attempting to measure.

Test-retest Reliability

If your survey respondent had to take your survey again would they answer the same questions the same way?  Test-retest reliability measures reliability over time.  A number of factors can affect reliability over time such as a person’s mood, time of the day, where the questions are placed in the survey (context effect), circumstantial events, vagueness, etc.  A good test will take into account factors that may influence survey results over time and minimize them so that results show little variation. If a test is unreliable then any one of a number of factors can lead to varying results depending on when the question is asked.  In general the more time a person takes between retesting the more variation you can expect in the responses.

If you ask Joe Q what he thinks about Candidate X on Tuesday he may view him favorably because X gave a really good speech on Monday.   Say Candidate X is indited later in the week in a corruption scandal.  Joe previously indicated that a candidate’s integrity is very important.  Last week he said that he was leaning toward Candidate X.  Now that Candidate X has been exposed you may think he is likely to give you a different response if you asked Joe the same question next Tuesday.   The reliability of opinion polls can be doubtful depending on the questions we ask because opinions tend to fluctuate over time.  What does Joe Q mean when he rates integrity as very important?  Perhaps Joe Q considers anyone that shares his ideology to have integrity.  Its possible that Joe Q would vote for Candidate X no matter what he thought of him personally because they share the same ideology.  Probing Joe’s past voting record would be more indicative of voter preference than asking a subjective question about integrity.  Asking him more objective questions that would not fluctuate from week to week would have higher test-retest reliability.

Parallel Forms Reliability

Another challenge reliability faces is in knowing what the best questions to ask are.  What does Joe Q mean when he rates integrity as very important?  Could we come up with better questions to predict how voters like Joe Q would vote? Another way to improve the reliability of a survey is to ensure that it is representative of the data you are trying to collect.   To do this increase the sample size.  If you are gathering research to find out whether voters like Joe Q are likely to vote for Candidate X then you need to find more people like Joe and ask them different questions or question sets based on the same construct.

You come up with a large set of questions to ask in your survey.  The construct that you are measuring is voter preference. The large question set is split in half and you administer each set to half of the targeted population.  You can then take a look at which questions are better indicators of voter preference. This combines what is known as a split test method with parallel form evaluation.

You can use parallel forms to measure a construct for people that are not like Joe Q.  Here you would divide a population that is representative of all likely voters in two.  Develop a large question set that measures a particular construct and then administer to each half of your representative population.  Now you can learn which questions are better indicators for voter preference for a representative population.

Inter-rater Reliability

This is necessary if you are conducting your survey using an interview process.   If  multiple people are interviewing Joe Q to ask what his opinion on politics is then inter-rater reliability measures the degree to which the observers agree.  This is the best way to measure reliability if you are using observation for your research.

Internal Consistency Reliability

The purpose of asking questions in surveys is to assess a particular construct or idea.  Therefore different questions that measure the same construct should yield similar results.  Reliability is determined on the basis of whether results are consistent for different items that measure the same construct.  For example, you could check for reliability on your survey by asking a respondent two similar questions meant to measure the same thing.

• Average Inter-Item Correlation – when we ask a respondent two similar questions to measure the same construct.  This compares correlations between this and any other paired questions to measure the same construct by calculating the mean of all paired comparisons.
• Average Itemtotal Correlation – where you take the average inter-item correlation and calculate a total score for each item.
• Split-half Correlation – you divide items that measure the same construct into two tests,  apply them to the same group of people, and calculate the correlation between the two scores.
• Cronbach’s Alpha – when we calculate the average split half estimates from a sample population.

In order to draw conclusions, formulate theories, or make generalizations about your research you need to ensure the reliability of the data you collect.  In general reliability is threatened when assessments are taken over time, rely on different standards of judgment, or assessments are highly subjective.  You can improve reliability by ensuring that your surveys are written clearly and without ambiguity.  You should construct your response options so that they are appropriate and meaningful.

Conclusions drawn from analyzing survey data are only acceptable to the degree to which they are determined valid.  Validity is used to determine whether research measures what it intended to measure and to approximate the truthfulness of the results. Researchers often use their own definition when it comes to what is considered valid.  In quantitative research testing for validity and reliability is a given.  However some qualitative researchers have gone so far as to suggest that validity does not apply to their research even as they acknowledge the need for some qualifying checks or measures in their work.  This is wrong.   To disregard validity is to put the trustworthiness of your work in question and to call into question others confidence in its results.   Even when qualitative measures are used in research they need to be looked at using measures of reliability and validity in order to sustain the trustworthiness of the results.  Validity and reliability make the difference between “good” and “bad” research reports. Quality research depends on a commitment to testing and increasing the validity as well as the reliability of your research results.

Any research worth its weight is concerned with whether what is being measured is what is intended to be measured and considers the ways in which observations are influenced by the circumstances in which they are made.   The basis of how our conclusions are made play an important role in addressing the broader substantive issues of any given study. For this reason we are going to look at various validity types that have been formulated as a part of legitimate research methodology.

Face Validity

This is the least scientific method of validity as it is not quantified using statistical methods.  This is not validity in a technical sense of the term.  It is concerned with whether it seems like we measure what we claim.  Here we look at how valid a measure appears on the surface and make subjective judgments based off of that.  For example,  if you give a survey that appears to be valid to the respondent and the questions are selected because they look valid to the administer.   The administer asks a group of random people, untrained observers,  if the questions appear valid to them.  In research its never sufficient to rely on face judgments alone and more quantifiable methods of validity are necessary in order to draw acceptable conclusions.  There are many instruments of measurement to consider so face validity is useful in cases where you need to distinguish one approach over another.  Face validity should never be trusted on its own merits.

Content Validity

This is also a subjective measure but unlike face validity we ask whether the content of a measure covers the full domain of the content. If a researcher wanted to measure introversion they would have to first decide what constitutes a relevant domain of content for that trait.  This is considered a subjective form of measurement because it still relies on people’s perception for measuring constructs that would otherwise be difficult to measure.   Where it distinguishes itself is through its use of experts in the field or individuals belonging to a target population.  This study can be made more objective through the use of rigorous statistical tests.  For example you could have a content validity study that informs researchers how items used in a survey represent their content domain, how clear they are, and the extent to which they maintain the theoretical factor structure assessed by the factor analysis.

Construct Validity

A construct represents a collection of behaviors that are associated in a meaningful way to create an image or an idea invented for a research purpose.  Depression is a construct that represents a personality trait which manifests itself in behaviors such as over sleeping, loss of appetite, difficulty concentrating, etc.  The existence of a construct is manifest by observing the collection of related indicators.  Any one sign may be associated with several constructs.  A person with difficulty concentrating may have A.D.D. but not depression.  Construct validity is the degree to which inferences can be made from operationalizations(connecting concepts to observations) in your study to the constructs on which those operationalizations are based.  To establish construct validity you must first provide evidence that your data supports the theoretical structure.  You must also show that you control the operationalization of the construct, in other words, show that your theory has some correspondence with reality.

• Convergent Validity - the degree to which an operation is similar to other operations it should theoretically be similar to.
• Discriminative Validity - if a scale adequately differentiates itself or does not differentiate between groups that should differ or not differ based on theoretical reasons or previous research.
• Nomological Network - representation of the constructs of interest in a study, their observable manifestations, and the interrelationships among and between these.  According to Cronbach and Meehl,  a nomological network has to be developed for a measure in order for it to have construct validity
• Multitrait-Multimethod Matrix - six major considerations when examining Construct Validity according to Campbell and Fiske.  This includes evaluations of the convergent validity and discriminative validity.  The others are trait method unit, multi-method/trait, truley different methodology, and trait characteristics.

Internal Validity

This refers to the extent to which the independent variable can accurately be stated to produce the observed effect.  If the effect of the dependent variable is only due to the independent variable(s) then internal validity is achieved. This is the degree to which a result can be manipulated.

Statistical Conclusion Validity

A determination of whether a relationship or co-variation exists between cause and effect variables.   Requires ensuring adequate sampling procedures,  appropriate statistical tests, and reliable measurement procedures. This is the degree to which a conclusion is credible or believable.

External Validity

This refers to the extent to which the results of a study can be generalized beyond the sample. Which is to say that you can apply your findings to other people and settings.   Think of this as the degree to which a result can be generalized.

Criterion-Related Validity

Can alternately be referred to as Instrumental Validity. The accuracy of a measure is demonstrated by comparing it with a measure that has been demonstrated to be valid.  In other words, correlations with other measures that have known validity. For this to work you must know that the criterion has been measured well.  And be aware that appropriate criteria do not always exist.  What you are doing is checking the performance of your operationalization against a criteria.  The criteria you use as a standard of judgment accounts for the different approaches you would use:

• Predictive Validity - operationalization’s ability to predict what it is theoretically able to predict.  The extent to which a measure predicts expected outcomes.
• Concurrent Validity - operationalization’s ability to distinguish between groups it theoretically should be able to.  This is where a test correlates well with a measure that has been previously validated.

When we look at validity in survey data we are asking whether the data represents what we think it should represent.  We depend on the respondent’s mind set and attitude in order to give us valid data.  In other words we depend on them to answer all questions honestly and conscientiously.  We also depend on whether they are able to answer the questions that we ask.  When questions are asked that the respondent can not comprehend or understand then the data does not tell us what we think it does.

The order in which questions are asked in your web surveys can influence the results of your survey.  For this reason you should group your questions into logically coherent sections.   Grouping questions that are similar together will make your survey easier to complete for your respondent.  As will grouping questions with a similar format. Your transition from one question to another should be smooth. A poorly constructed survey will confuse your respondents,  bias their answers, and jeopardize the quality of your work.

The following are some guidelines for  sequencing questions for a research survey:

• Introductory Questions – These questions should stimulate interest without being difficult to answer.  Make the first question related to the stated subject matter.  This should be simple and straightforward.  Pretend you’re having a conversation with someone for the first time and you do not want to offend or bore them.
• Sensitive Questions – Place these late in the survey.  If rapport is established then there is a greater likelihood that the respondent will be willing to answer these questions.
• Related Questions – Grouping related questions together gives the respondent a chance to concentrate on specific issues without being distracted.   At the same time remain cognizant of arranging consecutive questions that evoke reflexive (given without adequate thought) responses together.
• Logical Sequence – There should be a clear logical order to the particular series of questions that are being asked.  Questions should be arranged in a sequential or temporal order.

Context effect describes the influence of environmental factors on one’s perception of a stimulus.  Context effect in surveys involve how responses can be shaped by the order of questions, format of responses, and any visual images. Respondents rely on their own unique evaluations, impressions, values, and beliefs when formulating answers.   The context of a question influences what a respondent considers as they answer that question. A question’s context can affect a respondent’s answer by suggesting a standard of comparison.

The use of images in web based surveys may affect respondent’s answers.   One study measured the effect of showing respondents an image of a healthy woman exercising versus a sick woman in bed and then asked them to rate their own health.  When shown the healthy woman respondents consistently rated their own health lower than when exposed to the image of the unhealthy woman.  Three different surveys were used with varying factors such as size and placement of the image and also the location of the question in the survey.

You want to segment your subscriber list based on shared needs so that your subscribers respond similarly to your marketing strategy.   Ineffectiveness arises in email marketing when you do not address your subscribers’ needs or your subscribers become disenchanted with your products or services. Effective email marketing involves developing a relationship with your subscribers that is built around addressing their needs.  Through list segmentation you are able to differentiate between the different needs of your subscribers so that you can then send targeted mailings that your subscribers will respond to in a similar way.  Send messages that are irrelevant to your subscribers’ needs and you risk losing them.  The better you are able to segment your lists the higher your response rate.  Treat your subscribers as if you were in a relationship with them and they will respond to you more.  List segmentation should not be rigid or fixed.  If you are responsive to your subscribers they will be responsive to you. How you segment your subscribers will change over time as you learn more about them.

Sending out email surveys are a good way of collecting additional information about your subscribers.   This information can be used in addition to your subscriber fields to put you in a better position to segment your subscribers.  When using subscription forms you should only ask for critical information.  If you make your subscription form too long you may be creating a barrier for people that would otherwise be interested in signing up to your list.

Including a short survey link in an autoresponder you send out after a subscriber joins your list is a great way of collecting non critical information that can be used to more effectively segment your list. You do not need to ask your subscribers everything you ever wanted to know about them in one survey either.   A short questionnaire with only one or two relevant questions can be very effective.  Your subscriber will be more willing to participate especially if you explain to them that you are asking these questions so that you can send them information that is more relevant to them.  You can then send them more short surveys from time to time in order to collect additional information so that you can continuously segment your subscribers.

Another option would be to send out a survey when a subscriber purchases a product from your website.  You can use this survey to gather preference or opinion data that can be used to better segment your subscriber list.   You can learn how they feel about a particular brand or technology.  These are meaningful segments that allow you to send more relevant messages.   Are your subscribers brand loyal or do they look for the best deal irrespective of brand.    Are they purchasing a camera because they are a photographer or is it only out of necessity for a special occasion?  You can set this up by redirecting your subscriber to a page containing a survey embedded in an iframe after they purchase a product.

You are asking your subscribers about their needs, preferences, and behavioral patterns as part of your market research.   Your survey should ask questions related to the marketplace and an industry rather than focusing on a business.   This information is a good predictor of your subscribers’ future behavior and will form the basis of how they are segmented for truly targeted email marketing.

Many people interested in email marketing, especially those new to it, want to know how many subscribers they can send to through their list at one time.  The thinking here is that email marketing is a sheer numbers game that can be mastered by being handed a large list of prospective leads.  The more emails you can muscle out at one time the greater the chance you have of maximizing your returns. In doing so they succumb to a “cold calling” mentality.  They do this despite the fact that cold calling as a model offers the lowest possible return of all prospecting methods.  What happens is that you annoy the very people you are trying to reach to the extent that your message loses all relevancy to them.  You actually do the opposite of what you set out to accomplish.   You minimize the returns you can expect from your leads and in the process dilute the message you are trying to deliver by sending the same message to too large of a pool. You can also damage your sender reputation and so reduce your email deliverability.

This happens because expectations in such an approach are based on the faulty assumption that if you feed enough people with the same message you cost effectively reach more people that will buy into what you are selling.  The failure of this approach is due to the fact that the right questions aren’t asked in the beginning and there is no strategy mapped out for reaching the audience that is being marketed to.   It’s a brain dead approach to marketing with a clear lack of imagination.  They fail to ask themselves how the message they are attempting to deliver is being received by their subscribers.  Not everyone is going to respond to the same message.    People are too different to respond the way you want them to if you send them the same generic message.

One report showed that the number of read open and click through rates are 2.5 times higher for mailings sent to fewer than 100 subscribers than emails sent to 100,000 subscribers or more.  Effective email marketers send out to lists of subscribers that are segmented and targeted to specific demographics. The email design and email copywriting techniques are targeted specifically at the people the message is being sent to.  As a result their mailings are far more effective.

Ideally you would be able to craft a message that is specific to each and every person that you are trying to reach.   Your ideal subscriber list would consist of an army of one.  In such a scenario you could expect high read open and click through rates.   However,  such an approach is far too impractical to be seriously considered.

You can however increase the relevancy of your mailing by segmenting your list and sending them out to targeted groups of subscribers.  Segmenting lists involve separating lists into demographics that are based on similar needs or interests.  A 2006 study by Jupiter Research found that engaging audiences with more relevant communication can increase net profits by an average of 18 times.

You can begin segmenting your list by creating additional subscriber fields in your subscription forms and sending out targeted mailings to specific groups based on those fields.    List segmentation is an expansive subject which we will explore in greater depth in future articles.

The focus in email marketing has turned away from list building and toward relationship building.   Email marketers learn more information about their subscribers so they can segment their lists into groups that they can reasonably expect to respond similarly.    They then send out targeted mailings to smaller but more specific lists of subscribers.  As a result they can expect higher open and click through rates.

We are now including a set of Spanish language files for Knowledge Management.   A special thanks to Ramon for contributing this.

You can get the language files by downloading the latest version of Knowledge Management.

Let us know if you have any questions!

Knowledge Management includes a powerful user / user group system for managing users. It also allows you to connect any number of external login sources into the software in a seamless and efficient manner. You are able to connect to any external sources by using one of our preset login sources or creating your own. Connect to external databases (MySQL, MSSQL, PostgreSQL, Oracle, Etc…), any Active Directory / LDAP source, or literally any other source by using custom login source files.

Active Directory / LDAP Integration…

With your purchase of any Knowledge Management license you are able to integrate with Active Directory/LDAP. Simply go to the “login sources” page and enable the LDAP login source. You can then choose the user group that should be used for your LDAP users.

Various other login sources available…

By default we include a set of login sources that can be used (without any code changes) that include: External ActiveCampaign Products, vBulletin, Active Directory / LDAP. You can also easily create your own login source.

Create your own custom login source(s)…

While we do include some popular login source files, some users may need to use a login source that we do not have a login source file for yet. The process of adding new/custom login source files is quite simple and quick. Often it only takes about 10 lines of code! We have an easy-to-follow tutorial to guide you through the process. And if you have any questions you can contact support and we will be glad to assist you.

Set the order and permissions of each source…

You can have multiple login sources enabled. Thus you can have it check LDAP first, then an external database, and then the local user system. The order is entirely up to you! You can also set the user group(s) for each login source. This allows you to set detailed permissions for each login source.