ANECDOTAL EVIDENCE

Every time a student of astrology or an astrologer looks at the astrological birth chart of a person and compares the information provided in the birth chart and the person’s life, research is being conducted. Every time the astrologer looks at the current positions of the planets and compares them to a person’s birth chart and notes the events in the person’s life, research is being conducted.

However, this kind of research is prone to several problems. There is selection bias, which means that these charts are not representative of some larger population of charts. For example, suppose I do the astrology charts of friends and family, or suppose that I am a professional astrologer and most of the charts I do are of clients. Suppose also that I find that in these charts there are many cases of Uranus on the 7th house cusp and people having sudden breakups in relationships. Suppose also that most of my clients are women over the age of 40 years old and have no children and they are American. Can I generalize my findings to men, to women in other countries, to younger women, to women more affluent, less affluent, better educated, or more poorly educated than my clients? Can I even generalize to most women in the community in which I live, given that my clients tend to have certain belief systems, life styles, etc. If the correlation exists, why does it exist? If we have an idea of the mechanism by which astrology functions, it may help us to better guess what population we can generalize to.

Another big problem is that the belief that Uranus on the 7th house cusp is associated with breakups in close relationships is based on a very crude and subjective measurement. Are we noticing this correlation because we expect to see it and have we evoked this information from our clients and do we selectively perceive this event because we expect to see it? Do we bias our perceptions by interpreting any breakup as support for our belief, and do we rationalize breakups of other people as being less severe than they really are.

For example, imagine this conversation with a client who has Venus in Capricorn trine Saturn:

Client: I am having a consultation with you because I just went through a divorce with my husband and I am very upset, and I need to talk to someone.

Astrologer: Were you married for a long time?

Client: Yes, 20 years.

Astrologer: And you trusted him and trust is very important to you. You expect loyalty in a relationship and you are loyal. Why did you divorce?

Client: I found out that he was having an affair. And you are correct. It is the issue of loyalty and faithfulness that is at the heart of this. I always thought we would be married until one of us died.

Astrologer: Yes, the Venus in Capricorn trine Saturn in your chart shows how important a lasting, loyal relationship is. You must feel devastated.

Client: I do. You are so incredibly accurate and this is really helpful. It is really my sense of loyalty and dedication and wanting commitment that is so important to me.

Astrologer: Yes, exactly! You are using the words that describe this planetary configuration.

Now imagine this conversation with a client who has Uranus conjunct the 7th house cusp:

Client: I am having a consultation with you because I just went through a divorce with my husband and I am very upset, and I need to talk to someone.

Astrologer: Were you married for a long time?

Client: Yes, 20 years.

Astrologer: Wow, that is a long time. I mean it’s a long time given this sensitive astrological influence in your chart. There is this issue of freedom and a kind of rebelliousness and irresponsibility that we think of as something that teenagers are inclined to, but it can happen in your relationships.

Client: Well, my ex-husband was always a little less responsible than me. When we go out he tends to drink, not excessively, but enough to be dangerous when driving, and he always thinks he is fine and can drive, but I have to take the car keys from him and drive him home or he would be out driving drunk.

Astrologer: Like a self-willed and irresponsible teenager!

Client: Yes, (laughs). That’s Joe, a big lawyer who makes lots of money but never grew up emotionally.

Astrologer: But still, even though he never grew up emotionally, you were shocked by his affair?

Client: Yes, completely shocked.

Astrologer: These kinds of upsets in relationships are indicated in your chart, but I am going to give you some pointers on how to avoid this.

Client: Oh, thank you so much. You are describing my situation so precisely. I also had a huge breakup with a boyfriend when I was 17 years old and I don’t want to go through this again.

The astrologer is having a very sensitive and deep conversation with the client that is very helpful. In this hypothetical scenario, however, both conversations could happen with the same client. Amazingly, it is even possible that the astrology chart served as a guide to help the astrologer identify issues in the relationships even if the astrology chart was calculated for the wrong person! Why was the astrologer correct when surmising that the breakup was a complete shock, even if the client did not actually have Uranus conjunct the 7th cusp? It might be intuition, or an unconscious realization that the breakup must have been a shock for the client to schedule the appointment, or it might be some kind of divinatory process that can occur in consultations that is not completely understood, but could be classified as a kind of intuitive or psychic phenomenon. The astrological language is so profound and so descriptive of the human situation that it may somehow serve to act as a guide to help astrologers even though there is no direct measurable relationship between the astrological variables and the human behavior. Although it may seem a bit far-fetched that astrologers can provide helpful and accurate information to clients without the astrological variables actually accurately describing a person in an objective sense when used outside of the context of the astrological reading, this is an idea proposed by astrologers, not just by non-believers in astrology.

The casual or uncontrolled observations that astrologers and students of astrologers make typically can be classified as anecdotal evidence. There is no rigorous control of either how the data is obtained or how the data that is correlated are measured when anecdotal evidence is gathered. Anecdotal evidence, however, is not completely useless. Understanding must begin somewhere and anecdotal evidence is a starting point. Anecdotal evidence can also suggest relationships that are later studied in more detail.

We have only skimmed the surface of problems with anecdotal evidence. Developing good research designs is often a complex process because of the many issues that can arise. For this reason, academic research is usually conducted with feedback from content and methodology experts in a constant attempt to be aware of issues, problems, and weaknesses in the research design. No research design is perfect! Research is a pragmatic enterprise where one does the best that one can given the problem being tackled and the resources available.

When astrology is studied in a more controlled environment where the population being generalized to is clearly stated, the charts are sampled from this population in a reasonable way, and some form of clear measurement of a trait is used to correlate with an astrological factor, rarely has a statistically signfiicant correlation been found, and when it has been found, it has not been consistently replicated to the satisfaction of all experts who have studied the research. There have been studies such as the studies by Michel Gauquelin that show promising results, but no study has been consistently replicated, so whether astrology actually works in a kind of scientific and measurable way is still not known and there is no solid evidence that it does. There are, however, some promising studies that suggest that there might be a measurable effect, so a raging debate continues as to whether quantitative research in astrology is even a wortwhile endeavor.

QUALITATIVE RESEARCH

The failure of quantitative research in astrology to produce findings that support astrology has inclined some astrologers to feel that astrology may lie outside the boundaries of a science that can be studied with these kinds of research methods, but qualitative research studies are still useful and informative. Qualitative research is an important research method regardless of one's attitude towards quantitative research. Qualitiative research can help understand the meaning of astrological variables but does not prove that astrology works, so it may be more important to astrologers than to skeptics of astrology.

Qualitative research does not attempt to quantify findings and establish a measurable effect of some astrological variable in a quantitative manner. In qualitative research, numbers and statistics may be used but the conclusions are not based on the assumption that observations can be reduced to numerical values and compared based on these numerical values.

Table 1: A comparison of qualitative and quantitative designs according to one researcher:

Note: Table 1 is from http://www.fortunecity.com/greenfield/grizzly/432/rra2.htm and downloaded on September 24 2011.

Another comparison of qualitative and quantitative research is given in Table 2 according to another researcher. These tables give a good sense of the differences between qualitative and quantitative research. Note in Table 2 the difference in the goals of the investigation. Qualitative research focuses on understanding and description, while quantitative research attempts to predict or confirm a hypothesis.

Table 2: Characteristics of Qualitative and Quantitative Research

Merriam, S.B. (1988). Case study research in education: A qualitative approach. San Francisco: Jossey-Bass, p. 18.

Note: The above table and reference is from http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage21.htm
and was downloaded on September 26, 2011.

There are many kinds of qualitative research designs, and I will not attempt to discuss them here. This short introduction to research methods is designed to simply orient you to the basic directions for research without going into any detail.

EXPLORATORY RESEARCH

Some research designs are not hypothesis tests and do not attempt to answer a specific question. A hypothesis test attempts to find an answer to a question such as “Do people with stelliums in Cancer in the 4th house more likely to work in an area involving domestic products, homes, real estate, children, or stay at home to work than other people?” In exploratory research we explore the data to see what relationships exist in the data. Sometimes hypothesis tests can be done later based on the findings of the exploratory research. Exploratory research might investigate what astrological factors occur in the charts of scientists, musicians, painters, and other professions, and then based on these findings develop a hypothesis to be tested. Exploratory research is prone to find many relationships that are not replicable findings because the findings are just random occurrences in the data that is sampled, but if done properly, exploratory research can also help discover relationships that can be replicated in future studies.

Note that some astrological studies are exploratory but are then are incorrectly used as strong evidence in favor of astrlogy. This is especially a problem when no strong theoretical justification for the findings is given. For example, periodically a study is reported where certain sun signs are found to be more likely to be in particular professions or to differ in some other way. However, these studies typially are exploratory. In an exploratory study one does not state a clear hypothesis before the data is collected. Furthermore, one does ot restrict observations to those that fit one's expectations based on previous studies and/or a clear theoretical framework. In order to draw a conclusion that there is a corelation of the astrological variable and the behavior measuerd, one must also consider the possibility of confounding variables that may exist given that the study is not an experimental design. This problem is discussed below.

DESCRIPTIVE RESEARCH

Surveys are often conducted simply to be able to describe a situation. We may wish to know how many people in a given community or urban area hold a certain political position, preference, or interest. This can be useful to people who may wish to open businesses or plan future development for a community. Descriptive research is generally not a kind of research that astrologers pursue. What may seem like descriptive research is usually actually exploratory research to determine relationships of astrological variables and behavior in order to study these relationships in more detail in the future. Because descriptive research is a common kind of research study and is very important in other disciplines, it is worthy of mention in this introduction to research designs, and it is possible that some astrologers will engage in descriptive research to understand more about a particular population being studied or for some other reason.

QUASI-EXPERIMENTAL DESIGNS, EXPERIMENTAL DESIGNS, AND NATURAL EXPERIMENTS

In an experimental design treatment is controlled. The word “treatment” is used loosely to mean the predictor variables (Experimental and Quasi-Experimental Designs by Shadish, Cook, and Campbell, 2002, page 12). In astrology the treatment is the astrological factors such as planets in zodiac signs, house, or in aspect. The researcher does not control these astrological influences; we cannot manipulate them. The astrological factors are naturally occurring effect and is referred to as a natural experiment (Shadish, Cook and Campbell, 2002, page 12).

Quasi-experimental designs are an area of very interest and research in recent years. In a quasi-experimental design the treatment is done through self-selection rather than random assignment. A study of the relationship of smoking and lung cancer is typically done as a quasi-experiment. People self-select to smoke. We cannot randomly assign people to smoke and then see if they get cancer.

Although drawing causal inferences from quasi-experimental studies is difficult, it is not impossible. Donald Rubin’s potential outcomes framework (very often referred to as the counterfactual framework but Rubin’s preferred term is potential outcomes framework). The essence of the potential outcomes framework is whether the occurrence of event B depends on whether event A occurs. The potential outcomes framework eliminates the complex philosophical issues of mechanisms that are regarded as causal and simply replaces the idea of causality with a question of whether a particular behavior occurs dependent on some other earlier behavior.

Rubin's potential outcomes framework views causal inference as a kind of missing data problem. We may observe, for example, that smokers more often have lung cancer than non-smokers. The data that we do not have is whether the person would have lung cancer if he/she had not smoked. Data from people who do smoke does not provide this missing data because the non-smokers may differ in many ways from smokers. The non-smokers may not only avoid smoking, they may also avoid excessive alcohol, may exercise more, may differ in age, education, race, gender, urban/rural place of residence, region of the country, emotional well-being, etc. These variables are referred to as covariates and they are referred to as confounding variables if they influence outcome (lung cancer) as well as are correlated with treatment (smoking / non-smoking). A confounding variable may be the cause for boh treatment and outcome and thus invalidate any causal relationship between smoking and lung cancer. Not that we are using the word "causal" as it is used in the counterfactual framework, as indicating whether one behavior (lung cancer) occurs if another behavior (smoking) occurs. This use of the word causal removes issues of how this causal relationship exists; it does not need to occur through some kind of Newtonian model of material causality such as in the laws of motion and inertia.

Researchers have devised many elegant mathematical models and research methods to assist in drawing causal inferences in quasi-experimental designs. One breakthrough, for example, is the use of propensity scores to balance the treated and untreated groups. A propensity score is the measurement of treatment assignment given the covariates. Several matching mathematical algorithms have been developed to match the groups based on propensity scores and relatively new methods such as data mining methods such as boosted regression have been applied in simulation studies to determine if they provide more accurate estimates of propensity scores than with the moer traditional methods such as logistic regression. These are very hot areas of reseach now because improving the ability to draw causal inferences (as defined according to the potential outcomes framework) is vitally important especially in the social sciences.

I have gone into some detail in describing Rubin's potential outcomes framework because it is central to current research in quasi-experimental designs, and quasi-experimental designs, like the natural experiments that astrologers frequently use when conducting research, share a common problem of not having the advantages of a randomized experiment. In a randomized experiment, covariates that are potential confounders are also randomly distributed among the experimental and control groups so causal inference is much easier in a randomized experiment. However, like medical researchers studing the relationship of smoking to cancer, we cannot randomly assign people to have different astrological variables and see what the resulting behavior would be if everything else remained the same. Like medical researchers and social science researchers, however, we can do the best we can given these realities.

Quasi-experiments and natural experiments are similar in many respects. In both cases we can ask the counterfactual question of whether the outcome variable occurs dependent on the predictor variable. In the case of astrology, we ask the question of whether an outcome, such as talking a lot (which can be measured as the number of words spoken in a day) is related to the predictor of planets in Gemini. Although some astrologers might argue that astrology works through synchronicity and Gemini does not cause talking, that is not relevant to the potential outcomes framework. We simply ask the counterfactual question of whether the person would talk a lot of the planets were not in Gemini, and if we can answer this question, then a causal relationship exists even if that causal relationship is through some kind of synchronistic mechanism. In the potential outcomes framework causality is not restricted to causality in the sense of Newtonian laws of inertia as material causes. Because planetary positions are determined by mathematical formulae and are not influenced by human behavior, the astrological variable is the predictor variable (also referred to as the treatment in some of the literature) and the human behavior is the outcome variable.

CONDUCTING ASTROLOGICAL RESEARCH

Any of the major astrology programs can calculate charts and save them in a database to be analyzed for research. If you are unsure if your software can conduct the research you would like to do and you cannot figure it out from the Help or documentation provided with the program, then contact the software manufacturer for help. I am one of the authors of the Kepler and Sirius software programs and I use this software in all of my research. With over 40,000 charts included with Sirius, including the Gauquelin database as well as company data and data for famous people, and the extraordinary range of research features provided, one can implement an enormous number of research designs with this software. I continue to add more features as I and other users conduct research that requires additional features.

Because astrological research is usually a natural experiment rather than a randomized experimental design, how do we deal with the problem of possible confounding variables in quantitative research designs? The most common confounding variables are cyclic or social causes of a distribution of the astrological variables in the group being studied that is different fron the distribution that we had anticipated.

For example, suppose you do a study of some particular group of people and you find that many of them have Sun in Taurus and Aries rising, which is a combination that I happen to have in my chart. Actually, there are some seasonal variations in some countries and birth in the Spring is sometimes more common so Sun in Taurus may be an artifact of being born in the Spring season. How about Aries rising? Some studies have shown that with natural births, there is a greater tendency for birth to occur before sunrise. (Sorry, I have not looked up the references of these studies, but I am using them primarily for didactic purposes here). So Aries rising is common as well.

However, in hospitals where there are a large percent of births through operation, more babies are born during office hours of 9 AM to 5 PM! The cyclic motion of planets is complex and in some years particular aspects occur more often than others. Consider also that births often peak in communities 9 months after large numbers of military troops return home (perhaps this would not occur if half the military were women). Some studies also indicated a rise in births 9 months after snow storms or holidays provide couples the opportunity to spend more time being intimate. Even without these social causes of varying birth rates and thus varying disributions of astrological variables, the complexity of planetary motions can result in unexpected varying distributions of planetary configurations.

An interesting example of confounding effects that I encountered was in a strudy of introversion/extraversion. While analyzing the data I noticed an effect on introversion by outer planets. At first, I felt excited as it looked like I was on the track to finding a measurable effect from an astrological variable. Later it occurred to me that perhaps these outer planet placements were different for older people in the study and perhaps older people are more introverted. Thus, age would be the confounding variable. I analyzed the data and this proved to be true.

Furthermore, a search in academic journals of articles on the relationship of age and introversion produced papers that showed a positive correlation of age and introversion. My study confirmed these findings and perhaps this study will be of equal interest to academic rsearchers of the relationship of age and introversion as to those interested in astrological research. I reanalyzed the data including age as a variable using a structural equation model, and the results showed an extremely strong effect of age (p<.001) and also a significant astrological effect (p<.02). (http://astrosoftware.com/articles/GauqPaper2/GauquelinPaper2.htm)

I present this study as an example of the ways in which the concepts being presented in this paper are very real and important issues that you can encounter in actual rsearch. Also, this story gives a sense of the experiences one can have while conducting research in which one is sincerely trying to learn how astrological variables may work rather than superficially engaging in research either to support one's previous beliefs, whether they are pro-astrology or anti-astrology. The search for a measurable astrological effect can be a rocky road of joyful hopeful results and depressing findings, but I do learn a great deal about how astrology works in every study that I conduct, and these findings complement and augment the understanding gained through less controlled astrological study and research. Still, the goal of quantitative research of a replicable measuable astrological effect eludes us.

In astrological research, we generally try to make sure that the groups we are comparing cannot have varying distributions of the astrological variables as a consequence of confounding variables. One must consider varous possible cyclic and social effects on the distributions. There are also ways to simulate control groups. Propensity scores and other balancing scores that are used in quasi-experimental designs are not likely to be as helpful for astrological research as they are in many quasi-experimental designs. I will not discuss these issues in detail here because this subject is a large one and can be the subject of a separate paper.

In my own research in harmonic astrology, there is an advantage that higher harmonics tend to be fairly evenly distributed across relatively short periods of time so that there is less likelihood of the effects of confounding than there is for the occurrence of conjunctions and oppositions, for example. However, there are not guarantee prophylactics against confounding variables or statistical aberrations. Research takes time and most discoveries emerge gradually as replications and variations of studies are conducted rather than in a single eureka-like moment of discovery.

Some additional guidelines for conducting research are given at the beginning of a paper at http://astrosoftware.com/Proveast.htm. The 12 guidelines given in this paper can be very helpful for anyone who wishes to embark on any kind of astrological research study, whether it be qualitiative or quantitative.

CONCLUDING REMARKS

Astrologers embrace many different theories, including Vedic, Hellenistic, medieval, harmonics, midpoints, modern psychological, with almost endless variations within each of these traditions as well as other traditions. Astrologers also embrace a great range of theoretical frameworks regarding how astrology works, such as divination, motivation, “as above, so below”, energetic patterns, etc.

There is also a wide spectrum of research methods that can be used according to the research questions of interest to the astrological researcher, the theoretical framework, and the astrological tradition being evaluated. Research methodology has evolved rapidly in recent decades with rapid advancement in methods like multi-level modeling, structural equation modeling, data mining methods, etc. as well as tremendous improvements in statistical software to make analyses feasible. Astrologers can benefit greatly by employing these methods and this article is an attempt to introduce a few fundamental concepts in research methodology that are relevant to astrological research.