To request a blog written on a specific topic, please email James@StatisticsSolutions.com with your suggestion. Thank you!

Wednesday, February 3, 2010

Probability

Probability is a value that specifies whether or not an event is likely to happen. The value of probability generally lies between zero to one. If the probability of a happening of an event comes out to be zero, then that event would be considered successful. If the probability of a happening of an event comes out to be one, then that event would be considered a failure.
There are certain definitions of probability.

Statistics Solutions is the country's leader in probability and dissertation statistics. Contact Statistics Solutions today for a free 30-minute consultation.

A sample space S in probability is a non empty set whose elements are called outcomes. The events in the probability are nothing but the subsets of the sample space.

A probability space consists of the sample space and the probability function, which involves the mapping of the events to the real numbers in an interval of zero in such a way that the probability of the sample space is one. If A0 ,A1, ….. is the sequence of disjointed events, then the probability of the union of the sequence will be equal to the sum of the probability of all the disjointed events.

Conditional probability is that type of probability that denotes the probability of a particular event when it is given that another particular event has occurred, provided that the probability of the occurrence of the other particular event is not equal to zero.

There is a product rule in probability that states that the probability of the intersection of any two particular events is equal to the product between the probability of the second event and the conditional probability of the events.

The theorem of total probability states that if the sample space is the disjointed union of events, for example B1, B2, …. then for all events of A, then the probability of A will be equal to the sum of the probability of the intersection between the event A and the disjointed events Bi.

Suppose the two events, A and B, have a positive probability. In this case, the event A would be independent of B if and only if the conditional probability of A given the events B is equal to the probability of A. It is important to remember that this independence probability would be applicable only when the probability of the event B would not be equal to zero.

There is also an independence product rule in probability that states that the probability of the intersection of the two events is equal to the product of the probability of the event A and the probability of the event B. It is important to remember that in the theory of probability, the disjointed events are not the same as that of the independent events.

The theory of probability is the logic of science. According to James Clerk Maxwell (1850), the true logic involves the calculus of probability, which takes into consideration the magnitude of the probability that is supposed to be reasonable.

The theory of probability can be described with a popular example— the tossing of a coin with possible outcomes of “heads” or “tails.” Suppose “heads” is considered a success and “tails” is considered a failure. Thus, the probability of a success (“heads”) will be the probability of the value one, and the probability of failure (“tails”) is the value of zero. Similarly, rolling dice is another popular example based on the theory of probability.

Monday, February 1, 2010

F-test

An F-test is conducted by the researcher on the basis of the F statistic. The F statistic in the F-test is defined as the ratio between the two independent chi square variates that are divided by their respective degree of freedom. The F-test follows the Snedecor’s F- distribution.

Statistics Solutions is the country's leader in F-test and dissertation statistics. Contact Statistics Solutions today for a free 30-minute consultation.

The F-test contains some applications that are used in statistical theory. This document will detail the applications of the F-test.

The F-test is used by a researcher in order to carry out the test for the equality of the two population variances. If a researcher wants to test whether or not two independent samples have been drawn from a normal population with the same variability, then he generally employs the F-test.

The F-test is also used by the researcher to determine whether or not the two independent estimates of the population variances are homogeneous in nature.

An example depicting the above case in which the F-test is applied is, for example, if two sets of pumpkins are grown under two different experimental conditions. In this case, the researcher would select a random sample of size 9 and 11. The standard deviations of their weights are 0.6 and 0.8 respectively. After making an assumption that the distribution of their weights is normal, the researcher conducts an F-test to test the hypothesis on whether or not the true variances are equal.

The researcher uses the F-test to test the significance of an observed multiple correlation coefficient. The F-test is also used by the researcher to test the significance of an observed sample correlation ratio. The sample correlation ratio is defined as a measure of association as the statistical dispersion in the categories within the sample as a whole. Its significance is tested by the researcher using the F-test.

The researcher should note that there is some association between the t and F distributions of the F-test. According to this association, if a statistic t follows a student’s t distribution with ‘n’ degrees of freedom, then the square of this statistic will follow Snedecor’s F distribution, as in the F-test, with 1 and n degrees of freedom.

The F-test also has some other associations, like the association between the F-test and chi square distribution.

Due to such relationships, the F-test has many properties, like chi square. The F-values in the F-test are all non negative. The F-distribution in the F-test is always non-symmetrically distributed. The mean in F-distribution in the F-test is approximately one. There are two independent degrees of freedom in F distribution, one in the numerator and the other in the denominator. There are many different F distributions in the F-test, one for every pair of degree of freedom.

The F-test is a parametric test that helps the researcher draw out an inference about the data that is drawn from a particular population. The F-test is called a parametric test because of the presence of parameters in the F- test. These parameters in the F-test are the mean and variance. The mode of the F-test is the value that is most frequently in a data set and it is always less than unity. According to Karl Pearson’s coefficient of skewness, the F-test is highly positively skewed. The probability distribution of F increases steadily before reaching the peak, and then it starts decreasing in order to become tangential at infinity. Thus, we can say that the axis of F is asymptote to the right tail.

Thursday, December 17, 2009

Autocorrelation

Autocorrelation occurs due to the chance correlation of the error term of a particular household with some other household or firm. Autocorrelation is also named chance correlation. Autocorrelation is also applied in the case of time series analysis.

Statistics Solutions is the country's leader in autocorrelation and dissertation statistics. Contact Statistics Solutions today for a free 30-minute consultation.

The process of autocorrelation is defined as the correlation that exists between the members of the series of the observations that are planned with respect to time.

If two types of data are considered – a cross sectional type of data and a time series type of data—then for the cross sectional type of data, if the change in the income of a particular person affects the consumption expenditure of another household other than his, then autocorrelation is present in the data. Similarly, for the time series type of data, if an output is low in one quarter due to a labor strike, and if the data showing low output continues in the next quarter as well, then autocorrelation is supposed to be present in the data.

The process of autocorrelation is defined as the type of lag correlation for a given type of series with itself, which is lagged by several numbers of time units. On the other hand, serial autocorrelation is that type of autocorrelation that is defined as the process of lag correlation between two series in time series data.

There are certain patterns that are exhibited by autocorrelation.

Autocorrelation exhibits patterns among the residual errors. Autocorrelation also occurs in cases when the error shows a cyclical kind of pattern, etc.

The major reason why autocorrelation occurs is because of the inertia or sluggishness that is present in time series data.

The occurrence of the non stationary property in time series data also gives rise to the phenomenon of autocorrelation. Thus, to make the time series almost free of the problem of autocorrelation, the researcher should always make the data stationary.

The researcher should know that autocorrelation can be positive as well as negative. Economic time series generally exhibits positive autocorrelation as the series moves in an upward or downward pattern. If the series moves in a constant upward and downward movement, then autocorrelation is negative.

The major consequence of using ordinary least square (OLS) in the presence of autocorrelation is that it will simply make the estimator inefficient. As a result, the hypothesis testing procedures will give inaccurate results due to the presence of autocorrelation.

There is a popular test called the Durbin Watson test that detects the presence of autocorrelation. This test is conducted under the null hypothesis that there is no autocorrelation in the data. A test statistic called ‘d’ is computed, which is defined as the ratio between the sum of the square of the difference in the residuals with ith and (i-1) time and the square of the residual in ith time. If the upper critical value of the test comes out to be less than the value of ‘d,’ then there is no autocorrelation. If the lower critical value of the test is more than the value of ‘d,’ then there is autocorrelation.

If one detects autocorrelation in the data, then the first thing a researcher should do is that he should try to find whether or not the autocorrelation is pure. If it is pure autocorrelation, then one can transform it into the original model, which is free from pure autocorrelation.

Wednesday, December 16, 2009

Canonical Correlation

A canonical correlation is a correlation between two canonical or latent types of variables. In canonical correlation, one variable is an independent variable and the other variable is a dependent variable. It is important for the researcher to know that unlike regression analysis, in canonical correlation, the researcher can find a relationship between many dependent and independent variables. A statistic called the Wilk’s Lamda is used for testing the significance of the canonical correlation. The work of the canonical correlation is the same as in simple correlation. In both of these, the point is to provide the percentage of the variances in the dependent variable that are explained by the independent variable. So, canonical correlation is defined as the tool that measures the degree of the relationship between the two variates.

Statistics Solutions is the country's leader in canonical correlation and dissertation statistics. Contact Statistics Solutions today for a free 30-minute consultation.

The process of canonical correlation is considered the member of the multiple general linear hypotheses, and therefore the assumptions of multiple regressions are also assumed in canonical correlation as well.

There are concepts and terms associated with canonical correlation. These concepts and terms will help a researcher better understand canonical correlation. They are as follows:

1. Canonical variable or variate: A canonical variable in canonical correlation is defined as the linear combination of the set of original variables. These variables in canonical correlation are a form of latent variables.

2. Eigen values: The value of the Eigen values in canonical correlation are considered as approximately being equal to the square of the value of the canonical correlation. The Eigen values basically reflect the proportion of the variance in the canonical variate, which is explained by the canonical correlation that relates to the two sets of variables.

3. Canonical Weight: The other name for canonical weight is the canonical coefficient. The canonical weight in canonical correlation must first be standardized. It is then used to assess the relative importance of the contribution of the individual’s variable.

4. Canonical communality coefficient: This coefficient in canonical correlation is defined as the sum of the squared structure coefficients for the given type of variable.

5. Redundancy coefficient, d: This coefficient in canonical correlation basically measures the percent of the variance of the original variables of one set that is predicted from the other set through canonical variables.

6. Likelihood ratio test: This significance test in canonical correlation is used to carry out the significance test of all the sources of the linear relationship between the two canonical variables.

There are certain assumptions that are made by the researcher for conducting canonical correlation. They are as follows:

1. It is assumed that the interval type of data is used to carry out canonical correlation.

2. It is assumed in canonical correlation that the relationships should be linear in nature.

3. It is assumed that there should be low multicollinearity in the data while performing canonical correlation. If the two sets of data are highly inter-correlated, then the coefficient of the canonical correlation is unstable.

4. There should be unrestricted variance in canonical correlation. If the variance is not unrestricted, then this might make the canonical correlation look unstable.

Most researchers think that canonical correlation is computed in SPSS. However, canonical correlation is obtained while computing MANOVA in SPSS. In MANOVA, canonical correlation is used in data sets where one refers to the one set of variables as the dependent and the other as the covariates.

Tuesday, December 15, 2009

Chi Square test

Parametric tests are those kinds of tests that involve the use of parameters, and the chi square test is a parametric tests.

Statistics Solutions is the country's leader in chi square tests and dissertation statistics. Contact Statistics Solutions today for a free 30-minute consultation.

There are varieties of chi square tests that are used by the researcher. They are cross tabulation, chi square test for the goodness of fit, likelihood ratio test, chi square test, etc.

The task of the chi square test is to test the statistical significance of the observed relationship with respect to the expected relationship. The chi square statistic is used by the researcher for determining whether or not a relationship exists.

In the chi square test, the null hypothesis is assumed as there not being an association between the two variables that are observed in the study. The chi square test is calculated by evaluating the cell frequencies that involve the expected frequencies in those types of cases when there is no association between the variables. The comparison between the expected type of frequency and the actual observed frequency is then made in the chi square test. The computation of the expected frequency in the chi square test is calculated as the product of the total number of observations in the row and the column, which is divided by the total size of the sample.

The calculation of the chi square type of statistic in the chi square test is done by computing the sum of the square of the deviation between the observed and the expected frequency, which is divided by the expected frequency.

The researcher should know that the greater the difference between the observed and expected cell frequency, the larger the value of the chi square statistic in the chi square test.

In order to determine if the association between the two variables exists, the probability of obtaining a value of chi square should be larger than the one obtained from the chi square test of cross tabulation.

There is one more popular test called the chi square test for goodness of fit.

This type of chi square test called the chi square test for goodness of fit helps the researcher to understand whether or not the sample drawn from a certain population has a specific distribution and whether or not it actually belongs to that specified distribution. This type of chi square test can be applicable to only discrete types of distribution, like Poisson, binomial, etc. This type of chi square test is an alternative test for the non parametric test called the Kolmogorov Smrinov goodness of fit test.

The null hypothesis assumed by the researcher in this type of chi square test is that the data drawn from the population follows the specified distribution. The chi square statistic in this chi square test is defined in a similar manner to the definition in the above type of test. One of the important points to be noted by the researcher is that the expected number of frequencies in this type of chi square test should be at least five. This means that the chi square test will not be valid for those whose expected cell frequency is less than five.

There are certain assumptions in the chi square test.

The random sampling of data is assumed in the chi square test.

In the chi square test, a sample with a sufficiently large size is assumed. If the chi square test is conducted on a sample with a smaller size, then the chi square test will yield inaccurate inferences. The researcher, by using the chi square test on small samples, might end up committing a Type II error.

In the chi square test, the observations are always assumed to be independent of each other.

In the chi square test, the observations must have the same fundamental distribution.

Tuesday, November 24, 2009

Analysis Of Variance (ANOVA)

The question that one usually asks about Analysis of Variance (ANOVA) is about the definition of Analysis of Variance (ANOVA). Analysis of Variance (ANOVA) is defined as the process of examining the differences among the means for two or more populations. The next question that arises in the researcher’s mind is what null hypothesis is assumed in the Analysis of Variance (ANOVA). The answer is that the null hypothesis is assumed as the following: “there exists no significant difference in the means of all the populations that are being examined in the Analysis of Variance (ANOVA).”

Statistics Solutions is the country's leader in Analysis of Variance (ANOVA) and dissertation statistics. Contact Statistics Solutions today for a free 30-minute consultation.

The type of variable on which the Analysis of Variance (ANOVA) is applicable is also an important issue. Analysis of Variance (ANOVA) is applicable in cases where the interval or a ratio type of the dependent variable and one or more categorical type of independent variable is involved. The researchers should also note that the categorical type of variables is considered as the factors in the Analysis of Variance (ANOVA). The combination of the factor levels or the categories in the Analysis of Variance (ANOVA) is generally termed as the treatments.

The Analysis of Variance (ANOVA) technique, which consists of only one categorical type of independent variable, or in other words a single factor, is called one way Analysis of Variance (ANOVA). On the other hand, if the Analysis of Variance (ANOVA) technique consists of two or more than two factors or categorical types of variables or independent variables, then it is called n way Analysis of Variance (ANOVA). In this, the term ‘n’ refers to the number of factors in the Analysis of Variance (ANOVA).

Like regression analysis, the process of Analysis of Variance (ANOVA) also requires the calculation of multiple sums of squares for evaluating the test statistic that is used for testing the null and alternative hypothesis. There is also one difference in Analysis of Variance (ANOVA) and regression analysis, and that is that Analysis of Variance (ANOVA) uses separate and combined means and variances for the samples while evaluating the values that are applicable for the sum of the squares.

Often, the researcher questions what type of test statistic is used for testing the significant difference. The test statistic is nothing but the F statistic that is used in Analysis of Variance (ANOVA). The F test statistic is defined as the ratio between the sample variances. The task of the F test in Analysis of Variance (ANOVA) is to carry out the test of significance of the variability of the components existing in the study.

The most important question is about the assumptions in Analysis of Variance (ANOVA).

The first assumption of Analysis of Variance (ANOVA) is that each sample has been drawn from the population by the process of random sampling.

The second assumption of Analysis of Variance (ANOVA) is that the population from which each sample is randomly drawn should follow normal distribution. In other words, this means that in Analysis of Variance (ANOVA), it is assumed that the error term is normally distributed having its mean as zero and the variance as σ2e.

The third assumption of Analysis of Variance (ANOVA) is that there is homogeneity within the variances of the populations from which the sample has been drawn.

The fourth assumption of Analysis of Variance (ANOVA) is that the population that consists of the random effects (A) is normally distributed having ‘0’ as the mean and σ2a as the variance.

Thursday, November 19, 2009

Validity

Validity refers to the state in which the researcher or the investigator can get assurance that the inferences drawn from the data are error free or accurate. If there is validity in the sample, then there is validity in the population from where that sample has been drawn.

Statistics Solutions is the country's leader in validity and dissertation statistics. Contact Statistics Solutions today for a free 30-minute consultation.

There are basically four major types of Validity. These types are Internal Validity, External Validity, Statistically Conclusive Validity and Construct Validity.

Internal Validity refers to that type of validity where there is a causal relationship between the variables. Internal Validity signifies the causal relationship between the dependent and the independent type of variable. Internal Validity refers to those factors that are the reason for affecting the dependent variable. This type of validity is used in the case of the design of experiments where the treatments are randomly assigned.

External Validity refers to that type of validity where there is a causal relationship between the cause and the effect. The cause and effect in this type of validity are those that are generalized or transferred either to different people or different treatment variables and the measurement variable.

Statistically conclusive validity refers to that type of validity in which the researcher is interested about the inference on the degree of association between the two variables. For instance, in the study of the association between the two variables, the researcher reaches statistically conclusive Validity only if he has performed statistical significance tests upon the hypotheses predicted by him. This type of validity is violated when the researcher reaches two types of errors, namely type I error and type II error.

Type I error causes violation of this type of validity because in this type of error, the researcher rejects the hypothesis which was indeed true.

Type II error causes violation of this type of validity because in this type of error, the researcher accepts the hypothesis which was indeed false.

Construct Validity refers to that type of validity in which the construct of the test is involved in predicting the relationship for the dependent type of variable. For example, construct validity can be drawn with the help of Cronbach’s alpha. In Cronbach’s alpha, it is assumed that if its value is 0.80, then it is considered good for confirmation, and if its value is 0.70, then it is adequate. So, if the construct satisfies such conditions, then the validity holds. Otherwise, it does not.

Convergent/divergent validation and factor analysis is also used to test this type of validity.
There is a strong relationship between validity and reliability. A test is said to be unreliable if it does not hold the conditions of validity. Reliability is a necessary property of the test, but is not the sufficient condition for validity.

Thus, validity plays the significant role in making an accurate inference about the data.
There are certain things that act as a threat to validity. These are as follows:

If the researcher collects insufficient data to attain validity in the inference, this is not feasible because insufficient data will not represent the population as a whole.

If the researcher measures the sample of the population with too few measurement variables, then he also cannot achieve validity of that sample.

If the researcher selects the wrong type of sample, then he too cannot achieve validity in the inference about the population.

If the researcher selects an inaccurate measurement method during analysis, then the researcher would not be able to achieve validity.