The challenges of planning and managing education in developing countries are more difficult than ever before. The aspirations for education are rising throughout the world faster than the resources to fund them, and the measures to fulfill them. At the same time, education and training are increasingly important to the futures of developing countries. International economic, technological and trading trends make the future of these countries partly dependent on their ability to expand and improve their education systems.

• Can indicate the range of innovations that have been tried for improving education structures, teacher preparation, and courses of study, curriculum, and teaching methodologies.
• May indicate problems and opportunities that have arisen when implementing a given innovation, strategies used in response, and the extent to which the actual implementation corresponded with the planned implementation.
• Often indicate how well various innovations have met their objectives.
• Can indicate unanticipated side effects (both positive and negative) that may occur from a given innovation.
• Can indicate the costs of a given innovation.
• Can indicate whether a given innovation was soon abandoned or long retained.

There are three basic ways of finding research and evaluation studies that may be of use in the planning of education. The first is to use search engines that will help you find appropriate studies located throughout most of the Web. The second is to use indexes to specific fields of research, such as education. The third is to check the web sites of organizations that do many studies applicable to education in developing countries.

Search engines are a good way to find research and evaluation reports conducted by international development organizations, NGOs, some government agencies, and university research centers. Search engines are not a good way to find the scholarly and professional journals, because most of those journals are not yet available on public web sites. The Northern Light search engine, however, does make the full-text of about 50 education journals available on the web. These include Academe, British Journal of Sociology of Education, Change, Comparative Education, Harvard Education Review, Review of Education Research, and Sociology. If a search in Northern Light finds an article in one of these journals, it will allow you to read an abstract for free, and then for a fee of U.S. $1.00 to $4.00 it will allow you to view and print the full text of the article. The fee is automatically charged to the user's individual or corporate credit card.

Each of the above search engines appears easy to use. It seems that all you have to do is type a word or phrase in the search box on the web site. Occasionally it works that easily, but without some knowledge about how to use each engine, you are likely to get no hits or thousands of hits unrelated to your interests. It takes an hour or two of study and practice to become moderately skilled in using any of these search engines. The needed instruction can be found on the Web:

Each search engine has some instructions for its use on its web site, but finding the appropriate link to click can be difficult because of the cluttered home pages. Look for something labeled [Search Tips], [Simple Searches], [Advanced Searches], [Help], and [FAQ]. Most search engines have a [simple] and an [advanced] search mode. If one does not work well, try the other. The advanced mode of some engines actually provides better guidance than the simple mode. Some search engines also have an e-mail address to which you can send questions about how to use the search engine when you have had difficulties. It usually is on one of the just indicated pages, towards the bottom.

The best book on the use of search engines is Alfred and Emily Glossbrenner's Search Engines for the World Wide Web. Since search engines are changing frequently, make sure to get the latest edition. The second edition came out in early 1999. It costs U.S. $17.99 and can be ordered through amazon.com and barnesandnoble.com.

If your search yields few or no hits:

If your search yields too many hits:

Eldis is a gateway to information sources on development that are available on the Web. It indexes and provides links to appropriate information throughout the WWW. It also has put the full-text of selected documents on its own web site. Eldis aims to provide policy makers, managers, NGOs and academics with easy access to the high quality information about development that is on the Web. It currently indexes and links to 5,000 selected documents available in full-text, to 2,700 home pages of organizations, and to descriptions of 120,000 books and articles.

From the home page, click on [Search] near the upper left corner. That will take you to the Eldis simple search page. It is not simple, so click on the [Help with Searching] for some brief instructions, then go back to the simple search page. If you are looking for organizations and cannot find appropriate ones that way, do the following from that page: click near the bottom on [more search options], then click near the bottom on the [subject guides], and then click on the specific subject of your interest. That will take you to a page, which lists organizations, statistical sources, bibliographic sources, research centers, and training courses related to the subject. On the left side there are links to various related tools and resources, including a link labeled [Education online broad search] which will list more than 8,000 documents related to education for development.

Several countries maintain indexes to the articles published in their education journals and to other documents on education that are published in the country. Presently we understand that only the United States makes its index available publicly on the Web. Its ERIC system has indexed more than 1 million journal articles, monographs, reports, conference papers, and books related to all topics in education. This probably includes half of the formal education research and evaluation conducted in the world. Most of the indexed studies have been conducted within the U.S., but some have been done abroad. In addition, some of those in the U.S. will be applicable in other countries. The ERIC system also has produced hundreds of Digests that provide excellent brief reviews of the literature on any given topic. These Digests are available in full-text on the Web. For other documents, ERIC only provides the document citations and abstracts on the Web. The abstracts, however, sometimes are quite informative. The full text of most indexed documents, except journals and books, can be ordered on microfiche for a small fee.

The ERIC search system looks easy to use, but without moderate competency you will miss important documents and get many unrelated documents. The ERIC Clearinghouse on Higher Education at The George Washington University has created a web-based tutorial that will teach moderate competency with the ERIC search system in less than two hours. For those who still are not able to find what they want, ERIC accepts e-mail inquiries that go to expert searchers who do the search and e-mail it back. When making such an inquiry, be specific about what you are trying to find and also indicate related topics that are NOT of interest.

For human assistance with ERIC

While existing research and evaluation can be very helpful when planning improvements in educational systems, it can also be misleading. Some reports are nothing more than the writer's opinion based on no empirical evidence. Some are the result of seriously flawed methodology. Some report only the positive outcomes and fail to mention the negative ones that were discovered. Sometimes there may be many studies of a given issue and they have yielded inconsistent results. In addition, an innovation that has worked well in one context may not work well in another context.

There are several strategies that can be used to avoid being misled by incompetent studies and biased reports. Below are suggestions on how to interpret individual studies and how to interpret a set of studies with inconsistent results.

The table of questions and warnings below will help you interpret individual research and evaluation studies and consider their application to your own situation.

While one study can help inform decision-making, several applicable studies will provide more reliable information. There are many existing reviews of the research and evaluation literature that synthesize findings of several studies on various topics. They are published in scholarly journals and also in the reports of some organizations. They can be located most efficiently with Eldis and ERIC, but web search engines may help, and some can be found by scouting the web sites of major education and development organizations.

Just as with individual studies, reviews of the literature can both guide and mislead you. Some are deliberately biased to support a predetermined agenda, whereas others are skilled efforts to discover whatever has been learned from the existing research and evaluation on a given topic. To check for possible biases, consider the relevant questions and warnings in the attached Table.

Inconsistency in the findings of several studies on a given topic may be due to several different conditions. The contexts in which the innovation is tried may not have been the same in all the studies. The intended beneficiaries may have differed in ways that influence the outcomes of the innovation. The innovations may have differed in some important feature, either in the original plans or in the actual implementation. Finally, the methods of the evaluation may have varied and affected the results. If distinct differences in these characteristics are noted, it is advisable to sort the studies into categories of those characteristics, and examine whether the findings vary from one category to another. That will provide an estimate of what the result is likely to be when you use the innovation under any given condition.

If your examination of the individual studies suggests that the studies were conducted under similar contexts, implemented similar innovations, and used similar evaluation procedures, then the best estimate of the effects is the average finding. The best average is that of the magnitude of the findings, assuming they have all been measured on the same scale. If the magnitudes are sometimes not reported, or reported on different scales, the next best simple strategy is to calculate the ratio of positive findings to negative findings. If it is well above 0.5, you can be reasonably confidant that the innovation will have positive effects when used as it was in the studies.

Meta-analysis uses stratification and averaging approaches with the assistance of multiple regression statistical methods. Meta-analysis computes and averages the effect-sizes and analyzes how the variation in the study outcomes is related to variations in the study contexts, interventions, and evaluations. Meta-analysis requires considerable time and the skills of a professional statistician.

Conclusion

Prior research and evaluation studies can be a powerful tool for the planning and management of educational innovation. Web developments over the last two years now make a substantial portion of the world's collection of education research and evaluation easily available to those in developing countries that are working to improve their education systems. When used skillfully and carefully, this resource can expand the perspectives and analysis available to inform decisions.