Most people making their first attempt at program evaluation will ask the same general kinds of questions at the outset of their endeavor: Why evaluate? How much will it cost? Can I do it myself, or will consultants be required? How scientifically rigorous must it be? Should I have a control group? How big should the sample be? etc. All of these questions can be answered, but only after a set of important assumptions about the program being evaluated has been carefully specified. In most cases, all of these questions can be answered quite adequately by the person(s) asking them. Very little technical expertise is required. In this article we present an approach to the formulation of a general evaluation strategy, including a method for answering some of the questions that need to be answered first.

Thedesign ofacomprehensive evalua- tionofsubsidized rural primary careprograms ona large national scale isdescribed. Itsmajor purpose is toderive datawhoseanalysis will answer major policy questions about thefactors influencing theoutcome of themajor types ofsuchprograms indifferent commu- nities. This first paper also delineates atypology which Inthe1960s, thelongrecognized problem ofmaking goodquality personal health services available topeople in rural areaswasgiven greatly increased attention inthe United States. Formanyyears prior tothis, various mea- sures tocorrect this persistent deficiency hadbeentried by rural people themselves, byagencies suchasgovernments at local, state, andnational levels, andbylocal andnational foundations. Fewofthese efforts seemed toprovide satisfac- tory solutions. During the1970s, growing concern about this problem ledtogreatly expanded andstrengthened efforts andsomenewapproaches whichincluded therequirement thatsubsidized primary careprojects adopt specific pro- grammatic goals andparticular methods oforganization, staffing, andoperation. Towardtheendofthe1970s, national economic con- straints produced increasing competition forthepublic dol- lar, thus heightening aninterest incomprehensive evaluation studies thatmight leadtopolicy changes forfuture sup- port. 2A review oftheliterature indicates that, although a goodmanyevaluation studies havebeendoneinthis field, theyhavebeenlimited bytherepresentativeness ofthe practices studied andtheexamination ofalimited setof relevant characteristics. Forexample, several studies have examined factors associated withfinancial self-sufficiency without simultaneously examining theimpact oftheprogram on thecommunity being served. A partial, butrepre- sentative, list isincluded inthereferences.3-19

The use of mid-level practitioners (nurse practitioners, physician's assistants, and so on) is advocated to improve the access of rural people to health care. A remote rural area of southern Appalachia is served by a network of three clinics staffed by mid-level practitioners (MLPs) and an M.D. During the first three years of operation 76% of the geographically defined target population of 5,500 received services. MLPs provided care in half of the 40,252 medical encounters and 89% of their contacts were managed without consultation with or referral to the M.D. They managed 36% of first-year visits, 51% of second-year visits, and 54% of third-year visits. Concurrent with this shift in patient care responsibility from M.D. to MLP, differences in the types of conditions managed by M.D. and MLP decreased with time. Population surveys indicate that consumer satisfaction with MLP services is high and that health care from this system is perceived as being more accessible than care from alternative sources. In this setting the role of the MLP evolved in the direction of, but was not limited to, that of an M.D. substitute. Experience with this delivery system suggests that, as members of a health care team, MLPs can manage a majority of problems encountered in rural primary care with a high level of consumer satisfaction and improved access.