Ways to Work, Inc.

REQUEST FOR PROPOSAL

PROGRAM IMPACT EVALUATION SERVICES

Contact Person: Emily Putnam
Evaluation and Research Services
Alliance for Children and Families
11700 W. Lake Park Drive
Milwaukee, WI 53224
Telephone: 414-359-6603
Email: [email protected]

Issue Date: January 7, 2011

Letter of Intent Date: January 20, 2011

Proposal Due Date: February 7, 2011

Contract Type: Fixed cost

Period of Performance: 7- months

Project: Program evaluation and credit score impact study for the Ways to Work program

THIS IS A FULL AND OPEN COMPETITION

BACKGROUND

Ways to Work Inc., (WtW; www.waystowork.org) is a unique Community Development Financial Institution (CDFI) with a national program model combining financial literacy training, low interest character based loans, and high touch case management support. This unique program provides a hand-up to working families striving to establish increased economic self-sufficiency and to establish or rebuild their credit. The WtW program model transforms an immediate need for reliable transportation into a teachable moment for delivering curriculum based financial education. Financial education is reinforced through individualized budget coaching sessions, a low-interest, character based car loan serves as the incentive for program participation, and case management supports success in repaying the loan and building an improved credit standing. The Ways to Work national office provides the program model, loan capital, central lending system and technical and operational support. Local nonprofit agencies operate the program in their local communities and assume the risk of loan default costs.

The WtW national office and its government, philanthropic and banking partners seek deeper evaluation of program effectiveness and impact. Through this request for proposals, WtW seeks qualified evaluators to carry out a program evaluation of the WtW model as it has been implemented at current active sites (see List of Sites Appendix 1) and for a third, bi-annual Credit Score Impact study. Current numbers from the WtW national office identify 40+ sites and participation of over 10,000 clients over the 10 year period. WtW serves around 1,300 clients annually.

These evaluation results will serve the critical purpose of demonstrating mission achievement and impact to existing and future funders making investments in the WtW program and to continue to inform sites about best practices. WtW anticipates awarding a seven month contract as a result of this RFP process in order to provide impact information and updated evaluation results to potential funders for the Fall 2011 funding cycle. The total contract amount is expected to fall within the range of $125,000 to $175,000.

Through this proposal, WtW seeks a qualified evaluation provider who:

• can establish a positive and facilitative partnership with the national WtW office and program sites.
• demonstrates experience in developing, selecting and applying principals of program evaluation while maintaining respect for program participant experience.
• has experience in integrating quantitative and qualitative evaluation methods.
• has the capacity to work with multiple data collection methods and sources within the specified time frame.
• can produce reports and documents that capture relevant data, creates a compelling case that is accessible by the funding community and on-going program development, and can advise on and produce credible products. Proposals from evaluators with a strong interest in economic security programs and a desire to present and publish these findings are strongly encouraged.
SCOPE OF WORK

Evaluation is a significant component of the success of WtW. Prior program evaluations are regularly used to establish outcomes and refine the program model. This request for proposals seeks evaluation services that integrate and build off of prior program evaluations and credit studies with the intent of establishing program impact. (See the WtW website for copies of prior evaluations and credit impact studies, www.waystowork.org.) The evaluation must establish whether there exists a compelling link between program activities and outcomes for borrowers generally, and between variation in the implementation of the model, level of borrower outcomes achieved, and program-level outcomes (e.g., loan repayment rate).

A competitive proposal will introduce a plan for working with the WtW national office to integrate existing data and the collection of new data to:

1. Update and expand the 2006 Ways to Work program evaluation. The external evaluator should attempt to tie variations in program sites to program and borrower outcomes.

2. Demonstrate the impact of WtW with a focus on "return on investment".

3. Update and enhance the 2009 Credit Impact Study and connect these trends with best practices.

WtW will provide the contracted evaluator with all available materials including existing program data from an extensive administrative loan database (e.g. loan recipient credit scores, credit histories for WtW selected loan originations), and site records (e.g. organizational profiles, community specific economic and demographic characteristics for 2009 and 2010). However, additional data will need to be collected to complete this project. WTW will arrange local sites' compliance with the evaluator and help to facilitate securing of borrower consent but the primary responsibility for these tasks will rest with the contracted evaluator.

For the Credit Score Impact Study, the contracted evaluator will be provided with de-identified data from WtW for multiple cases of borrowers from 2001 to 2011. WtW is interested in examining the data across cohorts. Proposals should account for the credit score impact study components in the proposed project schedule. In addition, proposals should include methods for linking the credit impact study trends to other project components. Ways to Work seeks the advice of the contracted evaluator on how to best present findings of the credit impact study.

Focus of Project

The proposed design should address the following questions and examine identified outcomes. The WtW national office is also interested in having the contracted evaluators propose and explore additional questions and alternative approaches.

1. What individual and familial outcomes do borrowers experience from participation in the Ways to Work loan program? Outcomes include but are not limited to employment, financial assets, access to child care, access to transportation, financial relationships with credit institutions, independence from or return to public assistance/support, family characteristics and dynamics, amount and use of free time, child health and wellbeing indicators, such as child obesity, school attendance, etc.

2. Based on past data collected and new sources of data identified by the evaluator, what are the social and financial benefits of WtW? A strong argument should be made about the program's ROI should be framed and presented in such a way that the report indicates:

"for every program dollar spent, sponsors may expect between $## and $## return over X amount of time".

3. How should impact be viewed in the context of the current economic environment? How has the national economic downturn in early 2008 to 2010 affected borrowers, loan program sites, and the communities they serve? In the current economic environment, why have some WtW sites closed and why have new sites opened?

4. To what extent and in what ways does participation in WtW affect borrower use of safety net services, especially governmental and community-based resources related to food insecurity, housing stability, foreclosure prevention, and mortgage including food stamp usage, TANF usage, usage of food pantries, use of subsidized child care, use of transportation subsidies, and use of housing assistance? Additionally, what is the nature and extent of impact of borrower participation in WtW on the cost of support, worker productivity, tardiness and days missed, and general quality of life? What effect does borrower participation in WtW affect local businesses' worker retention rates and/or employee turnover? How does this translate into local businesses employee-related cost savings? What is the extent of impact of borrower participation in WtW on federal and state income taxes paid ?

5. What are the WtW site level outcomes? What effect does the duration of a WtW site have on program outcomes? What is the nature and extent of variation among sites with respect to program implementation vis-à-vis the basic program model? To what extent do site differences, program variations, or community characteristics relate to outcomes for borrowers and for the agencies operating WtW loan sites? (e.g., How does the timing and duration of financial education or the choice of curriculum affect borrower outcomes? What is the nature and extent of affect of cross-referrals on program and borrower outcomes?)How are outcomes related or linked to best practices? Outcomes should include but are not limited to loan default rates, relationships with community financial institutions, car dealers, car repair shops and professionals, cross-program referrals coordination of services, etc.

The Credit Score Impact Study at a minimum should:

6. Provide descriptive information about the participants in each cohort, and comparisons across cohorts and between participants with and without credit scores.

7. Examine the impact on credit scores on borrowers as a result of participation in the WtW program including the trends evident for those with at least two years and five years of scores or more.

8. Examine the prevalence of negative and positive changes in scores for each cohort and what factors, if any, are associated with negative and positive changes in scores over time. WtW is especially interested in those participants who "drop off the radar".

9. Examine the loan status for each cohort and what factors, if any, are associated with positive loan status as well as what factors, if any, are associated with default loan status. This analysis should also look at the change in credit scores of WtW participants who did not default against those who did.

10. Explore differences site-level characteristics associated with positive and negative change scores. These characteristics are to be determined during the contract period with the WtW national office and not to exceed 3-5 readily verifiable characteristics (e.g. strong banking relationships) provided this data can be acquired during the outcome evaluation phase.

Project Components

Desired proposals will include a well established plan that allows for the elements below. These elements are listed in order of importance to the WtW national office. They do not need to be organized or specified in this order but the proposal and project plan should reflect upon these elements accordingly. It is fully expected that the proposal draw on methods and design from prior evaluations and credit impact studies. The proposal should identify which design elements from prior evaluations will be used and which design elements are innovative to this proposal.

A. Plan for a partnership between the evaluator and WtW national office and program sites for the duration of the project. The proposal should include a strong plan for communication with the national office, how information about WtW priorities and needs will be gathered and then structured, how project team members will be integrated into the evaluation process as needed as well as a philosophy or approach for helping non-evaluation staff understand evaluation components and methods.

B. Participant outcomes using a representative sample of borrowers with loan originations between 2001 and 2010. Outcomes should capture both short- and long-term outcomes and lived experiences.

C. Return on investment to capture the social and economic impact of the WtW. The proposal should justify the measures selected and how the findings will be situated in the current economic climate and across sites.

D. Sampling procedures drawing on the full national pool of WtW participants across all active sites and all program years. A fully justified explanation of sampling procedures should capture variation in program operations, client enrollment, and duration as well as other factors that might contribute to impact and borrower outcomes.

E. Loan Program Site outputs and outcomes including an examination of reasons for and handling of loan defaults, local credit reporting practices, policies and practices of the local loan committee, source of loan funds, and relationship with banking partner. Evaluation should explore site director and/or loan coordinator philosophy, program wisdom, and rationale guiding loan decisions and lending practices.

F. Profile of Operations/Practices and Lending Outcomes that documents program features, site outcomes and operating agency impact. The evaluation should also contextualize the evaluation within the current community demographics, geographic location and locale, recent economic exigencies, and current financial indicators of community well-being, such as unemployment and foreclosure rates, etc.

G. Credit Impact Study update and plan for integration into the larger evaluation project.

H. Best practices, longevity and/or lending outcomes using a selective sample of sites.

I. General Program Overview that captures the history, core values, lending and program strategies, and current program model. Time and attention to program history should be limited to what is necessary to frame the impact studies.

DELIVERABLES

Project deliverables include:

• A bi-monthly progress report
• A comprehensive program evaluation report
• A comprehensive Credit Impact Study report (if appropriate)
• A separate executive summary for each report suitable for dissemination to boards of directors, funders, program sites, and national and local media outlets
• Electronic copies of all reports, protocols, instruments, interview schedules, on-line questionnaires, coding guides, data dictionaries, databases or electronic information systems developed for this evaluation, analysis outputs.

PROJECT TIMING EXPECTATIONS

• Proposal due: February 7, 2011
• Selection of Evaluation Provider: February 24, 2011
• Notification of selection and contract approval: February 28, 2011
• Date to begin work: March 1, 2011
• Progress update and review of preliminary findings: July 15, 2011
• Full report outline: July 30, 2011
• Final edit copy: August 15, 2011
• End of feedback period: August 26, 2011
• Dates for final deliverables: September 8, 2011

PROPOSAL REQUIREMENTS

DECLARATION OF INTENT TO SUBMIT A PROPOSAL: Interested parties must indicate their intention to submit by completing the online form found at: http://www.surveymonkey.com/s/HMXFJTR by January 20, 2011.

Interested evaluators must include the following information in their proposal:

1. Include the name of your firm, address of office responding, telephone number, fax number, email, contact person with title, and historic information on establishment including parent company, if applicable. Indicate the number of professional consultants on staff who have worked on a project of this scope.

2. State your experience with projects of this size and scope. Provide at least one and no more than three examples of relevant, completed consulting work. For each example project, specify the date and scope of each engagement, a description of services provided, a list of key personnel, telephone numbers, and the contact person to use as a reference.

3. Specify the key people you would use on this project if awarded the contract and include a résumé for each. Vitae and résumés should be in a standard format, providing project title, credentials or professional registrations, awards, previous employment, and a brief summary of experience that relates to this project.

4. Include a narrative describing a multi-level, mixed methods evaluation detailing the evaluation design elements, the project activities, and strategies to achieve the scope of the work and furnish each deliverable.

5. Include a confidentiality, privacy protections and data security plan to maintain respondent confidentiality. The plan should specify security and storage of data, backup procedures and data integrity. The plan must also state how all personally identifying information about borrowers will be protected and when it will be destroyed.

6. Provide a detailed budget and budget justification for the project. Include all costs that might be associated with all components of the research including items like borrower incentives, any travel, etc. Please express your cost quote for the work described including the provision of one final copy of all protocols, data collection instruments, reports, etc.

7. Provide details of hourly rates for appropriate levels of project staff. For salaried personnel indicate percentage of time to be expended on proposed evaluation.

8. Estimate the number of work hours by level of personnel (e.g., Principal Evaluator, Project Manager, Senior Consultant, Administrative Assistant, Research Assistants, etc.) required to complete the various aspects of the scope of work. Exercise reasonable care since your estimation will be a factor in evaluation of your proposal.

9. Specify any terms or conditions desired in a contract. If you have a standard services contract, please include a copy for our review

PROPOSAL SUBMISSION INFORMATION

PROPOSAL SUBMISSION: Proposals MUST be submitted electronically to [email protected] as well as two complete hard copies, including all attachments. Paper copies of the proposal must be signed by an officer authorized to bind the evaluator or company. Paper versions must be postmarked by 5:00 p.m. Pacific Time on February 7, 2011.

Hard copies of proposals may be mailed or hand delivered. For materials submitted in hard copy by postal mail or mail courier services, the responsibility for timely delivery is entirely the responsibility of the evaluator submitting the proposal.

Director of Evaluation and Research Services
Alliance for Children and Families
11700 West Lake Park Drive
Milwaukee, WI 53224

PROPOSAL FORMAT REQUIREMENTS: Electronic documents and paper copies of proposals shall be no more than 10 pages, single-spaced (excluding appendices such as logic models, management plans and timelines, project staff vita and resumes, table of previous evaluation clients or projects, and samples of past work). Faxed proposals will not be accepted.

PROPOSAL WITHDRAWAL: No proposal may be withdrawn after having been received by Ways to Work.

PROPOSAL REJECTION: If adequate submissions are not received, WtW reserves the right to reject any and all proposals, and, at its discretion, re-issue the RFP.

EVALUATOR RESPONSIBILITY RE: COSTS INCURRED FOR PROPOSAL SUBMISSION: The submitting evaluator agrees that this Request for Proposals does not obligate WtW to pay any costs incurred by the submitting evaluator during preparation or submission of a proposal.

CONTRACT CANCELLATION: Ways to Work reserves the right to cancel the contract if services provided are not meeting WtW needs. Notification will be provided in writing. Payment will be made on a prorated basis for those services performed and deemed acceptable prior to contract nullification.

DATA RIGHTS: Ways to Work will retain all rights to the plan, all data collected and all subsequent usage, including but not limited to discussion, presentation, and publication. WtW is open to requests from the contracted evaluator to present and publish the data.
 
PROPOSAL REVIEW CRITERIA

The proposal selection process is single-stage. Contractor selection will be made by a team comprised of staff in the WtW national office, the Alliance for Children and Families ("Alliance") finance office and the Alliance evaluation and research services department. A final decision about the contract award remains at the sole discretion of the WtW President.

A. Evaluation of proposals will be based on, but not limited to, the following:

Selection Criteria
Soundness of Evaluation Plan

• Clarity of research plan and methods
• Likelihood that proposed methods will achieve the scope of the work outlined in the RFP
• Likelihood that the project can be completed within the proposed time frame for the work
Cost
• Reasonableness of the proposed costs for the proposed evaluation activities
Experience/Qualifications
• Prior experience in conducting program evaluation
• Documentation of research experience related to the research activities and methods outlined in the proposal
• Documentation of experience and capacity to undertake a large-scale, multi-site evaluation
• Relevant education, training, and experience of key staff
Capability
• Infrastructure to undertake the proposed research activities (e.g., research facility, data processing resources, interview technology, etc.)
Work Products
• Quality of previous work products
References

B. WtW reserves the option to request additional information to complete proposal review.

C. WtW reserves the right to contract separately for components of the evaluation.

D. All proposers will be notified, upon final determination, of the firm selected to perform the requested work.