I’m frequently asked about when and how ROI Solutions started to use machine learning to build MiLo Intelligence predictive data models, particularly after my recent presentation at the Bridge to Integrated Marketing and Fundraising Conference.
We started MiLo Intelligence back in 2018 as a kind of skunkworks project with a few goals:
- Leverage the new but sometimes ‘over-hyped’ machine learning tools and technologies being introduced. We wanted to find practical applications that could help nonprofits move the needle.
- Take full advantage of the rich data we store for our clients in Revolution CRM to make it actionable.
- Let our clients ease into modeling by incorporating model scores into their existing processes.
- Help our clients make their fundraising programs more efficient and cost-effective by targeting the right constituents at the right time, especially as budgets shrink and mailing costs continue to climb.
From the start, we committed to building custom models for each client to ensure that the nuances of their programs were reflected in the modeling. We also wanted to leverage the shared knowledge about each organization’s data fully. Every organization is different, and what data represents a strong or a loose connection to the organization varies widely – for some clients, flags on an account represent constituents raising their hands for more information; in others, it revolves around activism and a willingness to promote the organization’s mission; and finally, for some clients, it’s measured in volunteer hours given to support an event or fundraising walk.
For all model builds, the process is essentially the same:
- Collaborate to define the modeling objective and use case. There is limited value in producing model scores that can’t be deployed to impact fundraising directly, so we always start the modeling process with a concrete use case.
- Select and evaluate modeling data. This is another collaborative process to identify data essential to the constituent-organization relationship. Once we have determined the data points (“features” in modeling terminology), we work together to identify the best historical data to use for modeling, whether it’s a set of fundraising campaigns or pledges representative of the current sustainer file. We always pull point-in-time data to look at each record as it existed when modeling.
- Build Models: Many different algorithms are associated with machine learning modeling, and as part of our model build process, we create many different models based on these algorithms and compare the results to determine which one is most predictive. Based on the review of initial results, we will often add/modify features and recreate models until we are satisfied with the results.
- Back-test against previous campaigns to prove efficacy. In addition to testing against holdout data in the modeling process (building the models on 80% of the modeling data and then validating with the remaining 20%), we always back-test the model by scoring data/campaigns that were not involved in the creation of the model to validate accuracy.
- Incorporate scores into existing processes or create new ones. Models are only valuable if you use them to inform decisions about marketing or other outreach to constituents; that’s one of the reasons it helps to have a well-defined use case from the start.
- Monitor and fine-tune models based on real-world results. Any new model you introduce into your program needs to be tested – results may look great on the back-tests, but it’s essential to test on current data in case conditions have changed.
- Remodel as necessary to improve results. The accuracy of any model will change over time as the world changes and constituent sentiment evolves, so remodeling is always necessary at some point.
You’ve spent years collecting, validating, and storing your constituent data, so let’s work together to put it to use! For anyone currently using one of our models, we provide regular updates on model performance and look at ways to improve the model.
If you are interested in one of our existing models or have an idea for a model that could impact your organization, please get in touch with the MiLo Intelligence team to get the collaboration started.