How to conduct valid surveys that will help you choose between Westlaw and Lexis.
Its contract renewal time for the firm’s Westlaw or Lexis subscription. You’ve worked with the sales reps from both companies, maybe allowed them to do a demo to the firm and gone to the trouble of getting competing proposals from both vendors. How do you decide which vendor to choose?
In my 25 years in this industry, I’ve seen countless firm’s struggle deciding between competing proposals from Westlaw and Lexis. Many firms, in a well-meaning but misguided effort to make a more informed decision, opt to conduct firm-wide surveys to find out what the “lawyers prefer”.
While this may seem like a good idea, without the right work done in advance, these surveys result in ill-informed decisions confirmed by unsubstantiated opinions that all too often locks a firm into an unfavorable and needlessly costly contract.
So, what’s wrong with doing a survey? Nothing, but you must follow a process that ensures the responses are fact-based, with defined concerns, and not ambiguous, generalized references to “preference, familiarity or beliefs” if it’s to provide reliable guidance.
Survey Do’s and Don’t
Do: Set up a trial of the competitor’s product
Don’t: Proceed with a survey without a trial
I know, this seems self-evident. How can you assess competing products if you don’t have access to both? (I’ve seen it done!) I recommend establishing a limited trial period prior to sending out your survey, at most 2 weeks in duration. Anything longer than 2 weeks ensures that the group will delay using it and likely forget about the task. I suggest creating a sense of urgency and starting the trial at 1 week then expanding to 2 if needed. Both Westlaw and Lexis will gladly establish trials for firms. These trials generally must be no longer than 30 days.
Do: Select a limited group to trial product
Don’t: Open to entire firm
Select trial participants from these groups: Power users of your current vendor, users that are considered thought leaders/influencers in the firm and those that you believe will approach the task objectively. Limiting the trial participants to these groups engages the necessary stakeholders giving the ultimate survey results more credibility within the firm.
Do: Require participants to take training on the competitor’s product
Don’t: Let participants slide on this!
This is probably the most difficult, but also the most important aspect of generating reliable survey results. Participants MUST spend some time with the non-incumbent vendor’s training team to make sure they can truly assess the product. This is not as onerous as it sounds; I recommend setting up an individual, telephonic training session for each participant (both Westlaw and Lexis will happily comply). Doing this allows the participant to ask questions that are both unique to their practice and commensurate with their familiarity of the tool.
Do: Communicate to keep them on track
Don’t: Set up a trial and assume they will try it
Make sure you touch base with the participants throughout the trial to reinforce the timing expectations and to ensure they keep the request top of mind. Even though you handpicked them, these are obviously busy people, so the chance for this to fall by the wayside is high. A simple email to the group reinforcing the importance of their feedback to the firm and reminding them of the time frame of the trial will go a long way to make sure the survey responses are well-informed and helpful to your decision-making process.
Do: Create survey questions that require fact-based responses
Don’t: Ask for generalized “feedback”
Remember, the purpose of conducting a survey is to elicit responses to questions that will help the firm make an informed, fact-based decision to determine if a vendor’s product is adequate for the firm’s needs. An example of such a question is, “Please indicate if you found all the resources necessary to conduct full and complete research on your topic. If no, please specify the content that was missing.” The question tells you not just whether the user feels the vendor has the specific content they need, more importantly, it requires them to specify exactly what was missing if they could not answer in the affirmative. You can now reach out to that vendor and confirm or deny that content (or functionality) does or does not exist. The fact is, there are few instances where content or functionality is not in Lexis or Westlaw’s offering; in most cases the user just didn’t know where to find it. Generalized questions like, “What did you think or Give me your feedback” are going to produce vague, emotion and preference-based responses that do nothing to help you make an informed, fact-based decision.
Do: Consider opinions only from those who participated in the trial
Don’t: Send surveys to participants with little or no meaningful use during the trial
This is simple. If they participated in the trial, they participate in the survey. If they didn’t log into the service during the trial, they should not participate in the survey. Both Westlaw and Lexis can track usage during a trial by individual participant and they will make that information available to you on a weekly basis. Review it and use it to find out who needs a reminder to log in and to determine whether someone has enough use to justify including their opinion in the decision-making process.
The takeaway… When conducted correctly, surveys are an extremely effective tool that allows firms to make an informed, fact-based decision about which legal vendor is right for them. As outlined above, the process is not difficult. The challenge is to set the correct expectation at the outset and stick to them throughout the process. The result could be a significant reduction in overhead. Given the increasingly high cost of legal research contracts, allowing decisions to be made based on “preference and beliefs” is a costly mistake firm’s simply can no longer afford.