The trouble with secondary sources in WestlawNext

With WestlawNext (WLN) finally going live for law school faculty, librarians and staff yesterday afternoon, many law school librarians are finally getting permanent access to the much-hyped interface. This comes only a couple of weeks after Julie Jones's fascinating panel at the AALL Annual Meeting, "The Economics of Interface: Vendors Respond." Following presentation's by Thomson Reuters' Mike Dahn, LexisNexis' Molly Miller and Fastcase's Ed Walters, Larry Abraham of Fordham Law stepped up to the audience microphone and told the vendor representatives that their interfaces discourage users from using secondary sources, instead emphasizing primary law materials. Following Laurence's comments, a few other audience members (myself included) complained to the panel about how vendors treat secondary sources both within their systems and in training provided to subscribers.

With that fresh in mind, I wanted to hit on some specific problems I've encountered while searching secondary sources in WLN. My concerns aren't new. I mentioned them in my initial review of the product back in January. Specifically, when searching state-specific secondary sources, the results are often cluttered with irrelevant materials at the top of the results list. Here's part of what I had to say back in January:
The search algorithms for secondary sources are also problematic. When I select California as my jurisdiction, if there are California-specific resources available on my topic, those need to show up first in my results. Period. When I ran a search for the term "rape," the first 12 results were from "National" sources such as Am.Jur. Trials, Am.Jur. Proof of Facts, and several law reviews. More alarming, none of these were general overviews of rape law, but instead provided information on specific aspects of rape law in various jurisdictions. I discussed this problem with Mike Dahn, vice president of new product development at Thomson Reuters Legal. He assured me that this was not by design and that they were working to make sure jurisdiction-specific resources for a user's selected jurisdiction float to the top of search results, but as of now this isn't working correctly.

The problem I described then still occurs. Here's what the top search results for the Secondary Sources facet look like when I select California as my jurisdiction and search for the word "rape":

The first result from a secondary source covering California law doesn't appear until the 11th result. This means a researcher, despite having already selected a jurisdiction, has to wade through numerous off-topic, out-of-jurisdiction articles to find the ones that are on point for the search.

There is, however, a way around this problem. Having already narrowed the search results to the "Secondary Sources" facet, the left sidebar now displays additional filters. Find the one labeled "Jurisdiction" and select "California":

With this one tweak, the results reflect more of what a researcher expects to see when performing a jurisdiction-specific search:

The jurisdiction sub-facet illustrates part of the problem with the original set of results. Despite a researcher having selected California as the jurisdiction before running the search, WestlawNext still includes "National" sources in the results. In a state with fewer high-quality secondary sources than California — say Nevada — including these national sources is probably a necessary evil (though one would hope these national search results would be more on point). Just realize that if you want only state-specific resources you'll need to do that extra step of filtering.

Oddly, even if you narrow your search target to only "California Secondary Sources" by browsing the WLN sources before running your search, WLN still considers those national sources to be "California Secondary Sources," so you'll again need to apply a post-search jurisdiction filter to get true state-specific results.

But the problem with secondary sources in WLN doesn't go away by selecting a sub-facet. This refers back to another problem I discussed in January:

In these results, very little specific information about each source displays. Most of the time what a researcher sees is the title of the specific section, the title of the source, and a couple of text snippets in which the search terms are highlighted. Little to no information about where within that source the section appears shows in these results. For example, when a section of "Witkin and Epstein's California Criminal Law" title simply "Generally" appears, that's the extent of the citation information displayed. The precise article within "Witkin & Epstein" in which this section appears is nowhere to be found, so researchers won't know until viewing the document itself whether it's from an article about sex crimes, robbery or murder. The threaded information available in search results for statutory sections (Title, Chapter, Subpart, etc.) needs to be included in secondary sources as well, or else the results are confusing at best and unusable at worst.

Take another look at that last screenshot showing results from California-only secondary sources. This uses WLN's default "More Detail" display type. In these results, the first result looks to be the most relevant. After all, it is titled "Rape" in big bold print. The second and third results also look to be on point, but the information provided is rather cryptic about what specific crime they refer to. If the source in question isn't part of your subscription plan, the only way to find out if they're relevant is to click on each and incur an out-of-plan charge. Not exactly the best way to go about cost-effective research. Worse still, that first result — the one titled "Rape" — that looks so relevant? It turns out this is a section from an article about First Degree Murder that only discusses rape insofar as it pertains to the commission of a homicide. The second result ("Nature of Crime") is the only one in the top three actually from Witkin & Epstein's article about the crime of rape. The third document in the results ("In General") is, like the first, about homicide.

There is a more detailed view available than this one, but it doesn't solve the problem. To change the display type, go to the display options at the top of the search results (the icon showing one to three horizontal lines) and select "Most Detail":

Even with the most detail display available for search results, however, the full threaded information isn't included and it's still impossible to tell whether each result concerns the specific crime of rape without retrieving the document itself:

This isn't a problem so long as the source is part of your subscription plan and you can pull up as many documents as you want. But if "Witkin & Epstein's California Criminal Law" isn't in your plan, you might be faced with quite the dilemma when deciding how important it is to find the right document. Or perhaps you'll skip secondary sources altogether, jumping instead into the primary law results directly.

Truth be told, the more specific your search terms ("unlawful sexual intercourse with a minor" vs. "rape"), the stronger likelihood of having relevant search results float to the top. This is true of any search engine, and I'm not suggesting the search algorithm is the problem. It's the display. Given that general searches like "rape" are common in legal research, particularly when searching secondary sources, the lack of useful identifying information about a document is a problem that needs to be addressed by Thomson Reuters Legal.

I don't want this post to be interpreted as a negative review of WLN. Most of my review in January was positive and I stand by that overall opinion. WestlawNext is a vast improvement across the board over Classic Westlaw, and I don't share the opinion of many that it "dumbs down" legal research. To the contrary, I've found that it adds considerable power to most of the research I've done using the new system. Figuring out what database I need to search in Classic Westlaw doesn't mean I understand the sources of law any better. It just means I know what database I need to search in Westlaw.

But as superior as it is to its predecessor, it still has legitimate problems. The issues associated with secondary sources in WLN need to be highlighted. Even if Thomson Reuters opts to not fix these problems, we as researchers need to be aware of them. And as instructors — both in the classroom and at the reference desk — we need to be prepared to educate others about them, too.