Meeting unmet legal needs with NLP

One of the great powers of a technology like natural language processing is that it can be applied to both general and specific tasks. People query popular large-language models with a wide variety of questions on topics ranging from pop music to probability theory — and everything in between. But natural language processing, or NLP as it’s known, can also be applied to very specific tasks.

One example of a precise application of NLP is a recent collaboration between an Australia-based charity called Justice Connect and Timothy Baldwin, Acting Provost, Associate Provost for Academic Affairs and Professor of Natural Language Processing at MBZUAI.

As an organization, Justice Connect focuses on providing support to people who need legal assistance, explained Tom O’Doherty, Head of Innovation at Justice Connect. The organization offers a variety of services, including one-on-one legal support, self-help resources on the web, pro bono legal work and digital tools.

Despite the huge volume of legal work undertaken by organizations such as Justice Connect, there’s a much larger unmet need for these services. “There are many millions of people throughout the world who are in need of legal aid,” O’Doherty explained, “and several million in Australia alone.”

One of O’Doherty’s responsibilities is to make sure that people who use Justice Connect’s digital tools have a positive experience when they interact with them. O’Doherty was assessing how the organization’s intake tool was working for a potential client. This intake tool is designed to help route requests for help so that they can be reviewed by people who are knowledgeable about the related area of law.

Falling through the cracks

“The person I was with was able to easily fill out the first two questions in the intake form, which related to their location and the kind of issue they were having,” O’Doherty said. “But when they got to a question that asked about the area of law that their issue was related to, they didn’t know how to answer.”

Confused and frustrated, the person chose an in applicable area of law, simply to get through the process. It turns out that this was a common occurrence. “Almost all of the users had a problem identifying which area of law their problem related to, which stems from legal language being difficult to understand for most people,” O’Doherty said.

Answering that simple question with the wrong answer has real consequences for people who are seeking help. By getting the question wrong, more time is spent getting them the resources they need. Ιn some cases, they become frustrated and end their search for support all together.

With this knowledge, O’Doherty thought that it might represent an opportunity for an innovative application of natural language processing.

O’Doherty connected with Baldwin, who was working at the University of Melbourne at the time. The two devised a way to address the problem that melded the capabilities of NLP and the specialized expertise of attorneys.

Though NLP has been applied to the legal field for tasks related to finding evidence and predicting legal judgments, Baldwin and O’Doherty’ project was different. “What we’re doing with Justice Connected is removed from most of the mainstream legal NLP work, which is very much about helping attorneys,” Baldwin noted.

Justice Connect has a large amount of data that has been collected over the years through the intake form. This data proved to be a useful starting point, but the team needed to train their model to make sense of it. Enter Justice Connect’s roster of motivated and socially minded lawyers.

Expert volunteers/the power of pro bono

Led by O’Doherty and Baldwin, the joint team established an interactive process by which lawyers, who worked pro bono, categorized historical responses to the intake form, training the model to associate different terms and phrases with related areas of law. They were able to get attorneys to help them with this by turning the often-tedious task of training into a game that tapped into the attorneys’ competitive nature.

This gamification was extremely successful. Nearly 300 lawyers participated and categorized 200,000 pieces of text.

“The network of pro-bono lawyers that are engaged through Justice Connect is such an inspiring community,” Baldwin said. “The idea that we can get highly skilled, highly paid lawyers to volunteer thousands of hours to do the data annotation to help in the development of AI tools is almost unheard of.”

The result is that user experience of the survey has improved as well, as measured by the number of people who start the form and don’t finish, which is down by more than 50%. It has also resulted in a 10% increase in the total number of services that are initiated through the survey.

That said, there are challenges that must be addressed, O’Doherty said. He and his team have been working to take into account the usability of the intake form for underrepresented communities by adding more responses from these communities to their models. “We’re able to train the model with samples from specific groups, which improves the model for those groups,” O’Doherty said. “One of our goals as a social justice organization is to make sure that our work is inclusive of the people who are in most need of legal resources.”

Justice Connect plans to expand their offering of tools that are powered by natural language processing. They hope to add more functionality to their intake form to make it responsive, providing it with the capability to ask follow-up questions to that it can better route requests.

“There’s a very real human element to it, on the side of the help seeker as well as the legal triaging,” Baldwin said. “It’s very compelling to be able to contribute to this high-impact initiative.”