How Della was able to help review documents in a new language with only 100 contracts
A Dutch client, Legadex, wanted to analyse 900 contracts (leases and commercial customer/supplier contracts) in a new language for Della; Dutch.
The challenge was to understand a new language very quickly, without training our core model and with no help from native speakers (unfortunately nobody in the team speaks Dutch). 🙂
After training, Della was able to automatically detect 85% of the relevant legal points. Taking into account the time to teach Della’s models, Legadex completed the review 4x faster than manually.
The success of the project was entirely based on the learning capacity of our AI model and transferring this learning between different languages without affecting performance in other languages.
How did they do it?
Step 1 – Checklist creation
A checklist of 12 questions was created (in Dutch) dealing with parties, dates, renewal conditions, termination date and so on.
Step 2 – Initial Review
The client reviewed 100 contracts in Dutch to start the apprenticeship. At this stage 36% of the data was already detected by Della’s machine learning model – simply using the text of the question to direct the AI. Saving time during the initial review.
36% of the answers were correct at step 2.
Step 3 – Second Review
Della then trained our AI on the corrected answers.
After updating the model from this first set of review, a second batch of 260 contracts was analysed by the client’s team and submitted to the client for validation.
76% of the answers were correct at step 3.
Step 4 – Finalise the Review
After updating the model with this second batch, the remaining contracts were then reviewed by the client using Della.
85% of the answers were correct in step 4.
The time taken by the client to review a document was divided by three between step 2 and 4.
Lesson learned: clear success in using AI to review contract faster in other languages
In the end, the entire process took 3 weeks. The precision of Della’s results was similar to that of the lawyer who had reviewed these contracts manually.
Better still, Della was able to automatically classify the contracts even though only half of them originally were.
The time saving was estimated at 75% by the customer even taking into account the initial training in Dutch and the round trips during the learning phase.
The ability of Della’s analysis engine to handle new languages without having to modify the general model has been proven.
A hundred contracts are sufficient to validate the ability to answer a question in a new language.
New opportunities are opening up …
Legadex is a next-generation legal service provider with its head office in Amsterdam, the Netherlands. The company was founded in 2008 by Hans-Martijn Roos and Luc van Daele, who saw that corporate law departments were looking for better alternatives for handling their legal work and becoming more cost-conscious.