Improving document processing with automation

Adapted from "Automated Document Processing for Approving e-Filings" in AI Readiness for State Courts.

What the AI tool does

Document automation software by CSI/Tyler Technologies processes court filings using a combination of AI and Robotic Process Automation. In the Texas courts, the software is used to verify that e-filings meet the minimum standards to be electronically filed. 

Identifying the need

The court determined that because the volume of e-filings was so high, it was taking too long for the clerk's office to accept the filings. The staff's capacity to keep up with this task wasn't high enough to meet the demand using human labor alone, so the court looked for a way to automate parts of the workflow. 

Design, development & testing

The main stakeholders were the clerk's office and the court staff who do caseflow processing tasks downstream from the clerk's office. One big impact of their feedback was that the court eventually decided to only operate the new system during business hours. Although one of the benefits of automating tasks is that they can be done 24 hours per day, the high efficiency of the technology in this case was creating stress for the staff. They would arrive to work on Monday morning to find 2.5 days' worth of filings that had been approved over the weekend and were ready for staff review. They shared that this made them feel like they were always behind. The purpose of the technology is to lighten the load for court personnel, not to make them feel more stressed, so the court now uses the technology to process new filings during the same hours that staff are working.

Implementation & evaluation

It was important to be transparent about the process and let personnel know when things were happening. It was also critical to manage expectations and help people understand that perfection wasn't the goal. A common misstep in court technology projects is to want the tool to be perfect before turning it "on." The project team had to make a deliberate effort not to let perfection get in the way of having a good product that could be improved over time.

The new system was initially implemented for only one case type, so the court could assess how it performed. Right out of the box, CSI's model was getting about 60% right, before it had the chance to learn from the court's data. But once the data from each clerk's office started being fed into each model, that percentage increased. Staff spent time reviewing the decisions that the technology made and provided feedback for gradual adjustments and improvements. Over time, the system performed better, and the court gained confidence in its decisions. Now, the accuracy rate is almost 95%, which is better than human performance. Eventually, the system was expanded to other case types.

Monitoring, maintaining & updating

Initially, the project required extra staff time to train, test, and improve the models. The court allocated some staff time to the project to determine what other tasks would decrease as a result. As the technology improved over time, staff spent less and less time reviewing the output. Once the technology was fully implemented and the new, more efficient workflow was in place, the ongoing costs were the subscription fees for the software and the typical IT staff resources that the court uses to maintain all of its systems.