The Art and Logic of eDiscovery
Ep. 38: The Art and Logic of eDiscovery: Navigating Managed Review for Cost Savings and Defensibility
This episode of Outside-IN we talk with Todd Kunz, the National Director of Managed Review Services at IST, shares his unique journey into the eDiscovery and Managed Review business, driven by a knack for logic and problem-solving.
Todd shares an insightful perspective on optimizing managed review processes, starting from the crucial, often overlooked, eDiscovery agreement stage where the biggest cost savings can be achieved by keeping options open for technology use. We delve into the cutting-edge technologies like continuously active learning (CAL) and Large Language Models (LLMs) used to efficiently manage vast amounts of data, while emphasizing the critical importance of human vetting and iterative processes to ensure defensibility and accuracy. Todd also candidly addresses managing client expectations, explaining why the process is inherently "imperfect" and how a robust, documented QC (Quality Control) process is vital to producing high-quality, defensible work.
Furthermore, the discussion highlights the strategic design of workflows to reduce unnecessary "document touches" and the importance of talent, including how remote work has expanded recruitment pools and how IST attracts top performers through competitive pay and genuine engagement. Todd reveals that creativity is at the heart of effective managed review, constantly seeking better, more logical ways to approach data and solve complex challenges.
Tune in to discover the intricate blend of process, technology, and human ingenuity that defines modern managed review.
Episode Highlights
Connect with: Todd Kunz | LinkedIn
Learn More about: Seeing Beyond the Pixels | Speeding Document Review with Analytics | Considerations for Outsourcing Document Review | Maximizing Managed Review Efficiences
Key Takeaways:
- Early Intervention is Crucial for Cost Savings: The most significant cost savings in managed review occur before the review even begins, during the eDiscovery agreement phase. It's essential to leave options open in the agreement to utilize various technologies and avoid locking into processes that could increase costs later. For example, agreeing to eliminate redundant emails in a thread early on can reduce data volume by half.
- Technology is a Game-Changer, but Requires Human Oversight: Tools like continuously active learning (CAL) and Large Language Models (LLMs) are vital for efficiently managing and reducing vast amounts of data. However, these technologies are not self-sufficient; they require continuous human vetting and an iterative process to ensure accuracy and defensibility. Just like a Google search, prompt adjustments and repeated checks are necessary to refine results and avoid consistent errors.
- Managed Review is Inherently Imperfect; Robust QC is Paramount: Clients must understand that the review process, regardless of tools or reviewers, is not perfect; different individuals will interpret documents differently. To ensure high-quality, defensible work, it is critical to have robust, documented Quality Control (QC) processes in place. This includes systemic searches for common pitfalls (like privileged information or critical merger terms) and continuous feedback loops with reviewers.
- Strategic Workflow Design Optimizes Efficiency and Accuracy: Workflows should be meticulously designed to minimize unnecessary "document touches," as each touch adds cost. Asking reviewers to identify too many issues (e.g., 50 instead of 5) on a single document can lead to decreased accuracy and increased costs. Focusing on what is truly needed and finding more efficient ways to collect information, rather than manual review for every detail, is key.
- Talent Evaluation and Management are Critical: The quality of the review team significantly impacts efficiency and accuracy; a small group of highly effective reviewers can outperform a much larger, less capable one. It's crucial to continuously monitor reviewer pace and accuracy metrics against benchmarks to identify and address underperformance early. Matching individual strengths and subject matter expertise to specific case types also contributes to success.
- Creativity is Essential for Problem-Solving and Optimization: Despite being a technical and rules-heavy field, creativity is at the heart of effective managed review. It involves thinking differently about data and finding innovative, logical ways to approach problems and design workflows, ultimately leading to better and more efficient processes and avoiding unnecessary work.
Listen, Watch and Subscribe!
Check out our previous episodes here:
