Redaction software · Buyer's guide
Every tool passes its own demo.
These questions are the harder test.
Watch enough redaction demos and they blur together: the vendor drives, the PDF is clean, the automation catches everything on the sample page, and the products all start to look interchangeable. The differences live where demos never go. A scanned exhibit. A long thread with privileged advice buried in the middle. A protective order with defined categories. An output file addressed to opposing counsel. These eight questions drag those differences into the room, and they work on any vendor. Including us.
Question 01
Does the automation read the content, or apply patterns configured at setup?
A pattern rule is a shape matcher. Configure it once and it will catch a Social Security number or a card number every time, because those strings have a format. Privilege has no format. Counsel's assessment of a claim, forwarded into the middle of a long thread, reads like every other paragraph of business email, and no shape defined at setup will ever find it. The layer that reads meaning decides whether the tool can find the content that would actually hurt your client.
A good answer separates the two layers honestly: patterns for formatted identifiers, reading for everything that only exists in context.
Question 02
Does it propose a mark with the exemption attached, or flag a document as risky?
In legal work the reason is the redaction. Text is withheld because it is privileged, because it is PHI, because a protective order defines it as confidential, or because it is a client's personal information. A tool that only scores documents as risky hands back a to-do list; someone still opens each file, hunts for the passage, and supplies the reason from scratch. A proposed mark that arrives already carrying its reason is most of that labor done, reduced to a judgment call.
A good answer points at one highlighted passage with the reason attached, privilege or PHI or a protective-order category, waiting on your call.
Question 03
Who approves each mark, and is the decision recorded by name?
Sooner or later a production gets questioned. A clawback dispute asks why a privileged passage went out. A sanctions motion asks who decided a category was applied too broadly or not at all. "The software did it" is not an answer anyone wants to give a judge. Every mark should carry a named human decision and a timestamp, so that when the fight arrives you produce a record rather than a reconstruction from memory.
A good answer opens any single mark and reads out the passage, the reason, the reviewer's name, and when the call was made.
Question 04
Can anything leave before review is finished, and is the output verified?
The nightmare here is specific: a draft with undecided marks reaching opposing counsel, or a produced file whose text sits intact underneath a drawn rectangle, one select-all and copy away from disclosure. Both failures have happened to real firms. Ask whether export refuses to run while any mark is undecided. Then ask whether the finished file is checked to confirm the underlying text is removed from the document, not merely covered by a black box.
A good answer shows you the export refusing to run, then verifies a finished file to prove the text is gone from the layer beneath.
Question 05
Is there one log for the whole production?
A production is not a folder of PDFs. It is documents, email exports, attachments, and spreadsheets, and the redaction decisions across all of them have to hang together as one defensible story. If every format keeps its own separate trail, someone stitches them together by hand the night before the deadline. One log across the entire production is the difference between defending a set and defending a pile.
A good answer filters a single log by custodian, by reason, or by reviewer, across every file type in the matter.
Question 06
What happens to a whole email export?
Email is where document counts explode. One custodian's mailbox export can hold thousands of messages plus their attachments, and the same thread routinely shows up in three custodians' files. Ask whether the tool takes the export whole, splits each attachment into its own reviewable record, and recognizes the duplicate thread the third time so it is decided once. If the answer begins with "first you convert it," that conversion is unpaid work assigned to your team.
A good answer accepts the raw export and returns individual records, attachments separated, duplicates matched across custodians.
Question 07
What are the limits behind the headline?
Case volume is spiky. Discovery lands, a supplemental production doubles it, and the month before a deadline can outweigh the rest of the year combined. Whatever the headline plan promises, ask what meters underneath it: page allowances, per-document overage, seat minimums, annual commitments. Then price your heaviest plausible month and watch whether the number surprises anyone on the vendor's side of the table.
A good answer names every meter in plain language and can quote your worst month on the spot, without a follow-up call.
Question 08
Where does the content go, and does it train anything?
These are client documents, and confidentiality is not a preference. It is the professional obligation the entire engagement rests on. Before a single client file uploads, get the answers in writing: where processing happens, what is retained and for how long, and whether client content is ever used to train AI models. If the training answer is anything other than a flat no, the evaluation can end right there.
A good answer is a document you can forward to your own security review: processing locations, retention terms, and an unqualified no on training.
The checklist
The same eight, ready to copy.
Keep them in the room when the vendor is driving.
Eight questions for any redaction demo
- Does the automation read the content, or apply patterns configured at setup?
- Does it propose a mark with the exemption attached, or flag a document as risky?
- Who approves each mark, and is the decision recorded by name?
- Can anything leave before review is finished, and is the output verified?
- Is there one log for the whole production?
- What happens to a whole email export?
- What are the limits behind the headline?
- Where does the content go, and does it train anything?
Copy this into your notes and bring it to any demo, including ours.
Ask us these same questions.
This list is not neutral. We publish it because the questions play to how RecordBeacon is built, and that claim deserves your skepticism. So do not take it on faith. Start a free workspace or book a walkthrough, bring your own documents instead of ours, and grade our answers the way you would grade anyone else's.
Questions about the list
Do these questions apply to small firms or solo practices?
Yes, arguably more. A large firm has litigation support staff to absorb a tool's gaps; a solo practitioner is the review process. The duty of confidentiality does not scale down with headcount, and neither does a clawback fight. Skip the procurement ceremony, keep the list open beside a free trial, and check each answer yourself on one real matter.
What should we bring to a demo?
Your ugliest scanned document and a real email export. Never evaluate on the vendor's sample file; it was chosen because the product performs well on it. A crooked fax from 2009, a thread with an attachment inside an attachment, a spreadsheet with names in the wrong column: fifteen minutes on your own worst documents tells you more than an hour of prepared material.
Why does RecordBeacon publish evaluation criteria?
Because when buyers ask these questions we tend to win, so publishing them is in our interest, and pretending otherwise would be silly. The list is useful for exactly that reason: every item on it is checkable against any product, including ours. Verify it yourself and keep whichever tool survives.
Doing public records work? This guide has a version written for records offices: evaluating redaction software for public records.