I Asked 10 AI Assistants for an AI Compliance Checklist. The Gap Was Embarrassing.

For twelve weeks I’ve been asking ten AI assistants the same questions about Australian business compliance, then scoring the answers. Question 12 was about automated decision-making and AI compliance specifically, the changes landing in December. The gap between the best and worst responses here was wider than anything I’ve seen in this series.

Here’s why the question matters. The Privacy Act’s automated decision-making transparency obligations kick in on 10 December 2026. If your business uses AI to screen job applications, automate marketing follow-ups, approve or reject loan-style decisions, or send personalised offers, you’re in scope. Get it wrong, and the OAIC can now impose fines up to $50 million. That’s not hypothetical. The enforcement machinery is already in place.

I asked all ten assistants a single question: give me an AI compliance checklist for Australian businesses preparing for the Privacy Act’s automated decision-making changes. Then I scored them on accuracy, Australian specificity, and practical usefulness.

What the best answers included

Grok and Meta AI both earned the top score. What set them apart was knowing which Australian-specific obligations are actually in play right now.

Grok was the only assistant to accurately cover the SMS Sender ID Register, which went live 1 July 2026. If you send marketing texts, your sender ID needs to be registered or your messages can be blocked. That’s a live obligation, not a future one, and most assistants either missed it or didn’t know it existed. Grok also correctly described the Privacy Act’s ADM transparency requirements with enough precision to be useful, not just “you need to tell people” but the specific obligation to provide meaningful explanation of automated decisions that significantly affect individuals.

Meta AI caught two things the others didn’t. First, the no-grandfather clause: businesses that were already using AI before the Privacy Act amendments passed don’t get a grace period. You’re in scope from day one if you’re in scope at all. Second, Meta AI named the Scams Prevention Framework (which passed in May 2026) and its relevance to AI-driven communications, and correctly described the government’s Voluntary AI Safety Standard, specifically the 10 guardrails that Australian government agencies are expected to follow, and which the government expects industry to use as a reference framework.

DeepSeek named the statutory tort for serious privacy invasions, which was introduced in June 2025. That one matters because it opens a civil liability path for individuals, separate from OAIC enforcement. Claude produced the most readable response and correctly handled the Fair Work Act’s nuances around employee records, which are partially exempt from the Privacy Act but not from the ADM transparency rules.

Where most assistants fell short

ChatGPT had the 10 December 2026 deadline right. That’s something. But the rest of its response was generic privacy-law boilerplate that could have applied to any jurisdiction. Nothing about the OAIC, the Privacy Act amendments specifically, the Spam Act, the Scams Prevention Framework, or any Australian regulator or enforcement mechanism. If a small business owner had acted on that answer alone, they’d have a checklist that looked tidy and missed most of what actually applies to them in Australia.

Gemini was accurate but thin. It correctly mentioned APP 8, the cross-border data transfer provision, which was actually a point none of the others flagged. But it didn’t go much further. A mention with no practical guidance on what APP 8 means for businesses using US-hosted AI tools isn’t much help.

Matilda was the worst performer this round. The response was structurally broken, incomplete sentences, a conflation of EU AI Act requirements with Australian law, and no shareable link. Matilda is an Australian-built tool, which made the poor performance on Australian compliance law particularly notable.

Copilot got the $50M penalty figure right but framed the consent obligations slightly wrong. Under the amended Privacy Act, the requirement isn’t purely consent-based, it’s a transparency and notification obligation. The distinction matters for how you set up your processes.

What this means for your business

10 December 2026 is four months away. If you’re using AI in any customer-facing or employment-related context, here’s what to do before then.

Map your automated decisions. Write down every place where an AI tool makes or substantially influences a decision that affects a customer or employee. Marketing segmentation, job application screening, credit or payment decisions, personalised pricing, all of these potentially fall within the ADM transparency obligations.

Check your privacy policy. You’ll need to disclose that you use automated decision-making and explain in plain language what it does. Vague statements about “using technology to improve your experience” won’t satisfy the obligation.

Review your AI vendors. If your AI tool processes personal information and the vendor is offshore (US, EU), APP 8 cross-border transfer obligations apply. Check whether your vendor has an Australian data residency option or a compliant data processing agreement.

Look at the Voluntary AI Safety Standard. If you’re a government contractor or supply chain participant, agencies may start requiring alignment with the 10 guardrails. Get familiar with them now rather than scrambling when a contract renewal comes up.

Talk to a privacy lawyer. The AI assistants in this series gave me useful starting points. None of them should be your final step before a compliance decision with a $50M penalty on the other side of it.

📖 More deep dives: News Deep Dives

📊 Compare AI tools side by side | 💼 Free resources & AI prompt packs

: News Deep Dives →

The Actual Checklist

The AI assistants were asked for this. Most gave you a version. Here is a sharper one, built from the best answers in the series and cross-checked against the actual Privacy Act amendments, the Voluntary AI Safety Standard, and the ACMA’s SMS Sender ID requirements. Print it. Assign it. Set a calendar reminder before 10 December.

1. Map every automated decision your business makes. Start with a list. Anything where AI, software, or an algorithm produces an output that affects a person counts. Job application screening. Loan or credit assessments. Personalised pricing. Automated marketing suppression. Dynamic service access. If a person’s privacy is affected and a machine made the call, it is in scope. You do not need a fully autonomous system. Partial automation counts if it materially influences the outcome.
2. Update your privacy policy before 10 December 2026. The new transparency obligations require you to disclose: what types of automated decisions you make, what personal information feeds those decisions, whether the logic is human-reviewable, and how an individual can request more information or seek review. A vague “we may use automated tools” line does not cut it. Write a plain-English section specifically about ADM. If your policy is currently silent on this, you are already behind.
3. Review your AI vendors for APP 8 compliance. Australian Privacy Principle 8 covers cross-border disclosure of personal information. If your AI tools process or store customer data overseas, that is a cross-border disclosure. You need either a contractual guarantee that the overseas recipient treats the data to Australian standards, or you need informed consent. Check your contracts. Most SaaS vendors have a data processing addendum available on request. Ask for it. If they cannot produce one, that is useful information too.
4. Register your SMS Sender ID if you send marketing texts. The Australian Communications and Media Authority’s SMS Sender ID Register opened on 1 July 2026. If you send commercial texts using a named sender ID (your brand name instead of a phone number), registration is required. Unregistered sender IDs face blocking. This is separate from the Privacy Act changes but was consistently mentioned by the stronger assistants in this series, and it is the kind of thing businesses miss because it sat in a different regulatory bucket.
5. Review the Voluntary AI Safety Standard’s 10 guardrails. The Department of Industry released this in September 2024. It is voluntary for now. The keyword is “now.” Larger businesses that use AI in high-risk settings are the most likely candidates for mandatory adoption when the regulatory conversation matures. The 10 guardrails cover governance, transparency, human oversight, and incident response. Reading them now tells you where your gaps are before anyone makes them compulsory.
6. Document your ADM logic. If a regulator or individual asks how a decision was made, you need to be able to answer. That means documenting: which system made the decision, which inputs it used, what the output criteria were, and who has authority to override it. This does not require publishing your algorithm. It requires having a record that survives staff turnover and a system upgrade. A shared document that nobody can find is not documentation. Assign an owner.
7. Set a hard review date before 10 December 2026. Book it now. Put it in your calendar, your project tool, your team’s shared diary. December compliance deadlines have a way of becoming January regrets when November gets busy. The OAIC has been explicit about enforcement intent. The $50 million penalty ceiling is designed to land on businesses that treated compliance as optional. A calendar reminder costs nothing.

One thing none of the AI assistants mentioned: compliance is not a one-time box-tick. Privacy Act obligations are ongoing. When you add a new AI tool, when a vendor changes their data handling, when you expand into a new customer segment, the checklist runs again. Build the habit, not just the document.

After 12 Questions: Who’s Actually Useful?

Twelve rounds of the same format. Same question structure, same scoring criteria, same ten assistants. The pattern is clearer now than it was at question three.

Grok and Meta AI have been the standout performers. Not because they sound authoritative, but because they consistently surface specific Australian details. Legislation names with dates. Regulatory bodies with actual jurisdiction. Enforcement mechanisms that exist, not vague gestures toward “relevant authorities.” When Grok named the SMS Sender ID Register in this question, that was not a lucky guess. It is the kind of specificity that only emerges from training data that includes recent Australian regulatory sources.

Claude scores well on readability and structure. The answers are clear, well-organised, and practically useful. Where Claude falls short is the same place it fell short in earlier questions: the Australian specificity sometimes goes shallow. You get the right topic but not the operative detail.

ChatGPT and Gemini have been the most consistently frustrating. Both sound knowledgeable. Both produce confident, well-formatted responses. Both consistently miss the Australian-specific layer that makes the difference between a useful answer and a generic one that could apply to any common law jurisdiction. Gemini in particular has a habit of producing responses that read like a compliance professional wrote them, until you check the citations and realise they are either US-focused or simply not linked to anything.

Copilot and Matilda have been reliably middling. Not wrong enough to be dangerous, not useful enough to be worth the effort.

The broader pattern: assistants trained on large, recent, jurisdiction-agnostic datasets tend to produce confident general-purpose answers. Australian-specific compliance is a narrow enough domain that general capability does not substitute for specific coverage. If you are using AI assistants for Australian compliance research, treat the output as a starting framework, not a finished product. The gap between “sounds right” and “is right for Australia” is where the risk lives.

The running scorecard is at smallbizai.au/nine-ai-assistants-scorecard. Check it before question 13.

Sources

• DeepSeek
• ChatGPT
• Claude
• Copilot
• Grok
• Manus
• Meta AI
• Gemini
• Perplexity

📊 See the running scorecard → Who’s winning after every question.