Ethical and Regulatory Challenges of Using AI in Accounting

Ethical and Regulatory Challenges of Using AI in Accounting

Accounting is changing in ways that were unthinkable ten years ago due to artificial intelligence (AI).

AI promises quicker, smarter, and more effective operations, from automating journal entries to identifying fraud.

However, great power also comes with great responsibility, and in this situation, that obligation includes a long list of unavoidable ethical and legal issues.

According to Karbon’s 2025 global poll of accountants:

• Users of advanced AI save 71% more time each day (around 79 minutes compared to 49).
• When a firm invests in AI training, employees save 22% more time than those who don’t- it’s a difference of 40 hours annually, per employee.
• Firms investing in AI training unlock seven weeks of staff time yearly per employee.

Major providers also observe this increase in productivity: According to Intuit, small businesses can save up to 12 hours a month by using its QuickBooks AI agents.

Let’s break it down into practical issues and what businesses, experts, and regulators need to consider to stay on the right side of innovation.

Data Privacy & Security Risks

Data is what AI lives on. It gets more accurate the more it has. However, it poses a privacy concern due to that same dependency.

AI systems in accounting handle extremely sensitive data, including:

• Tax returns
• Payroll information
• Income records
• Client names

If this data is not handled securely, the consequences could be disastrous.

According to IBM’s 2024 Cost of a Data Breach Report, the average cost of a breach in the financial sector is $6.08 million, second highest across industries.

Strict controls on the collection, processing, and storage of personal and financial data are required under data protection regulations such as:

• NDPR
• PDPA
• CCPA
• GDPR

in several jurisdictions.

This environment is made more challenging by AI, particularly if the system is trained on customer data without explicit consent or anonymization.

Pro Tip

• Ensure AI accounting solutions comply with national and international privacy regulations
• Implement encryption or anonymization techniques
• Restrict access through robust identity management

Bias and Discrimination

Not all AI is neutral.

The quality of AI systems depends on the quality of the data they are trained on.

AI may unintentionally reinforce inequality if the data exhibits previous biases, whether in auditing, lending, or spending clearance.

According to a recent May 2025 arXiv study, well-known AI models (GPT-4, Claude, and Gemini) gave profiles markedly different risk scores depending on gender and nationality, demonstrating that bias still exists even in sophisticated systems.

Pro Tip

• Make use of representative and varied training data
• Carry out frequent bias audits
• Choose AI systems that offer decision-making logic that can be explained

The “Black Box” Problem

One of the most debated concerns in AI is its lack of transparency.

Although many machine learning models, particularly deep learning systems, provide insightful information, they don’t necessarily explain their reasoning.

In accounting, where every choice must be traceable and auditable, this becomes a significant problem.

The UK’s Financial Reporting Council observed in 2024 that a number of big auditing companies that used AI lacked appropriate KPIs to monitor how these tools were affecting judgment.

During a regulatory assessment, auditors might not be able to defend a system’s risk flags if they are unable to explain how they were determined.

Pro Tip

• Select AI technologies that facilitate explainability (XAI)
• Keep track of model logic documentation
• Make sure decision-making processes incorporate human evaluation

Example

XAI is used to assess financial qualifications for loans or mortgage applications and to detect financial fraud.

Accountability and Legal Responsibility

Who is accountable if an AI system makes a mistake, like failing to detect a fraud alert or incorrectly classifying a revenue entry?

Human professionals are held accountable by the majority of laws and ethical regulations.

Therefore, businesses need to make sure AI technologies are regularly monitored, evaluated, and audited, not just when they are deployed.

The EU’s Artificial Intelligence Act (Regulation (EU) 2024/1689), effective from 1 August 2024, adopts a risk-based approach and explicitly requires human oversight for high-risk AI systems, which include tools used in accounting and auditing

Pro Tip

• Keep responsibility frameworks clear
• Establish human-in-the-loop protocols
• Refrain from depending too much on automation when making important judgments

Regulatory Compliance: A Moving Target

The laws attempting to regulate AI are not keeping up with its rapid advancement.

• The EU AI Act categorizes AI tools used in accounting and auditing as “high-risk” systems, requiring mandatory testing, transparency reports, and external audits.
• In the US, multiple states and federal bodies are proposing laws around AI fairness, transparency, and workplace use.
• In Nigeria, the NDPR requires all data-processing technologies to comply with consent, minimization, and accuracy principles.
• Singapore’s IMDA has released Model AI Governance Frameworks promoting ethical AI across finance and enterprise sectors.

A 2025 report from Legalfly revealed that while 90% of financial firms use AI, only 18% have formal policies, and just 29% enforce them consistently, leaving data protection compliance widely neglected

Pro Tip

• Make use of adaptable compliance designs
• Assign policy owners
• Keep a close eye on regulatory changes

Explore how accounts receivable automation balances efficiency with human oversight.

Ethical Boundaries in Automation

AI lacks moral judgment. While it enhances fraud detection and anomaly spotting, professional scrutiny remains vital.

AI can help with accounting, but there’s a thin line between that and depending on it to take the place of expert judgment.

Complex decision-making that is automated without ethical consideration may result in negative financial consequences or damage to one’s reputation.

AI improves anomaly detection and expedites fraud discovery in auditing. However, professional obligations necessitate:

• Examining results produced by AI,
• Evaluating the importance of the context, and
• Relying on human expertise to make final choices.

Key consideration: Keep human judgment in high-stakes situations, promote ethical education, and refrain from placing uncritical faith in AI-generated results.

Workforce Readiness and Training

Although AI tools may replace repetitive accounting work, job loss is not the result. Instead, reskilling is required.

Future accountants will need to be knowledgeable with data pipelines, model behavior, and automation ethics. Upskilling is necessary, not optional.

In 2024, UK unions warned that up to 54% of banking jobs and 48% of insurance roles could be disrupted, urging widespread reskilling

Pro Tip

• Provide continuous learning opportunities around AI tools, ethics, and emerging technologies.

Final Thoughts: The Future Is Intelligent, But It Needs to Be Secure

AI in accounting is quickly becoming a competitive requirement rather than a luxury. However, great capability also carries great responsibility.

Adopting AI ethically means more than installing software-it means embedding trust, transparency, and accountability into your systems and workflows.

Finance professionals, tech leaders, and regulators must work together to build a future where AI empowers good decisions without compromising ethics or compliance.

Start by asking the correct questions when examining AI in your accounting procedures: “How can we use it responsibly?” rather than “What can it do?”