Not surprisingly, GenAI is quickly becoming embedded in workflows across organizations. Professionals are using it to draft emails, summarize reports, prepare presentation outlines and synthesize research. What began as experimentation has rapidly evolved into routine productivity support.

Within equipment leasing and finance firms, the same shift is underway. GenAI tools are increasingly used to support market research, summarize industry developments, assist with credit narratives and help structure internal analyses or executive communications. In an industry built on data, interpretation and documentation, the productivity gains are easy to see.

But have you ever researched a new equipment segment where GenAI provided a well-reasoned summary including precise market growth statistics, projections and other industry information? While the analysis appears to be authoritative, were you able to validate the sources against credible research studies? Did you even think to check? 

Or perhaps you heard about the attorneys who submitted a legal brief containing six judicial decisions that looked legitimate, complete with case names and quotations. There was just one problem: the cases did not exist, and the attorneys were ultimately sanctioned by the court.

These situations illustrate a growing issue in the age of GenAI: hallucinations.

An AI hallucination occurs when AI assistants such as ChatGPT, Copilot or Claude produce information that is confidently stated — but is factually incorrect or even fabricated. In other words, AI assistants can generate content that sounds authoritative but has no grounding in verifiable evidence. 

Why does this happen? Quite simply, AI assistants are designed to generate responses based on patterns and probabilities in data, rather than independently verifying information against authoritative facts. The challenge is that AI-generated errors appear credible and do not signal uncertainty.

For executive teams increasingly experimenting with GenAI, the risk is not obvious mistakes. It is confident fabrications that slip quietly into strategy discussions, forecasts and board-level materials.

The Mechanics of Fabricated Authority

So why do AI hallucinations occur in the first place? 

AI assistants rely on large language models as their core ‘brain.’ These models generate responses by recognizing patterns and probabilities in large quantities of data. When information is incomplete or unclear, these models predict the response and produce answers that are the most plausible. The resulting answers appear definitive but are not independently verified against a trusted set of facts. 

Hallucinations are not a user error or a system glitch — they are a direct consequence of structural limitations of large language models. AI produces what looks right, not what it knows to be true.

It is important to distinguish hallucination from bias and simple error.

Bias reflects skewed framing or emphasis embedded in the data itself. The underlying facts may be real, but the perspective is imbalanced.

A simple error, by contrast, is an incorrect fact — such as a wrong date or outdated statistic.

Hallucination is different. It involves manufacturing information that has no basis in reality: a nonexistent court case, an invented research study or an invented quotation. That distinction matters because fabricated authority carries different reputational and governance risks than a typical mistake.

Where Hallucinations Hide in Finance Workflows

In equipment leasing and finance, AI hallucinations can appear in more places than many executives expect. 

AI hallucinations are not limited to fabricated citations of market trends or industry statistics. They also appear in GenAI-supported workflows such as credit narratives, asset valuation analysis, residual forecasting, portfolio analytics and industry research. Here AI hallucinations rarely appear as obviously false statements. Instead, they surface as plausible but incorrect assumptions embedded within otherwise polished analysis. Said another way, this is where human interpretation of data outputs is essential to avoid costly portfolio mistakes and drains on capital.

Consider a few examples.

In credit underwriting, a GenAI tool might summarize a borrower’s industry outlook and cite a growth trend that appears credible — but is based on outdated or misinterpreted market data. An experienced credit professional would recognize whether that trend aligns with current sector conditions.

In asset valuation, an AI-generated analysis might confidently explain residual value behavior by referencing market comparables that are incomplete, misclassified or entirely fabricated. Human valuation specialists must verify whether the secondary market data supports the assumptions.

In residual forecasting, GenAI may extrapolate historical equipment demand trends without recognizing structural shifts in the market that materially affect asset values. For example, an AI-generated analysis might project stable resale values for mid-sized construction equipment based on historical auction data, while failing to account for emerging emissions regulations, changes in fleet replacement cycles or shifts in contractor demand. To someone unfamiliar with the sector, the analysis may appear convincing. To an experienced leasing professional, the assumptions may raise questions.

In portfolio risk analysis, GenAI may produce convincing narratives explaining changes in performance metrics, attributing them to correlations that appear logical but are not supported by the underlying data.

And in industry research, AI tools may generate market sizing figures or growth projections that read like professional research reports but cannot be traced to credible primary sources.

Experienced professionals bring context that AI systems lack: familiarity with how equipment markets behave across cycles, insight into how credit risk evolves in specific sectors and the ability to recognize when an analytical explanation does not align with real-world market behavior.

As GenAI moves closer to credit decisions, asset valuations and portfolio strategy, the need for disciplined human oversight does not diminish — it increases.

Along the same lines, clients are asking us whether there is still a role for consulting firms in the age of GenAI. The answer is yes — and the nature of that value is becoming clearer.

GenAI excels at accessing information, summarizing content and accelerating analysis. What it does not possess is experience — particularly the kind developed over decades of working across economic cycles, asset classes, regulatory shifts and market disruptions.

It cannot see around corners, recognize early signals of market inflection or distinguish between a temporary anomaly and a structural trend — all hiding places for AI hallucinations.

Consulting firms play an important role in helping organizations interpret AI-generated insights, challenge elegant but unsupported conclusions and apply experience-based judgment. GenAI can accelerate analysis. Consulting expertise provides context and the wisdom to prevent clients from making false starts based on AI hallucinations.

Governance Over Avoidance: A Roadmap for Leaders

So how should equipment leasing companies prevent AI hallucinations in corporate decision making? The answer is not avoidance, but governance. A practical rule of thumb is simple: GenAI can help organizations think faster, but it cannot determine what is true. The following guardrails will serve organizations well in minimizing AI hallucinations.

1. Use AI for synthesis and drafting — not as the final authority on facts. GenAI is well suited for organizing ideas, summarizing information and producing early drafts. It should not be treated as a definitive source for statistics, legal interpretations, market sizing or financial projections without independent validation. Disclose the use of AI assistants in these contexts.

2. Match human review to the level of risk. The greater the reputational, regulatory or financial consequences of a statement, the more rigorous the human verification required. Internal brainstorming may tolerate AI-assisted drafts; board materials, investor communications, credit documentation and publications require full human validation.

3. Never rely on AI output alone for financial or risk decisions. AI-generated insights should never be the sole basis for investment decisions, market entry strategies, credit assumptions, asset valuations or portfolio risk assessments. These decisions must incorporate verified data, domain expertise and management judgment.

4. Verify specific claims and sources independently. Statistics, market growth rates, research citations, regulatory references and attributed quotations should be checked against reliable primary sources. If the origin of a fact cannot be verified, it should not appear in final materials.

5. Maintain accountability for the final output. Regardless of how extensively AI tools are used in drafting or analysis, responsibility for accuracy remains with the professionals and executives presenting the information. Technology may assist in preparation; accountability does not transfer to the tool.

There is good reason to believe hallucinations will diminish over time as model architectures improve and enterprise controls become more robust.

But equipment leasing leadership cannot wait for that future state. Today’s message is straightforward: GenAI is a powerful enabler of better work. Hallucinations are a known — and manageable — limitation. •

Valerie L. Gerard is co-chief executive officer of The Alta Group and leads the Strategy & Competitive Alignment practice. She brings deep leadership and advising experience to Alta. In her work with clients, Gerard helps companies design and implement value-creating solutions. She partners with leadership teams on both strategic and tactical issues from growth strategies and business model optimization to multi-vendor customer financing programs and long-term capitalization

Gerard has been actively involved in industry leadership and research initiatives throughout her career. She is a two-time trustee of the Equipment Leasing & Finance Foundation, member of the Executive Office, former chair of the Foundation’s Research Committee, Industry Future Council and on the editorial board of its Journal of Equipment Leasing and Finance. She is also a founding member of the Equipment Leasing and Finance Association Women’s Leadership Forum and has served as head of the Association’s Investor Relations Advisory Task Force. Her additional industry service includes membership on the American Financial Services Association Investor Relations Committee. She currently serves on the Research Committee of the Canadian Finance & Leasing Association.

Industry recognition includes the 2023 Steven R. LeBarron Award for Principled Research from the Equipment Leasing & Finance Foundation and being named one of the top women leaders in equipment finance by Monitor in its inaugural 2018 listing. She can be reached at vgerard@thealtagroup.com.