AI hallucination in HS code classification: the risk

Why AI sounds so sure about HS codes even when it's wrong

We are working hard on exciting new things at Customaite! Some lessons I've learned in the last few months: AI technology can be genuinely great, but it can also be deceiving, and the difference comes down entirely to how you use it. Utilizing AI models in your business seems like a no-brainer, but be aware: there are real risks tied to this too, and AI hallucination in HS code classification is one of the clearest examples I've run into.

Why AI hallucination is a real risk in HS code classification

Take the case of HS code classification. HS codes are an internationally standardized 6-digit numerical system used by customs authorities to classify traded products. Asking an AI what HS codes are is an easy task, and one an AI model is perfectly capable of doing from memory. Asking an AI what the top 10 most used HS codes are, with example descriptions for goods, is something it cannot do from memory.

To be more precise, I asked an AI model: "Give me the top 10 most used 6-digit HS codes with example descriptions that you might find on an invoice." As a response I got: "Here are the top 10 used HS codes!", followed by a list where most codes only conformed to 4 digits (one of which was 851712, which it claimed was "LED lamps with ballast").

To actually answer this reliably, the model would need to consult an external, up-to-date source. The pitfall is that it will answer your question anyway, and do so very convincingly. That's the real risk of using AI models: they sound convincing every time you ask something, but they have no real grounding tied to that confidence.

How context engineering closes the gap

Knowing this, we at Customaite are fully aware of these risks, and with some expertise, we can luckily dodge these pitfalls. Giving AI the correct context to work with is key here. Letting the AI know what it is doing, what it is working with, and what it needs to accomplish in this.

The real challenge here is in setting the scene correctly. Giving no context at all simply isn't good enough to create a reliable, customs-compliant application. Customs context is ever so complex, but using the business knowledge we've built around these topics for 5+ years is key here. Suddenly the AI model is not wildly guessing about what is expected; it is guided with technological and business-sound logic.

For my example, to build the context I took the HS codes and the descriptions of those HS codes themselves, together with excerpts of legal texts, and gave it all to an AI model with the same prompt. This gave the model enough text to stop recalling details from memory and start using the ones I provided.

From guessing to guided logic

Going back to the initial problem: the fact that the model couldn't provide accurate HS codes is because only a weak link exists between the code itself and its description in the model's memory. It cannot recall these things reliably from memory alone. Instead of letting it guess details, I passed them along directly. By providing the model with rich context, it had no problem linking any of these. To actually get the top 10, it still needs to do a web search, but putting it all together, it can generate a genuinely high-quality result.

The problem shifts from "How can I get a good result out of the box?" to "How can I provide the correct anchor points so that it doesn't start guessing where I don't want it to guess?"

The takeaway: context is the hard part, not the AI

So, to summarize my thoughts: integrating AI models in a reliable and consistent manner is way harder than it looks. Collecting and crafting high-quality context around customs is key for creating a reliable, high-performing product, which is what our users expect from us. Being reliable also takes away the work load and mental pressure that would've been required from our users 10 years ago. This way of using AI lets us elevate our product to a next level because of the awareness this creates internally around these topics. Crafting that context is where the real challenge lies, given the complex nature of customs and all the rulings tied to it.

Curious how Customaite grounds AI in real customs data instead of letting it guess? Talk to us or see the platform in action.