Artificial Intelligence. It can do many things: understand whalesong, compose film scripts on-demand, and decrease the fuel consumption of a ship at sea. However, “AI” has also quickly become the greatest buzzword of the 21st century – wooing customers, investors and governments alike.

It’s vital that serious researchers working to popularise this exciting approach continue to pursue rigorous methods of proving the real value of what they’re creating. In fact, we believe every end-user looking to employ this sort of service, in shipping or any industry, should demand it.

No-bullshit AI

We have pioneered a way of verifying the accuracy – and therefore utility – of a ship’s AI-generated model in real-world conditions. This is important – the more accurate the virtual model, the more efficient a ship can be made, and vice-versa.

The new approach was developed by seven of our thirteen-strong team of research scientists, and first published in May 2022.

The few models that currently provide an estimation of their accuracy all do so based on testing with data obtained from the same distribution (i.e. representative of similar conditions and containing similar biases) as the data used to train the model. For example, if the model is trained on data from the vessel’s historical behaviour, in a narrow range of well-experienced wind speeds or drafts, it is also tested on data with these speeds and drafts. Thus, the tests performed can’t tell if the model is reproducing the biases in the training data – and whether it will work as well in different, never-seen-before conditions. As anyone familiar with maritime data will know, real ship-at-sea data is actually highly variable. Most model accuracy figures reported in publications and marketing materials thus bear no relation to the actual utility of those models in real use cases.

DeepSea has long researched approaches to solving the technical challenge of boosting models’ ability to understand unseen (“out-of-domain”) conditions. However, before this recent publication, there has been no benchmark for evaluating this sort of competence within a vessel model. With this announcement, we are signalling that this rigorous test is a key part of our AI methodology. Moreover, we are releasing the details of the approach for global researchers to utilise themselves, in the hope of catalysing greater transparency across the industry.