Scaling up and accelerating digital transformation using inherently SAFE Muon Tomography for NII to aid Customs and build a data rich capability and ecosystem

By Kevin Davies – Decision Sciences International Corporation (DSIC)

Where are we today?

In today’s modern world and the Global Supply chain on which we rely so heavily, we are living in an age of increasing and emerging threats –including radioactive materials being used as a weapon (dirty bomb or WMD) due to the increasingly porous borders and lack of controls around areas such as western Ukraine; and of course the ever present threats of explosives and biological pathogens and illicit narcotics being traded illegally and landing on our streets. With these things in mind and in the face of escalating cross-border smuggling, authorities need to rigorously monitor imports, exports, transit traffic and people for illicit goods and dangerous materials. They increasingly require more powerful and diverse toolsets exceeding the capabilities of conventional high energy X-Ray Non-Intrusive Inspection (NII) systems, to complement and facilitate a higher degree of assured detection for the items/cargo under inspection. In a strategically deployed layered security approach Muon Tomography can offer unique capabilities in this area, allowing truly safe NII, data driven, with unparalleled penetration. Its unique attributes to detect Shielded Nuclear Materials (SNM), added to the ability to effectively screen the densest of material and cargo, this can easily be seen in the comparative illustration later in the article.

Recap on the technology – Muon Tomography (Muography)
A naturally occurring harmless flux of muons are constantly raining down onto earth from the atmosphere 24 hours a day, 7 days a week. Known as “Primary Cosmic Rays” they shatter in the Earth’s atmosphere to produce a continuous flux of muons and electrons.

Primary cosmic rays are subatomic particles, composed mostly of protons (>90%) + and helium nuclei, accelerated to large energies by astrophysical objects. Cosmic rays “shatter” in the Earth’s atmosphere through collision events with atomic nuclei and produce a continuous flux of elementary charged particles known as muons and electrons. This harmless flux of muons is constantly raining down onto earth from the atmosphere 24 hours a day, 7 days a week, everywhere around the globe. On average at sea level over 10000 muons and 2000 electrons pass through your body every minute.

Muon Tomography harnesses the characteristics of these “Charged Particles” to produce 3-Dimensional imagery with material classification capabilities in a safe manner WITHOUT the need for radiation protection, as it uses a naturally occurring phenomena. One characteristic that facilitates the “data lake” able to be tapped into is the highly penetrative characteristic as shown in the illustration comparatively to the likes of High Energy X-Ray.

The science and the “magic” in this inspection technology is largely due to the unique AI / ML and algorithms that have been developed by the Physicists / Data Scientists and Engineers working in this field, and it is these we will explore in a little more detail within the article.

Advanced Technology Adoption for NII
The increasing adoption of software-based algorithms Machine Learning (ML) and Artificial Intelligence (AI) being injected into the “process flow” of inspections and particularly NII at a Port or a Border crossing, is leading to increasingly automated screening solutions. These solutions and systems not only advance the screening procedures and detection capabilities, allowing authorities to monitor for and identify complex threats, but reduce the operator burden, create higher throughput, and maintain system uptime to help keep both people and property safe, without the need for large numbers of highly skilled and very costly security personnel.

AI and ML facilitate our increasing ability to respond quickly to ever changing threats; they leverage new and developing technologies that produce detailed management information from an increasingly complex screening operation that in turn can provide infinitely more data. It is already commonplace to integrate certain technologies, camera-based and sensor-based, from different suppliers and vendors. Taking this further to incorporate Muon Tomography with its data rich outputs feeding ML algorithms is a natural progression to enhance the data models and software algorithms needed. We argue that it is a more flexible approach by authorities to include a SAFE and capable technology, such as Muon Tomography, that accelerate innovation with positive results. Giving much needed diverse screening options to Customs Authorities moving away from High-Energy X-Ray system technology and its limited data availability and inherent health and safety considerations.

What can we expect to see from Ports and Borders operation of NII in the future?

A Wider adoption of emerging technologies such as Muon Tomography imaging systems, these can offer unique capabilities complimenting existing and developing systems. Access to additional data allows concepts such as federated learning models, that protect data privacy, integrity and cyber security requirements, whilst providing that “stream of commerce” data needed by developers to ensure “fit for purpose” training datasets that increase an automated probability of detection for operators, with minimal false alarm rates. All of which advance the detection capabilities through ever more accurate and effective algorithms.

Interoperability between competing vendors and their systems across sites and country borders, increased collaboration between those established and new players on centralized platforms with greater adoption of OA and remote screening facilities. Conformance and adoption of Cyber security standards with adoption of federated techniques to give vendors unfettered access to the essential training datasets which ultimately will translate to higher probability of detection with less false positives. A standardization and inclusion of generic API’s for multiple asset access across multiple platforms with true data integration and facilitating interoperability across all systems.

With Software Architectures moving away from monolithic to containerized structures multiple solutions / approaches can be offered. These services can take advantage of AI / ML services and solutions using SaaS, PaaS models and can be provided On Premise, in the Cloud or a mixture of both, as you need it using and taking advantage of the latest in Edge Computing capabilities while truly maintaining Cyber Security.

AI / ML applied to Muography Imagery
Advanced detection capabilities enabled by AI and embedded in NII systems can and will further enhance existing security systems, particularly in a Ports and Borders environment AI) powered software algorithms using Deep Learning (DL) are at the heart of a Muography Imaging system. As trading volumes increase but the necessary resource availability for inspection is decrease, AI tools fill the void and increasing the probability of detection, ensuring anomalies are highlighted using tools such as VOI (Volume of Interest), a marking tool in a 3D environment like ROI (Region of Interest) commonly used in existing X-Ray inspection systems.

The Approach, how and what ML / DL techniques are used
In one approach, DL refers to the use of Convolutional Neural Networks (CNN) to generate predictions either per voxel or per scan, depending on the algorithm. To refresh understandings, CNNs are a class of artificial neural networks that currently achieve state-of-the-art results in some computer vision tasks, and are increasingly being used/relied on for NII system imagery, but not exclusively, as they are also in use in areas such as medical image classification and segmentation which has similar challenges.

The Muon Tomography imagery are 3-channel images, with channels (importantly) that utilize both Muon and Electron information in separate reconstructions, which are then concatenated. There are only a few commercial companies with applications and have successfully applied modern computer vision technology and deep learning to Muon-based reconstructions and incorporate these models into a product, and only one that has done this in a commercially available product! Generally, they will use multi-scale CNNs and supervised DL modelling approaches for the models developed so far. The models are trained on proprietary datasets, collected, labelled, and curated by the developer in a federated manner to ensure data integrity and facilitate cyber security requirements. This library includes models based on semantic segmentation, object detection, and image regression.

Image Segmentation
The cargo segmentation model assigns a cargo class label to each voxel in the reconstruction. It utilizes a four-layer UNET architecture with skip connections trained on patches of the reconstruction. The UNET architecture is an encoder-decoder often used for semantic segmentation. Additionally, the last layer is a SoftMax function with 4 classes. The loss function is used because it gives control over the false positives / false negatives for each class. The model achieves very good performance in a production environment and is easy to apply and maintain.

Image Acceleration
The image acceleration model has the same architecture as the Image segmentation model (UNET CNN with 4 layers and skip connections), but instead of a SoftMax last layer, there is a regression layer with a loss function. The model is also trained on patches of the reconstruction and uses a selected time restricted reconstruction as output. The purpose of the image enhancement model is to accelerate the generation of a longer time reconstruction, thus greatly reducing the time the cargo needs to spend in the detector. Muon Tomography in current systems relies on passive naturally occurring Muon / Electrons for reconstruction generation, and thus, the time required for a quality reconstruction is measured in minutes (≈ 3min). The model developed for image acceleration achieves very good performance in production.

Anomaly Detection / Region of Interest
The Region of Interest (RoI) problem is posed as an object detection problem. In contrast to semantic segmentation, which outputs predictions per voxel and the localization and categorization of objects needs several analytical steps, you can use a state-of-the-art single-stage detector. The architecture generally used is Retina U-net, which fuses the RetinaNet one-stage detector with UNET architecture, thus allowing full leverage of the full “per voxel” supervision signal. This is especially beneficial in smaller datasets. RetinaNet UNET provides strong detection performance, generally only achieved by two-stage detectors.

Costs
The initial acquisition cost of a Muon Tomography system for Customs NII purposes will be approximately twice the cost of a standard High Energy X-Ray system. I would suggest, however, that the important cost that should be considered is the Total Cost of Ownership (TCO) over the equipment lifetime. The Muography system has no moving parts with no radiation protection walls/building needed; its annual maintenance cost is minimal, with experience shown to be, on average, only about 30% that of a typical HE X-ray system. This significantly reduced annual cost easily translates to a considerable savings to the Customs administration. Mapping and considering the running and acquisition costs together, the conventional X-ray system and Muography system reach a comparable level of cost in year 5 of an average 15 year expected lifespan, with the Muography system thereafter costing significantly less in real terms year on year with all the advantages that come with it: No harmful radiation, 3D imagery and unrivalled penetration leading to an assured probability of detection.

Real world examples of the impact of AI / ML on Muon Tomography images
These exciting applications of Machine Learning in the field of Muon Tomography are not merely a futuristic possibility but are very real and available today. One simple example of Image Segmentation applied to Muon Tomography data is shown here, where the models have been trained to identify voxels filled with material vs those filled only with air, allowing the system to clean up signal smear and produce a much clearer image. The images show a side view of the rear portion of a cargo container and trailer. The 2 trailer wheels and axles are visible at the bottom of the images, directly above them is a pallet of gravel, and forward is a 3-ton block of granite. In the Raw Image, cloudiness is visible throughout the image surrounding the materials, an artifact of the reconstruction algorithms used to produce the image. In the segmented image the materials have been separated from the surrounding air and the signal smear is removed, producing a much sharper image for an operator to inspect.

A second application of ML in Muon Tomography is Image Acceleration, analogous to photographic age progression for scan images. In the first example of Accelerated Imaging, two pallets of gravel can be seen from a top view perspective. The lower pallet consists entirely of gravel, while the upper pallet contains a fentanyl surrogate embedded inside. The Raw Image displayed uses 60 seconds of Muon data, showing significant variation in signal resulting in a “noisy” appearance. With the application of Accelerated Imaging models, the signal variation is greatly reduced, providing the operator with an image of much higher quality and clarity.

A real-world example of Accelerated Imaging comes in the context of a seizure by U.S. Customs and Border Protection of more than 2 tons of marijuana at the Mariposa Port of Entry along the U.S. Mexico border (link to new article). The method of smuggling in this instance was to conceal drugs inside metal containers, weld them shut, then conceal these containers inside large rolls of sheet metal. This elaborate process produces large interior volumes for concealing contraband shrouded in metal, presenting a significant challenge in terms of penetration for x-ray systems to detect. As shown in the images to the right.

To the left is a Muon Tomography image of one of the steel rolls using 60 seconds of Muon data. Again, a “noisy” appearance is seen in the raw image, and the much cleaner Accelerated Image. The internal structure of the steel rolls is clearly visible to an operator as it is below on an image of the complete truck

Region of Interest / Volume of Interest is a broad application for AI and ML, as there are many reasons to consider a region “interesting.” A volume with an identifiable object within it, or other notable substructure indicative of an anomaly is one way a volume may be of interest. Another way a volume may be interesting is by being different than the others as might be the case of an anomalous pallet of contraband material being shipped alongside a container loaded with a particular commodity. If/when manifest data becomes available, this application will extend into manifest verification for commercial shipping. One exciting example application of Volumes of Interest is that of Human Detection. Humans produce a recognizable signature in the cosmic ray muon and electron data, and AI models can detect their presence within a cargo consignment. Left is shown an example of this capability. Three gentlemen are standing in various positions in a cargo container with pallets of material throughout. Along with the reconstructed image of the entire scene, the 3 figures are clearly shown in red, the voxel-by-voxel output of a Human Detection ML model.

Conclusions and Possible Impact
Muon tomography is an innovative technique that offers distinct advantages over traditional X-ray inspection methods. Muons are minimally interactive with matter, resulting in a SAFE technology for use as a NII toolset Additionally, the technology employed in muon tomography only relies on naturally occurring radiation and can gather data over extended periods, allowing for continuous monitoring and more comprehensive assessments with a larger dataset. The advancements in AI and ML are not just theoretical; they have been successfully demonstrated to have real-world applications in enhancing the capabilities of Muon Tomography imagery for NII.

By utilizing the continued use of DL models and advanced image processing techniques, these passive and SAFE systems can quickly and accurately identify anomalies in cargo, reducing false alarms and increasing the probability of detecting. As this technology continues to evolve, and with partnership access to elements of meta data from the authorities, it promises to play a critical role in safeguarding global trade and ensuring the safety of people and goods. Muon tomography is a powerful and data rich alternative for inspection, offering deeper penetration, 3-dimensional imagery, continuous monitoring capabilities, and potential significant cost advantages, particularly when it comes to the infrastructure required for high-energy X-ray machines, making it ideal for Customs use in NII applications in an environment where traditional X-ray methods may fall short.