An end-to-end production case study: Leveraging data-driven machine learning and autonomous process control in AM

“You are just too young.” If one were to imagine a discussion between a personified CNC milling machine and a multi-laser Laser Powder Bed Fusion (PBF-LB) machine, this sentence would eventually come up. The CNC machine would point out the obvious problems that the brand-new metal AM machine is having. This would be unfair, however; milling and other subtractive manufacturing technologies have benefitted from a slow but steady evolution over the last century or so.

Metal AM has some way to go to achieve broad recognition as a value-adding process for a finite but broad variety of applications. The quality, repeatability, reliability, and traceability of the process are not currently at the level of subtractive manufacturing. It is difficult to achieve scalability in production or repeatability during fleet expansions, for example when opening new production sites. While significant steps have been taken to bridge this gap, it is fair to say that, historically, they have been sporadic, with multiple changes in sensor technologies, non-standardised processes for quality, niche proprietary software solutions often targeting only single OEMs, and technology costs that are often too high to justify the investment.

If we followed the argument of the CNC machine in the example above, we would indeed need to wait for a greater level of technological maturity. However, we can and should continue to ask ourselves what are the key differentiators between additive and subtractive manufacturing. One could rightfully argue that metal AM is a ‘natively digital’ process, which comes with its advantages. But, in short, while metal AM is ‘too young,’ it also has – on paper – the potential to leap through decades of trial and error by optimally combining all the technologies that are currently available.

If done in a smart, relentlessly organised, and pragmatic way, we believe that the future can be steered to a brighter place. The use case that we present here is meant to show that the relatively optimistic timeline described in a recent Metal AM magazine article [1] can be further reduced.

To accelerate the mainstreaming of AM, we need to do something which is simple to describe but complex to achieve: establish an end-to-end traceability process (covering both the physical and digital world) to minimise in-field failures, increase repeatability in (decentralised) production and guarantee that liability is assumed by the correct responsible actors if issues arise. And, while we are there, let us do it cost-effectively. Fig. 2 shows the workflow and the technologies needed to establish it.

The necessary actors and an application case: Baker Hughes VRT component

The general process proposed can be summarised in a few steps of an AM production infinity loop (Fig. 3) where a similar concept to a product lifecycle has been used. We will describe every step and dive deeper into the core technological aspects related to process monitoring and part inspection with examples. We will follow the path that energy technology company (Baker Hughes) and manufacturing software provider (Oqton) are taking to achieve an end-to-end workflow for the serial production of the VRT component shown in Fig. 1.

This is a key component packaged in a valve body that provides a tortuous path configuration for a high-pressure drop liquid application. We will explore how Baker Hughes is working to maintain a high and repeatable production quality that its clients can easily certify. It also allows full history and traceability of the parts during the operating life of the component in case of malfunction or extraordinary maintenance.

The starting point is the assumption that one or more AM machines are installed at the manufacturer’s facilities and that a part design is available. Here are the steps that need to be followed afterward:

1 – Setting up a digital factory

The fundamental step of this process is to create a digital factory, or a digital replica of the shop floor. This digital twin needs to be integrated into the different enterprise systems so it works flawlessly from the order management perspective. This means that each AM machine gets a unique, digital version of itself containing essential information. Furthermore, each interacting member of an organisation is assigned a profile with the corresponding privileges and approval rights.

Materials, vendors, customers, processing, and post-processing activities are also defined, and parts and orders are submitted through various demand generation, order management, and data integration systems. This set-up step is so important that it could merit an entire article [2], but, for the current case, let’s just assume that we have a digital factory and that orders and parts can be created and scheduled in advance.

2 – Build preparation and first preventive actions via build simulation

Once the part design is available, we need to set up the build workflow with the corresponding data structure for the first article. This step is also part of the quality and traceability process. We select the materials, machine and all the hundreds of process parameters typical in metal AM, optimise part orientation, semi-automatically create support strategies, and perform simulation-based corrections. The simulation will reveal the potential risk for cracks, recoater crashes, unwanted shrink lines and overheating, and will suggest pre-deformation to achieve the desired dimensional tolerances. Once the build parameters, supports, orientation, etc. are optimised and satisfactory slicing is performed, build files can be created.

The VRT component built was optimised and simulated in 3DXpert, Oqton’s industrial Additive Manufacturing software. Fig. 4 shows the simulation results after a mechanical and thermal analysis.
At this stage, a connection to step 1 is created, each part is assigned a unique ID, and relevant files and metadata – including simulation summaries – are uploaded to the digital factory. A unique tie between the build prep information and the corresponding virtual part and order is established. Now we have both a physical and a digital factory and a virtual part with all the information needed for manufacturing. We can also evaluate the preventive work done to achieve the desired quality.

3 – Building the first articles and in-situ monitoring

The AM machine is instructed to build a component under the expert supervision of manufacturing engineers, workshop managers, and operators. The physical AM machines are securely connected to the digital factory using a gateway – an industrial computer with connectivity and computational capabilities (see next section).

The AM machine is streaming both IoT and camera image data to an Oqton gateway where the analyses are done in-situ and in real-time. This data analysis will reveal process deviations and operators will receive an alert in case the set-up thresholds are exceeded. Artificial Intelligence (AI) plays an important role in image processing for anomaly detection. Operators can be informed via email or directly on the cloud dashboard about possible anomalies. This AI-automated auditing process releases significant specialised talent from repetitive routines and tedious work, although if necessary they can also remotely monitor the growth or self-correction of anomalies– intervening when deemed necessary.

Fig. 5 shows the dashboard displaying the anomalies of an early test of the VRT while Fig. 6, left, shows an early prototype with anomalies caused by a non-optimised support structure.

Once the first article has been built, a very compact summary report is created and automatically linked to the virtual part in the digital factory.

4 – Analysis and inspection of the first item

Given the importance of the VRT component and the foreseen production amount, we need to analyse the anomalies results file that was generated after the build. The results can be exported and opened in 3DXpert for further inspection. The distribution of anomalies over the build – their size, criticality, location, and possible correlation with, for example, simulation data and scan path – are performed, giving unique insights into the part quality and possible root causes (see Fig. 6, right.)

If the results are satisfactory, we can continue the production process. If the results aren’t satisfactory, the component might be scrapped and the design re-optimised, or sent for further Non-Destructive Testing (e.g., CT scan). It is recommended to do a CT scan of the first article regardless, which, together with the IoT and image anomalies analysis, gives us a comprehensive picture. A complete dataset for quality and comparison of the production part is now available. The summary report together with the connected data can now be considered as a ‘golden set’ and made available in the digital factory.

5 – Serial production and possible de-localisation

Baker Hughes has multiple production facilities and may even use a qualified external vendor for serial production, decentralisation or localisation. For this reason, the company needs to be able to control the quality of its parts even when they’re produced outside of its facilities.

Production facilities receiving production orders for the VRTs will also receive the required quality specifications, and be required to comply with the monitoring process and traceability that is aligned to the level of criticality of the to-be-manufactured component. Therefore, if a contractor carries out the production, Baker Hughes could monitor the compliance on the required specification directly online and/or simply receive a report. Details of post-processing steps, including inspection criteria, should also be traced and should be part of the digital twin factory setup. With this, each part can be evaluated as ‘ready for deployment,’ ‘to be further inspected,’ or ‘to scrap’. Fig. 7 shows a summarised traceability report.

6 – Deployment of parts

Finally, the parts are deployed into operation. In some cases, the produced parts might need to last up to thirty years in operation. For critical components, the full set of information contained in the streamlined preparation and quality report constitutes the must-go-to documentation to be used in case of auditing requests and/or on-field issues in the future. When a problem arises, the engineering team will be in the best possible position and fastest route to analyse root causes, and quickly discriminate equipment faults, trace other components in other fleets at risk, and assess liabilities.

The technological prerequisites and blockers

Developing this end-to-end production integration came with certain technological challenges and requirements. We will analyse those challenges here with the intent to underline that if the devil is generally in the details – well, here the details are many, with all of the fine print typical to advanced technology. All stars and planets need to be meticulously brought into alignment to achieve successful end-to-end production.

1 – Best-in-class physics-based computational technologies

Physics-based simulation sounds like it’s difficult to master. The end-to-end approach aims to allow AM experts to use simulation without being computational mechanics or finite elements experts. To speed up this process the use of models that can be used with simple-to-understand parameters that are ‘good enough’ should be used. The target is a balance between good predictive capabilities, ease of use, and sheer computational speed. The end-to-end approach leverages an extremely fast GPU-native voxel FE solver with automatic meshing that focuses on the prediction of some anomalies, overheating and pre-deformation calculation. Fig. 8 shows a challenging simulation example with complex power-supports which can be run by non-experts when the correct technical choices are made.

2 – Best-in-class AI models for anomaly detection and aggregation

AI is not used here as a buzzword. Physics-based models are mostly deterministic and have great predictive capabilities, but AI models are good at detecting and bringing evidence to the fore based on real sensor information as input. In the end-to-end approach, we focused on branches of Machine Learning that are mature and industry-ready. We used image classification and segmentation with volumetric aggregation of topologically complex anomalies and validated it through a large development effort.

Once again, ‘the devil is in the detail,’ and here the details take the shape of the massive amount of data and images needed to train the AI models and the invaluable experience of partners like Baker Hughes and 3D Systems to steer the model in the right direction. Only cooperation between OEMs, AM experts, and software developers can have a chance of serious success. Fig. 9 shows one of the many validation examples for AI algorithm development.

3 – Cybersecurity

In the end-to-end example of Baker Hughes’s VRT, the connection that tied the process is the underlying digital thread. In our case, this is centralised in the cloud-based Oqton Manufacturing OS. It’s easy to assume that data can freely stream from OT to IT networks to general-purpose cloud platforms such as Google Cloud or AWS without anyone raising an eyebrow. The reality is of course different. Issues like IP protection, data leakage, external access to OT networks, etc. are paramount [3]. This is why Baker Hughes rigorously scrutinised the security of the solution through months of evaluation and cybersecurity tests.

4 – Dataset standardisation, sensor availability, and machine agnosticism

If metal AM is to become a production method of choice, large and small players alike need to be able to use different equipment from different vendors. Because of the competition in the AM machine market, multiple OEMs are highly specialised in specific features and are particularly advanced for certain applications but might perform poorly on others. Customers want to start benefitting from this wide range of options, but the lack of standardisation is hindering them. This applies to sensors, (IoT) data types and communication protocols, specific proprietary software, and the openness of the AM machine’s software ecosystem for usage of the valuable streamed data.

Baker Hughes and Oqton made a conscious choice at the start of this project to focus on what is practical and feasible and not on what is theoretically optimal. Baker Hughes had invested several years in evaluating various monitoring systems with specialised high-end sensors and high-resolution with various multiphysics principles and had concluded that although one could develop a specialised system it was not practical, not scalable at an industrial level from the implementation perspective and the small, aggregated value did not provide a return for the investment.

We therefore used sensor data that is common to basically every machine (i.e., oxygen levels, pressures, temperatures, etc.) and optical images from standard cameras looking at the build plate and taken before and after the recoating happens. By optimally using this minimal set of data we can cover Baker Hughes’s portfolio of large AM machines with a standardised set of information that can be used to compare and analyse part quality. The principle is that with this standard set of data, the overwhelming requirement is to have a high-end processing engine with robust AI and Machine Learning (ML) models with tens of thousands of trained images to generate the accuracy required to make concise operational decisions. The difference on this last approach is that once the models are trained and are machine and part agnostic then it is scalable and ready for an autonomous process, which enables AM at an industrial scale with a practical and economic solution.

This forms a strong basis to then further add sensors and information that the industry regards as important and avoids exotic and expensive sensors with uncertain added value.

5 – Data size, real-time performance, and edge computing

While the set of information is the minimum required for a thorough analysis of quality, it can still be very large. This information also needs to be digitally stored, backed and possibly maintained with variable retention policies. Certainly, it also needs to be processed at a time that, for the metal AM application, can be considered real-time. In a matter of a few seconds – say up to twenty seconds – images of single layers need to be processed and run through the AI model, anomalies need to be aggregated considering all possible topological variations during the build, and IoT data need to be aggregated, synced, and eventually, streamed to a cloud tenant for visualisation and fruition. It goes without saying that ‘every byte counts.’ Advanced numerics need to happen on so-called edge devices or gateways – small industrial PCs with enough connectivity and computing power to ingest and elaborate data locally and relay only the relevant information.

In our end-to-end application, Oqton’s gateways played a crucial role. Even if the cloud solution is not used, one still needs to create databases that are indexable and searchable through the years: storing and retrieving petabytes or more of data is not a viable option.

6 – Ease of use

AM engineers, operators and shopfloor managers need to be allowed to do their job and not waste time on the intricacies of the underlying technologies. This end-to-end solution was developed with the aid of a strong team of UX designers focusing on ease of use for the problem at hand.

In short, all the above details need to be aligned for such a mature solution to be effectively developed, tested, and deployed – and for the end-user to benefit from it. This end-to-end example shows that multiple players were needed to make this happen: Baker Hughes pushed the boundary to embrace a new fully digital paradigm, AM machine OEMs, such as 3D Systems and SLM Solutions, provided machine know-how and connectivity support, while Oqton provided the software. It is extremely important to understand that developing AM machines, developing software, and producing AM products are three very different businesses and each player needs to be optimally positioned to excel in their own skills. Anything else would be suboptimal.

An extended set of possibilities and applications

While the end-to-end workflow was showcased in the VRT use case, there are multiple benefits that such a complete solution can provide to other applications. Some of the most prominent are summarised here:

Where and when did the problem occur?

The above workflow was recently used to show that a crack on a component did not appear during the build process, but rather during a post-processing step. This saved a huge amount of time and investigations on build preparation modifications.

Is it all scrap?

When a localised anomaly occurs, it can affect multiple parts. For example, a re-coater line can be caused by a small, localised deformation. The lack of powder on a straight line across the build plate can negatively affect other parts. Visual inspection after the build will only identify visible anomalies, but the part needs to be sent to CT or be scrapped if an internal wall is affected. The anomaly detection system allows us to partially recover large trays with multiple components if we discover that some of the parts are not affected by a critical anomaly.

Does the cause of part failure lie in the manufacturing process?

When one of multiple parts starts failing after a long time in service, how can we show that the manufacturing process is not the root cause? We trace every step of the production process and provide the information in an easy-to-read and easy-to-access report. From this, one can easily discard or further investigate slight deviations or potential risks highlighted during production. This reassures end-users and helps to properly assign liability. Comparison with similar parts or different instances of the same parts is a simple and effective way to eliminate doubts.

Act before it’s too late

Today, some very large parts are being produced. Their cost is often very high and the machine and operator time expensive. By stopping a build early at the onset of an anomaly one saves time, material, and operator effort.

We can give many other use case examples, but this initial list of benefits which became apparent in the first deployments of the solution is a good summary of what to expect.

A bright future lies ahead

We strongly believe that this standardised end-to-end approach can bring clarity and significant added value in the streamlining of metal AM part development and production. It is a first step in helping the process to scale to the industrial level of other mature manufacturing processes. This is just the beginning – but it is a very concrete and solid step, not just a promise of something to come in the coming years. It is ‘here and now’ and we can build from it and keep enhancing the process.

Of course, multiple steps need to be taken for the wider proliferation of such an approach. For this reason and to increase the end-users’ trust, the solution provided is modular and one can opt-in for just some of the features. The benefits of the full workflow will become apparent along the way. Technology development in AI, physics-based simulation, and combinations of the two with the advancements in cybersecurity and edge computing will pave the way for more useful extensions starting from a strong base. While sources such as [1] suggest a five or more years estimate for the technology to mature, we hope that this use case shows that the future is closer than expected. If we look a bit closer, it’s happening now.

Authors

Tommaso Tamarozzi
Product Director Additive Monitoring, Inspection and Simulation
Oqton
www.oqton.com

Juan Carlos Flores
Executive Director, Additive Manufacturing
Baker Hughes
www.bakerhughes.com

References

[1] Atela, F. L., & Godfrey, D. (2023, Vol 9 No 4 Winter 2023). Mission possible: The five-year plan to gain FAA and EASA acceptance of in-process monitoring. Metal AM, pp. 147-152.

[2] Buttle, G. (2024, January 19). What is a Manufacturing Operating System? Retrieved from Oqton.com: https://oqton.com/posts/whats-the-difference-between-a-manufacturing-os-and-an-mes/

[3] Yampolskiy, M., & Kowen, J. (2023, Vol 9 No 4 Winter 2023). Cybersecurity in Additive Manufacturing: Securing the industry’s future. Metal AM, pp. 129-137.