LLMs, Generative AI & Design for Manufacturing
Challenges and Opportunities in 3D Modeling and Design Processes
The recent emergence of readily accessible Large Language Models (LLMs) like GPT-4 has ignited a blaze of excitement and breathless speculation.
The online sphere is abuzz with their seemingly boundless potential, painting a rosy picture of a future transformed by AI-driven design.
Yet, as we delve deeper into the practical applications of LLMs, especially in precision-demanding fields like engineering, a more nuanced narrative emerges.
It's a tale that balances the dazzle of innovation with the sobering real-world challenges of gathering accurate data and the physics of manufacturing.
A recent paper titled "How Can Large Language Models Help Humans in Design And Manufacturing?" was authored by a collaborative team of researchers from multiple institutions including MIT, Harvard and University of Washington to explore what the capability is today (published July 25, 2023) of using LLMs for design and manufacturing data, and the current issues that need to be overcome to make LLMs a viable design and manufacturing assistant, eventually.
The primary contributors include Liane Makatura, Michael Foshey, Bohan Wang, Pingchuan Ma, Bolei Deng, Megan Tjandrasuwita, Crystal Elaine Owens, Peter Yichen Chen, Allan Zhao, Wil J Norton, Edward Gu, Joshua Jacob, and Yifei Li, all affiliated with MIT, USA. Additionally, Felix Hähnlein and Amy Zhu from the University of Washington, USA, and Andrew Spielberg from Harvard University, USA, have also made contributions to the research.
The paper investigates the application of Large Language Models (LLMs), specifically GPT-4, in design and manufacturing processes. The research focuses on LLMs' capabilities in converting text prompts into design specifications, generating manufacturing instructions, producing design variations, computing design performance metrics, and conducting design searches based on performance criteria. Through empirical examples, the authors delineate the operational strengths and weaknesses of LLMs in these areas.
Challenges encountered with LLMs encompass reasoning discrepancies, such as GPT-4's ambiguity in distinguishing between design parameters and performance metrics. In 3D modeling tasks, GPT-4 demonstrated difficulties in accurately positioning primitives in 3D space, particularly with tools like OpenJSCAD. Additionally, there were inconsistencies in generating Featurescript code for sketch-based CAD modeling. However, the model showed adaptability when provided with user feedback, adjusting its outputs accordingly.
Exploring 3D Modeling with GPT-4 LLM.
The researchers explored a number of pure 3D modeling experiments using GPT-4, following are some key findings for each.
OpenJSCAD Modeling:
Benefits: GPT-4 was able to use the OpenJSCAD library without any additional user explanation or restriction of the API. This allowed for the direct construction and modification of solid primitives based on user input.
The task was to design a cabinet using OpenJSCAD. The cabinet was to be constructed with 3/4" wood sheets and have exterior dimensions of 30 inches tall, 20 inches wide, and 18 inches deep. The stationary part of the cabinet was to consist of six boards: bottom, top, back, two sides, and one shelf centered inside the cabinet.
Issues: The initial design had the shelf oriented vertically rather than horizontally. Additionally, there were instances where the boards protruded into one another. Corrections were made in subsequent iterations to address these issues.
GPT-4 faced challenges in positioning primitives in 3D space. It often forgot that OpenJSCAD positions elements relative to the center of a given primitive rather than an external point, leading to incorrect arrangements. While GPT-4 could adjust designs when reminded of this convention, it wasn't always successful in rectifying the issue.
Sketch-based CAD with OnShape:
Benefits: GPT-4 was introduced to a simplified DSL (Domain Specific Language) to generate models using Onshape's Featurescript. This approach allowed for the creation of 3D models based on planar sketches that are lifted into 3D and subsequently modified.
The Task was to design a chair using OnShape's Featurescript. The chair was to have 4 legs, a rectangular seat, and a rectangular back.
Design Variations:
Legs: The design sometimes included cylindrical legs, while other times it had rectangular legs.
Construction Direction: The design was consistently constructed in a single direction. Even though the provided language description documented the use of other plane directions, GPT-4 predominantly used the one direction for sketching planes.
Design Types: Two main types of designs were observed:
Designs constructed in both the negative and positive direction starting from the seat. These designs had a higher likelihood of being correct.
Designs that began from a leg. These designs often faced issues, especially when changes in the coordinate system were introduced. For instance, when selecting the top plane of the first leg as a sketch plane for the seat, discrepancies arose in the sketch.
Issues: GPT-4 struggled with generating Featurescript code directly due to the lack of documentation and examples. The model sometimes produced inconsistent designs, such as chairs with either cylindrical or rectangular legs. The reliance on GUI in CAD systems made it challenging for GPT-4 to generate accurate textual representations.
Design Interpolation with CAD DSL:
Benefits: GPT-4 was tested for its ability to interpolate between two designs, allowing for the creation of new designs inspired by existing ones. This approach can be beneficial when designers want to merge features from two different designs.
Chair Interpolation:
Input Designs:
A chair with cylindrical legs.
A chair with a backrest featuring splats.
Task: Mix the two designs to create a chair with cylindrical legs and splats in the back.
Result: GPT-4 successfully combined the designs, producing a chair with both cylindrical legs and splats in the backrest. The result can be seen in Fig. 27 (c). The design process was facilitated by descriptive variable names in the code, such as
leg4_solidandsplat_3_sketch.
Temple Design Interpolation:
Input Designs:
A temple with 4 pillars on each side.
A temple with 10 pillars on each side.
Task: Design a temple with steps, a roof, and 6 pillars on each side.
Result: GPT-4 managed to extract the construction logic of the pillars from the input designs and introduced variables and a looping structure to place them correctly. The resulting design had steps, a roof, and 6 pillars on each side, as visualized in Fig. 28 (c). However, without mentioning the steps and roof in the prompt, GPT-4 would often focus solely on the pillars.
Bike Design Interpolation:
Input Designs:
A bicycle design.
A quad-bike design.
Task: Design a tricycle.
Result: GPT-4 correctly reasoned about the number of wheels for the tricycle and adjusted the size of the quad-bike’s vertical bar to ensure all wheels were on the same plane. However, it struggled with extracting the complete fork design from the bicycle, resulting in an incomplete tricycle design as seen in Fig. 29 (c).
Issues: Interpolating two designs posed challenges such as matching dimensions between sub-designs, combining operations, and extracting subparts from a design. Accurately referring to two sub-designs in a text prompt was also a challenge.
These interactions provide a promising basis for interactive user control of the design, but the process is somewhat tedious at the moment, as GPT-4 requires very explicit instructions about the design or correction intent. The addition of highly-detailed user constraints also seems to confuse GPT-4 to an extent, as it seems to “forget” the larger context of the design in the process, so it must be frequently reminded.
Generation of Manufacturing and Assembly Data.
Three examples of attempts at generating manufacturing data and assembly instructions using Large Language Models (LLMs) like GPT-4, along with the benefits and issues associated with each:
Additive Design for 3D Printing:
Benefits: GPT-4 has the potential to improve the additive design process, which is often time-consuming and requires spatial reasoning. The model can comprehend complex specifications in natural language, generate designs efficiently, simulate outcomes, and explore innovative possibilities.
Issues: Directly using GPT-4 to generate G-code from a natural language description was challenging due to the complexity and length of G-code. GPT-4 failed to output complete code that precisely modeled the specified shape. A two-stage approach was developed to overcome this, transforming the concept into an intermediate 3D shape representation and then translating this into G-Code.
Optimal Manufacturing Process Selection:
Benefits: GPT-4 can be used to select optimal manufacturing techniques based on a part’s features. It can propose and implement modifications to a design to improve its manufacturability, leading to more efficient production processes.
Issues: While GPT-4 was successful in selecting an optimal manufacturing process for most cases, there were instances where it suggested inappropriate processes. For example, GPT-4 suggested an injection molding process for a part that was not suitable for such a process.
Manufacturing and Assembly Instructions Creation:
Benefits: GPT-4 has the potential to develop manufacturing and assembly instructions for various processes, streamlining the transition from design to production.
Issues: The exact capabilities and limitations of GPT-4 in this area were not detailed in the provided excerpts, but it's implied that the model's outputs would need verification and refinement to ensure accuracy and applicability in real-world manufacturing scenarios.
The design spaces created by GPT-4 are based both on semantic knowledge and on code interpretation. However, it does not take into account geometric considerations, such as intersecting or non-connecting parts. As a result, generated parameter bounds can create non-valid geometry and it has proven difficult to make GPT-4 correct these. However, in general the generation of valid parameter bounds and constraints is a difficult problem for which mainly approximations have been proposed
Simulation with LLMs
Quadcopter State-Space Matrices for LQR Control:
Task: Derive the A and B matrices for the LQR controller by linearizing the dynamics of the quadcopter around a fixed point. The system was assumed to have 12 states and 6 degrees of freedom. The control inputs should be total thrust and x,y,z torques.
Approach: The researchers used symbolic computation with the
sympylibrary to define the state variables, control inputs, and constants. They then formulated the system dynamics and computed the Jacobians.Issues: While GPT-4 formulated the problem correctly and used the right kinematics equations and rotation matrix, there were errors in the initial derivation of the state-space matrices. For instance, GPT-4 indexed the wrong elements of the rotation matrix.
Chair Design Strategies:
Task: Simulate the design of a chair with 4 legs, a rectangular back, and a rectangular backrest.
Approach: GPT-4 was tasked with simulating the design of a chair in two distinct strategies. In the first strategy, the simulation began with the seat as the starting point. In the second strategy, the simulation initiated with one of the chair legs. The simulation process involved creating sketches of the chair components and extruding them to achieve the desired 3D shapes.
Issues: The simulation faced challenges in coordinate system management. In the first strategy, where the seat was the starting point, GPT-4 managed the coordinate systems effectively, ensuring the legs' sketch coordinates aligned correctly with the seat. However, in the second strategy, where a chair leg was the starting point, GPT-4 encountered difficulties. When attempting to sketch the seat on the top face of one of the legs, GPT-4 failed to correctly change the coordinate system. This oversight affected the accuracy and feasibility of the simulated design.
Sourcing and Assembly of ‘Off the Shelf’ Parts.
The researchers also investigated using GPT to source off the shelf parts to solve a given problem.
Bracket Design for Specific Scenarios:
Task: The user sought a bracket that could be used in a tight space without room for a wrench and would be submerged in saltwater.
Results: GPT-4 recommended "316 Stainless Steel Wing Nuts" and "316 Stainless Steel Knurled-Head Thumb Nuts".
Issues: While GPT-4 identified the correct form and material for the bracket, the exact compatibility and fit for the specific scenario would still need to be verified.
Quadcopter Parts Sourcing:
Task: The user asked GPT-4 for a parts list that would encompass everything needed for a quadcopter design.
Results: GPT-4 compiled a comprehensive list that included batteries, frames, propellers, transmitters and receivers, electronic speed controllers, and more. The list was found to be comprehensive and accurate.
Issues: When pushed beyond specification, GPT-4 produced lists with incompatible parts or named parts that did not exist. Although GPT-4 was able to provide domain-specific knowledge and complete parts lists, the exact specifications and compatibility of the parts needed verification.
At-Home Carbonation System:
Task: The user sought assistance in finding parts for an at-home carbonation system, specifically requesting hose, fittings, and a pressure gauge suitable for a jar with a 5-gallon capacity.
Results: GPT-4 provided a comprehensive list of parts, including regulators, pressure relief valves, tubing, adapters, bulkhead fittings, and diffusion stones. The parts were sourced from McMaster-Carr and homebrew stores.
Issues: While GPT-4's suggestions were geometrically compatible, the exact part numbers provided were not always accurate. For instance, all part numbers were incorrect, except for the pressure relief valve. The BOM (Bill of Materials) provided by GPT-4 also omitted smaller items such as clamps or the CO2 tank, and optional or safety items like check valves or non-return valves.
The Entire Design and Manufacturing Process.
The research paper presented a detailed exploration of GPT-4's involvement in the entire design process of a wood cabinet. This encompassed stages from 3D design generation, where GPT-4 assisted in creating and modifying cabinet models, to part sourcing, where it recommended specific components like brackets. The model also potentially aided in providing manufacturing instructions, assembly guidance, and performance evaluations. While GPT-4 showcased its potential in assisting with these tasks, the results also highlighted challenges, such as inaccuracies in design modifications and part recommendations, underscoring the need for human intervention and verification in the design and manufacturing process.
Paper Conclusions:
Capabilities and Limitations: The research underscored GPT-4's significant capabilities in supporting design processes, particularly its extensive knowledge base in design and manufacturing. The model demonstrated proficiency in solving a wide array of problems and automatically completing specifications. However, the study also pinpointed certain limitations in GPT-4's performance.
Design Specific Language (DSL) Insights: Historically, DSLs have been developed with human users in mind. The research suggests that when creating DSLs for an AI coder like GPT-4, new perspectives and considerations emerge. The paper hints at the potential for new insights into DSL design, especially within the design and manufacturing domain.
API Usage and Framework Development: The research observed a myriad of possibilities in terms of API usage and framework development. Approaches ranged from dividing problems into parts for GPT-4 to tackle, to iterative solutions, and even reframing problems to ask GPT-4 to generate problem-solving code. Each approach has its potential advantages and disadvantages.
Recommendations:
Optimal Framework Development: The researchers suggest that there's a need to determine what an optimal framework would look like, especially when integrating GPT-4 into design and manufacturing processes. They hope their research will aid others in formulating answers to this question.
Harnessing LLM Potential: The paper concludes that while substantial strides have been made in understanding how Large Language Models like GPT-4 can be utilized in design and manufacturing, there remains a rich avenue of opportunities for further exploration and innovation.
The rapid dissemination of information often leads to a whirlwind of excitement and anticipation around emerging technologies. Large Language Models (LLMs) like GPT-4 are no exception, with the online community buzzing with their potential applications and transformative capabilities.
The allure of a machine that can seemingly understand and generate human-like text is undeniably captivating, and the initial demonstrations often border on the miraculous.
However, as with many technological advancements, there's a chasm between the hype and the grounded reality, especially in fields that demand precision, like engineering. The research on LLMs in design and manufacturing processes reveals that while these models can indeed assist and augment certain tasks, they are not a panacea. The intricacies of engineering solutions require not just the generation of correct information, but also the synthesis of this information in a contextually relevant and applicable manner. LLMs, in their current state, can falter in the face of complex, multi-faceted problems, necessitating human oversight, verification, and iterative refinement.
In conclusion, while LLMs represent a monumental leap in natural language processing and have undeniable potential in various domains, their application in engineering underscores a vital lesson: technology, no matter how advanced, must be approached with a balanced perspective, discerning the line between its genuine capabilities and the surrounding hype. As we move forward, it's imperative to harness the strengths of LLMs judiciously, always grounding their use in the rigorous standards and requirements of the engineering world.
