Advances in Artificial Intelligence (AI) have reinvigorated the imagination of adopting AI for quality engineering applications, especially for process monitoring. Existing process monitoring methods are often based on statistical methods applied to limited scopes such at the end of a process. Existing modeling framework is not built to handle the entire process. Possible correlations of upstream and downstream processes are ignored because of modeling complexity. The current process monitoring practice is isolated. The opportunities of monitoring the entire process and providing mitigation strategies are often missed.
Shing Chang, a quality engineering research and educator, has explored the possible use of generative AI modeling for holistic process monitoring and control in two projects. The first project is the monitoring of scams targeting older adults. The second project is related to a hydroponic application.
Project 1
Empowering Older Adults to Use Smart Assistive Technology for Combating Financial Exploitation
In this NSF funded project, a smart assistive technology (SAT) is proposed to empower older adults to combat financial exploitation. Through an ensemble of data-driven algorithms using generative AI technologies and data monitoring, SAT can proactively provide feedback to its users or their caregivers about warning signs of fraud-scam susceptibility. The proposed project integrates social phycological factors in SAT design. In return, SAT serves as a conduit for research teams to study various social aspects of smart technology, such as ergonomics, privacy concerns, loneliness, and fraud-scam susceptibility. This project will advance knowledge and understanding in intelligent information technology and aging in order to guide smart in-home product designers to create effective and innovative solutions to mitigate financial exploitation among older adults. This integrative research project will take place in five counties representing rural and urban areas in Kansas and impacting more than half a million adults aged 60 and older.
The proposed SAT takes conversations between a scammer and an older adult as inputs. In other words, the quality characteristic is a pair of dialogs in either text or voice format. Voice can easily be converted into text format. The conventional quality characteristics are usually a number, a numerical vector, a profile, or an image but, in this case, it is a dialog. The proposed Large Multimodal Model (LMM) framework must be portable for mobile device usage and address key technical challenges: training with limited training data and training sources, reducing model size without sacrificing performance, achieving real-time processing with acceptable latency, and providing timely model updates for evolving scams. Overcoming these challenges will expand knowledge of multimodal model development for AI-driven fraud detection in real time.
This project is transformative in integrating social science research into the proposed SAT system, customizing it to individual social-emotional experiences and socio-demographic factors (e.g., trust in and competency with technology, marital status, and occupation, such as urban retired white-collar workers vs. rural farmers). Additionally, SAT incorporates multiple forms of communication, including email solicitations, social media activities, and phone conversations, to combat financial fraud.
Project 2
Agri-Generative Digital Twin Integrating AI, Plant Sciences, Water Management, Social-economic Studies, and Urban Farming Communities
Climate change and water scarcity are increasingly impairing food production and threatening global food security. Farmers are grappling with extreme weather conditions like droughts, intense rainfall, floods, and severe derechos, which disrupt their livelihoods. Meanwhile, consumers, especially those in inner cities, face reduced access to fresh produce, leading to nutrient deficiencies and health impacts.
To address these challenges, urban food systems such as vertical farming (VF) and high-tunnel (HT) systems offer promising solutions. Both VF and HT methods protect crops from extreme weather, conserve water, and reduce food mileage, creating a more resilient food supply chain. This project proposes harnessing the power of artificial intelligence (AI) to optimize and support the VF and HT operations of both new and experienced urban and crop farmers. By integrating AI into VF and HT, this project aims to enhance the sustainability and resilience of urban farming, contributing to a stronger, more adaptive food system.
At the heart of the proposed infrastructure is a knowledge hub designed to curate and advance knowledge on urban farming systems, including vertical farming (VF) and high tunnel (HT). The hub will gather data from sensors and cameras, monitoring variables like lighting, temperature, water/soil nutrient levels, and root and aerial crop growth images across various VF and HT operations. This data will be hosted on the Agri-Generative Digital Twin (AGDT) cloud platform, where it will be used to train foundational AI models. The AGDT architecture will integrate a large language model (LLM), crop imaging data, field sensor inputs, and outputs from predictive AI models to build a powerful multimodal model (LMM) customized to each farming operation. As more users connect their VF and HT systems to the cloud platform, the AGDT framework will address a key challenge in AI development by amassing large, diverse datasets necessary for robust model training. This data and the resulting AI models will support research, education, and workforce development initiatives, benefiting urban food producers and related communities. Farmers will also be able to provide feedback in natural language, enabling the platform to evolve based on practical insights. Economic and social impact studies will assess the feasibility and effectiveness of smart farming practices for urban food systems.
The underlying digital twin in the proposed AGDT is a mathematical representation of the complex interactions between a crop and its growing environment at the physiology level, which provides domain knowledge support to the LMM for agricultural stakeholders. AGDT consists of several main components: (1) data storage (database & file system): CEA parameter data and plant images, (2) predictive and prescriptive AI models, (3) generative AI models, and (4) user interface (aka AI assistant). The concept of digital twins derives from manufacturing operations where data generated from sensors scattered throughout a factory is collected in a cloud platform. A virtual or digital copy via sensor reading replicates physical factory operations. In other words, this digital replicate is the digital twin of the physical factory. A digital twin allows simulated changes in the virtual factory without interrupting production. The proposed AGDT adopts this concept with the application for VF or HT operations. Many VF or HT operations may reside in AGDT to share data for AI models for common use. The individual data stream from each VF or HT system through the trained data provides customized solutions. An analog of the foundation model is a self-driving AI model trained on driving data from millions of cars no matter what city the cars are in. The proposed LMM approach advances the self-driving example one more step in allowing users to provide inputs (e.g., customized routes), ask questions, and provide feedback verbally.