Our Students

haihui-yan-180x180.jpg

Research Interests Mechanism design of robotic systems, vision and perception systems.

PhD Project: 3D Printing Soft Robotic Grippers for Automated Strawberry Harvesting

In 2019, strawberries were the second most produced fruit in England dominating ~21% (141.6 thousand tons) of the market, but achieved the highest fruit value of 2.46 million pounds/thousand ton. However, strawberries are commonly harvested by hand, which is a very labour-intensive job. Moreover, there has been a consistent decline in the number of available pickers, in the autumn of 2019, UK farmers reported a 30 percent shortage in pickers. Therefore, there is an urgent need for a highly effective and smart or human-like strawberry harvesting design to meet this gap. The main challenges to strawberry robotic harvesting are bruising, abrasions and other mechanical damages of strawberries. To address these challenges a new solution is proposed, which involves the design of a gripper structure, allowing highly efficient harvesting of strawberries with no mechanical damages. General rigid robotic grippers have been designed with pneumatic and hydraulic gripper solutions. However, it is often difficult to control the gripper precisely as they are rigid and non-compliant. Soft robotics offers more compliancy and flexibility. They are also 3D printable with more possible solutions.

3D printing of soft robotics has been extensively explored recently. These include a range of soft materials including Liquid crystal elastomers (LCE). They have a lot of potential to overcome current grippers’ problems. They are more flexible, lightweight and they are 3D printable. These characteristics maximise the potential these materials to produce a flexible and compliant gripper suitable for bruise-free strawberry harvesting.

This project will involve the design of a novel 3D printed robotic gripper targeted for strawberry harvesting. This will involve producing a specification for strawberry harvesting and using it to produce a design.

PhD Project: Collaborative Lifelong Learning for Robust Site-Specific Crop Management

Farms are not, in general, homogeneous. As such, the farm-wide treatment and maintenance of crops leads to sub-optimal crop yield or quality. By taking a finer grained approach, crop would receive the necessary care for their needs, rather than the average need of the farm. However, it is impractical for human experts to manually analyse the needs of crop on such a granular scale.

To address this need, we first propose novel machine learning approaches for crop care, such as soil-moisture optimisation via LSTM neural networks. To collect real-time data of the farm a custom sensor system will be developed and several of them deployed to collect relevant environmental data from a small region of the farm.

Finally, to improve the long-term autonomy and overall performance of an agent, a collaborative multi-agent system will be constructed to facilitate the lifelong learning from both an agent’s environment, but also from other agents. This will improve the robustness and performance of agents over long periods of time.

Publications:

• Foster, J., Schoepf, S., & Brintrup, A. (2024). Fast Machine Unlearning without Retraining through Selective Synaptic Dampening. Proceedings of the AAAI Conference on Artificial Intelligence, 38(11), 12043-12051. https://doi.org/10.1609/aaai.v38i11.29092
• Foster, J., Fogarty, K., Schoepf, S., Öztireli, C., & Brintrup, A. (2024). Zero-shot machine unlearning at scale via lipschitz regularization. arXiv preprint arXiv:2402.01401.
• Schoepf, S., Foster, J., & Brintrup, A. (2024). Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening. arXiv preprint arXiv:2402.10098.
• Schoepf, S., Foster, J., & Brintrup, A. (2024). Potion: Towards Poison Unlearning. arXiv preprint arXiv:2406.09173.
• Foster, J., Brintrup, A. Aiding food security and sustainability efforts through graph neural network-based consumer food ingredient detection and substitution. Sci Rep 13, 18809 (2023). https://doi.org/10.1038/s41598-023-44859-0
• Foster, J., Gudelis, M., & Esfahani, A. G. (2022). Robotic Perception in Agri-food Manipulation: A Review. arXiv preprint arXiv:2208.10580.
• Foster, J., & Brintrup, A. (2023). Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization. arXiv preprint arXiv:2309.08546.

bethan-moncur-180x180.jpg

Research Interests Food manufacturing, strategic technology management, and industrial sustainability.

PhD Project: Deciding to implement emerging technologies: the help of digital technologies in planning for the implementation of robotics and autonomous systems in food manufacturing firms

Novel technologies are radically changing the way that food is produced, processed, and traded. However, successfully implementing emerging technologies is an important yet challenging task for organisations: they operate within complex, uncertain, dynamic environments where they must make decisions based on incomplete information. Therefore, companies often used tools and techniques developed by technology management researchers (such as roadmapping, scenario planning and scoring methods) to reconcile opportunities and risks associated with implementing emerging technologies. 

Digital technologies can be used to alter the interface through which information is presented to decision makers, thus influencing decision-making behaviour and affecting the decision outcome. The aim of the proposed research is to incorporate AR and VR into technology management decision support tools and study the effects. The focus will be on facilitating decision-making in the context of implementing emerging technologies, such as robotics and autonomous systems, that are necessary for the transition to a more sustainable food system. The insights gained will be used to develop a decision-making tool to test in the agri-food industry. 

Publications:

• Moncur, B., Galvez Trigo, M.J., Mortara, L. (2023). Augmented Reality to Reduce Cognitive Load in Operational Decision-Making. In: Schmorrow, D.D., Fidopiastis, C.M. (eds) Augmented Cognition. HCII 2023. Lecture Notes in Computer Science(), vol 14019. Springer, Cham. https://doi.org/10.1007/978-3-031-35017-7_21
•  Leeb, C., Mortara, L., Felicini, N., Phaal, R., & Moncur, B. (2023). (Best) Practices in the Integration of Social and Digital Decision - Making Approaches Across Industries. Institute for Manufacturing. https://doi.org/10.17863/CAM.100091
• Moncur, Bethan; Clawson, Garry; Bennett, James; Fogarty, Kyle; Fox, Charles (2022). Towards Open Source Hardware Robotic Woodwind: an Internal Duct Flute Player. University of Lincoln. Conference contribution. https://hdl.handle.net/10779/lincoln.25179038.v1

kyle-fogarty-180x180.jpg

Research Interests Development of long-term autonomy for robotics, the theoretical development and practical application of computer vision, computer graphics, and machine learning techniques.

PhD research: 3D modelling of natural structures

The agri-food industry currently faces pressures including those from a growing population and increased consumption; for sustainable food security, innovation in this sector is needed.  The use of synthetic pesticides to control weeds, insects and diseases is widespread in global agriculture; with increasing regulations and registration costs, the number of new pesticides entering the agriculture market has been reduced¹. Now, attention is turning to alternative methods of weeding and controlling the spread of disease; work in the field of autonomous systems has developed robotics that reduces the need for herbicides by deploying camera guided robotic weeding devices. Central to many of these approaches is the application of computer vision to perceive the world and make decisions about the health of plants and crops. A recent paper² demonstrated that `off-the-shelf CNNs’, the YOLOv3 architecture more precisely, can be used relatively effectively for the detection and localisation of iceberg lettuce. Nevertheless, the author of this paper comments on one of the fundamental issues in these sorts of perception task, the need for lots of training data and the general lack of these datasets. The collection of training data, in the form of labelled training images, can be a time consuming and arduous task. Generative models provide a potential solution to this problem. Generative approaches have proven successful for the synthesis of photorealistic, novel 2D images of objects such as human faces. Recent work has also begun to explore the ideas of generative models for 3D rendering³. This project will focus on the development of 3D modelling techniques of natural structures, like plants and crops, and their applications in tackling pertinent problems in the agri-food industry like the use of pesticides.

Publications: 

• Foster, J., Fogarty, K., Schoepf, S., Öztireli, C., & Brintrup, A. (2024). Zero-shot machine unlearning at scale via lipschitz regularization. arXiv preprint arXiv:2402.01401.
• Zhong, F., Fogarty, K., Hanji, P., Wu, T., Sztrajman, A., Spielberg, A., ... & Oztireli, C. (2023). Neural fields with hard constraints of arbitrary differential order. arXiv preprint arXiv:2306.08943.
• K. Fogarty, E. Ametova, G. Burca, A. M. Korsunsky, S. Schmidt, P. J. Withers, W. R. B. Lionheart; Recovering the second moment of the strain distribution from neutron Bragg edge data. Appl. Phys. Lett. 18 April 2022; 120 (16): 164102. https://doi.org/10.1063/5.0085896
• Yang, J., Fogarty, K., Zhong, F., & Oztireli, C. (2024). SYM3D: Learning Symmetric Triplanes for Better 3D-Awareness of GANs. arXiv preprint arXiv:2406.06432.
• Zhou, C., Zhong, F., Hanji, P., Guo, Z., Fogarty, K., Sztrajman, A., ... & Oztireli, C. (2023). FrePolad: Frequency-Rectified Point Latent Diffusion for Point Cloud Generation. arXiv preprint arXiv:2311.12090.
• Moncur, Bethan; Clawson, Garry; Bennett, James; Fogarty, Kyle; Fox, Charles (2022). Towards Open Source Hardware Robotic Woodwind: an Internal Duct Flute Player. University of Lincoln. Conference contribution. https://hdl.handle.net/10779/lincoln.25179038.v1

rachel-trimble-180x180.jpg

Machine Learning application for agriculture and conservation

PhD Project: Using reinforcement learning to optimise adaptive control of invading plant disease epidemics

Invasive plant diseases threaten agricultural production and natural ecosystems. For example, Xylella Fastidiosa has killed large numbers of olive trees in Italy with future economic impact estimated in the billions of euros. At the same time, the resources to manage these pathogens are limited. Fast responses to outbreaks can minimise the damage. However, this means decisions on how resources are deployed must be made at the start of an outbreak when information about the disease progress and epidemic parameters in a new environment may be limited.

This project aims to explore and evaluate the use of reinforcement learning approaches to optimise control of invasive plant diseases. This will mean building plant disease models and reinforcement learning agents and evaluating the performance of the agents. The work will investigate the simulated cost of control, epidemic outcomes and robustness to uncertainty in the epidemic model. The techniques will aim to be generally applicable but will also be tested with real case studies. 

Publications: 

• Russell, R., & Cunniffe, N.J. (2024) Optimal Control Prevents Itself from Eradicating Stochastic Disease Epidemics, pre-print.
• Trimble, R., & Fox, C. (2023) Skid-steer friction calibration protocol for digital twin creation. In: TAROS, September 12-15 2023, Cambridge, UK
• Accelerate AI in Biological Science workshop (May 2023): Can we use Machine Learning to Improve Control of Invasive Plant Diseases?
• 12th International Congress of Plant Pathology – ICPP satellite event 2023 (August 2023): Integrating Reinforcement Learning and Epidemiological Models for Disease Control Optimisation with Limited Information.

jack_bradley.jpg

Plant diseases (AMR), Machine Learning, Automation, Agricultural Engineering, Sensing (Electrochemical)

PhD Project: Bioreactor arrays for analysis and control of food production using genetically modified microbes

Food production using genetically modified algae and bacteria is increasingly becoming a preferable option for high-value products, such as vitamins [1], and could play an important role in future bioeconomy. Compared to traditional agriculture, this new branch offers solutions to several food-security challenges: water pollution, fertilizer use, climate change, land scarcity, etc [2]. Algae and bacteria can be cultured independent of land, without fertiliser, reduced greenhouse gas emissions, [3] and in limited space, which has also recently gained attention for food production in spaceships and on board of the ISS, where spatial constraints are a critical factor. The biomass production capacities of algal and bacterial species are much higher compared to terrestrial plants and can be easily harnessed with genetic modifications. Furthermore, these organisms can be easily and efficiently cultivated in water without fertilisers, making them an environmentally safe, and sustainable choice for the future food production.  

However, several technological challenges still need to be addressed to realise the full potential of genetically modified algae and bacteria in food production [4]. These include: 1) Low genetic stability of genetically modified microbes (GMMs), 2) susceptibility towards contamination, 3) lack of optimisation for genetic design and culture conditions. Cultures of GMM are susceptible to loss of function due to genetic instability or contaminating cells, which lead to reduced production. On the other hand, lack of a systematic approach to optimise GMMs for production rate and functional stability and selecting conditions where the function and functional lifetime is maximised, prevents efficient use of these systems.  

To address these challenges, this proposal aims to develop a feedback-controlled bioreactor array, which could be used to run parallel production and evolution experiments on GMMs to a) select optimised designs for function and functional lifetime and b) to run distributed production experiments where effects of contamination can be minimised through redundancy and continuous harvest of the produced goods. Furthermore, this array will enable easy optimisation of culture conditions and population size to maximize functional lifetime of GMMs for long-term cultivation in an automated manner. 

Accordingly, the specific aims of this project are:  

1. Engineer a bioreactor array for parallel production and evolution assays.  
2. Design and calibrate a control algorithm for feedback control of the cultures and product harvest using optical measurements of abundance and composition.  
3. Analyse genetic stability of GMMs using this setup through functional lifetime analysis. 
4. Optimise culture conditions and population size to maximise production. 
5. Perform test production runs with distributed control for continuous product harvest and elimination of contaminated cultures. 

personalpicomarfaris.jpg

Soft robots, tactile sensing, neuromorphic event-based systems, grasping and manipulation

PhD Project: Tactile-sensing and augmented reality for robotic manipulation of fruit

Fruit manipulation is a complex task. It is time-consuming, physically exhaustive, causes fruit loss due to human errors, and suffers from shortages as it depends on seasonal workers, making it an ideal application for robotics. However, handling fruits requires high-level robot skills due to different weights, shapes, and textures, to prevent fruit damage and achieve successful picking patterns. The project goal is to combine multimodal sensing (tactile and vision) with augmented reality (AR) to deliver capable robotic system for food handling, in particular fruits. Safe, accurate, and efficient manipulation will be achieved through advanced control of manipulators and through the development of (soft) grippers with tactile and visual sensing capabilities. The robot will be ‘programmed’ using innovative methods based on augmented reality. This is crucial for rapid prototyping / deploying within the unstructured environment, which is typical of the agriculture setting. The student will have full access to all teaching courses at undergraduate, master, and PhD level, in engineering and wider domains (communication, management, etc.). The student will also learn from a large body of activities at the Control Laboratory, at the Bio-Inspired Robotic Laboratory, and at the Observatory for Human-Machine Collaboration.

img_20230220_135144.jpg

Distributed Sensing, Wireless Sensor Networks, Edge Computing, IoT, Biosensors, Energy Harvesting, Predictive Models

PhD Project: Biosensor Development for In-Field Plant Diagnostics and Physiology Monitoring

The project aims to explore the potential benefits of combining biosensing, machine learning, and robotics for in-field plant diagnostics and physiology monitoring. Plant diagnostics will primarily focus on molecular diagnostics platforms that enable early and precise detection and identification of pathogens, such as fungi, as supported by prior work. Biosensor-enabled plant physiology monitoring will also target early disease detection as well as elements of plant growth (e.g., healing, ripening, senescence). 

This project brings device-level developments to the CDT for system-level applications. The project plan aims to solve unmet needs between the industry and state-of-the-art agricultural research within the scope of the AgriFoRwArdS CDT. From a top-level perspective, this project lies firmly within the CDT’s sensing and perception research area but aspires to interact with projects focusing on manipulation, mobile robotics, and automation. Autonomous Mobile Diagnostics Platform, Plant-as-a-Sensor Paradigm, and Olfactory Fruit Harvesting are all subprojects that may be explored under this research theme in collaboration with other CDT students and the industry partner.