Machine learning and computer vision
Machine learning and computer vision
Dense 3D face modelling and reconstruction
Before and after using Reflectance Transformation Imaging (RTI). Courtesy of 'Prof. Mieczyslaw Domaradzki' Archaeological Museum, Septemvri, Bulgaria
Machine learning is the science of constructing algorithms that learn from data and in the case of computer vision, to image and video data.
The proliferation of data and the availability of high performance computing makes this a fertile and very applicable area of research. It draws on ideas in computer science, statistics and applied mathematics, together with biologically inspired paradigms such as neural computation.
Machine learning research at Exeter spans the range of data, applications and methodologies: from kernel methods to deep neural architectures and reinforcement learning applied to both continuous and discrete, graph-based data.
The group collaborates with industrial partners and with the Impact Lab, and contributes strongly to the University’s membership of the Alan Turing Institute.
Group members
- Key Contact: Professor Richard Everson - Professor of Machine Learning (Research Lead)
- Prof Alberto Arribas - Associate Professor
- Dr Jacqueline Christmas - Senior Lecturer
- Dr Fabrizio Costa - Lecturer
- Dr Leon Danon - Senior Lecturer
- Dr Anjan Dutta - Lecturer
- Professor Jonathan Fieldsend - Professor
- Dr Dmitry Kangin - Research Fellow
- Professor Ed Keedwell - Professor
- Dr Lorenzo Livi - Lecturer
- Dr Chunbo Luo - Senior Lecturer
- Dr Sareh Rowlands - Lecturer
- Dr Wenjie Ruan - Senior Lecturer
- Prof Hywel Williams - Associate Professor
- Dr Johan Wahlström - Lecturer
Former members
- Dr Nicolas Pugeault - Lecturer
- Dr Anastasios (Tassos) Roussos - Lecturer
Research Projects
Computational fluid dynamics (CFD) is fundamental to modern engineering design, from aircraft and cars to household appliances. It allows the behaviour of fluids to be computationally simulated and new designs to be evaluated. However, each CFD evaluation can take a long time, possibly hours or days.
This 3-year EPSRC-funded project aims to accelerate the optimisation process by substituting computationally simpler, dynamically generated 'surrogate' models in place of full CFD evaluation - learning appropriate surrogates from a relatively few well-chosen full evaluations, with multiple competing quality criteria for the final designs.
Our diverse range of industrial collaborations will ensure research is driven by realistic industrial problems and builds on existing industrial experience.
Project team
Professor Richard Everson (project lead)
Professor Gavin Tabor (Engineering)
Funded by Innovate UK, this project is learning the best parameters to enable digital twins to faithfully represent real-world systems.
Contact: Professor Richard Everson
Machine learning and machine vision are being used to automatically detect and classify defects in waste-water pipes from CCTV inspections in this collaboration with Innovate UK and South West Water.
Contact: Professor Richard Everson
Developing novel methods for modelling and reconstructing the 3D geometry and dynamics of the human face from image data, with unprecedented quality.
Contact: Dr Anastasios (Tassos) Roussos
This project is using machine learning methods to model the sea’s surface from maritime radar observations, and then using the model to make predictions of that surface for up to two minutes into the future.
Contact: Dr Jacqueline Christmas
We are using our knowledge of the constraints on the shapes of sea waves and their spectra, to learn the scale and shapes of waves from video recorded by a single, monoscopic camera installed on a ship.
Contact: Dr Jacqueline Christmas
This project involves developing image processing techniques, based on inexpensive, off-the-shelf digital photographic equipment, to model the shape of a surface and hence read its inscriptions.
Contact: Dr Jacqueline Christmas
We are working with the Royal Society to develop collaborative strategies to enable multiple drones to efficiently detect marine oil spills.
Contact: Dr Chunbo Luo
Using deep learning to enhance infrared images via super resolution and detect/recognise low-pixel targets. This project is a collaboration with CENSIS, Thales and the Scottish Funding Council (SFC).
This interdisciplinary ESRC-funded project uses machine learning, text analysis and network science to understand how people are exposed to political news online. Find out more on the ExpoNet webpage.
Contact: Dr Hywel Williams
Wind storms can cause great damage to property and infrastructure. The windstorm footprint is an important summary of the hazard, of great relevance to the insurance industry and to infrastructure providers.
Windstorm footprints are conventionally estimated from meteorological data and numerical weather model analyses. However there are several interesting, less structured data sources (video feeds, social media data, amateur meteorological observations) that could contribute to their estimation, and more importantly will raise the spatial resolution of estimates.
These will be investigated as part of this 4-year NERC-funded project which will develop and compare statistical process modelling and machine learning approaches to the problem.
- View details of the BigFoot NERC grant
Project team
Professor Peter Challenor (Mathematics, project lead)
Dr Theo Economou (Mathematics)
Professor David Stephenson (Mathematics)
Funded by the ESRC, this project considers social media as a data source to monitor pollen and air pollution alongside their health impacts such as hayfever and asthma. Visit the social sensing Exeter website.
Contact: Dr Hywel Williams
This project is developing our understanding the behaviour of recurrent neural networks with dynamical systems theory.
Contact: Dr Lorenzo Livi
We are designing a statistical-geometrical framework to perform change detection on sequences of attributed graphs of varying size. This project is in collaboration with the Swiss National Science Foundation (SNF).
Contact: Dr Lorenzo Livi
Using machine learning to predict and control epileptic seizures in drug-resistant patients.
Contact: Dr Lorenzo Livi
Visual attention and pre-attentive perception
How much of driving is pre-attentive? Quite a lot, it appears. For an experienced driver, the act of driving requires little attention, allowing for extended periods of driving, while at the same time having a conversation, looking for directions or daydreaming. It is precisely because little attention is necessary that drivers' inattention is such a common cause of accident.
Research in collaboration with the University of Surrey has been investigating exactly how much of a driver’s actions could be explained by pre-attentive perception only [1]. This research employed a computational model of pre-attentive perception with machine learning to mimic the driver’s behaviour. Experiments showed that up to 80% of steering and braking actions of a driver can be fully explained by such a pre-attentive model of perception. Moreover, this model was able predict the driver's steering accurately up to a full second before they started turning the wheel.
Ongoing research is investigating the full role of attention in dynamic tasks such as driving, and how a successful attentional strategy can be learnt from experience. In particular, recent experiments have allowed us to explain what is learnt by Deep Saliency models of visual attention [2].
Funding
This research is funded by the EPSRC under project DEVA: Autonomous Driving with Emergent Visual Attention.
References
- Pugeault N, Bowden R. (2015) How much of driving is pre-attentive? IEEE Transactions on Vehicular Technologies, volume 64, pages 1-15, article no. 12, DOI:10.1109/TVT.2015.2487826.
- He S, Pugeault N. (2017) Deep saliency: What is learnt by a deep network about saliency? 2nd Workshop on Visualization for Deep Learning at the 34 International Conference on Machine Learning, Sidney, Australia, 10th Aug 2017.
- He S, Kangin D, Mi Y, Pugeault N. (2018) Aggregated Sparse Attention for Steering Angle Prediction, International Conference on Pattern Recognition, Beijing, China, 20th-24th Aug 2018.
Autonomous control
Consider a team of robots charged with exploring an unknown environment completely autonomously. Their task will be to discover the environment’s layout, to localise themselves with respect to visible landmarks and each other, and to plan trajectories ahead.
The state of the art in robotics research provides robust methods to model a robot’s environment from vision and motion, so called SLAM (Simultaneous Localisation and Mapping), as well as for localising itself in this map using visual landmarks. Once the environment’s layout and the robot position are known, planning and navigation are also well studied problems.
What is less understood by previous research is:
- how should a robot plan its actions and motions initially in order to discover its environment most efficiently and accurately? [1]
- if we assume that the exploration can done by multiple robots simultaneously, how should these robots decide when and how to cooperate to improve exploration? [2]
In response to those questions, we have developed novel algorithms for planning optimal collaborative exploration of unknown environments by teams of autonomous robots.
This project is in collaboration with the Centre for Vision Speech and Signal Processing (CVSSP) at the University of Surrey.
Video: Humans can localise without LiDAR, can robots?
References
- Mendez Maldonado, Oscar, Hadfield, Simon, Pugeault, Nicolas and Bowden, Richard (2016) Next-best stereo: extending next best view optimisation for collaborative sensors. British Machine Vision Conference, 19-22 September 2016, York, UK.
- Mendez Maldonado, Oscar, Hadfield, Simon, Pugeault, Nicolas and Bowden, Richard (2017) Taking the Scenic Route to 3D: Optimising Reconstruction from Moving Cameras. IEEE International Conference on Computer Vision 2017, 22-29 October 2017, Venice, Italy.
- O Mendez, S Hadfield, N Pugeault, R Bowden (2018) SeDAR-Semantic Detection and Ranging: Humans can localise without LiDAR, can robots? Proceedings of the International Conference on Robotics and Automation (ICRA'2018), IEEE.
3D face modelling with unprecedented quality
We have recently created the most accurate digital 3D model of human faces [1,2]. We introduced the largest-scale 3D morphable model of facial shapes ever constructed, based on a dataset of more than 10,000 distinct facial identities from a huge range of gender, age and ethnicity combinations.
For this large scale facial model (LSFM), we proposed a fully automated system that establishes dense correspondences among 3D facial scans, yielding state-of-the-art results.
We are currently investigating novel methodologies that will allow us to extend our 3D face modelling so that it can also represent the 3D shapes of emotive faces with unprecedented accuracy. Towards that goal, we have recently collected a new dataset of dynamic 3D facial scans of around 5,000 individuals. This large-scale dataset includes facial scans of the participants in several different facial expressions. This data collection was done during a special exhibition and technology demonstration at the Science Museum, London.
This work is in collaboration with scientists from Imperial College London and craniofacial surgeons from Great Ormond Street Hospital and Royal Free Hospital.
Science magazine article
Science magazine featured this research project and its applications.
References
- J. Booth, A. Roussos, S. Zafeiriou, A. Ponniah, and D. Dunaway. A 3D Morphable Model learnt from 10,000 faces. International Conference on Computer Vision and Pattern Recognition (CVPR 2016), Las Vegas, Nevada, USA, June 2016.
- J. Booth, A. Roussos, A. Ponniah, D. Dunaway, and S. Zafeiriou. Large Scale 3D Morphable Models, International Journal of Computer Vision (IJCV), 2017.
Dense 3D face reconstruction from images and videos under unconstrained conditions
As part of our ongoing research, we are exploiting our highly-accurate 3D face modelling for problems of Computer Vision and Computer Graphics. Reconstructing the detailed 3D shape and dynamics of the human face has numerous applications, such as facial expression recognition, human-computer interaction, augmented reality, performance capture, computer games and visual effects, to name a few.
Despite the important advances in the related scientific fields, the existing methods have several limitations, since they can only work reliably under restrictive acquisition conditions. In our work, we seek to develop novel formulations and pioneering methodologies for dense 3D reconstruction and modelling of human faces that will be able to deal with challenging real-life image data.
We are paying particular attention to computational efficiency and to achieving unprecedented accuracy and robustness to challenging scenarios, such as low-resolution image data, severe occlusions, strong illumination changes and large intra-subject variability in facial morphology.
This work is in collaboration with scientists from Imperial College London.
References
- J. Booth, A. Roussos, E. Ververas, E. Antonakos, S. Ploumpis, Y. Panagakis, and S. Zafeiriou. 3D Reconstruction of "In-the-Wild" Faces in Images and Videos. Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), Accepted for publication.