At the core of the FashionBrain research at the Information School is the effective combination of machine learning and a human element.
The fashion industry moves quickly and, as with most facets of modern life, the rise of technology and the internet is only increasing its pace. Trends come and go seemingly instantaneously, and with ease of access to information higher than ever before and social networks spreading this information further than previously imaginable, European fashion retailers face innumerable challenges in keeping up, especially in an online world dominated by North American multinationals. The €2.8 million FashionBrain project, funded by the European Commission through Horizon2020 and coordinated at the Information School in Sheffield, aims to tackle this problem, whilst enhancing the consumer experience, too.
FashionBrain brings together three major Universities (the University of Sheffield, the University of Fribourg in Switzerland and Beuth University of Applied Sciences in Berlin) with three commercial companies (Zalando, MonetDB Solutions and Fashwell). Dr Gianluca Demartini, who led the Sheffield team and also coordinated the whole consortium, says “We are building new data solutions for different types of data and bring value to Zalando and other fashion retailers across Europe”.
Computer algorithms have the capacity to handle truly vast quantities of data, but as anyone who has used Google Translate will know, they still have blind spots and struggle with some tasks in which humans perform better. At the core of the FashionBrain research at the Information School is the effective combination of machine learning and a human element. “We are using hybrid human/machine solutions that leverage the scalability of computers to process any amount of data with the quality of human intelligence to make those algorithms do better” says Dr Demartini of the techniques the Information School team uses in the project.
The fashion industry
moves quickly and,
as with most facets
of modern life, the rise
of technology and the
internet is only
increasing its pace.
The three-year project began in January 2017 and in Sheffield, the team (completed by post-doctoral researcher Alessandro Checco and project officer Kathryn MacKellar) are focussed on applying crowdsourcing and machine learning techniques to data from various sources such as product catalogues and online reviews, as well as social media platforms like Instagram. “In an Instagram image, we need to understand that there is, for example, a pair of shoes and a bag, and then work out exactly which product it is from a catalogue of products” says Dr Demartini. “One of the things we are building is an app which allows you to take a photo of a friend and buy the same pair of shoes they are wearing.” Of course, to make such a simple-seeming app is a many step process, in which all partners are involved.
The European fashion market is fast increasing in size and competency. Italy has many small, family-driven fashion companies and Zalando is currently the main online fashion retailer in Europe, but it faces a threat from American corporate giant Amazon, who look poised to branch into fashion in the near future. They have been organising workshops on machine learning over the last few years and it seems they are bringing their knowledge to bear in the upcoming Amazon Echo Look, a product for the home which will take photos and videos of you trying on outfits and tell you which looks best. Dr Demartini says “the problem we are trying to solve is that online fashion is driven by search engines and social networks, which are typically American-based companies, who can drive customers where they desire. With this project, we want to take out these middlemen, which will help the European market.”
The European fashion market is fast increasing in size and competency.
Project partners in Fribourg are using time series forecasting to predict upcoming fashion trends, whilst research at Beuth, Berlin, is employing text mining and deep learning techniques to gain the highest quality results from large datasets. All the research is underpinned by Zalando’s vast catalogue of products, manufacturers and reviews, whilst MonetDB Solutions provides data storage and indexing solutions on a large scale and Swiss start-up Fashwell extracts the fashion product data from Instagram images.
Though the project is in its early stages, a lot of data has already been collected and crowdsourcing experiments have begun in Sheffield. Product reviews are being analysed to identify specific product issues; for example, a recurring problem with the sizing of a specific shoe model which can then be highlighted to other users, making them aware of the issue before purchase.
The next project milestone at month 18 of the project (June 2018) will see preliminary versions of the image searching app and trend detection techniques presented to the European commission. The three universities are already on the way to publishing their findings in academic journals and the outcomes will eventually be put into commercial use, hopefully making the European market more resilient to mounting corporate monopoly. Dr Demartini is leaving Sheffield shortly to pursue a new role at the University of Queensland, Brisbane, but the project will continue under the guidance of Professor Paul Clough at the Information School, with Dr Demartini likely to remain in a consultancy role.
Having started the project with a solely data-related research background, Dr Demartini is learning a lot about fashion during the FashionBrain project. “I was very simplistic about shoe sizing issues – I was thinking ‘it’s either too big or too small’. It’s far more complex than I was imagining!” This exemplifies the need for a human input; how could an algorithm possibly resolve the issue of which end of a shoe is too tight? Whether or not an expertise in fashion is required to provide this human element is still to be determined, but whatever the outcome, the future of shopping for your fashionable clothing may be changing.