The Transformer architecture revolutionized the world of Natural Language Processing. Hybridization with Quantum Computing may give it an additional boost on the long road toward the Holy Grail of understanding
In recent years, a novel neural network architecture called Transformer first introduced in the ground-breaking paper Attention is all you need [arXiv:1706.03762] revolutionized the analysis of sequential data, with particular focus on Natural Language Processing tasks such as Machine Translation or generation of text starting from human prompt. A generalization of the Transformer architecture led a researcher to applications in various fields such as image generation (an image is a sequence of pixels after all).
However, the main issue with this marvellous kind of neural networks is that the appalling size of parameters (in the order of hundreds of billions…
Using the PennyLane Quantum Machine Learning library it’s easy to create a quantum/digital-hybrid equivalent of the glorious LSTM layer.
In recent years, Toronto-based startup company Xanadu has introduced a python framework called PennyLane which allows users to create hybrid Quantum Machine Learning (QML) models. While it’s still too early to claim that quantum computing has taken over, there are some areas where it can give an advantage as for example in drug discovery or finance. One field that so far has been poorly explored in QML is Natural Language Processing (NLP), the sub-field of Artificial Intelligence that gives computers the ability to read, write and to some extent comprehend written text.
How a statistical test used to uncover tax frauds may help us find an explanation to a long-standing puzzle in particle physics
Humans like looking for patterns, be it shapes in the clouds or relationships among numbers. We are probably evolved to do so for survival, and just can’t help. In science, pattern recognition helps researcher to tell the shape of galaxies or to identify the decay of short-lived fundamental particles such as top quarks. Especially in physics, scientists have been scratching their heads for years trying to come up with an explanation of the apparently random distribution of the mass of fundamental particles or at least those that belong to the theory of the Standard Model of Particle Physics. Is there…
Elenco in questo post un po’ di risorse per farsi una “infarinatura” sul machine learning. Per iniziare, potete dare una letta a queste pagine / video lezioni:
* https://www.kaggle.com/learn/intro-to-machine-learning
* https://www.youtube.com/watch?v=0VH1Lim8gL8 (state of the art 2020)
Molti articoli interessanti appaiono praticamente ogni giorno sul sito Towards Data Science.
Quello del machine learning è un settore molto ampio, probabilmente le parole-chiave con le quali dovrete stringere amicizia sono:
- Tecniche di manipolazione dati, feature engineering
- Modelli: neural networks, decision trees
- Obiettivi: classification, regression, metriche per stimare la qualità (accuracy, F1-score)
- Tecniche di “training” dei modelli: supervised learning, un-supervised learning /…
Is it possible to predict the result of each series by leveraging historical basketball data and machine learning?
Paul the Octopus was a short-lived (26 January 2008–26 October 2010) cephalopod kept at the Sea Life Centre in Oberhausen, Germany, who became instantly famous because of his alleged ability to predict the results of FIFA World Cup football matches. All he had to do was to eat food from one of two boxes, identical in all aspects except for the flag of the contending national teams. As many regard octopuses as the closest to alien intelligence on Earth, one may wonder if the German invertebrate (no offence intended) knew some secret about football that presumably smarter hairless apes (a.k.a…
New evidence suggests that about 12,800 years ago a comet fell on North America causing the onset of an Ice Age, the extinction of large animals, and the disappearance of the Clovis people. History might have been very different if Native Americans had horses when the Europeans arrived in 1492.
In the 1997 non-fiction book “Guns, germs and steel”, Jared Diamond tried to answer the apparently simple question of why it was the Europeans who conquered the people who lived in the Americas, and not vice versa. The solution is found in a multi-disciplinary argument involving, as the title suggests, an interplay of weapons, infections and technology. What the Pulitzer-prize winner did not know at the time is that a critical piece of information was missing: Native American populations suffered a cataclysmic event, probably a comet strike, very early in their history. This event wiped out most of the large-sized…
While many people love science, few would agree to invest money with very low chances of getting anything in return. Are we at a tipping point in history?
The Gothic Cathedral of Science
It took decades to go from inception to construction, but here we are: humans have created a machine to explore the deepest recesses of Nature. It’s called the Large Hadron Collider, it’s a 27 km long vacuum tube in which protons are accelerated and smashed head-on at four different points. Around each of them is a detector, a sort of enormous 3D camera that takes snapshots of proton-proton collision. A single detector is used by scientists to run hundreds of different small experiments. One of them notoriously led to the significant discovery in 2012 of a fundamental particle called the…
A controversial theory claims that a thin disk of Dark Matter along the galactic plane is able to dislodge comets and create havoc on Earth. Let’s take a look with open data.
After I moved to Canada in 2014, I started to cross the Atlantic Ocean on a regular basis for work. On my first flight back home, I noticed a really weird feature situated somewhere in north of the St Lawrence river. Not having an internet connection available right away, I had to wait until we landed to find out it was called the Manicouagan Reservoir, an annular lake created by the impact of an asteroid about 215 million years ago (MYA). …
It is my belief that science and technology advance by making mistakes. Here’s an account of the two times I thought I had made a discovery, but more mundane explanations were at play.
Lately, I have read a book called “ Anomaly!” written by Tommaso Dorigo. The book is about a part of the history of the CDF Experiment at Fermilab, more or less spanning the period between its inception in the early 1980s to the end of the first part of data taking (2000). This spurred my intention to look back at the mistakes that are routinely taken during the scientific process, a fundamental ingredient without which science would never advance. Still, one would never publish articles to describe the things that went wrong. So let me tell you this story. …
Supplementing pre-trained models with variational quantum circuits brings us to the era of quantum-enhanced Natural Language Processing
In recent time, a trend appeared quite evident: large, pre-trained models are needed to achieve state-of-the-art performance in computer vision and language understanding. In particular, the field of natural language processing (NLP) is seeing an exponential expansion in terms of capabilities but also network size as Transformer architecture-based models appeared (for example BERT, GPT-2). In this article, I’ll focus on the classification of short documents (pre-prints of scientific papers) by using Universal Sentence Encoder (USE) embeddings and a quantum machine learning kind of operator called variational quantum circuit which is, roughly speaking, the equivalent of a fully-connected (dense) layer in…
Riccardo Di Sipio
NLP Machine Learning engineer at Ideal. Previously smashing protons at the CERN LHC. Views are my own.
