Quantum Natural Language Processing — early steps in my learning path

A very basic introduction to a subject I am trying to learn

Introduction

Natural Language Processing (NLP) is a subject that always amazed me in Data Science. I used it a lot in my professional career, but mainly using non-semantic models based on bag of words, because the type of texts I had to classify were too short, with too many spelling errors and abbreviations.

I have been gradually diving into semantic models as I had to face problems with larger and more complex texts. I recently used BERTopic for a topic modeling problem whose performance amazed me, because it far outperformed techniques such as Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA).

In my previous posts on QML, I focused on binary classification problems using Variational Quantum Classifiers (VQC) and Quantum Enhanced SVMs. I am still performing tests with these techniques, but while my models are still running I decided to start studying another QML field of study: Quantum Natural Language Processing (QNLP). I am still starting my studies, but, as always, I would like to share my first discoveries, and if you have any ideas, suggestions or corrections to this post feel free to share them with me!

Text classification using QNLP

There are some types of QNLP problems, but here I will focus on the “simplest” of them, which is text classification. I must highlight the simplest, because in this case is not simple at all!

The process of creating a QNLP text classification model consists of the following steps:

Figure 1 — Simple diagram of a QNLP text classification model from author

DisCoCat Diagrams

Figure 2 — A Disco Cat (sorry, the pun was stronger than me) created by the author using gencraft

The first part of any NLP model is converting the text of the corpus into computable elements. In traditional NLP, the words and phrases are converted into vectors, which might be encoded by a bag of words method, a neural network such as a Word2Vec or a transformer in Large Language Models (LLM). In QNLP these texts must be converted into quantum circuits, which is a format that can be processed by quantum computers.

DisCoCat (Categorical Compositional Distribution) is a mathemtical framework that represents sentences as diagrams. For newcomers as me, it resembles graphs, but with specific properties.

Furthermore, some optimization problems envolving graphs have been successfully solved by quantum computer with considerable speedups and encoding data the way DisCoCat does seems very quantum friendly.

This is an example on how DisCoCat represents the sentence “Skillful man bakes dinner”:

Figure 3— DisCoCat representation of sentence “Skillful man bakes dinner”

This is a noun phrase in English, where we have a subject (“Skillful man”), a verb (“bakes”) and an object (“dinner”). The verb is the center of the phrase and expects a noun to its left and right to have a complete meaning.

To be quite honest, this is a complex topic and I’m still looking for the best references to learn about it. So, if you have any interesting references on the subject, feel free to suggest them in the comments!

Diagrams to quantum circuits

This is the step where we propely encode our data into a quantum format. Here a quantum circuit is built based on each sentence in our corpus.

Figure 4 — Quantum circuit representing “Skillful man baskes dinner”

Note that, in Figure 4, each word has a certain number of parameters and this depends on how the word is positioned in the sentence. Here the word man is part of the subject and has 3 parameters, represented by the Rx (man_n0), Rz (man_n1) and Rx (man_n2) gates in the upper left corner of Figure 4.

In another case, for the sentence “Man bakes tasty dinner” we have the following circuit:

Figure 5— Quantum circuit representing “Man bakes tasty dinner”

In this case, man is also the subject and we are using the same parameters compared to the previous sentence, but the gates and parameters are placed in other parts of the circuit.

Now we have an example sentence using the word man as part of the object (“Woman loves man”):

Figure 6 — Quantum circuit for the sentence “Woman loves man”

As we can see, we have a set of parameters for each word that are applied to the quantum gates, but the way they are positioned in the circuit depends on the sentence and the context of the word in it. Thus, this approach takes into account the context of the word in the sentence. The word parameters are the same in the three cases mentioned. However, depending on the word and its context in the sentence, the number of parameters in each circuit may vary.

These operations are complex to manually code, but there is a Python package that creates the DisCoCat diagrams and tranforms them into quantum circuits: lambeq, from Quantinuum, which is open source.

Model training

Now that we have a training dataset, composed of a number of parameterizable quantum circuits, and training labels, we are able to train our classification model.

The training task is made in a hybrid environment, where the quantum circuits are run in quantum computers (or simulated devices) and the parameters optimization is run in a classic computer.

Figure 7 — Hybrid computing diagram

This type of model can be trained using pyTorch, but it is important to note that QNLP models might become very consuming in terms of quantum memory (QRAM) and computation, so we are currently very limited in terms of complexity regarding vocabulary (quantity of parameters) and sentences (number of words in each phrase and amont of sentences in corpus).

Once the model is trained, it is ready for evaluation and prediction.

If you are willing to dive into a simple QNLP text classification model, this tutorial from Pennylane is a nice introduction.