Home / Deep Learning Techniques

Example Illustrating a Summary of an English Text

Summary text

A total of 47 bodies have been exhumed from two mass graves. Iraqis find mass graves inside presidential palace compound in Tikrit . ISIS claimed to have executed 1,700 Iraqi soldiers captured outside Camp Speicher.

Table 1.1. Example illustrating a summary of an English text


Maybe you are interested!

Summary text

After the survey, Ms. Nguyen Thi Lang - Head of the Propaganda Department of the Provincial Labor Federation - together with trade union officials worked with local authorities and organized a dialogue conference with the presence of business representatives. industry and workers. Tinh Loi Garment Company Limited, which has nearly 1,000 female workers staying here, has agreed to sponsor Huong Sen Kindergarten with an additional 3 million VND each month to upgrade and open more classrooms, receiving more than 200 students. I'm the child of a worker and I'm going to school.

Example Illustrating a Summary of an English Text

Table 1.2. Example illustrating a summary of a Vietnamese text

1.1.2. Classification of text summarization problem


Text summarization problems are classified based on different criteria, including some common types of problems as follows:

- Single-text summary: The source text has only one text.

- Multi-text summary: The source text set includes many documents (these documents often have related content). The resulting text is a single text from the input source text set. Multi-text summarization faces some difficulties such as the problem of duplicating information between source documents, complex text preprocessing, and requiring high compression ratios.

- Extraction-oriented text summarization: Is the process of shortening the text so that the resulting text contains the linguistic units in the source text.

- Summary-oriented text summarization: Is the process of shortening the text so that the resulting text contains a number of new corpus units generated from corpus units located in the source text or not in the text. source version. From this information, perform transformations to create a new text while still ensuring the content and meaning of the input source text is maintained. Summary-oriented text summarization is a complex text summarization problem, with many difficulties in semantic representation and natural language generation from source text.

- Single-language summary: The source text and summary text have only one language.

- Multilingual summarization: The source text contains only one language, but the resulting text can be summarized in many different languages.

- Mixed-language summary: The source text can include many different languages.

Among these types of text summaries, extraction-oriented summarization creates a summary text based on sentence shortening that brings high linguistic efficiency, while summary-oriented summarization produces a summary text. The shortcut is guaranteed in terms of syntax and grammar

meaning by shortening sentences [58,59,60]. Currently proposed text summarization methods are often extraction-oriented summarization because it is easier to perform than the sentence reduction of summary-oriented summarization. However, using a summary-oriented text summarization approach often results in summarizing texts with less coherent information. Besides, single-text summaries are also made easier, the generated text has less duplicate information compared to multi-text summaries. Therefore, the problems of single-text summarization, multi-text summarization, extraction-oriented text summarization and summary-oriented text summarization have gained the attention and development of researchers in the field. natural language processing in general and text summarization in particular [61].

1.1.3. Steps taken in text summary


With input source text, to generate a summary, a TTVB system needs to perform the main steps shown in Figure 1.1 below.

Figure 1.1. Steps taken in text summary

Analysis: A text or set of source texts is analyzed to return information used for searching and evaluating important linguistic units and input parameters for the next step.

Transform: This step uses a transformation that acts on the output information of the analysis step to simplify and create a unified whole. The returned results are summarized corpus units.

Generate summary text: This step will link the corpus units received from the transformation step according to a certain criterion to generate summary text.

With each type of TTVB system there will be certain differences. For summary-oriented TTVB systems, there are all the above steps, but for extraction-oriented TTVB systems, there is no transformation step but only two steps of analysis and summary text generation.

1.1.4. Some characteristics of the text


Sentence position: The importance of a sentence in a text based on position characteristics is determined as the position value of the sentence in the text. Many methods often consider the first sentence in a text to be more important than other sentences in the text [62,63].

TF-IDF: TF-IDF (Term Frequency - Inverse Document Frequency) is the weight of a word that represents the importance of that word in a text that is located in a set of documents [64] . The TF-IDF weight is calculated according to the frequency of occurrence of words (TF) and the inverse of the frequency of occurrence of words in a text of a set of documents (IDF) as follows:

- TF = Number of occurrences of the word in the text/Total number of words in the text.

- IDF = log (Total number of documents in the text set/Number of documents containing that word).

- TF-IDF = TF*IDF.

Central sentence: The importance of a sentence in a text based on the characteristics of the central sentence is calculated by the average value of the similarity between a sentence and other sentences in the text. This feature considers the co-occurrence of words between a sentence and other sentences in the text [65].


1.2. Some methods for automatically evaluating summary text


With the text summarization problem, the effectiveness of the summary text plays an important role. To evaluate the effectiveness of summary documents, it is necessary to rely on parameters such as compression ratio, accuracy, cohesion, etc. There are a number of methods to evaluate the effectiveness of summary documents presented below. This.

1.2.1. The method is based on content similarity


Evaluate the similarity of the content of the result document generated by the considered TTVB system compared to the corresponding result document generated by other methods. Suppose, the result text of the application under consideration is S , the corresponding summary result text of other n evaluation methods is: J 1 , J 2 ,…,J n (with the same original source text ) then the formula for calculating similarity is:

sim ( M , S ,{ J , J

, J }) M ( S , J 1 ) M ( S , J 2 ) M ( S , J 3 )

(1.1)


in there:

1 2 3 3

- M is the criterion for calculating the content similarity between two documents X and Y, M

( ) ( )

x

2

y

2

i

i

i

It is usually calculated according to the following formula [66]:


cos( X , Y )

x iy

(1.2)


with: X, Y are two documents represented as vectors respectively.

- Or M can be calculated another way by formula [67]:

LCS ( X , Y ) length ( X ) length ( Y ) d ( X , Y )

2


(1.3)

with:


+ X, Y are two texts represented as strings of corresponding words.

+ d(X,Y) is the minimum number of addition and deletion operations needed to make the variable

convert text X to text Y.

+ LCS(X,Y) is the length of the greatest common substring between X and Y.

+ length(X), length(Y) are the length of the two documents X, Y, respectively.

1.2.2. The method is based on appropriate correlation


The correlation-based method is suitable for evaluating the BTTB system based on queries: With a query Q and a set of documents { D i } and a tool to sort the documents D i in order The degree of compatibility between D i and Q in decreasing order, then from the set { D i }, we have the set { S i } which is the summary text set of { D i } generated by the system under consideration, we use Use the sorting tool above to sort { S i } like

above. To evaluate, it is necessary to determine the correlation between these two sorted lists.

The common formula for determining correlation is the linear correlation between two sets of matching points x and y:

i

( x )

x

2

i

i

( y )

y

2

i

i

r

i( x ix ) ( yy )


(1.4)


where: x and y are the average value of each corresponding set of matching points for the document set D i .

1.2.3. ROUGE method


Evaluating text summarization results is a difficult task at the present time because using the opinions of language experts is considered the best way to evaluate but is costly. Therefore, automatic evaluation solutions are considered the optimal solution to evaluate the quality of summaries generated by text summarization systems. The automatic evaluation solution must find a measure closest to human evaluation to evaluate summary text and ROUGE (Recall- Oriented Understudy for Gisting Evaluation) [68] is an effective automatic evaluation measure. The fruit is commonly used today.


1.2.3.1. ROUGE measure

The ROUGE metric is used as a standard metric to evaluate the effectiveness of text summarization systems. ROUGE performs a comparison of a summary automatically generated from the summary model and a set of reference summaries (natural human summaries). Therefore, to get a good evaluation, calculating Recall and Precision [69,70] through duplicate words is used in the ROUGE measure.

Recall: Shows how much of the reference summary the system summary captures, calculated according to the formula:

R c

a

(1.5)

where: c is the number of words in the recapture system summary, a is the total number of words in the reference summary.

If all the words in the reference summary have been summarized by the system, it cannot be confirmed that the system summary is of real quality because a summary generated from the system can be very poor. long and contains all the words in the reference summary, but most of the remaining words in the system summary are redundant, which makes the summary lengthy. Therefore, precision is used to overcome this problem.

Precision: Shows how many parts the system summary actually has relative to the reference summary, calculated by the formula:

P c

b

(1.6)

where: c is the number of words the system summary captures relative to the reference summary, b is the total number of words in the system summary.

A commonly used measure is the F1 measure ( F1 score ) [70]. The F1 measure is calculated based on the recall R and precision P according to the formula:

F1 2 R * P

R P

(1.7)

The F1 measure represents the summary quality of a text summarization system more objectively because it tends to be closer to the smaller value between the two values ​​of recall and precision , the F1 value is large if both Great recall and precision values.


1.2.3.2. Popular ROUGE measures

Common ROUGE [68] metrics often used to evaluate the quality of a system summary compared to a reference summary in the text summarization problem include:

Rouge recall – N (symbol R N ): Represents the use of one word (uni-gram), two words (bi-gram), three words (tri-gram) or N words (N-gram) appear simultaneously in the system summary and reference summary. The recall R N (usually N = 1 ÷ 4) is calculated according to the formula:

S R S g ram S Count sample ( gram N )​

R N

S R S

N


grams N S

Count ( grams N )

(1.8)

in there:


+ N : is N-gram (with N = 1, 2, 3,...).

+ RS: is a set of reference summary documents.

+ Count match (gram N ): is the number of N-grams appearing simultaneously in the system summary and reference summary.

+ Count(gram N ): is the number of N-grams in the reference summary.

Rouge precision – N (symbol P N ): Represents the use of one word (uni-gram), two words (bi-gram), three words (tri-gram) or N words (N-gram) appears in the system summary relative to the reference summary. The recall P N (usually N = 1 ÷ 4) is calculated according to the formula:

S R S g ram S Count sample ( gram N )​

P N

N

grams N SS

Count ( grams N )

(1.9)

with: SS: is the system summary text.

Rouge F1 measure – N (symbol R – N): R–N measure (usually N = 1 ÷ 4) is calculated based on recall R N and precision P N according to the formula:

R N 2 R N * P N

R N P N

(1.10)

In formula (1.10), when N = 1 we have measure R-1, N = 2 we have measure R-2 which are commonly used measures to evaluate the effectiveness of text summarization models. .

Rouge – L F1 measure (symbol R – L): Represents the use of the longest string of words appearing simultaneously in the system summary and the reference summary based on the longest common substring (LCS - Longest Common Subsequence). LCS is the problem of finding the longest common substring for all internal strings

a set of strings (usually two strings). The RL measure is calculated based on recall

u

R lcs and P lcs accuracy are as follows:

LCS ( r i , C )

R lc s


P lc s

i 1

m

u

LCS ( r i , C )

i 1

n

(1.11)


(1.12)

(1 2 ) R * P

P

R Llc s lc s

(1.13)

2

R lc s lc s

where: C is the candidate summary set; r i is the review sentence in the reference summary; u is the number of sentences of the reference summary; m is the number of words in the reference summary set

mat; n is the number of words in the candidate summary set C ;

LCS ( r i , C )

is the point of the set

is determined by the union of the longest common substring set between sentence r i and every sentence in the set C , this score is calculated by the total length of the union of the largest common substrings divided by the length of r i ; is the coefficient that controls the relative importance of R lcs and P lcs ( is a parameter usually chosen equal to 1).

In formula (1.13), when

1

We have a commonly used measure

evaluates the quality of the summary and is calculated according to the formula:

R L2 R lc s * P lc s

R lc s P lc s


(1.14)

F1 measure of Rouge-S (symbol RS): RS measure determines the similarity between any pair of words in a sentence combined in the correct order. The RS measure is calculated based on the recall R S and precision P S as follows:

R SKIP 2 ( X , Y )

S C ( m , 2)

P SKIP 2 ( X , Y )

S C ( n , 2)

(1 2 ) R * P

(1.15)


(1.16)

R S SS

(1.17)

R 2 P

SS

where: m is the number of words of the reference summary; n is the number of words in the summary set

off candidate C ; X is the reference summary set; Y is the candidate summary set;

SKIP 2 ( X , Y )

is the number of matching skip bi-gram pairs between X and Y ; C(m,2) , C(n,2) are respectively the 2nd convolutional combination functions of m elements, 2nd convolutional combinational functions of n elements; is the coefficient that controls the relative importance of R S and P S ( is an optional parameter and is usually set to 1).

In formula (1.17), when

R S 2 R S * P S

R S P S

1 we have the following formula to calculate the measure:


(1.18)

Rouge-St F1 measure (symbol R-St): When using the RS measure , a number of meaningless word pairs may appear such as “the the” , “is is” , etc. v.... To minimize word pairs

This nonsense, we can limit the distance that can form word pairs to t (in t-skip bi-gram ), meaning that only words that are no more than t words apart can form valid word pairs. (Because pairs of meaningless words are often not located close to each other, choosing a small t will limit the creation of pairs of meaningless words). Then, the R-St measure is calculated based on the recall R St and the precision P St as follows:

R SKIP 2, t ( X , Y )

(1.19)

St t

( m i 1)

i 0

P SKIP 2, t ( X , Y )

(1.20)

St t

( n i 1)

i 0

(1 2 ) R * P

R StSt St

(1.21)

R 2 P

St St

where: m is the number of words of the reference summary; n is the number of words in the summary set

off candidate C ; X is the reference summary set; Y is the candidate summary set;

SKIP 2, t ( X , Y )

is the number of matching skip bi-gram pairs between X and Y ; is the control coefficient

relative importance of equals 1).

R St and

P St

( is an optional parameter and is often chosen

In formula (1.21), when

R St 2 R St * P St

R St P St

1 we have the measure calculated by the formula:


(1.22)

In formula (1.22), when t = 4, we have the R-S4 measure , which is a commonly used measure to evaluate the effectiveness of text summarization models.

F1 measure of Rouge-SUt (symbol R-SUt): Is an extended measure of R-St measure by adding a word (uni-gram) as a counting unit to overcome the case of a candidate sentence The member has no co-occurrence pairs with the reference summary. The R-SUt measure is obtained from R-St by adding sentence start markers to the beginning of candidate sentences and reference summary sentences. When t = 4, we have the R-SU4 measure obtained from the R-S4 measure , which is a commonly used measure to evaluate the effectiveness of text summarization models.

Currently, ROUGE metrics are used as a popular standard metric to evaluate the effectiveness of text summarization models. Therefore, the thesis will use the metrics R-1 , R-2 , RL , R-S4 and R-SU4 to evaluate the effectiveness of the proposed text summarization models.


1.3. Methods for combining text in multi-text summarization


For the multi-text summarization problem, the first question is how to combine the documents in this source text set?

Figure 1.2. Method for processing summaries of each single text in multi-text summarization

Currently there are two commonly used methods to solve this problem:

- First method: Combine all input documents into a single text called hypertext , then summarize on this hypertext to generate the final summary. This method turns the multi-text summarization problem into a single-text summarization problem and can use single-text summarization techniques to generate the final summary.

- Second method: First, each text of the multi-text set is summarized to generate the corresponding summary text. These summaries will then be combined into a comprehensive summary text. After that, this synthesized summary text will be summarized using single-text summarization techniques to produce the final summary text, which is also the summary of the results of the source multi-text set. need summary. Figure 1.2 represents the idea of ​​the method of summarizing each single text in multi-text summarization.

The first approach is easier to capture novel information than the second approach. The second approach summarizes each text first, reducing the input text length of the multi-text summarization model so the final summary will have high accuracy.

Comment


Agree Privacy Policy *