R/S4classes.R, R/textstat.R, R/corpus.R, and 3 more
textstat-class.RdThe textstat S4 class is the superclass for the classes features,
context, and partition. Usually, these subclasses, which are designed to
serve a specified analytical purpose, will be used . Common standard generic
methods such as head, tail, dim, nrow, colnames are defined for the
textstat class and are available for subclasses by inheritence. The core of
textstat and its childs is a data.table in the slot stat for keeping
data on text statistics of a corpus, or a partition. The textstat class
inherits from the corpus class, keeping information on the corpus
available.
# S4 method for class 'textstat'
name(x)
# S4 method for class 'character'
name(x)
# S4 method for class 'textstat'
name(x) <- value
# S4 method for class 'textstat'
round(x, digits = 2L)
# S4 method for class 'textstat'
sort(x, by, decreasing = TRUE)
as.bundle(object, ...)
# S4 method for class 'textstat,textstat'
e1 + e2
# S4 method for class 'textstat'
subset(x, subset)
# S3 method for class 'textstat'
as.data.table(x, ...)
# S4 method for class 'textstat'
show(object)
# S4 method for class 'textstat'
p_attributes(.Object)
# S4 method for class 'textstat'
knit_print(x, options = knitr::opts_chunk, ...)
# S4 method for class 'textstat'
get_corpus(x)
# S4 method for class 'textstat'
format(x, digits = 2L)
restore(file)
cp(x)
# S4 method for class 'textstat'
view(.Object)An object (textstat or class inheriting from textstat).
A character vector to assign as name to slot name
of a textstat class object.
Number of digits.
Column that will serve as the key for sorting.
Logical, whether to return decreasing order.
a textstat object
Argument that will be passed into a call of the format method
on the object x.
A texstat object.
Another texstat object.
A logical expression indicating elements or rows to keep.
A textstat object.
Chunk options.
An rds file to restore (filename).
A head-method will return the first rows of the data.table in
the stat-slot. Use argument n to specify the number of rows.
A tail-method will return the last rows of the data.table in
the stat-slot. Use argument n to specify the number of rows.
The methods dim, nrow and ncol will return information
on the dimensions, the number of rows, or the number of columns of the
data.table in the stat-slot, respectively.
Objects derived from the textstat class can be indexed with simple
square brackets ("[") to get rows specified by an numeric/integer vector,
and with double square brackets ("[[") to get specific columns from the
data.table in the slot stat.
The colnames-method will return the column names of the data-table
in the slot stat.
The methods as.data.table, and as.data.frame will extract the
data.table in the slot stat as a data.table, or data.frame,
respectively.
textstat objects can have a name, which can be retrieved, and set using
the name-method and name<-, respectively.
The round()-method looks up all numeric columns in the
data.table in the stat-slot of the textstat object and
rounds values of these columns to the number of decimal places specified by
argument digits.
The knit_print method will be called by knitr to render
textstat objects or objects inheriting from the textstat class as a
DataTable htmlwidget when rendering a R Markdown document as html.
It will usually be necessary to explicitly state "render = knit_print" in
the chunk options. The option polmineR.pagelength controls the number of
lines displayed in the resulting htmlwidget. Note that including
htmlwidgets in html documents requires that pandoc is installed. To avoid
an error, a formatted data.table is returned by knit_print if
pandoc is not available.
The format()-method returns a pretty-printed and minimized version
of the data.table in the stat-slot of the textstat-object: It will
round all numeric columns to the number of decimal numbers specified by
digits, and drop all columns with token ids. The return value is a
data.table.
Using the reference semantics of data.table objects (i.e. inplace
modification) has great advantages for memory efficiency. But there may be
unexpected behavior when reloading an S4 textstat object (including classes
inheriting from textstat) with a data.table in the stat slot. Use
restore to copy the data.table once to have a restored object that works
for inplace operations after saving / reloading it.
It is not possible to add columns to the data.table in the stat
slot of a textclass object, when the object has been saved and loaded
using save()/load(). This scenario applies for instance, when the
objects of an interactive R session are saved, and loaded when starting the
next interactive R session. The cp() function will create a copy of the
object, including an explicit copy of the data.table in the stat slot.
Inplace modifications of the new object are possible. The function can also
be used to avoid unwanted side effects of modifying an object.
p_attributeObject of class character, p-attribute of the query.
corpusA corpus specified by a length-one character vector.
statA data.table with statistical information.
nameThe name of the object.
annotation_colsA character vector, column names of
data.table in slot stat that are annotations.
encodingA length-one character vector, the encoding of the corpus.
use(pkg = "polmineR", corpus = "GERMAPARLMINI")
#> ✔ corpus loaded: GERMAPARLMINI (version: 0.1.0 | build date: 2023-04-16)
use(pkg = "RcppCWB", corpus = "REUTERS")
#> ✔ corpus loaded: REUTERS
P <- partition("GERMAPARLMINI", date = ".*", p_attribute = "word", regex = TRUE)
#> ... get cpos and strucs
#> ... getting counts for p-attribute(s): word
y <- cooccurrences(P, query = "Arbeit")
# generics defined in the polmineR package
x <- count("REUTERS", p_attribute = "word")
name(x) <- "count_reuters"
name(x)
#> [1] "count_reuters"
get_corpus(x)
#> [1] "REUTERS"
# Standard generic methods known from data.frames work for objects inheriting
# from the textstat class
head(y)
#> word word_id count_partition count_coi count_ref exp_coi
#> <char> <int> <int> <int> <int> <num>
#> 1: Bundesagentur 5360 15 11 4 0.10741211
#> 2: für 66 1625 55 1570 11.63631205
#> 3: Sozial 12775 9 7 2 0.06444727
#> 4: " 493 468 21 447 3.35125787
#> 5: ihre 597 121 12 109 0.86645770
#> 6: schafft 675 36 8 28 0.25778907
#> exp_ref ll rank_ll
#> <num> <num> <int>
#> 1: 14.892588 91.39629 1
#> 2: 1613.363688 86.51204 2
#> 3: 8.935553 59.67241 3
#> 4: 464.648742 42.65782 4
#> 5: 120.133542 41.95489 5
#> 6: 35.742211 41.32776 6
tail(y)
#> word word_id count_partition count_coi count_ref
#> <char> <int> <int> <int> <int>
#> 1: reguläre 2987 1 1 0
#> 2: sozialversicherungspflichtigen 13356 1 1 0
#> 3: tete 1440 1 1 0
#> 4: unsichere 12856 1 3 0
#> 5: verhelfen 13422 1 1 0
#> 6: Überweisungsvorschlag 2473 7 7 0
#> exp_coi exp_ref ll rank_ll
#> <num> <num> <num> <int>
#> 1: 0.007160807 0.9928392 NA 518
#> 2: 0.007160807 0.9928392 NA 519
#> 3: 0.007160807 0.9928392 NA 520
#> 4: 0.021482422 2.9785176 NA 521
#> 5: 0.007160807 0.9928392 NA 522
#> 6: 0.050125652 6.9498743 NA 523
nrow(y)
#> [1] 523
ncol(y)
#> [1] 9
dim(y)
#> [1] 523 9
colnames(y)
#> [1] "word" "word_id" "count_partition" "count_coi"
#> [5] "count_ref" "exp_coi" "exp_ref" "ll"
#> [9] "rank_ll"
# Use brackets for indexing
if (FALSE) { # \dontrun{
y[1:25]
y[,c("word", "ll")]
y[1:25, "word"]
y[1:25][["word"]]
y[which(y[["word"]] %in% c("Arbeit", "Sozial"))]
y[ y[["word"]] %in% c("Arbeit", "Sozial") ]
} # }
sc <- partition("GERMAPARLMINI", speaker = "Angela Dorothea Merkel")
#> ... get cpos and strucs
cnt <- count(sc, p_attribute = c("word", "pos"))
cnt_min <- subset(cnt, pos %in% c("NN", "ADJA"))
cnt_min <- subset(cnt, pos == "NE")
use(pkg = "RcppCWB", corpus = "REUTERS")
#> ✔ corpus loaded: REUTERS
# Get statistics in textstat object as data.table
count_dt <- corpus("REUTERS") %>%
subset(grep("saudi-arabia", places)) %>%
count(p_attribute = "word") %>%
as.data.table()
# textstat objects stored as *.rds files should be loaded using restore().
# Before moving to examples, this is a brief technical dip why this is
# recommended: If we load the *.rds file with readRDS(), the data.table in
# the slot 'stat' will have the pointer '0x0', and the data.table cannot be
# augmented without having been copied previously.
k <- kwic("REUTERS", query = "oil")
kwicfile <- tempfile(fileext = ".rds")
saveRDS(k, file = kwicfile)
problemprone <- readRDS(file = kwicfile)
problemprone@stat[, "newcol" := TRUE]
#> Key: <match_id>
#> match_id left node
#> <int> <char> <char>
#> 1: 1 its contract prices for crude oil
#> 2: 2 in the light of falling oil
#> 3: 3 prices and a weak crude oil
#> 4: 4 in a line of U.S oil
#> 5: 5 last two days citing weak oil
#> 6: 6 halt the current slide in oil
#> 7: 7 current slide in oil prices oil
#> 8: 8 said The movement to higher oil
#> 9: 9 Research Associates CERA Analysts and oil
#> 10: 10 problem OPEC faces is excess oil
#> 11: 11 excess oil supply in world oil
#> 12: 12 that way said Paul Mlotok oil
#> 13: 13 to regain the initiative in oil
#> 14: 14 for one of the major oil
#> 15: 15 winter demand to sell their oil
#> 16: 16 that the demand for OPEC oil
#> 17: 17 production Demand for their OPEC oil
#> 18: 18 it will pay for crude oil
#> 19: 19 Canada last changed its crude oil
#> 20: 20 for all grades of crude oil
#> 21: 21 emergency OPEC meeting to review oil
#> 22: 22 after recent weakness in world oil
#> 23: 23 Traders and analysts in international oil
#> 24: 24 a challenge to any international oil
#> 25: 25 had guaranteed markets for its oil
#> 26: 26 committee Referring to pressure by oil
#> 27: 27 new report To counter falling oil
#> 28: 28 to boost exports outside the oil
#> 29: 29 year s fall in world oil
#> 30: 30 industry Growth in the non oil
#> 31: 31 halting the current decline in oil
#> 32: 32 producers had had difficulty selling oil
#> 33: 33 03 yesterday REUTER The Gulf oil
#> 34: 34 last year's decline in world oil
#> 35: 35 budget due to uncertainty surrounding oil
#> 36: 36 was also no projection for oil
#> 37: 37 expectations of positive signs regarding oil
#> 38: 38 OPEC accord to boost world oil
#> 39: 39 recent fall in free market oil
#> 40: 40 it will never sell its oil
#> 41: 41 a year of turmoil on oil
#> 42: 42 was shown clearly in the oil
#> 43: 43 Traders and analysts in international oil
#> 44: 44 over producing REUTER Saudi crude oil
#> 45: 45 mln bpd in January Gulf oil
#> 46: 46 December to back new official oil
#> 47: 47 below Opec levels Saudi Arabian oil
#> 48: 48 export refineries They put crude oil
#> 49: 49 200,000 bpd each REUTER Deputy oil
#> 50: 50 to discuss coordination of crude oil
#> 51: 51 by Gulf Cooperation Council GCC oil
#> 52: 52 each other market their crude oil
#> 53: 53 OPEC accord to boost world oil
#> 54: 54 recent fall in free market oil
#> 55: 55 it will never sell its oil
#> 56: 56 agreed to cut its total oil
#> 57: 57 dollars a barrel Reuter Kuwait's oil
#> 58: 58 the recent weakness in world oil
#> 59: 59 of crude daily bpd Crude oil
#> 60: 60 sharply last week as international oil
#> 61: 61 was closed when a Cypriot oil
#> 62: 62 He said there was no oil
#> 63: 63 the high tide After delivering oil
#> 64: 64 and future impact of low oil
#> 65: 65 oil prices on the domestic oil
#> 66: 66 called for new research for oil
#> 67: 67 that such moves as increasing oil
#> 68: 68 and future impact of low oil
#> 69: 69 oil prices on the domestic oil
#> 70: 70 called for new research for oil
#> 71: 71 full economic benefits of cheap oil
#> 72: 72 that such moves as increasing oil
#> 73: 73 its posted prices for crude oil
#> 74: 74 West Coast grades of crude oil
#> 75: 75 April one NYMEX will allow oil
#> 76: 76 This will change the way oil
#> 77: 77 serve the industry because the oil
#> 78: 78 CFTC spokeswoman Reuter Argentine crude oil
#> match_id left node
#> <int> <char> <char>
#> right newcol
#> <char> <lgcl>
#> 1: by 1.50 dlrs a barrel TRUE
#> 2: product prices and a weak TRUE
#> 3: market a company spokeswoman said TRUE
#> 4: companies that have cut its TRUE
#> 5: markets Reuter OPEC may be TRUE
#> 6: prices oil industry analysts said TRUE
#> 7: industry analysts said The movement TRUE
#> 8: prices was never to be TRUE
#> 9: industry sources said the problem TRUE
#> 10: supply in world oil markets TRUE
#> 11: markets OPEC's problem is not TRUE
#> 12: analyst with Salomon Brothers Inc TRUE
#> 13: prices But some other analysts TRUE
#> 14: companies who spoke on condition TRUE
#> 15: but in late March and TRUE
#> 16: has been rising through the TRUE
#> 17: is clearly above 15.8 mln TRUE
#> 18: 64 Canadian cts a barrel TRUE
#> 19: postings on Feb 19 Reuter TRUE
#> 20: one dlr a barrel effective TRUE
#> 21: policies after recent weakness in TRUE
#> 22: prices Sheikh Ali al Khalifa TRUE
#> 23: markets estimate OPEC is producing TRUE
#> 24: company that declared Kuwait sold TRUE
#> 25: because of its local and TRUE
#> 26: companies on OPEC members in TRUE
#> 27: revenues the government has launched TRUE
#> 28: sector and attract new investment TRUE
#> 29: prices which forced it to TRUE
#> 30: sector is low because of TRUE
#> 31: prices Oil industry sources said TRUE
#> 32: at official OPEC prices but TRUE
#> 33: state of Qatar recovering slightly TRUE
#> 34: prices announced its first budget TRUE
#> 35: revenues Sheikh Abdul Aziz said TRUE
#> 36: revenue Qatar an OPEC member TRUE
#> 37: price trends foremost among them TRUE
#> 38: prices and stabilise the market TRUE
#> 39: prices Nazer said Saudi Arabia TRUE
#> 40: at prices below the pronounced TRUE
#> 41: markets which saw prices slump TRUE
#> 42: market He said contacts among TRUE
#> 43: markets estimate OPEC is producing TRUE
#> 44: output last month fell to TRUE
#> 45: sources said They said exports TRUE
#> 46: prices averaging 18 dlrs a TRUE
#> 47: minister Hisham Nazer yesterday reiterated TRUE
#> 48: exports through Yanbu at 100,000 TRUE
#> 49: ministers from six Gulf Arab TRUE
#> 50: marketing the official Emirates news TRUE
#> 51: ministers to help each other TRUE
#> 52: Four of the GCC states TRUE
#> 53: prices and stabilize the market TRUE
#> 54: prices Nazer said Saudi Arabia TRUE
#> 55: at prices below the pronounced TRUE
#> 56: output ceiling by 7.25 pct TRUE
#> 57: minister said in a newspaper TRUE
#> 58: prices Sheikh Ali al Khalifa TRUE
#> 59: prices fell sharply last week TRUE
#> 60: traders and analysts estimated the TRUE
#> 61: tanker Seapride II ran aground TRUE
#> 62: spill but the ship is TRUE
#> 63: to a refinery in Paulsboro TRUE
#> 64: prices on the domestic oil TRUE
#> 65: industry U.S policy now is TRUE
#> 66: exploration and development techniques It TRUE
#> 67: reserves and more exploration and TRUE
#> 68: prices on the domestic oil TRUE
#> 69: industry U.S policy now is TRUE
#> 70: exploration and development techniques It TRUE
#> 71: But the group did not TRUE
#> 72: reserves and more exploration and TRUE
#> 73: one to 1.50 dlrs a TRUE
#> 74: the company said Reuter The TRUE
#> 75: traders that do not hold TRUE
#> 76: is transacted in the real TRUE
#> 77: market does not close when TRUE
#> 78: production was down 10.8 pct TRUE
#> right newcol
#> <char> <lgcl>
"newcol" %in% colnames(problemprone@stat) # is FALSE!
#> [1] TRUE
attr(problemprone@stat, ".internal.selfref")
#> <pointer: 0x59bb7b066580>
identical(attr(problemprone@stat, ".internal.selfref"), new("externalptr"))
#> [1] FALSE
# Restore stored S4 object with copy of data.table in 'stat' slot
k <- kwic("REUTERS", query = "oil")
kwicfile <- tempfile(fileext = ".rds")
saveRDS(k, file = kwicfile)
k2 <- restore(kwicfile)
enrich(k2, s_attribute = "id")
"id" %in% colnames(k2) # is TRUE
#> [1] TRUE
k <- kwic("REUTERS", query = "oil")
rdata_file <- tempfile(fileext = ".RData")
save(k, file = rdata_file)
rm(k)
load(rdata_file)
k <- cp(k) # now it is possible to columns by reference
enrich(k, s_attribute = "id")
"id" %in% colnames(k)
#> [1] TRUE