Package 'xgboost'

Description

When it comes to serializing XGBoost models, it's possible to use R serializers such as save() or saveRDS() to serialize an XGBoost model object, but XGBoost also provides its own serializers with better compatibility guarantees, which allow loading said models in other language bindings of XGBoost.

Note that an xgb.Booster object (as produced by xgb.train(), see rest of the doc for objects produced by xgboost()), outside of its core components, might also keep:

• Additional model configuration (accessible through xgb.config()), which includes model fitting parameters like max_depth and runtime parameters like nthread. These are not necessarily useful for prediction/importance/plotting.

• Additional R specific attributes - e.g. results of callbacks, such as evaluation logs, which are kept as a data.table object, accessible through attributes(model)$evaluation_log if present.

The first one (configurations) does not have the same compatibility guarantees as the model itself, including attributes that are set and accessed through xgb.attributes() - that is, such configuration might be lost after loading the booster in a different XGBoost version, regardless of the serializer that was used. These are saved when using saveRDS(), but will be discarded if loaded into an incompatible XGBoost version. They are not saved when using XGBoost's serializers from its public interface including xgb.save() and xgb.save.raw().

The second ones (R attributes) are not part of the standard XGBoost model structure, and thus are not saved when using XGBoost's own serializers. These attributes are only used for informational purposes, such as keeping track of evaluation metrics as the model was fit, or saving the R call that produced the model, but are otherwise not used for prediction / importance / plotting / etc. These R attributes are only preserved when using R's serializers.

In addition to the regular xgb.Booster objects produced by xgb.train(), the function xgboost() produces objects with a different subclass xgboost (which inherits from xgb.Booster), which keeps other additional metadata as R attributes such as class names in classification problems, and which has a dedicated predict method that uses different defaults and takes different argument names. XGBoost's own serializers can work with this xgboost class, but as they do not keep R attributes, the resulting object, when deserialized, is downcasted to the regular xgb.Booster class (i.e. it loses the metadata, and the resulting object will use predict.xgb.Booster() instead of predict.xgboost()) - for these xgboost objects, saveRDS might thus be a better option if the extra functionalities are needed.

Note that XGBoost models in R starting from version ⁠2.1.0⁠ and onwards, and XGBoost models before version ⁠2.1.0⁠; have a very different R object structure and are incompatible with each other. Hence, models that were saved with R serializers like saveRDS() or save() before version ⁠2.1.0⁠ will not work with latter xgboost versions and vice versa. Be aware that the structure of R model objects could in theory change again in the future, so XGBoost's serializers should be preferred for long-term storage.

Furthermore, note that model objects from XGBoost might not be serializable with third-party R packages like qs or qs2.

Details

Use xgb.save() to save the XGBoost model as a stand-alone file. You may opt into the JSON format by specifying the JSON extension. To read the model back, use xgb.load().

Use xgb.save.raw() to save the XGBoost model as a sequence (vector) of raw bytes in a future-proof manner. Future releases of XGBoost will be able to read the raw bytes and re-construct the corresponding model. To read the model back, use xgb.load.raw(). The xgb.save.raw() function is useful if you would like to persist the XGBoost model as part of another R object.

Use saveRDS() if you require the R-specific attributes that a booster might have, such as evaluation logs or the model class xgboost instead of xgb.Booster, but note that future compatibility of such objects is outside XGBoost's control as it relies on R's serialization format (see e.g. the details section in serialize and save() from base R).

For more details and explanation about model persistence and archival, consult the page https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html.

Examples

data(agaricus.train, package = "xgboost")

bst <- xgb.train(
  data = xgb.DMatrix(agaricus.train$data, label = agaricus.train$label, nthread = 1),
  nrounds = 2,
  params = xgb.params(
    max_depth = 2,
    nthread = 2,
    objective = "binary:logistic"
  )
)

# Save as a stand-alone file; load it with xgb.load()
fname <- file.path(tempdir(), "xgb_model.ubj")
xgb.save(bst, fname)
bst2 <- xgb.load(fname)

# Save as a stand-alone file (JSON); load it with xgb.load()
fname <- file.path(tempdir(), "xgb_model.json")
xgb.save(bst, fname)
bst2 <- xgb.load(fname)

# Save as a raw byte vector; load it with xgb.load.raw()
xgb_bytes <- xgb.save.raw(bst)
bst2 <- xgb.load.raw(xgb_bytes)

# Persist XGBoost model as part of another R object
obj <- list(xgb_model_bytes = xgb.save.raw(bst), description = "My first XGBoost model")
# Persist the R object. Here, saveRDS() is okay, since it doesn't persist
# xgb.Booster directly. What's being persisted is the future-proof byte representation
# as given by xgb.save.raw().
fname <- file.path(tempdir(), "my_object.Rds")
saveRDS(obj, fname)
# Read back the R object
obj2 <- readRDS(fname)
# Re-construct xgb.Booster object from the bytes
bst2 <- xgb.load.raw(obj2$xgb_model_bytes)

Description

This data set is originally from the Mushroom data set, UCI Machine Learning Repository.

Usage

data(agaricus.test)

Format

A list containing a label vector, and a dgCMatrix object with 1611 rows and 126 variables

Details

It includes the following fields:

• label: The label for each record.

• data: A sparse Matrix of 'dgCMatrix' class with 126 columns.

References

https://archive.ics.uci.edu/ml/datasets/Mushroom

Bache, K. & Lichman, M. (2013). UCI Machine Learning Repository http://archive.ics.uci.edu/ml. Irvine, CA: University of California, School of Information and Computer Science.

Description

This data set is originally from the Mushroom data set, UCI Machine Learning Repository.

Usage

data(agaricus.train)

Format

A list containing a label vector, and a dgCMatrix object with 6513 rows and 127 variables

Details

It includes the following fields:

• label: The label for each record.

• data: A sparse Matrix of 'dgCMatrix' class with 126 columns.

References

https://archive.ics.uci.edu/ml/datasets/Mushroom

Bache, K. & Lichman, M. (2013). UCI Machine Learning Repository http://archive.ics.uci.edu/ml. Irvine, CA: University of California, School of Information and Computer Science.

Description

Extracts the coefficients from a 'gblinear' booster object, as produced by xgb.train() when using parameter booster="gblinear".

Note: this function will error out if passing a booster model which is not of "gblinear" type.

Usage

## S3 method for class 'xgb.Booster'
coef(object, ...)

Arguments

Value

The extracted coefficients:

• If there is only one coefficient per column in the data, will be returned as a vector, potentially containing the feature names if available, with the intercept as first column.

• If there is more than one coefficient per column in the data (e.g. when using objective="multi:softmax"), will be returned as a matrix with dimensions equal to ⁠[num_features, num_cols]⁠, with the intercepts as first row. Note that the column (classes in multi-class classification) dimension will not be named.

The intercept returned here will include the 'base_score' parameter (unlike the 'bias' or the last coefficient in the model dump, which doesn't have 'base_score' added to it), hence one should get the same values from calling predict(..., outputmargin = TRUE) and from performing a matrix multiplication with model.matrix(~., ...).

Be aware that the coefficients are obtained by first converting them to strings and back, so there will always be some very small lose of precision compared to the actual coefficients as used by predict.xgb.Booster.

Examples

library(xgboost)

data(mtcars)

y <- mtcars[, 1]
x <- as.matrix(mtcars[, -1])

dm <- xgb.DMatrix(data = x, label = y, nthread = 1)
params <- xgb.params(booster = "gblinear", nthread = 1)
model <- xgb.train(data = dm, params = params, nrounds = 2)
coef(model)

Description

Returns a vector of numbers of rows and of columns in an xgb.DMatrix.

Usage

## S3 method for class 'xgb.DMatrix'
dim(x)

Arguments

Details

Note: since nrow() and ncol() internally use dim(), they can also be directly used with an xgb.DMatrix object.

Examples

data(agaricus.train, package = "xgboost")

train <- agaricus.train
dtrain <- xgb.DMatrix(train$data, label = train$label, nthread = 2)

stopifnot(nrow(dtrain) == nrow(train$data))
stopifnot(ncol(dtrain) == ncol(train$data))
stopifnot(all(dim(dtrain) == dim(train$data)))

Description

Only column names are supported for xgb.DMatrix, thus setting of row names would have no effect and returned row names would be NULL.

Usage

## S3 method for class 'xgb.DMatrix'
dimnames(x)

## S3 replacement method for class 'xgb.DMatrix'
dimnames(x) <- value

Arguments

Details

Generic dimnames() methods are used by colnames(). Since row names are irrelevant, it is recommended to use colnames() directly.

Examples

data(agaricus.train, package = "xgboost")

train <- agaricus.train
dtrain <- xgb.DMatrix(train$data, label = train$label, nthread = 2)
dimnames(dtrain)
colnames(dtrain)
colnames(dtrain) <- make.names(1:ncol(train$data))
print(dtrain, verbose = TRUE)

Description

Get or set information of xgb.DMatrix and xgb.Booster objects

Usage

## S3 method for class 'xgb.Booster'
getinfo(object, name)

## S3 method for class 'xgb.Booster'
setinfo(object, name, info)

getinfo(object, name)

## S3 method for class 'xgb.DMatrix'
getinfo(object, name)

setinfo(object, name, info)

## S3 method for class 'xgb.DMatrix'
setinfo(object, name, info)

Arguments

Details

The name field can be one of the following for xgb.DMatrix:

• label

• weight

• base_margin

• label_lower_bound

• label_upper_bound

• group

• feature_type

• feature_name

• nrow

See the documentation for xgb.DMatrix() for more information about these fields.

For xgb.Booster, can be one of the following:

• feature_type

• feature_name

Note that, while 'qid' cannot be retrieved, it is possible to get the equivalent 'group' for a DMatrix that had 'qid' assigned.

Important: when calling setinfo(), the objects are modified in-place. See xgb.copy.Booster() for an idea of this in-place assignment works.

See the documentation for xgb.DMatrix() for possible fields that can be set (which correspond to arguments in that function).

Note that the following fields are allowed in the construction of an xgb.DMatrix but are not allowed here:

• data

• missing

• silent

• nthread

Value

For getinfo(), will return the requested field. For setinfo(), will always return value TRUE if it succeeds.

Examples

data(agaricus.train, package = "xgboost")

dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))

labels <- getinfo(dtrain, "label")
setinfo(dtrain, "label", 1 - labels)

labels2 <- getinfo(dtrain, "label")
stopifnot(all(labels2 == 1 - labels))
data(agaricus.train, package = "xgboost")

dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))

labels <- getinfo(dtrain, "label")
setinfo(dtrain, "label", 1 - labels)

labels2 <- getinfo(dtrain, "label")
stopifnot(all.equal(labels2, 1 - labels))

Description

Predict values on data based on XGBoost model.

Usage

## S3 method for class 'xgb.Booster'
predict(
  object,
  newdata,
  missing = NA,
  outputmargin = FALSE,
  predleaf = FALSE,
  predcontrib = FALSE,
  approxcontrib = FALSE,
  predinteraction = FALSE,
  training = FALSE,
  iterationrange = NULL,
  strict_shape = FALSE,
  avoid_transpose = FALSE,
  validate_features = FALSE,
  base_margin = NULL,
  ...
)

Arguments

Details

Note that iterationrange would currently do nothing for predictions from "gblinear", since "gblinear" doesn't keep its boosting history.

One possible practical applications of the predleaf option is to use the model as a generator of new features which capture non-linearity and interactions, e.g., as implemented in xgb.create.features().

Setting predcontrib = TRUE allows to calculate contributions of each feature to individual predictions. For "gblinear" booster, feature contributions are simply linear terms (feature_beta * feature_value). For "gbtree" booster, feature contributions are SHAP values (Lundberg 2017) that sum to the difference between the expected output of the model and the current prediction (where the hessian weights are used to compute the expectations). Setting approxcontrib = TRUE approximates these values following the idea explained in http://blog.datadive.net/interpreting-random-forests/.

With predinteraction = TRUE, SHAP values of contributions of interaction of each pair of features are computed. Note that this operation might be rather expensive in terms of compute and memory. Since it quadratically depends on the number of features, it is recommended to perform selection of the most important features first. See below about the format of the returned results.

The predict() method uses as many threads as defined in xgb.Booster object (all by default). If you want to change their number, assign a new number to nthread using xgb.model.parameters<-(). Note that converting a matrix to xgb.DMatrix() uses multiple threads too.

Value

A numeric vector or array, with corresponding dimensions depending on the prediction mode and on parameter strict_shape as follows:

If passing strict_shape=FALSE:

• For regression or binary classification: a vector of length nrows.

• For multi-class and multi-target objectives: a matrix of dimensions ⁠[nrows, ngroups]⁠.

  Note that objective variant multi:softmax defaults towards predicting most likely class (a vector nrows) instead of per-class probabilities.

• For predleaf: a matrix with one column per tree.

  For multi-class / multi-target, they will be arranged so that columns in the output will have the leafs from one group followed by leafs of the other group (e.g. order will be group1:feat1, group1:feat2, ..., group2:feat1, group2:feat2, ...).

  If there is more than one parallel tree (e.g. random forests), the parallel trees will be the last grouping in the resulting order, which will still be 2D.

• For predcontrib: when not multi-class / multi-target, a matrix with dimensions ⁠[nrows, nfeats+1]⁠. The last "+ 1" column corresponds to the baseline value.

  For multi-class and multi-target objectives, will be an array with dimensions ⁠[nrows, ngroups, nfeats+1]⁠.

  The contribution values are on the scale of untransformed margin (e.g., for binary classification, the values are log-odds deviations from the baseline).

• For predinteraction: when not multi-class / multi-target, the output is a 3D array of dimensions ⁠[nrows, nfeats+1, nfeats+1]⁠. The off-diagonal (in the last two dimensions) elements represent different feature interaction contributions. The array is symmetric w.r.t. the last two dimensions. The "+ 1" columns corresponds to the baselines. Summing this array along the last dimension should produce practically the same result as predcontrib = TRUE.

  For multi-class and multi-target, will be a 4D array with dimensions ⁠[nrows, ngroups, nfeats+1, nfeats+1]⁠

If passing strict_shape=TRUE, the result is always a matrix (if 2D) or array (if 3D or higher):

• For normal predictions, the dimension is ⁠[nrows, ngroups]⁠.

• For predcontrib=TRUE, the dimension is ⁠[nrows, ngroups, nfeats+1]⁠.

• For predinteraction=TRUE, the dimension is ⁠[nrows, ngroups, nfeats+1, nfeats+1]⁠.

• For predleaf=TRUE, the dimension is ⁠[nrows, niter, ngroups, num_parallel_tree]⁠.

If passing avoid_transpose=TRUE, then the dimensions in all cases will be in reverse order - for example, for predinteraction, they will be ⁠[nfeats+1, nfeats+1, ngroups, nrows]⁠ instead of ⁠[nrows, ngroups, nfeats+1, nfeats+1]⁠.

References

1. Scott M. Lundberg, Su-In Lee, "A Unified Approach to Interpreting Model Predictions", NIPS Proceedings 2017, https://arxiv.org/abs/1705.07874

2. Scott M. Lundberg, Su-In Lee, "Consistent feature attribution for tree ensembles", https://arxiv.org/abs/1706.06060

See Also

xgb.train()

Examples

## binary classification:

data(agaricus.train, package = "xgboost")
data(agaricus.test, package = "xgboost")

## Keep the number of threads to 2 for examples
nthread <- 2
data.table::setDTthreads(nthread)

train <- agaricus.train
test <- agaricus.test

bst <- xgb.train(
  data = xgb.DMatrix(train$data, label = train$label, nthread = 1),
  nrounds = 5,
  params = xgb.params(
    max_depth = 2,
    nthread = nthread,
    objective = "binary:logistic"
  )
)

# use all trees by default
pred <- predict(bst, test$data)
# use only the 1st tree
pred1 <- predict(bst, test$data, iterationrange = c(1, 1))

# Predicting tree leafs:
# the result is an nsamples X ntrees matrix
pred_leaf <- predict(bst, test$data, predleaf = TRUE)
str(pred_leaf)

# Predicting feature contributions to predictions:
# the result is an nsamples X (nfeatures + 1) matrix
pred_contr <- predict(bst, test$data, predcontrib = TRUE)
str(pred_contr)
# verify that contributions' sums are equal to log-odds of predictions (up to float precision):
summary(rowSums(pred_contr) - qlogis(pred))
# for the 1st record, let's inspect its features that had non-zero contribution to prediction:
contr1 <- pred_contr[1,]
contr1 <- contr1[-length(contr1)]    # drop intercept
contr1 <- contr1[contr1 != 0]        # drop non-contributing features
contr1 <- contr1[order(abs(contr1))] # order by contribution magnitude
old_mar <- par("mar")
par(mar = old_mar + c(0,7,0,0))
barplot(contr1, horiz = TRUE, las = 2, xlab = "contribution to prediction in log-odds")
par(mar = old_mar)


## multiclass classification in iris dataset:

lb <- as.numeric(iris$Species) - 1
num_class <- 3

set.seed(11)

bst <- xgb.train(
  data = xgb.DMatrix(as.matrix(iris[, -5], nthread = 1), label = lb),
  nrounds = 10,
  params = xgb.params(
    max_depth = 4,
    nthread = 2,
    subsample = 0.5,
    objective = "multi:softprob",
    num_class = num_class
  )
)

# predict for softmax returns num_class probability numbers per case:
pred <- predict(bst, as.matrix(iris[, -5]))
str(pred)
# convert the probabilities to softmax labels
pred_labels <- max.col(pred) - 1
# the following should result in the same error as seen in the last iteration
sum(pred_labels != lb) / length(lb)

# compare with predictions from softmax:
set.seed(11)

bst <- xgb.train(
  data = xgb.DMatrix(as.matrix(iris[, -5], nthread = 1), label = lb),
  nrounds = 10,
  params = xgb.params(
    max_depth = 4,
    nthread = 2,
    subsample = 0.5,
    objective = "multi:softmax",
    num_class = num_class
  )
)

pred <- predict(bst, as.matrix(iris[, -5]))
str(pred)
all.equal(pred, pred_labels)
# prediction from using only 5 iterations should result
# in the same error as seen in iteration 5:
pred5 <- predict(bst, as.matrix(iris[, -5]), iterationrange = c(1, 5))
sum(pred5 != lb) / length(lb)

Description

Predict values on data based on XGBoost model.

Usage

## S3 method for class 'xgboost'
predict(
  object,
  newdata,
  type = "response",
  base_margin = NULL,
  iteration_range = NULL,
  validate_features = TRUE,
  ...
)

Arguments

Value

Either a numeric vector (for 1D outputs), numeric matrix (for 2D outputs), numeric array (for 3D and higher), or factor (for class predictions). See documentation for parameter type for details about what the output type and shape will be.

Examples

data("ToothGrowth")
y <- ToothGrowth$supp
x <- ToothGrowth[, -2L]
model <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)
pred_prob <- predict(model, x[1:5, ], type = "response")
pred_raw <- predict(model, x[1:5, ], type = "raw")
pred_class <- predict(model, x[1:5, ], type = "class")

# Relationships between these
manual_probs <- 1 / (1 + exp(-pred_raw))
manual_class <- ifelse(manual_probs < 0.5, levels(y)[1], levels(y)[2])

# They should match up to numerical precision
round(pred_prob, 6) == round(manual_probs, 6)
pred_class == manual_class

Description

Print information about xgb.Booster.

Usage

## S3 method for class 'xgb.Booster'
print(x, ...)

Arguments

Value

The same x object, returned invisibly

Examples

data(agaricus.train, package = "xgboost")
train <- agaricus.train

bst <- xgb.train(
  data = xgb.DMatrix(train$data, label = train$label, nthread = 1),
  nrounds = 2,
  params = xgb.params(
    max_depth = 2,
    nthread = 2,
    objective = "binary:logistic"
  )
)

attr(bst, "myattr") <- "memo"

print(bst)

Description

Prints formatted results of xgb.cv().

Usage

## S3 method for class 'xgb.cv.synchronous'
print(x, verbose = FALSE, ...)

Arguments

Details

When not verbose, it would only print the evaluation results, including the best iteration (when available).

Examples

data(agaricus.train, package = "xgboost")

train <- agaricus.train
cv <- xgb.cv(
  data = xgb.DMatrix(train$data, label = train$label, nthread = 1),
  nfold = 5,
  nrounds = 2,
  params = xgb.params(
    max_depth = 2,
    nthread = 2,
    objective = "binary:logistic"
  )
)
print(cv)
print(cv, verbose = TRUE)

Description

Print information about xgb.DMatrix. Currently it displays dimensions and presence of info-fields and colnames.

Usage

## S3 method for class 'xgb.DMatrix'
print(x, verbose = FALSE, ...)

Arguments

Examples

data(agaricus.train, package = "xgboost")

dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))
dtrain

print(dtrain, verbose = TRUE)

Description

Prints basic properties of an XGBoost model object.

Usage

## S3 method for class 'xgboost'
print(x, ...)

Arguments

Value

Same object x, after printing its info.

Description

Returns the feature / variable / column names from a fitted booster object, which are set automatically during the call to xgb.train() from the DMatrix names, or which can be set manually through setinfo().

If the object doesn't have feature names, will return NULL.

It is equivalent to calling getinfo(object, "feature_name").

Usage

## S3 method for class 'xgb.Booster'
variable.names(object, ...)

Arguments

Description

These methods allow to manipulate the key-value attribute strings of an XGBoost model.

Usage

xgb.attr(object, name)

xgb.attr(object, name) <- value

xgb.attributes(object)

xgb.attributes(object) <- value

Arguments

Details

The primary purpose of XGBoost model attributes is to store some meta data about the model. Note that they are a separate concept from the object attributes in R. Specifically, they refer to key-value strings that can be attached to an XGBoost model, stored together with the model's binary representation, and accessed later (from R or any other interface). In contrast, any R attribute assigned to an R object of xgb.Booster class would not be saved by xgb.save() because an XGBoost model is an external memory object and its serialization is handled externally. Also, setting an attribute that has the same name as one of XGBoost's parameters wouldn't change the value of that parameter for a model. Use xgb.model.parameters<-() to set or change model parameters.

The ⁠xgb.attributes<-⁠ setter either updates the existing or adds one or several attributes, but it doesn't delete the other existing attributes.

Important: since this modifies the booster's C object, semantics for assignment here will differ from R's, as any object reference to the same booster will be modified too, while assignment of R attributes through ⁠attributes(model)$<attr> <- <value>⁠ will follow the usual copy-on-write R semantics (see xgb.copy.Booster() for an example of these behaviors).

Value

• xgb.attr() returns either a string value of an attribute or NULL if an attribute wasn't stored in a model.

• xgb.attributes() returns a list of all attributes stored in a model or NULL if a model has no stored attributes.

Examples

data(agaricus.train, package = "xgboost")
train <- agaricus.train

bst <- xgb.train(
  data = xgb.DMatrix(train$data, label = train$label, nthread = 1),
  nrounds = 2,
  params = xgb.params(
    max_depth = 2,
    nthread = 2,
    objective = "binary:logistic"
  )
)

xgb.attr(bst, "my_attribute") <- "my attribute value"
print(xgb.attr(bst, "my_attribute"))
xgb.attributes(bst) <- list(a = 123, b = "abc")

fname <- file.path(tempdir(), "xgb.ubj")
xgb.save(bst, fname)
bst1 <- xgb.load(fname)
print(xgb.attr(bst1, "my_attribute"))
print(xgb.attributes(bst1))

# deletion:
xgb.attr(bst1, "my_attribute") <- NULL
print(xgb.attributes(bst1))
xgb.attributes(bst1) <- list(a = NULL, b = NULL)
print(xgb.attributes(bst1))

Description

Constructor for defining the structure of callback functions that can be executed at different stages of model training (before / after training, before / after each boosting iteration).

Usage

xgb.Callback(
  cb_name = "custom_callback",
  env = new.env(),
  f_before_training = function(env, model, data, evals, begin_iteration, end_iteration)
    NULL,
  f_before_iter = function(env, model, data, evals, iteration) NULL,
  f_after_iter = function(env, model, data, evals, iteration, iter_feval) NULL,
  f_after_training = function(env, model, data, evals, iteration, final_feval,
    prev_cb_res) NULL
)

Arguments

Details

Arguments that will be passed to the supplied functions are as follows:

• env The same environment that is passed under argument env.

  It may be modified by the functions in order to e.g. keep tracking of what happens across iterations or similar.

  This environment is only used by the functions supplied to the callback, and will not be kept after the model fitting function terminates (see parameter f_after_training).

• model The booster object when using xgb.train(), or the folds when using xgb.cv().

  For xgb.cv(), folds are a list with a structure as follows:

  • dtrain: The training data for the fold (as an xgb.DMatrix object).

  • bst: Rhe xgb.Booster object for the fold.

  • evals: A list containing two DMatrices, with names train and test (test is the held-out data for the fold).

  • index: The indices of the hold-out data for that fold (base-1 indexing), from which the test entry in evals was obtained.

  This object should not be in-place modified in ways that conflict with the training (e.g. resetting the parameters for a training update in a way that resets the number of rounds to zero in order to overwrite rounds).

  Note that any R attributes that are assigned to the booster during the callback functions, will not be kept thereafter as the booster object variable is not re-assigned during training. It is however possible to set C-level attributes of the booster through xgb.attr() or xgb.attributes(), which should remain available for the rest of the iterations and after the training is done.

  For keeping variables across iterations, it's recommended to use env instead.

• data The data to which the model is being fit, as an xgb.DMatrix object.

  Note that, for xgb.cv(), this will be the full data, while data for the specific folds can be found in the model object.

• evals The evaluation data, as passed under argument evals to xgb.train().

  For xgb.cv(), this will always be NULL.

• begin_iteration Index of the first boosting iteration that will be executed (base-1 indexing).

  This will typically be '1', but when using training continuation, depending on the parameters for updates, boosting rounds will be continued from where the previous model ended, in which case this will be larger than 1.

• end_iteration Index of the last boostign iteration that will be executed (base-1 indexing, inclusive of this end).

  It should match with argument nrounds passed to xgb.train() or xgb.cv().

  Note that boosting might be interrupted before reaching this last iteration, for example by using the early stopping callback xgb.cb.early.stop().

• iteration Index of the iteration number that is being executed (first iteration will be the same as parameter begin_iteration, then next one will add +1, and so on).

• iter_feval Evaluation metrics for evals that were supplied, either determined by the objective, or by parameter custom_metric.

  For xgb.train(), this will be a named vector with one entry per element in evals, where the names are determined as 'evals name' + '-' + 'metric name' - for example, if evals contains an entry named "tr" and the metric is "rmse", this will be a one-element vector with name "tr-rmse".

  For xgb.cv(), this will be a 2d matrix with dimensions ⁠[length(evals), nfolds]⁠, where the row names will follow the same naming logic as the one-dimensional vector that is passed in xgb.train().

  Note that, internally, the built-in callbacks such as xgb.cb.print.evaluation summarize this table by calculating the row-wise means and standard deviations.

• final_feval The evaluation results after the last boosting round is executed (same format as iter_feval, and will be the exact same input as passed under iter_feval to the last round that is executed during model fitting).

• prev_cb_res Result from a previous run of a callback sharing the same name (as given by parameter cb_name) when conducting training continuation, if there was any in the booster R attributes.

  Sometimes, one might want to append the new results to the previous one, and this will be done automatically by the built-in callbacks such as xgb.cb.evaluation.log, which will append the new rows to the previous table.

  If no such previous callback result is available (which it never will when fitting a model from start instead of updating an existing model), this will be NULL.

  For xgb.cv(), which doesn't support training continuation, this will always be NULL.

The following names (cb_name values) are reserved for internal callbacks:

• print_evaluation

• evaluation_log

• reset_parameters

• early_stop

• save_model

• cv_predict

• gblinear_history

The following names are reserved for other non-callback attributes:

• names

• class

• call

• params

• niter

• nfeatures

• folds

When using the built-in early stopping callback (xgb.cb.early.stop), said callback will always be executed before the others, as it sets some booster C-level attributes that other callbacks might also use. Otherwise, the order of execution will match with the order in which the callbacks are passed to the model fitting function.

Value

An xgb.Callback object, which can be passed to xgb.train() or xgb.cv().

See Also

Built-in callbacks:

• xgb.cb.print.evaluation

• xgb.cb.evaluation.log

• xgb.cb.reset.parameters

• xgb.cb.early.stop

• xgb.cb.save.model

• xgb.cb.cv.predict

• xgb.cb.gblinear.history

Examples

# Example constructing a custom callback that calculates
# squared error on the training data (no separate test set),
# and outputs the per-iteration results.
ssq_callback <- xgb.Callback(
  cb_name = "ssq",
  f_before_training = function(env, model, data, evals,
                               begin_iteration, end_iteration) {
    # A vector to keep track of a number at each iteration
    env$logs <- rep(NA_real_, end_iteration - begin_iteration + 1)
  },
  f_after_iter = function(env, model, data, evals, iteration, iter_feval) {
    # This calculates the sum of squared errors on the training data.
    # Note that this can be better done by passing an 'evals' entry,
    # but this demonstrates a way in which callbacks can be structured.
    pred <- predict(model, data)
    err <- pred - getinfo(data, "label")
    sq_err <- sum(err^2)
    env$logs[iteration] <- sq_err
    cat(
      sprintf(
        "Squared error at iteration %d: %.2f\n",
        iteration, sq_err
      )
    )

    # A return value of 'TRUE' here would signal to finalize the training
    return(FALSE)
  },
  f_after_training = function(env, model, data, evals, iteration,
                              final_feval, prev_cb_res) {
    return(env$logs)
  }
)

data(mtcars)

y <- mtcars$mpg
x <- as.matrix(mtcars[, -1])

dm <- xgb.DMatrix(x, label = y, nthread = 1)
model <- xgb.train(
  data = dm,
  params = xgb.params(objective = "reg:squarederror", nthread = 1),
  nrounds = 5,
  callbacks = list(ssq_callback)
)

# Result from 'f_after_iter' will be available as an attribute
attributes(model)$ssq

Description

This callback function saves predictions for all of the test folds, and also allows to save the folds' models.

Usage

xgb.cb.cv.predict(save_models = FALSE, outputmargin = FALSE)

Arguments

Details

Predictions are saved inside of the pred element, which is either a vector or a matrix, depending on the number of prediction outputs per data row. The order of predictions corresponds to the order of rows in the original dataset. Note that when a custom folds list is provided in xgb.cv(), the predictions would only be returned properly when this list is a non-overlapping list of k sets of indices, as in a standard k-fold CV. The predictions would not be meaningful when user-provided folds have overlapping indices as in, e.g., random sampling splits. When some of the indices in the training dataset are not included into user-provided folds, their prediction value would be NA.

Value

An xgb.Callback object, which can be passed to xgb.cv(), but not to xgb.train().

Description

This callback function determines the condition for early stopping.

The following attributes are assigned to the booster's object:

• best_score the evaluation score at the best iteration

• best_iteration at which boosting iteration the best score has occurred (0-based index for interoperability of binary models)

The same values are also stored as R attributes as a result of the callback, plus an additional attribute stopped_by_max_rounds which indicates whether an early stopping by the stopping_rounds condition occurred. Note that the best_iteration that is stored under R attributes will follow base-1 indexing, so it will be larger by '1' than the C-level 'best_iteration' that is accessed through xgb.attr() or xgb.attributes().

At least one dataset is required in evals for early stopping to work.

Usage

xgb.cb.early.stop(
  stopping_rounds,
  maximize = FALSE,
  metric_name = NULL,
  verbose = TRUE,
  save_best = FALSE
)

Arguments

Value

An xgb.Callback object, which can be passed to xgb.train() or xgb.cv().

Description

Callback for logging the evaluation history

Usage

xgb.cb.evaluation.log()

Details

This callback creates a table with per-iteration evaluation metrics (see parameters evals and custom_metric in xgb.train()).

Note: in the column names of the final data.table, the dash '-' character is replaced with the underscore '_' in order to make the column names more like regular R identifiers.

Value

An xgb.Callback object, which can be passed to xgb.train() or xgb.cv().

See Also

xgb.cb.print.evaluation

Description

Callback for collecting coefficients history of a gblinear booster

Usage

xgb.cb.gblinear.history(sparse = FALSE)

Arguments

Details

To keep things fast and simple, gblinear booster does not internally store the history of linear model coefficients at each boosting iteration. This callback provides a workaround for storing the coefficients' path, by extracting them after each training iteration.

This callback will construct a matrix where rows are boosting iterations and columns are feature coefficients (same order as when calling coef.xgb.Booster, with the intercept corresponding to the first column).

When there is more than one coefficient per feature (e.g. multi-class classification), the result will be reshaped into a vector where coefficients are arranged first by features and then by class (e.g. first 1 through N coefficients will be for the first class, then coefficients N+1 through 2N for the second class, and so on).

If the result has only one coefficient per feature in the data, then the resulting matrix will have column names matching with the feature names, otherwise (when there's more than one coefficient per feature) the names will be composed as 'column name' + ':' + 'class index' (so e.g. column 'c1' for class '0' will be named 'c1:0').

With xgb.train(), the output is either a dense or a sparse matrix. With with xgb.cv(), it is a list (one element per each fold) of such matrices.

Function xgb.gblinear.history provides an easy way to retrieve the outputs from this callback.

Value

An xgb.Callback object, which can be passed to xgb.train() or xgb.cv().

See Also

xgb.gblinear.history, coef.xgb.Booster.

Examples

#### Binary classification:

## Keep the number of threads to 1 for examples
nthread <- 1
data.table::setDTthreads(nthread)

# In the iris dataset, it is hard to linearly separate Versicolor class from the rest
# without considering the 2nd order interactions:
x <- model.matrix(Species ~ .^2, iris)[, -1]
colnames(x)
dtrain <- xgb.DMatrix(
  scale(x),
  label = 1 * (iris$Species == "versicolor"),
  nthread = nthread
)
param <- xgb.params(
  booster = "gblinear",
  objective = "reg:logistic",
  eval_metric = "auc",
  reg_lambda = 0.0003,
  reg_alpha = 0.0003,
  nthread = nthread
)

# For 'shotgun', which is a default linear updater, using high learning_rate values may result in
# unstable behaviour in some datasets. With this simple dataset, however, the high learning
# rate does not break the convergence, but allows us to illustrate the typical pattern of
# "stochastic explosion" behaviour of this lock-free algorithm at early boosting iterations.
bst <- xgb.train(
  c(param, list(learning_rate = 1.)),
  dtrain,
  evals = list(tr = dtrain),
  nrounds = 200,
  callbacks = list(xgb.cb.gblinear.history())
)

# Extract the coefficients' path and plot them vs boosting iteration number:
coef_path <- xgb.gblinear.history(bst)
matplot(coef_path, type = "l")

# With the deterministic coordinate descent updater, it is safer to use higher learning rates.
# Will try the classical componentwise boosting which selects a single best feature per round:
bst <- xgb.train(
  c(
    param,
    xgb.params(
      learning_rate = 0.8,
      updater = "coord_descent",
      feature_selector = "thrifty",
      top_k = 1
    )
  ),
  dtrain,
  evals = list(tr = dtrain),
  nrounds = 200,
  callbacks = list(xgb.cb.gblinear.history())
)
matplot(xgb.gblinear.history(bst), type = "l")
#  Componentwise boosting is known to have similar effect to Lasso regularization.
# Try experimenting with various values of top_k, learning_rate, nrounds,
# as well as different feature_selectors.

# For xgb.cv:
bst <- xgb.cv(
  c(
    param,
    xgb.params(
      learning_rate = 0.8,
      updater = "coord_descent",
      feature_selector = "thrifty",
      top_k = 1
    )
  ),
  dtrain,
  nfold = 5,
  nrounds = 100,
  callbacks = list(xgb.cb.gblinear.history())
)
# coefficients in the CV fold #3
matplot(xgb.gblinear.history(bst)[[3]], type = "l")


#### Multiclass classification:
dtrain <- xgb.DMatrix(scale(x), label = as.numeric(iris$Species) - 1, nthread = nthread)

param <- xgb.params(
  booster = "gblinear",
  objective = "multi:softprob",
  num_class = 3,
  reg_lambda = 0.0003,
  reg_alpha = 0.0003,
  nthread = nthread
)

# For the default linear updater 'shotgun' it sometimes is helpful
# to use smaller learning_rate to reduce instability
bst <- xgb.train(
  c(param, list(learning_rate = 0.5)),
  dtrain,
  evals = list(tr = dtrain),
  nrounds = 50,
  callbacks = list(xgb.cb.gblinear.history())
)

# Will plot the coefficient paths separately for each class:
matplot(xgb.gblinear.history(bst, class_index = 0), type = "l")
matplot(xgb.gblinear.history(bst, class_index = 1), type = "l")
matplot(xgb.gblinear.history(bst, class_index = 2), type = "l")

# CV:
bst <- xgb.cv(
  c(param, list(learning_rate = 0.5)),
  dtrain,
  nfold = 5,
  nrounds = 70,
  callbacks = list(xgb.cb.gblinear.history(FALSE))
)
# 1st fold of 1st class
matplot(xgb.gblinear.history(bst, class_index = 0)[[1]], type = "l")

Description

The callback function prints the result of evaluation at every period iterations. The initial and the last iteration's evaluations are always printed.

Does not leave any attribute in the booster (see xgb.cb.evaluation.log for that).

Usage

xgb.cb.print.evaluation(period = 1, showsd = TRUE)

Arguments

Value

An xgb.Callback object, which can be passed to xgb.train() or xgb.cv().

See Also

xgb.Callback

Description

Callback for resetting booster parameters at each iteration

Usage

xgb.cb.reset.parameters(new_params)

Arguments

Details

Note that when training is resumed from some previous model, and a function is used to reset a parameter value, the nrounds argument in this function would be the the number of boosting rounds in the current training.

Does not leave any attribute in the booster.

Value

An xgb.Callback object, which can be passed to xgb.train() or xgb.cv().

Description

This callback function allows to save an xgb-model file, either periodically after each save_period's or at the end.

Does not leave any attribute in the booster.

Usage

xgb.cb.save.model(save_period = 0, save_name = "xgboost.ubj")

Arguments

Value

An xgb.Callback object, which can be passed to xgb.train(), but not to xgb.cv().

Description

Accessors for model parameters as JSON string

Usage

xgb.config(object)

xgb.config(object) <- value

Arguments

Details

Note that assignment is performed in-place on the booster C object, which unlike assignment of R attributes, doesn't follow typical copy-on-write semantics for assignment - i.e. all references to the same booster will also get updated.

See xgb.copy.Booster() for an example of this behavior.

Value

Parameters as a list.

Examples

data(agaricus.train, package = "xgboost")

## Keep the number of threads to 1 for examples
nthread <- 1
data.table::setDTthreads(nthread)
train <- agaricus.train

bst <- xgb.train(
  data = xgb.DMatrix(train$data, label = train$label, nthread = 1),
  nrounds = 2,
  params = xgb.params(
    max_depth = 2,
    nthread = nthread,
    objective = "binary:logistic"
  )
)

config <- xgb.config(bst)

Description

Creates a deep copy of an 'xgb.Booster' object, such that the C object pointer contained will be a different object, and hence functions like xgb.attr() will not affect the object from which it was copied.

Usage

xgb.copy.Booster(model)

Arguments

Value

A deep copy of model - it will be identical in every way, but C-level functions called on that copy will not affect the model variable.

Examples

library(xgboost)

data(mtcars)

y <- mtcars$mpg
x <- mtcars[, -1]

dm <- xgb.DMatrix(x, label = y, nthread = 1)

model <- xgb.train(
  data = dm,
  params = xgb.params(nthread = 1),
  nrounds = 3
)

# Set an arbitrary attribute kept at the C level
xgb.attr(model, "my_attr") <- 100
print(xgb.attr(model, "my_attr"))

# Just assigning to a new variable will not create
# a deep copy - C object pointer is shared, and in-place
# modifications will affect both objects
model_shallow_copy <- model
xgb.attr(model_shallow_copy, "my_attr") <- 333
# 'model' was also affected by this change:
print(xgb.attr(model, "my_attr"))

model_deep_copy <- xgb.copy.Booster(model)
xgb.attr(model_deep_copy, "my_attr") <- 444
# 'model' was NOT affected by this change
# (keeps previous value that was assigned before)
print(xgb.attr(model, "my_attr"))

# Verify that the new object was actually modified
print(xgb.attr(model_deep_copy, "my_attr"))

Description

May improve the learning by adding new features to the training data based on the decision trees from a previously learned model.

Usage

xgb.create.features(model, data)

Arguments

Details

This is the function inspired from the paragraph 3.1 of the paper:

Practical Lessons from Predicting Clicks on Ads at Facebook

(Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yan, xin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, Joaquin Quinonero Candela)

International Workshop on Data Mining for Online Advertising (ADKDD) - August 24, 2014

https://research.facebook.com/publications/practical-lessons-from-predicting-clicks-on-ads-at-facebook/.

Extract explaining the method:

"We found that boosted decision trees are a powerful and very convenient way to implement non-linear and tuple transformations of the kind we just described. We treat each individual tree as a categorical feature that takes as value the index of the leaf an instance ends up falling in. We use 1-of-K coding of this type of features.

For example, consider the boosted tree model in Figure 1 with 2 subtrees, where the first subtree has 3 leafs and the second 2 leafs. If an instance ends up in leaf 2 in the first subtree and leaf 1 in second subtree, the overall input to the linear classifier will be the binary vector ⁠[0, 1, 0, 1, 0]⁠, where the first 3 entries correspond to the leaves of the first subtree and last 2 to those of the second subtree.

...

We can understand boosted decision tree based transformation as a supervised feature encoding that converts a real-valued vector into a compact binary-valued vector. A traversal from root node to a leaf node represents a rule on certain features."

Value

A dgCMatrix matrix including both the original data and the new features.

Examples

data(agaricus.train, package = "xgboost")
data(agaricus.test, package = "xgboost")

dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))
dtest <- with(agaricus.test, xgb.DMatrix(data, label = label, nthread = 2))

param <- list(max_depth = 2, learning_rate = 1, objective = 'binary:logistic', nthread = 1)
nrounds = 4

bst <- xgb.train(params = param, data = dtrain, nrounds = nrounds)

# Model accuracy without new features
accuracy.before <- sum((predict(bst, agaricus.test$data) >= 0.5) == agaricus.test$label) /
                   length(agaricus.test$label)

# Convert previous features to one hot encoding
new.features.train <- xgb.create.features(model = bst, agaricus.train$data)
new.features.test <- xgb.create.features(model = bst, agaricus.test$data)

# learning with new features
new.dtrain <- xgb.DMatrix(
  data = new.features.train, label = agaricus.train$label, nthread = 1
)
new.dtest <- xgb.DMatrix(
  data = new.features.test, label = agaricus.test$label, nthread = 1
)
bst <- xgb.train(params = param, data = new.dtrain, nrounds = nrounds)

# Model accuracy with new features
accuracy.after <- sum((predict(bst, new.dtest) >= 0.5) == agaricus.test$label) /
                  length(agaricus.test$label)

# Here the accuracy was already good and is now perfect.
cat(paste("The accuracy was", accuracy.before, "before adding leaf features and it is now",
          accuracy.after, "!\n"))

Description

The cross validation function of xgboost.

Usage

xgb.cv(
  params = xgb.params(),
  data,
  nrounds,
  nfold,
  prediction = FALSE,
  showsd = TRUE,
  metrics = list(),
  objective = NULL,
  custom_metric = NULL,
  stratified = "auto",
  folds = NULL,
  train_folds = NULL,
  verbose = TRUE,
  print_every_n = 1L,
  early_stopping_rounds = NULL,
  maximize = NULL,
  callbacks = list(),
  ...
)

Arguments

Details

The original sample is randomly partitioned into nfold equal size subsamples.

Of the nfold subsamples, a single subsample is retained as the validation data for testing the model, and the remaining nfold - 1 subsamples are used as training data.

The cross-validation process is then repeated nrounds times, with each of the nfold subsamples used exactly once as the validation data.

All observations are used for both training and validation.

Adapted from https://en.wikipedia.org/wiki/Cross-validation_%28statistics%29

Value

An object of class 'xgb.cv.synchronous' with the following elements:

• call: Function call.

• params: Parameters that were passed to the xgboost library. Note that it does not capture parameters changed by the xgb.cb.reset.parameters() callback.

• evaluation_log: Evaluation history stored as a data.table with the first column corresponding to iteration number and the rest corresponding to the CV-based evaluation means and standard deviations for the training and test CV-sets. It is created by the xgb.cb.evaluation.log() callback.

• niter: Number of boosting iterations.

• nfeatures: Number of features in training data.

• folds: The list of CV folds' indices - either those passed through the folds parameter or randomly generated.

• best_iteration: Iteration number with the best evaluation metric value (only available with early stopping).

Plus other potential elements that are the result of callbacks, such as a list cv_predict with a sub-element pred when passing prediction = TRUE, which is added by the xgb.cb.cv.predict() callback (note that one can also pass it manually under callbacks with different settings, such as saving also the models created during cross validation); or a list early_stop which will contain elements such as best_iteration when using the early stopping callback (xgb.cb.early.stop()).

Examples

data(agaricus.train, package = "xgboost")

dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))

cv <- xgb.cv(
  data = dtrain,
  nrounds = 3,
  params = xgb.params(
    nthread = 2,
    max_depth = 3,
    objective = "binary:logistic"
  ),
  nfold = 5,
  metrics = list("rmse","auc")
)
print(cv)
print(cv, verbose = TRUE)

Description

Helper function to supply data in batches of a data iterator when constructing a DMatrix from external memory through xgb.ExtMemDMatrix() or through xgb.QuantileDMatrix.from_iterator().

This function is only meant to be called inside of a callback function (which is passed as argument to function xgb.DataIter() to construct a data iterator) when constructing a DMatrix through external memory - otherwise, one should call xgb.DMatrix() or xgb.QuantileDMatrix().

The object that results from calling this function directly is not like an xgb.DMatrix - i.e. cannot be used to train a model, nor to get predictions - only possible usage is to supply data to an iterator, from which a DMatrix is then constructed.

For more information and for example usage, see the documentation for xgb.ExtMemDMatrix().

Usage

xgb.DataBatch(
  data,
  label = NULL,
  weight = NULL,
  base_margin = NULL,
  feature_names = colnames(data),
  feature_types = NULL,
  group = NULL,
  qid = NULL,
  label_lower_bound = NULL,
  label_upper_bound = NULL,
  feature_weights = NULL
)

Arguments

Value

An object of class xgb.DataBatch, which is just a list containing the data and parameters passed here. It does not inherit from xgb.DMatrix.

See Also

xgb.DataIter(), xgb.ExtMemDMatrix().

Description

Interface to create a custom data iterator in order to construct a DMatrix from external memory.

This function is responsible for generating an R object structure containing callback functions and an environment shared with them.

The output structure from this function is then meant to be passed to xgb.ExtMemDMatrix(), which will consume the data and create a DMatrix from it by executing the callback functions.

For more information, and for a usage example, see the documentation for xgb.ExtMemDMatrix().

Usage

xgb.DataIter(env = new.env(), f_next, f_reset)

Arguments

Value

An xgb.DataIter object, containing the same inputs supplied here, which can then be passed to xgb.ExtMemDMatrix().

See Also

xgb.ExtMemDMatrix(), xgb.DataBatch().

Description

Construct an 'xgb.DMatrix' object from a given data source, which can then be passed to functions such as xgb.train() or predict().

Usage

xgb.DMatrix(
  data,
  label = NULL,
  weight = NULL,
  base_margin = NULL,
  missing = NA,
  silent = FALSE,
  feature_names = colnames(data),
  feature_types = NULL,
  nthread = NULL,
  group = NULL,
  qid = NULL,
  label_lower_bound = NULL,
  label_upper_bound = NULL,
  feature_weights = NULL,
  data_split_mode = "row",
  ...
)

xgb.QuantileDMatrix(
  data,
  label = NULL,
  weight = NULL,
  base_margin = NULL,
  missing = NA,
  feature_names = colnames(data),
  feature_types = NULL,
  nthread = NULL,
  group = NULL,
  qid = NULL,
  label_lower_bound = NULL,
  label_upper_bound = NULL,
  feature_weights = NULL,
  ref = NULL,
  max_bin = NULL
)

Arguments

Details

Function xgb.QuantileDMatrix() will construct a DMatrix with quantization for the histogram method already applied to it, which can be used to reduce memory usage (compared to using a a regular DMatrix first and then creating a quantization out of it) when using the histogram method (tree_method = "hist", which is the default algorithm), but is not usable for the sorted-indices method (tree_method = "exact"), nor for the approximate method (tree_method = "approx").

Note that DMatrix objects are not serializable through R functions such as saveRDS() or save(). If a DMatrix gets serialized and then de-serialized (for example, when saving data in an R session or caching chunks in an Rmd file), the resulting object will not be usable anymore and will need to be reconstructed from the original source of data.

Value

An 'xgb.DMatrix' object. If calling xgb.QuantileDMatrix, it will have additional subclass xgb.QuantileDMatrix.

Examples

data(agaricus.train, package = "xgboost")

## Keep the number of threads to 1 for examples
nthread <- 1
data.table::setDTthreads(nthread)
dtrain <- with(
  agaricus.train, xgb.DMatrix(data, label = label, nthread = nthread)
)
fname <- file.path(tempdir(), "xgb.DMatrix.data")
xgb.DMatrix.save(dtrain, fname)
dtrain <- xgb.DMatrix(fname, nthread = 1)

Description

Checks whether an xgb.DMatrix object has a given field assigned to it, such as weights, labels, etc.

Usage

xgb.DMatrix.hasinfo(object, info)

Arguments

See Also

xgb.DMatrix(), getinfo.xgb.DMatrix(), setinfo.xgb.DMatrix()

Examples

x <- matrix(1:10, nrow = 5)
dm <- xgb.DMatrix(x, nthread = 1)

# 'dm' so far does not have any fields set
xgb.DMatrix.hasinfo(dm, "label")

# Fields can be added after construction
setinfo(dm, "label", 1:5)
xgb.DMatrix.hasinfo(dm, "label")

Description

Save xgb.DMatrix object to binary file

Usage

xgb.DMatrix.save(dmatrix, fname)

Arguments

Examples

data(agaricus.train, package = "xgboost")

dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))
fname <- file.path(tempdir(), "xgb.DMatrix.data")
xgb.DMatrix.save(dtrain, fname)
dtrain <- xgb.DMatrix(fname, nthread = 1)

Description

Dump an XGBoost model in text format.

Usage

xgb.dump(
  model,
  fname = NULL,
  fmap = "",
  with_stats = FALSE,
  dump_format = c("text", "json", "dot"),
  ...
)

Arguments

Value

If fname is not provided or set to NULL the function will return the model as a character vector. Otherwise it will return TRUE.

Examples

data(agaricus.train, package = "xgboost")
data(agaricus.test, package = "xgboost")

train <- agaricus.train
test <- agaricus.test

bst <- xgb.train(
  data = xgb.DMatrix(train$data, label = train$label, nthread = 1),
  nrounds = 2,
  params = xgb.params(
    max_depth = 2,
    nthread = 2,
    objective = "binary:logistic"
  )
)

# save the model in file 'xgb.model.dump'
dump_path = file.path(tempdir(), 'model.dump')
xgb.dump(bst, dump_path, with_stats = TRUE)

# print the model without saving it to a file
print(xgb.dump(bst, with_stats = TRUE))

# print in JSON format:
cat(xgb.dump(bst, with_stats = TRUE, dump_format = "json"))

# plot first tree leveraging the 'dot' format
if (requireNamespace('DiagrammeR', quietly = TRUE)) {
  DiagrammeR::grViz(xgb.dump(bst, dump_format = "dot")[[1L]])
}

Description

Create a special type of XGBoost 'DMatrix' object from external data supplied by an xgb.DataIter() object, potentially passed in batches from a bigger set that might not fit entirely in memory.

The data supplied by the iterator is accessed on-demand as needed, multiple times, without being concatenated, but note that fields like 'label' will be concatenated from multiple calls to the data iterator.

For more information, see the guide 'Using XGBoost External Memory Version': https://xgboost.readthedocs.io/en/stable/tutorials/external_memory.html

Usage

xgb.ExtMemDMatrix(
  data_iterator,
  cache_prefix = tempdir(),
  missing = NA,
  nthread = NULL
)

Arguments

Details

Be aware that construction of external data DMatrices will cache data on disk in a compressed format, under the path supplied in cache_prefix.

External data is not supported for the exact tree method.

Value

An 'xgb.DMatrix' object, with subclass 'xgb.ExtMemDMatrix', in which the data is not held internally but accessed through the iterator when needed.

See Also

xgb.DataIter(), xgb.DataBatch(), xgb.QuantileDMatrix.from_iterator()

Examples

data(mtcars)

# This custom environment will be passed to the iterator
# functions at each call. It is up to the user to keep
# track of the iteration number in this environment.
iterator_env <- as.environment(
  list(
    iter = 0,
    x = mtcars[, -1],
    y = mtcars[, 1]
  )
)

# Data is passed in two batches.
# In this example, batches are obtained by subsetting the 'x' variable.
# This is not advantageous to do, since the data is already loaded in memory
# and can be passed in full in one go, but there can be situations in which
# only a subset of the data will fit in the computer's memory, and it can
# be loaded in batches that are accessed one-at-a-time only.
iterator_next <- function(iterator_env) {
  curr_iter <- iterator_env[["iter"]]
  if (curr_iter >= 2) {
    # there are only two batches, so this signals end of the stream
    return(NULL)
  }

  if (curr_iter == 0) {
    x_batch <- iterator_env[["x"]][1:16, ]
    y_batch <- iterator_env[["y"]][1:16]
  } else {
    x_batch <- iterator_env[["x"]][17:32, ]
    y_batch <- iterator_env[["y"]][17:32]
  }
  on.exit({
    iterator_env[["iter"]] <- curr_iter + 1
  })

  # Function 'xgb.DataBatch' must be called manually
  # at each batch with all the appropriate attributes,
  # such as feature names and feature types.
  return(xgb.DataBatch(data = x_batch, label = y_batch))
}

# This moves the iterator back to its beginning
iterator_reset <- function(iterator_env) {
  iterator_env[["iter"]] <- 0
}

data_iterator <- xgb.DataIter(
  env = iterator_env,
  f_next = iterator_next,
  f_reset = iterator_reset
)
cache_prefix <- tempdir()

# DMatrix will be constructed from the iterator's batches
dm <- xgb.ExtMemDMatrix(data_iterator, cache_prefix, nthread = 1)

# After construction, can be used as a regular DMatrix
params <- xgb.params(nthread = 1, objective = "reg:squarederror")
model <- xgb.train(data = dm, nrounds = 2, params = params)

# Predictions can also be called on it, and should be the same
# as if the data were passed differently.
pred_dm <- predict(model, dm)
pred_mat <- predict(model, as.matrix(mtcars[, -1]))

Description

A helper function to extract the matrix of linear coefficients' history from a gblinear model created while using the xgb.cb.gblinear.history callback (which must be added manually as by default it is not used).

Usage

xgb.gblinear.history(model, class_index = NULL)

Arguments

Details

Note that this is an R-specific function that relies on R attributes that are not saved when using XGBoost's own serialization functions like xgb.load() or xgb.load.raw().

In order for a serialized model to be accepted by this function, one must use R serializers such as saveRDS().

Value

For an xgb.train() result, a matrix (either dense or sparse) with the columns corresponding to iteration's coefficients and the rows corresponding to boosting iterations.

For an xgb.cv() result, a list of such matrices is returned with the elements corresponding to CV folds.

When there is more than one coefficient per feature (e.g. multi-class classification) and class_index is not provided, the result will be reshaped into a vector where coefficients are arranged first by features and then by class (e.g. first 1 through N coefficients will be for the first class, then coefficients N+1 through 2N for the second class, and so on).

See Also

xgb.cb.gblinear.history, coef.xgb.Booster.

Description

Get DMatrix Data

Usage

xgb.get.DMatrix.data(dmat)

Arguments

Value

The data held in the DMatrix, as a sparse CSR matrix (class dgRMatrix from package Matrix). If it had feature names, these will be added as column names in the output.

Description

Get Number of Non-Missing Entries in DMatrix

Usage

xgb.get.DMatrix.num.non.missing(dmat)

Arguments

Value

The number of non-missing entries in the DMatrix.

Description

Get the quantile cuts (a.k.a. borders) from an xgb.DMatrix that has been quantized for the histogram method (tree_method = "hist").

These cuts are used in order to assign observations to bins - i.e. these are ordered boundaries which are used to determine assignment condition ⁠border_low < x < border_high⁠. As such, the first and last bin will be outside of the range of the data, so as to include all of the observations there.

If a given column has 'n' bins, then there will be 'n+1' cuts / borders for that column, which will be output in sorted order from lowest to highest.

Different columns can have different numbers of bins according to their range.

Usage

xgb.get.DMatrix.qcut(dmat, output = c("list", "arrays"))

Arguments

Value

The quantile cuts, in the format specified by parameter output.

Examples

data(mtcars)

y <- mtcars$mpg
x <- as.matrix(mtcars[, -1])
dm <- xgb.DMatrix(x, label = y, nthread = 1)

# DMatrix is not quantized right away, but will be once a hist model is generated
model <- xgb.train(
  data = dm,
  params = xgb.params(tree_method = "hist", max_bin = 8, nthread = 1),
  nrounds = 3
)

# Now can get the quantile cuts
xgb.get.DMatrix.qcut(dm)

Description

Get number of boosting in a fitted booster

Usage

xgb.get.num.boosted.rounds(model)

## S3 method for class 'xgb.Booster'
length(x)

Arguments

Details

Note that setting booster parameters related to training continuation / updates through xgb.model.parameters<-() will reset the number of rounds to zero.

Value

The number of rounds saved in the model as an integer.

Description

Visualizes distributions related to the depth of tree leaves.

• xgb.plot.deepness() uses base R graphics, while

• xgb.ggplot.deepness() uses "ggplot2".

Usage

xgb.ggplot.deepness(
  model = NULL,
  which = c("2x1", "max.depth", "med.depth", "med.weight")
)

xgb.plot.deepness(
  model = NULL,
  which = c("2x1", "max.depth", "med.depth", "med.weight"),
  plot = TRUE,
  ...
)

Arguments

Details

When which = "2x1", two distributions with respect to the leaf depth are plotted on top of each other:

1. The distribution of the number of leaves in a tree model at a certain depth.

2. The distribution of the average weighted number of observations ("cover") ending up in leaves at a certain depth.

Those could be helpful in determining sensible ranges of the max_depth and min_child_weight parameters.

When which = "max.depth" or which = "med.depth", plots of either maximum or median depth per tree with respect to the tree number are created.

Finally, which = "med.weight" allows to see how a tree's median absolute leaf weight changes through the iterations.

These functions have been inspired by the blog post https://github.com/aysent/random-forest-leaf-visualization.

Value

The return value of the two functions is as follows:

• xgb.plot.deepness(): A "data.table" (invisibly). Each row corresponds to a terminal leaf in the model. It contains its information about depth, cover, and weight (used in calculating predictions). If plot = TRUE, also a plot is shown.

• xgb.ggplot.deepness(): When which = "2x1", a list of two "ggplot" objects, and a single "ggplot" object otherwise.

See Also

xgb.train() and xgb.model.dt.tree().

Examples

data(agaricus.train, package = "xgboost")
## Keep the number of threads to 2 for examples
nthread <- 2
data.table::setDTthreads(nthread)

## Change max_depth to a higher number to get a more significant result
model <- xgboost(
  agaricus.train$data, factor(agaricus.train$label),
  nrounds = 50,
  max_depth = 6,
  nthreads = nthread,
  subsample = 0.5,
  min_child_weight = 2
)

xgb.plot.deepness(model)
xgb.ggplot.deepness(model)

xgb.plot.deepness(
  model, which = "max.depth", pch = 16, col = rgb(0, 0, 1, 0.3), cex = 2
)

xgb.plot.deepness(
  model, which = "med.weight", pch = 16, col = rgb(0, 0, 1, 0.3), cex = 2
)

Description

Represents previously calculated feature importance as a bar graph.

• xgb.plot.importance() uses base R graphics, while

• xgb.ggplot.importance() uses "ggplot".

Usage

xgb.ggplot.importance(
  importance_matrix = NULL,
  top_n = NULL,
  measure = NULL,
  rel_to_first = FALSE,
  n_clusters = seq_len(10),
  ...
)

xgb.plot.importance(
  importance_matrix = NULL,
  top_n = NULL,
  measure = NULL,
  rel_to_first = FALSE,
  left_margin = 10,
  cex = NULL,
  plot = TRUE,
  ...
)

Arguments

Details

The graph represents each feature as a horizontal bar of length proportional to the importance of a feature. Features are sorted by decreasing importance. It works for both "gblinear" and "gbtree" models.

When rel_to_first = FALSE, the values would be plotted as in importance_matrix. For a "gbtree" model, that would mean being normalized to the total of 1 ("what is feature's importance contribution relative to the whole model?"). For linear models, rel_to_first = FALSE would show actual values of the coefficients. Setting rel_to_first = TRUE allows to see the picture from the perspective of "what is feature's importance contribution relative to the most important feature?"

The "ggplot" backend performs 1-D clustering of the importance values, with bar colors corresponding to different clusters having similar importance values.

Value

The return value depends on the function:

• xgb.plot.importance(): Invisibly, a "data.table" with n_top features sorted by importance. If plot = TRUE, the values are also plotted as barplot.

• xgb.ggplot.importance(): A customizable "ggplot" object. E.g., to change the title, set + ggtitle("A GRAPH NAME").

See Also

graphics::barplot()

Examples

data(agaricus.train)

## Keep the number of threads to 2 for examples
nthread <- 2
data.table::setDTthreads(nthread)

model <- xgboost(
  agaricus.train$data, factor(agaricus.train$label),
  nrounds = 2,
  max_depth = 3,
  nthreads = nthread
)

importance_matrix <- xgb.importance(model)
xgb.plot.importance(
  importance_matrix, rel_to_first = TRUE, xlab = "Relative importance"
)

gg <- xgb.ggplot.importance(
  importance_matrix, measure = "Frequency", rel_to_first = TRUE
)
gg
gg + ggplot2::ylab("Frequency")

Description

Visualizes SHAP contributions of different features.

Usage

xgb.ggplot.shap.summary(
  data,
  shap_contrib = NULL,
  features = NULL,
  top_n = 10,
  model = NULL,
  trees = NULL,
  target_class = NULL,
  approxcontrib = FALSE,
  subsample = NULL
)

xgb.plot.shap.summary(
  data,
  shap_contrib = NULL,
  features = NULL,
  top_n = 10,
  model = NULL,
  trees = NULL,
  target_class = NULL,
  approxcontrib = FALSE,
  subsample = NULL
)

Arguments

Details

A point plot (each point representing one observation from data) is produced for each feature, with the points plotted on the SHAP value axis. Each point (observation) is coloured based on its feature value.

The plot allows to see which features have a negative / positive contribution on the model prediction, and whether the contribution is different for larger or smaller values of the feature. Inspired by the summary plot of https://github.com/shap/shap.

Value

A ggplot2 object.

See Also

xgb.plot.shap(), xgb.ggplot.shap.summary(), and the Python library https://github.com/shap/shap.

Examples

# See examples in xgb.plot.shap()

Description

Creates a data.table of feature importances.

Usage

xgb.importance(
  model = NULL,
  feature_names = getinfo(model, "feature_name"),
  trees = NULL
)

Arguments

Details

This function works for both linear and tree models.

For linear models, the importance is the absolute magnitude of linear coefficients. To obtain a meaningful ranking by importance for linear models, the features need to be on the same scale (which is also recommended when using L1 or L2 regularization).

Value

A data.table with the following columns:

For a tree model:

• Features: Names of the features used in the model.

• Gain: Fractional contribution of each feature to the model based on the total gain of this feature's splits. Higher percentage means higher importance.

• Cover: Metric of the number of observation related to this feature.

• Frequency: Percentage of times a feature has been used in trees.

For a linear model:

• Features: Names of the features used in the model.

• Weight: Linear coefficient of this feature.

• Class: Class label (only for multiclass models). For objects of class xgboost (as produced by xgboost()), it will be a factor, while for objects of class xgb.Booster (as produced by xgb.train()), it will be a zero-based integer vector.

If feature_names is not provided and model doesn't have feature_names, the index of the features will be used instead. Because the index is extracted from the model dump (based on C++ code), it starts at 0 (as in C/C++ or Python) instead of 1 (usual in R).

Examples

# binary classification using "gbtree":
data("ToothGrowth")
x <- ToothGrowth[, c("len", "dose")]
y <- ToothGrowth$supp
model_tree_binary <- xgboost(
  x, y,
  nrounds = 5L,
  nthreads = 1L,
  booster = "gbtree",
  max_depth = 2L
)
xgb.importance(model_tree_binary)

# binary classification using "gblinear":
model_tree_linear <- xgboost(
  x, y,
  nrounds = 5L,
  nthreads = 1L,
  booster = "gblinear",
  learning_rate = 0.3
)
xgb.importance(model_tree_linear)

# multi-class classification using "gbtree":
data("iris")
x <- iris[, c("Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width")]
y <- iris$Species
model_tree_multi <- xgboost(
  x, y,
  nrounds = 5L,
  nthreads = 1L,
  booster = "gbtree",
  max_depth = 3
)
# all classes clumped together:
xgb.importance(model_tree_multi)
# inspect importances separately for each class:
num_classes <- 3L
nrounds <- 5L
xgb.importance(
  model_tree_multi, trees = seq(from = 1, by = num_classes, length.out = nrounds)
)
xgb.importance(
  model_tree_multi, trees = seq(from = 2, by = num_classes, length.out = nrounds)
)
xgb.importance(
  model_tree_multi, trees = seq(from = 3, by = num_classes, length.out = nrounds)
)

# multi-class classification using "gblinear":
model_linear_multi <- xgboost(
  x, y,
  nrounds = 5L,
  nthreads = 1L,
  booster = "gblinear",
  learning_rate = 0.2
)
xgb.importance(model_linear_multi)

Description

Checks whether two booster objects refer to the same underlying C object.

Usage

xgb.is.same.Booster(obj1, obj2)

Arguments

Details

As booster objects (as returned by e.g. xgb.train()) contain an R 'externalptr' object, they don't follow typical copy-on-write semantics of other R objects - that is, if one assigns a booster to a different variable and modifies that new variable through in-place methods like xgb.attr<-(), the modification will be applied to both the old and the new variable, unlike typical R assignments which would only modify the latter.

This function allows checking whether two booster objects share the same 'externalptr', regardless of the R attributes that they might have.

In order to duplicate a booster in such a way that the copy wouldn't share the same 'externalptr', one can use function xgb.copy.Booster().

Value

Either TRUE or FALSE according to whether the two boosters share the underlying C object.

See Also

xgb.copy.Booster()

Examples

library(xgboost)

data(mtcars)

y <- mtcars$mpg
x <- as.matrix(mtcars[, -1])

model <- xgb.train(
  params = xgb.params(nthread = 1),
  data = xgb.DMatrix(x, label = y, nthread = 1),
  nrounds = 3
)

model_shallow_copy <- model
xgb.is.same.Booster(model, model_shallow_copy) # same C object

model_deep_copy <- xgb.copy.Booster(model)
xgb.is.same.Booster(model, model_deep_copy) # different C objects

# In-place assignments modify all references,
# but not full/deep copies of the booster
xgb.attr(model_shallow_copy, "my_attr") <- 111
xgb.attr(model, "my_attr") # gets modified
xgb.attr(model_deep_copy, "my_attr") # doesn't get modified

Description

Load XGBoost model from binary model file.

Usage

xgb.load(modelfile)

Arguments

Details

The input file is expected to contain a model saved in an XGBoost model format using either xgb.save() in R, or using some appropriate methods from other XGBoost interfaces. E.g., a model trained in Python and saved from there in XGBoost format, could be loaded from R.

Note: a model saved as an R object has to be loaded using corresponding R-methods, not by xgb.load().

Value

An object of xgb.Booster class.

See Also

xgb.save()

Examples

data(agaricus.train, package = "xgboost")
data(agaricus.test, package = "xgboost")

## Keep the number of threads to 1 for examples
nthread <- 1
data.table::setDTthreads(nthread)

train <- agaricus.train
test <- agaricus.test

bst <- xgb.train(
  data = xgb.DMatrix(train$data, label = train$label, nthread = 1),
  nrounds = 2,
  params = xgb.params(
    max_depth = 2,
    nthread = nthread,
    objective = "binary:logistic"
  )
)

fname <- file.path(tempdir(), "xgb.ubj")
xgb.save(bst, fname)
bst <- xgb.load(fname)

Description

User can generate raw memory buffer by calling xgb.save.raw().

Usage

xgb.load.raw(buffer)

Arguments

Description

Parse a boosted tree model text dump into a data.table structure.

Usage

xgb.model.dt.tree(model, trees = NULL, use_int_id = FALSE, ...)

Arguments

Details

Note that this function does not work with models that were fitted to categorical data, and is only applicable to tree-based boosters (not gblinear).

Value

A data.table with detailed information about tree nodes. It has the following columns:

• Tree: integer ID of a tree in a model (zero-based index).

• Node: integer ID of a node in a tree (zero-based index).

• ID: character identifier of a node in a model (only when use_int_id = FALSE).

• Feature: for a branch node, a feature ID or name (when available); for a leaf node, it simply labels it as "Leaf".

• Split: location of the split for a branch node (split condition is always "less than").

• Yes: ID of the next node when the split condition is met.

• No: ID of the next node when the split condition is not met.

• Missing: ID of the next node when the branch value is missing.

• Gain: either the split gain (change in loss) or the leaf value.

• Cover: metric related to the number of observations either seen by a split or collected by a leaf during training.

When use_int_id = FALSE, columns "Yes", "No", and "Missing" point to model-wide node identifiers in the "ID" column. When use_int_id = TRUE, those columns point to node identifiers from the corresponding trees in the "Node" column.

Examples

# Basic use:

data(agaricus.train, package = "xgboost")
## Keep the number of threads to 1 for examples
nthread <- 1
data.table::setDTthreads(nthread)

bst <- xgb.train(
  data = xgb.DMatrix(agaricus.train$data, label = agaricus.train$label, nthread = 1),
  nrounds = 2,
  params = xgb.params(
    max_depth = 2,
    nthread = nthread,
    objective = "binary:logistic"
  )
)

# This bst model already has feature_names stored with it, so those would be used when
# feature_names is not set:
dt <- xgb.model.dt.tree(bst)

# How to match feature names of splits that are following a current 'Yes' branch:
merge(
  dt,
  dt[, .(ID, Y.Feature = Feature)], by.x = "Yes", by.y = "ID", all.x = TRUE
)[
  order(Tree, Node)
]

Description

Only the setter for XGBoost parameters is currently implemented.

Usage

xgb.model.parameters(object) <- value

Arguments

Details

Just like xgb.attr(), this function will make in-place modifications on the booster object which do not follow typical R assignment semantics - that is, all references to the same booster will also be updated, unlike assingment of R attributes which follow copy-on-write semantics.

See xgb.copy.Booster() for an example of this behavior.

Be aware that setting parameters of a fitted booster related to training continuation / updates will reset its number of rounds indicator to zero.

Value

The same booster object, which gets modified in-place.

Examples

data(agaricus.train, package = "xgboost")

train <- agaricus.train

bst <- xgb.train(
  data = xgb.DMatrix(train$data, label = train$label, nthread = 1),
  nrounds = 2,
  params = xgb.params(
    max_depth = 2,
    learning_rate = 1,
    nthread = 2,
    objective = "binary:logistic"
  )
)

xgb.model.parameters(bst) <- list(learning_rate = 0.1)

Description

Convenience function to generate a list of named XGBoost parameters, which can be passed as argument params to xgb.train(). See the online documentation for more details.

The purpose of this function is to enable IDE autocompletions and to provide in-package documentation for all the possible parameters that XGBoost accepts. The output from this function is just a regular R list containing the parameters that were set to non-default values. Note that this function will not perform any validation on the supplied arguments.

If passing NULL for a given parameter (the default for all of them), then the default value for that parameter will be used. Default values are automatically determined by the XGBoost core library upon calls to xgb.train() or xgb.cv(), and are subject to change over XGBoost library versions. Some of them might differ according to the booster type (e.g. defaults for regularization are different for linear and tree-based boosters).

Usage

xgb.params(
  objective = NULL,
  verbosity = NULL,
  nthread = NULL,
  seed = NULL,
  booster = NULL,
  eta = NULL,
  learning_rate = NULL,
  gamma = NULL,
  min_split_loss = NULL,
  max_depth = NULL,
  min_child_weight = NULL,
  max_delta_step = NULL,
  subsample = NULL,
  sampling_method = NULL,
  colsample_bytree = NULL,
  colsample_bylevel = NULL,
  colsample_bynode = NULL,
  lambda = NULL,
  reg_lambda = NULL,
  alpha = NULL,
  reg_alpha = NULL,
  tree_method = NULL,
  scale_pos_weight = NULL,
  updater = NULL,
  refresh_leaf = NULL,
  grow_policy = NULL,
  max_leaves = NULL,
  max_bin = NULL,
  num_parallel_tree = NULL,
  monotone_constraints = NULL,
  interaction_constraints = NULL,
  multi_strategy = NULL,
  base_score = NULL,
  eval_metric = NULL,
  seed_per_iteration = NULL,
  device = NULL,
  disable_default_eval_metric = NULL,
  use_rmm = NULL,
  max_cached_hist_node = NULL,
  extmem_single_page = NULL,
  max_cat_to_onehot = NULL,
  max_cat_threshold = NULL,
  sample_type = NULL,
  normalize_type = NULL,
  rate_drop = NULL,
  one_drop = NULL,
  skip_drop = NULL,
  feature_selector = NULL,
  top_k = NULL,
  num_class = NULL,
  tweedie_variance_power = NULL,
  huber_slope = NULL,
  quantile_alpha = NULL,
  aft_loss_distribution = NULL,
  lambdarank_pair_method = NULL,
  lambdarank_num_pair_per_sample = NULL,
  lambdarank_normalization = NULL,
  lambdarank_score_normalization = NULL,
  lambdarank_unbiased = NULL,
  lambdarank_bias_norm = NULL,
  ndcg_exp_gain = NULL
)

Arguments