murnmf {rnnmf}R Documentation

murnmf .

Description

Multiplicative update Non-negative matrix factorization with regularization.

Usage

murnmf(
  Y,
  L,
  R,
  W_0R = NULL,
  W_0C = NULL,
  lambda_1L = 0,
  lambda_1R = 0,
  lambda_2L = 0,
  lambda_2R = 0,
  gamma_2L = 0,
  gamma_2R = 0,
  epsilon = 1e-07,
  max_iterations = 1000L,
  min_xstep = 1e-09,
  on_iteration_end = NULL,
  verbosity = 0
)

Arguments

Y

an r \times c matrix to be decomposed. Should have non-negative elements; an error is thrown otherwise.

L

an r \times d matrix of the initial estimate of L. Should have non-negative elements; an error is thrown otherwise.

R

an d \times c matrix of the initial estimate of R. Should have non-negative elements; an error is thrown otherwise.

W_0R

the row space weighting matrix. This should be a positive definite non-negative symmetric r \times r matrix. If omitted, it defaults to the properly sized identity matrix.

W_0C

the column space weighting matrix. This should be a positive definite non-negative symmetric c \times c matrix. If omitted, it defaults to the properly sized identity matrix.

lambda_1L

the scalar \ell_1 penalty for the matrix L. Defaults to zero.

lambda_1R

the scalar \ell_1 penalty for the matrix R. Defaults to zero.

lambda_2L

the scalar \ell_2 penalty for the matrix L. Defaults to zero.

lambda_2R

the scalar \ell_2 penalty for the matrix R. Defaults to zero.

gamma_2L

the scalar \ell_2 penalty for non-orthogonality of the matrix L. Defaults to zero.

gamma_2R

the scalar \ell_2 penalty for non-orthogonality of the matrix R. Defaults to zero.

epsilon

the numerator clipping value.

max_iterations

the maximum number of iterations to perform.

min_xstep

the minimum L-infinity norm of the step taken. Once the step falls under this value, we terminate.

on_iteration_end

an optional function that is called at the end of each iteration. The function is called as on_iteration_end(iteration=iteration, Y=Y, L=L, R=R, Lstep=Lstep, Rstep=Rstep, ...)

verbosity

controls whether we print information to the console.

Details

This function uses multiplicative updates only, and may not optimize the nominal objective. It is also unlikely to achieve optimality. This code is for reference purposes and is not suited for usage other than research and experimentation.

Value

a list with the elements

L

The final estimate of L.

R

The final estimate of R.

Lstep

The infinity norm of the final step in L.

Rstep

The infinity norm of the final step in R.

iterations

The number of iterations taken.

converged

Whether convergence was detected.

Note

This package provides proof of concept code which is unlikely to be fast or robust, and may not solve the optimization problem at hand. User assumes all risk.

Author(s)

Steven E. Pav shabbychef@gmail.com

References

Merritt, Michael, and Zhang, Yin. "Interior-point Gradient Method for Large-Scale Totally Nonnegative Least Squares Problems." Journal of Optimization Theory and Applications 126, no 1 (2005): 191–202. https://scholarship.rice.edu/bitstream/handle/1911/102020/TR04-08.pdf

Pav, S. E. "An Iterative Algorithm for Regularized Non-negative Matrix Factorizations." Forthcoming. (2024)

Lee, Daniel D. and Seung, H. Sebastian. "Algorithms for Non-negative Matrix Factorization." Advances in Neural Information Processing Systems 13 (2001): 556–562. http://papers.nips.cc/paper/1861-algorithms-for-non-negative-matrix-factorization.pdf

See Also

aurnmf, gaurnmf

Examples


 nr <- 100
 nc <- 20
 dm <- 4

 randmat <- function(nr,nc,...) { matrix(pmax(0,runif(nr*nc,...)),nrow=nr) }
 set.seed(1234)
 real_L <- randmat(nr,dm)
 real_R <- randmat(dm,nc)
 Y <- real_L %*% real_R
# without regularization
 objective <- function(Y, L, R) { sum((Y - L %*% R)^2) }
 objective(Y,real_L,real_R)

 L_0 <- randmat(nr,dm)
 R_0 <- randmat(dm,nc)
 objective(Y,L_0,R_0)
 out1 <- murnmf(Y, L_0, R_0, max_iterations=5e3L)
 objective(Y,out1$L,out1$R)
# with L1 regularization on one side
 out2 <- murnmf(Y, L_0, R_0, max_iterations=5e3L,lambda_1L=0.1)
# objective does not suffer because all mass is shifted to R
 objective(Y,out2$L,out2$R)
list(L1=sum(out1$L),R1=sum(out1$R),L2=sum(out2$L),R2=sum(out2$R))
sum(out2$L)
# with L1 regularization on both sides
 out3 <- murnmf(Y, L_0, R_0, max_iterations=5e3L,lambda_1L=0.1,lambda_1R=0.1)
# with L1 regularization on both sides, raw objective suffers
 objective(Y,out3$L,out3$R)
list(L1=sum(out1$L),R1=sum(out1$R),L3=sum(out3$L),R3=sum(out3$R))


# example showing how to use the on_iteration_end callback to save iterates.
max_iterations <- 1e3L
it_history <<- rep(NA_real_, max_iterations)
quadratic_objective <- function(Y, L, R) { sum((Y - L %*% R)^2) }
on_iteration_end <- function(iteration, Y, L, R, ...) {
  it_history[iteration] <<- quadratic_objective(Y,L,R)
}
out1b <- murnmf(Y, L_0, R_0, max_iterations=max_iterations, on_iteration_end=on_iteration_end)


# should work on sparse matrices too, but beware zeros in the initial estimates
if (require(Matrix)) { 
 real_L <- randmat(nr,dm,min=-1)
 real_R <- randmat(dm,nc,min=-1)
 Y <- as(real_L %*% real_R, "sparseMatrix")
 L_0 <- randmat(nr,dm)
 R_0 <- randmat(dm,nc)
 out1 <- murnmf(Y, L_0, R_0, max_iterations=1e2L)
}


[Package rnnmf version 0.3.0 Index]