API

Here you can find description of functions and methods avaliable in t3f.

Module contents

class t3f.TensorTrain(tt_cores, shape=None, tt_ranks=None, convert_to_tensors=True, name='TensorTrain')

Bases: t3f.tensor_train_base.TensorTrainBase

Represents a Tensor Train object (a TT-tensor or TT-matrix).

t3f represents a Tensor Train object as a tuple of TT-cores.

left_tt_rank_dim

The dimension of the left TT-rank in each TT-core.

right_tt_rank_dim

The dimension of the right TT-rank in each TT-core.

tt_cores

A tuple of TT-cores.

Returns:
A tuple of 3d or 4d tensors shape
[r_k-1, n_k, r_k]
or
[r_k-1, n_k, m_k, r_k]
class t3f.TensorTrainBase(tt_cores)

Bases: object

An abstract class that represents a collection of Tensor Train cores.

dtype

The DType of elements in this tensor.

eval(feed_dict=None, session=None)

Evaluates this sparse tensor in a Session.

Calling this method will execute all preceding operations that produce the inputs needed for the operation that produces this tensor. N.B. Before invoking SparseTensor.eval(), its graph must have been launched in a session, and either a default session must be available, or session must be specified explicitly.

Parameters:
  • feed_dict – A dictionary that maps Tensor objects to feed values. See [Session.run()](../../api_docs/python/client.md#Session.run) for a description of the valid feed values.
  • session – (Optional.) The Session to be used to evaluate this sparse tensor. If none, the default session will be used.
get_raw_shape()

Get tuple of TensorShapes representing the shapes of the underlying TT-tensor.

Tuple contains one TensorShape for TT-tensor and 2 TensorShapes for TT-matrix

Returns:A tuple of TensorShape objects.
get_shape()

Get the TensorShape representing the shape of the dense tensor.

Returns:A TensorShape object.
get_tt_ranks()

Get the TT-ranks in an array of size `num_dims`+1.

The first and the last TT-rank are guarantied to be 1.

Returns:TensorShape of size `num_dims`+1.
graph

The Graph that contains the tt_cores tensors.

is_tt_matrix()

Returns True if the TensorTrain object represents a TT-matrix.

is_variable()

True if the TensorTrain object is a variable (e.g. is trainable).

name

The name of the TensorTrain.

Returns:String, the scope in which the TT-cores are defined.
ndims()

Get the number of dimensions of the underlying TT-tensor.

Returns:A number.
op

The Operation that evaluates all the cores.

tt_cores

A tuple of TT-cores.

class t3f.TensorTrainBatch(tt_cores, shape=None, tt_ranks=None, batch_size=None, convert_to_tensors=True, name='TensorTrainBatch')

Bases: t3f.tensor_train_base.TensorTrainBase

Represents a batch of Tensor Train objects (TT-tensors or TT-matrices).

t3f represents a Tensor Train object as a tuple of TT-cores.

batch_size

The number of elements or None if not known.

get_shape()

Get the TensorShape representing the shape of the dense tensor.

The first dimension is the batch_size.

Returns:A TensorShape object.
left_tt_rank_dim

The dimension of the left TT-rank in each TT-core.

right_tt_rank_dim

The dimension of the right TT-rank in each TT-core.

tt_cores

A tuple of TT-cores.

Returns:
A tuple of 4d or 5d tensors shape
[batch_size, r_k-1, n_k, r_k]
or
[batch_size, r_k-1, n_k, m_k, r_k]
t3f.add(tt_a, tt_b, name='t3f_add')

Returns a TensorTrain corresponding to elementwise sum tt_a + tt_b.

The shapes of tt_a and tt_b should coincide. Supports broadcasting:

add(TensorTrainBatch, TensorTrain)

adds TensorTrain to each element in the batch of TTs in TensorTrainBatch.

Parameters:
  • tt_a – TensorTrain, TensorTrainBatch, TT-tensor, or TT-matrix
  • tt_b – TensorTrain, TensorTrainBatch, TT-tensor, or TT-matrix
  • name – string, name of the Op.
Returns
a TensorTrain object corresponding to the element-wise sum of arguments if
both arguments are `TensorTrain`s.
OR a TensorTrainBatch if at least one of the arguments is
TensorTrainBatch
Raises
ValueError if the arguments shapes do not coincide
t3f.add_n_projected(tt_objects, coef=None)

Adds all input TT-objects that are projections on the same tangent space.

add_projected((a, b)) is equivalent add(a, b) for a and b that are from the same tangent space, but doesn’t increase the TT-ranks.
Parameters:
  • tt_objects – a list of TT-objects that are projections on the same tangent space.
  • coef –

    a list of numbers or anything else convertable to tf.Tensor. If provided, computes weighted sum. The size of this array should be

    len(tt_objects) x tt_objects[0].batch_size
Returns:

TT-objects representing the sum of the tt_objects (weighted sum if coef is provided). The TT-rank of the result equals to the TT-ranks of the arguments.
t3f.assign(ref, value, validate_shape=None, use_locking=None, name=None)
t3f.batch_size(tt, name='t3f_batch_size')

Return the number of elements in a TensorTrainBatch.

Parameters:
  • tt – TensorTrainBatch object.
  • name – string, name of the Op.
Returns:

0-D integer tensor.
Raises:

ValueError if got TensorTrain which doesn’t have batch_size as input.
t3f.bilinear_form(A, b, c, name='t3f_bilinear_form')

Bilinear form b^t A c; A is a TT-matrix, b and c can be batches.

Parameters:
  • A – TensorTrain object containing a TT-matrix of size N x M.
  • b – TensorTrain object containing a TT-matrix of size N x 1 or TensorTrainBatch with a batch of TT-matrices of size N x 1.
  • c – TensorTrain object containing a TT-matrix of size M x 1 or TensorTrainBatch with a batch of TT-matrices of size M x 1.
  • name – string, name of the Op.
Returns:

A number, the value of the bilinear form if all the arguments are

`TensorTrain`s.

OR tf.Tensor of size batch_size if at least one of the arguments is

TensorTrainBatch
Raises:

ValueError if the arguments are not TT-matrices or if the shapes are – not consistent.
Complexity:
O(batch_size r_A r_c r_b n d (r_b + r_A n + r_c))

d is the number of TT-cores (A.ndims()); r_A is the largest TT-rank of A max(A.get_tt_rank()) n is the size of the axis dimensions e.g.

if b and c are tensors of shape (3, 3, 3), A is a 27 x 27 matrix of tensor shape (3, 3, 3) x (3, 3, 3) then n is 3
t3f.cast(tt, dtype, name='t3f_cast')

Casts a tt-tensor to a new type.

Parameters:
  • tt – TensorTrain object.
  • dtype – The destination type.
  • name – string, name of the Op.
Raises:
  • TypeError – If tt cannot be cast to the dtype.
  • ValueError – If tt is not a TensorTrain or TensorTrainBatch.
t3f.clean_raw_shape(shape, name='t3f_clean_raw_shape')

Returns a tuple of TensorShapes for any valid shape representation.

Parameters:
  • shape – An np.array, a tf.TensorShape (for tensors), a tuple of tf.TensorShapes (for TT-matrices or tensors), or None
  • name – string, name of the Op.
Returns:

A tuple of tf.TensorShape, or None if the input is None
t3f.concat_along_batch_dim(tt_list, name='t3f_concat_along_batch_dim')

Concat all TensorTrainBatch objects along batch dimension.

Parameters:
  • tt_list – a list of TensorTrainBatch objects.
  • name – string, name of the Op.
Returns:

TensorTrainBatch
t3f.cores_regularizer(core_regularizer, scale, scope=None)

Returns a function that applies given regularization to each TT-core.

Parameters:
  • core_regularizer – a function with signature core_regularizer(core) that returns the penalty for the given TT-core.
  • scale – A scalar multiplier Tensor. 0.0 disables the regularizer.
  • scope – An optional scope name.
Returns:

A function with signature regularizer(weights) that applies the regularization.
Raises:

ValueError – If scale is negative or if scale is not a float.
t3f.expand_batch_dim(tt, name='t3f_expand_batch_dim')

Creates a 1-element TensorTrainBatch from a TensorTrain.

Parameters:
  • tt – TensorTrain or TensorTrainBatch.
  • name – string, name of the Op.
Returns:

TensorTrainBatch
t3f.eye(shape, dtype=tf.float32, name='t3f_eye')

Creates an identity TT-matrix.

Parameters:
  • shape – array which defines the shape of the matrix row and column indices.
  • dtype – [tf.float32] dtype of the resulting matrix.
  • name – string, name of the Op.
Returns:

TensorTrain containing an identity TT-matrix of size np.prod(shape) x np.prod(shape)
t3f.flat_inner(a, b, name='t3f_flat_inner')

Inner product along all axis.

The shapes of a and b should coincide.

Parameters:
  • a – TensorTrain, TensorTrainBatch, tf.Tensor, or tf.SparseTensor
  • b – TensorTrain, TensorTrainBatch, tf.Tensor, or tf.SparseTensor
  • name – string, name of the Op.
Returns
a number
sum of products of all the elements of a and b
OR or a tf.Tensor of size batch_size
sum of products of all the elements of a and b for each element in the batch.
t3f.frobenius_norm(tt, epsilon=1e-05, differentiable=False, name='t3f_frobenius_norm')

Frobenius norm of TensorTrain or of each TT in TensorTrainBatch

Frobenius norm is the sqrt of the sum of squares of all elements in a tensor.

Parameters:
  • tt – TensorTrain or TensorTrainBatch object
  • epsilon – the function actually computes sqrt(norm_squared + epsilon) for numerical stability (e.g. gradient of sqrt at zero is inf).
  • differentiable – bool, whether to use a differentiable implementation or a fast and stable implementation based on QR decomposition.
  • name – string, name of the Op.
Returns
a number which is the Frobenius norm of tt, if it is TensorTrain OR a Tensor of size tt.batch_size, consisting of the Frobenius norms of each TensorTrain in tt, if it is TensorTrainBatch
t3f.frobenius_norm_squared(tt, differentiable=False, name='t3f_frobenius_norm_squared')

Frobenius norm squared of TensorTrain or of each TT in TensorTrainBatch.

Frobenius norm squared is the sum of squares of all elements in a tensor.

Parameters:
  • tt – TensorTrain or TensorTrainBatch object
  • differentiable – bool, whether to use a differentiable implementation or a fast and stable implementation based on QR decomposition.
  • name – string, name of the Op.
Returns
a number which is the Frobenius norm squared of tt, if it is TensorTrain OR a Tensor of size tt.batch_size, consisting of the Frobenius norms squared of each TensorTrain in tt, if it is TensorTrainBatch
t3f.full(tt, name='t3f_full')

Converts a TensorTrain into a regular tensor or matrix (tf.Tensor).

Parameters:
  • tt – TensorTrain or TensorTrainBatch object.
  • name – string, name of the Op.
Returns:

tf.Tensor.
t3f.gather_nd(tt, indices, name='t3f_gather_nd')

out[i] = tt[indices[i, 0], indices[i, 1], …]

Equivalent to
tf.gather_nd(t3f.full(tt), indices)

but much faster, since it does not materialize the full tensor.

For batches of TT works indices should include the batch dimension as well.

Parameters:
  • tt – TensorTrain or TensorTrainBatch object representing a tensor (TT-matrices are not implemented yet)
  • indices –

    numpy array, tf.Tensor, placeholder with 2 or more dimensions. The last dimension indices.shape[-1] should be equal to the numbers of dimensions in TT:

    indices.shape[-1] = tt.ndims for TensorTrain indices.shape[-1] = tt.ndims + 1 for TensorTrainBatch
  • name – string, name of the Op.
Returns:

tf.Tensor with elements specified by indices.
Raises:
  • ValueError if indices have wrong shape.
  • NotImplementedError if tt is a TT-matrix.
t3f.get_variable(name, dtype=None, initializer=None, regularizer=None, trainable=True, collections=None, caching_device=None, validate_shape=True)

Returns TensorTrain object with tf.Variables as the TT-cores.

Parameters:
  • name – The name of the new or existing TensorTrain variable. Used to name the TT-cores.
  • dtype – Type of the new or existing TensorTrain variable TT-cores (defaults to DT_FLOAT).
  • initializer – TensorTrain or TensorTrainBatch, initializer for the variable if one is created.
  • regularizer – A (TensorTrain -> Tensor or None) function; the result of applying it on a newly created variable will be added to the collection GraphKeys.REGULARIZATION_LOSSES and can be used for regularization.
  • trainable – If True also add the variable to the graph collection GraphKeys.TRAINABLE_VARIABLES (see tf.Variable).
  • collections – List of graph collections keys to add the Variables (underlying TT-cores). Defaults to [GraphKeys.GLOBAL_VARIABLES] (see tf.Variable).
  • caching_device – Optional device string or function describing where the Variable should be cached for reading. Defaults to the Variable’s device. If not None, caches on another device. Typical use is to cache on the device where the Ops using the Variable reside, to deduplicate copying through Switch and other conditional statements.
  • validate_shape – If False, allows the variable to be initialized with a value of unknown shape. If True, the default, the shape of initial_value must be known.
Returns:

The created or existing TensorTrain object with tf.Variables TT-cores.
Raises:

ValueError – when creating a new variable and shape is not declared, when violating reuse during variable creation, or when initializer dtype and dtype don’t match. Reuse is set inside variable_scope.
t3f.glorot_initializer(shape, tt_rank=2, dtype=tf.float32, name='t3f_glorot_initializer')

Constructs a random TT matrix with entrywise variance 2.0 / (n_in + n_out)

Parameters:
  • shape –

    2d array, shape[0] is the shape of the matrix row-index, shape[1] is the shape of the column index. shape[0] and shape[1] should have the same number of elements (d) Also supports omitting one of the dimensions for vectors, e.g.

    glorot_initializer([[2, 2, 2], None])
    and
    glorot_initializer([None, [2, 2, 2]])

    will create an 8-element column and row vectors correspondingly.

  • tt_rank – a number or a (d+1)-element array with ranks.
  • dtype – [tf.float32] dtype of the resulting matrix.
  • name – string, name of the Op.
Returns:

TensorTrain containing a TT-matrix of size

np.prod(shape[0]) x np.prod(shape[1])
t3f.gradients(func, x, name='t3f_gradients', runtime_check=True)

Riemannian autodiff: returns gradient projected on tangent space of TT.

Computes projection of the gradient df/dx onto the tangent space of TT tensor at point x.

Warning: this is experimental feature and it may not work for some function, e.g. ones that include QR or SVD decomposition (t3f.project, t3f.round) or for functions that work with TT-cores directly (in contrast to working with TT-object only via t3f functions). In this cases this function can silently return wrong results!

Example

# Scalar product with some predefined tensor squared 0.5 * <x, t>**2. # It’s gradient is <x, t> t and it’s Riemannian gradient is # t3f.project(<x, t> * t, x) f = lambda x: 0.5 * t3f.flat_inner(x, t)**2 projected_grad = t3f.gradients(f, x) # t3f.project(t3f.flat_inner(x, t) * t, x)

Parameters:
  • func – function that takes TensorTrain object as input and outputs a number.
  • x – point at which to compute the gradient and on which tangent space to project the gradient.
  • name – string, name of the Op.
  • runtime_check – [True] whether to do a sanity check that the passed function is invariant to different TT representations (otherwise the Rieamnnian gradient doesn’t even exist). It makes things slower, but helps catching bugs, so turn it off during production deployment.
Returns:

TensorTrain, projection of the gradient df/dx onto the tangent space at point x.

See also

t3f.hessian_vector_product

t3f.gram_matrix(tt_vectors, matrix=None, name='t3f_gram_matrix')

Computes Gramian matrix of a batch of TT-vectors.

If matrix is None, computes
res[i, j] = t3f.flat_inner(tt_vectors[i], tt_vectors[j]).
If matrix is present, computes
res[i, j] = t3f.flat_inner(tt_vectors[i], t3f.matmul(matrix, tt_vectors[j]))
or more shortly
res[i, j] = tt_vectors[i]^T * matrix * tt_vectors[j]

but is more efficient.

Parameters:
  • tt_vectors – TensorTrainBatch.
  • matrix – None, or TensorTrain matrix.
  • name – string, name of the Op.
Returns:

tf.tensor with the Gram matrix.
Complexity:
If the matrix is not present, the complexity is O(batch_size^2 d r^3 n)

where d is the number of TT-cores (tt_vectors.ndims()), r is the largest TT-rank

max(tt_vectors.get_tt_rank())
and n is the size of the axis dimension, e.g.
for a tensor of size 4 x 4 x 4, n is 4; for a 9 x 64 matrix of raw shape (3, 3, 3) x (4, 4, 4) n is 12
If the matrix of TT-rank R is present, the complexity is
O(batch_size^2 d R r^2 n (r + nR))

where the matrix is of raw-shape (n, n, …, n) x (n, n, …, n); r is the TT-rank of vectors tt_vectors; R is the TT-rank of the matrix.

t3f.he_initializer(shape, tt_rank=2, dtype=tf.float32, name='t3f_he_initializer')

Constructs a random TT matrix with entrywise variance 2.0 / n_in

Parameters:
  • shape –

    2d array, shape[0] is the shape of the matrix row-index, shape[1] is the shape of the column index. shape[0] and shape[1] should have the same number of elements (d) Also supports omitting one of the dimensions for vectors, e.g.

    he_initializer([[2, 2, 2], None])
    and
    he_initializer([None, [2, 2, 2]])

    will create an 8-element column and row vectors correspondingly.

  • tt_rank – a number or a (d+1)-element array with ranks.
  • dtype – [tf.float32] dtype of the resulting matrix.
  • name – string, name of the Op.
Returns:

TensorTrain containing a TT-matrix of size

np.prod(shape[0]) x np.prod(shape[1])
t3f.hessian_vector_product(func, x, vector, name='t3f_hessian_vector_product', runtime_check=True)

P_x [d^2f/dx^2] P_x vector, i.e. Riemannian hessian by vector product.

Computes
P_x [d^2f/dx^2] P_x vector

where P_x is projection onto the tangent space of TT at point x and d^2f/dx^2 is the Hessian of the function.

Note that the true Riemannian hessian also includes the manifold curvature term which is ignored here.

Warning: this is experimental feature and it may not work for some function, e.g. ones that include QR or SVD decomposition (t3f.project, t3f.round) or for functions that work with TT-cores directly (in contrast to working with TT-object only via t3f functions). In this cases this function can silently return wrong results!

Example

# Quadratic form with matrix A: <x, A x>. # It’s gradient is (A + A.T) x, it’s Hessian is (A + A.T) # It’s Riemannian Hessian by vector product is # proj_vec = t3f.project(vector, x) # t3f.project(t3f.matmul(A + t3f.transpose(A), proj_vec), x) f = lambda x: t3f.bilinear_form(A, x, x) res = t3f.hessian_vector_product(f, x, vector)

Parameters:
  • func – function that takes TensorTrain object as input and outputs a number.
  • x –
    point at which to compute the Hessian and on which tangent space to
    project the gradient.

    vector: TensorTrain object which to multiply be the Hessian. name: string, name of the Op. runtime_check: [True] whether to do a sanity check that the passed

    function is invariant to different TT representations (otherwise the Rieamnnian gradient doesn’t even exist). It makes things slower, but helps catching bugs, so turn it off during production deployment.
Returns:

TensorTrain, result of the Riemannian hessian by vector product.

See also

t3f.gradients

t3f.is_batch_broadcasting_possible(tt_a, tt_b)

Check that the batch broadcasting possible for the given batch sizes.

Returns true if the batch sizes are the same or if one of them is 1.

If the batch size that is supposed to be 1 is not known on compilation stage, broadcasting is not allowed.

Parameters:
  • tt_a – TensorTrain or TensorTrainBatch
  • tt_b – TensorTrain or TensorTrainBatch
Returns:

Bool
t3f.l2_regularizer(scale, scope=None)

Returns a function that applies L2 regularization to TensorTrain weights.

Parameters:
  • scale – A scalar multiplier Tensor. 0.0 disables the regularizer.
  • scope – An optional scope name.
Returns:

A function with signature l2(tt) that applies L2 regularization.
Raises:

ValueError – If scale is negative or if scale is not a float.
t3f.lazy_batch_size(tt, name='t3f_lazy_batch_size')

Return static batch_size if available and dynamic otherwise.

Parameters:
  • tt – TensorTrainBatch object.
  • name – string, name of the Op.
Returns:

A number or a 0-D tf.Tensor
Raises:

ValueError if got TensorTrain which doesn’t have batch_size as input.
t3f.lazy_raw_shape(tt, name='t3f_lazy_raw_shape')

Returns static raw shape of a TensorTrain if defined, and dynamic otherwise.

This operation returns a 2-D integer numpy array representing the raw shape of the input if it is available on the graph compilation stage and 2-D integer tensor of dynamic shape otherwise. If the input is a TT-tensor, the raw shape will have 1 x ndims() elements. If the input is a TT-matrix, the raw shape will have 2 x ndims() elements representing the underlying tensor shape of the matrix.

Parameters:
  • tt – TensorTrain object.
  • name – string, name of the Op.
Returns:

A 2-D numpy array or tf.Tensor of size 1 x ndims() or 2 x ndims()
t3f.lazy_shape(tt, name='t3f_lazy_shape')

Returns static shape of a TensorTrain if defined, and dynamic otherwise.

This operation returns a 1-D integer numpy array representing the shape of the input if it is available on the graph compilation stage and 1-D integer tensor of dynamic shape otherwise.

Parameters:
  • tt – TensorTrain object.
  • name – string, name of the Op.
Returns:

A 1-D numpy array or tf.Tensor
t3f.lazy_tt_ranks(tt, name='t3f_lazy_tt_ranks')

Returns static TT-ranks of a TensorTrain if defined, and dynamic otherwise.

This operation returns a 1-D integer numpy array of TT-ranks if they are available on the graph compilation stage and 1-D integer tensor of dynamic TT-ranks otherwise.

Parameters:
  • tt – TensorTrain object.
  • name – string, name of the Op.
Returns:

A 1-D numpy array or tf.Tensor
t3f.lecun_initializer(shape, tt_rank=2, dtype=tf.float32, name='t3f_lecun_initializer')

Constructs a random TT matrix with entrywise variance 1.0 / n_in

Parameters:
  • shape –

    2d array, shape[0] is the shape of the matrix row-index, shape[1] is the shape of the column index. shape[0] and shape[1] should have the same number of elements (d) Also supports omitting one of the dimensions for vectors, e.g.

    lecun_initializer([[2, 2, 2], None])
    and
    lecun_initializer([None, [2, 2, 2]])

    will create an 8-element column and row vectors correspondingly.

  • tt_rank – a number or a (d+1)-element array with ranks.
  • dtype – [tf.float32] dtype of the resulting matrix.
  • name – string, name of the Op.
Returns:

TensorTrain containing a TT-matrix of size

np.prod(shape[0]) x np.prod(shape[1])
t3f.matmul(a, b, name='t3f_matmul')

Multiplies two matrices that can be TT-, dense, or sparse.

Note that multiplication of two TT-matrices returns a TT-matrix with much larger ranks. Also works for multiplying two batches of TT-matrices or a product between a TT-matrix and a batch of TT-matrices (with broadcasting).

Parameters:
  • a – TensorTrain, TensorTrainBatch, tf.Tensor, or tf.SparseTensor of size M x N
  • b – TensorTrain, TensorTrainBatch, tf.Tensor, or tf.SparseTensor of size N x P
  • name – string, name of the Op.
Returns
If both arguments are TensorTrain objects, returns a TensorTrain
object containing a TT-matrix of size M x P.
If at least one of the arguments is a TensorTrainBatch object, returns
a TensorTrainBatch object containing a batch of TT-matrices of size M x P.

Otherwise, returns tf.Tensor of size M x P.

t3f.matrix_batch_with_random_cores(shape, tt_rank=2, batch_size=1, mean=0.0, stddev=1.0, dtype=tf.float32, name='t3f_matrix_batch_with_random_cores')

Generate a batch of TT-matrices of given shape with N(mean, stddev^2) cores.

Parameters:
  • shape –

    2d array, shape[0] is the shape of the matrix row-index, shape[1] is the shape of the column index. shape[0] and shape[1] should have the same number of elements (d) Also supports omitting one of the dimensions for vectors, e.g.

    matrix_batch_with_random_cores([[2, 2, 2], None])
    and
    matrix_batch_with_random_cores([None, [2, 2, 2]])
  • create a batch of one 8-element column and row vector correspondingly. (will) –
  • tt_rank – a number or a (d+1)-element array with ranks.
  • batch_size – an integer.
  • mean – a number, the mean of the normal distribution used for initializing TT-cores.
  • stddev – a number, the standard deviation of the normal distribution used for initializing TT-cores.
  • dtype – [tf.float32] dtype of the resulting matrix.
  • name – string, name of the Op.
Returns:

TensorTrainBatch containing a batch of TT-matrices of size

np.prod(shape[0]) x np.prod(shape[1])
t3f.matrix_ones(shape, dtype=tf.float32, name='t3f_matrix_ones')

Generate a TT-matrix of the given shape with each entry equal to 1.

Parameters:
  • shape –

    2d array, shape[0] is the shape of the matrix row-index, shape[1] is the shape of the column index. shape[0] and shape[1] should have the same number of elements (d) Also supports omitting one of the dimensions for vectors, e.g.

    matrix_ones([[2, 2, 2], None])
    and
    matrix_ones([None, [2, 2, 2]])

    will create an 8-element column and row vectors correspondingly.

  • dtype – [tf.float32] dtype of the resulting matrix.
  • name – string, name of the Op.
Returns:

TensorTrain containing a TT-matrix of size

np.prod(shape[0]) x np.prod(shape[1]) with each entry equal to 1
t3f.matrix_with_random_cores(shape, tt_rank=2, mean=0.0, stddev=1.0, dtype=tf.float32, name='t3f_matrix_with_random_cores')

Generate a TT-matrix of given shape with N(mean, stddev^2) cores.

Parameters:
  • shape –

    2d array, shape[0] is the shape of the matrix row-index, shape[1] is the shape of the column index. shape[0] and shape[1] should have the same number of elements (d) Also supports omitting one of the dimensions for vectors, e.g.

    matrix_with_random_cores([[2, 2, 2], None])
    and
    matrix_with_random_cores([None, [2, 2, 2]])

    will create an 8-element column and row vectors correspondingly.

  • tt_rank – a number or a (d+1)-element array with ranks.
  • mean – a number, the mean of the normal distribution used for initializing TT-cores.
  • stddev – a number, the standard deviation of the normal distribution used for initializing TT-cores.
  • dtype – [tf.float32] dtype of the resulting matrix.
  • name – string, name of the Op.
Returns:

TensorTrain containing a TT-matrix of size

np.prod(shape[0]) x np.prod(shape[1])
t3f.matrix_zeros(shape, dtype=tf.float32, name='t3f_matrix_zeros')

Generate a TT-matrix of the given shape with each entry equal to 0.

Parameters:
  • shape –

    2d array, shape[0] is the shape of the matrix row-index, shape[1] is the shape of the column index. shape[0] and shape[1] should have the same number of elements (d) Also supports omitting one of the dimensions for vectors, e.g.

    matrix_zeros([[2, 2, 2], None])
    and
    matrix_zeros([None, [2, 2, 2]])

    will create an 8-element column and row vectors correspondingly.

  • dtype – [tf.float32] dtype of the resulting matrix.
  • name – string, name of the Op.
Returns:

TensorTrain containing a TT-matrix of size

np.prod(shape[0]) x np.prod(shape[1]) with each entry equal to 0
t3f.multiply(tt_left, right, name='t3f_multiply')

Returns a TensorTrain corresponding to element-wise product tt_left * right.

Supports broadcasting:

multiply(TensorTrainBatch, TensorTrain) returns TensorTrainBatch consisting of element-wise products of TT in TensorTrainBatch and TensorTrain

multiply(TensorTrainBatch_a, TensorTrainBatch_b) returns TensorTrainBatch consisting of element-wise products of TT in TensorTrainBatch_a and TT in TensorTrainBatch_b

Batch sizes should support broadcasting

Parameters:
  • tt_left – TensorTrain OR TensorTrainBatch
  • right – TensorTrain OR TensorTrainBatch OR a number.
  • name – string, name of the Op.
Returns
a TensorTrain or TensorTrainBatch object corresponding to the element-wise product of the arguments.
Raises
ValueError if the arguments shapes do not coincide or broadcasting is not possible.
t3f.multiply_along_batch_dim(batch_tt, weights, name='t3f_multiply_along_batch_dim')

Multiply each TensorTrain in a batch by a number.

Parameters:
  • batch_tt – TensorTrainBatch object, TT-matrices or TT-tensors.
  • weights – 1-D tf.Tensor (or something convertible to it like np.array) of size tt.batch_size with weights.
  • name – string, name of the Op.
Returns:

TensorTrainBatch
t3f.ones_like(tt, name='t3f_ones_like')

Constructs t3f.ones with the shape of tt.

In the case when tt is TensorTrainBatch constructs t3f.ones with the shape of a TensorTrain in tt.

Parameters:
  • tt – TensorTrain object
  • name – string, name of the Op.
Returns:

TensorTrain object of the same shape as tt but with all entries equal to 1.
t3f.orthogonalize_tt_cores(tt, left_to_right=True, name='t3f_orthogonalize_tt_cores')

Orthogonalize TT-cores of a TT-object.

Parameters:
  • tt – TenosorTrain or a TensorTrainBatch.
  • left_to_right – bool, the direction of orthogonalization.
  • name – string, name of the Op.
Returns:

The same type as the input tt (TenosorTrain or a TensorTrainBatch).
t3f.pairwise_flat_inner(tt_1, tt_2, matrix=None, name='t3f_pairwise_flat_inner')

Computes all scalar products between two batches of TT-objects.

If matrix is None, computes
res[i, j] = t3f.flat_inner(tt_1[i], tt_2[j]).
If matrix is present, computes
res[i, j] = t3f.flat_inner(tt_1[i], t3f.matmul(matrix, tt_2[j]))
or more shortly
res[i, j] = tt_1[i]^T * matrix * tt_2[j]

but is more efficient.

Parameters:
  • tt_1 – TensorTrainBatch.
  • tt_2 – TensorTrainBatch.
  • matrix – None, or TensorTrain matrix.
  • name – string, name of the Op.
Returns:

tf.tensor with the matrix of pairwise scalar products (flat inners).
Complexity:
If the matrix is not present, the complexity is O(batch_size^2 d r^3 n)

where d is the number of TT-cores (tt_vectors.ndims()), r is the largest TT-rank

max(tt_vectors.get_tt_rank())
and n is the size of the axis dimension, e.g.
for a tensor of size 4 x 4 x 4, n is 4; for a 9 x 64 matrix of raw shape (3, 3, 3) x (4, 4, 4) n is 12
A more precise complexity is
O(batch_size^2 d r1 r2 n max(r1, r2))

where r1 is the largest TT-rank of tt_a and r2 is the largest TT-rank of tt_b.

If the matrix is present, the complexity is
O(batch_size^2 d R r1 r2 (n r1 + n m R + m r2))

where the matrix is of raw-shape (n, n, …, n) x (m, m, …, m) and TT-rank R; tt_1 is of shape (n, n, …, n) and is of the TT-rank r1; tt_2 is of shape (m, m, …, m) and is of the TT-rank r2;

t3f.pairwise_flat_inner_projected(projected_tt_vectors_1, projected_tt_vectors_2)

Scalar products between two batches of TTs from the same tangent space.

res[i, j] = t3f.flat_inner(projected_tt_vectors_1[i], projected_tt_vectors_1[j]).

pairwise_flat_inner_projected(projected_tt_vectors_1, projected_tt_vectors_2) is equivalent to

pairwise_flat_inner(projected_tt_vectors_1, projected_tt_vectors_2)

, but works only on objects from the same tangent space and is much faster than general pairwise_flat_inner.

Parameters:
  • projected_tt_vectors_1 – TensorTrainBatch of tensors projected on the same tangent space as projected_tt_vectors_2.
  • projected_tt_vectors_2 – TensorTrainBatch.
Returns:

tf.tensor with the scalar product matrix.
Complexity:
O(batch_size^2 d r^2 n), where

d is the number of TT-cores (projected_tt_vectors_1.ndims()); r is the largest TT-rank max(projected_tt_vectors_1.get_tt_rank())

(i.e. 2 * {the TT-rank of the object we projected vectors onto}.
and n is the size of the axis dimension, e.g.
for a tensor of size 4 x 4 x 4, n is 4; for a 9 x 64 matrix of raw shape (3, 3, 3) x (4, 4, 4) n is 12.
t3f.project(what, where)

Project what TTs on the tangent space of where TT.

project(what, x) = P_x(what) project(batch_what, x) = batch(P_x(batch_what[0]), …, P_x(batch_what[N]))

This function implements the algorithm from the paper [1], theorem 3.1.

[1] C. Lubich, I. Oseledets and B. Vandereycken, Time integration of
Tensor Trains.
Parameters:
  • what – TensorTrain or TensorTrainBatch. In the case of batch returns batch with projection of each individual tensor.
  • where – TensorTrain, TT-tensor or TT-matrix on which tangent space to project
Returns:

a TensorTrain with the TT-ranks equal 2 * tangent_space_tens.get_tt_ranks()
Complexity:
O(d r_where^3 m) for orthogonalizing the TT-cores of where

+O(batch_size d r_what r_where n (r_what + r_where))

d is the number of TT-cores (what.ndims()); r_what is the largest TT-rank of what max(what.get_tt_rank()) r_where is the largest TT-rank of where n is the size of the axis dimension of what and where e.g.

for a tensor of size 4 x 4 x 4, n is 4; for a 9 x 64 matrix of raw shape (3, 3, 3) x (4, 4, 4) n is 12
t3f.project_matmul(what, where, matrix)

Project matrix * what TTs on the tangent space of where TT.

project(what, x) = P_x(what) project(batch_what, x) = batch(P_x(batch_what[0]), …, P_x(batch_what[N]))

This function implements the algorithm from the paper [1], theorem 3.1.

[1] C. Lubich, I. Oseledets and B. Vandereycken, Time integration of
Tensor Trains.
Parameters:
  • what – TensorTrain or TensorTrainBatch. In the case of batch returns batch with projection of each individual tensor.
  • where – TensorTrain, TT-tensor or TT-matrix on which tangent space to project
  • matrix – TensorTrain, TT-matrix to multiply by what
Returns:

a TensorTrain with the TT-ranks equal 2 * tangent_space_tens.get_tt_ranks()
Complexity:
O(d r_where^3 m) for orthogonalizing the TT-cores of where

+O(batch_size d R r_what r_where (n r_what + n m R + m r_where))

d is the number of TT-cores (what.ndims()); r_what is the largest TT-rank of what max(what.get_tt_rank()) r_where is the largest TT-rank of where matrix is of TT-rank R and of raw-shape (m, m, …, m) x (n, n, …, n).

t3f.project_sum(what, where, weights=None)

Project sum of what TTs on the tangent space of where TT.

project_sum(what, x) = P_x(what) project_sum(batch_what, x) = P_x(sum_i batch_what[i]) project_sum(batch_what, x, weights) = P_x(sum_j weights[j] * batch_what[j])

This function implements the algorithm from the paper [1], theorem 3.1.

[1] C. Lubich, I. Oseledets and B. Vandereycken, Time integration of
Tensor Trains.
Parameters:
  • what – TensorTrain or TensorTrainBatch. In the case of batch returns projection of the sum of elements in the batch.
  • where – TensorTrain, TT-tensor or TT-matrix on which tangent space to project
  • weights – python list or tf.Tensor of numbers or None, weights of the sum
Returns:

a TensorTrain with the TT-ranks equal 2 * tangent_space_tens.get_tt_ranks()
Complexity:
O(d r_where^3 m) for orthogonalizing the TT-cores of where

+O(batch_size d r_what r_where n (r_what + r_where))

d is the number of TT-cores (what.ndims()); r_what is the largest TT-rank of what max(what.get_tt_rank()) r_where is the largest TT-rank of where n is the size of the axis dimension of what and where e.g.

for a tensor of size 4 x 4 x 4, n is 4; for a 9 x 64 matrix of raw shape (3, 3, 3) x (4, 4, 4) n is 12
t3f.quadratic_form(A, b, c, name='t3f_bilinear_form')

Outdated, see bilinear_form.

t3f.random_matrix(shape, tt_rank=2, mean=0.0, stddev=1.0, dtype=tf.float32, name='t3f_random_matrix')

Generate a random TT-matrix of the given shape with given mean and stddev.

Entries of the generated matrix (in the full format) will be iid and satisfy E[x_{i1i2..id}] = mean, Var[x_{i1i2..id}] = stddev^2, but the distribution is in fact not Gaussian.

In the current implementation only mean 0 is supported. To get a random_matrix with specified mean but tt_rank greater by 1 you can call x = t3f.random_matrix(shape, tt_rank, stddev=stddev) x = mean * t3f.ones_like(x) + x

Parameters:
  • shape –

    2d array, shape[0] is the shape of the matrix row-index, shape[1] is the shape of the column index. shape[0] and shape[1] should have the same number of elements (d) Also supports omitting one of the dimensions for vectors, e.g.

    random_matrix([[2, 2, 2], None])
    and
    random_matrix([None, [2, 2, 2]])

    will create an 8-element column and row vectors correspondingly.

  • tt_rank – a number or a (d+1)-element array with ranks.
  • mean – a number, the desired mean for the distribution of entries.
  • stddev – a number, the desired standard deviation for the distribution of entries.
  • dtype – [tf.float32] dtype of the resulting matrix.
  • name – string, name of the Op.
Returns:

TensorTrain containing a TT-matrix of size

np.prod(shape[0]) x np.prod(shape[1])
t3f.random_matrix_batch(shape, tt_rank=2, batch_size=1, mean=0.0, stddev=1.0, dtype=tf.float32, name='t3f_random_matrix_batch')

Generate a batch of TT-matrices with given shape, mean and stddev.

Entries of the generated matrices (in the full format) will be iid and satisfy E[x_{i1i2..id}] = mean, Var[x_{i1i2..id}] = stddev^2, but the distribution is in fact not Gaussian.

In the current implementation only mean 0 is supported. To get a random_matrix_batch with specified mean but tt_rank greater by 1 you can call x = t3f.random_matrix_batch(shape, tt_rank, batch_size=bs, stddev=stddev) x = mean * t3f.ones_like(x) + x

Parameters:
  • shape –

    2d array, shape[0] is the shape of the matrix row-index, shape[1] is the shape of the column index. shape[0] and shape[1] should have the same number of elements (d) Also supports omitting one of the dimensions for vectors, e.g.

    random_matrix_batch([[2, 2, 2], None])
    and
    random_matrix_batch([None, [2, 2, 2]])
  • create a batch of one 8-element column and row vector correspondingly. (will) –
  • tt_rank – a number or a (d+1)-element array with ranks.
  • batch_size – an integer.
  • mean – a number, the desired mean for the distribution of entries.
  • stddev – a number, the desired standard deviation for the distribution of entries.
  • dtype – [tf.float32] dtype of the resulting matrix.
  • name – string, name of the Op.
Returns:

TensorTrainBatch containing a batch of TT-matrices of size

np.prod(shape[0]) x np.prod(shape[1])
t3f.random_tensor(shape, tt_rank=2, mean=0.0, stddev=1.0, dtype=tf.float32, name='t3f_random_tensor')

Generate a random TT-tensor of the given shape with given mean and stddev.

Entries of the generated tensor (in the full format) will be iid and satisfy E[x_{i1i2..id}] = mean, Var[x_{i1i2..id}] = stddev^2, but the distribution is in fact not Gaussian (but is close for large tensors).

In the current implementation only mean 0 is supported. To get a random_tensor with specified mean but tt_rank greater by 1 you can call x = t3f.random_tensor(shape, tt_rank, stddev=stddev) x = mean * t3f.ones_like(x) + x

Parameters:
  • shape – array representing the shape of the future tensor.
  • tt_rank – a number or a (d+1)-element array with the desired ranks.
  • mean – a number, the desired mean for the distribution of entries.
  • stddev – a number, the desired standard deviation for the distribution of entries.
  • dtype – [tf.float32] dtype of the resulting tensor.
  • name – string, name of the Op.
Returns:

TensorTrain containing a TT-tensor
t3f.random_tensor_batch(shape, tt_rank=2, batch_size=1, mean=0.0, stddev=1.0, dtype=tf.float32, name='t3f_random_tensor_batch')

Generate a batch of TT-tensors with given shape, mean and stddev.

Entries of the generated tensors (in the full format) will be iid and satisfy E[x_{i1i2..id}] = mean, Var[x_{i1i2..id}] = stddev^2, but the distribution is in fact not Gaussian (but is close for large tensors).

In the current implementation only mean 0 is supported. To get a random_tensor_batch with specified mean but tt_rank greater by 1 you can call x = t3f.random_tensor_batch(shape, tt_rank, batch_size=bs, stddev=stddev) x = mean * t3f.ones_like(x) + x

Parameters:
  • shape – array representing the shape of the future tensor.
  • tt_rank – a number or a (d+1)-element array with ranks.
  • batch_size – an integer.
  • mean – a number, the desired mean for the distribution of entries.
  • stddev – a number, the desired standard deviation for the distribution of entries.
  • dtype – [tf.float32] dtype of the resulting tensor.
  • name – string, name of the Op.
Returns:

TensorTrainBatch containing TT-tensors.
t3f.raw_shape(tt, name='t3f_raw_shape')

Returns the shape of a TensorTrain.

This operation returns a 2-D integer tensor representing the shape of the input. If the input is a TT-tensor, the shape will have 1 x ndims() elements. If the input is a TT-matrix, the shape will have 2 x ndims() elements representing the underlying tensor shape of the matrix.

Parameters:
  • tt – TensorTrain or TensorTrainBatch object.
  • name – string, name of the Op.
Returns:

A 2-D Tensor of size 1 x ndims() or 2 x ndims()
t3f.renormalize_tt_cores(tt, epsilon=1e-08, name='t3f_renormalize_tt_cores')

Renormalizes TT-cores to make them of the same Frobenius norm.

Doesn’t change the tensor represented by tt object, but renormalizes the TT-cores to make further computations more stable.

Parameters:
  • tt – TensorTrain or TensorTrainBatch object
  • epsilon – parameter for numerical stability of sqrt
  • name – string, name of the Op.
Returns:

TensorTrain or TensorTrainBatch which represents the same tensor as tt, but with all cores having equal norm. In the batch case applies to each TT in TensorTrainBatch.
t3f.round(tt, max_tt_rank=None, epsilon=None, name='t3f_round')

TT-rounding procedure, returns a TT object with smaller TT-ranks.

Parameters:
  • tt – TensorTrain object, TT-tensor or TT-matrix
  • max_tt_rank –

    a number or a list of numbers If a number, than defines the maximal TT-rank of the result. If a list of numbers, than max_tt_rank length should be d+1 (where d is the rank of tens) and max_tt_rank[i] defines the maximal (i+1)-th TT-rank of the result. The following two versions are equivalent

    max_tt_rank = r
    and
    max_tt_rank = r * np.ones(d-1)
  • epsilon –

    a floating point number or None If the TT-ranks are not restricted (max_tt_rank=np.inf), then the result would be guarantied to be epsilon close to tt in terms of relative Frobenius error:

    ||res - tt||_F / ||tt||_F <= epsilon

    If the TT-ranks are restricted, providing a loose epsilon may reduce the TT-ranks of the result. E.g.

    round(tt, max_tt_rank=100, epsilon=0.9)

    will probably return you a TT-tensor with TT-ranks close to 1, not 100. Note that providing a nontrivial (= not equal to None) epsilon will make the TT-ranks of the result undefined on the compilation stage (e.g. res.get_tt_ranks() will return None, but t3f.tt_ranks(res).eval() will work).

  • name – string, name of the Op.
Returns:

TensorTrain object containing a TT-tensor.
Raises:

ValueError if max_tt_rank is less than 0, if max_tt_rank is not a number and – not a vector of length d + 1 where d is the number of dimensions (rank) of the input tensor, if epsilon is less than 0.
t3f.shape(tt, name='t3f_shape')

Returns the shape of a TensorTrain.

This operation returns a 1-D integer tensor representing the shape of
the input. For TT-matrices the shape would have two values, see raw_shape for the tensor shape.
If the input is a TensorTrainBatch, the first dimension of the output is the
batch_size.
Parameters:
  • tt – TensorTrain or TensorTrainBatch object.
  • name – string, name of the Op.
Returns:

A Tensor
t3f.squeeze_batch_dim(tt, name='t3f_squeeze_batch_dim')

Converts batch size 1 TensorTrainBatch into TensorTrain.

Parameters:
  • tt – TensorTrain or TensorTrainBatch.
  • name – string, name of the Op.
Returns:

TensorTrain if the input is a TensorTrainBatch with batch_size == 1 (known

at compilation stage) or a TensorTrain.

TensorTrainBatch otherwise.
t3f.tangent_space_to_deltas(tt, name='t3f_tangent_space_to_deltas')

Convert an element of the tangent space to deltas representation.

Tangent space elements (outputs of t3f.project) look like:
dP1 V2 … Vd + U1 dP2 V3 … Vd + … + U1 … Ud-1 dPd.

This function takes as input an element of the tangent space and converts it to the list of deltas [dP1, …, dPd].

Parameters:
  • tt – TensorTrain or TensorTrainBatch that is a result of t3f.project, t3f.project_matmul, or other similar functions.
  • name – string, name of the Op.
Returns:

A list of delta-cores (tf.Tensors).
t3f.tensor_batch_with_random_cores(shape, tt_rank=2, batch_size=1, mean=0.0, stddev=1.0, dtype=tf.float32, name='t3f_tensor_batch_with_random_cores')

Generate a batch of TT-tensors of given shape with N(mean, stddev^2) cores.

Parameters:
  • shape – array representing the shape of the future tensor.
  • tt_rank – a number or a (d+1)-element array with ranks.
  • batch_size – an integer.
  • mean – a number, the mean of the normal distribution used for initializing TT-cores.
  • stddev – a number, the standard deviation of the normal distribution used for initializing TT-cores.
  • dtype – [tf.float32] dtype of the resulting tensor.
  • name – string, name of the Op.
Returns:

TensorTrainBatch containing TT-tensors
t3f.tensor_ones(shape, dtype=tf.float32, name='t3f_tensor_ones')

Generate TT-tensor of the given shape with all entries equal to 1.

Parameters:
  • shape – array representing the shape of the future tensor
  • dtype – [tf.float32] dtype of the resulting tensor.
  • name – string, name of the Op.
Returns:

TensorTrain object containing a TT-tensor
t3f.tensor_with_random_cores(shape, tt_rank=2, mean=0.0, stddev=1.0, dtype=tf.float32, name='t3f_tensor_with_random_cores')

Generate a TT-tensor of the given shape with N(mean, stddev^2) cores.

Parameters:
  • shape – array representing the shape of the future tensor.
  • tt_rank – a number or a (d+1)-element array with the desired ranks.
  • mean – a number, the mean of the normal distribution used for initializing TT-cores.
  • stddev – a number, the standard deviation of the normal distribution used for initializing TT-cores.
  • dtype – [tf.float32] dtype of the resulting tensor.
  • name – string, name of the Op.
Returns:

TensorTrain containing a TT-tensor
t3f.tensor_zeros(shape, dtype=tf.float32, name='t3f_tensor_zeros')

Generate TT-tensor of the given shape with all entries equal to 0.

Parameters:
  • shape – array representing the shape of the future tensor
  • dtype – [tf.float32] dtype of the resulting tensor.
  • name – string, name of the Op.
Returns:

TensorTrain object containing a TT-tensor
t3f.to_tt_matrix(mat, shape, max_tt_rank=10, epsilon=None, name='t3f_to_tt_matrix')

Converts a given matrix or vector to a TT-matrix.

The matrix dimensions should factorize into d numbers. If e.g. the dimensions are prime numbers, it’s usually better to pad the matrix with zeros until the dimensions factorize into (ideally) 3-8 numbers.

Parameters:
  • mat – two dimensional tf.Tensor (a matrix).
  • shape – two dimensional array (np.array or list of lists) Represents the tensor shape of the matrix. E.g. for a (a1 * a2 * a3) x (b1 * b2 * b3) matrix shape should be ((a1, a2, a3), (b1, b2, b3)) shape[0]` and shape[1]` should have the same length. For vectors you may use ((a1, a2, a3), (1, 1, 1)) or, equivalently, ((a1, a2, a3), None)
  • max_tt_rank –

    a number or a list of numbers If a number, than defines the maximal TT-rank of the result. If a list of numbers, than max_tt_rank length should be d+1 (where d is the length of shape[0]) and max_tt_rank[i] defines the maximal (i+1)-th TT-rank of the result. The following two versions are equivalent

    max_tt_rank = r
    and
    max_tt_rank = r * np.ones(d-1)
  • epsilon –

    a floating point number or None If the TT-ranks are not restricted (max_tt_rank=np.inf), then the result would be guarantied to be epsilon close to mat in terms of relative Frobenius error:

    ||res - mat||_F / ||mat||_F <= epsilon

    If the TT-ranks are restricted, providing a loose epsilon may reduce the TT-ranks of the result. E.g.

    to_tt_matrix(mat, shape, max_tt_rank=100, epsilon=0.9)

    will probably return you a TT-matrix with TT-ranks close to 1, not 100. Note that providing a nontrivial (= not equal to None) epsilon will make the TT-ranks of the result undefined on the compilation stage (e.g. res.get_tt_ranks() will return None, but t3f.tt_ranks(res).eval() will work).

  • name – string, name of the Op.
Returns:

TensorTrain object containing a TT-matrix.
Raises:

ValueError if max_tt_rank is less than 0, if max_tt_rank is not a number and – not a vector of length d + 1 where d is the number of dimensions (rank) of the input tensor, if epsilon is less than 0.
t3f.to_tt_tensor(tens, max_tt_rank=10, epsilon=None, name='t3f_to_tt_tensor')

Converts a given tf.Tensor to a TT-tensor of the same shape.

Parameters:
  • tens – tf.Tensor
  • max_tt_rank –

    a number or a list of numbers If a number, than defines the maximal TT-rank of the result. If a list of numbers, than max_tt_rank length should be d+1 (where d is the rank of tens) and max_tt_rank[i] defines the maximal (i+1)-th TT-rank of the result. The following two versions are equivalent

    max_tt_rank = r
    and
    max_tt_rank = r * np.ones(d-1)
  • epsilon –

    a floating point number or None If the TT-ranks are not restricted (max_tt_rank=np.inf), then the result would be guarantied to be epsilon close to tens in terms of relative Frobenius error:

    ||res - tens||_F / ||tens||_F <= epsilon

    If the TT-ranks are restricted, providing a loose epsilon may reduce the TT-ranks of the result. E.g.

    to_tt_tensor(tens, max_tt_rank=100, epsilon=0.9)

    will probably return you a TT-tensor with TT-ranks close to 1, not 100. Note that providing a nontrivial (= not equal to None) epsilon will make the TT-ranks of the result undefined on the compilation stage (e.g. res.get_tt_ranks() will return None, but t3f.tt_ranks(res).eval() will work).

  • name – string, name of the Op.
Returns:

TensorTrain object containing a TT-tensor.
Raises:

ValueError if the rank (number of dimensions) of the input tensor is – not defined, if max_tt_rank is less than 0, if max_tt_rank is not a number and not a vector of length d + 1 where d is the number of dimensions (rank) of the input tensor, if epsilon is less than 0.
t3f.transpose(tt_matrix, name='t3f_transpose')

Transpose a TT-matrix or a batch of TT-matrices.

Parameters:
  • tt_matrix – TensorTrain or TensorTrainBatch object containing a TT-matrix (or a batch of TT-matrices).
  • name – string, name of the Op.
Returns:

TensorTrain or TensorTrainBatch object containing a transposed TT-matrix

(or a batch of TT-matrices).
Raises:

ValueError if the argument is not a TT-matrix.
t3f.tt_ranks(tt, name='t3f_tt_ranks')

Returns the TT-ranks of a TensorTrain.

This operation returns a 1-D integer tensor representing the TT-ranks of the input.

Parameters:
  • tt – TensorTrain or TensorTrainBatch object.
  • name – string, name of the Op.
Returns:

A Tensor
t3f.zeros_like(tt, name='t3f_zeros_like')

Constructs t3f.zeros with the shape of tt.

In the case when tt is a TensorTrainBatch constructs t3f.zeros with the shape of a TensorTrain in tt.

Parameters:
  • tt – TensorTrain object
  • name – string, name of the Op.
Returns:

TensorTrain object of the same shape as tt but with all entries equal to 0.

t3f.nn module

Utils for simplifying building neural networks with TT-layers

class t3f.nn.KerasDense(input_dims, output_dims, tt_rank=2, activation=None, use_bias=True, kernel_initializer='glorot', bias_initializer=0.1, **kwargs)

Bases: tensorflow.python.keras.engine.base_layer.Layer

call(x)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:
  • inputs – Input tensor, or list/tuple of input tensors.
  • **kwargs – Additional keyword arguments. Currently unused.
Returns:

A tensor or list/tuple of tensors.
compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
Returns:An input shape tuple.

t3f.utils module

t3f.utils.in_eager_mode()

Checks whether tensorflow eager mode is avaialable and active.

t3f.utils.replace_tf_svd_with_np_svd()

Replaces tf.svd with np.svd. Slow, but a workaround for tf.svd bugs.

t3f.utils.unravel_index(indices, shape)

t3f.kronecker module

t3f.kronecker.cholesky(kron_a, name='t3f_kronecker_cholesky')

Computes the Cholesky decomposition of a given Kronecker-factorized matrix.

Parameters:
  • kron_a – TensorTrain or TensorTrainBatch object containing a matrix or a batch of matrices of size N x N, factorized into a Kronecker product of square matrices (all tt-ranks are 1 and all tt-cores are square). All the cores must be symmetric positive-definite.
  • name – string, name of the Op.
Returns:

TensorTrain object containing a TT-matrix of size N x N if the argument is

TensorTrain

TensorTrainBatch object, containing TT-matrices of size N x N if the

argument is TensorTrainBatch
Raises:
  • ValueError if the tt-cores of the provided matrix are not square,
  • or the tt-ranks are not 1.
t3f.kronecker.determinant(kron_a, name='t3f_kronecker_determinant')

Computes the determinant of a given Kronecker-factorized matrix.

Note, that this method can suffer from overflow.

Parameters:
  • kron_a – TensorTrain or TensorTrainBatch object containing a matrix or a batch of matrices of size N x N, factorized into a Kronecker product of square matrices (all tt-ranks are 1 and all tt-cores are square).
  • name – string, name of the Op.
Returns:

A number or a Tensor with numbers for each element in the batch. The determinant of the given matrix.
Raises:
  • ValueError if the tt-cores of the provided matrix are not square,
  • or the tt-ranks are not 1.
t3f.kronecker.inv(kron_a, name='t3f_kronecker_inv')

Computes the inverse of a given Kronecker-factorized matrix.

Parameters:
  • kron_a – TensorTrain or TensorTrainBatch object containing a matrix or a batch of matrices of size N x N, factorized into a Kronecker product of square matrices (all tt-ranks are 1 and all tt-cores are square).
  • name – string, name of the Op.
Returns:

TensorTrain object containing a TT-matrix of size N x N if the argument is

TensorTrain

TensorTrainBatch object, containing TT-matrices of size N x N if the

argument is TensorTrainBatch
Raises:
  • ValueError if the tt-cores of the provided matrix are not square,
  • or the tt-ranks are not 1.
t3f.kronecker.slog_determinant(kron_a, name='t3f_kronecker_slog_determinant')

Computes the sign and log-det of a given Kronecker-factorized matrix.

Parameters:
  • kron_a – TensorTrain or TensorTrainBatch object containing a matrix or a batch of matrices of size N x N, factorized into a Kronecker product of square matrices (all tt-ranks are 1 and all tt-cores are square).
  • name – string, name of the Op.
Returns:

Two number or two Tensor with numbers for each element in the batch. Sign of the determinant and the log-determinant of the given matrix. If the determinant is zero, then sign will be 0 and logdet will be -Inf. In all cases, the determinant is equal to sign * np.exp(logdet).
Raises:
  • ValueError if the tt-cores of the provided matrix are not square,
  • or the tt-ranks are not 1.

t3f.approximate module

t3f.approximate.add_n(tt_objects, max_tt_rank, name='t3f_approximate_add_n')

Adds a bunch of TT-object and round after each summation.

This version implements a slow-to-compile but fast-to-execute (at least on a GPU) version: summing in a binary tree order. I.e. it uses the following idea:

round(a + b + c + d) ~= round(round(a + b) + round(c + d))

and so is able to compute the answer in log(N) parallel adds/rounds.

Parameters:
  • tt_objects – a list of TensorTrainBase objects.
  • max_tt_rank – a number, TT-rank for each individual rounding.
  • name – string, name of the Op.
Returns:

Object of the same type as each input.

See also

t3f.approximate.reduce_sum_batch

t3f.approximate.reduce_sum_batch(tt_batch, max_tt_rank, coef=None, name='t3f_approximate_reduce_sum_batch')

Sum of all TT-objects in the batch with rounding after each summation.

This version implements a slow-to-compile but fast-to-execute (at least on a GPU) version: summing in a binary tree order. I.e. it uses the following idea:

round(a + b + c + d) ~= round(round(a + b) + round(c + d))

and so is able to compute the answer in log(batch_size) parallel adds/rounds.

Parameters:
  • tt_batch – TensorTrainBatch object.
  • max_tt_rank – a number, TT-rank for each individual rounding.
  • coef –

    tf.Tensor, its shape is either batch_size, or batch_size x N. If coef is a vecotor of size batch_size, the result will

    be (approximate) weighted sum.
    If coef is a matrix of shape batch_size x N, the result will be
    a TensorTrainBatch res containing N TT-object such that
    res[j] ~= sum_i tt_batch[i] coef[i, j]
  • name – string, name of the Op.
Returns:

If coefficients are absent or is a vector of numbers, returns

a TensorTrain object representing (approximate) element-wise sum of all the objects in the batch, weighted if coef is provided.

If coefficients is a matrix, returns TensorTrainBatch.

See also

t3f.approximate.add_n