Tensors

Tips:

• 0 ∈ ｛0 1}
• 0 1} represents a set with two elements 0 and 1. Therefore, 0 belongs to this set and is represented by 0 ∈ {0 1}. ∈ represents belonging to.
• {0} ∈ {0 1} is wrong, one set cannot belong to another set.
• Reverse E: predicate logic, exists
• The existence of the quantifier ∃ x:P(x) means that there is at least one x such that P(x) is true. n ∈ N:n is an even number.
• Inverted A: predicate logic, for all; for any; for each
• The universal quantifier ∀ x:P(x) means that P(x) is true for all x. n ∈ N:n2 ≥ n.
• ∧ Logical conjunction statement A ∧ B is true if both A and B are true; otherwise it is false. n < 4 ∧ n >2 ⇔ n = 3 when n is a natural number.
• ∨ Logical disjunction The statement A ∨ B is true if A or B (or both) is true; if both are false, the statement is false. n ≥ 4 ∨ n ≤ 2 ⇔ n ≠ 3 when n is a natural number.

scalar, vector, matrix, tensor

(1) Scalar:

In computer learning, I think a scalar can be understood as a sign used for statistics or marking a value in this type of mathematical event. For example, when it represents the slope of a line, it has only one use, and it only needs this one. It does not need any other definition, it just represents the slope. It only has one value. For another example, we can define a scalar in an array, and this scalar is the number of the array. So I understand a scalar likeThe only answer to a math problem. Just like The scalar is equal to the answer. The value of the scalar is equal to the result. If the question changes, then the scalar is not the original scalar.

Display method:

Express the slope of a line: Let s ∈ R represent the slope of a line’

Express the number of arrays: Let n ∈ N represent the number of elements

(2)Vector:

A vector represents a column of numbers. These numbers must be arranged in order. We can determine each individual number according to the sequence of the index, which is usually represented by lowercase bold font, such asx. The elements in a vector can be represented by italics with subscripts. The first element of the vector x is, then the second element is, and so on. If each element belongs to a scalar R, and the vector has n elements, then the vector belongs to the set of n Cartesian products of the set R of real numbers. When we need to explicitly represent the elements in a vector, we arrange the elements into a column surrounded by square brackets

Figure 1 Vector

We can think of vectors as points in space, with each element being a coordinate on a different axis.

Sometimes we need to index some elements in a vector. In this case, we define an index collection containing these elements and then write this collection at the footer. For example, specify,and, we define the set s={1,3,6}, and then write。

(3) Matrix:

A matrix is a two-dimensional array in which each element is indexed by two instead of one. You can think of two vectors put together like a zipper. Only the combination of two can be counted as a whole. Of course, the matrix does not necessarily mean that there are two columns. There can be several columns. It is common to give matrices variable names in bold uppercase, sayA. If a real matrix has height m and width n then we sayA ∈ Rmxn . When we represent elements in a matrix. It is usually written in italics without bolding names, with indexes separated by commas. For example, A1,1 represents the element in the upper left corner of matrix A, and Am,n represents the element in the lower right corner of A. We use: to indicate that a row or a column is to be used. For example, Am,: means that I want to use all elements of m rows.

Figure 2 Matrix

Sometimes we need indices of matrix-valued expressions rather than individual elements. In this case, we follow the expression with a subscript but do not have to lowercase the variable names of the matrix. (That is, I want to evaluate the element function in row i and column j. At this time, our a can be capitalized as A) For example, if f(A)i,j is written like this, it means that the function f() acts on the element in row i and column j of the matrix on A.

Matrix is not too important in machine learning! In fact, if we now have data of N users, and each piece of data contains M features, then it actually corresponds to an NThe matrix of M; for another example, a picture consists of 16Composed of 16 pixels, then this is a 16*16 matrix. Only now do I realize that the matrix principle we learned in freshman year is so useful! If the teacher had popularized it first when giving lectures, many students would not feel confused when learning matrix.

(4)Tensor:

The tensor defined in geometric algebra is based on the generalization of vectors and matrices. To put it more simply, we can regard scalars as zero-order tensors, vectors as first-order tensors, and matrices as second-order tensors. For example, any color picture can be represented as a third-order tensor, and the three dimensions are the height, width and color data of the picture.

Representing this picture as a tensor is the table at the bottom:

Figure 3 Tensor

The horizontal axis of the table represents the width value of the picture, and only 0 is intercepted here._{319; The vertical axis of the table represents the height value of the picture, and only 0 is intercepted here.}4; Each square in the table represents a pixel. For example, the table data in the first row and column is [1.0, 1.0, 1.0], which represents the values of the RGB three primary colors at this position in the picture (i.e. R=1.0, G=1.0, B=1.0).

Of course, we can continue to expand this definition, that is: we can use a fourth-order tensor to represent a data set containing multiple pictures. The four dimensions are: the number of the picture in the data set, the picture height, width, and color data.

Tensor is a very important concept in deep learning, because it is a core component in a deep learning framework, and almost all subsequent operations and optimization algorithms are based on tensors.

To understand in depth from a mathematical perspective:

concept

The tensor defined in geometric algebra is based on the generalization of vectors and matrices. To put it more simply, we can regard scalars as zero-order tensors, vectors as first-order tensors, and matrices as second-order tensors.

Figure 4 Tensor concept

definition

The strict definition of tensors is described by linear mapping. Similar to vectors, a set consisting of several ordered numbers that satisfy a certain coordinate transformation relationship when the coordinate system changes is defined as a tensor. From a geometric point of view, it is a real geometric quantity, that is to say, it is something that does not change with the coordinate transformation of the reference system (actually, the basis vector changes). The final result is that the combination of the basis vector and the components on the corresponding basis vector (that is, the tensor) remains unchanged, such as a first-order tensor (vector)acan be expressed asa= xi+ yj. Since basis vectors can have rich combinations, tensors can represent very rich physical quantities.

Change the definition

A (p, q) type tensor is a mapping:

Figure 5 (p,q) type tensor

where V is a vector space and V* is the corresponding dual space.

Say it a bit

If a physical quantity has only a single value at a certain position of the object, then it is an ordinary scalar quantity, such as density. If it has different values ​​at the same position and viewed from different directions, and this number happens to be calculated by multiplying the viewing direction by a matrix, it is a tensor.

Understanding of tensors: **Tensors are quantities with magnitude and multiple directions. **The direction here refers to the order of the tensor.

Space dimension n: Generally we use 3-dimensional space, but it can also be 4-dimensional and above.

Tensor order m: Let’s talk about the order of tensor in a fixed 3-dimensional space. The order is less than or equal to the dimension, that is, m<=n.

This quantity is distinguished below: the difference between the order of the tensor (the number of directions of the tensor) and the dimension of the space where it is located (the number of directions of the space where it is located).

In two-dimensional space, each direction of the two-dimensional second-order tensor (plane stress tensor) can be represented by two directions in two-dimensional space. (Distinguish between the two directions of the second-order tensor and the two directions x and y of the two-dimensional space) So there are 2^2=4 directions in total.

In three-dimensional space, each direction of the three-dimensional second-order tensor (space stress tensor) can be represented by three directions in three-dimensional space. (Distinguish between the two directions of the second-order tensor and the three directions x, y, z of the three-dimensional space) So there are 3^2=9 directions in total.

tensor product

Do you know matrix products?

Figure 6 Matrix product

inner product of vectors

Figure 7 Inner product of vectors

and matrix and vector multiplication

Figure 8 Multiplication of matrices and vectors

So you find something in common. There is a similar indicator that is invisible after summing. If you just put two quantities together, without summing, and just construct a multilinearity, you have discovered the tensor product, such as vector

Figure 9 Tensor product

So you construct a matrix, which is a second-order tensor. Similarly, for matrices, of course,

Figure 10 Tensor product

Here you construct a fourth-order tensor.

There are many ways to understand things like tensor product, and there will be different views in different contexts. But if we compare it with matrix product, I think a better way to say it is that tensor product is a universal product, and matrix multiplication is a concretization.

We now have many matrices in our hands, and then we want to multiply two matrices. You will definitely not be able to think of how to multiply at first, but you can guess some of the most basic properties of products. For example, it must match the multiplication of numbers, and it must also match addition, which is the distributive law. No matter what this product is, it should have these basic properties. Then the tensor product appears at this time. It represents the broadest product and is also the weakest product. It only meets the basic properties mentioned above. Precisely because it is the weakest, all specific products can be regarded as being concretized from the results of tensor products, that is, they can be regarded as universal products, or the product of an envelope.

inMathematicsopen in new windowin,tensor product，

Figure 11 Tensor product

There are two (or more)tensor productGeneral formula for components. For example, ifUandVThe ranks are respectivelynandmof twocovarianceopen in new windowtensors, then the components of their tensor product are given as

Figure 12 Tensor product

So the components of the tensor product of two tensors are the ordinary products of the components of each tensor.

What can vectors represent?

For example, we can use the normal vector of a plane to represent the plane; in physics, we can use vectors to represent forces, etc. It seems that vectors can represent many things, but if you think about it carefully, vectors only represent two elements: magnitude and direction.

There are many ways to represent a vector. We can use [0, 1] to represent a two-dimensional vector, or we can use a line with an arrow in a plane, three-dimensional or higher-dimensional space to represent a vector. We all know that (0, 0) —> (1, 1) can represent a directed line segment (vector) from (0, 0) to (1, 1). So, why can [0, 1] be used to represent a vector?

According to the previous explanation, we know that a vector is a directed line segment in space, which can be represented by the product combination of the basis of a set of coordinate systems and the corresponding components of the vector. Since there are many ways to define coordinate systems, there are many kinds of bases, and there are many kinds of corresponding components. But if everyone uses the same set of basis vectors by default, then the basis vectors are not needed. At this time, if you want to represent a vector, you only need to give these three components. For example, use 0, 1 to represent a vector. If you add two parentheses, this is the column representation of vectors we often see in books (0, 1), and the three-dimensional ones are (1, 2, 1). Post a very loving picture