Deep Learning - Getting started with TensorFlow

Contents:


What is TensorFlow (TF)?

The best description can be found at the TensorFlow website:

" TensorFlow is a programming system in which you represent computations as graphs. Nodes in the graph are called ops (short for operations). An op takes zero or more Tensors, performs some computation, and produces zero or more Tensors. A Tensor is a typed multi-dimensional array. For example, you can represent a mini-batch of images as a 4-D array of floating point numbers with dimensions [batch, height, width, channels].

A TensorFlow graph is a description of computations. To compute anything, a graph must be launched in a Session. A Session places the graph ops onto Devices, such as CPUs or GPUs, and provides methods to execute them. These methods return tensors produced by ops as numpy ndarray objects in Python, and as tensorflow::Tensor instances in C and C++. The computation graph

TensorFlow programs are usually structured into a construction phase, that assembles a graph, and an execution phase that uses a session to execute ops in the graph.

For example, it is common to create a graph to represent and train a neural network in the construction phase, and then repeatedly execute a set of training ops in the graph in the execution phase.

TensorFlow can be used from C, C++, and Python programs. It is presently much easier to use the Python library to assemble graphs, as it provides a large set of helper functions not available in the C and C++ libraries.

The session libraries have equivalent functionalities for the three languages. "


Installing TensorFlow

Since I wanted to use TF with Windows 10, but TF is only supported for Linux + Mac OS X, I set up a VMWare which runs Ubuntu.

Then I followed these instructions, which uses pip to install TF (there are several approaches to install TF)

1. Installing pip

sudo apt-get install python-pip python-dev
or install pip3 for the Python3 package manager:
sudo apt-get install python3-pip

2. Checking your Python version:

python --version
gave in my case the result: Python 2.7.6 and
python3 --version
gave in my case the result: Python 3.4.3

3. Setting up paths

For Ubuntu/Linux 64-bit, CPU only, Python 2.7:

		export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0rc0-cp27-none-linux_x86_64.whl
and for Ubuntu/Linux 64-bit, CPU only, Python 3.4:

		export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0rc0-cp34-cp34m-linux_x86_64.whl

4. Using pip/pip3 to get the TF package:

For Python2:
sudo pip install --upgrade $TF_BINARY_URL
and for Python3:
sudo pip3 install --upgrade $TF_BINARY_URL

TensorFlow Hello World!

		
            # tf_add_two_values.py
            #
            # A very very simple TensorFlow graph that can
            # add two numbers! ;-)
            # 
            # It shows how to feed the input data in, using a dictionary
            # and running the computation graph several times (3x in this case)
            # with different inputs a,b.
            #
            # by Prof. Dr. Juergen Brauer, www.juergenbrauer.org

            import tensorflow as tf

            a = tf.placeholder("float")
            b = tf.placeholder("float")

            y = tf.add(a, b)

            with tf.Session() as sess:
                print("%f should equal   5.0" % sess.run(y, feed_dict={a: 2, b: 3})   )
                print("%f should equal  17.0" % sess.run(y, feed_dict={a: 9, b: 8})   )
                print("%f should equal 101.0" % sess.run(y, feed_dict={a: 1, b: 100}) )
        
Output:
5.000000 should equal   5.0
17.000000 should equal  17.0
101.000000 should equal 101.0
        

A first TensorFlow program

In the following Python script a simple TF graph is constructed and executed.

For more theory on TF graphs, here is a short description.

Note: In TF tensors are n-dimensional arrays. We can create a constant tensor (that does not change over time) with tf.constant():

Here is a first simple TensorFlow script:

		
			# A first simple TensorFlow program
			#
			# Constructs a very simple graph that
			# does just a little bit linear algebra
			#
			# by Prof. Dr. Juergen Brauer, www.juergenbrauer.org


			# import the TensorFlow module
			import tensorflow as tf

			# also import numpy for defining n-dimensional arrays directly
			import numpy as np



			# create two tensors (n-dimensional arrays)
			M1 = np.array([(1,2,3),(4,5,6),(7,8,9)])
			print("\nM1=",M1)
			tensor1 = tf.constant(M1)

			M2 = np.array([(10,20,30),(40,50,60),(70,80,90)])
			print("\nM2=",M2)
			tensor2 = tf.constant(M2)

			# create a graph op for adding the two matrices
			op1 = tf.add(tensor1,tensor2)

			# now get a session object:
			# A session is a specific distribution of the graph operations to
			# be performed on your computing resources: it depends on how many
			# CPUs and GPUs you have
			sess = tf.Session()

			# now let the graph work and compute the result
			result = sess.run(op1)

			# output the result of the computation
			print("\nresult=",result)

			print("\nData types of the objects used here:")
			print("tensor1 has data type", type(tensor1))
			print("tensor2 has data type", type(tensor2))
			print("op1 has data type",     type(op1))
			print("sess has data type",    type(sess))
			print("result has data type",  type(result))
			print("\n")
		

And here is the output generated by this script:

		
M1= [[1 2 3]
 [4 5 6]
 [7 8 9]]

M2= [[10 20 30]
 [40 50 60]
 [70 80 90]]

result= [[11 22 33]
 [44 55 66]
 [77 88 99]]

Data types of the objects used here:
tensor1 has data type <class 'tensorflow.python.framework.ops.Tensor'>
tensor2 has data type <class 'tensorflow.python.framework.ops.Tensor'>
op1 has data type <class 'tensorflow.python.framework.ops.Tensor'>
sess has data type <class 'tensorflow.python.client.session.Session'>
result has data type <class 'numpy.ndarray'>
		

A first TensorFlow program that learns

In the following Python script a simple TF graph is constructed that allows to do learn the parameters W,b of a line y(x)=W*x+b, just by the help of 100 random samples (x,y).

		
			# A first simple TensorFlow learner
			#
			# Given some random samples of 2D data points (x,y),
			# the following TF script tries to find a line
			# that best describes the data points in terms of
			# a minimal sum of squared errors (SSE)
			#
			# by Prof. Dr. Juergen Brauer, www.juergenbrauer.org

			import tensorflow as tf
			import numpy as np


			# 1.1 create 1D array with 100 random numbers drawn uniformly from [0,1)
			x_data = np.random.rand(100)
			print("\nx_data:",x_data)
			print("data type:", type(x_data))

			# 1.2 now compute the y-points
			y_data = x_data * 1.2345 + 0.6789

			# so now we have ground truth samples (x,y)
			# and the TF learner will have to estimate the line parameters
			# y=W*x+b with W=1.2345 and b=0.6789
			#
			# These parameters are called variables in TF.


			# 2. We initialize them with a random W and b=0
			W = tf.Variable(tf.random_uniform([1], -1.0, 1.0))
			b = tf.Variable(tf.zeros([1]))
			y = W * x_data + b


			# 3.1 Now define what to optimize at all:
			#     here we want to minimize the SSE.
			#     This is our "loss" of a certain line model
			loss_func = tf.reduce_mean(tf.square(y - y_data))

			# 3.2 We can use different optimizers in TF for model learning
			my_optimizer = tf.train.GradientDescentOptimizer(0.5)

			# 3.3 Tell the optimizer object to minimize the loss function
			train = my_optimizer.minimize(loss_func)


			# 4. Before starting, we have to initialize the variables
			init = tf.initialize_all_variables()


			# 5. Now distribute the graph to the computing hardware
			sess = tf.Session()


			# 6. Initialize the graph
			sess.run(init)


			# 7. Print inial value of W and b
			print("\n")
			print("initial W", sess.run(W))
			print("initial b", sess.run(b))


			# 8. For 200 steps...
			print("\n")
			for step in range(201):

				# 8.1 Do another gradient descent step to come to a better
				#     W and b
				sess.run(train)

				# 8.2 From time to time, print the current value of W and b
				if step % 10 == 0:
					print(step, sess.run(W), sess.run(b))

		

And here is the output of the script above:

x_data: [ 0.08570703  0.93417424  0.14190885  0.06902993  0.69436633  0.45719928
  0.10836465  0.34898391  0.92762256  0.72039116  0.4573859   0.26536097
  0.49598964  0.25114231  0.93833748  0.78320411  0.41994859  0.50204453
  0.10232796  0.46088877  0.07684852  0.79552978  0.0611582   0.88433202
  0.14683716  0.68133636  0.44524855  0.5703985   0.55787588  0.04207022
  0.30575287  0.59084949  0.81356629  0.69896455  0.62073628  0.51289546
  0.90904004  0.74049462  0.43985826  0.44551302  0.89189255  0.19392645
  0.33792151  0.4264049   0.06543103  0.21384447  0.5728932   0.72418163
  0.94863564  0.15639689  0.3534044   0.32512409  0.60381707  0.31953207
  0.82927607  0.37372332  0.72746845  0.81321755  0.67602859  0.64793116
  0.73948743  0.72175944  0.52924276  0.29625595  0.47429952  0.91291742
  0.98289152  0.76036165  0.89692963  0.63484564  0.61186549  0.70529708
  0.4011296   0.88547995  0.88947476  0.63369736  0.19920077  0.34257096
  0.80183467  0.71985152  0.50683508  0.87701425  0.38403252  0.47755784
  0.79836205  0.93505285  0.14845976  0.97750596  0.44757491  0.31257701
  0.52933848  0.34397994  0.82448212  0.93973516  0.61538141  0.96637458
  0.61971817  0.14383735  0.87284328  0.22125581]
data type: <class 'numpy.ndarray'>


initial W [-0.47123861]
initial b [ 0.]


0 [ 0.53861499] [ 1.61340666]
10 [ 0.71406066] [ 0.98095357]
20 [ 0.94207531] [ 0.84861481]
30 [ 1.07019317] [ 0.77425891]
40 [ 1.14217973] [ 0.73248005]
50 [ 1.18262732] [ 0.70900542]
60 [ 1.20535398] [ 0.69581556]
70 [ 1.21812344] [ 0.6884045]
80 [ 1.22529829] [ 0.68424046]
90 [ 1.22932982] [ 0.68190062]
100 [ 1.23159504] [ 0.68058598]
110 [ 1.23286784] [ 0.6798473]
120 [ 1.23358285] [ 0.67943227]
130 [ 1.23398471] [ 0.67919904]
140 [ 1.23421061] [ 0.67906797]
150 [ 1.23433733] [ 0.67899442]
160 [ 1.2344085] [ 0.67895311]
170 [ 1.23444855] [ 0.67892987]
180 [ 1.23447096] [ 0.67891681]
190 [ 1.23448384] [ 0.67890936]
200 [ 1.23449087] [ 0.67890525]