Estimation theory (Jan 10)

Today we studied estimation theory (recap of least squares and first look at maximum likelihood).

At the beginning of the first lecture, we saw the solutions to the Moodle exercise worth 3 points:
 

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

if __name__ == '__main__':

    # Question 1: load data using numpy
    
    location_data = np.loadtxt("locationData.csv")
    print(np.shape(location_data))

    # Question 2: visualize using plt functions
    
    plt.plot(location_data[:, 0], location_data[:, 1], 'b-')
    plt.show()
    
    ax = plt.subplot(1, 1, 1, projection = "3d")
    plot3D = ax.plot(location_data[:,0], 
                      location_data[:,1], 
                      location_data[:,2])
    plt.show()
    
    # Question 3: Normalize data to zero mean and unit variance
    
    def normalize_data(data):
        
        return (data - np.mean(data, axis = 0)) / np.std(data, axis = 0)
        
    X_norm = normalize_data(location_data)
    
    print(np.mean(X_norm, axis = 0)) # Should be [0, 0, 0]
    print(np.std(X_norm, axis = 0)) # Should be [1, 1, 1]

The main point of the assignment was to make sure everyone has used Python and maybe installed Anaconda. I looked at a random sample of the returns, and they all look good. Thus I will add 3 points to all who have submitted any solution.

Some points raised while discussing the solution above:

• plt.plot used many many times in a sequence will not wipe old plots away.
• The function uses broadcasting. In Numpy, this is an efficient way to vectorize computation. For example, the statement
  data - np.mean(data, axis = 0)
  operates with data of dimensions 600x3 and 1x3. This is allowed if the trailing dimensions agree (both have last dimension == 3).

On the second hour the focus was on linear regression models: least squares and maximum likelihood. In regression the prediction target is real-valued and it is eay to consider accuracy in terms of output variance. In general, minimum variance unbiased estimators are desired but often hard to derive. Our key focus is in least squares and maximum likelihood.

It seems that this time the video lecture worked but the screen was missing (some cable was loose). The admins promised the next time all should work just fine. Also, there will be a live cast from now on.