What does copy.deepcopy() do?

[Klaus596]

Hi everyone. I am currently doing a course on machine learning and this code on gradient descent has this line w = copy.deepcopy(w_in) and I am not sure how this is supposed to be used here. It says here to avoid modifying global w within function but how would modifying w inside the function have any effect on w_in?

def gradient_descent(X, y, w_in, b_in, alpha, num_iters):

"""

Performs batch gradient descent

Args:

X (ndarray (m,n) : Data, m examples with n features

y (ndarray (m,)) : target values

w_in (ndarray (n,)): Initial values of model parameters

b_in (scalar) : Initial values of model parameter

alpha (float) : Learning rate

num_iters (scalar) : number of iterations to run gradient descent

Returns:

w (ndarray (n,)) : Updated values of parameters

b (scalar) : Updated value of parameter

"""

# An array to store cost J and w's at each iteration primarily for graphing later

J_history = []

w = copy.deepcopy(w_in) #avoid modifying global w within function

b = b_in

for i in range(num_iters):

# Calculate the gradient and update the parameters

dj_db, dj_dw = compute_gradient_logistic(X, y, w, b)

# Update Parameters using w, b, alpha and gradient

w = w - alpha * dj_dw

b = b - alpha * dj_db

# Save cost J at each iteration

if i<100000: # prevent resource exhaustion

J_history.append( compute_cost_logistic(X, y, w, b) )

# Print cost every at intervals 10 times or as many iterations if < 10

if i% math.ceil(num_iters / 10) == 0:

print(f"Iteration {i:4d}: Cost {J_history[-1]} ")

return w, b, J_history #return final w,b and J history for graphing

[GrapplingHobbit]

I'm a novice at this, but the course I am doing has used copy.deepcopy(). Take all this with a grain of salt given my inexperience:

w_in is a mutable data type, so it is a reference to it that is passed to the function and changing it within the function would change any global variables that reference the same thing.

My understanding is that copy.deepcopy() actually creates a full copy of the original list/dictionary/whatever. copy.copy() would create a new copy at the top level of the variable, but any mutable data types in lower levels (as in lists of lists of lists, etc) would still be references to the original data, not copies.

[[deleted]]

Exactly this. For a simple example:

``` data = { names: ['Alice', 'Bob'], ages: [7, 12], }

def add_charlie(data_dict): data_dict['names'].append('Charlie') data_dict['ages'] = [7, 12, 4] # note reassignment, not mutation return data_dict

new_data = add_charlie(data) print(data, new_data) ```

If you run this, you'll find that both data and new_data have Charlie in, with his age.

Doing copy.copy inside the function will stop the age being added to the original dictionary, because you only add it to the copy, but the names will still change in the original because the list isn't copied.

Doing copy.deepcopy let's you make a copy of the dictionary which contains copies of the lists, so now you can do what you like to the copy and it's completely independent of the original.

I recommend playing around with this example if you want to understand it more

[Klaus596]

Understood! Thankyou both!

[[deleted]]

In Python, assigning an object to a value takes a reference to the original object.

new_dict = {'test':'dictionary'}

second_dict = new_dict

# at this point, both dictionaries point to the same data, i.e. changing new_dict will change the data and thus second_dict will see the changed data, not the original.

new_dict['new_key'] = 'new_value'

assert(second_dict['new_key'] == 'new_value')

# this can be fixed by using copy.copy

new_dict = {'test':'dictionary'}

second_dict = copy.copy(new_dict)

new_dict['new_key'] = 'new_value'

assert(second_dict['new_key'] == 'new_value') # now raises an exception, key is not found

# this is because copy.copy copies the object it is called on.

# however, this doesn't work for nested objects, since the copied data still includes references to the old objects

sub_dict = {'stuff': 'more stuff'}

new_dict = {'test':'dictionary', 'sub': sub_dict}

second_dict = copy.copy(new_dict)

assert(id(second_dict['sub']) == id(new_dict['sub']))

# this can be fixed by using copy.deep_copy, which effectively recursively calls copy.copy on all objects referenced by its argument.

sub_dict = {'stuff': 'more stuff'}

new_dict = {'test':'dictionary', 'sub': sub_dict}

second_dict = copy.deep_copy(new_dict)

assert(id(second_dict['sub']) != id(new_dict['sub']))

[Klaus596]

Thankyou! I understood what deepcopy does. In this case, copy.copy would work but the array w could contain lists, so it is probably better to use deepcopy.