python - NaN values in Pandas are not being filled by the interpolate function when it's applied to a full dataframe - S

So, i'm a beginner at the Pandas Python and noticed the interpolate function is pretty interesting, but i have one problem when using the line:

result = df.interpolate(method='linear')

I found out that even though it did filled a lot of the NaN's in 'df', the four first NaN's of Ambient_temp and the first NaN on the Intake_temp column are not filled the way i wanted. Any hints on how to get this working? The interpolation worked very well with every other column besides those two.

Image of said dataframe

Example:

amb_temp = [np.nan, np.nan, 32, 32]
in_temp = [ 29, 27, 23, 22]
volts = [np.nan, 13, 11, 11]

dict = {'ambient_temperature': amb_temp, 'temperature_inside': in_temp, 'volts': volts} 

df = pd.DataFrame(dict)

(it's not exactly the same dataframe, but encapsulates the same problem. I got this one based off and example on 'geeksforgeeks' and used numpy.nan to simulate the absence of data.)

So, i'm a beginner at the Pandas Python and noticed the interpolate function is pretty interesting, but i have one problem when using the line:

result = df.interpolate(method='linear')

I found out that even though it did filled a lot of the NaN's in 'df', the four first NaN's of Ambient_temp and the first NaN on the Intake_temp column are not filled the way i wanted. Any hints on how to get this working? The interpolation worked very well with every other column besides those two.

Image of said dataframe

Example:

amb_temp = [np.nan, np.nan, 32, 32]
in_temp = [ 29, 27, 23, 22]
volts = [np.nan, 13, 11, 11]

dict = {'ambient_temperature': amb_temp, 'temperature_inside': in_temp, 'volts': volts} 

df = pd.DataFrame(dict)

(it's not exactly the same dataframe, but encapsulates the same problem. I got this one based off and example on 'geeksforgeeks' and used numpy.nan to simulate the absence of data.)

• python
• pandas

Share Improve this question Follow edited Jan 5 at 17:01 Salier1 asked Jan 4 at 23:06 Salier1Salier1 1533 bronze badges 4

• 1 Please post a small example (not a picture) that people can copy/paste. Looks like for the 2 columns you quoted the first values were NaN, if it is the case it's normal that these nan were not interpolated as the algorithm needs a first value to start interpolating. – rehaqds Commented Jan 4 at 23:44
• 1 So what values would you want to replace the first NaN values? – user19077881 Commented Jan 5 at 0:36
• I'll edit it and use a better example. Thanks, for the two answers, i think i now understand that pandas can't fill the first missing values. It does makes a lot of sense that it has these NaN's being the way the user 'rehaqds' said. And yes, i do want the first NaN's to be filled 'user19077881'. – Salier1 Commented Jan 5 at 16:53
• With what values or logic you want the NaN to be filled? – rehaqds Commented Jan 6 at 7:35

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

This is it:

import numpy as np
import pandas as  pd
amb_temp = [np.nan, np.nan, 32, 32]
in_temp = [ 29, 27, 23, 22]
volts = [np.nan, 13, 11, 11]

dict1 = {'ambient_temperature': amb_temp, 'temperature_inside': in_temp, 'volts': volts} 
keys = list(dict1.keys())

for k in keys:
    data_array = np.array(dict1[k])
    print("1   {}".format(data_array))
    not_nan = ~np.isnan(data_array)
    indices = np.arange(len(data_array))
    dict1[k] = np.interp(indices, indices[not_nan], data_array[not_nan])
print(dict1)