python - Inverse Transform ValueError: operands could not be broadcast together - Stack Overflow

I'm working through a baseball stats forecaster. I'm virtually complete, after having resolved numerous issues with the original model I pulled from a YouTube video. But I'm stuck at the very end, where I want to compile a summary df of the work.

At the point in question, I'm trying to revert my data back to its original values. Earlier in the program, I scaled numerous columns (labeled selected_columns) via MinMax Scaler and specifically, with this line:

batting.loc[:, selected_columns] = scaler.fit_transform(batting[selected_columns])

At the end of the program, I want to get back to the original data and tried doing so by using the following (upcoming_season is a dataframe of the next year's players along with their player_id, team, and all their stats which are set to 0 since they haven't played yet):

upcoming_season = upcoming_season.drop(['Name', 'Team', 'H', 'L-Hits', 'Next_Hits', 'player_season', 'h_diff'], axis=1)
next_season = pd.DataFrame(upcoming_season)
next_season = next_season.apply(pd.to_numeric)
inverse_next_season = scaler.inverse_transform(next_season)

The error I receive is generated by the last line (the inverse_transform line). Here is the Traceback:

ValueError                                Traceback (most recent call last)
Cell In[82], line 1
----> 1 inverse_next_season = scaler.inverse_transform(next_season)

File /opt/anaconda3/lib/python3.12/site-    packages/sklearn/preprocessing/_data.py:574, in MinMaxScaler.inverse_transform(self, X)
    564 xp, _ = get_namespace(X)
    566 X = check_array(
    567     X,
    568     copy=self.copy,
   (...)
    571     force_all_finite="allow-nan",
    572 )
--> 574 X -= self.min_
    575 X /= self.scale_
    576 return X

ValueError: operands could not be broadcast together with shapes (285,141) (137,) (285,141) 

What I can't figure out is where the "third" dataset (137,) is coming from.

I can include additional code, but I don't know what would be necessary to help you help me. So if you let me know what more may be useful, I'll be happy to comply.

I was only able to find two other questions that were somewhat similar, but not enough to help me with this specific situation.

python numpy ValueError: operands could not be broadcast together with shapes

tensorflow scaler.inverse_transform ValueError: operands could not be broadcast together with shapes (342,22) (23,) (342,22)

I'm working through a baseball stats forecaster. I'm virtually complete, after having resolved numerous issues with the original model I pulled from a YouTube video. But I'm stuck at the very end, where I want to compile a summary df of the work.

At the point in question, I'm trying to revert my data back to its original values. Earlier in the program, I scaled numerous columns (labeled selected_columns) via MinMax Scaler and specifically, with this line:

batting.loc[:, selected_columns] = scaler.fit_transform(batting[selected_columns])

At the end of the program, I want to get back to the original data and tried doing so by using the following (upcoming_season is a dataframe of the next year's players along with their player_id, team, and all their stats which are set to 0 since they haven't played yet):

upcoming_season = upcoming_season.drop(['Name', 'Team', 'H', 'L-Hits', 'Next_Hits', 'player_season', 'h_diff'], axis=1)
next_season = pd.DataFrame(upcoming_season)
next_season = next_season.apply(pd.to_numeric)
inverse_next_season = scaler.inverse_transform(next_season)

The error I receive is generated by the last line (the inverse_transform line). Here is the Traceback:

ValueError                                Traceback (most recent call last)
Cell In[82], line 1
----> 1 inverse_next_season = scaler.inverse_transform(next_season)

File /opt/anaconda3/lib/python3.12/site-    packages/sklearn/preprocessing/_data.py:574, in MinMaxScaler.inverse_transform(self, X)
    564 xp, _ = get_namespace(X)
    566 X = check_array(
    567     X,
    568     copy=self.copy,
   (...)
    571     force_all_finite="allow-nan",
    572 )
--> 574 X -= self.min_
    575 X /= self.scale_
    576 return X

ValueError: operands could not be broadcast together with shapes (285,141) (137,) (285,141) 

What I can't figure out is where the "third" dataset (137,) is coming from.

I can include additional code, but I don't know what would be necessary to help you help me. So if you let me know what more may be useful, I'll be happy to comply.

I was only able to find two other questions that were somewhat similar, but not enough to help me with this specific situation.

python numpy ValueError: operands could not be broadcast together with shapes

tensorflow scaler.inverse_transform ValueError: operands could not be broadcast together with shapes (342,22) (23,) (342,22)

• python
• inverse-transform

Share Improve this question Follow edited Feb 1 at 2:41 Richard C asked Feb 1 at 2:38 Richard CRichard C 111 silver badge44 bronze badges 2

• Have you looked at the shapes of X and self.min_ ? – Frank Yellin Commented Feb 1 at 2:45
• I wouldn't know where to begin looking for self.min. And X is a return value. Both of those are embedded within python (AFaIK) and not in my code directly. – Richard C Commented Feb 1 at 2:51

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

I was able to figure this out. Apparently, somewhere along the way I deleted several columns in the dataset I was attempting to reverse scaling on. Once I discovered that and made the necessary changes, the code, as written, worked as expected.

I suspect that the inverse.transform line was referencing a version of the dataset from earlier in the program, but at this time I can't see where. But this was my solution.