API: argmin/argmax behaviour for nullable dtypes with skipna=False

[jorisvandenbossche]

Now we are adding ExtensionArray.argmin/argmax (#24382 / #27801), we also need to decide on its behaviour for nullable / masked arrays.
For the default of skipna=True, I think the behaviour is clear (we skip those values, and calculate the argmin/argmax of the remaining values), but we need to decide on the behaviour in case of skipna=False in presence of missing values.

I don't think it makes much sense to follow the current behaviour for Series with NaNs:

In [25]: pd.Series([1, 2, 3, np.nan]).argmin() 
Out[25]: 0

In [26]: pd.Series([1, 2, 3, np.nan]).argmin(skipna=False)  
Out[26]: -1

(the -1 is discussed separately in #33941 as well)

We also can't really compare with numpy, as there NaNs are regarded as the largest values (which follows somewhat from the sorting behaviour with NaNs).

For other reductions, the logic is: once there is an NA present, the result is also an NA with skipna=False (with the logic here that the with NAs, the min/max is not known, and thus also not the location).

So something like:

>>> pd.Series([1, 2, 3, pd.NA], dtype="Int64").argmin(skipna=False)  
<NA>

However, this also means that argmin/argmax return something that is not an integer, and since those methods are typically used to get a value that can be used to index, that might also be annoying.
An alternative could also be to raise an error (similar as for empty arrays).

[dsaxton]

I think your suggestion that argmin / argmax should return NA when we have NA and skipna is False makes total sense (I would even say that any other behavior is wrong / inconsistent with how NA is already defined / implemented)

[arw2019]

I'm +1 on returning NA with skipna=False

cc @pandas-dev/pandas-core @pandas-dev/pandas-triage there's a PR in the works to implement this once we decide on the behavior

[jbrockmendel]

Discussed on last week's dev call, agreed that argmax/argmin should always return an integer regardless of dtype. Agreed to deprecate the current cases that return -1 in favor of raising.

[rhshadrach]

I'm a bit late to the party, but was idxmin/max discussed? In groupby with categorical groups and observed=False we need to decide on a behavior: #10694

[jbrockmendel]

we agreed last week that the relevant cases (where argmin/argmax return -1s) should also be deprecated and eventually raise. ive got an upcoming PR for that deprecation. Will that have downstream consquences we may not have considered?

[rhshadrach]

Not necessarily - at most it would be from a high level "API Consistency" type of view, but perhaps not even that. I've never been sure what idxmin/idxmax should return in the case of a "missing value", and raising a better error message than the one that occurs in #10694 sounds like a good path forward to me.