DEPR: deprecate errors='ignore' in to_datetime and make output dtype predictable

[MarcoGorelli]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

currently, if you do to_datetime(inputs), you don't really know what the dtype of the output will be. It could be Index or DatetimeIndex

Deprecating parsing mixed offsets goes part of the way to addressing this

Feature Description

Can we go all the way there, and deprecate errors='ignore'? Then, if the computation succeeds, then you get DatetimeIndex (potentially with some NaTs if errors='coerce')

Alternative Solutions

none that I can think of

Additional Context

there's talk about query optimisation in pandas, and greater predictability should help with that

[jbrockmendel]

+1

[mroeschke]

Likewise for to_timedelta and to_numeric

[Cyddharth-Gupta]

@MarcoGorelli I am willing to take up this issue but I was hoping if I can get more info into this, if we deprecate errors='ignore' then shall it return a warning that it is deprecated?

[MarcoGorelli]

yup, that's right!

[Cyddharth-Gupta]

okay, then I will take up this issue.

[Cyddharth-Gupta]

take

[SteffenMeinecke]

@MarcoGorelli @Cyddharth-Gupta For my applications, errors="ignore" is what I need. What is the recommended code for pandas 3.0 to get previous behavior?
Consider the following example:

df1 = pd.DataFrame([[1, 1, 2, 4, 6], [7, 7, 5., 4.1, "h"], [None, 3, 5.2, 4, "ghjh"]])
print(df1.dtypes)
# code instead of `df2 = df1.apply(pd.to_numeric, errors='ignore')`
print(df2.dtypes)

and the expected output:

0 object
1 object
2 object
3 object
4 object
dtype: object

0 float64
1 int64
2 float64
3 float64
4 object
dtype: object

[MarcoGorelli]

that's a different function, this issue is just about to_Datetime

[SteffenMeinecke]

Thanks @MarcoGorelli for replying. Since @mroeschke mentioned that it is the same for to_numeric(), I thought it would be this issue. Can you guide me to the right one?

[dbrtly]

what is the recommended refactor for a line of code like this that gets a warning in pytest?

expected[col] = pd.to_datetime(expected[col], errors="ignore").dt.time

[MarcoGorelli]

i'd suggest to wrap it in try-except

[MohitBurkule]

@MarcoGorelli @Cyddharth-Gupta For my applications, errors="ignore" is what I need. What is the recommended code for pandas 3.0 to get previous behavior? Consider the following example:

df1 = pd.DataFrame([[1, 1, 2, 4, 6], [7, 7, 5., 4.1, "h"], [None, 3, 5.2, 4, "ghjh"]])
print(df1.dtypes)
# code instead of `df2 = df1.apply(pd.to_numeric, errors='ignore')`
print(df2.dtypes)

and the expected output:

0 object 1 object 2 object 3 object 4 object dtype: object

0 float64 1 int64 2 float64 3 float64 4 object dtype: object

The below might work

df2 = df1.apply(pd.to_numeric, errors='coerce')
df2=df2.fillna(df1)