Pandas Arrays
For most data types, pandas uses NumPy arrays as the concrete objects contained with a Index, Series, or DataFrame.
For some data types, pandas extends NumPy’s type system.
Pandas and third-party libraries can extend NumPy’s type system (see Extension Types). The top-level array() method can be used to create a new array, which may be stored in a Series, Index, or as a column in a DataFrame.
array(data[, dtype, copy]) | Create an array. |
Datetime Data
NumPy cannot natively represent timezone-aware datetimes. Pandas supports this with the arrays.DatetimeArray extension array, which can hold timezone-naive or timezone-aware values.
Timestamp, a subclass of datetime.datetime, is pandas’ scalar type for timezone-naive or timezone-aware datetime data.
Timestamp | Pandas replacement for datetime.datetime |
Properties
Methods
Timestamp.astimezone | Convert tz-aware Timestamp to another time zone. |
Timestamp.ceil | return a new Timestamp ceiled to this resolution |
Timestamp.combine(date, time) | date, time -> datetime with same date and time fields |
Timestamp.ctime | Return ctime() style string. |
Timestamp.date | Return date object with same year, month and day. |
Timestamp.day_name | Return the day name of the Timestamp with specified locale. |
Timestamp.dst | Return self.tzinfo.dst(self). |
Timestamp.floor | return a new Timestamp floored to this resolution |
Timestamp.freq | |
Timestamp.freqstr | |
Timestamp.fromordinal(ordinal[, freq, tz]) | passed an ordinal, translate and convert to a ts note: by definition there cannot be any tz info on the ordinal itself |
Timestamp.fromtimestamp(ts) | timestamp[, tz] -> tz’s local time from POSIX timestamp. |
Timestamp.isocalendar | Return a 3-tuple containing ISO year, week number, and weekday. |
Timestamp.isoformat | |
Timestamp.isoweekday | Return the day of the week represented by the date. |
Timestamp.month_name | Return the month name of the Timestamp with specified locale. |
Timestamp.normalize | Normalize Timestamp to midnight, preserving tz information. |
Timestamp.now([tz]) | Returns new Timestamp object representing current time local to tz. |
Timestamp.replace | implements datetime.replace, handles nanoseconds |
Timestamp.round | Round the Timestamp to the specified resolution |
Timestamp.strftime | format -> strftime() style string. |
Timestamp.strptime | string, format -> new datetime parsed from a string (like time.strptime()). |
Timestamp.time | Return time object with same time but with tzinfo=None. |
Timestamp.timestamp | Return POSIX timestamp as float. |
Timestamp.timetuple | Return time tuple, compatible with time.localtime(). |
Timestamp.timetz | Return time object with same time and tzinfo. |
Timestamp.to_datetime64 | Returns a numpy.datetime64 object with ‘ns’ precision |
Timestamp.to_julian_date | Convert TimeStamp to a Julian Date. |
Timestamp.to_period | Return an period of which this timestamp is an observation. |
Timestamp.to_pydatetime | Convert a Timestamp object to a native Python datetime object. |
Timestamp.today(cls[, tz]) | Return the current time in the local timezone. |
Timestamp.toordinal | Return proleptic Gregorian ordinal. |
Timestamp.tz_convert | Convert tz-aware Timestamp to another time zone. |
Timestamp.tz_localize | Convert naive Timestamp to local time zone, or remove timezone from tz-aware Timestamp. |
Timestamp.tzname | Return self.tzinfo.tzname(self). |
Timestamp.utcfromtimestamp(ts) | Construct a naive UTC datetime from a POSIX timestamp. |
Timestamp.utcnow() | Return a new Timestamp representing UTC day and time. |
Timestamp.utcoffset | Return self.tzinfo.utcoffset(self). |
Timestamp.utctimetuple | Return UTC time tuple, compatible with time.localtime(). |
Timestamp.weekday | Return the day of the week represented by the date. |
A collection of timestamps may be stored in a arrays.DatetimeArray. For timezone-aware data, the .dtype of a DatetimeArray is a DatetimeTZDtype. For timezone-naive data, np.dtype("datetime64[ns]") is used.
If the data are tz-aware, then every value in the array must have the same timezone.
arrays.DatetimeArray(values[, dtype, freq, copy]) | Pandas ExtensionArray for tz-naive or tz-aware datetime data. |
DatetimeTZDtype([unit, tz]) | A np.dtype duck-typed class, suitable for holding a custom datetime with tz dtype. |
Timedelta Data
NumPy can natively represent timedeltas. Pandas provides Timedelta for symmetry with Timestamp.
Timedelta | Represents a duration, the difference between two dates or times. |
Properties
Methods
A collection of timedeltas may be stored in a TimedeltaArray.
Timespan Data
Pandas represents spans of times as Period objects.
Period
Period | Represents a period of time |
Properties
Methods
Period.asfreq | Convert Period to desired frequency, either at the start or end of the interval |
Period.now | |
Period.strftime | Returns the string representation of the Period, depending on the selected fmt. |
Period.to_timestamp | Return the Timestamp representation of the Period at the target frequency at the specified end (how) of the Period |
A collection of timedeltas may be stored in a arrays.PeriodArray. Every period in a PeriodArray must have the same freq.
arrays.DatetimeArray(values[, dtype, freq, copy]) | Pandas ExtensionArray for tz-naive or tz-aware datetime data. |
PeriodDtype | A Period duck-typed class, suitable for holding a period with freq dtype. |
Interval Data
Arbitrary intervals can be represented as Interval objects.
Interval | Immutable object implementing an Interval, a bounded slice-like interval. |
Properties
A collection of intervals may be stored in an arrays.IntervalArray.
Nullable Integer
numpy.ndarray cannot natively represent integer-data with missing values. Pandas provides this through arrays.IntegerArray.
Categorical Data
Pandas defines a custom data type for representing data that can take only a limited, fixed set of values. The dtype of a Categorical can be described by a pandas.api.types.CategoricalDtype.
CategoricalDtype([categories, ordered]) | Type for categorical data with the categories and orderedness |
Categorical data can be stored in a pandas.Categorical
Categorical(values[, categories, ordered, …]) | Represents a categorical variable in classic R / S-plus fashion |
The alternative Categorical.from_codes() constructor can be used when you have the categories and integer codes already:
The dtype information is available on the Categorical
np.asarray(categorical) works by implementing the array interface. Be aware, that this converts the Categorical back to a NumPy array, so categories and order information is not preserved!
A Categorical can be stored in a Series or DataFrame. To create a Series of dtype category, use cat = s.astype(dtype) or Series(..., dtype=dtype) where dtype is either
- the string
'category'
- an instance of
CategoricalDtype.
If the Series is of dtype CategoricalDtype, Series.cat can be used to change the categorical data. See Categorical Accessor for more.
Sparse Data
Data where a single value is repeated many times (e.g. 0 or NaN) may be stored efficiently as a SparseArray.
The Series.sparse accessor may be used to access sparse-specific attributes and methods if the Series contains sparse values. See Sparse Accessor for more.