KeyError when using for loop on dataframe to plot histograms


Keywords:python 


Question: 

I have a dataframe similar to:

df = pd.DataFrame({'Date': ['2016-01-05', '2016-01-05', '2016-01-05', '2016-01-05', '2016-01-08', '2016-01-08', '2016-02-01'], 'Count': [1, 2, 2, 3, 2, 0, 2]})

and I am trying to plot a histogram of Count for each unique Date

I've tried:

for date in df.Date.unique(): 
    plt.hist([df[df.Date == '%s' %(date)]['Count']])
    plt.title('%s' %(date))

which results in

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-17-971a1cf07250> in <module>()
      1 for date in df.Date.unique():
----> 2     plt.hist([df[df.Date == '%s' %(date)]['Count']])
      3     plt.title('%s' %(date))

c:~\anaconda3\lib\site-packages\matplotlib\pyplot.py in hist(x, bins, range, normed, weights, cumulative, bottom, histtype, align, orientation, rwidth, log, color, label, stacked, hold, data, **kwargs)
   2963                       histtype=histtype, align=align, orientation=orientation,
   2964                       rwidth=rwidth, log=log, color=color, label=label,
-> 2965                       stacked=stacked, data=data, **kwargs)
   2966     finally:
   2967         ax.hold(washold)

c:~\anaconda3\lib\site-packages\matplotlib\__init__.py in inner(ax, *args, **kwargs)
   1816                     warnings.warn(msg % (label_namer, func.__name__),
   1817                                   RuntimeWarning, stacklevel=2)
-> 1818             return func(ax, *args, **kwargs)
   1819         pre_doc = inner.__doc__
   1820         if pre_doc is None:

c:~\anaconda3\lib\site-packages\matplotlib\axes\_axes.py in hist(self, x, bins, range, normed, weights, cumulative, bottom, histtype, align, orientation, rwidth, log, color, label, stacked, **kwargs)
   5925 
   5926         # basic input validation
-> 5927         flat = np.ravel(x)
   5928 
   5929         input_empty = len(flat) == 0

c:~\anaconda3\lib\site-packages\numpy\core\fromnumeric.py in ravel(a, order)
   1482         return asarray(a).ravel(order=order)
   1483     else:
-> 1484         return asanyarray(a).ravel(order=order)
   1485 
   1486 

c:~\anaconda3\lib\site-packages\numpy\core\numeric.py in asanyarray(a, dtype, order)
    581 
    582     """
--> 583     return array(a, dtype, copy=False, order=order, subok=True)
    584 
    585 

c:~\anaconda3\lib\site-packages\pandas\core\series.py in __getitem__(self, key)
    581         key = com._apply_if_callable(key, self)
    582         try:
--> 583             result = self.index.get_value(self, key)
    584 
    585             if not lib.isscalar(result):

c:~\anaconda3\lib\site-packages\pandas\indexes\base.py in get_value(self, series, key)
   1978         try:
   1979             return self._engine.get_value(s, k,
-> 1980                                           tz=getattr(series.dtype, 'tz', None))
   1981         except KeyError as e1:
   1982             if len(self) > 0 and self.inferred_type in ['integer', 'boolean']:

pandas\index.pyx in pandas.index.IndexEngine.get_value (pandas\index.c:3332)()

pandas\index.pyx in pandas.index.IndexEngine.get_value (pandas\index.c:3035)()

pandas\index.pyx in pandas.index.IndexEngine.get_loc (pandas\index.c:4018)()

pandas\hashtable.pyx in pandas.hashtable.Int64HashTable.get_item (pandas\hashtable.c:6610)()

pandas\hashtable.pyx in pandas.hashtable.Int64HashTable.get_item (pandas\hashtable.c:6554)()

KeyError: 0

But when I try to simply print it, there is no problem:

for date in df.Date.unique(): 
    print([df[df.Date == '%s' %(date)]['Count']])

[0    1
1    2
2    2
3    3
Name: Count, dtype: int64]
[4    2
5    0
Name: Count, dtype: int64]
[6    2
Name: Count, dtype: int64]

What is the issue with calling plt.hist on my dataframe the way that I have it here?


2 Answers: 

Essentially you have two square brackets too much in your code.

plt.hist([series])  # <- wrong
plt.hist(series)    # <- correct

In the first case matplotlib would try to plot a histogram of a list of one element, which is non-numeric. That won't work.

Instead, removing the brackts and directly supplying the series, works fine

for date in df.Date.unique(): 
    plt.hist(df[df.Date == '%s' %(date)]['Count'])
    plt.title('%s' %(date))

Now this will create all histograms in the same plot. Not sure if this is desired. If not, consider the incredibly short alternative:

df.hist(by="Date")

enter image description here



You're passing a list of dataframes, which is causing a problem here. You could deconstruct a groupby object and plot each one separately.

gps = df.groupby('Date').Count
_, axes = plt.subplots(nrows=gps.ngroups)

for (_, g), ax in zip(df.groupby('Date').Count, axes):
    g.plot.hist(ax=ax)

plt.show()

enter image description here

Take a look at the Visualisation docs if you need more sugar in your graph.