Posts

Showing posts with the label pandas-groupby

Create new columns from aggregated categories

Image
Clash Royale CLAN TAG #URR8PPP Create new columns from aggregated categories I have a dataframe looks like: SK_ID_CURR CREDIT_ACTIVE 0 215354 Closed 1 215354 Active 2 215354 Active 3 215354 Active 4 215354 Active 5 215354 Active 6 215354 Active 7 162297 Closed 8 162297 Closed 9 162297 Active I would like to aggregate the number of active and closed credits for each id, and then make a new column for Active_credits , Closed_credits with the number of corresponding active and closed credits for each id. Active_credits Closed_credits What is your expected output? – Akshay Nevrekar 10 mins ago 2 Answers 2 You can use pandas.crosstab , which avoids your suggested intermediary step: pandas.crosstab res = pd.crosstab(df['...

Perform cumulative count on a pandas column taking order into account

Image
Clash Royale CLAN TAG #URR8PPP Perform cumulative count on a pandas column taking order into account I have a pandas data frame pd.DataFrame(columns=["A", "B"], data=[['id1','a'],['id1','a'], ['id1','a'], ['id1','b'], ['id1','b'], ['id1','a'], ['id1','a'], ['id2','c'], ['id2','c'], ['id2','a'], ['id2','c']]) A B 0 id1 a 1 id1 a 2 id1 a 3 id1 b 4 id1 b 5 id1 a 6 id1 a 7 id2 c 8 id2 c 9 id2 a 10 id2 c I want to do a group by that gives me cumulative sum of occurrences in B column considering the order A B C 0 id1 a 3 1 b 2 2 a 2 3 id2 c 2 4 a 1 5 c 1 1 Answer 1 You can use itertools.groupby followed by a list comprehension. This wor...

Pandas Multiindex Series Processing

Image
Clash Royale CLAN TAG #URR8PPP Pandas Multiindex Series Processing I have a pandas 2 index serie that I got from a .groupby() on my original DataFrame: label ncsc False 0 297 1 537 2 333 3 207 4 51 5 12 6 4 7 2 True 0 29 1 68 2 35 3 29 4 35 5 18 6 8 7 2 Name: ncsc, dtype: int64 I would like to be able to compute for each 'ncsc' the True rate, ie, for 'ncsc'=6, True rate = 8/(4+8) = 0.66. Do you see a way to do that with =out using a loop but with a 'pandas synthax'?? Thanks 1 Answer 1 Convert the wanted index level to a column and group by that column: df.reset_index(level=1).groupby('ncsc').mean() By clicking ...

Add auto-increment column on pandas multi-index

Image
Clash Royale CLAN TAG #URR8PPP Add auto-increment column on pandas multi-index I have the following multi-index panda below. I am trying to create: However I am unsure how I would be able to do this. Any pointers would help Multi-index before: duration_in_status lob project_rank Commodities CM LOB 2.0 Index Book Migration 25.0 Cross Platform CM LOB 0.0 CSAVA 16.0 Calypso Migration 0.0 EMD / Delta One 0.0 FRTB 68.0 Index Book Migration 1.0 Instruments 3.0 KOJAK 0.0 LOB BOW 324.0 Non-Tradin...

How to keep pandas group by column when applying transform function?

Image
Clash Royale CLAN TAG #URR8PPP How to keep pandas group by column when applying transform function? This is my pandas dataframe look's like: sampling_time MQ2_LPG MQ2_CO MQ2_SMOKE MQ2_ALCOHOL MQ2_CH4 MQ2_H2 MQ2_PROPANE 0 2018-07-15 08:41:49.028 4.41 32.87 19.12 7.70 10.29 7.59 4.49 1 2018-07-15 08:41:49.028 2.98 19.08 12.47 4.72 6.34 5.15 3.02 2 2018-07-15 08:41:49.028 2.73 16.88 11.33 4.22 5.69 4.72 2.76 3 2018-07-15 08:41:49.028 2.69 16.47 11.11 4.13 5.57 4.64 2.71 4 2018-07-15 08:41:49.028 2.66 16.26 11.00 4.09 5.50 4.60 2.69 When I'm doing group by (split apply combine method), my sampling time column was removed. transformed = dataframe.groupby('sampling_time').transform(lambda x: (x - x.mean()) / x.std()) transformed.head() MQ2_LPG MQ2_CO MQ2_SMOKE MQ2_ALCOHOL MQ2_CH4 MQ2_H2 MQ2_PROPANE 0 15.710127 15.975636 15....