pandas


pandas series or tidy dataframe: index level values to dataframe columns


I have a tidy Dataframe (which I'm not sure how to produce from scratch) such as:
signal condition epoch time value
0 A 0 -1100 0.12
1 A 0 -1080 0.09
2 A 0 -1060 0.08
...
With several conditions A,B,C. I want to get a DataFrame with multiindex, and the values of value column as values in the new (and only) columns A, B, C so in the end it looks like:
value A B C
epoch time
0 -1100 0.12 0.23 0.09
-1080 0.09 0.22 0.10
-1060 0.08 0.19 0.06
...
I tried to first put everything except value into the index with set_index(['condition','epoch','time']) and then transpose or pivot but can't get it right somehow (transpose gives me hierarchical columns and pivot KeyError)
I tried e.g.: df.pivot(('epoch','time'),'condition')
You were on the right track using pivot which is explicitly designed for such long-to-wide transformations:
# show dummy df
print(df)
signal condition epoch time value
0 0 A 0 -1100 0.12
1 1 A 0 -1080 0.09
2 2 A 0 -1060 0.08
3 0 B 0 -1100 0.42
4 1 B 0 -1080 0.29
5 2 B 0 -1060 0.18
6 0 C 0 -1100 0.32
7 1 C 0 -1080 0.59
8 2 C 0 -1060 0.38
# pivot
pd.pivot_table(df, values="value", columns="condition", index=["epoch", "time"])
condition A B C
epoch time
0 -1100 0.12 0.42 0.32
-1080 0.09 0.29 0.59
-1060 0.08 0.18 0.38
try
df.set_index(['epoch', 'time', 'condition']).value.unstack().rename_axis('value', 1)
value A B C
epoch time
0 -1100 0.12 0.23 0.09
-1080 0.09 0.22 0.10
-1060 0.08 0.19 0.06
setup
import pandas as pd
from io import StringIO
txt = """signal condition epoch time value
0 A 0 -1100 0.12
1 A 0 -1080 0.09
2 A 0 -1060 0.08
0 B 0 -1100 0.23
1 B 0 -1080 0.22
2 B 0 -1060 0.19
0 C 0 -1100 0.09
1 C 0 -1080 0.10
2 C 0 -1060 0.06 """
df = pd.read_csv(StringIO(txt), delim_whitespace=True)

Related Links

How can I select out columns where the first values are NaN?
Record limitation in pandas dataframe when importing from a csv file
count of unique values in pandas dataframe column
Rolling sums on pandas dataframe
Pandas Bug - Error when inserting list serialize as string
Formatting index of a pandas table in a plot
resample over consecutive chunks of large size CSV
Too many possibilities for categorical fields
How to install pandas on virtual machine?
How to change particular column value when defined mask is true?
Pandas/NumPy: concisely label first N values matching a mask
fast way to make index prefix with an alphabet
Is there a way to better format Pandas data frames when printing them in Sublime text?
Pandas pivot certain rows to columns
Combine three dataframes
Merging dataframes by file name

Categories

HOME
jdo
urbancode
react-virtualized
objectgears
leon
jsrender
rdf
jxls
ckeditor
angular-ui-bootstrap
esper
applepay
slurm
primary-key
kibana-4
kentor-authservices
imacros
u-sql
highlight.js
facebook-page
flux
excel-vba-mac
viewport
visjs
object-detection
scriptcs
realex-payments-api
language-agnostic
java-7
sqlcipher
tinymce-4
data-manipulation
protovis
mapdb
html5-fullscreen
file-format
stacked
automake
x11-forwarding
user-controls
sql-server-agent
firebase-admin
fakeiteasy
babel-core
qwt
outlook-2013
xml-documentation
network-flow
prolog-setof
paxos
alphabet
typescript1.8
domain-model
idisposable
knpmenubundle
linode
setuptools
static-ip-address
ado.net-entity-data-model
ctest
ipconfig
php-parse-error
cloudbees
photobucket
iad
reactive-banana
cannon.js
ruby-2.2
responsive-images
service-accounts
processmodel
lustre
pick
relocation
contenttype
ember-charts
message-driven-bean
odata4j
excel-2003
chronoforms
type-equivalence
free-variable
semantic-diff
deobfuscation
subgurim-maps
f#-powerpack
celltable
xfbml
web-application-design
icanhaz.js
preference
asp.net-profiles
vc90
meego
compiler-specific
zend-tool
kdbg
xetex
boost-smart-ptr
nerddinner
w3wp
data-acquisition
django-notification

Resources

Database Users
RDBMS discuss
Database Dev&Adm
javascript
java
csharp
php
android
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App