pandas


Assigning one column to another column between pandas DataFrames (like vector to vector assignment)


I have a super strange problem which I spent the last hour trying to solve, but with no success. It is even more strange since I can't replicate it on a small scale.
I have a large DataFrame (150,000 entries). I took out a subset of it and did some manipulation. the subset was saved as a different variable, x.
x is smaller than the df, but its index is in the same range as the df. I'm now trying to assign x back to the DataFrame replacing values in the same column:
rep_Callers['true_vpID'] = x.true_vpID
This inserts all the different values in x to the right place in df, but instead of keeping the df.true_vpID values that are not in x, it is filling them with NaNs. So I tried a different approach:
df.ix[x.index,'true_vpID'] = x.true_vpID
But instead of filling x values in the right place in df, the df.true_vpID gets filled with the first value of x and only it! I changed the first value of x several times to make sure this is indeed what is happening, and it is. I tried to replicate it on a small scale but it didn't work:
df = DataFrame({'a':ones(5),'b':range(5)})
a b
0 1 0
1 1 1
2 1 2
3 1 3
4 1 4
z =Series([random() for i in range(5)],index = range(5))
0 0.812561
1 0.862109
2 0.031268
3 0.575634
4 0.760752
df.ix[z.index[[1,3]],'b'] = z[[1,3]]
a b
0 1 0.000000
1 1 0.812561
2 1 2.000000
3 1 0.575634
4 1 4.000000
5 1 5.000000
I really tried it all, need some new suggestions...
Try using df.update(updated_df_or_series)
Also using a simple example, you can modify a DataFrame by doing an index query and modifying the resulting object.
df_1
a b
0 1 0
1 1 1
2 1 2
3 1 3
4 1 4
df_2 = df_1.ix[3:5]
df_2.b = df_2.b + 2
df_2
a b
3 1 5
4 1 6
df_1
a b
0 1 0
1 1 1
2 1 2
3 1 5
4 1 6

Related Links

Pandas Column Construction with np.where()
MultiIndex-based indexing in pandas
How to get the number of the most frequent value in a column?
Using pandas.ols on multiple dependent variables at once
Insert 0-values for missing dates within MultiIndex
Reindexing dataframes
pandas access axis by user-defined name
Trouble with groupss and aggregation
Replace MultiIndex's contents with DataFrame columns
What's the `DataFrameGroupBy`-equivalent of `dict.keys`?
How to split a dataframe according to a boolean criterion?
Pandas Rolling Computations on Sliding Windows (Unevenly spaced)
Resampling Minute data
How to get the last n row of pandas dataframe?
Resample time series in pandas to a weekly interval
Suppress output of object when plotting in ipython

Categories

HOME
wso2-am
tizen
json.net
onedrive
lodash
iot
objectgears
rubygems
twitter-bootstrap-4
v8
dxl
win32gui
ebean
enterprise-library-5
tomcat6
google-cloud-spanner
hapi
iolanguage
uitypeeditor
clojurescript
libtiff
django-admin
angular-ui
footer
pythonanywhere
nhibernate-envers
textfield
one-hot-encoding
intentfilter
h2db
dynamic-featured-image
chromebook
url-scheme
wpfdatagrid
info.plist
uninstall
phpfox
fopen
windows-error-reporting
fifo
grails-3.1
pim
protovis
gammu
mapbox-gl
jmonkeyengine
swisscomdev
lto
hilbert-curve
uft-api
streamreader
tasker
trim
email-templates
ncalc
powershell-dsc
node-gyp
password-encryption
skeleton-css-boilerplate
avconv
taffy
dotcover
rainbowtable
dropbox-php
titanium-android
removechild
ado.net-entity-data-model
trash
mongocsharpdriver
sdhc
qdialog
feedback
sframe
notify
bigbluebutton
block-device
asp.net-web-api-odata
device-manager
code-access-security
castle
yorick
hippomocks
android-2.2-froyo
access-rights
cos
comaddin
pygit2
propertyeditor
onsubmit
appconkit
windows-phone-7.1.1
amazon-appstore
semantic-diff
nsdatecomponents
gwt-ext
libc++
f#-powerpack
xdomainrequest
invite
pyinotify
sudzc
jquery-ui-droppable
virtual-functions
noscript
sector
windows-live-messenger

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App