pandas


Two unaligned Pandas Series: concat raises error, adding does not but it returns a weird answer


Note: I am working with a rather old-ish Pandas 0.16.2, in Python 2.7.11.
My simplistic conceptual model for the adding of two Series was that it would involve an index-matching step that is similar to what goes on in a pd.concat(..., axis=1), ie. the Series indexes are lined up and then the values are added.
Therefore (modulo NaN handling I guess), I would expect u+v to work if, and only if concat([u, v], axis=1) works.
In the example below I build two Series with 'unalignable' indexes. My confusion is that concat does raise an error (as expected) but the adding does not -- and even more confusing is that the result of adding comes back with everything duplicated.
First I create a couple of series which have equal indexes (containing duplicates):
import string, pandas as pd
# Create a series with an index that has duplicates
u = pd.Series(range(5), index=list(string.ascii_lowercase)[:5])
u = pd.concat([u, u])
# Create another, same index but values reversed
v = pd.Series(range(5)[::-1], index=list(string.ascii_lowercase)[:5])
v = pd.concat([v, v])
Here they are:
In [2]: u
Out[2]:
a 0
b 1
c 2
d 3
e 4
a 0
b 1
c 2
d 3
e 4
dtype: int64
In [3]: v
Out[3]:
a 4
b 3
c 2
d 1
e 0
a 4
b 3
c 2
d 1
e 0
dtype: int64
They can be added since the indices are equal:
In [4]: u+v
Out[4]:
a 4
b 4
c 4
d 4
e 4
a 4
b 4
c 4
d 4
e 4
dtype: int64
If we sort v then its index gets reordered and since there is no longer an obvious way to line up v with u any more it is not surprising that concat raises an error:
In [5]: v.sort()
In [6]: v
Out[6]:
e 0
e 0
d 1
d 1
c 2
c 2
b 3
b 3
a 4
a 4
dtype: int64
In [7]: pd.concat([u, v], axis=1)
....
ValueError: cannot reindex from a duplicate axis
However, adding still works but bizarrely returns a longer series:
In [8]: u+v
Out[8]:
a 4
a 4
a 4
a 4
b 4
b 4
b 4
b 4
c 4
c 4
c 4
c 4
d 4
d 4
d 4
d 4
e 4
e 4
e 4
e 4
dtype: int64
What happened here?

Related Links

Is there a GraphLab SFrame.show() equivalent in Pandas?
Filter data frame with a boolean vector based on one column
Putting a vector as a Pandas data frame element
Extra lane in heat map (pandas)
How to concat multiple pandas dataframes into one dask dataframe larger than memory?
Upsampling Dataframe in Pandas with Index + Column
Reading Google Bigquery data into dataframe
How do you divide pandas columns by a scalar?
pandas - not in index
Can I retrieve all the results from a groupby?
Pandas - 2 dataframes, add Index column of df1 to df2 on second column
How to improve the efficiency of pandas.nlargest?
Matplotlib index of datapoints from rectangleselector
How to create a view of dataframe in pandas?
Python 3 pandas add a column with if then statement using length
updating a column based on a condition without using .loc

Categories

HOME
debugging
keras
cplex
alfresco
google-oauth
v8
c#-2.0
serverspec
opengl-es-2.0
azure-storage-tables
bs4
contact
synchronization
applepay
kibana-4
vault
node-pdfkit
phaser
visual-studio-cordova
circuit
vaadin7
telephony
graphlab
invantive-sql
ab-testing
crystal-reports-2008
paging
xlsxwriter
web-sql
fopen
c++-amp
newline
unboundid
bcd
ecto
phonegap
xquery-3.0
gtrendsr
catch-all
usbserial
vsts-build-task
subset-sum
openoffice.org
jmonkeyengine
stacked
bitbucket-pipelines
x11-forwarding
mixture-model
twitch
webix-treetable
temporary-files
bootstrapper
awt
sql-server-agent
firebase-admin
total-commander
acoustics
reactive-cocoa-5
filepicker
upstart
revapi
jquery-nestable
worksheet
datastax-startup
integrity
pg-dump
nsarray
r-forge
imanage
pycaffe
cubes
ableton-live
underscore.js-templating
blackberry-10
sigabrt
ubuntu-10.04
captivenetwork
itextpdf
natvis
dereference
device-orientation
apache-commons-net
ng-animate
pick
codeigniter-url
gui-test-framework
asp.net-dynamic-data
titanium-modules
typeof
wp-query
comexception
rtmfp
oracle-warehouse-builder
algebraic-data-types
quantlib-swig
cdc
odata4j
ruby-datamapper
ril
eventual-consistency
bluepill
word-processor
multipage
subscript
fireworks
window-management
javax.script
yetanotherforum
appender
database-management
webkit.net
gacutil
spec#

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App