Mixing
Issue Extracting Data from CSV and Appending to Dict
I'm getting an odd result when trying to insert cells of data from a table (a CSV here but could be other tables) into a list of dictionaries.
import csv
keylist = ["ID", "RN", "PD"]
myID = 0
t =
t.append(dict.fromkeys(keylist, ))
with open("dataset.csv") as csv_file:
csv_reader = csv.reader(csv_file, delimiter=',')
for row in csv_reader:
for j in range (len(row)):
#printing here works as expected
print keylist[j], row[j]
#when appending to the value list the result is not as expected
t[myID][keylist[j]].append(row[j])
The unexpected result appears to append the whole row instead of just the item at row[j].
For example if the CSV were something like:
0, "foo", "bar"
0, "foo2", "bar2"
0, "foo3", "bar3"
The result for t[0]["ID"] would be this:
[0, "foo", "bar", 0, "foo2", "bar2", 0, "foo3", "bar3"]
Instead of the expected result of:
[0, 0, 0]
Any help would be much appreciated.
python arrays csv dictionary append
asked Nov 21 '18 at 21:57
zarf.snacklerzarf.snackler
105
add a comment |
I'm getting an odd result when trying to insert cells of data from a table (a CSV here but could be other tables) into a list of dictionaries.
import csv
keylist = ["ID", "RN", "PD"]
myID = 0
t =
t.append(dict.fromkeys(keylist, ))
with open("dataset.csv") as csv_file:
csv_reader = csv.reader(csv_file, delimiter=',')
for row in csv_reader:
for j in range (len(row)):
#printing here works as expected
print keylist[j], row[j]
#when appending to the value list the result is not as expected
t[myID][keylist[j]].append(row[j])
The unexpected result appears to append the whole row instead of just the item at row[j].
For example if the CSV were something like:
0, "foo", "bar"
0, "foo2", "bar2"
0, "foo3", "bar3"
The result for t[0]["ID"] would be this:
[0, "foo", "bar", 0, "foo2", "bar2", 0, "foo3", "bar3"]
Instead of the expected result of:
[0, 0, 0]
Any help would be much appreciated.
python arrays csv dictionary append
asked Nov 21 '18 at 21:57
zarf.snacklerzarf.snackler
105
add a comment |
I'm getting an odd result when trying to insert cells of data from a table (a CSV here but could be other tables) into a list of dictionaries.
import csv
keylist = ["ID", "RN", "PD"]
myID = 0
t =
t.append(dict.fromkeys(keylist, ))
with open("dataset.csv") as csv_file:
csv_reader = csv.reader(csv_file, delimiter=',')
for row in csv_reader:
for j in range (len(row)):
#printing here works as expected
print keylist[j], row[j]
#when appending to the value list the result is not as expected
t[myID][keylist[j]].append(row[j])
The unexpected result appears to append the whole row instead of just the item at row[j].
For example if the CSV were something like:
0, "foo", "bar"
0, "foo2", "bar2"
0, "foo3", "bar3"
The result for t[0]["ID"] would be this:
[0, "foo", "bar", 0, "foo2", "bar2", 0, "foo3", "bar3"]
Instead of the expected result of:
[0, 0, 0]
Any help would be much appreciated.
python arrays csv dictionary append
asked Nov 21 '18 at 21:57
zarf.snacklerzarf.snackler
105
I'm getting an odd result when trying to insert cells of data from a table (a CSV here but could be other tables) into a list of dictionaries.
import csv
keylist = ["ID", "RN", "PD"]
myID = 0
t =
t.append(dict.fromkeys(keylist, ))
with open("dataset.csv") as csv_file:
csv_reader = csv.reader(csv_file, delimiter=',')
for row in csv_reader:
for j in range (len(row)):
#printing here works as expected
print keylist[j], row[j]
#when appending to the value list the result is not as expected
t[myID][keylist[j]].append(row[j])
The unexpected result appears to append the whole row instead of just the item at row[j].
For example if the CSV were something like:
0, "foo", "bar"
0, "foo2", "bar2"
0, "foo3", "bar3"
The result for t[0]["ID"] would be this:
[0, "foo", "bar", 0, "foo2", "bar2", 0, "foo3", "bar3"]
Instead of the expected result of:
[0, 0, 0]
Any help would be much appreciated.
python arrays csv dictionary append
python arrays csv dictionary append
asked Nov 21 '18 at 21:57
zarf.snacklerzarf.snackler
105
asked Nov 21 '18 at 21:57
zarf.snacklerzarf.snackler
105
asked Nov 21 '18 at 21:57
zarf.snacklerzarf.snackler
105
asked Nov 21 '18 at 21:57
zarf.snacklerzarf.snackler
105
asked Nov 21 '18 at 21:57
zarf.snacklerzarf.snackler
105
105
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
dict.fromkeys initializes with the same value for all keys, so they all get the same instance of the list. It's not meant to initialize with mutable objects.
Instead, use collections.defaultdict to create a new list if a key doesn't yet exist:
import csv
from collections import defaultdict
keylist = ['ID', 'RN', 'PD']
myID = 0
t = [defaultdict(list)]
with open('dataset.csv',newline='') as csv_file: # Use newline='' per csv docs.
csv_reader = csv.reader(csv_file,skipinitialspace=True) # handles spaces after commas.
for row in csv_reader:
for col,value in enumerate(row):
t[myID][keylist[col]].append(value)
print(t[myID])import csv
from collections import defaultdict
keylist = ['ID', 'RN', 'PD']
myID = 0
t = [defaultdict(list)]
with open('dataset.csv',newline='') as csv_file:
csv_reader = csv.reader(csv_file,skipinitialspace=True)
for row in csv_reader:
for i,v in enumerate(row):
t[myID][keylist[i]].append(v)
print(t[myID])
Output:
defaultdict(<class 'list'>, {'ID': ['0', '0', '0'], 'RN': ['foo', 'foo2', 'foo3'], 'PD': ['bar', 'bar2', 'bar3']})
Note this still doesn't give you integers for your zeros. You'll need more code for that. Something like:
for col,value in enumerate(row):
t[myID][keylist[col]].append(int(value) if col==0 else value)
Output:
defaultdict(<class 'list'>, {'ID': [0, 0, 0], 'RN': ['foo', 'foo2', 'foo3'], 'PD': ['bar', 'bar2', 'bar3']})
answered Nov 22 '18 at 2:06
Mark TolonenMark Tolonen
94.1k12113176
add a comment |
I believe the problem stems from this initialization of the dictionary:
dict.fromkeys(keylist, )
The same list object is shared among all the dictionary's keys, and all items get appended to the same list.
The following initialization achieved the correct result:
t.append({k: for k in keylist})
Edit: a simple example to illustrate what's happening:
a = b =
a.append(3)
b.append('foo')
a
gives:
[3, 'foo']
Because while a and b are different variables, they are referring to the same object. Likewise in your example, the different keys in the dictionary all refer to the same list object passed in the fromkeys method.
answered Nov 21 '18 at 22:08
andersourceandersource
51418
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53421032%2fissue-extracting-data-from-csv-and-appending-to-dict%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
dict.fromkeys initializes with the same value for all keys, so they all get the same instance of the list. It's not meant to initialize with mutable objects.
Instead, use collections.defaultdict to create a new list if a key doesn't yet exist:
import csv
from collections import defaultdict
keylist = ['ID', 'RN', 'PD']
myID = 0
t = [defaultdict(list)]
with open('dataset.csv',newline='') as csv_file: # Use newline='' per csv docs.
csv_reader = csv.reader(csv_file,skipinitialspace=True) # handles spaces after commas.
for row in csv_reader:
for col,value in enumerate(row):
t[myID][keylist[col]].append(value)
print(t[myID])import csv
from collections import defaultdict
keylist = ['ID', 'RN', 'PD']
myID = 0
t = [defaultdict(list)]
with open('dataset.csv',newline='') as csv_file:
csv_reader = csv.reader(csv_file,skipinitialspace=True)
for row in csv_reader:
for i,v in enumerate(row):
t[myID][keylist[i]].append(v)
print(t[myID])
Output:
defaultdict(<class 'list'>, {'ID': ['0', '0', '0'], 'RN': ['foo', 'foo2', 'foo3'], 'PD': ['bar', 'bar2', 'bar3']})
Note this still doesn't give you integers for your zeros. You'll need more code for that. Something like:
for col,value in enumerate(row):
t[myID][keylist[col]].append(int(value) if col==0 else value)
Output:
defaultdict(<class 'list'>, {'ID': [0, 0, 0], 'RN': ['foo', 'foo2', 'foo3'], 'PD': ['bar', 'bar2', 'bar3']})
answered Nov 22 '18 at 2:06
Mark TolonenMark Tolonen
94.1k12113176
add a comment |
dict.fromkeys initializes with the same value for all keys, so they all get the same instance of the list. It's not meant to initialize with mutable objects.
Instead, use collections.defaultdict to create a new list if a key doesn't yet exist:
import csv
from collections import defaultdict
keylist = ['ID', 'RN', 'PD']
myID = 0
t = [defaultdict(list)]
with open('dataset.csv',newline='') as csv_file: # Use newline='' per csv docs.
csv_reader = csv.reader(csv_file,skipinitialspace=True) # handles spaces after commas.
for row in csv_reader:
for col,value in enumerate(row):
t[myID][keylist[col]].append(value)
print(t[myID])import csv
from collections import defaultdict
keylist = ['ID', 'RN', 'PD']
myID = 0
t = [defaultdict(list)]
with open('dataset.csv',newline='') as csv_file:
csv_reader = csv.reader(csv_file,skipinitialspace=True)
for row in csv_reader:
for i,v in enumerate(row):
t[myID][keylist[i]].append(v)
print(t[myID])
Output:
defaultdict(<class 'list'>, {'ID': ['0', '0', '0'], 'RN': ['foo', 'foo2', 'foo3'], 'PD': ['bar', 'bar2', 'bar3']})
Note this still doesn't give you integers for your zeros. You'll need more code for that. Something like:
for col,value in enumerate(row):
t[myID][keylist[col]].append(int(value) if col==0 else value)
Output:
defaultdict(<class 'list'>, {'ID': [0, 0, 0], 'RN': ['foo', 'foo2', 'foo3'], 'PD': ['bar', 'bar2', 'bar3']})
answered Nov 22 '18 at 2:06
Mark TolonenMark Tolonen
94.1k12113176
add a comment |
dict.fromkeys initializes with the same value for all keys, so they all get the same instance of the list. It's not meant to initialize with mutable objects.
Instead, use collections.defaultdict to create a new list if a key doesn't yet exist:
import csv
from collections import defaultdict
keylist = ['ID', 'RN', 'PD']
myID = 0
t = [defaultdict(list)]
with open('dataset.csv',newline='') as csv_file: # Use newline='' per csv docs.
csv_reader = csv.reader(csv_file,skipinitialspace=True) # handles spaces after commas.
for row in csv_reader:
for col,value in enumerate(row):
t[myID][keylist[col]].append(value)
print(t[myID])import csv
from collections import defaultdict
keylist = ['ID', 'RN', 'PD']
myID = 0
t = [defaultdict(list)]
with open('dataset.csv',newline='') as csv_file:
csv_reader = csv.reader(csv_file,skipinitialspace=True)
for row in csv_reader:
for i,v in enumerate(row):
t[myID][keylist[i]].append(v)
print(t[myID])
Output:
defaultdict(<class 'list'>, {'ID': ['0', '0', '0'], 'RN': ['foo', 'foo2', 'foo3'], 'PD': ['bar', 'bar2', 'bar3']})
Note this still doesn't give you integers for your zeros. You'll need more code for that. Something like:
for col,value in enumerate(row):
t[myID][keylist[col]].append(int(value) if col==0 else value)
Output:
defaultdict(<class 'list'>, {'ID': [0, 0, 0], 'RN': ['foo', 'foo2', 'foo3'], 'PD': ['bar', 'bar2', 'bar3']})
answered Nov 22 '18 at 2:06
Mark TolonenMark Tolonen
94.1k12113176
dict.fromkeys initializes with the same value for all keys, so they all get the same instance of the list. It's not meant to initialize with mutable objects.
Instead, use collections.defaultdict to create a new list if a key doesn't yet exist:
import csv
from collections import defaultdict
keylist = ['ID', 'RN', 'PD']
myID = 0
t = [defaultdict(list)]
with open('dataset.csv',newline='') as csv_file: # Use newline='' per csv docs.
csv_reader = csv.reader(csv_file,skipinitialspace=True) # handles spaces after commas.
for row in csv_reader:
for col,value in enumerate(row):
t[myID][keylist[col]].append(value)
print(t[myID])import csv
from collections import defaultdict
keylist = ['ID', 'RN', 'PD']
myID = 0
t = [defaultdict(list)]
with open('dataset.csv',newline='') as csv_file:
csv_reader = csv.reader(csv_file,skipinitialspace=True)
for row in csv_reader:
for i,v in enumerate(row):
t[myID][keylist[i]].append(v)
print(t[myID])
Output:
defaultdict(<class 'list'>, {'ID': ['0', '0', '0'], 'RN': ['foo', 'foo2', 'foo3'], 'PD': ['bar', 'bar2', 'bar3']})
Note this still doesn't give you integers for your zeros. You'll need more code for that. Something like:
for col,value in enumerate(row):
t[myID][keylist[col]].append(int(value) if col==0 else value)
Output:
defaultdict(<class 'list'>, {'ID': [0, 0, 0], 'RN': ['foo', 'foo2', 'foo3'], 'PD': ['bar', 'bar2', 'bar3']})
answered Nov 22 '18 at 2:06
Mark TolonenMark Tolonen
94.1k12113176
edited Nov 22 '18 at 5:24
answered Nov 22 '18 at 2:06
Mark TolonenMark Tolonen
94.1k12113176
answered Nov 22 '18 at 2:06
Mark TolonenMark Tolonen
94.1k12113176
answered Nov 22 '18 at 2:06
Mark TolonenMark Tolonen
94.1k12113176
94.1k12113176
add a comment |
add a comment |
I believe the problem stems from this initialization of the dictionary:
dict.fromkeys(keylist, )
The same list object is shared among all the dictionary's keys, and all items get appended to the same list.
The following initialization achieved the correct result:
t.append({k: for k in keylist})
Edit: a simple example to illustrate what's happening:
a = b =
a.append(3)
b.append('foo')
a
gives:
[3, 'foo']
Because while a and b are different variables, they are referring to the same object. Likewise in your example, the different keys in the dictionary all refer to the same list object passed in the fromkeys method.
answered Nov 21 '18 at 22:08
andersourceandersource
51418
add a comment |
I believe the problem stems from this initialization of the dictionary:
dict.fromkeys(keylist, )
The same list object is shared among all the dictionary's keys, and all items get appended to the same list.
The following initialization achieved the correct result:
t.append({k: for k in keylist})
Edit: a simple example to illustrate what's happening:
a = b =
a.append(3)
b.append('foo')
a
gives:
[3, 'foo']
Because while a and b are different variables, they are referring to the same object. Likewise in your example, the different keys in the dictionary all refer to the same list object passed in the fromkeys method.
answered Nov 21 '18 at 22:08
andersourceandersource
51418
add a comment |
I believe the problem stems from this initialization of the dictionary:
dict.fromkeys(keylist, )
The same list object is shared among all the dictionary's keys, and all items get appended to the same list.
The following initialization achieved the correct result:
t.append({k: for k in keylist})
Edit: a simple example to illustrate what's happening:
a = b =
a.append(3)
b.append('foo')
a
gives:
[3, 'foo']
Because while a and b are different variables, they are referring to the same object. Likewise in your example, the different keys in the dictionary all refer to the same list object passed in the fromkeys method.
answered Nov 21 '18 at 22:08
andersourceandersource
51418
I believe the problem stems from this initialization of the dictionary:
dict.fromkeys(keylist, )
The same list object is shared among all the dictionary's keys, and all items get appended to the same list.
The following initialization achieved the correct result:
t.append({k: for k in keylist})
Edit: a simple example to illustrate what's happening:
a = b =
a.append(3)
b.append('foo')
a
gives:
[3, 'foo']
Because while a and b are different variables, they are referring to the same object. Likewise in your example, the different keys in the dictionary all refer to the same list object passed in the fromkeys method.
answered Nov 21 '18 at 22:08
andersourceandersource
51418
edited Nov 21 '18 at 22:17
answered Nov 21 '18 at 22:08
andersourceandersource
51418
answered Nov 21 '18 at 22:08
andersourceandersource
51418
answered Nov 21 '18 at 22:08
andersourceandersource
51418
51418
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53421032%2fissue-extracting-data-from-csv-and-appending-to-dict%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
