Append value to each array in a numpy array

I have a numpy array of arrays, for example:

x = np.array([[1,2,3],[10,20,30]])

Now lets say I want to extend each array with [4,40], to generate the following resulting array:

[[1,2,3,4],[10,20,30,40]]

How can I do this without making a copy of the whole array? I tried to change the shape of the array in place but it throws a ValueError:

x[0] = np.append(x[0],4)

x[1] = np.append(x[1],40)

ValueError : could not broadcast input array from shape (4) into shape (3)

asked Nov 21 '18 at 18:48

hirschme

11919

3

You can't. Numpy arrays are not really designed to change size because it screws with having contiguous memory. It's probably more efficient to use lists until you have the final structure

– roganjosh
Nov 21 '18 at 18:49

I have edited my answer. It's not clear to me whether or not you can reformulate your problem to fit that use case, but it's worth considering.

– roganjosh
Nov 21 '18 at 19:18

np.append does not work in-place. It's a version of concatenate and makes a new array. It's a hard function to use correctly (if ever).

– hpaulj
Nov 21 '18 at 19:35

add a comment |

I have a numpy array of arrays, for example:

x = np.array([[1,2,3],[10,20,30]])

Now lets say I want to extend each array with [4,40], to generate the following resulting array:

[[1,2,3,4],[10,20,30,40]]

How can I do this without making a copy of the whole array? I tried to change the shape of the array in place but it throws a ValueError:

x[0] = np.append(x[0],4)

x[1] = np.append(x[1],40)

ValueError : could not broadcast input array from shape (4) into shape (3)

asked Nov 21 '18 at 18:48

hirschme

11919

3

You can't. Numpy arrays are not really designed to change size because it screws with having contiguous memory. It's probably more efficient to use lists until you have the final structure

– roganjosh
Nov 21 '18 at 18:49

I have edited my answer. It's not clear to me whether or not you can reformulate your problem to fit that use case, but it's worth considering.

– roganjosh
Nov 21 '18 at 19:18

np.append does not work in-place. It's a version of concatenate and makes a new array. It's a hard function to use correctly (if ever).

– hpaulj
Nov 21 '18 at 19:35

add a comment |

I have a numpy array of arrays, for example:

x = np.array([[1,2,3],[10,20,30]])

Now lets say I want to extend each array with [4,40], to generate the following resulting array:

[[1,2,3,4],[10,20,30,40]]

How can I do this without making a copy of the whole array? I tried to change the shape of the array in place but it throws a ValueError:

x[0] = np.append(x[0],4)

x[1] = np.append(x[1],40)

ValueError : could not broadcast input array from shape (4) into shape (3)

asked Nov 21 '18 at 18:48

hirschme

11919

I have a numpy array of arrays, for example:

x = np.array([[1,2,3],[10,20,30]])

Now lets say I want to extend each array with [4,40], to generate the following resulting array:

[[1,2,3,4],[10,20,30,40]]

How can I do this without making a copy of the whole array? I tried to change the shape of the array in place but it throws a ValueError:

x[0] = np.append(x[0],4)

x[1] = np.append(x[1],40)

ValueError : could not broadcast input array from shape (4) into shape (3)

python arrays numpy

asked Nov 21 '18 at 18:48

hirschme

11919

asked Nov 21 '18 at 18:48

hirschme

11919

asked Nov 21 '18 at 18:48

hirschme

11919

asked Nov 21 '18 at 18:48

hirschme

11919

asked Nov 21 '18 at 18:48

hirschme

11919

3

You can't. Numpy arrays are not really designed to change size because it screws with having contiguous memory. It's probably more efficient to use lists until you have the final structure

– roganjosh
Nov 21 '18 at 18:49

I have edited my answer. It's not clear to me whether or not you can reformulate your problem to fit that use case, but it's worth considering.

– roganjosh
Nov 21 '18 at 19:18

np.append does not work in-place. It's a version of concatenate and makes a new array. It's a hard function to use correctly (if ever).

– hpaulj
Nov 21 '18 at 19:35

add a comment |

3

You can't. Numpy arrays are not really designed to change size because it screws with having contiguous memory. It's probably more efficient to use lists until you have the final structure

– roganjosh
Nov 21 '18 at 18:49

I have edited my answer. It's not clear to me whether or not you can reformulate your problem to fit that use case, but it's worth considering.

– roganjosh
Nov 21 '18 at 19:18

np.append does not work in-place. It's a version of concatenate and makes a new array. It's a hard function to use correctly (if ever).

– hpaulj
Nov 21 '18 at 19:35

You can't. Numpy arrays are not really designed to change size because it screws with having contiguous memory. It's probably more efficient to use lists until you have the final structure

– roganjosh
Nov 21 '18 at 18:49

I have edited my answer. It's not clear to me whether or not you can reformulate your problem to fit that use case, but it's worth considering.

– roganjosh
Nov 21 '18 at 19:18

np.append does not work in-place. It's a version of concatenate and makes a new array. It's a hard function to use correctly (if ever).

– hpaulj
Nov 21 '18 at 19:35

add a comment |

2 Answers
2

active

oldest

votes

You can't do this. Numpy arrays allocate contiguous blocks of memory, if at all possible. Any change to the array size will force an inefficient copy of the whole array. You should use Python lists to grow your structure if possible, then convert the end result back to an array.

However, if you know the final size of the resulting array, you could instantiate it with something like np.empty() and then assign values by index, rather than appending. This does not change the size of the array itself, only reassigns values, so should not require copying.

edited Nov 21 '18 at 19:16

answered Nov 21 '18 at 18:54

roganjosh

6,74931328

add a comment |

Create a new matrix

Insert the values of your old matrix

Then, insert your new values in the last positions

x = np.array([[1,2,3],[10,20,30]])

new_X = np.zeros((2, 4))

new_X[:2,:3] = x 

new_X[0][-1] = 4

new_X[1][-1] = 40

x=new_X

Or Use np.reshape() or np.resize() instead

answered Nov 21 '18 at 19:22

Felipe Rigel

How is this more efficient than a copy?

– roganjosh
Nov 21 '18 at 19:24

new_x[:, -1] = [4, 50] is better numpy code. And set the dtype of new_X to match x.

– hpaulj
Nov 21 '18 at 19:26

Thanks for the suggestion, but I needed to avoid using double the storage space. I ended up converting the np.arrays into lists, modifying them, and then converting back.

– hirschme
Nov 21 '18 at 22:26

@roganjosh You start your answer with "You can't". You can but is not efficient, and no, is not more eficient than a copy cause is a copy, but It's faster to copy than cast is Memory VS Time.

– Felipe Rigel
Dec 4 '18 at 22:06

"How can I do this without making a copy of the whole array?". You can't. It will be copied. I don't know what point you're making. What you're doing is no different than a copy in principle, just less efficient still

– roganjosh
Dec 4 '18 at 22:10

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53418727%2fappend-value-to-each-array-in-a-numpy-array%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

2 Answers
2

active

oldest

votes

2 Answers
2

active

oldest

votes

edited Nov 21 '18 at 19:16

answered Nov 21 '18 at 18:54

roganjosh

6,74931328

add a comment |

edited Nov 21 '18 at 19:16

answered Nov 21 '18 at 18:54

roganjosh

6,74931328

add a comment |

edited Nov 21 '18 at 19:16

answered Nov 21 '18 at 18:54

roganjosh

6,74931328

edited Nov 21 '18 at 19:16

answered Nov 21 '18 at 18:54

roganjosh

6,74931328

edited Nov 21 '18 at 19:16

answered Nov 21 '18 at 18:54

roganjosh

6,74931328

answered Nov 21 '18 at 18:54

roganjosh

6,74931328

answered Nov 21 '18 at 18:54

roganjosh

6,74931328

add a comment |

Create a new matrix

Insert the values of your old matrix

Then, insert your new values in the last positions

x = np.array([[1,2,3],[10,20,30]])

new_X = np.zeros((2, 4))

new_X[:2,:3] = x 

new_X[0][-1] = 4

new_X[1][-1] = 40

x=new_X

Or Use np.reshape() or np.resize() instead

answered Nov 21 '18 at 19:22

Felipe Rigel

How is this more efficient than a copy?

– roganjosh
Nov 21 '18 at 19:24

new_x[:, -1] = [4, 50] is better numpy code. And set the dtype of new_X to match x.

– hpaulj
Nov 21 '18 at 19:26

Thanks for the suggestion, but I needed to avoid using double the storage space. I ended up converting the np.arrays into lists, modifying them, and then converting back.

– hirschme
Nov 21 '18 at 22:26

@roganjosh You start your answer with "You can't". You can but is not efficient, and no, is not more eficient than a copy cause is a copy, but It's faster to copy than cast is Memory VS Time.

– Felipe Rigel
Dec 4 '18 at 22:06

"How can I do this without making a copy of the whole array?". You can't. It will be copied. I don't know what point you're making. What you're doing is no different than a copy in principle, just less efficient still

– roganjosh
Dec 4 '18 at 22:10

add a comment |

Create a new matrix

Insert the values of your old matrix

Then, insert your new values in the last positions

x = np.array([[1,2,3],[10,20,30]])

new_X = np.zeros((2, 4))

new_X[:2,:3] = x 

new_X[0][-1] = 4

new_X[1][-1] = 40

x=new_X

Or Use np.reshape() or np.resize() instead

answered Nov 21 '18 at 19:22

Felipe Rigel

How is this more efficient than a copy?

– roganjosh
Nov 21 '18 at 19:24

new_x[:, -1] = [4, 50] is better numpy code. And set the dtype of new_X to match x.

– hpaulj
Nov 21 '18 at 19:26

Thanks for the suggestion, but I needed to avoid using double the storage space. I ended up converting the np.arrays into lists, modifying them, and then converting back.

– hirschme
Nov 21 '18 at 22:26

@roganjosh You start your answer with "You can't". You can but is not efficient, and no, is not more eficient than a copy cause is a copy, but It's faster to copy than cast is Memory VS Time.

– Felipe Rigel
Dec 4 '18 at 22:06

"How can I do this without making a copy of the whole array?". You can't. It will be copied. I don't know what point you're making. What you're doing is no different than a copy in principle, just less efficient still

– roganjosh
Dec 4 '18 at 22:10

add a comment |

Create a new matrix

Insert the values of your old matrix

Then, insert your new values in the last positions

x = np.array([[1,2,3],[10,20,30]])

new_X = np.zeros((2, 4))

new_X[:2,:3] = x 

new_X[0][-1] = 4

new_X[1][-1] = 40

x=new_X

Or Use np.reshape() or np.resize() instead

answered Nov 21 '18 at 19:22

Felipe Rigel

Create a new matrix

Insert the values of your old matrix

Then, insert your new values in the last positions

x = np.array([[1,2,3],[10,20,30]])

new_X = np.zeros((2, 4))

new_X[:2,:3] = x 

new_X[0][-1] = 4

new_X[1][-1] = 40

x=new_X

Or Use np.reshape() or np.resize() instead

answered Nov 21 '18 at 19:22

Felipe Rigel

answered Nov 21 '18 at 19:22

Felipe Rigel

answered Nov 21 '18 at 19:22

Felipe Rigel

answered Nov 21 '18 at 19:22

Felipe Rigel

How is this more efficient than a copy?

– roganjosh
Nov 21 '18 at 19:24

new_x[:, -1] = [4, 50] is better numpy code. And set the dtype of new_X to match x.

– hpaulj
Nov 21 '18 at 19:26

Thanks for the suggestion, but I needed to avoid using double the storage space. I ended up converting the np.arrays into lists, modifying them, and then converting back.

– hirschme
Nov 21 '18 at 22:26

@roganjosh You start your answer with "You can't". You can but is not efficient, and no, is not more eficient than a copy cause is a copy, but It's faster to copy than cast is Memory VS Time.

– Felipe Rigel
Dec 4 '18 at 22:06

"How can I do this without making a copy of the whole array?". You can't. It will be copied. I don't know what point you're making. What you're doing is no different than a copy in principle, just less efficient still

– roganjosh
Dec 4 '18 at 22:10

add a comment |

How is this more efficient than a copy?

– roganjosh
Nov 21 '18 at 19:24

new_x[:, -1] = [4, 50] is better numpy code. And set the dtype of new_X to match x.

– hpaulj
Nov 21 '18 at 19:26

Thanks for the suggestion, but I needed to avoid using double the storage space. I ended up converting the np.arrays into lists, modifying them, and then converting back.

– hirschme
Nov 21 '18 at 22:26

@roganjosh You start your answer with "You can't". You can but is not efficient, and no, is not more eficient than a copy cause is a copy, but It's faster to copy than cast is Memory VS Time.

– Felipe Rigel
Dec 4 '18 at 22:06

"How can I do this without making a copy of the whole array?". You can't. It will be copied. I don't know what point you're making. What you're doing is no different than a copy in principle, just less efficient still

– roganjosh
Dec 4 '18 at 22:10

How is this more efficient than a copy?

– roganjosh
Nov 21 '18 at 19:24

new_x[:, -1] = [4, 50] is better numpy code. And set the dtype of new_X to match x.

– hpaulj
Nov 21 '18 at 19:26

Thanks for the suggestion, but I needed to avoid using double the storage space. I ended up converting the np.arrays into lists, modifying them, and then converting back.

– hirschme
Nov 21 '18 at 22:26

@roganjosh You start your answer with "You can't". You can but is not efficient, and no, is not more eficient than a copy cause is a copy, but It's faster to copy than cast is Memory VS Time.

– Felipe Rigel
Dec 4 '18 at 22:06

"How can I do this without making a copy of the whole array?". You can't. It will be copied. I don't know what point you're making. What you're doing is no different than a copy in principle, just less efficient still

– roganjosh
Dec 4 '18 at 22:10

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

Search This Blog

Ufyukyu